Sun Jul 5 15:31:42 PDT 2009 kevan@isnotajoke.com * Modify bench_dirnode.py to use CachingDict. New patches: [Modify bench_dirnode.py to use CachingDict. kevan@isnotajoke.com**20090705223142 Ignore-this: 9ba62a16fd37ef281368715a887fd9f8 ] hunk ./src/allmydata/test/bench_dirnode.py 83 packstr = pack(N) def pack(N): - return testdirnode._pack_contents(dict(children[:N])) + return testdirnode._pack_contents(dirnode.CachingDict(children[:N])) def unpack(N): return testdirnode._unpack_contents(packstr) Context: [directories: update the directory benchmarks to exercise the unpack-and-repack functionality, and add optional profiling zooko@zooko.com**20090705162953 Ignore-this: 4a1b11c9b1880772c923b3c03e10770b ] [test/benchmark: benchmark the time to pack and unpack dirnodes zooko@zooko.com**20090704224300 Ignore-this: cd8f6a6ded44a3f6f102f9cd0b60ca62 See also the prof_benchmarks() function in this file which will run the benchmarks under profiling. ] [setup: the mac-exe build (using py2app) requires macholib>=1.2. zooko@zooko.com**20090703175702 Ignore-this: ea489a5076f2c9f20f6bfd2807cc5680 Here is someone else's description of this problem: http://rtmpy.org/ticket/7 ] [clean up debian packaging: we have control files for etch/lenny/sid, and "Brian Warner "**20090703072804 everything else uses one of those. Add dependency on python-pysqlite2 for platforms that use py2.4 by default. Update foolscap dependency to 0.4.1. ] [update NEWS to cover all recent changes, sort by end-user importance Brian Warner **20090703014303 Ignore-this: 6ddac78075d7547a19712d505818949c ] [Tolerate unknown URI types in directory structures. Part of #683. Brian Warner **20090703010749 Ignore-this: afd0e15e2e39d3b87743ec7ccd87054d The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better). ] [test_client.py: minor refactoring of BASECONFIG usage Brian Warner **20090703005450 Ignore-this: bf19b9ae7edeab293068cce2368f9364 ] [create_node_from_uri: take both writecap+readcap, move logic out of dirnode.py Brian Warner **20090702222537 Ignore-this: 93051498076e90d3f1dc85161ce8247a ] [dirnode.py: prepare to preserve both rwcap+rocap when copying Brian Warner **20090702211254 Ignore-this: f128c02da32f86d7e39527a35dfc2e02 This will make it easier to tolerate unknown nodes safely. ] [interfaces.py: wrap some lines to 80cols Brian Warner **20090702015728 Ignore-this: e2c777c1e89a684b43ceabeb0042456c ] [cli: webopen: when called with no arguments, open the Welcome page Brian Warner **20090701200548 Ignore-this: ae7d6cb42165d0c751926065378343dd ] [mutable repairer: skip repair of readcaps instead of throwing an exception. Brian Warner **20090701011343 Ignore-this: 2c24493426cdc1db8f0e3815ee2c5f87 This should improve the behavior of #625 a bit: at least all the files will get repaired. ] [interfaces.py: improve ICheckAndRepairResults docs a bit Brian Warner **20090701001925 Ignore-this: 3a57acfee3487c1e071e135743e42c2a ] [repairer.py: wrap to 80cols. No code changes. Brian Warner **20090701000047 Ignore-this: 4a84ac95a849be0656d362882876082a ] [setup: require pycryptopp>=0.5.14 if on Windows and with Python>=2.6 zooko@zooko.com**20090630184807 Ignore-this: f7e9beeb5d5613a7c0ffed14d1dda3c6 ] [edit NEWS Brian Warner **20090630174115 Ignore-this: c4461a2304fcd45bee95e11418693a18 ] [NEWS: list all user-visible changes since 1.4.1 . Needs lots of editing. Brian Warner **20090630170734 Ignore-this: f606a5d678d0db8065b9f84e796d59b0 ] [* find-trailing-spaces.py: exit rc=1 if whitespace found, to be a pre-commit hook Brian Warner **20090629224658 Ignore-this: 1465becc923ee04d170e5ee791cf605e ] [use 522-bit RSA keys in all unit tests (except one) Brian Warner **20090629223124 Ignore-this: 7a4c3685683ff9da5ceb2d8cb7b19b7 This reduces the total test time on my laptop from 400s to 283s. * src/allmydata/test/test_system.py (SystemTest.test_mutable._test_debug): Remove assertion about container_size/data_size, this changes with keysize and was too variable anyways. * src/allmydata/mutable/filenode.py (MutableFileNode.create): add keysize= * src/allmydata/dirnode.py (NewDirectoryNode.create): same * src/allmydata/client.py (Client.DEFAULT_MUTABLE_KEYSIZE): add default, this overrides the one in MutableFileNode ] [remove trailing whitespace Brian Warner **20090629200358 Ignore-this: 7a3756618dcfca0a40acb4c3d15f6440 ] [remove trailing whitespace from some util classes Brian Warner **20090629200341 Ignore-this: ded84c38885d6aabf2691957dde82bf3 ] [repairer: raise a better exception when faced with a readonly filenode. Still Brian Warner **20090626063230 Ignore-this: a100005b973a6a57566b943073352828 produces an error, though. ] [mutable/filenode.py: set _writekey to None, rather than leaving it missing Brian Warner **20090626062022 Ignore-this: be111c37dabd6c7aa47abd7bf160926e This will at least turn the really really weird error when a repair of a readonly mutable file is attempted into a merely really weird assertion that mentions "repair currently requires a writecap". ] [one last tweak Brian Warner **20090626052627 Ignore-this: 8d9d045a1ebe774aeed6f7e1cdd36973 ] [more 'tahoe cp' docs tweaks Brian Warner **20090626051652 Ignore-this: f594c395007af64da3d7302943ef8855 ] [cli.py: minor tweaks to test posthooks "Brian Warner "**20090626024557] [cli.py: update comments on 'tahoe cp' --help a bit Brian Warner **20090626023837 Ignore-this: 4090b06896542fbf260d808e8a991cea ] [cli: add some --help text to 'tahoe cp' warner@lothar.com**20090625235751] [make it possible to add/renew-lease from the WUI Brian Warner **20090626061824 Ignore-this: 6fb6a0e8ecfc630464ac37a779078e5 add add/renew-lease checkbox on the "more info" page check/deep-check forms ] [hush pyflakes warner@lothar.com**20090625021809] [Split out NoSharesError, stop adding attributes to NotEnoughSharesError, change humanize_failure to include the original exception string, update tests, behave better if humanize_failure fails. warner@lothar.com**20090625021707] [remove introducer/old.py, will create something similar when the RIIntroducer changes warner@lothar.com**20090624194038] [Basically just a trivial platform detection patch for NetBSD. midnightmagic@example.com**20090617233647] [check_grid.py: print stderr when a subcommand fails warner@lothar.com**20090623022833] [big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging warner@lothar.com**20090623021047] [web/welcome.xhtml: remove trailing whitespace warner@lothar.com**20090623020909] [rrefutil: add trap_deafref utility, to make the callRemote-plus-ignore-DeadReferenceError-plug-log-other-errors pattern easier warner@lothar.com**20090623020826] [PollMixin: snoop trial's error observer to halt the test early if an error is seen. This turns a lot of timeouts into fast failures. warner@lothar.com**20090623020731] [clean up storage_broker interface: should fix #732 warner@lothar.com**20090621235119 Ignore-this: fb93cd670e809eed2bc123142dd8d4ff ] [hush pyflakes with recent FileTooLarge removal warner@lothar.com**20090621231757 Ignore-this: 4231b38c7e9091b0577b07ec99ac2df0 ] [add docs/proposed/GridID.txt (cleaning out some of my old branches) warner@lothar.com**20090621191204] [docs: remove warning about inability to build modules on py2.6 on Windows with mingw, differentiate between clients and servers, reflow to a consistent column width (79), add hint about firewall/NAT docs. zooko@zooko.com**20090621175005 Ignore-this: 85e7c1ccb258317ca4dd37917afb48f5 ] [setup: copy in misc/show-tools-version.py from zfec -- it prints out platform and setuptools versions zooko@zooko.com**20090621055846 Ignore-this: 4e144886ab02414bbaaf0295ce2b337 ] [docs: start updating the NEWS and relnotes.txt files, add Kevan to CREDITS zooko@zooko.com**20090621055114 Ignore-this: 35e05a5739549ffa693d55df51ffcfd ] [change max filesize limit tests kevan@isnotajoke.com**20090620212822 Ignore-this: 38e7c62a308c3c93e79df4bf72f4f675 Instead of testing to see that the previous SDMF filesize limit was being obeyed, we now test to make sure that we can insert files larger than that limit. ] [remove upper limit on SDMF filesize kevan@isnotajoke.com**20090620213130 Ignore-this: 5bc48c7421c73827909a17e651799d0c ] [immutable/download: instrument do-you-have-block responses to investigate #732 warner@lothar.com**20090621041209] [Makefile: add jaunty support, rearrange debian sections in order of release warner@lothar.com**20090618050502] [test_util: add known-answer tests for hashutil tags warner@lothar.com**20090618045709] [tests: bump up a timeout that expired on Zandr's box zooko@zooko.com**20090613195703 Ignore-this: 853000f43665396b3734a99f72bd472d ] [util: hooray! A clean implementation of this simple utility! Black Dew pointed out that the inverse of time.gmtime() is hidden in the "calendar" module. zooko@zooko.com**20090613160112 Ignore-this: 8359cc48ca1b8e2793e8b2afe2050cf4 ] [util: Brian's horrible hack to figure out how much localtime and utctime differ. Now we'll see if it works on Windows. zooko@zooko.com**20090612204556 Ignore-this: 8c36431da4707da76472956c7750ecbd ] [util: oops, time.tzset() doesn't work on Windows -- hopefully the new "London" unit test passes on Windows when we skip tzset() on platforms that don't have it zooko@zooko.com**20090612000920 Ignore-this: 7d314b8334cfa3f65f9635e3d3eb727e ] [setup: edit install.html to warn Windows users away from Python v2.6 zooko@zooko.com**20090611225506 Ignore-this: 89ad63eab49ede883ef92f2de5b5fc54 ] [util: fix time_format.iso_utc_time_to_seconds() so that it works even in London zooko@zooko.com**20090611221129 Ignore-this: 14dbb9840587797f848226fdb3645c08 ] [setup: run the same "make quicktest" on Windows as on non-Windows zooko@zooko.com**20090611193214 Ignore-this: d0c78377d3892373ec3d91e9e98bd8bd I checked and it behaves about as well on Windows as the previous version did. ] [tests: significantly increase timeouts that triggered on Zandr's ARM box zooko@zooko.com**20090610161043 Ignore-this: 2b3c556c5166a8267b4b15664d3aadfb ] [test: multiple by 10 or so all timeouts that Zandr's ARM box just overran zooko@zooko.com**20090610125639 Ignore-this: bf62f063ab46814fd78de55a5fbc9d84 ] [tests: bump up timeout on a test that timed out on draco zooko@zooko.com**20090610044628 Ignore-this: f598b98cbae44dc947937c6ca54c10cb ] [tests: raise the timeout for test_cli since Zandr's ARM machine totally burst through the old one zooko@zooko.com**20090609210509] [test_cli.Backup: increase timeout massively, it takes 1200s on zandr's ARM linkstation warner@lothar.com**20090609052801] [tests: double the timeouts on some tests which time-out on Francois's box zooko@zooko.com**20090609021753 Ignore-this: b2727b04402f24a9b9123d2f84068106 ] [tests: bump up timeouts so that the tests can finish before timeout on Francois's little arm box zooko@zooko.com**20090608225557 Ignore-this: fb83698338b2f12546cd3e1dcb896d34 ] [tests: increase timeouts on some other tests that timed-out on Francois's arm box zooko@zooko.com**20090605143437 Ignore-this: 2903cc20d914fc074c8d7a6c47740ba6 ] [tests: bump up the timeout on a bunch of tests that took longer than the default timeout (120s) on François Lenny-armv5tel zooko@zooko.com**20090605031444 Ignore-this: 84d67849b1f8edc88bf7001e31b5f7f3 ] [backup: remove the --no-backupdb command, the handling of "can't import sqlite", and the related tests, and change an error message to more correctly indicate failure to load the database from disk rather than failure to import sqlite module zooko@zooko.com**20090604173131 Ignore-this: 8200a9fdfc49243c280ecd1d0c44fa19 Fixes #728. ] [setup: require pysqlite >= v2.0.5. if we are running on Python < 2.5 zooko@zooko.com**20090604154548 Ignore-this: cf04f46079821df209d01dad2e24b40b ] [setup: add pysqlite and sqlite to get_package_versions() zooko@zooko.com**20090604153728 Ignore-this: a1dea7fabeab2b08fb0d8d462facdb4d ] [more refactoring: move get_all_serverids() and get_nickname_for_serverid() from Client to storage_broker warner@lothar.com**20090602030750] [more storage_broker refactoring: downloader gets a broker instead of a client, warner@lothar.com**20090602022511 use Client.get_storage_broker() accessor instead of direct attribute access. ] [test_runner.py: remove test_client_no_noise: the issue in question is warner@lothar.com**20090601225007 ticketed in http://divmod.org/trac/ticket/2830 and doesn't need a Tahoe-side change, plus this test fails on win32 for unrelated reasons (and test_client is the place to think about the win32 issue). ] [remove plaintext-hashing code from the helper interface, to close #722 warner@lothar.com**20090601224916 and deny the Helper the ability to mount a partial-information-guessing attack. This will probably break compatibility between new clients and very old (pre-1.0) helpers. ] [start to factor server-connection-management into a distinct 'StorageServerFarmBroker' object, separate from the client and the introducer. This is the starting point for #467: static server selection warner@lothar.com**20090601210604] [mutable: catch and display first error, so code bugs which break all servers get displayed better warner@lothar.com**20090601210407] [misc/run-with-pythonpath.py: exec() the child (on unix), to remove the intermediate process warner@lothar.com**20090601210137] [docs: small edit to about.html zooko@zooko.com**20090528233422 Ignore-this: 1cfbb1f8426ed6d63b2d3952e4464ddc ] [docs: add links to Tahoe-LAFS for Paranoids and Tahoe-LAFS for Corporates in about.html zooko@zooko.com**20090528232717 Ignore-this: 7b70baa700d6b6f6e9ceec4132efe5 ] [docs: edit about.html and include network-and-reliance-topology.png (loaded from http://allmydata.org ) zooko@zooko.com**20090527150916 Ignore-this: 44adc61cde8ced8be2f0a7dfc7d95dad ] [docs: a few more edits to network-and-reliance-topology.svg zooko@zooko.com**20090527150458 Ignore-this: 2eac8c33fe71be25ff809b399c6193c1 ] [docs: update network-and-reliance-topology.svg for beauty and clarity zooko@zooko.com**20090527031123 Ignore-this: 5510914849771900ac29b4312470d84 ] [docs: update NEWS, relnotes.txt, CREDITS to mention WUI Style zooko@zooko.com**20090526233654 Ignore-this: 72d16ec833bc4a22af23d29ea1d5ff8b ] [Modify markup of Tahoe web pages to be more amenable to styling; some minor changes of wording. Kevin Reid **20090526232545 Ignore-this: 8845937f0df6c7ddc07abe3211428a6f ] [Tweak wording in directory page: not-read-only is "modifiable", mention creating a directory _in this directory_. Kevin Reid **20090526232414 Ignore-this: f006ec52ba2051802e025a60bcface56 ] [Comment on duplication of code/markup found during styling project. Kevin Reid **20090503203442 Ignore-this: a4b7f9f0ab57d2c03be9ba761be8d854 ] [Add CSS styles to spiff up the Tahoe WUI's appearance, particularly the welcome page and directories. Kevin Reid **20090503203142 Ignore-this: 5c50af241c1a958b5180ef2b6a49f626 ] [Link all Tahoe web pages to the /tahoe_css stylesheet which already exists. Kevin Reid **20090503202533 Ignore-this: 2ea8d14d3168b9502cf39d5ea3f2f2a8 ] [Fix broken link from Provisioning to Reliability page. Kevin Reid **20090501191050 Ignore-this: 56dc1a5e659b70cc02dc4df7b5d518cd ] [docs: network-and-reliance-topology.svg: nicer server icons, mv out of the "specifications" subdir zooko@zooko.com**20090526165842 Ignore-this: 8f47ab3a0ab782c1f0d46e10bcaebe5b ] [docs: update network-and-reliance-topology.svg zooko@zooko.com**20090526163105 Ignore-this: 2b864b4ed8743d4a15dfbb7eff3fa561 ] [accounting-overview.txt: more edits warner@lothar.com**20090523190359] [accounting-overview.txt: small edits warner@lothar.com**20090523184011] [_auto_deps.py: require foolscap-0.4.1, which adds an important fix for py2.4 warner@lothar.com**20090523011103] [immutable/encode.py: tolerate immediate _remove_shareholder by copying the warner@lothar.com**20090522184424 landlord list before iterating over it. This can probably only happen in unit tests, but cleaning it up makes certain test failures easier to analyze. ] [switch to using RemoteException instead of 'wrapped' RemoteReferences. Should fix #653, the rref-EQ problem warner@lothar.com**20090522004632] [switch all foolscap imports to use foolscap.api or foolscap.logging warner@lothar.com**20090522003823] [_auto_deps.py: bump our foolscap dependency to 0.4.0, since I'm about to start using its new features warner@lothar.com**20090522002100] [test_runner.py: fix minor typo warner@lothar.com**20090520033620] [setup: fix bug (wrong import) in error message, as noticed by pyflakes zooko@zooko.com**20090519195642 Ignore-this: f1b9f8c00b46c1b5f2f20e5fc424f341 ] [setup: fix trivial bug in recent patch to test base64.py at startup zooko@zooko.com**20090519195129 Ignore-this: f6be038f74b53ca69e7109fe34adfbc ] [setup: make Tahoe exit at startup with a useful error message if the base64.py module is buggy (fixes part of #710) zooko@zooko.com**20090519194555 Ignore-this: aa4d398235ddca8d417d61c9688e154 ] [test_introducer.py: add a test for the python2.4.0/2.4.1 bug in base64.b32decode warner@lothar.com**20090519034101] [immutable WriteBucketProxy: use pipeline to speed up uploads by overlapping roundtrips, for #392 warner@lothar.com**20090518234422] [util/pipeline.py: new utility class to manage size-limited work pipelines, for #392 warner@lothar.com**20090518234326] [docs: add a diagram that I'm about to show to the Boulder Linux Users Group: network-and-reliance-topology.svg zooko@zooko.com**20090514232059 Ignore-this: 2420c0a7c254c9f0f2349d9130490d33 ] [tests: mark test_runner as coded in utf-8 instead of ascii zooko@zooko.com**20090507223151 Ignore-this: ccf1ba9e5a9b53602701a36f9fdb545e ] [tests: raise timeout on test_runner.RunNode.test_introducer from 120s to 240s, since it hit the 120s time-out on François Lenny-armv5tel zooko@zooko.com**20090507215012 Ignore-this: ba18fe6832ba255d4971e8f623ed7da5 ] [setup: fix comment in setup.py zooko@zooko.com**20090507215003 Ignore-this: c46ef664630d52733138ef7fbc551c1c ] [docs: how_to_make_a_tahoe_release.txt: a couple of small edits zooko@zooko.com**20090507214932 Ignore-this: ae92aa835ad369f4b9e6e49d681957a3 ] [.darcs-boringfile: also ignore .gitignore warner@allmydata.com**20090415210550 Ignore-this: d29db314a1e506f6240859559436b4c3 ] [.darcs-boringfile: ignore .git, I'm starting to play around with it warner@allmydata.com**20090415205929 Ignore-this: 89234453516483c9586cd6e1351e88b5 ] [fix quicktest: stop using setuptools, add misc/run-with-pythonpath.py, to make it run faster warner@lothar.com**20090414201400] [TAG allmydata-tahoe-1.4.1 zooko@zooko.com**20090414025636 Ignore-this: de78fc32364c83e9f4e26b5abcfdea4a ] Patch bundle hash: 970239fd6100a76c8c0501a5ab5a0435b066f6c1