Sat Nov 21 05:56:44 GMT Standard Time 2009 david-sarah@jacaranda.org * storage server: detect disk space usage on Windows too (fixes #637) New patches: [storage server: detect disk space usage on Windows too (fixes #637) david-sarah@jacaranda.org**20091121055644 Ignore-this: 20fb30498174ce997befac7701fab056 ] { hunk ./_auto_deps.py 40 install_requires.append("pysqlite >= 2.0.5") ## The following block is commented-out because there is not currently a pywin32 package which -## can be easy_install'ed and also which actually makes "import win32api" succeed. Users have -## to manually install pywin32 on Windows before installing Tahoe. +## can be easy_install'ed and also which actually makes "import win32api" succeed. +## See http://sourceforge.net/tracker/index.php?func=detail&aid=1799934&group_id=78018&atid=551954 +## Users have to manually install pywin32 on Windows before installing Tahoe. ##import platform ##if platform.system() == "Windows": ## # Twisted requires pywin32 if it is going to offer process management functionality, or if hunk ./_auto_deps.py 48 ## # it is going to offer iocp reactor. We currently require process management. It would be ## # better if Twisted would declare that it requires pywin32 if it is going to offer process -## # management. Then the specification and the evolution of Twisted's reliance on pywin32 can -## # be confined to the Twisted setup data, and Tahoe can remain blissfully ignorant about such -## # things as if a future version of Twisted requires a different version of pywin32, or if a -## # future version of Twisted implements process management without using pywin32 at all, -## # etc.. That is twisted ticket #3238 -- http://twistedmatrix.com/trac/ticket/3238 . But -## # until Twisted does that, Tahoe needs to be non-ignorant of the following requirement: +## # management. That is twisted ticket #3238 -- http://twistedmatrix.com/trac/ticket/3238 . +## # On the other hand, Tahoe also depends on pywin32 for getting free disk space statistics +## # (although that is not a hard requirement: if win32api can't be imported then we don't +## # rely on having the disk stats). ## install_requires.append('pywin32') if hasattr(sys, 'frozen'): # for py2exe hunk ./docs/configuration.txt 305 reserved_space = (str, optional) If provided, this value defines how much disk space is reserved: the storage - server will not accept any share which causes the amount of free space (as - measured by 'df', or more specifically statvfs(2)) to drop below this value. + server will not accept any share which causes the amount of free disk space + to drop below this value. (The free space is measured by a call to statvfs(2) + on Unix, or GetDiskFreeSpaceEx on Windows, and is the space available to the + user account under which the storage server runs.) This string contains a number, with an optional case-insensitive scale suffix like "K" or "M" or "G", and an optional "B" or "iB" suffix. So hunk ./src/allmydata/storage/server.py 39 implements(RIStorageServer, IStatsProducer) name = 'storage' LeaseCheckerClass = LeaseCheckingCrawler + windows = False + + try: + import win32api, win32con + windows = True + # + win32api.SetErrorMode(win32con.SEM_FAILCRITICALERRORS | + win32con.SEM_NOOPENFILEERRORBOX) + except ImportError: + pass def __init__(self, storedir, nodeid, reserved_space=0, discard_storage=False, readonly_storage=False, hunk ./src/allmydata/storage/server.py 83 if reserved_space: if self.get_available_space() is None: - log.msg("warning: [storage]reserved_space= is set, but this platform does not support statvfs(2), so this reservation cannot be honored", + log.msg("warning: [storage]reserved_space= is set, but this platform does not support an API to get disk statistics (statvfs(2) or GetDiskFreeSpaceEx), so this reservation cannot be honored", umin="0wZ27w", level=log.UNUSUAL) self.latencies = {"allocate": [], # immutable hunk ./src/allmydata/storage/server.py 160 def _clean_incomplete(self): fileutil.rm_dir(self.incomingdir) - def do_statvfs(self): - return os.statvfs(self.storedir) + def get_disk_stats(self): + """Return disk statistics for the storage disk, in the form of a dict + with the following fields. + total: total bytes on disk + free_for_root: bytes actually free on disk + free_for_nonroot: bytes free for "a non-privileged user" [Unix] or + the current user [Windows]; might take into + account quotas depending on platform + used: bytes used on disk + avail: bytes available excluding reserved space + An AttributeError can occur if the OS has no API to get disk information. + An EnvironmentError can occur if the OS call fails.""" + + if self.windows: + # For Windows systems, where os.statvfs is not available, use GetDiskFreeSpaceEx. + # + # + # Although the docs say that the argument should be the root directory + # of a disk, GetDiskFreeSpaceEx actually accepts any path on that disk + # (like its Win32 equivalent). + + (free_for_nonroot, total, free_for_root) = self.win32api.GetDiskFreeSpaceEx(self.storedir) + else: + # For Unix-like systems. + # + # + # + s = os.statvfs(self.storedir) hunk ./src/allmydata/storage/server.py 189 - def get_stats(self): - # remember: RIStatsProvider requires that our return dict - # contains numeric values. - stats = { 'storage_server.allocated': self.allocated_size(), } - stats["storage_server.reserved_space"] = self.reserved_space - for category,ld in self.get_latencies().items(): - for name,v in ld.items(): - stats['storage_server.latencies.%s.%s' % (category, name)] = v - writeable = True - if self.readonly_storage: - writeable = False - try: - s = self.do_statvfs() # on my mac laptop: # statvfs(2) is a wrapper around statfs(2). # statvfs.f_frsize = statfs.f_bsize : hunk ./src/allmydata/storage/server.py 199 # wrong, and s.f_blocks*s.f_frsize is twice the size of my disk, # but s.f_bavail*s.f_frsize is correct - disk_total = s.f_frsize * s.f_blocks - disk_used = s.f_frsize * (s.f_blocks - s.f_bfree) - # spacetime predictors should look at the slope of disk_used. - disk_free_for_root = s.f_frsize * s.f_bfree - disk_free_for_nonroot = s.f_frsize * s.f_bavail + total = s.f_frsize * s.f_blocks + free_for_root = s.f_frsize * s.f_bfree + free_for_nonroot = s.f_frsize * s.f_bavail + + # valid for all platforms: + used = total - free_for_root + avail = max(free_for_nonroot - self.reserved_space, 0) hunk ./src/allmydata/storage/server.py 207 - # include our local policy here: if we stop accepting shares when - # the available space drops below 1GB, then include that fact in - # disk_avail. - disk_avail = disk_free_for_nonroot - self.reserved_space - disk_avail = max(disk_avail, 0) - if self.readonly_storage: - disk_avail = 0 - if disk_avail == 0: - writeable = False + return { 'total': total, 'free_for_root': free_for_root, + 'free_for_nonroot': free_for_nonroot, + 'used': used, 'avail': avail, } + + def get_stats(self): + # remember: RIStatsProvider requires that our return dict + # contains numeric values. + stats = { 'storage_server.allocated': self.allocated_size(), } + stats['storage_server.reserved_space'] = self.reserved_space + for category,ld in self.get_latencies().items(): + for name,v in ld.items(): + stats['storage_server.latencies.%s.%s' % (category, name)] = v + + try: + disk = self.get_disk_stats() + writeable = disk['avail'] > 0 # spacetime predictors should use disk_avail / (d(disk_used)/dt) hunk ./src/allmydata/storage/server.py 225 - stats["storage_server.disk_total"] = disk_total - stats["storage_server.disk_used"] = disk_used - stats["storage_server.disk_free_for_root"] = disk_free_for_root - stats["storage_server.disk_free_for_nonroot"] = disk_free_for_nonroot - stats["storage_server.disk_avail"] = disk_avail + stats['storage_server.disk_total'] = disk['total'] + stats['storage_server.disk_used'] = disk['used'] + stats['storage_server.disk_free_for_root'] = disk['free_for_root'] + stats['storage_server.disk_free_for_nonroot'] = disk['free_for_nonroot'] + stats['storage_server.disk_avail'] = disk['avail'] except AttributeError: hunk ./src/allmydata/storage/server.py 231 - # os.statvfs is available only on unix - pass - stats["storage_server.accepting_immutable_shares"] = int(writeable) + writeable = True + except EnvironmentError: + log.msg("OS call to get disk statistics failed", level=log.UNUSUAL) + writeable = False + + if self.readonly_storage: + stats['storage_server.disk_avail'] = 0 + writeable = False + + stats['storage_server.accepting_immutable_shares'] = int(writeable) s = self.bucket_counter.get_state() bucket_count = s.get("last-complete-bucket-count") if bucket_count: hunk ./src/allmydata/storage/server.py 244 - stats["storage_server.total_bucket_count"] = bucket_count + stats['storage_server.total_bucket_count'] = bucket_count return stats hunk ./src/allmydata/storage/server.py 247 - - def stat_disk(self, d): - s = os.statvfs(d) - # s.f_bavail: available to non-root users - disk_avail = s.f_frsize * s.f_bavail - return disk_avail - def get_available_space(self): hunk ./src/allmydata/storage/server.py 248 - # returns None if it cannot be measured (windows) + """Returns available space for share storage in bytes, or None if no + API to get this information is available.""" + + if self.readonly_storage: + return 0 try: hunk ./src/allmydata/storage/server.py 254 - disk_avail = self.stat_disk(self.storedir) - disk_avail -= self.reserved_space + return self.get_disk_stats()['avail'] except AttributeError: hunk ./src/allmydata/storage/server.py 256 - disk_avail = None - if self.readonly_storage: - disk_avail = 0 - return disk_avail + return None + except EnvironmentError: + log.msg("OS call to get disk statistics failed", level=log.UNUSUAL) + return 0 def allocated_size(self): space = 0 hunk ./src/allmydata/storage/server.py 270 def remote_get_version(self): remaining_space = self.get_available_space() if remaining_space is None: - # we're on a platform that doesn't have 'df', so make a vague - # guess. + # We're on a platform that has no API to get disk stats. remaining_space = 2**64 hunk ./src/allmydata/storage/server.py 272 + version = { "http://allmydata.org/tahoe/protocols/storage/v1" : { "maximum-immutable-share-size": remaining_space, "tolerates-immutable-read-overrun": True, hunk ./src/allmydata/storage/server.py 326 sf = ShareFile(fn) sf.add_or_renew_lease(lease_info) - # self.readonly_storage causes remaining_space=0 + # self.readonly_storage causes remaining_space <= 0 for shnum in sharenums: incominghome = os.path.join(self.incomingdir, si_dir, "%d" % shnum) hunk ./src/allmydata/test/test_storage.py 231 0x44, WriteBucketProxy_v2, ReadBucketProxy) class FakeDiskStorageServer(StorageServer): - def stat_disk(self, d): - return self.DISKAVAIL + DISKAVAIL = 0 + def get_disk_stats(self): + return { 'free_for_nonroot': self.DISKAVAIL, 'avail': max(self.DISKAVAIL - self.reserved_space, 0), } class Server(unittest.TestCase): hunk ./src/allmydata/test/test_storage.py 416 def test_reserved_space(self): ss = self.create("test_reserved_space", reserved_space=10000, klass=FakeDiskStorageServer) - # the FakeDiskStorageServer doesn't do real statvfs() calls + # the FakeDiskStorageServer doesn't do real calls to get_disk_stats ss.DISKAVAIL = 15000 # 15k available, 10k reserved, leaves 5k for shares hunk ./src/allmydata/test/test_storage.py 472 ss.disownServiceParent() del ss + def test_disk_stats(self): + # This will spuriously fail if there is zero disk space left (but so will other tests). + ss = self.create("test_disk_stats", reserved_space=0) + + disk = ss.get_disk_stats() + self.failUnless(disk['total'] > 0, disk['total']) + self.failUnless(disk['used'] > 0, disk['used']) + self.failUnless(disk['free_for_root'] > 0, disk['free_for_root']) + self.failUnless(disk['free_for_nonroot'] > 0, disk['free_for_nonroot']) + self.failUnless(disk['avail'] > 0, disk['avail']) + + def test_disk_stats_avail_nonnegative(self): + ss = self.create("test_disk_stats_avail_nonnegative", reserved_space=2**64) + + disk = ss.get_disk_stats() + self.failUnlessEqual(disk['avail'], 0) + def test_seek(self): basedir = self.workdir("test_seek_behavior") fileutil.make_dirs(basedir) hunk ./src/allmydata/test/test_storage.py 645 self.failUnlessEqual(writers, {}) stats = ss.get_stats() - self.failUnlessEqual(stats["storage_server.accepting_immutable_shares"], - False) + self.failUnlessEqual(stats["storage_server.accepting_immutable_shares"], 0) if "storage_server.disk_avail" in stats: hunk ./src/allmydata/test/test_storage.py 647 - # windows does not have os.statvfs, so it doesn't give us disk - # stats. But if there are stats, readonly_storage means - # disk_avail=0 + # Some platforms may not have an API to get disk stats. + # But if there are stats, readonly_storage means disk_avail=0 self.failUnlessEqual(stats["storage_server.disk_avail"], 0) def test_discard(self): hunk ./src/allmydata/test/test_storage.py 2424 d = self.render1(page, args={"t": ["json"]}) return d -class NoStatvfsServer(StorageServer): - def do_statvfs(self): +class NoDiskStatsServer(StorageServer): + def get_disk_stats(self): raise AttributeError hunk ./src/allmydata/test/test_storage.py 2428 +class BadDiskStatsServer(StorageServer): + def get_disk_stats(self): + raise OSError + class WebStatus(unittest.TestCase, pollmixin.PollMixin, WebRenderingMixin): def setUp(self): hunk ./src/allmydata/test/test_storage.py 2473 d = self.render1(page, args={"t": ["json"]}) return d - def test_status_no_statvfs(self): - # windows has no os.statvfs . Make sure the code handles that even on - # unix. - basedir = "storage/WebStatus/status_no_statvfs" + def test_status_no_disk_stats(self): + # Some platforms may have no disk stats API. Make sure the code can handle that + # (test runs on all platforms). + basedir = "storage/WebStatus/status_no_disk_stats" fileutil.make_dirs(basedir) hunk ./src/allmydata/test/test_storage.py 2478 - ss = NoStatvfsServer(basedir, "\x00" * 20) + ss = NoDiskStatsServer(basedir, "\x00" * 20) ss.setServiceParent(self.s) w = StorageStatus(ss) html = w.renderSynchronously() hunk ./src/allmydata/test/test_storage.py 2486 s = remove_tags(html) self.failUnless("Accepting new shares: Yes" in s, s) self.failUnless("Total disk space: ?" in s, s) + self.failUnless("Space Available to Tahoe: ?" in s, s) + self.failUnless(ss.get_available_space() is None) + + def test_status_bad_disk_stats(self): + # If the API to get disk stats exists but a call to it fails, then the status should + # show that no shares will be accepted, and get_available_space() should be 0. + basedir = "storage/WebStatus/status_bad_disk_stats" + fileutil.make_dirs(basedir) + ss = BadDiskStatsServer(basedir, "\x00" * 20) + ss.setServiceParent(self.s) + w = StorageStatus(ss) + html = w.renderSynchronously() + self.failUnless("

Storage Server Status

" in html, html) + s = remove_tags(html) + self.failUnless("Accepting new shares: No" in s, s) + self.failUnless("Total disk space: ?" in s, s) + self.failUnless("Space Available to Tahoe: ?" in s, s) + self.failUnless(ss.get_available_space() == 0) def test_readonly(self): basedir = "storage/WebStatus/readonly" } Context: [webapi: use t=mkdir-with-children instead of a children= arg to t=mkdir . Brian Warner **20091026011321 Ignore-this: 769cab30b6ab50db95000b6c5a524916 This is safer: in the earlier API, an old webapi server would silently ignore the initial children, and clients trying to set them would have to fetch the newly-created directory to discover the incompatibility. In the new API, clients using t=mkdir-with-children against an old webapi server will get a clear error. ] [nodemaker.create_new_mutable_directory: pack_children() in initial_contents= Brian Warner **20091020005118 Ignore-this: bd43c4eefe06fd32b7492bcb0a55d07e instead of creating an empty file and then adding the children later. This should speed up mkdir(initial_children) considerably, removing two roundtrips and an entire read-modify-write cycle, probably bringing it down to a single roundtrip. A quick test (against the volunteergrid) suggests a 30% speedup. test_dirnode: add new tests to enforce the restrictions that interfaces.py claims for create_new_mutable_directory(): no UnknownNodes, metadata dicts ] [test_dirnode.py: add tests of initial_children= args to client.create_dirnode Brian Warner **20091017194159 Ignore-this: 2e2da28323a4d5d815466387914abc1b and nodemaker.create_new_mutable_directory ] [update many dirnode interfaces to accept dict-of-nodes instead of dict-of-caps Brian Warner **20091017192829 Ignore-this: b35472285143862a856bf4b361d692f0 interfaces.py: define INodeMaker, document argument values, change create_new_mutable_directory() to take dict-of-nodes. Change dirnode.set_nodes() and dirnode.create_subdirectory() too. nodemaker.py: use INodeMaker, update create_new_mutable_directory() client.py: have create_dirnode() delegate initial_children= to nodemaker dirnode.py (Adder): take dict-of-nodes instead of list-of-nodes, which updates set_nodes() and create_subdirectory() web/common.py (convert_initial_children_json): create dict-of-nodes web/directory.py: same web/unlinked.py: same test_dirnode.py: update tests to match ] [dirnode.py: move pack_children() out to a function, for eventual use by others Brian Warner **20091017180707 Ignore-this: 6a823fb61f2c180fd38d6742d3196a7a ] [move dirnode.CachingDict to dictutil.AuxValueDict, generalize method names, Brian Warner **20091017180005 Ignore-this: b086933cf429df0fcea16a308d2640dd improve tests. Let dirnode _pack_children accept either dict or AuxValueDict. ] [test/common.py: update FakeMutableFileNode to new contents= callable scheme Brian Warner **20091013052154 Ignore-this: 62f00a76454a2190d1c8641c5993632f ] [The initial_children= argument to nodemaker.create_new_mutable_directory is Brian Warner **20091013031922 Ignore-this: 72e45317c21f9eb9ec3bd79bd4311f48 now enabled. ] [client.create_mutable_file(contents=) now accepts a callable, which is Brian Warner **20091013031232 Ignore-this: 3c89d2f50c1e652b83f20bd3f4f27c4b invoked with the new MutableFileNode and is supposed to return the initial contents. This can be used by e.g. a new dirnode which needs the filenode's writekey to encrypt its initial children. create_mutable_file() still accepts a bytestring too, or None for an empty file. ] [webapi: t=mkdir now accepts initial children, using the same JSON that t=json Brian Warner **20091013023444 Ignore-this: 574a46ed46af4251abf8c9580fd31ef7 emits. client.create_dirnode(initial_children=) now works. ] [replace dirnode.create_empty_directory() with create_subdirectory(), which Brian Warner **20091013021520 Ignore-this: 6b57cb51bcfcc6058d0df569fdc8a9cf takes an initial_children= argument ] [dirnode.set_children: change return value: fire with self instead of None Brian Warner **20091013015026 Ignore-this: f1d14e67e084e4b2a4e25fa849b0e753 ] [dirnode.set_nodes: change return value: fire with self instead of None Brian Warner **20091013014546 Ignore-this: b75b3829fb53f7399693f1c1a39aacae ] [dirnode.set_children: take a dict, not a list Brian Warner **20091013002440 Ignore-this: 540ce72ce2727ee053afaae1ff124e21 ] [dirnode.set_uri/set_children: change signature to take writecap+readcap Brian Warner **20091012235126 Ignore-this: 5df617b2d379a51c79148a857e6026b1 instead of a single cap. The webapi t=set_children call benefits too. ] [replace Client.create_empty_dirnode() with create_dirnode(), in anticipation Brian Warner **20091012224506 Ignore-this: cbdaa4266ecb3c6496ffceab4f95709d of adding initial_children= argument. Includes stubbed-out initial_children= support. ] [test_web.py: use a less-fake client, making test harness smaller Brian Warner **20091012222808 Ignore-this: 29e95147f8c94282885c65b411d100bb ] [webapi.txt: document t=set_children, other small edits Brian Warner **20091009200446 Ignore-this: 4d7e76b04a7b8eaa0a981879f778ea5d ] [Verifier: check the full cryptext-hash tree on each share. Removed .todos Brian Warner **20091005221849 Ignore-this: 6fb039c5584812017d91725e687323a5 from the last few test_repairer tests that were waiting on this. ] [Verifier: check the full block-hash-tree on each share Brian Warner **20091005214844 Ignore-this: 3f7ccf6d253f32340f1bf1da27803eee Removed the .todo from two test_repairer tests that check this. The only remaining .todos are on the three crypttext-hash-tree tests. ] [Verifier: check the full share-hash chain on each share Brian Warner **20091005213443 Ignore-this: 3d30111904158bec06a4eac22fd39d17 Removed the .todo from two test_repairer tests that check this. ] [test_repairer: rename Verifier test cases to be more precise and less verbose Brian Warner **20091005201115 Ignore-this: 64be7094e33338c7c2aea9387e138771 ] [immutable/checker.py: rearrange code a little bit, make it easier to follow Brian Warner **20091005200252 Ignore-this: 91cc303fab66faf717433a709f785fb5 ] [test/common.py: wrap docstrings to 80cols so I can read them more easily Brian Warner **20091005200143 Ignore-this: b180a3a0235cbe309c87bd5e873cbbb3 ] [immutable/download.py: wrap to 80cols, no functional changes Brian Warner **20091005192542 Ignore-this: 6b05fe3dc6d78832323e708b9e6a1fe ] [CHK-hashes.svg: cross out plaintext hashes, since we don't include Brian Warner **20091005010803 Ignore-this: bea2e953b65ec7359363aa20de8cb603 them (until we finish #453) ] [docs: a few licensing clarifications requested by Ubuntu zooko@zooko.com**20090927033226 Ignore-this: 749fc8c9aeb6dc643669854a3e81baa7 ] [setup: remove binary WinFUSE modules zooko@zooko.com**20090924211436 Ignore-this: 8aefc571d2ae22b9405fc650f2c2062 I would prefer to have just source code, or indications of what 3rd-party packages are required, under revision control, and have the build process generate o r acquire the binaries as needed. Also, having these in our release tarballs is interfering with getting Tahoe-LAFS uploaded into Ubuntu Karmic. (Technicall y, they would accept binary modules as long as they came with the accompanying source so that they could satisfy their obligations under GPL2+ and TGPPL1+, bu t it is easier for now to remove the binaries from the source tree.) In this case, the binaries are from the tahoe-w32-client project: http://allmydata.org/trac/tahoe-w32-client , from which you can also get the source. ] [setup: remove binary _fusemodule.so 's zooko@zooko.com**20090924211130 Ignore-this: 74487bbe27d280762ac5dd5f51e24186 I would prefer to have just source code, or indications of what 3rd-party packages are required, under revision control, and have the build process generate or acquire the binaries as needed. Also, having these in our release tarballs is interfering with getting Tahoe-LAFS uploaded into Ubuntu Karmic. (Technically, they would accept binary modules as long as they came with the accompanying source so that they could satisfy their obligations under GPL2+ and TGPPL1+, but it is easier for now to remove the binaries from the source tree.) In this case, these modules come from the MacFUSE project: http://code.google.com/p/macfuse/ ] [doc: add a copy of LGPL2 for documentation purposes for ubuntu zooko@zooko.com**20090924054218 Ignore-this: 6a073b48678a7c84dc4fbcef9292ab5b ] [setup: remove a convenience copy of figleaf, to ease inclusion into Ubuntu Karmic Koala zooko@zooko.com**20090924053215 Ignore-this: a0b0c990d6e2ee65c53a24391365ac8d We need to carefully document the licence of figleaf in order to get Tahoe-LAFS into Ubuntu Karmic Koala. However, figleaf isn't really a part of Tahoe-LAFS per se -- this is just a "convenience copy" of a development tool. The quickest way to make Tahoe-LAFS acceptable for Karmic then, is to remove figleaf from the Tahoe-LAFS tarball itself. People who want to run figleaf on Tahoe-LAFS (as everyone should want) can install figleaf themselves. I haven't tested this -- there may be incompatibilities between upstream figleaf and the copy that we had here... ] [setup: shebang for misc/build-deb.py to fail quickly zooko@zooko.com**20090819135626 Ignore-this: 5a1b893234d2d0bb7b7346e84b0a6b4d Without this patch, when I ran "chmod +x ./misc/build-deb.py && ./misc/build-deb.py" then it hung indefinitely. (I wonder what it was doing.) ] [docs: Shawn Willden grants permission for his contributions under GPL2+|TGPPL1+ zooko@zooko.com**20090921164651 Ignore-this: ef1912010d07ff2ffd9678e7abfd0d57 ] [docs: Csaba Henk granted permission to license fuse.py under the same terms as Tahoe-LAFS itself zooko@zooko.com**20090921154659 Ignore-this: c61ba48dcb7206a89a57ca18a0450c53 ] [setup: mark setup.py as having utf-8 encoding in it zooko@zooko.com**20090920180343 Ignore-this: 9d3850733700a44ba7291e9c5e36bb91 ] [doc: licensing cleanups zooko@zooko.com**20090920171631 Ignore-this: 7654f2854bf3c13e6f4d4597633a6630 Use nice utf-8 © instead of "(c)". Remove licensing statements on utility modules that have been assigned to allmydata.com by their original authors. (Nattraverso was not assigned to allmydata.com -- it was LGPL'ed -- but I checked and src/allmydata/util/iputil.py was completely rewritten and doesn't contain any line of code from nattraverso.) Add notes to misc/debian/copyright about licensing on files that aren't just allmydata.com-licensed. ] [build-deb.py: run darcsver early, otherwise we get the wrong version later on Brian Warner **20090918033620 Ignore-this: 6635c5b85e84f8aed0d8390490c5392a ] [new approach for debian packaging, sharing pieces across distributions. Still experimental, still only works for sid. warner@lothar.com**20090818190527 Ignore-this: a75eb63db9106b3269badbfcdd7f5ce1 ] [new experimental deb-packaging rules. Only works for sid so far. Brian Warner **20090818014052 Ignore-this: 3a26ad188668098f8f3cc10a7c0c2f27 ] [setup.py: read _version.py and pass to setup(version=), so more commands work Brian Warner **20090818010057 Ignore-this: b290eb50216938e19f72db211f82147e like "setup.py --version" and "setup.py --fullname" ] [test/check_speed.py: fix shbang line Brian Warner **20090818005948 Ignore-this: 7f3a37caf349c4c4de704d0feb561f8d ] [setup: remove bundled version of darcsver-1.2.1 zooko@zooko.com**20090816233432 Ignore-this: 5357f26d2803db2d39159125dddb963a That version of darcsver emits a scary error message when the darcs executable or the _darcs subdirectory is not found. This error is hidden (unless the --loud option is passed) in darcsver >= 1.3.1. Fixes #788. ] [de-Service-ify Helper, pass in storage_broker and secret_holder directly. Brian Warner **20090815201737 Ignore-this: 86b8ac0f90f77a1036cd604dd1304d8b This makes it more obvious that the Helper currently generates leases with the Helper's own secrets, rather than getting values from the client, which is arguably a bug that will likely be resolved with the Accounting project. ] [immutable.Downloader: pass StorageBroker to constructor, stop being a Service Brian Warner **20090815192543 Ignore-this: af5ab12dbf75377640a670c689838479 child of the client, access with client.downloader instead of client.getServiceNamed("downloader"). The single "Downloader" instance is scheduled for demolition anyways, to be replaced by individual filenode.download calls. ] [tests: double the timeout on test_runner.RunNode.test_introducer since feisty hit a timeout zooko@zooko.com**20090815160512 Ignore-this: ca7358bce4bdabe8eea75dedc39c0e67 I'm not sure if this is an actual timing issue (feisty is running on an overloaded VM if I recall correctly), or it there is a deeper bug. ] [stop making History be a Service, it wasn't necessary Brian Warner **20090815114415 Ignore-this: b60449231557f1934a751c7effa93cfe ] [Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. Brian Warner **20090815112846 Ignore-this: 1db1b9c149a60a310228aba04c5c8e5f * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo ] [docs: edits for docs/running.html from Sam Mason zooko@zooko.com**20090809201416 Ignore-this: 2207e80449943ebd4ed50cea57c43143 ] [docs: install.html: instruct Debian users to use this document and not to go find the DownloadDebianPackages page, ignore the warning at the top of it, and try it zooko@zooko.com**20090804123840 Ignore-this: 49da654f19d377ffc5a1eff0c820e026 http://allmydata.org/pipermail/tahoe-dev/2009-August/002507.html ] [docs: relnotes.txt: reflow to 63 chars wide because google groups and some web forms seem to wrap to that zooko@zooko.com**20090802135016 Ignore-this: 53b1493a0491bc30fb2935fad283caeb ] [docs: about.html: fix English usage noticed by Amber zooko@zooko.com**20090802050533 Ignore-this: 89965c4650f9bd100a615c401181a956 ] [docs: fix mis-spelled word in about.html zooko@zooko.com**20090802050320 Ignore-this: fdfd0397bc7cef9edfde425dddeb67e5 ] [TAG allmydata-tahoe-1.5.0 zooko@zooko.com**20090802031303 Ignore-this: 94e5558e7225c39a86aae666ea00f166 ] Patch bundle hash: 299212e00f98eb7b7ce509bcda9b95545cca18d9