Ticket #1382: interface.dpatch

Mon Mar 28 20:00:59 PDT 2011  Kevan Carstensen <kevan@isnotajoke.com>
  * interfaces: Create "IPeerSelector", an interface for an abstract peer selection class.

New patches:

[interfaces: Create "IPeerSelector", an interface for an abstract peer selection class.
Kevan Carstensen <kevan@isnotajoke.com>**20110329030059
 Ignore-this: 6bd8ad4056a7d75c7349876c59de8848
] hunk ./src/allmydata/interfaces.py 602
     """I am a node which represents a file: a sequence of bytes. I am not a
     container, like IDirectoryNode."""
 
+class IPeerSelector(Interface):
+    """
+    I select peers for an upload, maximizing some measure of health.
+
+    I keep track of the state of a grid relative to a file. This means
+    that I know about all of the peers that parts of that file could be
+    placed on, and about shares that have been placed on those peers.
+    Given this, I assign shares to peers in a way that maximizes the
+    file's health according to whichever definition of health I am
+    programmed with. I tell the uploader whether or not my assignment is
+    healthy. I keep track of failures during the process and update my
+    conclusions appropriately.
+    """
+    def add_peer_with_share(peerid, shnum, existing=False):
+        """
+        Update my internal state to reflect the fact that peer peerid
+        holds share shnum. Called for shares that are detected before
+        peer selection begins, and for shares that are successfully
+        allocated after peer selection ends.
+
+        I take an optional parameter called existing. This is a hint
+        that the peer => share relationship I'm being told about already
+        exists. I may use this hint in heuristics designed to minimize
+        the amount of share data uploaded to the grid.
+        """
+
+    def add_peers(peerids=set):
+        """
+        Update my internal state to include the peers in peerids as
+        potential candidates for storing a file.
+        """
+
+    def mark_full_peer(peerid):
+        """
+        Mark the peer peerid as full. This means that any
+        peer-with-share relationships I know about for peerid remain
+        valid, but that peerid will not be assigned any new shares.
+        """
+
+    def mark_bad_peer(peerid):
+        """
+        Mark the peer peerid as bad. This is typically called when an
+        error is encountered when communicating with a peer. I will
+        disregard any existing peer => share relationships associated
+        with peerid, and will not attempt to assign it any more shares.
+        """
+
+    def get_share_assignments(shallow=False):
+        """
+        Given what I know about the state of the grid, I'll attempt to
+        assign shares to peers in a way that maximizes my definition of
+        file health. I'll return a list of (share, peerid) tuples with
+        my decision.
+
+        I have an optional shallow parameter. If True, I will not
+        attempt to assign any new shares before checking happiness;
+        instead, I will rely only on the shares that I know are already
+        placed.
+        """
+
+    def is_healthy():
+        """
+        I return whether the share assignments I'm currently using
+        reflect a healthy file, based on my internal definitions.
+        """
+
+    def needs_recomputation():
+        """
+        I return True if the share assignments I last returned may have
+        become stale. This is a hint to the caller that they should call
+        get_share_assignments again.
+        """
+
 class IImmutableFileNode(IFileNode):
     def read(consumer, offset=0, size=None):
         """Download a portion (possibly all) of the file's contents, making

Context:

[NEWS: 'top' for node processes, WUI formatting, removal of GUI apps, documentation updates, foolscap dependency. refs #174, #1219, #1225
david-sarah@jacaranda.org**20110106005727
 Ignore-this: f61ac58b4d10e635feb6f7391b1b48fe
] 
[Makefile: update 'clean' target for files in bin/
david-sarah@jacaranda.org**20110103052738
 Ignore-this: 2bdbc4a50e13e508b66d0f65718c79b2
] 
[docs: update performance.rst to describe the difference between already-uploaded and not-already-uploaded, to parameterize segment size, and to use "~A" to mean "approximately A"
zooko@zooko.com**20110104065455
 Ignore-this: 8df0d79a062ee19854c0211bd202f606
] 
[bin/tahoe-script.template: On non-Windows, invoke support/bin/tahoe directly as a script (rather than via python), so that 'top' for example will show it as 'tahoe'. On Windows, simplify some code that set argv[0], which is never used. fixes #174
david-sarah@jacaranda.org**20101127232650
 Ignore-this: 42a86f3eecfdc1ea7b76a7cc68626898
] 
[test_runner: avoid unnecessary use of non-ASCII.
david-sarah@jacaranda.org**20110101100101
 Ignore-this: e2ff40dce6bb3b021306f2913d4e75df
] 
[docs/quickstart.html: fix redundant, badly nested tag. refs #1284
david-sarah@jacaranda.org**20110102175159
 Ignore-this: 2ae9cc0b47d2e87b9eb64a0f517c4eef
] 
[docs/quickstart.html: information about 'troublesome dependencies' and 'verified systems' de-emphasized by smaller italic font. Re-wrap so that the HTML source is readable (just about) as text. Minor wording tweaks. Improve organization by adding 'Windows Caveats' subsection. fixes #1284
david-sarah@jacaranda.org**20110102174212
 Ignore-this: e9dc57983974478200856651c5318fee
] 
[NEWS: update entry for removal of Mac and Windows apps. refs #1282
david-sarah@jacaranda.org**20101226042245
 Ignore-this: c8099bc6e8235718d042c9a13c1e2425
] 
[Move dependency imports from windows/depends.py (which has gone away) into src/allmydata/windows/tahoesvc.py. Also fix a pyflakes warning, and change the service display name from 'Allmydata Tahoe Node' to 'Tahoe-LAFS node'. refs #1282
david-sarah@jacaranda.org**20101226042100
 Ignore-this: ee45f324934e1251380206dbee6346d0
] 
[Remove unmaintained Windows GUI app, except for windows/tahoesvc.py which is moved to src/allmydata/windows. refs #1282
david-sarah@jacaranda.org**20101226040237
 Ignore-this: cae37b6622a7dd5940acc7d3e6a98b90
] 
[Remove the Makefile targets relating to the Mac GUI app. refs #1282
david-sarah@jacaranda.org**20101226025859
 Ignore-this: 75303be783974b41138744ec62b07965
] 
[NEWS: remove unmaintained Mac GUI app. refs #1282
david-sarah@jacaranda.org**20101226020858
 Ignore-this: 40474a07f4a550b48563d35350be7ab5
] 
[Remove unmaintained Mac GUI app. fixes #1282
david-sarah@jacaranda.org**20101226020508
 Ignore-this: b3613bf1abfd284d542bf7c753ec557a
] 
[Remove src/allmydata/util/find_exe.py which is no longer used. fixes #1150
david-sarah@jacaranda.org**20101226023206
 Ignore-this: 7436c9b53bf210aed34a1a973cd9cace
] 
[status_web_pages_review.darcs.patch 
freestorm77@gmail.com**20110102034214
 Ignore-this: 29f1ecb36177f10f3f846b3d56b313b2
 
 I make some changes on status web pages
 
 status.xhtml:
 - Delete unused webform_css link
 - Align tables on the left
 
 tahoe-css:
 - Do some minor changes on code synthax
 - changes table.status-download-events style to look like other tables
 
 status.py:
 - Align table on the left
 - Changes table header
 - Add heading tags
 - Modify google api graph: add image border, calculate height to feet data
 
 signed-off-by: zooko@zooko.com
 fixes #1219
] 
[test_storage.py: fix a pyflakes unused import warning.
david-sarah@jacaranda.org**20101231220756
 Ignore-this: df08231540cb7dff9d2b038e47ab30ee
] 
[test_storage.py: leave at least 512 MiB free when running test_large_share. refs #1195
david-sarah@jacaranda.org**20101231203215
 Ignore-this: b2144c0341c3452b5d4ba219e284ea0e
] 
[storage: use fileutil's version of get_disk_stats() and get_available_space(), use mockery/fakery in tests, enable large share test on platforms with sparse files and if > 4 GiB of disk space is currently available
zooko@zooko.com**20100910173629
 Ignore-this: 1304f1164c661de6d5304f993eb9b27b
] 
[fileutil: copy in the get_disk_stats() and get_available_space() functions from storage/server.py
zooko@zooko.com**20100910173520
 Ignore-this: 8b15569715f710f4fc5092f7ca109253
] 
[Update foolscap version requirement to 0.6.0, to address http://foolscap.lothar.com/trac/ticket/167
david-sarah@jacaranda.org**20101231060039
 Ignore-this: 98d2b8086a1a500b9f4565bca5a3810
] 
[docs/webapi.rst: typos.
david-sarah@jacaranda.org**20101230034422
 Ignore-this: d1f5166d72cc711f7e0d9981eac9105e
] 
[docs/webapi.rst: capitalization, formatting of section on URL character encoding, and a correction about Internet Explorer.
david-sarah@jacaranda.org**20101230034049
 Ignore-this: b3b9819d2fb264b4cdc5c8afd4e8c48d
] 
[docs: corrections and clarifications.
david-sarah@jacaranda.org**20101227051056
 Ignore-this: e33202858c7644c58f3f924b164294b6
] 
[docs: more formatting cleanups and corrections. Spell webapi and wapi as web-API.
david-sarah@jacaranda.org**20101227050533
 Ignore-this: 18b23cbfb780df585d8a722a1ec63e94
] 
[docs/debian.rst: bring description of building dependencies from source up-to-date, and change hostname from allmydata.com to tahoe-lafs.org.
david-sarah@jacaranda.org**20101212222912
 Ignore-this: f38462afc88b4475195610385a28391c
] 
[docs/architecture.rst: correct rst syntax.
david-sarah@jacaranda.org**20101212202003
 Ignore-this: 3fbe12feb28bec6f1c63aedbc79aad21
] 
[docs/architecture.rst: formatting.
david-sarah@jacaranda.org**20101212201719
 Ignore-this: 305fa5dfc2939355eaf6d0d2161eb1ff
] 
[docs: linkification, wording improvements.
david-sarah@jacaranda.org**20101212201234
 Ignore-this: 4e67287f527a8bc728cfbd93255d2aae
] 
[docs: formatting.
david-sarah@jacaranda.org**20101212201115
 Ignore-this: 2e0ed394ac7726651d3a4f2c4b0d3798
] 
[docs/configuration.rst: more formatting tweaks; which -> that.
david-sarah@jacaranda.org**20101212195522
 Ignore-this: a7becb7021854ca5a90edd892b36fdd7
] 
[docs/configuration.rst: more changes to formatting.
david-sarah@jacaranda.org**20101212194511
 Ignore-this: 491aac33e5f5268d224359f1447d10be
] 
[docs/configuration.rst: changes to formatting (mainly putting commands and filenames in monospace).
david-sarah@jacaranda.org**20101212181828
 Ignore-this: 8a1480e2d5f43bee678476424615b50f
] 
[scripts/backupdb.py: more accurate comment about path field.
david-sarah@jacaranda.org**20101212170320
 Ignore-this: 50e47a2228a85207bbcd188a78a0d4e6
] 
[scripts/cli.py: fix missing 'put' in usage example for 'tahoe put'.
david-sarah@jacaranda.org**20101212170207
 Ignore-this: 2cbadf066fff611fc03d3c0ff97ce6ec
] 
[docs/frontends/CLI.rst: changes to formatting (mainly putting commands and filenames in monospace), and to command syntax to reflect that DIRCAP/... is accepted. Clarify the syntax of 'tahoe put' and other minor corrections. Tahoe -> Tahoe-LAFS.
david-sarah@jacaranda.org**20101212165800
 Ignore-this: a123ef6b564aa8624d1e79c97068ea12
] 
[docs/frontends/CLI.rst: Unicode arguments to 'tahoe' work on Windows as of v1.7.1.
david-sarah@jacaranda.org**20101212063740
 Ignore-this: 3977a99dfa86ac33a44171deaf43aaab
] 
[docs/known_issues.rst: fix title and linkify another URL. refs #1225
david-sarah@jacaranda.org**20101212062817
 Ignore-this: cc91287f7fb51c23440b3d2fe79c449c
] 
[docs/known_issues.rst: fix an external link. refs #1225
david-sarah@jacaranda.org**20101212062435
 Ignore-this: b8cbf12f353131756c358965c48060ec
] 
[Fix a link from uri.rst to dirnodes.rst. refs #1225
david-sarah@jacaranda.org**20101212054502
 Ignore-this: af6205299f5c9a33229cab259c00f9d5
] 
[Fix a link from webapi.rst to FTP-and-SFTP.rst. refs #1225
david-sarah@jacaranda.org**20101212053435
 Ignore-this: 2b9f88678c3447ea860d6b61e8799858
] 
[More specific hyperlink to architecture.rst from helper.rst. refs #1225
david-sarah@jacaranda.org**20101212052607
 Ignore-this: 50424c768fca481252fabf58424852dc
] 
[Update hyperlinks between docs, and linkify some external references. refs #1225
david-sarah@jacaranda.org**20101212051459
 Ignore-this: cd43a4c3d3de1f832abfa88d5fc4ace1
] 
[docs/specifications/dirnodes.rst: fix references to mutable.rst. refs #1225
david-sarah@jacaranda.org**20101212012720
 Ignore-this: 6819b4b4e06e947ee48b365e840db37d
] 
[docs/specifications/mutable.rst: correct the magic string for v1 mutable containers. refs #1225
david-sarah@jacaranda.org**20101212011400
 Ignore-this: 99a5fcdd40cef83dbb08f323f6cdaaca
] 
[Move .txt files in docs/frontends and docs/specifications to .rst. refs #1225
david-sarah@jacaranda.org**20101212010251
 Ignore-this: 8796d35d928370f7dc6ad2dafdc1c0fe
] 
[Convert docs/frontends and docs/specifications to reStructuredText format (not including file moves).
david-sarah@jacaranda.org**20101212004632
 Ignore-this: e3ceb2d832d73875abe48624ddbb5622
] 
[scripts/cli.py: remove the disclaimer in the help for 'tahoe cp' that it does not handle non-ASCII filenames well. (At least, we intend to handle them.)
david-sarah@jacaranda.org**20101130002145
 Ignore-this: 94c003efaa20b9eb4a83503d79844ca
] 
[relnotes.txt: fifth -> sixth labor-of-love release
zooko@zooko.com**20101129045647
 Ignore-this: 21c245015268b38916e3a138d256c09d
] 
[Makefile: BB_BRANCH is set to the empty string for trunk, not the string 'trunk'.
david-sarah@jacaranda.org**20101128233512
 Ignore-this: 5a7ef8eb10475636d21b91e25b56c369
] 
[relnotes.txt: eleventh -> twelfth release.
david-sarah@jacaranda.org**20101128223321
 Ignore-this: 1e26410156a665271c1170803dea2c0d
] 
[relnotes.tst: point to known_issues.rst, not known_issues.txt.
david-sarah@jacaranda.org**20101128222918
 Ignore-this: 60194eb4544cac446fe4f60b3e34b887
] 
[quickstart.html: fix link to point to allmydata-tahoe-1.8.1.zip.
david-sarah@jacaranda.org**20101128221728
 Ignore-this: 7b3ee86f8256aa12f5d862f689f3ee29
] 
[TAG allmydata-tahoe-1.8.1
david-sarah@jacaranda.org**20101128212336
 Ignore-this: 9c18bdeaef4822f590d2a0d879e00621
] 
Patch bundle hash:
9a0a487d93c7ef13e5e40e21603f9feeb68466ca