[tahoe-lafs-trac-stream] [tahoe-lafs] #1382: immutable peer selection refactoring and enhancements

[tahoe-lafs]

#1382: immutable peer selection refactoring and enhancements
------------------------------------+---------------------------------------
     Reporter:  kevan               |       Owner:  kevan    
         Type:  enhancement         |      Status:  new      
     Priority:  major               |   Milestone:  undecided
    Component:  code-peerselection  |     Version:  1.8.2    
   Resolution:                      |    Keywords:           
Launchpad Bug:                      |  
------------------------------------+---------------------------------------

Comment (by kevan):

 First up is a draft of of {{{IPeerSelector}}}, an abstract peer selection
 class. The draft is contained in attachment:interface.dpatch.

 A significant annoyance with servers of happiness (and a source of many of
 its shortcomings) is the fact that it wasn't used to motivate share
 placement or server selection in a well thought out way; rather, it was
 just used to evaluate the job that the existing share placement strategy
 did. So, with {{{IPeerSelector}}} (and with the implicit assumption that
 the peer selection and share placement strategies you'd want to use when
 uploading a file are probably generally motivated by how you define file
 health), I wanted to combine both file health and share placement in one
 abstraction. I still have a few small issues to fix with the uploader
 changes that actually use the interface, so you'll have to wait to see the
 code that exercises and implements the interface, but the interaction is
 basically:

   * Initialize upload, upload initializes peer selector.
   * Upload tries to find existing shares as appropriate. For each existing
 share:
     * Call {{{add_peer_with_share}}} to tell the peer selector about that
 relationship.
   * Upload tells the peer selector all of the peers that it can write to
 (the peer selector doesn't assume that it can write to the peers it is
 told about for existing shares).
   * do:
     * Upload asks the peer selector for share assignments, checks their
 health. If unhealthy, the upload fails with an error as appropriate (so we
 can account for file repair edge cases here).
     * Upload tries to allocate buckets for the assigned share placements,
 reporting to the peer selector its successes (via
 {{{add_peer_with_share}}}) and failures (via {{{mark_full_peer}}} and
 {{{mark_bad_peer}}}) as appropriate.
    * while the peer selector needs recomputation.

 I have an immutable uploader that works pretty well with this sort of
 interaction.

 Things to look for if you're reviewing:

   1. Leaky abstractions; particularly leaky method names. I tried to avoid
 leaking graph theory into my interface definition. Did I succeed? If not,
 what should be improved?
   1. Does this seem like a generally useful interface? Could you imagine
 plugging it into an existing uploader without too much trouble? Could you
 imagine writing a new uploader that used it without too much trouble? How
 about a peer selector for an entirely different upload health metric?

 (of course, you're welcome to look for whatever you want -- that's just
 what I think I need feedback on :-)

-- 
Ticket URL: <http://tahoe-lafs.org/trac/tahoe-lafs/ticket/1382#comment:1>
tahoe-lafs <http://tahoe-lafs.org>
secure decentralized storage