[tahoe-dev] [tahoe-lafs] #1109: let the get_buckets() response include the first block

[tahoe-lafs]

#1109: let the get_buckets() response include the first block
------------------------------+---------------------------------------------
     Reporter:  zooko         |       Owner:  warner              
         Type:  enhancement   |      Status:  new                 
     Priority:  major         |   Milestone:  1.8.0               
    Component:  code-network  |     Version:  1.7.0               
   Resolution:                |    Keywords:  download performance
Launchpad Bug:                |  
------------------------------+---------------------------------------------

Comment (by warner):

 Yeah, in general, I think a more stateless immutable share-read interface
 would be better. The mutable share interface (which was written about 6
 months later) is stateless, and that makes life a bit easier. That
 interface takes a read vector and a set of share numbers, with an empty
 set meaning "all shares that you are holding", and I think the same
 interface would work for immutable files.

 Such an interface assumes that the server can efficiently open+seek+read
 the same file several times in quick succession (closing the filehandle
 between each call), whereas the current stateful interface keeps the
 filehandle open for the entire download. I suspect that most modern
 OS/filesystems cache recently-opened files and make this fairly quick, but
 it's worth doing some benchmarks to be certain. Also, we'd want to
 consider the interaction with GC and/or external tools which delete
 shares: keeping the filehandle open means a download will survive the
 share being deleted in the middle, whereas a stateless interface would
 not.

 A stateless interface would also make us slightly more resistant to a DoS
 attack in which the attacker opens lots of shares at once and tries to
 fill the file-descriptor table.

 I'd want to leave the signature of get_buckets() alone, and add a new
 method instead. The server-version-information dictionary could be used to
 advertise the availability of such a method.

 And overall, yeah, I'd like to optimize out that extra round trip, because
 my new downloader (#798) can currently retrieve a small file in just two
 roundtrips, and with this fix we could get that down to just a single
 roundtrip, which would be great.

 I was probably a bit over-enthusiastic about using Foolscap remote
 references when I wrote the immutable interface... incidentally, one
 conceivable benefit of the stateful interface could come up in server-
 driven share migration. That code (living in server A) could talk to
 server B and send it a "please copy my share" message, passing it the
 remote-reference to the share's {{{BucketReader}}}, or a client could use
 Foolscap's third-party-reference ("Gifts") feature to let A and B move the
 data directly between themselves without requiring client-side bandwidth
 for the copy. Of course, since all shares are publically readable, there's
 no authority-reducing benefit to doing it with bucket references over
 simply telling someone the storage-index and having them do the reads
 themselves. But when Accounting shows up, it might wind up to be handy to
 have a bucket-plus-read-authority object available to pass around.

-- 
Ticket URL: <http://tahoe-lafs.org/trac/tahoe-lafs/ticket/1109#comment:1>
tahoe-lafs <http://tahoe-lafs.org>
secure decentralized storage