let the get_buckets() response include the first block

[zooko]

We could optimize out a round trip in download if, when asking whether a storage server has blocks, by sending it a get_buckets(), you could optionally specify that if it does have one of those then could it please send you one of them. Then the reply would be a set of buckets (remote references to an object on the server) plus the full data of one block.

Brian: what do you think?

[warner]

Yeah, in general, I think a more stateless immutable share-read interface would be better. The mutable share interface (which was written about 6 months later) is stateless, and that makes life a bit easier. That interface takes a read vector and a set of share numbers, with an empty set meaning "all shares that you are holding", and I think the same interface would work for immutable files.

Such an interface assumes that the server can efficiently open+seek+read the same file several times in quick succession (closing the filehandle between each call), whereas the current stateful interface keeps the filehandle open for the entire download. I suspect that most modern OS/filesystems cache recently-opened files and make this fairly quick, but it's worth doing some benchmarks to be certain. Also, we'd want to consider the interaction with GC and/or external tools which delete shares: keeping the filehandle open means a download will survive the share being deleted in the middle, whereas a stateless interface would not.

A stateless interface would also make us slightly more resistant to a DoS attack in which the attacker opens lots of shares at once and tries to fill the file-descriptor table.

I'd want to leave the signature of get_buckets() alone, and add a new method instead. The server-version-information dictionary could be used to advertise the availability of such a method.

And overall, yeah, I'd like to optimize out that extra round trip, because my new downloader (#798) can currently retrieve a small file in just two roundtrips, and with this fix we could get that down to just a single roundtrip, which would be great.

I was probably a bit over-enthusiastic about using Foolscap remote references when I wrote the immutable interface... incidentally, one conceivable benefit of the stateful interface could come up in server-driven share migration. That code (living in server A) could talk to server B and send it a "please copy my share" message, passing it the remote-reference to the share's BucketReader, or a client could use Foolscap's third-party-reference ("Gifts") feature to let A and B move the data directly between themselves without requiring client-side bandwidth for the copy. Of course, since all shares are publically readable, there's no authority-reducing benefit to doing it with bucket references over simply telling someone the storage-index and having them do the reads themselves. But when Accounting shows up, it might wind up to be handy to have a bucket-plus-read-authority object available to pass around.

[zooko]

So perhaps for v1.8.0 we would add a new method called something like get_buckets_data() which implements the stateless interface, like slot_readv(). Then new downloaders could use that, perhaps invoking that on the first K servers that they know of which support it, and invoking get_buckets() on the remaining N-K servers which they will need to use only if some of the first K servers fail.