[tahoe-dev] Tahoe's webapi now handles "Range:" headers (mostly)

[Brian Warner]

I just pushed changes into tahoe trunk for ticket #527 "support Range
header". This makes the webapi server respect a "Range: bytes=0-100"
header in a GET /uri/FILECAP request, by returning only the first 101
bytes of the file. This turns out to be necessary to get, say, an
iPhone's quicktime library to be willing to play media files that are
coming out of a Tahoe grid.

The approach is a bit clunky: the file is downloaded to a cache
directory on the node's local disk, and GET requests are satisfied from
that cache. The ideal solution (which won't be in place for at least a
couple of weeks, possibly months) is to have the node only download the
segments that are necessary to satisfy the request, holding no more
than two segments in RAM at any one time, and not generally writing data
to disk.

But the cache-to-disk works well enough for now. The biggest drawback
is that it puts plaintext on disk, meaning that a webapi node
compromise can now reveal data that was retrieved before the compromise
occurred (i.e. this loses a measure of forward secrecy). I hope to fix
this to store ciphertext on the disk instead, but I'm not sure I'll get
it done before Zooko finishes the Repairer and we release 1.3.0 . We
also need some logic to clean out that disk cache after a while, but
that's an easier job, and I expect to get that into 1.3.0.

My list of things that I'm hoping we'll work on in the next few
weeks/months is:

 * solidify the FTP server (serious testing/bugfixing, unicode work)
 * SFTP server
 * Accounting
 * improve Helper usage: load-balancing, maybe announce Helpers
   through the Introducer
 * overhaul Download to tolerate lost peers, pipeline, support real seek
 * overhaul Upload to run faster with pipelining


cheers,
 -Brian