[Tahoe-dev] testnet flushed again

[Brian Warner]

I just upgraded our testnet to 0.4.0-145, which shortens the URIs (by
deriving the Storage Index from the encryption key). I had to flush the data
again, since the URI format changed. We're anticipating another such event
within a few days (as we move to zbase32 instead of RFC3548-base32),
shrinking the URIs further (and removing those annoying ===s from the end).
Hopefully we'll have an 0.5.0 release out by the end of the week, and then at
least the CHK file URIs have a chance of remaining stable for a little while.

Incidentally, my current work is on separating the upload process into
distinct read/encrypt/encode/push components, to allow these parts to live on
separate machines. The goal is to enable two specific features. The first is
to upload one segment at a time, say, when someone is pushing a 1GB file to
your node via HTTP PUT, and you don't want to spend a gig of disk space (and
you don't want to make multiple passes over the data, and you don't care
about convergence). The node will accept a 1MB chunk, encrypt/encode/push
that, then accept the next 1MB, etc. Our memory footprint is already designed
to be pretty low, but this will reduce the disk footprint too.

The second feature is called "offloaded upload", and basically it means that
one node does the encryption and a second node does the encoding/pushing.
This would let a e.g. a home user (on the slow end of an ADSL line) spend
just 1x the bandwidth to get the file to a service with better connectivity,
and then have that remote service do the hard work of encoding and pushing
the 3x or 4x shares, while preventing that remote service from seeing the
file's plaintext.

The idea is that nodes can always upload files by themselves, but if you have
someone who is willing to do the extra work for you (perhaps because you're
paying them), you can offload your bandwidth costs to them. This also makes
uploading in tahoe as fast as uploading via regular FTP or HTTP.

This feature probably won't be in 0.5.0, but it's coming soon, probably
within a few weeks.. stay tuned.

cheers,
 -Brian