[tahoe-dev] Automatic rebalancing

[Brian Warner]

On 12/5/10 3:52 PM, James A. Donald wrote:
> 
> A centralized coordinator is single point of failure and an additional
> configuration issue. If everyone runs the same algorithm, they will
> mostly agree without need for a central coordinator - though there
> will never be 100% agreement. If the system tolerates partial
> agreement without hard failure, one can usually do away with central
> coordinators.

Yeah. As I think I've written before, the Tahoe share-placement
algorithm needs to be a compromise between several different goals:

 * diversity (of location, of administration, of failure modes), to
   improve reliability
 * proximity (to the uploader, to the downloader), to improve upload
   speed, or download speed by various downloaders
 * discoverability (offline computability of likely share locations), to
   improve download speed by minimizing do-you-have-share queries
 * load-balancing (avoiding storage "hot spots", spreading bandwidth or
   storage requirements uniformly across servers, or better)
 * compression of location data (the filecap is the only per-file
   information transferred from uploader to downloader: putting a
   complete share-location list in each filecap would make them very
   large)

The current immutable downloader is tolerant of wrong information or
misplacement because it will fall back to asking every single server,
but that's problematic for large grids, especially if there's a chance
that your file doesn't really exist. The mutable retrieval code is less
forgiving, as it gives up eventually.

I like our tolerant downloader, and think we can avoid a central
coordinator, but we may need some new mechanisms to accomodate more of
those goals at the same time. For example, we could keep storage servers
more-or-less informed of the location of other shares for the files they
are handling: as long as a client can find any one shares, the resulting
server can help them file several other shares.

I think tickets #362, #447, #699, and others in the "code-peerselection"
category are relevant.

cheers,
 -Brian