[tahoe-dev] using the volunteer grid (dogfood tasting report, continued)

[Brian Warner]

Shawn Willden <shawn-tahoe at willden.org> writes:

> This means that it's tempting to set n close to m. But if you do that,
> just how bad *is* your reliability? I want to be able to provide users
> a simple way to make this tradeoff, presenting them with a list of
> reliability and performance estimates and letting them choose the one
> they feel is most appropriate.

Another thing to pay attention to is how exactly are you going to
communicate your choices to the eventual downloader. As discussed in
docs/specifications/outline.txt (section #3: Server Selection Algorithm,
filecap format), the uploader chooses servers to achieve some tradeoff
between availability, reliability, space efficiency, speed, and
load-balancing, but then you need to somehow tell the downloader where
they ought to look for the shares. They'll have some static
configuration, and can get additional information from the filecap
(although not very much, unless we're willing to make the filecaps
longer). And things may have changed since upload: servers may have come
and gone, and a repairer might have put shares in new places. The
current algorithm embodies a certain set of tradeoffs that seem to work
pretty well, but there are lots of reasons to desire something more
explicit.

I've been thinking recently about how to use the tahoe.cfg file to
express an explicit list of servers to use (#467). My main reason is to
enable alternative backends like an S3-based server, but it would also
be a convenient place to override the introducer-based permuted list,
and let users specify precisely which N servers they wanted their shares
to be placed upon.

It might be interesting to have the #467 server-specification syntax
also include a field for "availability", and then have part of the
t=check page for any given file include an estimate of availability and
reliability based upon those expectations. In some hypothetical future
world in which millions of Tahoe nodes collaboratively measure server
quality, this field could be automatically populated, and the Repairer
could use it to make decisions ("share#4 is on a bad server, better copy
it elsewhere").

cheers,
 -Brian