[tahoe-dev] documenting tahoe's design in standardizable form

[Zooko O'Whielacronx]

Yuck!  I sent HTML mail to the list.  How embarassing.

Here is the plaintext version:

Here are some notes from my blog [1] about a conversation I had on the
phone with Brian on Friday:

"""
add the following to the list of Things To Do After 1.3.0 release:
write up the notes from the discussion with Brian about how to
document standardizable explanation of tahoe:

    * document 1: file formats: integrity, confidentiality,
erasure-coding; immutable and mutable files (this would basically be
my lafs.pdf [2] with enough detail to facilitate compatible
implementation)
    * document 2: storage server protocol: how to connect to storage
servers and upload/download files or parts thereof; foolscap protocol
(extant), HTTP protocol (future work)
    * document 3: peer selection ; This is the hard one. We can
document the "TahoeTwo" system, which works okay for the allmydata.com
use case [3], but there are all sorts of use cases that are ill-served
by the "TahoeTwo" design — more than a few hundred servers per grid,
non-clique grids, secure decentralized introduction, different needs
for reliability, availability, efficiency, etc. (some user wants Tahoe
to store K+1 shares of each file in each of their several separate
data centers, Shawn wants it to store exactly K shares of each file on
his mom's computer, if that file is tagged as being a family photo),
how to tell which grid or grids or set-of-servers a given cap can be
found in.
    * document 4: directories ; How to encode a set of caps, each
annotated with metadata such as a name, timestamp, etc., in a file.
Traversal caps?

This project is very exciting because it could facilitate other people
re-using tahoe, re-implementing it, or at least stealing its best
ideas. The walls of separation between these four documents are there
to limit the damage from design mistakes, and by the same token to
explicitly identify what each part requires of the other parts.
In particular I am eager to wall off the designs in document 3 from
the others, as it is a field littered with the corpses of attractive
designs.
"""

Regards,

Zooko

[1] http://testgrid.allmydata.org:3567/uri/URI:DIR2-RO:j74uhg25nwdpjpacl6rkat2yhm:kav7ijeft5h7r7rxdp5bgtlt3viv32yabqajkrdykozia5544jqa/wiki.html
[2] http://allmydata.org/~zooko/lafs.pdf
[3] http://allmydata.org/trac/tahoe/wiki/UseCases