No subject

 * there will be a file, perhaps named $BASEDIR/private/servers.txt,
   which contains a list of storage servers that will be used for upload
   and download. Each server gets a single entry on this list, and the
   entries contain at least a nodeid, and maybe some extra data. The
   file is consulted on each upload/download attempt.
 * servers.txt may be modified or created by a "serverlist manager"
   component. Some entry in tahoe.cfg indicates which manager to use and
   how it should be configured.
 * The default configuration uses a "traditional Introducer" manager,
   which reads introducer.furl from the config and behaves just like our
   current system.

Given that, the question is what sorts of serverlist-managers we should
provide, and how they should work (and be configured).

== Initial Use Cases ==

The basic use case is a closed friendnet: a small number of people, each
running a client+server, who want to use each other's machines for
backup.

The other basic use case is the AllMyData commercial grid: a large
number of client machines, each using the same pool of storage servers.
All of the servers are managed by the same party.

== Actions ==

In general, new servers will be added to the list from time to time.
Members will join and leave the friendnet. The commercial grid will
acquire new servers as the existing ones get filled.

Servers may also be removed: when a friendnet participant stops behaving
in a friendly manner, clients may want to remove that server from their
lists. Or a server in a commercial grid might be compromised, and should
be removed from the list.

== Approaches ==

1 "static": The simplest approach is to let client admins manage their
            serverlist manually, and not enable a serverlist manager.
            Another form of this is to have client admins specify an
            explicit list of serverids, and use the Introducer to merely
            learn their contact information (ipaddr/port).

2 "update-furl": in this form, clients are given a FURL that points to a
                 server. The server provides a list of approved nodeids.
                 As in the "static" case, the Introducer is used to
                 distribute contact information, but not to authorize
                 servers. The client would fetch the nodeid list before
                 each upload/download operation. Variations involve
                 caching the list for a certain time, to reduce network
                 traffic at the expense of immediacy.

3 "one-key": The grid is defined by a single signing/verifying keypair.
             All authorized servers are given the signing privkey, and
             all clients are configured with the verifying pubkey. Each
             server signs its Announcement before distributing it
             through the Introducer. All servers get the same key.
             Variations include giving the client a list of pubkeys,
             and/or giving each server a list of privkeys.

4 "certchain": Each client gets a pubkey, as before, to which they
               delegate their server-selection. However Announcements
               are a chain of signed messages, each message delegating
               authority to the key which signs the next message. The
               first message is signed by the key that the client
               identified. The last message is signed by the storage
               server. Each storage server is given a "blessing prefix":
               a chain of one or more messages, in which the last
               message delegates some authority to the storage server's
               key. The server creates the Announcement by concatenating
               the prefix with their own signed message.

               To allow revocation, the blessing prefix can include an
               expiration time, which obligates the server to
               periodically renew its blessing, and allows a server to
               eventually be removed (by not renewing its blessing
               prefix).

               As with the client's serverlist, the server
               implementation would be split into two parts. The server
               would have a file (perhaps $BASEDIR/server-blessing.txt)
               which contains the blessing prefix that will be used for
               all introducer Announcements. Separately from that, the
               server would have a configuration setting that tells it
               how to update this file.

 4a: the server is manually configured with the blessing, and the admin
     is responsible for obtaining a new one before it expires.
 4b: the server is given a URL or a FURL to a service that will issue a
     renewed blessing on request, unless/until the service is instructed
     to stop issuing them (i.e. the server's authority is revoked). An
     http URL would be sufficient, because the blessing prefix is not
     secret, and the server can validate its integrity (assuming you
     tell the server which key to expect). A static http server, feeding
     files that are created by a daily cron job (emitting signed
     blessings for every key on a list), would be sufficient.

== Properties ==

The various approaches that I've thought about so far all vary on a
couple of different axes:

 * when a server is added, how much work is required? and on which
   nodes?
 * when a server is removed, how much work is required, and where? How
   long does it take for all clients to stop using that server? Is it
   even possible to remove a server?
 * efficiency/overhead: how frequently are control messages being sent?
 * where are the SPOFs? what happens when one goes down?
 * how complex is the system?

As usual, there are multi-way tradeoffs between these various
properties. To add or remove a server in #1-static, you must reconfigure
all clients (making it unsuitable for anything but very small, static
grids). However, once you're willing to do that, the changes are applied
immediately (you don't have to wait for anything to expire), and there's
no overhead (no messages are exchanged except when you change the
serverlist). #2-update-furl provides immediate add/revoke but requires a
message for every upload/download, and has a fail-stop SPOF (if that
server is unavailable, no uploads/downloads can happen).

#3-one-key is efficient, reasonably simple, and has no SPOFs, but it's
nearly impossible to remove a server (since every server knows the
single private key), and puts the grid into a forced-"egalitarian" mode
where every server can bring in new servers. This makes it suitable for
certain friendnets and possibly for the AllMyData commercial service (in
which all servers are run by the same company, and no clients provide
storage service), but the lack of revocation is a problem there too (a
single compromised server would allow attackers to disrupt reliable
service by introducing a lot of bogus servers).

#4-certchain is the most flexible, but is a bit more complex. It
provides a tradeoff between overhead and immediacy of revocation, and
requires us to define a way by which the server obtains a renewed
blessing prefix on a regular basis. The 4b approach (periodically fetch
a new blessing) would let the operator control this tradeoff: e.g. the
server renews their blessing every day, and the blessings are created
with an expiration time of two weeks. This gives a server-revocation
time (measured from the point at which you decide to revoke the server,
to the time at which the last client uses that server) of about two
weeks. It tolerates a 13-day downtime of the blessing service. You'd
probably want to have the server try more frequently as its blessing
gets more stale, to tolerate intermittently-available blessers too, and
admins might want to have the server email or page them if a renewal
attempt failed (so they'd have time to investigate and fix the blesser
problem).

There are a couple of different ways to approach revocation:
expiring/renewable certificates (with configurable renew-time and
expire-time values), OCSP-like online revocation-status checks, online
is-still-valid checks, etc. They vary on properties like: immediacy of
revocation, behavior in the face of network partition (fail-allow or
fail-deny), frequency of messages exchanged, etc. I think there is a
"three things, choose two" relationship between the properties. Any
scheme with an expiration time will provide a tradeoff between immediacy
and overhead, with the fully online systems at one end of the scale, and
the irrevocable systems at the other. Schemes that depend on an OCSP
check are fail-allow, while schemes with a renewal server are fail-deny.

== Conclusions ==

After talking with Zandr about what the ops team at an AMD-like service
would prefer, I think the development trajectory to follow is #1-static,
then #3-one-key, followed by #4b-certchain-with-renewal-URL. #1 requires
manual configuration, but is easy to understand, and will remain useful
for certain special cases (particularly personal grids, where all nodes
are run by the same person, like the way I run my own backupgrid). #3
lacks revocation, but makes it easy to add servers, and might be good
enough for some use cases. #4b fixes the lack of revocation, and would
be the most comfortable for an AMD-like service, as well as something
like our current volunteergrid (if we had a "master volunteer" who held
the private key and the membership authority).


what does everybody think?
 -Brian


[1]: http://github.com/warner/python-ecdsa  is a pure-python ECDSA
     library. It runs about 40x slower than Crypto++ would, though:
     256-bit-keys take 300ms/110ms/220ms to create/sign/verify. So I'd
     still really like http://allmydata.org/trac/pycryptopp/ticket/30
     to be fixed.

[2]: http://allmydata.org/trac/tahoe-lafs/ticket/798
     It mostly works, but I'm trying to come up with a good algorithm
     for deciding that a pending storage-server request is "overdue",
     which may require storing historical information about response
     times, and needs some heuristics that I haven't gotten comfortable
     with yet. It also wants some better visualization tools to guide
     the process.