[tahoe-dev] [tahoe-lafs] #467: change peer-selection to allow introducerless explicit serverlist, alternative backends (was: change peer-selection to prepare for rights-amplification step, alternative backends)

[tahoe-lafs]

#467: change peer-selection to allow introducerless explicit serverlist,
alternative backends
--------------------------------+-------------------------------------------
 Reporter:  warner              |           Owner:           
     Type:  enhancement         |          Status:  new      
 Priority:  major               |       Milestone:  undecided
Component:  code-peerselection  |         Version:  1.1.0    
 Keywords:                      |   Launchpad_bug:           
--------------------------------+-------------------------------------------

Old description:

> Another prep-task for the upcoming Accounting work is to change the
> client's
> peer-selection code. The goals are:
>
>  * the set of storage servers that are considered for peer-selection
> should
>    come from a config file, with one server per line, of various types
>  * one entry type means "ask the introducer", perhaps with some
> parameters
>    like "only accept servers that are blessed by the following key"
>  * if this file doesn't exist, it defaults to asking the introducer
>  * one entry type (or introducer tag) is used to point at a storage
> server's
>    "login FURL" instead of a fully-powered reference. (we need a better
>    name for this object). This is the object to which the client will
> pass
>    credentials and/or signed certificate chains.
>
> When the peer-selection process needs server references, the client
> should
> ask the login FURLs for server references, performing credential
> negotiation
> if necessary. The client should cache the results for later use, so
> ideally
> the negotiation only needs to take place once.

New description:

 (note: this is no longer closely related to Accounting, since my new
 plan for Accounting is to send signed requests over a Foolscap channel,
 instead of using rights-amplification to get from an initial Foolscap
 object to a per-account object).

 I'd like to have a section in the client's tahoe.cfg which lets it
 specify the servers available for storage. In contrast to #573 (which is
 about runtime/per-upload specification of which servers to use, out of
 the set provided by the introducer), this ticket is about boot-time
 configuration of the available set, potentially replacing the
 Introducer-provided list.

 My thought is that the tahoe.cfg should have a section that specifies a
 list of servers to use. Then another tahoe.cfg setting should have a
 flag which says "use the Introducer to populate this list", and the
 default configuration would use the Introducer. This latter section
 would also have a place to configure the #466-style "blesser" (a pubkey
 which tells the client to only accept server announcements which have
 been signed by the matching privkey).

 This would also make it possible to configure alternative server types.
 The first such server type I'd like to add is an S3-based server.
 Regular servers would be defined with a FURL; S3 servers would be
 defined with a service URL and a set of authorization secrets.

 The syntax I'm thinking of would look like this:

 {{{
 [client-server-selection]
 server.X.type = tahoe-foolscap
 server.X.nickname = alice
 server.X.furl = pb://...
 server.Y.type = tahoe-foolscap
 server.Y.nickname = bob
 server.Y.furl = pb://
 server.Z.type = s3
 server.Z.nickname = aws
 server.Z.url = http://...
 server.Z.creds = ...
 server.Z.num_shares = 3
 use_introducer = False
 permute_serverids = False
 }}}

 The {{{server.*}}} lines would basically define a list of dictionaries
 (the "X" and "Y" strings would be discarded after tahoe.cfg is parsed).

 The "use_introducer=False" line means that the client shouldn't bother
 talking to the Introducer. If it were True, the client would connect to
 the introducer and add whatever servers it knew about to the list.

 The "permute_serverids=False" line means that the client shouldn't
 permute the serverlist on each upload. Instead, it should assign 1 (or
 num_shares=) shares to each server in the order they appear in this
 list. The total-shares "N" value ought to equal the number of servers
 (or rather the sum of the num_shares= values).

 Having permute_serverids=False in the tahoe.cfg, rather than provided on
 a per-upload basis (as in #573) might prove more usable. It might be
 more appropriate for a fairly stable grid though: one in which new
 servers are not being added very frequently.

--

Comment(by warner):

 repurposes this ticket to talk about configuring the server list in
 tahoe.cfg, potentially instead of using the Introducer

-- 
Ticket URL: <http://allmydata.org/trac/tahoe/ticket/467#comment:1>
tahoe-lafs <http://allmydata.org>
secure decentralized file storage grid