[tahoe-dev] distributed-introducer: "grid-control" announcement

[Brian Warner]

My Accounting email mentioned needing to broadcast Recommendation
Records out to all nodes. Here's an idea about how to get that and also
start the move away from a centralized Introducer SPOF (#68):

* define a new type of announceable service called "grid-control". (we
  currently have only one type, called "storage", which announces the
  storage-server FURL and some version information)

* "grid-control" messages (which, like all #466 new-introducer messages,
  are signed by the nodekey) will initially contain just a list of
  Recommendation Records:

   { "service-name": "grid-control",
     "sequence-number": 123,
     "recommendations": [ {"who": "BOB-PUBKEY",
                           "petname": "Bob",
                           "what": "send-shares-to"},
                          {"who": "BOB-PUBKEY",
                           "petname": "Bob",
                           "what": "accept-shares-from"} ]
   }

* all clients and all server subscribe to "grid-control". The
  announcements are fed into the graph-reachability code described in my
  earlier message

That'll get us the broadcast channel needed for the Accounting project.
The (central) Introducer is responsible for distributing announcements
to everybody, as well as remembering earlier announcements so new nodes
can learn about existing servers.

To move to a #68 gossip-style system, I'm thinking this:

* add a "grid-control-FURL" key to the grid-control announcements
* this FURL will point to a Referenceable that provides the
  RIIntroducerPublisherAndSubscriberService_v2 service (just like the
  centralized Introducer does)
* any grid-control-FURLs that we receive will be handed to the
  IntroducerClient, which will connect to that FURL and treat it just
  like the normal Introducer
* the introducer code will need to be enhanced to avoid broadcast
  storms, using some of the selective-flooding ideas that Zooko proposed
  years back (gleaned from his experience with Mnet)
* all grid-control messages should be recorded in persistent storage, so
  they'll be available at reboot
* the central Introducer publishes a grid-control record that has only
  the grid-control-FURL key, nothing else
* clients seeded with an "introducer.furl" create a synthetic
  grid-control record and use it to bootstrap everything else
* clients can be seeded with a set of other grid-control-FURLs, from
  other nodes. When inviting someone to join your grid, the Invitation
  will include all of the grid-control-FURLs that you currently know
  about. If at least one of those is up, you should be able to bootstrap
  up to the whole grid.
* if you're worried that someone might be invited to join your grid and
  discover that nobody is online at that moment, you can run a dedicated
  Introducer. But otherwise, gossip should be enough.


## Open Questions

* should grid-control-FURL share announcements with recommendations? One
  benefit of doing it this way is that nodes would only be expected to
  forward messages about nodes that they themselves are willing to use,
  so bogus data (attackers injecting random+unreachable records) won't
  get propagated.
* is there a better name than "grid-control?
* as mentioned in the earlier email, putting all recommendations in a
  single Announcement means the limited-flood algorithm must work with
  coarser-granularity data: if only one of the recommendations has
  changed, it must still re-publish the whole Announcement. Is this ok?
  Could be do better without increasing complexity?
* I think we need to add a sequence-number to current Announcements, and
  only have clients replace earlier messages when the seqnum has
  increased.


thoughts?
 -Brian