[tahoe-dev] Opportunistic node connections

[Jeremy Fitzhardinge]

Drew Perttula wrote:
> Bringing in stuff like avahi sounds like a really heavyweight solution 
> that might not even pay off. In more exotic networks, VPNs might make 
> faraway machines look local, or nearby machines might be on opposite 
> sides of a router that doesn't conduct avahi. (I think I have both of 
> those effects at my house :)
>
> Why would you want anything other than "prefer to replicate to hosts 
> that are currently fast to talk to"? If we're looking for 10 hosts on 
> the list of 50, instead of picking at random, we might pick the ones 
> with low ping times. Of course that will have side effects regarding how 
> fast the nodes fill their disks, etc.
>   

Well, it comes from the old adage: "Never underestimate the bandwidth of 
a station-wagon full of tapes".  In this case, the observation is:

   1. people carry around a mass of data with them (often everything),
      either in a laptop, portable hard-disk, ipod, etc
   2. local network connections, wired or wireless, are getting faster
      faster than wan connections

If you want to back up several 10s or 100s of gigs of your photo 
collection at a friends place, the most efficient way to do it is carry 
it over there and copy it locally.

Extended more generally, in your day to day movements, you tend to 
encounter the same people fairly regularly.  They may be people in your 
office, at the cafe, friends, classmates, etc.  What if when you're near 
them, your respective devices quietly and automatically start 
replicating your newly generated data onto their storage, and vice 
versa?  Or, correspondingly, if you lose something, you retrace your 
steps - as it were - to recover your data.

And while you expect everyone will connect to the net to get in touch 
with the global grid to sync up the global knowledge of who has what, 
but you want to work with whoever's available even if you're disconnected.

Or more prosaically, if you're on a NATted network with other people who 
are part of the same grid, you can get in touch with local network 
addresses, even if you don't have globally visible network addresses 
(but can at least make outbound connections to other nodes and the 
introducer).


I went through trac today and came up with a list of open tickets which 
I think are relevant to this use case:

Easy adoption:
http://allmydata.org/trac/tahoe/ticket/126 - add "invite a friend" button

Deal with crummy NATted networks:
http://allmydata.org/trac/tahoe/ticket/49 - UPnP
http://allmydata.org/trac/tahoe/ticket/50 - STUNT/ICE
http://allmydata.org/trac/tahoe/ticket/169 - tcp hole punching

Opportunistic data sharing/distribution:
http://allmydata.org/trac/tahoe/ticket/213 - good handling of small 
numbers of servers, or strange choice of servers
http://allmydata.org/trac/tahoe/ticket/295 - distributed introduction 
and public-key-based server blessing
http://allmydata.org/trac/tahoe/ticket/403 - grid identifier
http://allmydata.org/trac/tahoe/ticket/444 - reduce number of active 
connections: connect-on-demand
http://allmydata.org/trac/tahoe/ticket/314 - DNS tricks to connect 
clients to their own (local) node

    J