[tahoe-dev] [tahoe-lafs] #867: use ipv6

[Greg Troxel]

  Would you be able to elaborate about this?  Specifically about my use case
  of two hosts on tunnel brokers, but link-local.  I feel it's important, and
  nobody's going to be typing in the furls manually, so who does it benefit
  to have less capability than more?

It's not about the benefit of less capability.  It's about avoiding
mess.  If the fe80:: addresses of a host are advertised to an
introducer, then almost all nodes (in general) will not be able to reach
that address.  Even worse, it will time out rather than get a connection
refused.  And it leaks the MAC address, which its itself a security
concern (possible with stateless autoconf, but people who have
configured static addresses to avoid this should not be exposed).

In the v6 specifications, the intent of link-local addresses (in
fe80::/16) is that they are only used for things like neighbor discovery
and routing protocols.  When you use a link-local address, the address
itself does not specify to the host stack which interface to use.
That's why you have the %wm0 or whatever showing up in the display
representation, which is based on the ifindex being inserted in one of
the bytes of the address.  To have the address be used by another host,
it has to not have that, and has to insert its own index.  So passing
them around is non-obvious; typically a routing protocol will just
record the other side's address and reuse it - but there it has its
*own* ifindex, specifying the interface the packet arrived on, rather
than tha remote side's index.

tahoe is designed primarily as a wide-area protocol; in the general case
all nodes are expected to be able to open a TCP connection to the
introducer, and then all clients are expected to open a connection to
all servers.  To do this sensibly, the introducer and each server node
need a globally-routable address.

I should point out that my bias is that running tahoe only locally
doesn't make sense in the first place.  There are other filesystems that
deal with resilience against disk failure and some of them have better
performance.  The real point is to get resilience against hazards that
affect an entire site, so wide-area connectivity is IMHO intrinsically
tied to the main use case.

  Other advantages are that they are not routed, so that they can be more
  "secret" than other addresses.  If you didn't want the world to know that
  you were using Tahoe, preferring more local over more remote addresses
  could be better.

I think this is the usual security-through-obscurity issue, and I don't
think it makes sense.  Outsiders can no more tell that you are using
tahoe with global addresses on your ethernet than they can tell about
private addresses.

I don't really understand your dual tunnel broker case.  With a normal
tunnel broker, you get a routable address for your end of the tunnel
(typically ::2 where the tunnel far end is ::1), and often an entire
/64.  So just using the routable addresses would work fine.  (I have 3
tunnels, one of which feeds a routed network of about 6 subnets carve up
From a /48, used by my group of about 100 at work).

  If you bring up a host, or set of hosts, in an environment without a DHCP
  server, and no IPv6 router, and don't run Avahi/Bonjour the only address
  that you'll come up with is the fe80 address.  With them included, your
  tahoe cluster can be brought up and connected to without any configuration,
  without any infrastructure, it would even work with only a crossover cable.

I really don't follow this.  You're saying that if you bring up 4
computers with no addressing, and then somehow figure out the link-local
address of the introducer, and put it in a furl and config files, and
then run tahoe, that this is somehow better than manually configuring 4
addresses (which then makes all sorts of other things easier)?  This is
IMHO a degenerate case, and it seems odd to want to add complexity to
tahoe to support it.  The furl is already a capability, so
auto-discovery seems inconsistent with tahoe's security goals.

A further complexity is that when you look at your link-local address on
the introducer, you'll see (assuming a:b:c:d:e:f ethernet address)
something that looks like
  fe80::a:b:c:d:e:f%wm0
But if you grab that out of buffer, it will look something like

  fe80::2:a:b:c:d:e:f

assuming 2 is the ifindex of wm0.  I don't remember which byte is used,
but I did figure this out in NetBSD recently.  (If my job weren't
developing new network protocols (some of which do use link-local
addresses), I can't imagine I would have dug into this.)  This is a
local OS decision how to represent the interface.  BSD does this (as
implemented by KAME), and I am 95% sure Linux does essentially the same
thing.  So to make this work, the server has to send the fe80:: address
without the interface ID to an introducer which is on the *same link*,
and the introdcer can then send it to clients which are also on the same
link.

But, how does the client know which interface to use to contact the
introducer?  So you need the client to take the actual LL address and
then add a e.g. %wm0 scopeid when they configure their client.

So if you really want to use fe80:: addresses, I would say the following
should let you do everything which will actually work, and avoid
cluttering everyone else with them:

  Users have to configure each client and server node with a LL address
  for the introducer, and put a %intfN scopeid on it.  Of course the
  introducer has to share a link with each node.

  Nodes contact the introducer, and send their bare (no scopeid) LL
  address.  The introducer keeps track of the incoming scopeid (it could
  have multiple interfaces), and applies it.  When sending addresses to
  a node, it checks that the scopeid matches the interface over which
  that node, and if so sends the bare LL address, and if not does not
  (because the two nodes are on different interfaces and therefore
  different links and thus will to interoperate).
  
I've been running IPV6 for 15 years or so (I don't really remember when
I brought up the 6bone connection).  The general plan has been to have
global addresses on systems, and to use them.  Those have varied from
6bone addresses to 6to4 addresses to modern addresses in 2001::.  The
only times I deal with link-local addresses are:

  looking at ndp entries

  looking at ripng status

  in occasional desparate times, using ssh to them to recover things.
  (I would never try to make tahoe work this way; I would fix the
  addresses and then have tahoe work with regular addresses.)

This is a little bit like putting RFC1918 addresses in the introducer,
where for a typical useful grid most nodes will not be able to connect
to most RFC1918 addresses.  But people do have significant RFC1918
privately routed networks.  They don't have routed link-local, by
definitions.

So to summarize, the only time it makes sense to use link-local
addresses are in situations where you will be talking to only hosts that
are on-link, and not talking to anything farther.  tahoe is almost never
that case.  Supporting link-local in tahoe requires a lot of complexity,
clutters the lists of addresses, and results in clients trying to make
connections that cannot succeed.

So my advice is to at least for now, limit things to global addresses.
It's easy enough to add in LL later, but I think it will be a tremendous
source of complexity.  And it should be optional and off by default,
because it's an irregular thing to do.

Greg
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 194 bytes
Desc: not available
URL: <http://tahoe-lafs.org/pipermail/tahoe-dev/attachments/20130216/731a5369/attachment.pgp>