`tahoe create-node` should require `--location` and/or `--hostname`, and not autodetect

[warner]

This splits out the most user-visible aspect of #2491. The create-node commands that create listening nodes should always be told what hostname or IP address(es) to advertise, rather than guessing by using IP-address autodetection functions.

That means the following commands should require a --location= argument:

• create-node (when storage service is enabled)
• create-introducer
• create-key-generator (deleted in #2783)
• create-stats-gatherer

These commands should also accept (but not require) a --port= argument, which says what port the server should listen on. These will be endpoint descriptors, so things like tcp:12345. Both location and port get written into tahoe.cfg. Unlike --location=, --port= is not mandatory, and the code should pick something sensible if it isn't provided.

I'm undecided on what exactly counts as "sensible". One option is just to allocate a free one inside create-node (different for each server). Another is for use to choose a port number for Tahoe servers to listen on (now, in this ticket, maybe register it with IANA or something), and let the option default to that. A third is a hybrid: attempt to listen on the default port during create-node and record that (in tahoe.cfg) if that succeeds, but if it's already claimed by some other process, allocate and record a random one.

Note: if the server is using a Tor "onion service", then the user should not be obligated to figure this out: the --tor option should provision and register the XYZ.onion location automatically. So --location= and --port= should *not* be required (and in fact should probably be rejected) if --tor is active.

But for plain TCP locations, the server should stop trying to figure it out on its own, and have the node-constructing command get this information from the user.

[warner]

Hm, I realized I'm not sure whether --location= is supposed to be just a hostname (example.com), or a full Foolscap connection hint (tcp:example.com:12345).

If it's just a hostname, then it makes sense for --port to be optional, because the command can build a connection hint from the given hostname and a self-allocated port.

If it's a full connection hint, then you basically must provide a --port that matches the hint: --location=tcp:example.com:12345 --port=12345. If you have some funky non-1-to-1 port-mapping firewall going on, you might provide different port numbers (where --location points at the external one, and --port provides the internal one). You'd do the same to use an externally-configured Tor onion service, --location=tor:xyz.onion:80 --port=tcp:12345.

I suppose we could say that if you pass --location, but not --port, and the location is in the form tcp:HOSTNAME:PORT, then we glean --port from the last component of the location, but this feels a bit magic to me.

We could also say that if you provide --location=tcp:HOSTNAME:PORT1 and a different --port=PORT2, then we could pretend that you really said --location=tcp:HOSTNAME:PORT2, but that would be even more magical, and would prevent setting up the portmapping example above.

I guess I'm most inclined to be fully explicit. --location= would be a full tcp:HOSTNAME:PORT connection hint (or a comma-separated list of them). --port= would be a full endpoint specification (so tcp:PORT or tcp:PORT:interface=127.0.0.1), with maybe the "tcp:" being optional.

Oh, what if we have a separate argument name --hostname=? Then we'd have the following possibilities:

• --hostname=example.com : self-allocate a port, use location=tcp:example.com:PORT, and port=tcp:PORT
• --hostname=example.com --port=12345: use tcp:example.com:12345 and tcp:12345
• --location=tcp:example.com:12345 --port=tcp:12345: use those as location and port
• it would be an error to provide both --hostname and --location, or to provide --location without --port.

And then using an automatically-configured Tor onion service would still be something different, like --tor or --listen-tor, which builds the service, then sets --location and --port internally.

[warner]

#1478 would be easier with something like this in place.

[warner]

At today's devchat, meejah and I worked something out.

The node creation commands are responsible for writing tub.location and tub.port into tahoe.cfg, with the same semantics as we have now: tub.location is a foolscap hint list (e.g. "tcp:HOSTNAME:PORT,tcp:HOST2:PORT2"), and tub.port is a twisted server endpoint specifiction ("tcp:12345" or "tcp:12345:interface=127.0.0.1"). These commands will also write information about Tor setup, when necessary. All allocation is finished by the time create-node exits, so nothing dynamic needs to happend at runtime (tahoe start).

Then the server-like node-creation commands (create-server, create-introducer, and create-node if it means client+server) have a couple of different allowable cases

• tahoe create-server --hostname=HOST: This allocates a TCP port (maybe an IANA-standardized Tahoe port, maybe just any free one), and sets location to tcp:HOSTNAME:PORT and port to tcp:PORT.

• --listen-tor: This allocates an onion address (inside create-server, which will need to start up a reactor, maybe start a tor instance, and talk to Tor's control port to create the private key), then sets location to tor:XYZ.onion:PORT1 and port to tcp:PORT2:interface=127.0.01. If we're able to use a unix socket for the inbound connections from tor, the port will be something like unix:PATH instead.

• --listen-i2p: same, but for I2P

• any combination of the above three

• --location= --port=: This explicitly provides both the FURL connection hints and the listening port. --location and --port must be provided as a pair (both, or neither, but never just one). If they are provided, none of --hostname/--listen-tor/--listen-i2p are accepted.

In the outbound direction, clients will automatically use Tor hints if possible. At node startup, they'll attempt to import txtorcon, and if that works, they'll register a foolscap connection handler for "tor" hints. The handler will provide Tor endpoints on request (synchronously). Then, when .connect() is called on one of those hints, the handler will attempt to spin up a Tor instance. If this fails (e.g. because Tor is not actually installed), the connection will fail, no big deal. Same story for i2p: try to import txi2p, if that works install the connection handler (which might not work), if not, don't.

tahoe.cfg will have an option to *not* use Tor, for the benefit of folks operating in an environment where they have txtorcon and Tor installed, but for whatever reason don't want want Tahoe to use them. I don't think this is common enough to need a CLI argument: folks can just edit tahoe.cfg after node creation.

The client-like node creation commands (create-client, maybe create-node) will then have an argument like --only-tor which indicates that all outbound TCP connections ought to route through Tor instead. It will also have an --anonymous flag that enforces restrictions on the other settings: it requires --only-tor, it looks at the server settings and complains if it sees non-onion hints in --location or non-localhost listening ports in --port.

The client-like commands will also take something like --socks-port, which registers a foolscap connection handler (for both tcp and tor) that sends everything to the given SOCKS server. This might be a Tor instance (in which case both tcp:HOST:PORT and tor:XYZ.onion:PORT will work), or it might be a plain SOCKS daemon (so tor: won't work).

All commands will accept --tor-controlport=endpoint, which means txtorcon ought to talk to the pre-existing Tor instance at that control port, rather than launching its own. This gets written into tahoe.cfg somehow.

It might be a good idea to have the node creation commands, when asked to use tor, check to see if it will actually work. That means spinning up a Tor executable (or connecting to the --tor-controlport= provided) and seeing if Tor can get the full descriptor list. Maybe even making a connection to a tahoe-lafs.org -hosted onion service to make sure it works.

And looking towards the future, meejah had a great idea about a sort of "setup wizard". tahoe create-node --web could run a server and pop open your web browser to a control panel. It could then run ifconfig and see if your box seemed to have a public IP address, or try to get a UPnP port, or provide instructions for setting up a firewall port-forwarding, etc. Then it could ask the user for permission to confirm connectivity, by having a service on tahoe-lafs.org connect back down to the purported address.

In the end, we really want running a server to be as easy as running a client, but the networking world in which we live doesn't make that trivial. I think --hostname= is about the simplest possible tool (but only works if you have a public-IP VPS server of some sort), and --location= --port= is the second-simplest tool.

[Brian Warner <warner@…>]

In f57d1e9/trunk:

Merge branch '2773-stats'

This changes "tahoe create-stats-gatherer" to take --hostname,
--location, and --port, according to (refs ticket:2773).

[warner]

BTW, that 2773-stats patch fixed the last of our "pending deprecations": warnings we'd get from running the test suite against the latest trunk version of Foolscap and Twisted.

[warner]

moving most tickets from 1.12 to 1.13 so we can release 1.12 with magic-folders