#2491 closed task (fixed)

synchronous node startup

Reported by: dawuud Owned by: warner
Priority: normal Milestone: 1.12.0
Component: code-nodeadmin Version: 1.10.1
Keywords: tor i2p static synchronous startup Cc:
Launchpad Bug:

Description

Synchronous node startup is part of the native Tor integration as specified in #517

Change History (15)

comment:1 Changed at 2015-08-29T13:27:58Z by dawuud

  • Keywords i2p static synchronous startup added

comment:2 Changed at 2015-08-29T14:36:26Z by dawuud

this untested rough draft attempt at synchronous startup for tahoe in my dev branch: https://github.com/david415/tahoe-lafs/tree/2491.synchronous-node-startup.0

this change uses get_all_ip_addresses from the new foolscap helper function specified in this trac ticket: http://foolscap.lothar.com/trac/ticket/238#comment:1

comment:3 Changed at 2016-03-28T23:52:37Z by warner

More details copied from #517:

At node-creation time (e.g. tahoe create-node):

  • tahoe create-node should allocate (or be told) a port to listen on, and write it into tahoe.cfg tub.port as a server endpoint descriptor string (so tcp:12345, or tcp:12345:interface=127.0.0.1).
  • it should also detect (or be told) an IP address to use in the FURL, and write it as tub.location (as a Foolscap connection hint, so tcp:HOST:PORT)

At runtime (e.g. tahoe start):

  • Try to get the port number from BASEDIR/client.port (which must not be "0"). If that's empty, read tahoe.cfg tub.port (which might be missing, which means "AUTO"; we can probably reject "0" here too).
  • if our port is "AUTO" (which only happens if we have a node created before the above tahoe create-node was deployed, but which has never been started with tahoe start), then we use the utility function to allocate a port, and we write it into BASEDIR/client.port.
  • Now we know what port to listen on. Build a server endpoint descriptor string and pass it to tub.listenOn().
  • Try to get the tub location from tahoe.cfg tub.location.
  • If that's empty, use the synchronous non-Twisted IP-address detection function to get a list of IP addresses. Combine that with the port we're listening on (from above) to build a list of hints. Join them with commas.
  • Pass the hints into tub.setLocation().
  • Now call the rest of the Node/Client startup functions.

Other code changes/accomplishments:

  • node.Node._tub_ready_observer and when_tub_ready go away
  • several client.Client methods stop using when_tub_ready (and stop using Deferreds): init_introducer_client, init_storage, init_control, init_helper, init_key_gen.
  • in particular, we call tub.setLocation() *before* calling any of those init methods, so they can safely use tub.registerReference

Other tasks necessary to make this work:

  • move the controlport and logport to separate Tubs, with a separate (ephemeral) key, listening on a runtime-allocated port, and 127.0.0.1
  • consider adding a config knob to override this: to set a stable port and key, so you can copy the logport.furl to a separate machine and use flogtool tail after a reboot.
Last edited at 2016-03-29T01:06:58Z by warner (previous) (diff)

comment:4 Changed at 2016-03-29T01:05:30Z by warner

I've pushed warner/2491-sync to my github, which rebases and builds upon your branch. It passes most tests, but a couple are still broken:

  • test_system leaves a dirty reactor
  • test_introducer.ClientSeqnums is broken.. probably shallow

I like how much simpler the setup code is without needing to wait for the Tub to be ready.

comment:5 Changed at 2016-03-29T01:06:38Z by warner

(so far, my branch addresses the "runtime" changes above, and the "other code changes/accomplishments", but doesn't change create-node, and doesn't touch controlport/logport)

comment:6 Changed at 2016-04-08T08:11:45Z by warner

I've split out the tahoe create-node --location= UI task to #2773.

comment:7 Changed at 2016-04-26T21:04:37Z by Brian Warner <warner@…>

In 6dfcd52/trunk:

iputil.py: copy allocate_tcp_port() from foolscap

refs ticket:2491

comment:8 Changed at 2016-04-26T21:04:38Z by Brian Warner <warner@…>

In e9e1265/trunk:

SystemTestMixin?: don't auto-detect IP addresses

The tubs created for test_system should always use 127.0.0.1 .

refs ticket:2491

comment:9 Changed at 2016-04-27T05:36:55Z by warner

  • Keywords review-needed added
  • Owner changed from daira to warner
  • Status changed from new to assigned

comment:10 Changed at 2016-04-27T21:33:07Z by Brian Warner <warner@…>

In 96c6259/trunk:

Teach tub to start synchronously even with ip autodetect

  • remove when_tub_ready() from all code
  • synchronous-ify all node/client/introducer startup code

refs ticket:2491

comment:11 Changed at 2016-04-27T21:36:00Z by warner

  • Keywords review-needed removed

Ok, that's merged, thanks meejah for the +1.

Still left to do for this ticket:

  • move controlport/logport to a separate Tub

And we should open a separate ticket for the changes to tahoe create-node.

comment:12 Changed at 2016-04-28T05:13:01Z by warner

This fixes #726, where port-already-in-use errors were written to the log instead of being signalled immediately.

comment:13 Changed at 2016-06-29T18:01:21Z by warner

  • Component changed from unknown to code-nodeadmin
  • Milestone changed from undecided to 1.13.0
  • Type changed from defect to task

#2773 is for the tahoe create-node argv changes: adding --hostname / --location / -- port

comment:14 Changed at 2016-06-30T02:49:53Z by warner

As mentioned in #1377, http://twistedmatrix.com/trac/ticket/823 might help address one of the problems this is addressing: errors at startup getting written to the logfile instead of the controlling terminal (and the tahoe start process reporting success, not failure). twistd basically now does the old buildbot trick of waiting for the new child process to get running (by reading from a pipe-to-self) and reporting a short error message if it fails.

comment:15 Changed at 2016-06-30T03:06:30Z by warner

  • Milestone changed from 1.13.0 to 1.12.0
  • Resolution set to fixed
  • Status changed from assigned to closed

#2794 is for moving the controlport/logport to a separate Tub, so we can close this ticket now

Note: See TracTickets for help on using tickets.