"client node probably started"

[zooko]

It would be nice if we could remove the "probably" from that message. How about doing a Foolscap "Hi there" with it? (That was Sam Stoller's suggestion.)

[warner]

The "probably" is there because the runner process has no clear way of knowing if the new process dies right away or continues running.

I've got some code in buildbot which watches the logfile and looks for the message that indicates startup has been successful.. perhaps we could snarf it for this purpose.

What do you mean by a 'Foolscap "Hi there"' message?

-Brian

[zooko]

If the runner process has some positive indication that the new process started up long enough to perform some action (such as writing a message to the log or connecting back to the runner process with Foolscap and saying "Hi there"), then the runner process should inform the user that the process has started, without the "probably".

So, yes, snarfing that code from buildbot would be fine with me.

[nejucomo]

I'm working on the foolscap approach. I believe it's possible to connect to the node and call get_version, so I'll use that if possible. (I've started by modifying the runner tests to start a node and fail if "probably started" appears in the output.)

[zooko]

This would fit nicely into the theme of v0.6.1: documentation, packaging, user-friendliness, etc.

[warner]

I'd advise the logfile-scanning approach. Benefits:

• any exceptions or warnings which occur during startup are displayed to the admin who is starting the node, at exactly the time and place they need to see it
• it displays exceptions even if foolscap fails to work (i.e. if pyopenssl isn't installed). Logfile writing is the only requirement

Downsides:

• it generally requires forking off a process, which is problematic under windows. I think I have a good-enough solution for this in buildbot, but I think it involves limited functionality

[zooko]

Bumping to v0.7 milestone.

Nejucomo: if you aren't planning to fix this ticket, would you please take your name off the "assigned" field?

[warner]

I built a prototype of this, watching twistd.log until the introducer has been contacted. I suspect it will have interactions with windows though (forking), and it probably breaks the 'start -m' (multiple nodes) functionality.

I plan to make it work better once I've gotten more progress down on #197.

[warner]

prototype implementation

[zooko]

I forget exactly how many people I have watched going through the Tahoe install and launch process. About half a dozen. Every single one has exclaimed at "Client node probably started.". I just watched another person do it, and they too exclaimed in exactly the same way, so let's say it's seven out of seven.

[guest]

I will add another vote that "probably" is not a very reassuring word choice. While things seem to be working, I still am unclear as to why I've been told that tahoe has only "probably started"

[warner]

Incidentally, I just learned that modern twistd can be run as a library. See ​http://divmod.org/trac/browser/trunk/Axiom/axiom/scripts/axiomatic.py for an example. This would make it easier to avoid the extra subprocess, and might make it easier to provide a more confident answer to this ticket.

In general, if we can instantiate the Client before the fork, then the parent process can be sure that:

1. the child was able to load all the correct Tahoe code, and import all the dependencies
2. the tahoe.cfg file was well-formed and none of its values caused immediate problems

To feel confident that the Client actually got started, we'll need to establish some form of communication between the "tahoe start" parent and the actual node process, whether that means tailing the logfile or connecting to the control.furl .

[zooko]

Jeremy Visser has packaged Tahoe-LAFS v1.6.1 for Ubuntu Lucid. He tried to test his package by following these instructions: ​http://allmydata.org/source/tahoe-lafs/trunk/docs/running.html but he got stuck and gave up on testing it (until I reminded him to try again). So I asked why he had given up:

<jayvee> I'm reading that, and not getting very far
<zooko> Why not?
<zooko> Sounds like I need to file a bug report on that page. :_)
<jayvee> oh, just the feedback I'm getting is not very descriptive
<jayvee> "introducer node probably started"
<jayvee> I'm basically expecting to see "tahoe successfully started, browse to
	 this_url to view contents"
<jayvee> but maybe I'm just a simpleton
<jayvee> 'tahoe run' blocks with no feedback. I presume that's intentional (no
	 news is good news), but a little disconcerting to someone who has
	 never used it before.

[zooko]

<jayvee> I ran 'tahoe start .' and 'tahoe run', and yet nothing is listening
	 on port 3456.
<jayvee> the documentation (using.html) says that should be the case

[zooko]

<jayvee> zooko, as a point of comparison, this is what upstart gives me
<jayvee> $ sudo start mythtv-backend
<jayvee> mythtv-backend start/running, process 17060
<jayvee> much more satisfying. even printing the PID makes me much more
	 confident.

[zooko]

It looks like ​http://twistedmatrix.com/trac/ticket/823 would solve this ticket with its --wait option.

See also #602 which is about "probably not started" not being sufficiently detailed and #529 which is about detecting problems on startup and failing loudly instead of quietly, and #371 which is about a common problem on startup.

[davidsarah]

We could just adapt the approach suggested in the twisted ticket (and implemented in ​this patch) rather than waiting for twisted to adopt it. That would also allow us to receive arbitrary messages from the child process and print them, addressing ticket:68#comment:53 for example.

[zooko]

It would be nice to contribute to Twisted. We either do so directly by contributing patches and code review for ​Twisted #823 and then waiting for it to be deployed and the using it in Tahoe-LAFS, or at least we could work on a patch within Tahoe-LAFS but be sure to carefully cross-link it with the relevant Twisted tickets and to try to get a similar patch committed to Twisted.

[warner]

I think this is resolved by ac3b26ecf29c08cb .. anyone want to confirm?

[zooko]

I ran tahoe start and it didn't print out any uncertainty-inducing messages:

Zooko-Ofsimplegeos-MacBook-Pro:~ pubvolgrid$ tahoe start
STARTING '/Users/pubvolgrid/.tahoe'

Hm, news-needed.

[zooko]

Hey does this mean that we can start running all these tests on cygwin and/or windows now:

test_runner.py​

It looks like both of these conditions which force tests to be skipped are now irrelevant and all tests should be runnable, but I'm not sure.

[davidsarah]

Replying to zooko:

Hey does this mean that we can start running all these tests on cygwin and/or windows now:

test_runner.py​

Possibly, I will investigate that.

[davidsarah]

Replying to zooko:

Hey does this mean that we can start running all these tests on cygwin and/or windows now: test_runner.py​

Apparently not.

The cygwin part of this is #908, and is due to a bug in twisted.internet.utils on cygwin that apparently causes it to hang. (I haven't tested it with recent cygwin, but it wouldn't have been affected by ac3b26ecf29c08cb.)

For native Windows, we currently skip the test_runner.RunNode tests because of #27 (twistd doesn't daemonize on windows). That is, tahoe start behaves like tahoe run on Windows, which is too different for the tests to work. It looks non-trivial to make them work without fixing either #27 or #1121 (test 'tahoe run').

[zooko]

Thanks for investigating!

[davidsarah]

Replying to davidsarah:

[...] That is, tahoe start behaves like tahoe run on Windows, [...]

More precisely: tahoe start now behaves like tahoe run. Prior to ac3b26ecf29c08cb, it used os.system to run twistd, which put the node in a different process to the tahoe command, although that process did not then daemonize. Since ac3b26ecf29c08cb, it runs the node in the same process as the tahoe command. Hmm, is that a regression?

(​Here is the code for twistd on Windows, and ​here is for Unix.)

[zooko]

Replying to davidsarah:

More precisely: tahoe start now behaves like tahoe run. Prior to ac3b26ecf29c08cb, it used os.system to run twistd, which put the node in a different process to the tahoe command, although that process did not then daemonize. Since ac3b26ecf29c08cb, it runs the node in the same process as the tahoe command. Hmm, is that a regression?

I don't think anybody benefited from or cared about the fact that it used to run it in a separate process. It just made it harder to kill it on Windows.