#174 closed enhancement (fixed)

make argv $0 be 'tahoe', not 'twistd'

Reported by: warner Owned by: davidsarah
Priority: minor Milestone: 1.8.1
Component: code-nodeadmin Version: 1.2.0
Keywords: usability unix reviewed Cc: kevinperson@…, zandr@…
Launchpad Bug:

Description

To make it easier to identify a running tahoe node with standard tools like 'ps' and 'top', it would be awfully nice if argv[0] (aka $0) were 'tahoe'. At the moment, the way we spawn twistd means that a 'allmydata-tahoe start' command actually results in a process with an argv of /usr/bin/python /usr/bin/twistd -y tahoe-client.tac --logfile logs/twistd.log which is kind of gross.

To accomplish this, we'd need to invoke twistd directly (as a regular python function call), rather than using os.system() to spawn off a new process. This is tricky. I've done it in buildbot, so we can probably steal code from there, but it has ramifications for both win32 (#27) and for tailing-the-logfile (#71).

We've changed the name of the .tac file to contain the word "tahoe" (#156), so at least 'ps ax |grep tahoe' results in something useful now, but it would be nice to complete this effort.

Attachments (1)

fix-174.darcs.patch (25.8 KB) - added by davidsarah at 2010-11-28T00:42:53Z.
bin/tahoe-script.template: On non-Windows, invoke support/bin/tahoe directly as a script (rather than via python), so that 'top' for example will show it as 'tahoe'. On Windows, simplify some code that set argv[0], which is never used. fixes #174

Download all attachments as: .zip

Change History (31)

comment:1 Changed at 2007-10-12T00:52:52Z by warner

So, I stumbled across a feature in Application that might work well enough for our purposes here. By adding service.IProcess(application).processName = "tahoe"

to our .tac file, we go from an argv that looks like:

/usr/bin/python /usr/bin/twistd -y tahoe-client.tac --logfile logs/twistd.log

to one that looks like:

tahoe /usr/bin/twistd --originalname -y tahoe-client.tac --logfile logs/twistd.log

It looks kind of weird now, but 'tahoe' shows up as argv0, which is probably a good thing.

I'll do some more experimentation.

comment:2 Changed at 2007-10-12T01:05:29Z by warner

Unfortunately, 'top' still shows the process as 'python', instead of 'tahoe'.

comment:3 Changed at 2008-02-11T19:28:14Z by warner

  • Summary changed from make argv $0 be 'tahoe', not 'twistd' to make argv $0 be 'tahoe', not 'twistd', and add BASEDIR to argv

Also, we'd like the node's basedir to show up in the argv list, so that 'ps ax |grep tahoe' would tell us which node is which.

comment:4 Changed at 2008-03-16T00:37:33Z by keturn

  • Cc kevinperson@… added

we had to write a C extension to modify argv[0], so I'm curious if you find a cleaner way.

comment:5 Changed at 2008-06-01T21:02:58Z by warner

  • Milestone changed from eventually to undecided

comment:6 Changed at 2008-08-18T22:17:57Z by zooko

Hm, I was just playing with dupfilefind and I noticed that its entry in top is named "dupfilefind" instead of "python", even though its entry in "ps eax" is:

 8571 pts/5    D+     9:43 /usr/bin/python /usr/bin/dupfilefind --profiles .

comment:7 Changed at 2008-09-22T20:59:18Z by zandr

  • Cc zandr@… added
  • Priority changed from major to critical
  • Version changed from 0.6.0 to 1.2.0

comment:8 Changed at 2009-02-24T21:51:11Z by zandr

Upon reflection, I think I want the nickname in argv for nodes that have them, and the base class (cpu-watcher, stats-gatherer, et al) for those that don't.

This was inspired by the behavior of Munin, which shows the hostname of the target of each munin-update process.

comment:9 Changed at 2009-07-29T17:55:14Z by zooko

  • Priority changed from critical to major

Since the Milestone is "undecided" and nobody is actively working on this ticket, it doesn't seem right to have its priority set to "critical".

comment:10 Changed at 2009-07-29T17:56:09Z by zooko

If it *is* still of critical importance to a user of Tahoe-LAFS, such as Zandr, then please say so and we'll see if someone actually wants to spend time fixing it.

comment:11 Changed at 2009-07-29T18:02:12Z by zooko

<exarkun> zooko: well, uh, how about setting processName to tahoe?

comment:12 Changed at 2009-07-29T18:03:26Z by zooko

<hagna> zooko, exarkun: see http://jcalderone.livejournal.com/30438.html

comment:13 Changed at 2010-01-07T00:30:20Z by davidsarah

  • Keywords usability added
  • Type changed from defect to enhancement

comment:14 follow-up: Changed at 2010-11-27T09:35:52Z by warner

This might have been mostly fixed with the recent import+call+twistd.run() change. At least argv[0] should now be 'tahoe' instead of twistd. If you passed an explicit basedir (i.e. tahoe start FOO), then I think FOO will appear in your ps args.

Anyone want to check?

comment:15 in reply to: ↑ 14 Changed at 2010-11-27T16:04:05Z by francois

Replying to warner:

Anyone want to check?

The cmdline as shown by ps ax looks fine but the process name displayed by top remains python.

$ ps ax | egrep [t]ahoe
21233 ?        S      0:03 /usr/bin/python /home/francois/dev/tahoe-upstream/support/bin/tahoe start /home/francois/.tahoe
$ top -b -n 1 -p 21233
top - 16:33:40 up 1 day, 23:01,  7 users,  load average: 0.56, 0.46, 0.39
Tasks:   1 total,   0 running,   1 sleeping,   0 stopped,   0 zombie
Cpu(s):  5.3%us,  1.6%sy,  0.0%ni, 92.4%id,  0.7%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   2056044k total,  1985072k used,    70972k free,    54296k buffers
Swap:  6024336k total,    45356k used,  5978980k free,   506808k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                        
21233 francois  20   0  210m  83m 2620 S    0  4.1   0:03.00 python                                                                         

$ cat /proc/21233/status | grep Name
Name:	python
$

comment:16 Changed at 2010-11-27T20:18:17Z by warner

Hm, I guess most interpreted-language programs (those with #! lines) will show up that way. Should we change it? That is, do folks think it's useful to know which version/instance of python is being used to run your node? I suppose if it helps you reconstruct the running program, it might be, although then there may be other ambiguities involved (i.e. if you just run "tahoe start", such that it searches your $PATH for "tahoe", will "ps" show you which one it found?).

I imagine we might be able to have the process rearrange its sys.argv array to try and change what ps sees, but I'm not 100% sure that it'd be a good idea..

comment:17 follow-up: Changed at 2010-11-27T20:47:06Z by davidsarah

  • Keywords unix added

Here's what twistd does to change argv[0]: http://twistedmatrix.com/trac/browser/trunk/twisted/scripts/_twistd_unix.py?rev=27324#L173

(name is given by twisted.application.service.IProcess(application).processName.)

This requires execv and so is only done on Unix. (I don't think there's any way to change how the Processes tab of Task Manager lists processes on Windows.)

comment:18 Changed at 2010-11-27T20:49:39Z by davidsarah

  • Priority changed from major to minor

Reducing priority to minor, since the commandline now does give all significant information.

comment:19 in reply to: ↑ 17 Changed at 2010-11-27T20:54:19Z by warner

Replying to davidsarah:

Here's what twistd does to change argv[0]: http://twistedmatrix.com/trac/browser/trunk/twisted/scripts/_twistd_unix.py?rev=27324#L173

Huh. That suggests that there isn't any way to affect argv from python (i.e. sys.argv is a one-time copy of the C data structure, and changes are not propagated back). Or at least that the Twisted devs decided it wasn't worth trying to use such a feature.

I'm not sure if that extra exec() is going to cause us problems. We're planning to set up a fairly specific environment before calling twistd.run(), and I don't think that re-execing the current contents of argv are likely to keep that same state around. It'll need some experimentation.

Changed at 2010-11-28T00:42:53Z by davidsarah

bin/tahoe-script.template: On non-Windows, invoke support/bin/tahoe directly as a script (rather than via python), so that 'top' for example will show it as 'tahoe'. On Windows, simplify some code that set argv[0], which is never used. fixes #174

comment:20 Changed at 2010-11-28T00:54:56Z by davidsarah

  • Keywords review-needed added
  • Owner set to warner

I agree that an extra exec isn't worth it. However I think that on Unix we can invoke the support/bin/tahoe script directly, rather than via python, which should achieve the same effect. (That script's shebang line should point to the same Python interpreter as the bin/tahoe script, since both should be the full path to the Python that ran setup.py build.) Please repeat the checks in comment:15 to confirm.

I don't think we need a test for this, since the existing tests should catch any regressions. (It would be difficult to test, since sys.argv[0] is the same for a script invoked directly or via python.)

comment:21 Changed at 2010-11-28T01:59:01Z by davidsarah

  • Keywords news-needed added

comment:22 Changed at 2010-11-28T03:39:30Z by davidsarah

  • Milestone changed from undecided to 1.9.0

comment:23 follow-up: Changed at 2010-11-28T07:57:33Z by warner

davidsarah: applying your patch on my OS-X box and then using 'tahoe start' to launch a couple of nodes gives me 'ps aux' output that looks the same as it did with current trunk:

185:warner@host-4-50% ps aux |grep tahoe
warner   50886   0.0  0.4  2480848  17332   ??  S    11:22PM   0:00.15 /usr/bin/python /Users/warner2/stuff/tahoe/t2-174/support/bin/tahoe start ../MY-TESTNET/node-3

"top" on OS-X just shows "Python" in the COMMAND column. (OS-X's top is a bit weird. Actually, all OS-X unix tools are a bit weird. sysv vs bsd, I guess.)

Is that what was supposed to happen?

(if it did the right thing, I suppose I'd r+ the patch, OTOH I get a deep sense of dread when seeing that weird .template file get changed, and it raises my "this file shouldn't even exist" hackles. I'd be much more keen to r+ a patch that deleted it altogether :-)

For the record, I'll be happy if the 'ps aux' output includes "tahoe", since "ps aux |grep tahoe" is what I always do to find out if a tahoe process is currently running. And I'm also happy if the argv array includes the target directory. Both of these conditions appear to be true right now.

comment:24 in reply to: ↑ 23 Changed at 2010-11-28T17:02:56Z by davidsarah

Replying to warner:

davidsarah: applying your patch on my OS-X box and then using 'tahoe start' to launch a couple of nodes gives me 'ps aux' output that looks the same as it did with current trunk:

185:warner@host-4-50% ps aux |grep tahoe
warner   50886   0.0  0.4  2480848  17332   ??  S    11:22PM   0:00.15 /usr/bin/python /Users/warner2/stuff/tahoe/t2-174/support/bin/tahoe start ../MY-TESTNET/node-3

"top" on OS-X just shows "Python" in the COMMAND column. (OS-X's top is a bit weird. Actually, all OS-X unix tools are a bit weird. sysv vs bsd, I guess.)

Is that what was supposed to happen?

No, it was supposed to change the command line to just "/Users/warner2/stuff/tahoe/t2-174/support/bin/tahoe start ../MY-TESTNET/node-3". Note that python setup.py build is needed, I should have mentioned that.

comment:25 follow-up: Changed at 2011-01-01T21:31:44Z by davidsarah

  • Keywords news-needed removed
  • Summary changed from make argv $0 be 'tahoe', not 'twistd', and add BASEDIR to argv to make argv $0 be 'tahoe', not 'twistd'

The patch doesn't help on OS X. Can someone try it on Linux?

comment:26 in reply to: ↑ 25 Changed at 2011-01-02T22:16:13Z by francois

Replying to davidsarah:

The patch doesn't help on OS X. Can someone try it on Linux?

This patch was tried on my Ubuntu 10.10 box. The output of the ps command did not changed but the output of the top command did changed.

In both cases.

francois@korn:~$ ps aux |grep tahoe
francois  6778  1.3  4.0 213176 82868 ?        S    22:36   0:01 /usr/bin/python /home/francois/dev/tahoe-upstream/support/bin/tahoe start
francois  6873  0.0  0.0   9212   844 pts/4    S+   22:37   0:00 grep tahoe

Without the patch.

francois@korn:~/dev/tahoe$ top -b -n 1 | egrep "python|tahoe"
 7208 francois  20   0  208m  81m 2620 S    0  4.0   0:00.97 python             

With it.

francois@korn:~/dev/tahoe$ top -b -n 1 | egrep "python|tahoe"
 7354 francois  20   0  208m  80m 2580 S    2  4.0   0:00.92 tahoe              

comment:27 Changed at 2011-01-03T00:38:41Z by davidsarah

  • Keywords reviewed added; review-needed removed
  • Owner changed from warner to davidsarah
  • Status changed from new to assigned

francois said on irc:

yes I read the code and haven't found any obvious flaw in it

comment:28 Changed at 2011-01-03T01:13:20Z by david-sarah@…

  • Resolution set to fixed
  • Status changed from assigned to closed

In 9815852a09582776:

bin/tahoe-script.template: On non-Windows, invoke support/bin/tahoe directly as a script (rather than via python), so that 'top' for example will show it as 'tahoe'. On Windows, simplify some code that set argv[0], which is never used. fixes #174

comment:29 Changed at 2011-01-06T01:28:43Z by david-sarah@…

In 1190ce614303b6fb:

NEWS: 'top' for node processes, WUI formatting, removal of GUI apps, documentation updates, foolscap dependency. refs #174, #1219, #1225

comment:30 Changed at 2011-07-19T01:40:41Z by davidsarah

  • Milestone changed from 1.9.0 to 1.8.1
Note: See TracTickets for help on using tickets.