#961 closed enhancement (somebody else's problem)

init scripts included in the apt-get install for tahoe-lafs on Debian/Ubuntu derivatives

Reported by: stott Owned by: zooko
Priority: normal Milestone: 1.10.1
Component: packaging Version: 1.5.0
Keywords: init security docs reviewed debian ubuntu Cc:
Launchpad Bug:

Description (last modified by zooko)

/etc/init scripts should be included by default to start tahoe-lafs if there are configured nodes, introducrs or helpers, Tahoe should auto-start. Run level 3, after most other services are up and running.

Attachments (8)

tahoelafsd (3.1 KB) - added by ioerror at 2010-03-04T08:06:14Z.
An init.d script created for Debian Lenny (from /etc/init.d/skel)
tahoelafsd.2 (300 bytes) - added by ioerror at 2010-03-04T08:07:03Z.
this file goes in /etc/default/tahoelafsd
tahoelafsd.3 (581 bytes) - added by ioerror at 2010-03-04T08:27:45Z.
New defaults file for debian
tahoelafsd.4 (591 bytes) - added by ioerror at 2010-03-04T08:29:18Z.
tahoe defaults file
tahoelafsd.5 (3.2 KB) - added by ioerror at 2010-03-04T08:30:05Z.
finial version of init.d script that uses defaults properly
debian-init-script-patch-final (9.5 KB) - added by ioerror at 2010-03-04T09:42:30Z.
the lenny files required to install, remove, and purge a .deb
fix-debian-building-instructions.dpatch (2.2 KB) - added by francois at 2010-03-04T19:57:07Z.
This patch fix a few missing commands in ioerror's modifications to docs/debian.txt
debian-docs-patch-final.txt (4.4 KB) - added by ioerror at 2010-03-05T00:41:22Z.
Only two packages aren't .debs now

Download all attachments as: .zip

Change History (44)

comment:1 follow-up: Changed at 2010-02-21T01:49:19Z by USSJoin

So I just spent a few hours trying to write an upstart (the new ubuntu replacement for init.d) script for Tahoe. It appears not to be possible, because the Tahoe daemon forks in such a way that the initctl processes can't follow it (with either the "expect daemon" or "expect fork" stanzas).

However, there's an easy workaround I've found, that I'm now using:

ln -s /usr/local/bin/tahoe /etc/init.d/tahoe

This *does* require that the "root" user be able to run tahoe, which requires an

ln -s /path/to/your/.tahoe /root/.tahoe

But that's also required by any (theoretical) upstart script.

comment:2 in reply to: ↑ 1 Changed at 2010-02-22T00:05:00Z by davidsarah

  • Keywords security added

Replying to USSJoin:

However, there's an easy workaround I've found, that I'm now using:

ln -s /usr/local/bin/tahoe /etc/init.d/tahoe

This *does* require that the "root" user be able to run tahoe, which requires an

ln -s /path/to/your/.tahoe /root/.tahoe

But that's also required by any (theoretical) upstart script.

Note that this conflicts with #725 ("we should whine if we're running as root"). Apart from being a really bad idea because of excess privilege, running the gateway as root causes directories and files to be owned by root and not accessible to other users, which then hoses any attempt to run the gateway as another user. (The CLI should still work provided that tahoe create-node or tahoe create-client wasn't run as root.)

comment:3 Changed at 2010-02-27T09:00:54Z by davidsarah

  • Milestone changed from eventually to 1.7.0

Changed at 2010-03-04T08:06:14Z by ioerror

An init.d script created for Debian Lenny (from /etc/init.d/skel)

Changed at 2010-03-04T08:07:03Z by ioerror

this file goes in /etc/default/tahoelafsd

comment:4 Changed at 2010-03-04T08:08:06Z by ioerror

  • Keywords review-needed added

comment:5 Changed at 2010-03-04T08:10:43Z by ioerror

A Debian user should do the following to securely run Tahoe:

addgroup --system tahoelafsd
adduser --disabled-login --system --home /var/lib/tahoelafs --shell /bin/bash --ingroup tahoelafsd tahoelafsd 
tahoe create client /var/lib/tahoelafs/

When using the init.d script that I have attached, Tahoe will run as the user tahoelafsd user and not as root.

The Tahoe packages should include this as part of the files provided in debian/

comment:6 Changed at 2010-03-04T08:18:24Z by ioerror

It's probably a good idea to keep the entire tahoelafs home directory hidden from the rest of the system:

chmod 750 /var/lib/tahoelafs/

It's important to note that you'll want to edit the /var/lib/tahoelafs/tahoe.cfg file after the creation of the client config or the init.d script will not work.

By default, I think that the init.d script should simply not start tahoe without a configuration... I'll add that and replace the DEFAULTS file....

Changed at 2010-03-04T08:27:45Z by ioerror

New defaults file for debian

Changed at 2010-03-04T08:29:18Z by ioerror

tahoe defaults file

Changed at 2010-03-04T08:30:05Z by ioerror

finial version of init.d script that uses defaults properly

comment:7 Changed at 2010-03-04T08:31:14Z by ioerror

It's likely that Tahoe (post-configuration) should start at boot:

# update-rc.d -f tahoelafsd defaults
 Adding system startup for /etc/init.d/tahoelafsd ...
   /etc/rc0.d/K20tahoelafsd -> ../init.d/tahoelafsd
   /etc/rc1.d/K20tahoelafsd -> ../init.d/tahoelafsd
   /etc/rc6.d/K20tahoelafsd -> ../init.d/tahoelafsd
   /etc/rc2.d/S20tahoelafsd -> ../init.d/tahoelafsd
   /etc/rc3.d/S20tahoelafsd -> ../init.d/tahoelafsd
   /etc/rc4.d/S20tahoelafsd -> ../init.d/tahoelafsd
   /etc/rc5.d/S20tahoelafsd -> ../init.d/tahoelafsd

Changed at 2010-03-04T09:42:30Z by ioerror

the lenny files required to install, remove, and purge a .deb

comment:8 Changed at 2010-03-04T09:46:23Z by ioerror

With the above patch, I've installed, uninstalled, reinstalled, purged, and then installed again. I have configured my daemon to run by editing /etc/defaults/allmydata-tahoe and having a configured /var/lib/tahoelafsd directory...

I've also configured it to run at boot like so:

 update-rc.d -f allmydata-tahoe defaults
 Adding system startup for /etc/init.d/allmydata-tahoe ...
   /etc/rc0.d/K20allmydata-tahoe -> ../init.d/allmydata-tahoe
   /etc/rc1.d/K20allmydata-tahoe -> ../init.d/allmydata-tahoe
   /etc/rc6.d/K20allmydata-tahoe -> ../init.d/allmydata-tahoe
   /etc/rc2.d/S20allmydata-tahoe -> ../init.d/allmydata-tahoe
   /etc/rc3.d/S20allmydata-tahoe -> ../init.d/allmydata-tahoe
   /etc/rc4.d/S20allmydata-tahoe -> ../init.d/allmydata-tahoe
   /etc/rc5.d/S20allmydata-tahoe -> ../init.d/allmydata-tahoe

comment:9 follow-up: Changed at 2010-03-04T19:41:56Z by davidsarah

  • Keywords docs added

Needs documentation. Also, is dpkg --remove supposed to remove the user/group?

The Debian FAQ says:

  • Remove a package (but not its configuration files): dpkg --remove foo
  • Remove a package (including its configuration files): dpkg --purge foo

(also see here).

Maybe --purge should remove the user/group and --remove should not (but I'm not an expert on Debian package management).

comment:10 in reply to: ↑ 9 Changed at 2010-03-04T19:45:20Z by davidsarah

Replying to davidsarah:

Also, is dpkg --remove supposed to remove the user/group?

Oh, you changed that in a later patch. Never mind, that looks right now (remove only the user on purge).

Changed at 2010-03-04T19:57:07Z by francois

This patch fix a few missing commands in ioerror's modifications to docs/debian.txt

Changed at 2010-03-05T00:41:22Z by ioerror

Only two packages aren't .debs now

comment:11 Changed at 2010-03-05T00:42:15Z by ioerror

What other documentation should I write up? Perhaps a man page? Or just continuing to flesh out the debian.txt?

I've attached my current debian.txt - it's how I build Tahoe on my current production system that is on the volunteer grid. I changed a few steps to match the documentation patch from francois.

comment:12 Changed at 2010-03-06T02:56:28Z by zooko

  • Owner changed from somebody to warner

comment:13 Changed at 2010-03-06T03:32:57Z by ioerror

There's one issue that is currently outstanding in this package but seemingly unrelated to Debian. It appears that setuptools has a multiple input warning:

# /etc/init.d/allmydata-tahoe start
/usr/lib/python2.5/site-packages/zope/__init__.py:19: UserWarning: Module allmydata was already imported from /usr/lib/python2.5/site-packages/allmydata/__init__.py, but /usr/lib/python2.5/site-packages is being added to sys.path
  import pkg_resources
STARTING /var/lib/tahoelafsd
client node probably started

comment:14 Changed at 2010-03-06T03:49:21Z by ioerror

diff -u allmydata-tahoe /tmp/allmydata-tahoe.init --- allmydata-tahoe.init 2010-03-04 00:48:45.000000000 -0800 +++ /tmp/allmydata-tahoe.init 2010-03-05 19:31:53.000000000 -0800 @@ -21,7 +21,6 @@

DAEMONHOME="/var/lib/tahoelafsd/" DAEMON=/usr/bin/$NAME DAEMON_ARGS=" start $DAEMONHOME"

-PIDFILE=/var/lib/tahoelafsd/twisted.pid

USERNAME=tahoelafsd SCRIPTNAME=/etc/init.d/$NAME

Version 0, edited at 2010-03-06T03:49:21Z by ioerror (next)

comment:15 Changed at 2010-03-10T03:26:00Z by zooko

Let's see... this ticket was "assigned" to warner (by me) but he never "accepted" it, so he hasn't indicated that he actually intends to review it soon. Could someone else review it?

comment:16 Changed at 2010-03-10T19:59:36Z by warner

FYI, "tahoe start" is a simple frontend to "twistd", the Twisted daemon-launching tool. Eventually we'll fix it so that any extra arguments you pass to "tahoe start" will be copied to "twistd", which will let you get a bit more control over how the daemon is started.

In the meantime, you can run twistd directly. "tahoe start BASEDIR" is equivalent to something like "cd BASEDIR && twistd -y *.tac --logfile logs/twistd.log". If you add --nodaemon to the twistd arguments, it won't daemonize. If you want to launch tahoe from an init.d script using start-stop-daemon, that's probably what you want to use. You might also want to use the --pidfile argument to control where the pidfile goes.

comment:17 Changed at 2010-04-12T17:37:20Z by zooko

  • Owner changed from warner to somebody

What's the status of this ticket? It was assigned to Brian but he hasn't accepted it, and it is marked as review-needed. I don't understand if Brian's comment:16 was intended as a code review requesting further changes or if it was just helpful commentary. I'm unassigning this from Brian and issuing a call for a reviewer. PatchReviewProcess

comment:18 Changed at 2010-06-17T04:39:31Z by zooko

  • Milestone changed from 1.7.0 to soon
  • Owner changed from somebody to warner

Brian: I wonder if people are put off from taking another step on this ticket because they aren't sure what you meant in comment:16. Just in case, would you please state whether the code here is good enough or whether in your opinion changes are needed? If the former, please remove review-needed from the keywords and add reviewed to the keywords. If the latter, please remove review-needed from the keywords. Thanks!

comment:19 Changed at 2010-06-17T22:37:16Z by taral

python-foolscap is packaged in debian sid.

Instead of packaging argparse and zbase32, you could just include them in the allmydata package directly. They are pretty small.

+1 to running twistd directly instead of using tahoe start. Or add a --nodaemon flag to tahoe start that causes it to exec twistd instead of forking?

Why do you have automatically generated sections in your postinst? Oh, I see. darcs unrecord is your friend.

Why not remove the group you created?

comment:20 Changed at 2010-06-17T23:04:25Z by warner

  • Owner changed from warner to nobody

Zooko: comment:16 was intended as helpful commentary. Folks who want to write init scripts but who are put off by "tahoe start"'s automatic daemonizing. Such folks could use the information in comment:16 to write a better init script.

I haven't touched tahoe+debian packaging in a long time.. I'm not currently familiar with how we're doing it (if at all). The last I remember was planning a new out-of-tree scheme using git-buildpackage and a collection of schroot jails (of various ubuntu/debian) versions on an EC2 host. My thought was to copy the debian.diff files from that effort back into the tahoe source tree. I vaguely remember thinking that I wanted to keep the in-tree stuff simple, and leave things like init.d scripts "to the professionals" (i.e. the actual ubuntu/debian packages that ship in Lenny, etc), and perhaps as a result not putting energy into landing tese patches.

So I cannot currently sign off on the patches: I don't have the time to page that stuff back into my brain and do a proper review. Maybe next week.

comment:21 Changed at 2010-07-14T06:19:55Z by zooko

  • Summary changed from There are no init scripts included in the Ubuntu apt-get install for tahoe-lafs to init scripts included in the Ubuntu apt-get install for tahoe-lafs

comment:22 Changed at 2010-07-17T23:40:52Z by davidsarah

  • Keywords reviewed added; review-needed removed

I had reviewed a previous version of these patches, and the changes since then look good. I think that any remaining issues (e.g. comment:13) are minor and new tickets should be opened for them.

I suggest committing these patches for 1.7.1, or failing that 1.8beta. They won't cause regressions; the worst that can happen is that the scripts don't do precisely what users want.

comment:23 Changed at 2010-07-18T04:35:33Z by zooko

  • Milestone changed from soon to 1.7.1
  • Owner changed from nobody to zooko
  • Status changed from new to assigned

comment:24 Changed at 2010-07-18T05:52:50Z by zooko

  • Keywords reviewed removed
  • Resolution set to fixed
  • Status changed from assigned to closed

applied in ca660a5fc6cb8d55, babbdf949437116b. Thanks, Jacob and David-Sarah!

comment:25 follow-up: Changed at 2013-07-06T14:43:22Z by zooko

  • Description modified (diff)
  • Resolution fixed deleted
  • Status changed from closed to reopened

I was talking with intrigeri about including Tahoe-LAFS in Tails (https://tails.boum.org/), and he pointed out this Debian bug report: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=652003 . That bug report by Thomas Pierson <contact@…> says that he can't find the actual init script anywhere. Sure enough, reviewing this ticket it appears to me that we accidentally omitted the actual init script (final version attached to this ticket by ioerror: attachment:debian-init-script-patch-final).

Daira: did you review attachment:debian-init-script-patch-final? You wrote review notes in comment:22.

ioerror: why did you post the diff in comment:14? Why remove the PID file variable setting?

comment:26 Changed at 2013-07-06T15:06:51Z by gdt

Is there a documented general strategy for how to place OS-specific helper files? It seems that these could be provided for a dozen systems, at least and thus there should be some subdirectory structure to avoid this getting cluttered.

There's also the issue of whether things are included with the sources or are part of packaging. Probably most things belong in the sources so they can be used/tested independently of packages. But user creation feels like it should be a packaging thing.

comment:27 in reply to: ↑ 25 ; follow-up: Changed at 2013-07-06T19:21:31Z by daira

  • Keywords reviewed added
  • Milestone changed from 1.7.1 to 1.11.0
  • Priority changed from minor to normal

Replying to zooko:

Daira: did you review attachment:debian-init-script-patch-final?

Yes (and I just rereviewed it). I'm not an expert on Debian packaging, but it looks fine to me, modulo this:

ioerror: why did you post the diff in comment:14? Why remove the PID file variable setting?

I'm not ioerror, but the PIDFILE variable appears to be unused (and the pid file is called twistd.pid anyway, not twisted.pid).

comment:28 in reply to: ↑ 27 Changed at 2013-07-06T19:24:19Z by daira

Replying to daira:

Replying to zooko:

Daira: did you review attachment:debian-init-script-patch-final?

Yes (and I just rereviewed it). I'm not an expert on Debian packaging, but it looks fine to me, modulo this: [...]

and misc/lenny no longer being an appropriate directory to put it in.

comment:29 Changed at 2013-07-07T18:42:26Z by zooko

Per ticket #1454, we decided to remove the debian package files from the upstream source repository since those files were now being maintained by debian. This turns out to be partially mistaken, as this particular set of files (the init scripts), we never successfully transfered to the Debian maintainers yet!

comment:30 Changed at 2013-07-07T23:26:59Z by zooko

Okay I created a branch in github, but it should not be merged to trunk because of #1454. As I said in the commit message, I'm creating this branch solely to give Debian/Ubuntu/Tails packagers and us a shared codebase to point at and talk about:

https://github.com/zooko/tahoe-lafs/commit/cb6d458ab5c638f7b203ef07b91b4fb5873d91e0

Last edited at 2013-07-07T23:32:06Z by zooko (previous) (diff)

comment:31 Changed at 2013-08-05T23:26:54Z by daira

  • Summary changed from init scripts included in the Ubuntu apt-get install for tahoe-lafs to init scripts included in the apt-get install for tahoe-lafs on Debian/Ubuntu derivatives

comment:32 Changed at 2013-08-05T23:27:16Z by daira

  • Keywords debian ubuntu added

comment:33 Changed at 2013-08-27T14:55:26Z by zooko

  • Resolution set to somebody else's problem
  • Status changed from reopened to closed

Okay, I've posted to http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=652003 asking the Debian maintainers to take over the init scripts. Closing this ticket as "Somebody Else's Problem".

comment:34 Changed at 2013-09-09T22:34:49Z by zooko

I'm leaving this ticket closed, but here I'm posting a copy of my code-review of bertagaz's patch which is now in Debian.

Thank you for working on this! I just reviewed http://anonscm.debian.org/gitweb/?p=tahoe/tahoe.git;a=blob;f=debian/README.Debian;h=14ea2a7c1b38d41df36dd052c44cdf22603fd775;hb=87f7666c2c3a5059dc28ea95c336b9de7f08ae47 . It is interesting that this is using the username within the local operating system as the "nick". That isn't what we intended the nickname to be for, but I don't see anything obviously wrong with it. It means that the node will announce to everyone who connects to it what the local username is. Is that what was expected?

Hm, actually now that I think this through, I think the human reading that doc needs a warning about this! I think you should add a sentence that says "This nickname will be announced to all participants in the grid." or something like that. Or, change the suggested command-lines so that it no longer fills in the "nick" field from the username.

comment:35 Changed at 2013-09-09T22:40:27Z by zooko

Okay, I just reviewed http://anonscm.debian.org/gitweb/?p=tahoe/tahoe.git;a=blob;f=debian/tahoe-lafs.init;h=13b505fb4b2d4be959e3df7edef02a369a48fc7c;hb=8428876521454b5fd2b0719048caf909c0ab68ee . Here are my comments.

Thank you for working on this patch! I'm excited about making Tahoe-LAFS be a more first-class citizen of the Debian universe. A lot of good can come of this. Thank you for your contribution.

I don't understand why /etc/init.d/tahoe-lafs restart does tahoe stop ; sleep 1 ; tahoe start and /etc/init.d/tahoe-lafs force-reload does tahoe restart. I think both of those should do tahoe restart.

I think /etc/init.d/tahoe-lafs stop should do tahoe stop instead of kill `cat twistd.pid`. That's because tahoe stop has a couple of features such as warning the user if the daemon doesn't stop after SIGKILL. See the code here.

Other than those two things, I didn't see anything strange or objectionable in this patch. Thanks again!

comment:36 Changed at 2013-10-01T03:43:49Z by zooko

I posted another, much bigger, review to http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=652003 but I'm not copying it into this ticket, so go read that thread.

Note: See TracTickets for help on using tickets.