#769 closed defect (invalid)

have buildslaves automatically build debian packages of foolscap, zfec, pycryptopp, pyutil, argparse, zbase32

Reported by: warner Owned by: nobody
Priority: critical Milestone: eventually
Component: packaging Version: 1.4.1
Keywords: debian ubuntu foolscap zfec pycryptopp pyutil argparse zbase32 install Cc: blackdew@…, bdew, exarkun
Launchpad Bug:

Description (last modified by warner)

I just tried to run tahoe --version from a .deb I built from trunk (on my sid system), and it fails with the following error:

pkg_resources.VersionConflict: (pycryptopp 0.5.2-1 (/usr/lib/python2.5/site-packages), Requirement.parse('pycryptopp>=0.5.15'))

I've updated the debian/control files to make this requirement visible to the debian package manager (i.e. it should not let me install this tahoe), but apart from that, we need updated pycryptopp debs before we can release 1.5.0 . We need them for all of the same platforms for which we provide Tahoe debs: etch/edgy/feisty/gutsy/hardy/hardy-amd64 .

I think we're ok with zfec and foolscap packages.

Change History (51)

comment:1 Changed at 2009-07-21T13:26:36Z by zooko

Whenever a new patch is committed to the pycryptopp repository, the buildbot builds new .deb packages for a couple of platforms -- Debian-unstable-i386 and Ubuntu-Jaunty-amd64:

http://allmydata.org/buildbot-pycryptopp/builders/BlackDew%20debian-unstable-i386/builds/10/steps/stdeb/logs/stdio

Probably the resulting .deb would work for you. How shall we arrange for it to be installed into an apt repository where your apt-get will find it?

comment:2 Changed at 2009-07-21T13:40:02Z by arthur

here is a copy of the debian files generated for i386 :

http://testgrid.allmydata.org:3567/uri/URI%3ACHK%3Aaws3mawi42itcztaotbeguefry%3A5nufi4nprsitbklcimg3rokoaxccitrpr4xi7yapjvbcpr672x4a%3A3%3A10%3A5438390

Content : pycryptopp_0.5.15-1.diff.gz pycryptopp_0.5.15-1.dsc pycryptopp_0.5.15-1_i386.build pycryptopp_0.5.15-1_i386.changes pycryptopp_0.5.15-1.tar.gz pycryptopp_0.5.15_i386.build pycryptopp_0.5.15.orig.tar.gz pycryptopp_0.5.15.tar.gz pycryptopp-0.5.15.tar.gz python-pycryptopp_0.5.15-1_i386.deb python-pycryptopp-dbg_0.5.15-1_i386.deb

comment:3 Changed at 2009-07-21T14:42:04Z by warner

  • Description modified (diff)

I updated the tahoe debian/control rules to declare a dependency upon pycryptopp-0.5.15: the tahoe .debs now correctly refuse to install.

Zooko: I set up a bunch of "flappserver upload-file" services for uploading tahoe .debs to the repository on hanford (which gets mirrored to allmydata.org). Let's do the same for the pycryptopp debs.

In buildslave@hanford, do this one or more times:

flappserver add ~/.flappserver -C DIST/main/binary-i386 upload-file ~/tahoe-debs/dists/DIST/main/binary-i386

(setting DIST appropriately for each time)

Each time you run that, it will emit a furl. Copy this to the buildslave, stored in something like ~/main-deb.furl . Then add an upload step to the buildmaster config, which runs:

flappclient --furlfile ~/main-deb.furl upload-file ../pycryptopp*.deb

You should also copy the "tahoe-update-apt.furl" and have a step to update the APT index by running

flappclient --furlfile ~/tahoe-update-apt.furl run-command

(take a look at any of the tahoe deb builders for an example)

I don't know how you ought manage the variety of platforms, though. We build and host tahoe debs for etch/edgy/feisty/gutsy/hardy/hardy-amd64, so it'd be nice to provide pycryptopp debs for all of those. The tahoe .deb process only actually has rules for etch, lenny, and sid: basically we use the etch rules for the py2.4 platforms (etch and edgy), sid for sid, and lenny for everything else. But we run those rules on actual edgy/feisty/etc systems so they declare the right dependencies (in general you can't install a newer deb on an older system, because it will declare a dependency upon newer versions of libc, etc).

You'd need to find some similar mapping for pycryptopp, and create debs for all the same platforms we do for Tahoe. It sounds like you're only currently creating debs for one of those platforms (sid), and that .deb probably won't install on any of the earlier ones.

comment:4 Changed at 2009-07-21T20:19:55Z by warner

Oh, or of course you could just log in to each of our debian/ubuntu buildslaves in turn, build .deb/.orig.tar.gz/.diff.gz packages, scp them over to hanford:~buildslave/tahoe-debs/... , and run 'make' to update the index and remirror to org. Any level of automation you want to apply to this process would make your life easier in the long run.

comment:5 Changed at 2009-07-22T12:59:15Z by zooko

Hm, I don't personally own or operate any of the Debian buildslaves currently connected to the pycryptopp buildbot -- http://allmydata.org/buildbot-pycryptopp/waterfall -- just the jaunty/amd64 one (yukyuk).

I guess in order to automatically produce .deb's for these various flavors of Debian, we need to add buildslaves for pycryptopp running each flavor? I'm not going to make time in the forseeable future to set up lots of buildslaves for various flavors of Debian, although I can definitely do Jaunty/amd64 right now...

Hm, the first step in your recipe doesn't work:

buildslave@hanford:~$ flappserver add ~/.flappserver -C jaunty/main/binary-amd64 upload-file ~/tahoe-debs/dists/jaunty/main/binary-amd64
Traceback (most recent call last):
  File "/usr/bin/flappserver", line 18, in <module>
    run_flappserver()
  File "/usr/lib/python2.5/site-packages/foolscap/appserver/cli.py", line 471, in run_flappserver
    r = dispatch(command, so)
  File "/usr/lib/python2.5/site-packages/foolscap/appserver/cli.py", line 440, in dispatch
    return c.run(options)
  File "/usr/lib/python2.5/site-packages/foolscap/appserver/cli.py", line 191, in run
    s = build_service(service_basedir, None, service_type, service_args)
  File "/usr/lib/python2.5/site-packages/foolscap/appserver/services.py", line 279, in build_service
    raise UnknownServiceType(service_type)
foolscap.appserver.services.UnknownServiceType: -C

Anyway, as I was saying, I'll spend some time setting up automation to produce .deb's for jaunty/amd64, and I'll also set up a server owned by allmydata.com for lenny/i386, but I don't care about the various other flavors of Debian enough to set up .deb-producing automation for them. If someone else wants to volunteer to do the buildslave side of this recipe for a Debian flavor that they care about then I'll do the buildmaster side of the recipe.

comment:6 follow-up: Changed at 2009-07-23T15:32:16Z by zooko

  • Cc blackdew@… added
  • Keywords debian added
  • Status changed from new to assigned

I set up a pycryptopp buildslave on a server owned by allmydata.com named "slave-etch":

http://allmydata.org/buildbot-pycryptopp/builders/allmydata.com%20debian-etch-i686

I looked around, but allmydata.com doesn't seem to run a Debian lenny server. As soon as Brian tells me what's wrong with the flappserver setup (above), I'll set that up to receive .deb's at hanford.

Meanwhile, does someone want to volunteer to run a Debian lenny (5.0) buildslave? We already have a Debian sid (unstable) buildslave thanks to Black Dew:

http://allmydata.org/buildbot-pycryptopp/builders/BlackDew%20debian-unstable-i386/builds/11/steps/stdeb/logs/stdio

Oh, I see that "stdeb 0.3-bdew-custom-fixes (/usr/lib/python2.5/site-packages/stdeb-0.3_bdew_custom_fixes-py2.5.egg)" doesn't satisfy the requirement that I just added of stdeb >= 0.3. :-) I'll loosen that requirement to just stdeb. Black Dew: what custom fixes are in that package? I think you reported them all to stdeb upstream (Andrew Straw), but I'm not sure. Black Dew: would you be willing to set up a flapp client to upload the resulting .deb's for Debuan unstable to our apt repository?

comment:7 Changed at 2009-07-23T17:16:49Z by warner

sorry, use lowercase "-c" or "--comment", not uppercase "-C".

You could manually build pycryptopp debs on each of the allmydata-owned debian buildslaves: we have one for each debian release that we make tahoe .debs for (obviously). This is what I do each time I make a new foolscap release, regardless of whether tahoe depends upon it the new version or not. I have a script to reduce the typing involved, though. The list is in the AdminServers on the (allmydata-only) dev trac instance, or in the comments on the buildmaster config's slavename list.

The reason I'm concerned about this is because the tahoe debs that we've been building for years now are useless without all of the support debs. pycryptopp is the only one that we're missing. Until we have updated pycryptopp debs, users of those older platforms (including Hardy, the current Ubuntu long-term-support release) will be forced to build their own, a relatively difficult and annoying process for someone used to simply typing "apt-get install allmydata-tahoe". In addition, servers which are already running hardy (such as the allmydata.com hosts) will be unable to upgrade until their admins manually build pycryptopp debs.

comment:8 Changed at 2009-07-23T17:24:17Z by zooko

I would be happy for Tahoe-LAFS (and pycryptopp) to be apt-get install'able by people who use various Debian platforms and who configure their /etc/apt/sources.list to point to allmydata.org's apt repositories. However, I have very little time to work on fun stuff (Tahoe-LAFS) nowadays, and manually building .deb's doesn't qualify as "fun". If someone pays me to do it, or if someone else does it, then .deb's will manually get built. Fortunately, setting up automation to automatically build .deb's counts as "fun". ;-)

comment:9 Changed at 2009-07-24T13:07:37Z by zooko

sorry, use lowercase "-c" or "--comment", not uppercase "-C".

I thought I tried that...

Yes, I did:

buildslave@hanford:~$ flappserver add ~/.flappserver -c jaunty/main/binary-amd64 upload-file ~/tahoe-debs/dists/jaunty/main/binary-amd64
Traceback (most recent call last):
  File "/usr/bin/flappserver", line 18, in <module>
    run_flappserver()
  File "/usr/lib/python2.5/site-packages/foolscap/appserver/cli.py", line 471, in run_flappserver
    r = dispatch(command, so)
  File "/usr/lib/python2.5/site-packages/foolscap/appserver/cli.py", line 440, in dispatch
    return c.run(options)
  File "/usr/lib/python2.5/site-packages/foolscap/appserver/cli.py", line 191, in run
    s = build_service(service_basedir, None, service_type, service_args)
  File "/usr/lib/python2.5/site-packages/foolscap/appserver/services.py", line 279, in build_service
    raise UnknownServiceType(service_type)
foolscap.appserver.services.UnknownServiceType: -c
buildslave@hanford:~$ flappserver add ~/.flappserver --comment jaunty/main/binary-amd64 upload-file ~/tahoe-debs/dists/jaunty/main/binary-amd64
Traceback (most recent call last):
  File "/usr/bin/flappserver", line 18, in <module>
    run_flappserver()
  File "/usr/lib/python2.5/site-packages/foolscap/appserver/cli.py", line 471, in run_flappserver
    r = dispatch(command, so)
  File "/usr/lib/python2.5/site-packages/foolscap/appserver/cli.py", line 440, in dispatch
    return c.run(options)
  File "/usr/lib/python2.5/site-packages/foolscap/appserver/cli.py", line 191, in run
    s = build_service(service_basedir, None, service_type, service_args)
  File "/usr/lib/python2.5/site-packages/foolscap/appserver/services.py", line 279, in build_service
    raise UnknownServiceType(service_type)
foolscap.appserver.services.UnknownServiceType: --comment

Since I get only about an hour to work on Tahoe-LAFS every day (at most, and in several small bursts), and since you and I are out of sync so that it takes about 24 hours for every round trip between us, maybe you could set up the flappserver on hanford?

Or, if you don't want to, then please write back and tell me what's wrong with my current flappserver setup command-line. :-)

comment:10 Changed at 2009-07-24T13:09:27Z by zooko

  • Cc bdew added

We now have three Debian buildslaves which are successfully generating .deb's:

http://allmydata.org/buildbot-pycryptopp/builders/linux-amd64-ubuntu-jaunty-yukyuk http://allmydata.org/buildbot-pycryptopp/builders/BlackDew%20debian-unstable-i386 http://allmydata.org/buildbot-pycryptopp/builders/allmydata.com%20debian-etch-i686

The one that I don't have a login to is the one that I most want to have .deb's for -- Debian unstable. Hopefully Black Dew will be willing to set up the client side flapp service to upload the .deb's that his buildslave is building.

comment:11 Changed at 2009-07-24T14:23:46Z by zooko

To my surprise, Hardy and Etch have been used a lot more than Jaunty, Lenny, or Sid in the last three months since the Tahoe-LAFS v1.4 release:

http://allmydata.org/pipermail/tahoe-dev/2009-July/002384.html

comment:12 Changed at 2009-07-24T15:17:22Z by zooko

Fixed statistics:

http://allmydata.org/pipermail/tahoe-dev/2009-July/002386.html

If this is true it means that Brian is more or less the only user of Tahoe-LAFS on Debian sid. :-)

comment:13 Changed at 2009-07-24T17:49:00Z by zooko

Arthur pointed out that people use older apt repos than their actual debian dist:

http://allmydata.org/pipermail/tahoe-dev/2009-July/002387.html

And he volunteered a Debian lenny buildslave:

http://allmydata.org/buildbot-pycryptopp/builders/Arthur%20debian-lenny-c7-i386

comment:14 Changed at 2009-07-25T22:41:53Z by zooko

So I skipped the --comment option, which I couldn't figure out, but it worked without the --comment option. Now we have automatic upload of pycryptopp .deb's and automatic rebuild of the apt repo index so that "apt-get" can find and download the .deb's! So far the jaunty-amd64 buildslave (my workstation) is the only one doing this:

http://allmydata.org/buildbot-pycryptopp/builders/linux-amd64-ubuntu-jaunty-yukyuk

The next step is for other buildslave operators to agree to install the furl files into their buildslave base directory, and then I'll set the buildmaster to invoke the "upload-deb" and "update-apt-repo" on those buildslaves. The furl files are confidential (knowledge of them gives the ability to upload the debs and regenerate the apt repo index, respectively), so write to zooko@… asking for yours and I'll e-mail you yours back.

comment:15 Changed at 2009-07-25T23:21:16Z by zooko

Hm, sudo apt-get install python-pycryptopp doesn't work on my system even after the buildbot appears to have uploaded the pycryptopp .deb and updated the apt repo for jaunty on amd64 in this step:

http://allmydata.org/buildbot-pycryptopp/builders/linux-amd64-ubuntu-jaunty-yukyuk/builds/23/steps/update%20apt%20repo/logs/stdio

It says:

""" MAIN yukyuk:~$ sudo apt-get install python-pycryptopp Reading package lists... 0% Reading package lists... 3% Reading package lists... Done Building dependency tree Reading state information... Done

Package python-pycryptopp is not available, but is referred to by another package. This may mean that the package is missing, has been obsoleted, or is only available from another source E: Package python-pycryptopp has no installation candidate """

My /etc/apt/sources.list includes:

deb http://allmydata.org/debian/  jaunty  main tahoe

Is there a manual step that needs to be done on the apt repository server to initialize the jaunty repo for the first time? (I think before this automated step started getting run there were no "jaunty"-flavored debs on this server.)

By the way, once we have the new automated apt-repo for pycryptopp sorted out then we need to update this wiki page, adding jaunty and setting the flags correctly for the imminent v1.5.0 release: http://allmydata.org/trac/tahoe/wiki/DownloadDebianPackages

comment:16 in reply to: ↑ 6 ; follow-up: Changed at 2009-07-26T06:50:26Z by bdew

Oh, I see that "stdeb 0.3-bdew-custom-fixes (/usr/lib/python2.5/site-packages/stdeb-0.3_bdew_custom_fixes-py2.5.egg)" doesn't satisfy the requirement that I just added of stdeb >= 0.3. :-) I'll loosen that requirement to just stdeb. Black Dew: what custom fixes are in that package? I think you reported them all to stdeb upstream (Andrew Straw), but I'm not sure.

The fixes there were needed to make it work with extras-require - and it was removed from the packages because of the problems IIRC, so it should work with unmodified stdeb i think.

I'll try replacing it with the original and see how it works.

Black Dew: would you be willing to set up a flapp client to upload the resulting .deb's for Debuan unstable to our apt repository?

Sure, send me the furl and i'll install flappclient later today.

comment:17 in reply to: ↑ 16 Changed at 2009-07-26T10:12:38Z by bdew

The fixes there were needed to make it work with extras-require - and it was removed from the packages because of the problems IIRC, so it should work with unmodified stdeb i think.

Ok i've checked and my changes aren't needed for now, bu there's a patch from Zooko (http://github.com/astraw/stdeb/issues/unreads#issue/2) that fixes a bug with pathes that contain spaces, and it's not included in 3.0

Also note that currently stdeb on zfec fails for me with:

The required version of setuptools (>=0.6c12dev) is not available, and
can't be installed while this script is running. Please install
 a more recent version first, using 'easy_install -U setuptools'.

(Currently using setuptools 0.6c9 (/usr/lib/python2.4/site-packages))

This happens because stdeb loads setuptools manually before calling setup.py, and setup.py gets the system default version, instead of the one it has bundled.

I can install 0.6c12dev as a default on my buildslave and it'll probably build the packages, but this isn't something to be expected on end users systems if they want to build the debs themself.

comment:18 Changed at 2009-07-26T12:07:55Z by zooko

In the long term we'll probably switch to the new fork of setuptools named "Distribute" http://mail.python.org/pipermail/distutils-sig/2009-July/012665.html and contribute patches to that project.

For now it wouldn't hurt for you to install setuptools (actually zetuptoolz 0.6c12dev on your system, but on the other hand we already have .deb's for zfec so it is okay if your system can't produce .deb's of zfec at the moment.

comment:19 Changed at 2009-07-27T13:13:37Z by zooko

Black Dew's debian-unstable buildslave is now trying to upload .deb's...

http://allmydata.org/buildbot-pycryptopp/builders/BlackDew%20debian-unstable-i386

comment:20 Changed at 2009-07-27T13:58:28Z by zooko

It worked! Now we need someone with Debian to test whether sudo apt-get install allmydata-tahoe works.

comment:21 Changed at 2009-07-28T13:32:12Z by zooko

Okay, and thanks to Arthur Lutz we now have a second Debian buildslave uploading .deb's, this one is lenny:

http://allmydata.org/buildbot-pycryptopp/builders/Arthur%20debian-lenny-c7-i386/builds/22/steps/upload%20deb/logs/stdio

Now we just need someone with Debian-or-Ubuntu/x86 to test whether the pycryptopp .deb's in the allmydata.org apt repo work for them (and whether therefore the allmydata-tahoe .deb's also work).

comment:22 Changed at 2009-07-28T14:23:09Z by arthur

This is what I get when doing a apt-get update :

Err http://allmydata.org lenny/main Packages

404 Not Found

Err http://allmydata.org lenny/tahoe Packages

404 Not Found

Err http://allmydata.org lenny/main Sources

404 Not Found

Err http://allmydata.org lenny/tahoe Sources

404 Not Found

W: Failed to fetch http://allmydata.org/debian/dists/lenny/main/binary-i386/Packages 404 Not Found

W: Failed to fetch http://allmydata.org/debian/dists/lenny/tahoe/binary-i386/Packages 404 Not Found

W: Failed to fetch http://allmydata.org/debian/dists/lenny/main/source/Sources 404 Not Found

W: Failed to fetch http://allmydata.org/debian/dists/lenny/tahoe/source/Sources 404 Not Found

E: Some index files failed to download, they have been ignored, or old ones used instead.

comment:23 Changed at 2009-07-28T14:42:34Z by terrell

in the browser - i confirm

404 for http://allmydata.org/debian/dists/lenny/main/binary-i386/Packages

but...

200 for http://allmydata.org/debian/dists/lenny/main/binary-i386/

listing python-pycryptopp_0.5.16-1_i386.deb

so something is wonky with the paths - looking for Packages and Sources that don't exist

comment:24 Changed at 2009-07-28T21:38:32Z by warner

I'm in an airport now, but when I'm back at home tomorrow I'll try to fix up the apt repo update scripts to include the new releases: lenny, jaunty, etc. (I had set up the flappserver receivers ahead of time, for more releases than we had APT automation to handle).

I'm a bit worried about the version numbers on the pycryptopp debs: it looks like they're being created with "0.5.16-1", with no "r1234"-type suffix. Is that right? The APT repository (and the debian clients which pull from it) will get confused if two different binaries are uploaded with the same version number, such as if 0.5.16-r1234 and 0.5.16-r1235 are both uploaded under the name "0.5.16-1". Is stdeb using the output of darcsver?

comment:25 Changed at 2009-07-29T13:51:02Z by zooko

Hm, no I get .deb files with the full version number including -r:

-rw-r--r--  1 zooko zooko 449358 2009-07-29 07:55 ./python-pycryptopp_0.5.16-r667-1_amd64.deb

Oh, except for the "release" versions, which contain no new patches since the most recent release tag. Those ones leave out the -r$COUNT but keep the -1 that stdeb seems to want to append:

-rw-r--r--  1 zooko zooko  449348 2009-07-29 08:03 ./python-pycryptopp_0.5.16-1_amd64.deb

Good enough?

comment:26 Changed at 2009-07-31T01:32:23Z by warner

yeah, that sounds great. thanks!

I failed to get the APT repo updated today. I'll be back online next tuesday or wednesday, and will try to get it done then. Note to self: look into using "mini-dinstall" instead of apt-ftparchive. Also, POX says to look at http://upsilon.cc/~zack/blog/posts/2009/04/howto:_uploading_to_people.d.o_using_dput/ .

comment:27 Changed at 2009-07-31T01:43:54Z by zooko

  • Milestone changed from 1.5.0 to 1.5.1

I think we should announce Tahoe-LAFS v1.5.0 Saturday even if we don't have this ticket (apt-get install allmydata-tahoe) isn't fixed and even if #773 (document installation on Windows) isn't fixed. I highly value those tickets, but this release is just taking way too long and I want to get it over with and move on! We can fix these tickets in the days following the v1.5.0 release.

Moving this from "v1.5.0" Milestone to "v1.5.1".

comment:28 Changed at 2009-08-01T17:59:06Z by zooko

Sebastian Kuzminsky might offer some help setting up apt repos, in which case I might have a whack at it myself today.

comment:29 Changed at 2009-08-03T05:39:09Z by zooko

I hooked up a buildslave for pycryptopp on the Ubuntu Hardy server operated by allmydata.com:

http://allmydata.org/buildbot-pycryptopp/builders/allmydata.com-hardy-i386%20syslib/builds/8

Arthur is already running a lenny buildslave on i386 which is uploading .deb's for pycryptopp:

http://allmydata.org/buildbot-pycryptopp/builders/Arthur%20debian-lenny-c7-i386/builds/22

I hope that these .deb's will make apt-get install allmydata-tahoe work for most current Debian users, in which case we can remove the note at the top of http://allmydata.org/trac/tahoe/wiki/DownloadDebianPackages that sadly says apt-get is not currently working.

comment:30 Changed at 2009-08-03T22:09:34Z by zooko

My friend Seb Kuzminsky uses a tool named "dpkg-scanpackages", I think it was called, to generate apt repo indexes from a set of .deb's.

(He also told me about a tool called "pbuilder" (??) which creates a chroot and installs only the debian packages that you specified in your build-depends. That way you find out if you forgot to include a dep in your build-depends.)

comment:31 Changed at 2009-08-10T15:50:28Z by zooko

Brian posted an update on this issue to the list:

http://allmydata.org/pipermail/tahoe-dev/2009-August/002547.html

comment:32 Changed at 2009-08-15T03:56:24Z by zooko

#644 was a duplicate of this.

comment:33 Changed at 2009-10-02T19:09:54Z by zooko

exarkun might be able to set up a grid once this ticket is fixed.

comment:34 Changed at 2009-10-02T19:11:47Z by exarkun

  • Cc exarkun added

comment:35 Changed at 2009-10-02T19:17:22Z by zooko

I wonder if exarkun gets mail when I comment on this ticket.

comment:36 Changed at 2009-10-02T19:39:43Z by exarkun

Signs point to yes.

:)

comment:37 Changed at 2009-10-26T20:40:51Z by zooko

  • Keywords ubuntu added
  • Milestone changed from 1.5.1 to eventually
  • Owner changed from zooko to nobody
  • Status changed from assigned to new

We need a hardy buildslave to move forward on this:

http://allmydata.org/buildbot-pycryptopp/waterfall

comment:38 Changed at 2009-10-26T21:12:49Z by zooko

  • Summary changed from debian 'sid' package fails to run: pycryptopp-0.5.15 not available in debian form to need .deb's of pycryptopp and zfec

comment:39 Changed at 2009-10-26T21:36:23Z by zooko

And we need a sid buildslave too. We already have a lenny one thanks to soultcer. We have a hardy-amd64 buildslave for Tahoe-LAFS thanks to Zandr Milewski, so maybe we could get him to set up buildslaves on that machine for pycryptopp and zfec.

comment:40 Changed at 2009-11-27T22:55:33Z by warner

The testgrid introducer is offline: it is hosted on a gutsy box, and was accidentally rebooted yesterday. Tahoe is not currently installed on it, because the current allmydata-tahoe .deb cannot be installed, due to a lack of a gutsy pycryptopp-0.5.15 debian package.

comment:41 Changed at 2009-11-28T18:01:19Z by zooko

According to wiki:DownloadDebianPackages there are six debian/ubuntu platforms that we currently intend to support (that's how I interpret the column titled priority): lenny i386, sid i386, hardy i386, hardy amd64, karmic i386, and karmic amd64.

Of those six two come with Tahoe-LAFS natively (Karmic both i386 anda amd64).

One is currently marked as fully working: hardy i386 has "apt-get installable: yes", "deb buildable: yes", "tahoe deb available: yes", "support debs available: yes", "runs-from-source: yes.

The other three all have missing pieces: sid i386 doesn't have "tahoe deb available", hardy amd64 doesn't have "support debs available" or "apt-get installable" and lenny i386 has a bunch of question marks and parentheticals that I don't understand.

Now the process of producing and hosting .deb's for zfec and pycryptopp is very automated. To set it up you:

  1. Set up a buildslave on the platform for which you wish to produce .deb's.
  2. Turn on the build_deb=True argument to the make_factory() for that builder in the master.cfg file.
  3. Turn on the upload_deb=True argument.
  4. Give that buildslave a flappclient furl in $WORKINGDIR/../../main-deb.furl which allows it to upload the resulting .deb to the right place.
  5. Give that buildslave a flappclient furl in $WORKINGDIR/../../tahoe-update-apt.furl which allows it to trigger the "rebuild our apt repository" service.

I am motivated to do these steps for hardy-amd64, but pycryptopp doesn't have a hardy-amd64 buildslave. I guess we're waiting on someone to contribute a hardy amd64 buildslave and contribute their time installing the flappclient furls.

I'm personally not motivated to do these steps for other platforms at this time. In particular I don't think we should support platforms that no longer get security fixes from their operating system provider, such as Ubuntu Gutsy, because there's little point in running a secure Tahoe-LAFS node on an insecure operating system.

But I guess if someone else goes to all the effort to maintain the buildslave for any platform then I'll go to the effort of configuring it in the master.cfg and creating the flappfurls.

comment:42 Changed at 2009-11-28T18:13:56Z by zooko

Brian reports that the pycryptopp .deb for Hardy-i386 that we currently host is built wrong:

<warner> the 0.5.16-r669 deb declares a dependency on python-central >=0.6.7

And python-central 0.6.7 is too new for Hardy, apparently. I just realized that pycryptopp doesn't have a buildslave for Hardy-i386 any more than it has one for Hardy-amd64, so I'll change the wiki:DownloadDebianPackages to show that we don't have .deb's of pycryptopp for Hardy-i386 and I'll eventually post to tahoe-dev asking for someone to contribute a hardy-i386 and hardy-amd64 buildslave for pycryptopp.

comment:43 Changed at 2009-11-28T18:18:35Z by zooko

Brian asked me to clarify that the fact that Tahoe-LAFS ships with Karmic doesn't mean that we provide .deb's of newer versions of Tahoe-LAFS for Karmic. I'll see if I can update wiki:DownloadDebianPackages to show that what you get in Karmic is only Tahoe-LAFS v1.5.0.

comment:44 Changed at 2009-11-28T18:35:32Z by warner

FYI, we have hardware for a hardy-i386 buildslave (a machine known as "deharo1"), but it's currently offline. I'll try to schedule some colo time next weekend to bring it back online. We also have a second machine which could probably be rebuilt to be a karmic buildslave.

comment:45 Changed at 2009-12-12T02:34:04Z by davidsarah

  • Keywords pycryptopp zfec install added

comment:46 Changed at 2009-12-12T04:21:42Z by zooko

  • Summary changed from need .deb's of pycryptopp and zfec to have buildslaves automatically build debian packages of foolscap, zfec, pycryptopp, pyutil, argparse, zbase32

#422 was a duplicate of this.

comment:47 Changed at 2009-12-13T00:02:01Z by davidsarah

  • Keywords foolscap pyutil argparse zbase32 added

comment:48 Changed at 2010-02-27T08:58:01Z by davidsarah

#498 was a duplicate of this.

comment:49 Changed at 2010-11-13T02:13:36Z by davidsarah

The remaining part of #978 was a duplicate of this. Copying a possibly-relevant comment here:

leif wrote:

Following ioerror's instructions in debian-docs-patch-final.txt (found on ticket #961) I was able to build a Debian lenny package.

The only thing I did differently than ioerror was that I also used stdeb to make debs of the two remaining things he easy_install'd: argparse and zbase32.

To make argparse's setup.py happy, I ran sed -i -e 's/__file__/"."/' setup.py .

comment:50 Changed at 2011-08-16T00:46:50Z by davidsarah

Is this still relevant given that there are official Debian packages of our dependencies now?

comment:51 Changed at 2011-08-26T22:22:07Z by davidsarah

  • Resolution set to invalid
  • Status changed from new to closed

The deb builders have been decommissioned.

Note: See TracTickets for help on using tickets.