#617 closed defect (fixed)

Debian etch package size mismatch

Reported by: swillden Owned by: warner
Priority: major Milestone:
Component: packaging Version: 1.2.0
Keywords: debian packaging Cc:
Launchpad Bug:


The r3571 build of Tahoe 1.2.0 in the allmydata.org debian repository has a size mismatch. The Packages file reports the size as 738,742 bytes, but the .deb file is actually 738,826 bytes in size. This causes apt to refuse to install the downloaded package.

Change History (13)

comment:1 Changed at 2009-02-10T18:13:03Z by swillden

  • Component changed from unknown to packaging
  • Owner changed from nobody to somebody

comment:2 Changed at 2009-02-10T18:26:17Z by warner

  • Owner changed from somebody to warner
  • Status changed from new to assigned

argg. What happens is that the Packages file was being rebuilt while a new .deb was being uploaded. This happens because we've got half a dozen different platforms, each of which uploads a .deb and then triggers the Packages-rebuild.. there's a lock in place to prevent overlapping rebuilds, but there's nothing to prevent the rebuild from seeing a partially-uploaded .deb and remembering its (truncated) size forever.

When this happens, we usually manually delete and rebuild the Packages file. But it happens a lot, and is really annoying.

One fix would be to use an uploader which keeps the file hidden (at least named something other than .deb) until the upload is finished. At the moment we're using scp or rsync (I forget which), which does not use a .tmp file. I'd like to change this to use the foolscap "xfer-client", to reduce the authority of the buildslave, and I'd also change xfer-client to use a .tmp file.

I'll rebuild the Packages file today, and then perhaps next week I'll get around to changing the upload process.

comment:3 Changed at 2009-06-16T22:49:44Z by warner

I'm working on foolscap#82, an "app server", which will be the clean way to fix this: buildslaves will get a single-purpose file-uploader FURL, and the server-side receiver will atomically move the inbound .deb files into place, not giving the update-index process a chance to see a partial file.

I'm getting close to finished.. I hope to have a new foolscap release out in the next few days, and then update the upload-deb process another day or two later.

comment:4 Changed at 2009-06-17T09:02:22Z by warner

Ok, foolscap-0.4.2 is out, and I've installed the new flappclient on all of our deb-producing machines except the feisty slave (I'm having ssh problems today). I've sent email to the admins of the non-allmydata deb-producing machines:

  • lenny-amd64: Eugen Leitl
  • hardy-amd64: Zandr (fieldcircus)
  • intrepid-amd64: Shawn Willden (might really be jaunty-amd64)

and I've set up the "flappserver" on the APT repository side.

Next step is to modify the buildmaster config to use flappclient upload-file to upload debs, at least on a few buildslaves, and see if it works.

comment:5 Changed at 2009-06-17T09:04:48Z by warner

Oh, another TODO item: replace the OS-X-10.5 xfer-server on hanford with this same flappserver. It's currently dropping Mac .dmg files into ~/public_html/dist/tahoe/mac-10.5/

comment:6 Changed at 2009-06-17T20:59:43Z by warner

Ok, we're now using flappclient to upload debs for all of our deb-producing machines (including feisty), and a quick test seems to work:

  • etch
  • edgy
  • feisty
  • gutsy
  • hardy

I'm waiting on responses from the non-AMD machines, with some folks saying they'll take care of it tonight:

  • lenny-amd64
  • hardy-amd64
  • jaunty-amd64

comment:7 Changed at 2009-06-20T01:37:00Z by warner

note to self, the ubuntu "watershed" package appears like it might handle the locking/idempotent-execution stuff I need to the APT-index-update step.

comment:8 Changed at 2009-06-25T16:00:23Z by zooko

  • Keywords debian packaging added

comment:9 Changed at 2009-08-06T18:42:31Z by zooko

  • Keywords changed from debian,packaging to debian packaging
  • Milestone changed from undecided to 1.5.1

comment:10 Changed at 2009-08-10T15:50:37Z by zooko

Brian posted an update on this issue to the list:


comment:11 Changed at 2009-10-27T01:06:54Z by zooko

Okay now all .deb uploader builders are using flappclient. I guess this ticket is still open because of idea of using "watershed" to lock for the apt-index-update?

comment:12 Changed at 2009-10-27T22:05:24Z by zooko

  • Milestone changed from 1.5.1 to eventually

comment:13 Changed at 2011-07-22T20:27:42Z by davidsarah

  • Milestone eventually deleted
  • Resolution set to fixed
  • Status changed from assigned to closed

This ticket appears to be long-obsolete. Brian: please open a new ticket about using watershed if that is still relevant.

Note: See TracTickets for help on using tickets.