[tahoe-dev] update on the buildbots

[Dan McNair]

On Sun, 12 Apr 2009, zooko wrote:

> One of them is Dan's ArchLinux buildbot, which is still languishing in the 
> Unsupported category because it used to mysteriously fail and then it 
> mysteriously started succeeding.
>
> On Mar 21, 2009, at 20:35 PM, Dan McNair wrote:
>
>> > What?  When did Dan's ArchLinux buildbot start succeeding?  Let's see, it 
>> > was still failing with the old mysterious deep recursion in 
>> > zope.interface at build #276:
>> > http://allmydata.org/buildbot/builders/Dan%20ArchLinux/builds/276
>> > but it has been consistently succeeding ever since, starting with build 
>> > #277:
>> > http://allmydata.org/buildbot/builders/Dan%20ArchLinux/builds/277
>> > The changes that Brian committed which triggered build #277 don't seem to 
>> > have anything to do with this issue.
>> > Okay, Dan, fess up.  What did you do?
>> 
>> Um ... I played with a few things ... looked at the source ... tried to 
>> grok what zope.interface was doing ...
>> 
>> Seriously, the only "change" is that I did a "setup.py install" in the 
>> buildbot workdir. Wait, no; I built an egg, dropped it in /tmp, and ran 
>> easy_install on that as a different user (one with sufficient permission; 
>> of course buildbot can't install software in the root).
>
> A tahoe .egg?
>
> I've noticed a clue.  See the difference between these two steps:
>
> http: //allmydata.org/buildbot/builders/Dan%20ArchLinux/builds/276/steps/show-tool-versions/logs/stdio
> http: //allmydata.org/buildbot/builders/Dan%20ArchLinux/builds/277/steps/show-tool-versions/logs/stdio
>
> That warning message is due to divmod issue #2830.
>
> So it seems like by step 277, when you installed Tahoe with easy_install then 
> this also must have installed Nevow as an egg where back in 276 it was 
> installed in some other way.  Does that make sense?

Before build 277 I wasn't installing tahoe; the buildbot was loading up 
all the deps (from a prior -sumo tarball) into its workdir and running 
tests using those packages.

I haven't touched Arch's package management for any of this. When I 
easy_install'd the tahoe .egg I just let easy_install find, download, and 
install all the dependencies. So whatever setuptools magic gets tools onto 
my machine was what was exercised here.

I think it's the same magic that was being used to resolve deps for the 
buildbot, except that we short-circuited the whole thing to avoid 
transient resolver issues that my router has and put in a tahoe-deps tree 
so that the buildbot didn't need to download deps. Which also meant its 
deps may have been a little out of date... and sure enough, a quick check 
shows that the buildbot workdir has Nevow-0.9.31 in its tahoe-deps dir, 
while my installed "tahoe --version" says Nevow-0.9.32.

> Do you think you could remove the current Nevow and then try re-installing 
> Nevow using the ArchLinux installation process, to see if that makes the 
> mysterious deep recursion in zope.interface recur?  What is the ArchLinux 
> install process for Python packages ("distributions"), anyway?

ArchLinux doesn't have nevow in its package management, so the current 
easiest best way I can see to install Nevow is exactly what I did, with 
easy_install. Is there a way I can tell the buildbot to use _only_ the 
versions in its tahoe-deps dir instead of the system installed versions? 
That would accomplish the reversion you desire; I don't feel like 
reverting my system, because I use tahoe on a daily basis, and want it to 
keep working.

Thanks! And sorry about taking so long to get back to you on this...