[tahoe-dev] problems with the build/install/test process

[zooko]

Folks:

Chris Galvan contributed some patches to our buildbot config to make  
each buildslave test what happens when you do commands like:

python setup.py build_tahoe
python setup.py trial
./bin/tahoe --version
python setup.py bdist_egg
mkdir installdir && easy_install -d installdir dist/*.egg
PATH=$PWD/support/bin:$PATH && PYTHONPATH=$PWD/installdir:$PWD/ 
installdir/support/lib/python2.5/site-packages && cd installdir &&  
trial allmydata

It's great to have these tested -- thanks, Chris! -- but currently  
our setup.py fails to do the right thing in many of these cases.

Here are my notes as I look at our numerous failing builders.

First: cygwin

http://allmydata.org/buildbot/builders/cygwin/builds/1660/steps/ 
installtest/logs/stdio

This failure is because setup.py doesn't pass "--reactor=poll"  
whenever starting twisted, including when starting trial.  I guess  
that the best way to solve this is to add a "reactor" parameter to  
the setuptools_trial plugin, but I'm not sure the best way to do  
that, so I think I'll just hack the sys.argv to append "-- 
reactor=poll" until Chris tells me a better way to do it.  :-)

There, done in the kludgey way in patches [3421] and [3422].


Next: edgy, etch,

http://allmydata.org/buildbot/builders/edgy/builds/1874/steps/ 
installtest/logs/stdio

This is the problem of what to do when a dependency is installed, and  
Tahoe requires a newer version of that dependency.  Ideally, I would  
like for the tahoe build process to install the newer version next to  
the older already-installed version, and arrange to use the newer  
version at runtime.  Setuptools comes with this functionality, but I  
don't know how to use that functionality when doing a plugin like  
"trial" -- I know how to do it only when installing with easy_install  
or when running "setup.py develop".  It is called "--multi-version",  
and is the subject of ticket #530 (use setuptools's --multi-version  
mode).


Next: hardy

http://allmydata.org/buildbot/builders/hardy/builds/480/steps/test- 
egg/logs/stdio

Hm, this points to a deeper problem: the buildbot's attempt to run  
"trial allmydata" to test the current source code from its newly  
installed location might accidentally run the unit tests of a  
different version of allmydata installed in the system.  This could  
be related to #145 ("make test" tests the installed version of  
allmydata, not the local sandbox version of allmydata).


Next:

Next: Mac OS X

http://allmydata.org/buildbot/builders/Mac%20OS-X/builds/806/steps/ 
mac_exe/logs/stdio

Hm.  The building of Macintosh packages is currently unsupported, but  
the changes we've made shouldn't have broken the tests that were  
previously passing.

By the way, does anybody want to take over the code to automatically  
build Macintosh packages of Tahoe?  There are many people who have  
expressed desire for such packages.


Regards,

Zooko

patches mentioned in this mail:

http://allmydata.org/trac/tahoe/changeset/3421
http://allmydata.org/trac/tahoe/changeset/3422

tickets mentioned in this mail:

http://allmydata.org/trac/tahoe/ticket/530 # use setuptools's --multi- 
version mode
http://allmydata.org/trac/tahoe/ticket/145 # "make test" tests the  
installed version of allmydata, not the local sandbox version of  
allmydata