[tahoe-dev] Karmic packaging updates

[Zooko Wilcox-O'Hearn]

On Saturday,2009-08-22, at 19:39 , Brian Warner wrote:

> Just a quick note about my debian packaging efforts before I run  
> off to dinner.

Yay!  Thank you!

> The packaging process got a lot easier after I ripped out most of  
> the custom stuff in our setup.py files, and all of the aliases in  
> setup.cfg .

Could you be more specific?  What went wrong before you ripped out  
most of the custom stuff in our setup.py files?  Is it a bug in our  
custom setup.py stuff or should we just treat Debian packaging as a  
sufficiently different use case that the setup.py shouldn't be  
expected to handle it?

> The debian diff includes a substantial patch to those two files.  
> I'd like to see how we might reduce our customizations in the trunk  
> too: in particular I'd like to make "setup.py build" go back to  
> normal and document some other command
> (perhaps "setup.py build_tahoe"?) that users should run after they  
> unpack the source tree and before they invoke ./bin/tahoe for the  
> first time.

I'm not sure exactly what the issues you experienced were, but I've  
seen problems in the past due to our desire to have a command,  
currently "python ./setup.py build", which makes a "bin/tahoe"  
executable that can "run from source" without being installed and  
without having to re-run build after every change to the source.   
"Normal", for Python packages, doesn't currently include those two  
features.  Instead you are expected to install the software before  
you run it, and you are expected to reinstall it every time you  
change it before you can run the new version.  (The setuptools  
"develop" command is one attempt to remove that latter restriction.)

I'm somewhat inclined to redefine "normal" by getting these sorts of  
features we want supported by future versions of the standard Python  
packaging tools.  ;-)

But anyway I would consider losing some of these features in order to  
gain some simplicity and maybe spend less of my time on maintaining  
the build system in the future.  On the other hand, maybe it is okay  
if the Debian diff contains a big patch that simplifies and re- 
arranges the build system just for the purpose of building .debs.   
Does that cause any harm?

One thing that gives me hope that I won't have to spend so much time  
on packaging in the future is that we recently added automated tests  
of various packaging desiderata.  Now we can use the test-driven,  
buildbot-powered, no-regressions development style on the build  
system just like we do on the Tahoe-LAFS code itself.  I guess that  
means that if you want to make any changes to the build system, you  
first have to add a test to the test suite that the current build  
system fails.  :-)

Here are the building/packaging tests we currently have:

1.  That "python ./setup.py build && python ./setup.py test" passes  
even if you don't have all the dependencies installed.  This gets  
tested for Tahoe-LAFS, zfec, and pycryptopp on various flavors of  
Linux, Mac, Windows, and Solaris (i.e. all of our Supported  
Platforms), and they all pass.

2.  That you can build a .egg with "python ./setup.py bdist_egg" and  
then install it with "easy_install $EGG" and this results in a  
working install.  This gets tested for Tahoe-LAFS, zfec, and  
pycryptopp on all of the aforementioned platforms and they all pass.

3.  That you can build a .deb using stdeb.  This gets tested for  
pycryptopp and zfec on various flavors of Linux and they all pass.

4.  That you can build a Macintosh app and a .dmg containing that  
app.  Tthis gets tested for Tahoe-LAFS on Mac.  This test is in the  
TODO state -- the test correctly determines that the build system  
fails to build a working Mac app.

5.  That if you build and then run "make clean", that it doesn't  
leave cruft behind.  This gets tested fro Tahoe-LAFS on one Linux  
builder.  It passes.

Regards,

Zooko