[tahoe-dev] [Nix-dev] Re: Tahoe-LAFS in NixOS

[Zooko Wilcox-O'Hearn]

[Cc: nix-dev and tahoe-dev]

On Saturday, 2009-10-31, at 16:56 , Ludovic Courtès wrote:

>> I would like to add a NixOS buildslave to our buildbot farm:
>>
>> http://allmydata.org/buildbot
>
> Can you give a pointer to info on how to setup and run a buildslave?

Here is the instructions that I recently gave someone who wanted to  
set up an OpenBSD buildslave:

http://allmydata.org/pipermail/tahoe-dev/2009-December/003279.html

The OpenBSD user, kyle at arbyte.us, was able to set up a buildslave  
easily enough.  The only confusing part was that I had not yet  
updated the config on the buildmaster side to accept connections from  
his buildslaves, so he wrote to me asking why they couldn't connect.   
I then updated the master config and then his buildslaves connected.

> In addition, most of Tahoe’s direct dependencies are defined in [3].
>
> [3] https://svn.nixos.org/repos/nix/nixpkgs/trunk/pkgs/top-level/ 
> python-packages.nix

How was this file generated?  Since Tahoe-LAFS's dependencies are  
already declared in a machine-readable format [1], I think it would  
be cool if there were a generic tool to translate those dependencies  
from that Python format into NixOS format.  I contributed some  
patches to such a tool to translate from Python format to Debian  
format [2].

> The “doCheck = false” in [0] means that tests aren’t run.  I just  
> checked what prompted me to disable them; here’s the end of the log:
...
> The following error occurred while trying to add or remove files in  
> the
> installation directory:
>
>     [Errno 2] No such file or directory: 'support/lib/python2.5/ 
> site-packages/test-easy-install-15898.pth'
>
> The installation directory you specified (via --install-dir, -- 
> prefix, or
> the distutils default setting) was:
>
>     support/lib/python2.5/site-packages
>
> This directory does not currently exist.  Please create it and try  
> again, or
> choose a different installation directory (using the -d or -- 
> install-dir
> option).
...
> It seems that ‘setuptools’ is trying to write something in a place  
> it isn’t allowed to.  Note that “python setup.py test” here is run  
> before Tahoe-LAFS is installed.  Is that a problem?  Any other idea?

It is okay to run "python setup.py test" before anything else.  (It  
is configured to execute the build step before it does the test step,  
in the Makefile style -- see line 48 of setup.cfg [3]).

I don't think that error message above says that the build was  
attempting to write to a disallowed location.  The location it is  
trying to write to is a relative path below the base dir (i.e.  
without '..').  That is allowed by the NixOS build system right?

I think what you see there is an example of a bug in our buildsystem  
which manifests on some platforms.  I am going to be trying to fix  
that bug in our buildsystem and looking at the buildbot results [4]  
to see if my attempted fixes cause it to build and run tests properly  
on more platforms without causing a regression on any platforms.  The  
sooner NixOS is included in that list of platforms the sooner I will  
be able to make this part work correctly on NixOS.

> (Ironically, it may be that build failure is related to the fact  
> that NixOS builders follow POLA whereas ‘setuptools’ expects some  
> form of ambient authority.)

This part is super cool!  Do the NixOS docs explicitly cite POLA?   
What paper or doc should I read for the top-level justification for  
NixOS's design?  I would like to show this to the POLA theorists.  :-)

Thanks for fixing the project name and licensing on the NixOS web  
page.  :-)

Regards,

Zooko

[1] http://allmydata.org/trac/tahoe/browser/_auto_deps.py
[2] http://github.com/astraw/stdeb
[3] http://allmydata.org/trac/tahoe/browser/setup.cfg?rev=3996#L48
[4] http://allmydata.org/buildbot/