Nevow doesn't declare its dependency on Twisted in a machine-readable way.

[zooko]

So Adam Langley's problem building Tahoe on Ubuntu turned out to be completely reproducible. Any time you run make in a Tahoe source tree on a system where neither Twisted nor Nevow is installed, setuptools will first install Twisted into a directory named Twisted-$VERSIONNUMBER.egg in the current working directory, because our setup.py​ told setuptools (in _auto_deps.py) that we need to have Twisted installed at build time.

We told it that for two reasons: 1. We require twisted to run tests, which is a build-time activity, and 2. We were attempting to work-around the fact that Nevow doesn't declare its dependency on Twisted in a machine-readable way, so that setuptools doesn't know whether to install Twisted first or Nevow first, and if it picks Nevow then the Nevow installation fails at install time when it can't import twisted modules.

However, it turns out that on at least Ubuntu Hardy (Adam's machine) and on my Macbook Pro (Mac OS 10.4), having Twisted listed as setup_requires and having Nevow being listed as install_requires leads to this bizarre error message which takes the name of PIL in vain:

 File "/home/agl/src/allmydata-tahoe-1.0.0-r2613/setuptools-0.6c8.egg/setuptools/sandbox.py",
line 27, in run_setup
 File "/home/agl/src/allmydata-tahoe-1.0.0-r2613/setuptools-0.6c8.egg/setuptools/sandbox.py",
line 63, in run
 File "/home/agl/src/allmydata-tahoe-1.0.0-r2613/setuptools-0.6c8.egg/setuptools/sandbox.py",
line 29, in <lambda>
 File "setup.py", line 8, in <module>
   #
 File "/tmp/easy_install-fqJkNC/Nevow-0.9.18/setupcommon.py", line 2,
in <module>
 File "/usr/lib/python2.5/site-packages/PIL/__init__.py", line 10, in <module>
   #
ImportError: No module named components

Here is a minimal Python project that you can use to test this yourself. Simply put the following two lines into a file named setup.py:

import setuptools
setuptools.setup(name="minproj_that_needs_nevow", setup_requires=["Twisted"], install_requires=["Nevow"]

And then execute python ./setup.py install. If your platform is like Adam's and mine, then you'll get this bizarre traceback that takes the name of PIL in vain.

Here is another minimal project's setup.py file:

import setuptools
setuptools.setup(name="minproj_that_needs_nevow", install_requires=["Nevow"]

If you try to install this project, you will get a nice ImportError from Nevow attempting to import twisted while it (Nevow) is being installed.

Now, one way that this could all be made to work is if the Nevow setup.py declared that it required Twisted. Twisted itself declares that it requires zope.interface, like this:

​http://twistedmatrix.com/trac/browser/trunk/setup.py?rev=23010#L75

The Nevow (and Twisted) maintainers have an admirable policy of not applying patches to code that isn't automatically tested. Therefore in order to get this changed in Nevow's setup.py it is first necessary to have automated testing of the process of installing Nevow. That is the topic of

​http://divmod.org/trac/ticket/2630 # installation of Nevow doesn't have automated tests

which I intend to help dash with as much as I can as soon as possible.

[zooko]

Okay, there is now an automated test of Nevow installation using setuptools (​http://divmod.org/trac/ticket/2630 # installation of Nevow doesn't have automated tests ), so hopefully this opens the way for improvements to the setuptools build/packaging/metadata...

[cgalvan]

Hey zooko, since ​http://bugs.python.org/setuptools/issue20 has been resolved, should this be closed as well? It seems like this was caused by that problem as well as #455.

[zooko]

Yes, the fix in ​http://bugs.python.org/setuptools/issue20 does make this problem stop happening for Tahoe. That's because setuptools gets lucky and accidentally installs Twisted before installing Nevow. So while I still intend to get ​http://divmod.org/trac/ticket/2629 (Nevow doesn't declare its dependency on Twisted in a machine-parseable way) fixes, we can in the meantime close this ticket as the issue is no longer negatively effecting Tahoe. In other words, the problem Tahoe had was really caused by #455 (setuptools sandbox isn't tight enough to hold Twisted's sand in so that it doesn't get in Nevow's eyes) and not by Nevow not declaring its dependency on Twisted in a machine-readable way. So I'm going to close this ticket as "Somebody Else's Problem".

[zooko]

This was wontfixed for tahoe-1.3.0.