setup.py: which "Twisted" distribution name should we depend on?

[warner]

We've learned a bit today about the overzealous downloading/installation of Twisted by our setup.py on platforms that supposedly already have it installed.

The entry in our setup.py (really _auto_deps.py) stanza that declares which other packages we depend upon refers to a setuptools "Distribution" (which is equivalent to a debian package). These distribution names are not case sensitive, and one can use either hyphens or underscores to separate the words. For example, "twisted-core", "Twisted-Core", and "Twisted_Core" are all equivalent.

When Twisted-8.0.1 is built by running 'python setup.py install' on the top-level setup.py, it produces a single .egg directory (i.e. a single "distribution", named "Twisted".

When Twisted-8.0.1 is built by running 'setup.py install' on each individual subproject's setup.py, it produces a separate .egg directory for each subproject: Twisted-Core, Twisted-Web, etc. This is how debian/sid current does it. In sid, the 'python-twisted' debian package is a metapackage that depends upon 'python-twisted-core', 'python-twisted-web', etc. It does not contain any .egg directories. So if you have debian sid, you have the "Twisted-Core" distribution, but not the "Twisted" distribution. The most recent debian release ("etch") has twisted-2.4.0, but no .egg files.

Ubuntu feisty and gutsy have Twisted-2.5.0, and have just the single "Twisted" distribution. Hardy is likely to be the same. Dapper has Twisted-2.2.0, but does not have any .egg files.

The extra download/installs we've been seeing are because the tahoe setup.py declares a dependency upon "Twisted", whereas the debian packages installed on the system provide "Twisted-Core". The setuptools dependency language doesn't provide for boolean-or, so we can't declare a dependency upon one or the other.

So we need to decide what to do in the Tahoe setup.py:

• depend upon "Twisted": builds on Feisty, Gutsy, and Hardy will not download anything extra, and will use the system Twisted instead. Builds on debian etch and sid will download Twisted-8.0.1 and install it into support/lib .
• depend upon "Twisted-Core": builds on debian sid will not download anything extra. Builds on debian etch, and on ubuntu feisty/gutsy/hardy will download and install Twisted-8.0.1 .
• modify our dependencies based upon whether pkg_resources.require("Twisted") succeeds or not. We could choose one of the two options above as our default, but then say that a system that provides "Twisted" will depend upon "Twisted" rather than "Twisted-Core", or vice-versa. This would avoid extra downloads on debian sid and ubuntu feisty/gutsy/hardy (i.e. all platforms on which the system Twisted has .egg_info data). However it would raise the question of what dependencies should wind up in the record on pypi.

I think Foolscap will need to make the same decision.