[tahoe-dev] building and pre-existing dependencies

[Zooko O'Whielacronx]

On Thu, Jul 22, 2010 at 7:51 PM, Greg Troxel <gdt at ir.bbn.com> wrote:
>
> I am trying to package tahoe for pkgsrc, the native packaging system on
> NetBSD and Dragonfly, also portable to Darwin, Linux, Solaris, OpenBSD,
> FreeBSD, IRIX, AIX, and a few others.

Sweet! I would be happy to help.

> I don't understand how setup.py decides if a particular python package
> is already present or needs to be downloaded and built.

In theory, it looks for a file or directory named
$DISTRIBUTIONNAME-$VERSION-py$PYTHONVERSION.egg-info in any directory
which is on the Python sys.path. In practice there are a few known
bugs in different versions of different tools which might cause it to
do this re-download behavior that you observed even if the .egg-info
file is present.

But let's start by checking whether there is a proper .egg-info for
pycrypto present. It looks like to me that this "egg.mk" script might
be intended to help manage those:
http://www.netbsdpkgsrc.info/packages/lang/python.html

> but python finds it:
>
> gdt 519 /usr/pkgsrc/filesystems/tahoe-lafs > python2.6
> Python 2.6.5 (r265:79063, Jul 16 2010, 15:06:40)
> [GCC 4.1.3 20080704 prerelease (NetBSD nb2 20081120)] on netbsd5
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import Crypto
>>>>

An import thing to keep in mind is that "Crypto" is the name of a
Python module (which is sometimes confusingly called a Python
"package") i.e., something that can be imported. "pycrypto" is the
name of the Python package (which is always confusingly called a
Python "distribution") i.e. something that can be installed onto a
system and that another distribution (package) can depend on.

So, to test whether the Python "distribution" is installed under the
expected name and version number it doesn't help to import Crypto,
which only tells you whether the Python module is importable. To see
whether the Python "distribution" is installed you have to do this:

python -c 'import pkg_resources;print pkg_resources.require("pycrypto")'

Or do the equivalent of it, which is to search for a file or directory
named "pycrypto-$VER-py$PYVER.egg-info".

Now you could if you want patch the Tahoe-LAFS setup.py to not use
this mechanism at all and just rely on the pkgsrc dependency
mechanism. That would be okay, but it would be one more patch you'd
have to maintain, and other Python tools besides the Tahoe-LAFS
setup.py might likewise be depending on the .egg-info. So it might be
good to get to the bottom of why there isn't a
pycrypto-$VER-py$PYVER.egg-info on your sys.path.

Regards,

Zooko