[tahoe-lafs-trac-stream] [tahoe-lafs] #1450: setuptools downloads and builds a correct version of a dependency in the install-to-egg step, but then adds a different version not satisfying the requirement to easy_install.pth

[tahoe-lafs]

#1450: setuptools downloads and builds a correct version of a dependency in the
install-to-egg step, but then adds a different version not satisfying the
requirement to easy_install.pth
----------------------------+------------------------------
     Reporter:  davidsarah  |      Owner:  somebody
         Type:  defect      |     Status:  new
     Priority:  major       |  Milestone:  undecided
    Component:  packaging   |    Version:  1.8.2
   Resolution:              |   Keywords:  setuptools build
Launchpad Bug:              |
----------------------------+------------------------------

Old description:

> In the attached install-to-egg build on the 'Eugen lenny' buildslave, the
> requirement for Twisted is >= 10.1.0.
>
> setuptools downloads and builds Twisted 10.2:
> {{{
> Searching for Twisted>=10.1.0
> Best match: Twisted 10.2.0
> Downloading http://tahoe-lafs.org/source/tahoe-lafs/deps/tahoe-dep-
> sdists/Twisted-10.2.0.tar.bz2
> Processing Twisted-10.2.0.tar.bz2
> Running Twisted-10.2.0/setup.py -q bdist_egg --dist-dir /tmp
> /easy_install-QTimkw/Twisted-10.2.0/egg-dist-tmp-yBXrHn
> [...]
> }}}
>
> but then it adds Twisted 8.1 to easy-install.pth, resulting in a failed
> Tahoe build:
> {{{
> Adding Twisted 8.1.0 to easy-install.pth file
> [...]
> Installed /home/tahoebuildslave/tahoebuildslave/Eugen lenny-
> amd64/build/egginstalldir/Twisted-8.1.0-py2.5-linux-x86_64.egg
> error: Could not find required distribution Twisted>=10.1.0
> }}}
>
> Since we're installing to an egg rather than globally, there's no good
> reason why that shouldn't work even if Twisted 8.1 is installed single-
> version-externally-managed. But if that's not supported for some reason,
> it's still incorrect to build 10.2 and then fail with a misleading error
> message.

New description:

 In the attached install-to-egg build on the 'Eugen lenny' buildslave, the
 requirement for Twisted is >= 10.1.0.

 setuptools downloads and builds Twisted 10.2:
 {{{
 Searching for Twisted>=10.1.0
 Best match: Twisted 10.2.0
 Downloading http://tahoe-lafs.org/source/tahoe-lafs/deps/tahoe-dep-
 sdists/Twisted-10.2.0.tar.bz2
 Processing Twisted-10.2.0.tar.bz2
 Running Twisted-10.2.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-
 QTimkw/Twisted-10.2.0/egg-dist-tmp-yBXrHn
 [...]
 }}}

 but then it adds Twisted 8.1 to easy-install.pth, resulting in a failed
 Tahoe build:
 {{{
 Adding Twisted 8.1.0 to easy-install.pth file
 [...]
 Installed /home/tahoebuildslave/tahoebuildslave/Eugen lenny-
 amd64/build/egginstalldir/Twisted-8.1.0-py2.5-linux-x86_64.egg
 error: Could not find required distribution Twisted>=10.1.0
 }}}

 Since we're installing to an egg rather than globally, there's no good
 reason why that shouldn't work even if Twisted 8.1 is installed single-
 version-externally-managed. But if that's not supported for some reason,
 it's still incorrect to build 10.2 and then fail with a misleading error
 message.

--

Comment (by daira):

 Replying to [comment:3 zooko]:
 > I don't know why {{{/usr/lib/python2.5/site-packages}}} is earlier than
 the Twisted egg. Setuptools normally puts all .egg's earlier on the
 {{{sys.path}}} earlier than all non-egg entries (which is the cause of the
 most widely hated setuptools issue:
 [http://bugs.python.org/setuptools/issue53 setuptools issue #53 -- respect
 the PYTHONPATH]).

 setuptools does not always put .egg entries earlier than non-egg entries.
 If a non-egg is needed to satisfy a dependency and it is in a shared
 package directory (such as a {{{site-packages}}} directory, either the
 global one or one in a virtualenv), then other packages from that
 directory can end up being found before the ones required by Tahoe, even
 when setuptools thinks theose requirements have been correctly satisfied
 (see ticket:1246#comment:6). This behaviour is nondeterministic.

-- 
Ticket URL: <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1450#comment:4>
tahoe-lafs <https://tahoe-lafs.org>
secure decentralized storage