[tahoe-lafs-trac-stream] [Tahoe-LAFS] #2077: pip packaging plan

[Tahoe-LAFS]

#2077: pip packaging plan
---------------------------+--------------------------------------
     Reporter:  daira      |      Owner:
         Type:  defect     |     Status:  new
     Priority:  normal     |  Milestone:  undecided
    Component:  packaging  |    Version:  1.10.0
   Resolution:             |   Keywords:  pip packaging setuptools
Launchpad Bug:             |
---------------------------+--------------------------------------

Comment (by daira):

 1. The
 `Requirement.parse('zope.interface==3.6.0,==3.6.1,==3.6.2,>=3.6.5')` error
 is already fixed on trunk. (Note that this was due to an incompatible,
 undocumented change in setuptools and we would have had this problem
 anyway if we had been using upstream setuptools or pip rather than
 zetuptoolz. In any case it is fixed, so please do not get distracted by
 it.)

 2. A couple of other difficulties with using pip have just been fixed on a
 branch, and will be in the Tahoe-LAFS v1.10.1 release.

 3. This ticket was filed some time ago, and I know more about pip and its
 relation with virtualenv and setuptools than I did then.

 4. Forking zetuptoolz was probably a mistake, and failing to keep up with
 changes to upstream for this long certainly was. You don't need to
 reiterate that; we get it.

 5. I've had a migraine today; actually this is the third time I've had a
 bad migraine after spending the previous night thinking about
 setuptools/pkg_resources. So forgive me if I sound a bit snappy or
 irritated.

 6. Tahoe-LAFS in fact does have an OS X installer, which will be in
 1.10.1. It may also have a Windows installer if that is finished in time.
 However you're right that it is independent of this ticket.

 7. Several issues are non-negotiable:

 * We need "download some file, extract it, run `python setup.py build`" to
 Just Work, given that Python is already installed. It must not rely on the
 user performing extra manual steps, ''especially'' not steps that vary by
 platform. (In particular Windows must work like any other platform.)

 * The file that the user downloads must contain only source; no binary
 blobs. It's acceptable for precompiled binaries of dependencies to be
 downloaded from other sites over https. Ideally this download would be
 separable from build or install, although `python setup.py build` should
 download dependencies by default. (There may be a different download that
 includes all dependencies, but that is a "nice to have", not a required
 feature.)

 8. We don't trust `pkg_resources` to correctly set up a `sys.path` that
 will result in importing the versions of dependencies asked for in
 `setup_requires` (there are several known cases where it doesn't, and
 these are ''not'' only associated with the use of eggs, nor are they
 easily fixable given the variety of ways in which dependent packages may
 have been installed on a user's system). To work around this problem, we
 explicitly check the versions that were actually imported. We do this in a
 way that does not involve duplicating information about version
 requirements in multiple places, and we would not want to lose that
 property.

--
Ticket URL: <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2077#comment:19>
Tahoe-LAFS <https://Tahoe-LAFS.org>
secure decentralized storage