[tahoe-lafs-trac-stream] [Tahoe-LAFS] #2077: pip packaging plan

[Tahoe-LAFS]

#2077: pip packaging plan
---------------------------+--------------------------------------
     Reporter:  daira      |      Owner:
         Type:  defect     |     Status:  new
     Priority:  normal     |  Milestone:  undecided
    Component:  packaging  |    Version:  1.10.0
   Resolution:             |   Keywords:  pip packaging setuptools
Launchpad Bug:             |
---------------------------+--------------------------------------

Comment (by glyph):

 Replying to [comment:19 daira]:

 > 4. Forking zetuptoolz was probably a mistake, and failing to keep up
 with changes to upstream for this long certainly was. You don't need to
 reiterate that; we get it.

 I was not trying to hammer home that it was a mistake by reiterating it,
 rather, it seemed as though there are feelings about the continued
 necessity of the status quo despite its unfortunate nature, and I was
 trying to dispel them.  I get that nobody is happy with the current state
 of that fork.

 > 5. I've had a migraine today; actually this is the third time I've had a
 bad migraine after spending the previous night thinking about
 setuptools/pkg_resources. So forgive me if I sound a bit snappy or
 irritated.

 Somebody put this on a T-shirt.  "Python Packaging: So Awful It Will
 Physically Damage Your Brain!" :-).  In all seriousness, I'm sorry to
 provoke this stress.  Please keep in mind that we are just talking about
 how to make our way to a brighter future, and none of this needs
 addressing immediately; as I said, if you need to ignore these messages
 then please do so, I don't mind if it takes you a couple of months at a
 time to get back to me.  If you want me to stop replying and insert the
 delay on my end, please let me know.

 > 7. Several issues are non-negotiable:
 >
 > * We need "download some file, extract it, run `python setup.py build`"
 to Just Work, given that Python is already installed. It must not rely on
 the user performing extra manual steps, ''especially'' not steps that vary
 by platform. (In particular Windows must work like any other platform.)

 I guess this is the main requirement I don't quite understand.  Why is
 manually unpacking a source tarball such an important use-case?  It's not
 like `wget` and `tar xvzf` are substantially less obscure than `pip`.
 Also: why is it so important that this file be called `setup.py`?  The
 general trend in the Python community is to unbundle these
 responsibilities as much as possible, and to have `setup.py` serve as a
 metadata calculator for distutils and setuptools, and other, external
 build scripts live elsewhere.

 > * The file that the user downloads must contain only source; no binary
 blobs. It's acceptable for precompiled binaries of dependencies to be
 downloaded from other sites over https. Ideally this download would be
 separable from build or install, although `python setup.py build` should
 download dependencies by default. (There may be a different download that
 includes all dependencies, but that is a "nice to have", not a required
 feature.)

 In my mind, there are three audiences here.

  1. There are developers interested in hacking on Tahoe itself, which
 ought to be retrieved from source control, and then use `pip install -e`
 into a virtualenv.
  2. There are developers interested in building an application that builds
 upon Tahoe, who ought to retrieve Tahoe from PyPI with `pip install
 allmydata-tahoe`, and ought to be able to tolerate binary blobs for their
 platform of choice.
  3. There are end-users who ought to be retrieving a fully integrated
 binary artifact (and presently that installer is in a questionable state,
 so some users may practically speaking want to stick with the `pip`
 workflow until it can be improved).

 Who is the audience for a direct tarball download?

 > 8. We don't trust `pkg_resources` to correctly set up a `sys.path` that
 will result in importing the versions of dependencies asked for in
 `setup_requires` (there are several known cases where it doesn't, and
 these are ''not'' only associated with the use of eggs, nor are they
 easily fixable given the variety of ways in which dependent packages may
 have been installed on a user's system). To work around this problem, we
 explicitly check the versions that were actually imported. We do this in a
 way that does not involve duplicating information about version
 requirements in multiple places, and we would not want to lose that
 property.

 Oh, are you literally talking about `setup_requires`, and not
 `install_requires`?  That feature seems to be a lost cause :-\.  What does
 Tahoe currently need `setup_requires` for?

--
Ticket URL: <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2077#comment:21>
Tahoe-LAFS <https://Tahoe-LAFS.org>
secure decentralized storage