setuptools delenda est

[davidsarah]

We need to stop using setuptools, for the following reasons:

• it frequently downloads, builds, installs, and/or runs the wrong code
• it frequently gives incorrect, misleading, or insufficient information about what it is doing
• it operates in a way that is incompatible with many OS packaging practices
• its behaviour when downloading dependencies is easily exploitable; I don't know of any way to use it securely
• its implementation is too complex to understand
• we have needed to maintain a fork in order to partially, and with limited success, mitigate these problems
• the bugs and design flaws that cause the above problems are not shallow, and it's unlikely that they're going to be fixed any time soon, because it is also poorly maintained.

Dealing with the effects of setuptools' problems on Tahoe-LAFS has inconvenienced users on many occasions and wasted a huge amount of core developer time. This ticket is to find, or to design and implement, an alternative.

[davidsarah]

See also ticket:1451#comment:4 .

[zooko]

I feel like this isn't as urgent as most bugs marked "Priority: Critical". If you disagree, then I apologize for overwriting the priority setting you left. Note that as the months and years have gone by, various other tools have been developed or improved; tools that we might be able to use, such as "Wheel" by Daniel Tolth, and "Paver". There are probably a few others that I haven't even heard of. Also pip and virtualenv have been ubiquitous and popular.

[davidsarah]

Major because of the sheer amount of pain setuptools has caused us. (pip doesn't solve most of those problems because it's just a frontend to setuptools, and the issues with setuptools are not primarily in limited to its easy_install frontend.)

[daira]

​peep is another tool that intends to address this problem. I couldn't get it to work on Tahoe; it downloaded the required sdists and then blew up with an exception from pip, which it depends on:

$ cp src/allmydata_tahoe.egg-info/requires.txt requirements.txt
$ peep install -r requirements.txt
[...]
  File "/usr/local/lib/python2.7/dist-packages/peep-0.2.1-py2.7.egg/EGG-INFO/scripts/peep", line 143, in hashes_of_requirements
    for req in reqs:  # InstallRequirements
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 1240, in parse_requirements
    skip_regex = options.skip_requirements_regex
AttributeError: 'NoneType' object has no attribute 'skip_requirements_regex'

If I understand correctly, pip by itself does not help you at all in verifying the integrity of dependencies; *at most* it will download the package you're directly installing over https.