'pip install allmydata-tahoe' now works

[Donald Stufft]

Leif Ryge <leif at ...> writes:

> 
> Unfortunately (unless I'm missing something; I haven't investigated fully) the
> statement "'pip install allmydata-tahoe' now works" is rather dangerously
> misleading as it implies that that is a safe command to run on an
> internet-connected computer.
> 
> Recent versions of pip verify SSL certificates and won't download over
> unencrypted HTTP unless you specifically tell it to. But, unless I'm mistaken,
> "pip install allmydata-tahoe" will still run tahoe's "setup.py build" which
> will brazenly download and execute unverified code.
> 
> If I am mistaken (and I hope I am!) someone should close
> https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2055 ("Building tahoe safely is
> non-trivial").
> 
> ~leif
> 
> On Mon, Jun 30, 2014 at 06:58:30AM -0700, Callme Whatiwant wrote:
> > Huzzah!
> > 
> > On Mon, Jun 23, 2014 at 12:47 PM, Brian Warner <warner at ...> wrote:
> > > Just a heads up, the new Nevow-0.11.1 release a few days ago fixed
> > > tahoe's #2032, which means that you should now be able to install tahoe
> > > with just:
> > >
> > >  pip install allmydata-tahoe
> > >
> > > That should grab all the necessary dependencies for you, including
Twisted.
> > >
> > > Hooray for easier installations!
> > >
> > > cheers,
> > >  -Brian
> > >
> > > #2032: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2032
> > > _______________________________________________
> > > tahoe-dev mailing list
> > > tahoe-dev at ...
> > > https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
> > _______________________________________________
> > tahoe-dev mailing list
> > tahoe-dev at ...
> > https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
> 
> 
> _______________________________________________
> tahoe-dev mailing list
> tahoe-dev at ...
> https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
> 

The answer to the question of "Is ``pip install <something>`` safe?" is "It
Depends".

It depends on what version of pip you're using. pip <1.3 did not verify TLS,
pip < 1.4 did not provide any method to disable spidering external (and
unsafe) links, pip < 1.5 would spider external (and unsafe) links by default.

So assuming you're using pip 1.5+ then the files that pip itself downloads
are protected by TLS (Typically using the Mozilla CA bundle, but some OSs
debundle that and add their own).

It also depends on what setuptools features people are using. pip disables
setuptools from fetching install_requires and extras_requires. However there
is currently no way for pip to tell setuptools *not* to do that. So it
depends on if any of the packages in the dependency set use setup_requires.

So assuming nothing uses setupt_requires, then only pip will be downloading
files, and you're back to the first "It depends".

It also depends on what version of setuptools people are using. If a project
*does* use setup_requires then those are downloaded by setuptools not by
pip. setuptools <0.7 did not verify TLS, and setuptools will spider external
(and unsafe) links by default. It does offer a feature that lets the end
user restrict what hostnames setuptools will connect to, but by default it
will allow all.

So assuming there is a setup_requires, and someone is using 0.7+ setuptools,
then it's "safe" depending on what URLs are linked from the
https://pypi.python.org/simple/<foo>/ page for anything that is specified in
a setup_requires. Authors now have control over what shows up there but it
may or not not be a "safe" download. [1]

It also depends on what you're doing in a setup.py. At this level there's
nothing pip/setuptools can really do except maybe block network access in a
setup.py. Obviously though, since a setup.py is regular old Python, you can
do anything you want in it.

Relating this all back to allmydata-tahoe, I know that there is stuff in
there that uses setup_requires. So the answer for allmydata-tahoe is "It
depends on pip version, setuptools version, and what all is in
setup_requires". The only thing I'm aware of that is in setup_requires is
cffi which is safe to install via setup_requires. Assuming that's the only
thing then it's just "Depending on pip and setuptools version".

One more thing, if someone is using pip >= 1.5, then it's "safe" for any
thing that has a .whl file on PyPI regardless of what they do or use in
their setup.py. (Also faster and all around better).

Obviously my definition of "safe" here is "Did I download the thing that the
PyPI server meant to send me" and it does not apply to "Is the PyPI server
evil?", "Are the PyPI Administrators Evil?", or "Are the Package Authors Evil?".

[1] I have some ideas on stealing control over this so that pip handles
downloading setup_requires, but it's not done yet.