[tahoe-lafs-trac-stream] [tahoe-lafs] #2055: Building tahoe safely is non-trivial

[tahoe-lafs]

#2055: Building tahoe safely is non-trivial
-----------------------------------+---------------------------
 Reporter:  leif                   |          Owner:  daira
     Type:  defect                 |         Status:  new
 Priority:  normal                 |      Milestone:  undecided
Component:  unknown                |        Version:  1.10.0
 Keywords:  install security eggs  |  Launchpad Bug:
-----------------------------------+---------------------------
 Summary: to safely build Tahoe on an untrustworthy (read: any) network it
 currently seems necessary to take an unintuitive step such as setting up a
 restrictive firewall or simply disconnecting from the internet in order to
 prevent {{{setup.py}}} from downloading and running arbitrary code via
 http.

 In this ticket I describe the two approaches I've tried: virtualenv v1.9.1
 (w/ pip v1.3), and the "Desert Island" build. If appropriate precautions
 are taken, both methods can yield what I believe are relatively "safe"
 builds (that is to say, they at least use HTTPS (and require CA-signed
 certificates) to ensure the integrity of the downloaded dependencies).

 The former requires blocking pip's port 80 connections and the latter
 requires disconnecting from the internet during the build.

 == virtualenv+pip ==

 Ideally, {{{pip install allmydata-tahoe}}} would be an easy and safe
 command to run!

 Version 1.3 of pip finally added certificate verification when making
 https connections, but when installing allmydata-tahoe v1.10 it still
 attempts to fetch foolscap and pycrypto via HTTP first. If that fails,
 perhaps because you've configured a firewall to not allow port 80
 connections, it will fall back to downloading them from PyPI via HTTPS.

 ^Note that using virtualenv 1.9 and pip 1.3, {{{pip install allmydata-
 tahoe}}} fails unless {{{pip install twisted}}} is run first. This might
 be because the former installs Twisted 11.0 while the latter installs
 Twisted 13.0.^

 == The "Desert Island" Build ==

 On the AdvancedInstall wiki page there are instructions for a "Desert
 Island" build, which consists of downloading and extracting https://tahoe-
 lafs.org/source/tahoe-lafs/deps/tahoe-deps.tar.gz in the tahoe-lafs source
 directory and running "{{{python setup.py build}}}".

 While this does work fine without an internet connection, it still tries
 repeatedly to connect to the internet. These are the lines of "{{{python
 setup.py build}}}" output which contain "Reading http":
 {{{
 Reading http://pypi.python.org/simple/zope.interface/
 Reading http://pypi.python.org/simple/
 Reading https://tahoe-lafs.org/source/tahoe-lafs/deps/tahoe-lafs-dep-
 sdists/
 Reading https://tahoe-lafs.org/source/tahoe-lafs/deps/tahoe-lafs-dep-eggs/
 Reading http://pypi.python.org/simple/mock/
 Reading http://pypi.python.org/simple/
 Reading http://pypi.python.org/simple/pyasn1/
 Reading http://pypi.python.org/simple/pycrypto/
 Reading http://pypi.python.org/simple/Nevow/
 Reading http://pypi.python.org/simple/pyOpenSSL/
 Reading http://pypi.python.org/simple/foolscap/
 Reading http://pypi.python.org/simple/simplejson/
 Reading http://pypi.python.org/simple/zfec/
 Reading http://pypi.python.org/simple/pyutil/
 Reading http://pypi.python.org/simple/zbase32/
 }}}

 Here is the context around one of them on my offline system (the others
 are similar):
 {{{
 Reading http://pypi.python.org/simple/foolscap/
 Download error: [Errno -2] Name or service not known -- Some packages may
 not be found!
 Couldn't retrieve index page for 'foolscap'
 Best match: foolscap 0.6.4
 Processing foolscap-0.6.4.tar.gz
 Running foolscap-0.6.4/setup.py -q bdist_egg --dist-dir /tmp/easy_install-
 vIEtM6/foolscap-0.6.4/egg-dist-tmp-Ipvbv_
 zip_safe flag not set; analyzing archive contents...
 foolscap.test.test_appserver: module references __file__
 Adding foolscap 0.6.4 to easy-install.pth file
 Installing flappserver script to support/bin
 Installing flappclient script to support/bin
 Installing flogtool script to support/bin

 Installed /fake-path-to-my-source-checkout/tahoe-
 lafs/support/lib/python2.7/site-packages/foolscap-0.6.4-py2.7.egg
 }}}

 I'm assuming (but have not confirmed) from the "Best match" part of this
 output that if any of these attempted requests were successful and the
 response indicated that there is a newer version of one of the
 dependencies than the corresponding egg in tahoe-deps, it would actually
 download and execute that code.

-- 
Ticket URL: <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2055>
tahoe-lafs <https://tahoe-lafs.org>
secure decentralized storage