debian 1.11.0-1 package looks broken without cffi, enum34, pycparser

[warner]

I updated my debian/sid box to the new tahoe-lafs-1.11.0-1 debian package today, and also did an apt-get autoremove which removed python-cffi and python-pycparser.

The resulting /usr/bin/tahoe fails at startup, because of our (#2749) internal version checks. The error message looks like this:

warner@cernio3:~$ tahoe --version
tahoe-lafs: 1.11.0 [master: 04a3e7993f70ac87c208d61f4387a59f5f419367]
foolscap: 0.10.1
pycryptopp: 0.6.0.1206569328141510525648634803928199668821045408958
zfec: 1.4.5
Twisted: 16.0.0
Nevow: 0.13.0
zope.interface: unknown
python: 2.7.11+
platform: Linux-debian_stretch/sid-x86_64-64bit_ELF
pyOpenSSL: 0.15.1
OpenSSL: 1.0.2g [ 1 Mar 2016]
simplejson: 3.8.1
pycrypto: 2.6.1
pyasn1: 0.1.9
service-identity: 16.0.0
characteristic: 14.3.0
pyasn1-modules: 0.0.7
cryptography: 1.2.3
cffi: None [(<type 'exceptions.ImportError'>, 'No module named cffi', ('/usr/lib/python2.7/dist-packages/allmydata/__init__.py', 239, 'get_package_versions_and_locations', '__import__(modulename)'))]
six: 1.10.0
enum34: 1.1.2
pycparser: None [(<type 'exceptions.ImportError'>, 'No module named pycparser', ('/usr/lib/python2.7/dist-packages/allmydata/__init__.py', 239, 'get_package_versions_and_locations', '__import__(modulename)'))]
setuptools: 20.3.1
idna: 2.0 [according to pkg_resources]

Warning: dependency 'cffi' (version None imported from None) was not found by pkg_resources.
Warning: dependency 'enum34' (version '1.1.2' imported from '/usr/lib/python2.7/dist-packages') was not found by pkg_resources.
Warning: dependency 'pycparser' (version None imported from None) was not found by pkg_resources.

For debugging purposes, the PYTHONPATH was
  None
install_requires was
  ['setuptools >= 11.3', 'zfec >= 1.1.0', 'simplejson >= 1.4', 'zope.interface >= 3.6.0, != 3.6.3, != 3.6.4', 'foolscap >= 0.10.1', 'pycrypto >= 2.1.0, != 2.2, != 2.4', 'pycryptopp >= 0.6.0', 'service-identity', 'characteristic >= 14.0.0', 'pyasn1 >= 0.1.8', 'pyasn1-modules >= 0.0.5', 'Twisted[tls] >= 15.1.0', 'Nevow >= 0.11.1', 'pyOpenSSL >= 0.14']
sys.path after importing pkg_resources was
  /usr/bin:
  /usr/lib/python2.7:
  /usr/lib/python2.7/plat-x86_64-linux-gnu:
  /usr/lib/python2.7/lib-tk:
  /usr/lib/python2.7/lib-old:
  /usr/lib/python2.7/lib-dynload:
  /usr/local/lib/python2.7/dist-packages:
  /usr/lib/python2.7/dist-packages

In retrospect, that's not actually a failure, but the successful list of versions scrolled off the screen, and what was left visible looked like a failure, so I'm going to claim that this is still a problem (to be fair, I'll note that tahoe create-introducer ran without noise).

We need to remove the transitive dependencies from our _auto_deps.py.

To fix this on Debian, I think the Debian package needs to either patch _auto_deps.py (which might involve patching __init__.py, I'm not sure), or re-introduce Depends: on the packages it wants (python-cffi, python-pycparser, maybe? I don't even know what that enum34 thing is, it's not in our install_requires= but it is listed in _auto_deps.py).

Or just delete all of __init__.py like #2749 suggests :).

[warner]

Installing python-cffi (which pulls in python-pycparser) is not sufficient to make it stop complaining. It changes the warning message for those two packages, though:

% sudo apt-get install python-cffi
% tahoe --version
tahoe-lafs: 1.11.0 [master: 04a3e7993f70ac87c208d61f4387a59f5f419367]
foolscap: 0.10.1
pycryptopp: 0.6.0.1206569328141510525648634803928199668821045408958
zfec: 1.4.5
Twisted: 16.0.0
Nevow: 0.13.0
zope.interface: unknown
python: 2.7.11+
platform: Linux-debian_stretch/sid-x86_64-64bit_ELF
pyOpenSSL: 0.15.1
OpenSSL: 1.0.2g [ 1 Mar 2016]
simplejson: 3.8.1
pycrypto: 2.6.1
pyasn1: 0.1.9
service-identity: 16.0.0
characteristic: 14.3.0
pyasn1-modules: 0.0.7
cryptography: 1.2.3
cffi: 1.5.2
six: 1.10.0
enum34: 1.1.2
pycparser: 2.14
setuptools: 20.3.1
idna: 2.0 [according to pkg_resources]

Warning: dependency 'cffi' (version '1.5.2' imported from '/usr/lib/python2.7/dist-packages') was not found by pkg_resources.
Warning: dependency 'enum34' (version '1.1.2' imported from '/usr/lib/python2.7/dist-packages') was not found by pkg_resources.
Warning: dependency 'pycparser' (version '2.14' imported from '/usr/lib/python2.7/dist-packages') was not found by pkg_resources.

For debugging purposes, the PYTHONPATH was
  None
install_requires was
  ['setuptools >= 11.3', 'zfec >= 1.1.0', 'simplejson >= 1.4', 'zope.interface >= 3.6.0, != 3.6.3, != 3.6.4', 'foolscap >= 0.10.1', 'pycrypto >= 2.1.0, != 2.2, != 2.4', 'pycryptopp >= 0.6.0', 'service-identity', 'characteristic >= 14.0.0', 'pyasn1 >= 0.1.8', 'pyasn1-modules >= 0.0.5', 'Twisted[tls] >= 15.1.0', 'Nevow >= 0.11.1', 'pyOpenSSL >= 0.14']
sys.path after importing pkg_resources was
  /usr/bin:
  /usr/lib/python2.7:
  /usr/lib/python2.7/plat-x86_64-linux-gnu:
  /usr/lib/python2.7/lib-tk:
  /usr/lib/python2.7/lib-old:
  /usr/lib/python2.7/lib-dynload:
  /usr/local/lib/python2.7/dist-packages:
  /usr/lib/python2.7/dist-packages

[exarkun]

Debian has since updated their package. Debian is also dropping Tahoe-LAFS due to lack of Python 3 support.