"tox" on debian (py3) can't run our setup.py

[warner]

rkirshnan reported that running "tox" on current debian/testing fails.

On that platform (as well as some Ubuntu ones I've seen), the system-packaged /usr/bin/tox is shbang'ed to use python3. Tox runs the project's setup.py to make an sdist tarball (which it then installs into the virtualenv). Tahoe's setup.py has a line that explicitly fails when run with python3, to avoid confusion, since Tahoe only works with python-2.7 . When tox hits this, it fails.

To fix this, I think we should take out the hard-fail. It might be good to leave in a warning (e.g. check sys.version_info), but move the version-based hard-fail to early in the entry-point script.

We should do an experiment and try to install tahoe in a py3 virtualenv, and see what happens: I suspect that some of the dependencies (zfec? pycryptopp? foolscap?) will fail to install, so nobody would ever get to the point of even running tahoe. If they made it that far, the next question would be whether importing our dependencies would succeed: that might produce different errors depending upon which subcommand was run (since we sometimes avoid importing things before we really need them).

[warner]

I started working on this, but it's not trivial. We basically need to make our setup.py work correctly under both py2 and py3. The problems:

• execfile(../_auto_deps.py)
• the code that runs git to compute a version string has bytes-vs-str issues

If I bypass those two issues, tox is able to install an sdist, and then run the tests.

The second issue will be fixed when we move to Versioneer (#2748).

The first needs us to either find a py2/py3-compatible equivalent of execfile(), or (my preference) remove _auto_deps.py and move the install_requires= list into setup.py. That will also entail removing a lot of the code from __init__.py that checks what versions of things are installed and complains if they don't look right (and frequently complains even if they do). #2749 covers this (#2283 is related, but doesn't go far enough).

I'm torn whether to try to address this in the upcoming release (for the sake of developers using system "tox" on recent debian/ubuntu), or to punt until the next release (to avoid the deep __init__.py changes it would require).

[warner]

Milestone renamed

[warner]

I surveyed a couple of releases. All current versions of debian ("sid", "testing", and even the latest release "jesse") use a python3-based tox. On ubuntu, trusty (14.04LTS) uses py2, but vivid (15.04) and beyond use py3. The next Ubuntu LTS (16.04 Xenial) is coming out in a month, and will use py3-based tox too.

We could fix this after 1.11.0 ships, under the theory that most people inclined to run unit tests will also be inclined to run from a git checkout.

But if we don't fix this before 1.11.0, most debian/ubuntu users will be unable to run the release's unit tests with our recommended process. They'll have to create a py2-based virtualenv, pip install tox into that, then run the resulting executable. Or, since they're going to make a virtualenv anyways (to run tahoe), they can just do "trial allmydata" from within the virtualenv.

That doesn't sound like a great experience. It doesn't affect people's ability to *use* tahoe, just their ability to run the tests in a standard way.

[warner]

Oh, I think we can do this. I figured out a 2+3 cross-compatible execfile() replacement to avoid changing the way we use _auto_deps.py for now. And I think I can 2+3-ify the version string code too (I've got a py3-compatible flavor now, but doing 2+3 shouldn't be too hard).

[Brian Warner <warner@…>]

In dd84abd/trunk:

setup.py: work with tox under py3

This allows a python3-based "tox" (as shipped with modern debian and
ubuntu systems) to run setup.py egg_info, update_version, and sdist
commands. It moves the main "tahoe requires py2" check out of setup.py
and into allmydata.scripts.runner.run, where it gets applied at runtime
rather than build time.

It also changes the execfile(_auto_deps.py) and Versioneer-like "ask git
what our version string should be" code to work under both py2 and py3.

fixes ticket:2747

[vu3rdd]

Very nice! Thanks. I can confirm that tox indeed works fine on Debian testing now on the latest master branch.