build failure: mock vs zetuptoolz

[warner]

Zooko and Jun Matsushita reported this one via email (as Trac was temporarily broken at the time). Building tahoe (either 1.10.1 or trunk) is currently broken.

The "mock" library (which we use for running unit tests, but not during runtime) recently received its first update in two years, and the new version (1.1.3) depends upon setuptools 17.1 or newer. Tahoe's forked "zsetuptoolz" doesn't meet the requirement.

I'm not entirely sure why the 17.1 requirement was added: the previous mock-1.1.2 didn't have that requirement, and Tahoe builds failed with an error like this:

  File "build/setuptools-0.6c16dev6.egg/setuptools/command/easy_install.py", line 385, in install_egg_scripts
    self.install_wrapper_scripts(dist)
  File "build/setuptools-0.6c16dev6.egg/setuptools/command/easy_install.py", line 577, in install_wrapper_scripts
    for args in get_script_args(dist, script_dir=self.script_dir):
TypeError: override_get_script_args() got an unexpected keyword argument 'script_dir'

It's possible that the dependency is actually imposed by the "pbr" library that mock now uses. The current mock-1.1.3 error looks like:

  File "build/setuptools-0.6c16dev6.egg/setuptools/command/easy_install.py", line 938, in build_and_install
    self.run_setup(setup_script, setup_base, args)
  File "build/setuptools-0.6c16dev6.egg/setuptools/command/easy_install.py", line 927, in run_setup
    run_setup(setup_script, args)
  File "build/setuptools-0.6c16dev6.egg/setuptools/sandbox.py", line 62, in run_setup
    lambda: execfile(
  File "build/setuptools-0.6c16dev6.egg/setuptools/sandbox.py", line 105, in run
    return func()
  File "build/setuptools-0.6c16dev6.egg/setuptools/sandbox.py", line 64, in <lambda>
    {'__file__':setup_script, '__name__':'__main__'}
  File "setup.py", line 10, in <module>
    #
AttributeError: 'module' object has no attribute 'version'

And it looks like mock trunk made a change yesterday (which will probably be released as 1.1.4) to get the setuptools version from the right place, which will probably change that error to something like:

mock requires setuptools>=17.1. Aborting installation

Some potential fixes/improvements:

• Remove "mock" from the set of things installed by "setup.py build": it's only for unit tests, and end-users don't need it. But this won't help developers, and might be hard to implement because tahoe has extensive (and overgrown, IMO) code to check the versions of all its dependencies, and that code doesn't have the notion of "tests_requires".
• Stop using "mock" altogether. Zooko points out that twisted.trial has a patch-during-tests utility that's better (it understands Deferreds). We have 9 test_*.py files that import mock.
• Stop forking setuptools. This is obviously the best long-term fix, and would be an opportunity to clean up a lot of our build code.
• Prolong the agony: modify zetuptoolz to handle whatever the newer mock needs.

[warner]

Daira implemented (in baaf2a4) the 5th option which I forgot about: change our mock dependency to reject any of the newer versions. This is a short-term fix, as really we want to be compatible with the latest version of everything. So I think we still need to implement one of the other longer-term fixes.

Lowering the priority, since this is no longer breaking all builds.

[daira]

... but it is still breaking builds of 1.10.1 and earlier :-(

I was hoping this would be something that could be easily fixed by releasing a new version of mock, but it sounds not. We might need to release a 1.10.2 to fix this.

(Note: this doesn't affect the Debian tahoe-lafs package because they won't have packaged mock 1.1.2+ yet.)

[daira]

It's really quite irritating that this happened a few weeks after releasing 1.10.1 when we'd held off that release for 2 years :-/

[daira]

I'm in favour of removing the mock dependency. The style of tests it results in are quite complicated to follow and I don't think it is buying us much, relative to using trial's patch method.

[daira]

I made a start on eliminating mock in this branch: ​https://github.com/tahoe-lafs/tahoe-lafs/commits/2465.eliminate-mock.0. It doesn't seem too painful.

[daira]

Review needed for ​https://github.com/tahoe-lafs/tahoe-lafs/pull/175.

[warner]

Landed in [c830419e]. I'll plan to release 1.10.2 early next week.