don't require newer foolscap if you have older Twisted

[zooko]

The current stable release—v1.8.1—​requires Twisted >= 2.4.0 and foolscap >= 0.5.1. The current trunk ​requires Twisted >= 2.4.0 and foolscap >= 0.6.0. Brian ​bemoaned this increased requirement on the mailing list, since there is actually nothing wrong with foolscap 0.5.1 unless you also happen to be using Twisted >= 10.2.

Brian gave as example that people on Ubuntu 10.04 Long-Term-Support "Lucid" have Twisted-10.0 and foolscap-0.5.1.

This patch makes it so that if you already have Twisted installed and the version is < 10.2 then the required version of foolscap is >= 0.5.1, else it is >= 0.6.0.

David-Sarah ​proposed a different approach, in which python setup.py build would build a local private copy of foolscap >= 0.6.0 even if the system already have foolscap-0.5.1 installed. I'm not sure users would be okay with that behavior -- it runs counter to the desiderata that Tahoe-LAFS will use the packages that are provided in the system if they are of an acceptable version number.

In any case, I would like to apply my patch right away for Tahoe-LAFS v1.8.2, so review-needed!

[zooko]

The desert island test is currently red because of this:

​http://tahoe-lafs.org/buildbot/builders/clean/builds/2667/steps/test-desert-island/logs/stdio

Downloading http://foolscap.lothar.com/releases/foolscap-0.6.0.tar.gz
Failed: build tried to download files
gmake: *** [test-desert-island] Error 1

I guess we need to either update the foolscap version in ​tahoe-deps or else apply the patch from this ticket. :-)

[davidsarah]

Replying to zooko:

David-Sarah ​proposed a different approach, in which python setup.py build would build a local private copy of foolscap >= 0.6.0 even if the system already have foolscap-0.5.1 installed.

There must be some confusion -- this is what trunk already does.

If having an installed foolscap-0.5.1 breaks the ability to build a private foolscap 0.6.0 in support/.../site-packages, that's a bug. It works fine for me, and on the buildbots, many of which have foolscap 0.5.1 installed.

(Potentially #1258 could cause such breakage, but only if the installed foolscap is in a directory shared with other packages that are Tahoe dependencies. If this causes an incorrect version to be found, check_all_requirements on the ticket1306 branch will tell you to remove the incorrect version from the shared directory.)

[zooko]

Replying to davidsarah:

Replying to zooko:

David-Sarah ​proposed a different approach, in which python setup.py build would build a local private copy of foolscap >= 0.6.0 even if the system already have foolscap-0.5.1 installed.

There must be some confusion -- this is what trunk already does.

Hm, yes I didn't understand your proposal. On that mailing list message, you wrote:

'setup.py build', by default, should either:

• have built a private copy of twisted < 10.2 (always used by bin/tahoe) that won't be affected by upgrades of the installed version, or
• have built a private copy of foolscap 0.6.0 (always used by bin/tahoe).

So if I understand correctly, you prefer for Tahoe-LAFS v1.8.2 to require foolscap >= 0.6.0 even if Twisted is < 10.2 (or alternately to require Twisted <= 10.2) for installation, because if Tahoe-LAFS v1.8.2 allows foolscap-0.5.1 at install time, then the user may later upgrade their Twisted to >= 10.2 and then execute bin/tahoe without re-running python setup.py build and they would then get a failure to start due to conflicting version requirements.

This is what trunk currently does.

The drawback to this approach is ​Brian's original complaint -- that it means Tahoe-LAFS requires foolscap >= 0.6.0 when the foolscap-0.5.1 that comes with Ubuntu Lucid should really be good enough.

Now, I don't feel so strongly about this, so I'm basically -0 on leaving trunk as it is, because I can live with it, but I would like to understand your motivation better in order to work with you on related issues.

My idea of packaging is that there is a "build/install" step where the packaging system can do stuff having to do with dependencies--either automatically resolve dependencies (complicated and controversial, but handy for users who don't know how to resolve dependencies themselves), or at least detect unresolved dependencies and clearly alert the user to the problem and exit loudly. Then, at run-time the packaging system could re-examine dependencies in order to give a better error message if a dependency is missing or is of an incompatible version, but it must not resolve dependencies at that time.

So I have the feeling that if the user changes installed deps after build time and before run-time, then we have no responsibility beyond erroring out cleanly, but you seem to think we should pay a cost (in the form of a higher version requirement on foolscap) in order to make Tahoe-LAFS run correctly even if the user has done that,.

[davidsarah]

After further thought and some IRC discussion, my complaint would be addressed if in the case where bin/tahoe fails because twisted >= 10.2 has been installed, it gives a message saying to re-run python setup.py build. I'll prepare a patch for that.

[warner]

Given that #1329 can be fixed by depending upon the new (just released a few hours ago) foolscap-0.6.1, perhaps this ticket has been rendered obsolete.

[zooko]

We agree to fix this by applying attachment:foolscap051060.darcs.patch​ foolscap051060.darcs.patch, adding code to catch version conflict exceptions at start-up time and print out an error message suggesting to the user to re-run python setup.py build, and fixing #1329.

Next step is for David-Sarah to write that patch to print a useful error message if there is a version error at start-up time.

[davidsarah]

It turned out to be much more difficult than I or zooko had thought to make the foolscap requirement dependent on the installed version of Twisted.

The problem is that:

• the first import of either pkg_resources or setuptools in setup.py must be after __requires__ has been set (because that import has side effects on the sys.path that depend on __requires__);
• __requires__ is set from the requirements computed by src/allmydata/_auto_deps.py (loaded using execfile);
• if _auto_deps.py imports pkg_resources in order to get the version of Twisted (for the current working set), then that first import would have to be both before and after setting __requires__: contradiction.

If _auto_deps.py were to import twisted directly, OTOH, that wouldn't necessarily be the same Twisted version that would later be imported at run time.

So we decided to just require foolscap >= 0.6.1 instead. The difference between 0.6.0 and 0.6.1 is to quiet a DeprecationWarning (#1329).