debian porting, twisted[tls] dependency

[Brian Warner]

On 3/30/16 6:45 AM, Ramakrishnan Muthukrishnan wrote:

> I made a package and am happy to report that it all went well. We no
> longer need a bunch of debian specific patches that we used to apply.

Huzzah!

> Another observation: My Debian machine which runs the 'testing' flavour
> had the version 15.2.1 of Twisted and with that I get an error:

>   File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py",
>   line 2610, in requires
>     "%s has no such extra feature %r" % (self, ext)
> pkg_resources.UnknownExtra: Twisted 15.2.1 has no such extra feature
> 'tls'

Hm. It looks like Twisted added the "tls" extra in twisted-15.1.0. So
that message will appear on any system that has twisted-15.0.0 or older
installed. I'm not sure how that happened on 15.2.1 .. maybe Debian
modifies the install to remove those "extras"? Maybe the .egg-info data

Pip doesn't have a full dependency resolver, so it sees the numerical
"Twisted >= 13.0.0" constraint satisfied, and doesn't/can't use a newer
version to additionally satisfy the [tls] constraint. But the runtime
(pkg_resources) check is much more strict.

> I am a bit puzzled because on the OS-X 10.10.5, I see that twisted
> version that comes with the system is 13.2.0 and that seem to work
> fine.

Was this in a virtualenv? In that case, you'd be isolated from the
system and wouldn't even see that twisted-13.2.0. Or maybe the older
setuptools there doesn't even notice the extras?



Our current _auto_deps.py calls for "Twisted[tls] >= 13.0.0". If we want
to use the extra, I guess that ought to be "Twisted[tls] >= 15.1.0".

But we don't really need that recent of a Twisted (see
https://tahoe-lafs.org/~warner/deps.png for a week-old graph of tahoe's
transitive dependencies).

And depending upon it would force your hand as a packager, to make sure
that the particular distribution you're targetting has a newer copy of
Twisted too (I'm thinking about backports here: jessie has
twisted-14.0.2, so anyone who's backporting Tahoe would also have to
backport twisted). Although I guess we've already forced your hand by
adding the [tls] constraint.

There's one other constraint: the latest Foolscap-0.11.0 requires
"Twisted[tls] >= 16.0.0". But the previous version (Foolscap-0.10.1)
only wanted >=10.1.0 . So a legal solution (according to the numeric
constraints) is (tahoe-lafs==1.11.0, foolscap==0.10.1,
twisted[tls]==13.0.0), and that *does* trigger the "no such extra [tls]"
error. So maybe it's best to be all-in, and upgrade/backport all of
tahoe,foolscap,twisted.

So in the short term, if twisted-16.0.0 is already in the pipeline for
testing/"stretch", then that should satisfy pkg_resources.

I'll update Tahoe's dependency from "Twisted[tls] >= 13.0.0" to
"Twisted[tls] >= 15.1.0". Porters take note: if it's inconvenient for
you to backport twisted and foolscap too, then you can modify tahoe's
src/allmydata/_auto_deps.py to remove the "[tls]" from "Twisted >=
13.0.0", as long as the OS package adds dependencies on whatever the
older Twisted needs to provide TLS support (things like
service-identity, pyopenssl, idna, maybe characteristic).

thanks,
 -Brian