tahoe-lafs packaging issues

Fri Mar 19 16:49:38 UTC 2021

On Fri, Mar 19, 2021 at 11:15 AM Greg Troxel <gdt at lexort.com> wrote:

>
> I've been paged out for a while, but previously looked after the
> tahoe-lafs package in pkgsrc, and am again because 2.7-only packages are
> causing extra maintenance burden and hence there is pressure to outright
> remove them.  Some of this may be well known, but I'll type more and
> assume less.
>

Hello Greg,

Thanks for reaching out and thanks for the excellent detail in this email.
The strategies / tradeoffs / challenges you outlined all seem to make sense
to me.

I think everyone who uses Tahoe is increasingly feeling the pressure from
the gradual wind-down of the Python 2.7 ecosystem.  The good news is that
the Python 3 porting effort is quite far along - modules that constitute
over 75% of the lines of Tahoe have been ported and many other modules are
partially ported.  It's hard to estimate when this will be done but I am
*hopeful* that we are looking at a Python 3-supporting release sometime
this Spring.  The effort has committed resources behind it so there's no
reason to think we won't be able to get over the finish line.

I'm curious about whether you think having magic-folder as an optional
dependency would be beneficial for your packaging efforts or the users of
those packages *after* the Python 3 port is complete, or if this is
mostly/entirely about working around the increasingly constrained state of
Python 2.7 libraries?

Regarding txtorcon, it is an optional dependency that enables Tahoe to run
over Tor.

Thanks for getting in touch,
Jean-Paul

> I'm assuming that it's important to the project to be in packaging
> systems, so people can "${pkgfoo} install tahoe-lafs", for their
> preferred value of pkgfoo, rather than having to engage with the
> underlying language mechanims, not have automatic updates, and not have
> automatic warnings of CVEs.  (Similarly, I'm assuming that while the
> tahoe-lafs developers almost certainly don't use a packaging system for
> tahoe, I would expect that they do for 95%+ of the other software they
> run.)
>
> I gather there is progress on a python 3 version from the trac roadmap,
> but it's hard for me to tell what kind of timeline there is.  I realize
> everyone understands that this is necessary.
>
> From the packaging viewpoint (which I realize you may not be super
> familiar with), the two basic problems are:
>
>   python2.7 is EOL and not receiving security updates, so its continued
>   presence is uncomfortable.
>
>   Many python packages are releasing new versions that drop python2
>   support, even though other programs in the open source world that
>   depend on those packages are still python2 only.  The old versions do
>   not receive security updates, and many other programs and modules
>   depend on updated versions.
>
> So, early on the strategy was to defer updates to packages if an update
> would break some 2.7 only reverse dependency (thing that depends on the
> package, to use the Debian term).  That was only ok for a few months,
> and now our strategy is a combination of:
>
>   Question the continued presence of things that are 2.7 only, in that
>   if they are unmaintained, don't have users, etc. it may be just as
>   well to rm them.
>
>   Introduce versioned dependencies, where when building for 2.7, choose
>   the last 2.7-capable (and thus no longer maintained) version of a
>   python package, and for 3.x, choose the maintained version.  Add those
>   old versions as necessary, when the work to do so seems worthwhile
>   depending on what's being kept usable.
>
> and at some point this will be untenable, and programs that don't
> support 3 will just be removed from the packaging system.
>
> (Debian is also heading down the path of removing python2 and has
> already removed tahoe-lafs.
>
>   https://tracker.debian.org/pkg/tahoe-lafs
>   https://lists.debian.org/debian-python/2019/07/msg00080.html
>
> https://www.linuxcompatible.org/story/update-on-python-2-removal-in-debian-sid-bullseye/
>   https://wiki.debian.org/Python/2Removal
> )
>
> pkgsrc was at 1.12.1 (old I know) and I have done a trial update to
> 1.15.0.  However, that ran into a number of problems.
>
>   magic-wormhole has a lot of dependencies and a few I have to add
>   (which is fine).   However, quite a number of them have current
>   releases which are 3.x only, so building a py27-magic-wormhole would
>   require adding a bunch of versioned dependencies.
>
>   There appear to be similar issues with tahoe-lafs itself requiring
>   2.7-supporting and hence old, unmaintained versions of a few python
>   packages (autobahn, txtorcon).  Perhaps txtorcon is actually optional;
>   I'm having enough other problems that it's hard to tell.
>
> It would help pkgsrc to update tahoe-lafs to 1.15.0, both for the usual
> notion of users having a recent release, and because a number of really
> old dependencies are no longer used (eg., py-crypto which we've patched
> to use cryptodome instead).  One thing that would help that is if
> magic-wormhole were not a hard requirement, and was only needed if not
> tried to use a feature that uses it.  I don't know how hard that is, and
> if that would really get us over the line to 1.15.x in pkgsrc.  And if a
> 3.x release is going to happen in the next 30 days, there's of course no
> point.
>
> I tried to just comment it out, very hackily, and the built tahoe runs
> enough to print the command help, as long as I have hand-installed an
> old py27-autobahn.
>
>
> My todo list is huge, and to be honest this isn't high on it, but I
> wanted to let you know how this is heading, as I'm not at all sure how
> the project views the situation.
>
> I'd like to hear how the project sees this.
>
>
> Greg
> _______________________________________________
> tahoe-dev mailing list
> tahoe-dev at tahoe-lafs.org
> https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://tahoe-lafs.org/pipermail/tahoe-dev/attachments/20210319/83c5a0c3/attachment.html>