#1658 closed enhancement (fixed)

drop support for Python < 2.6

Reported by: zooko Owned by: zooko
Priority: normal Milestone: 1.10.0
Component: packaging Version: 1.9.0
Keywords: packaging backward-compatibility review-needed Cc:
Launchpad Bug:

Description (last modified by zooko)

Let's drop support for versions of Python < 2.6. According to distrowatch's tables about the versions of packages that come with distros, this would exclude Tahoe-LAFS 1.10 from being usable with the included Python on the following versions of major distros:

distro excluded version excluded version release date oldest supported[*] version oldest supported[*] release date Python in oldest supported[*] notes next version Python in next version
Fedora 10 2008-11-25 15 2011-05-24 2.7 rawhide will be 17 2.7.2
Ubuntu 8.10 2008-10-30 10.04 LTS 2010-04-29 2.6 will be 12.04 precise 2.7.2
Debian 5.0 2009-02-15 6.0 2011-02-06 2.6 unstable sid will be 7.0 2.7.2
RedHat 5.7 2011-07-21 5.7 2011-07-21 2.4 I'm told that people tend to install their own newer version of Python when they want to use newer Python programs on older RHEL (current, not next) 6.2 2.6.6

Among other benefits, dropping support for Pythons older than 2.6 would allow us to remove the dependency on simplejson and the (complex imperatively decided) dependency on pysqlite.

See comment:11 -- apparently Python ≥ 2.6 is already ubiquitous in the *BSD world.

[*] that is, the oldest OS version supported by the OS distributors and still receiving security patches.

Attachments (2)

require-python-2.5.darcs.patch (122.0 KB) - added by davidsarah at 2012-05-16T02:51:44Z.
Require Python 2.5. Includes simplifications resulting from being able to use sqlite3 from the standard library. This also drops sqlite3 from the set of versions and paths we report.
use-SEEK_END.darcs.patch (102.1 KB) - added by davidsarah at 2012-05-16T21:42:56Z.
Since we now require Python 2.5, we can use os.SEEK_END. (Apply on top of previous patch.)

Download all attachments as: .zip

Change History (30)

comment:1 Changed at 2012-02-08T13:31:34Z by zooko

  • Description modified (diff)

comment:2 follow-up: Changed at 2012-02-09T01:56:46Z by davidsarah

I don't understand the "oldest supported" columns for FreeBSD and RedHat, if they're supposed to be referencing the "included" Python.

Another thing that could be dropped is the SHA-256 bindings in pycryptopp, since hashlib has SHA-256 since Python 2.5.

Last edited at 2012-02-09T01:57:21Z by davidsarah (previous) (diff)

comment:3 Changed at 2012-03-16T07:28:11Z by zooko

  • Description modified (diff)

comment:4 Changed at 2012-03-16T07:29:49Z by zooko

Twisted is planning to drop support for Python ≤ 2.5 after their next release, which means sometime this year or so:

http://twistedmatrix.com/trac/ticket/5553

comment:5 in reply to: ↑ 2 ; follow-up: Changed at 2012-03-16T07:30:16Z by zooko

Replying to davidsarah:

I don't understand the "oldest supported" columns for FreeBSD and RedHat, if they're supposed to be referencing the "included" Python.

I changed the title of the "oldest supported" column to "oldest supported version of the operating system". Does it make sense now?

comment:6 follow-up: Changed at 2012-03-16T14:45:25Z by gdt

As far as I can tell the FreeBSD column doesn't quite make sense. I'm a little fuzzy, but I think python isn't part of the base system, but is instead in "ports", which is similar to NetBSD. I think the culture is to run relatively recent ports even on older base systems, so my guess is that dropping <=2.5 will not be a significant issue for FreeBSD users.

The chart above doesn't have NetBSD, but python 2.6 has been default in pkgsrc for probably 2 years if not more, and the next quarterly release of pkgsrc (April) will have 2.7 as the default. So dropping support for <=2.5 is fine from that viewpoint also.

comment:7 in reply to: ↑ 5 ; follow-up: Changed at 2012-03-16T16:56:41Z by davidsarah

Replying to zooko:

Replying to davidsarah:

I don't understand the "oldest supported" columns for FreeBSD and RedHat, if they're supposed to be referencing the "included" Python.

I changed the title of the "oldest supported" column to "oldest supported version of the operating system". Does it make sense now?

No, that wasn't what I was confused about. I was confused as to how it is possible for an entry in the first column, i.e. "excluded version of the operating system", to be greater than or equal to the "oldest supported version of the operating system" -- i.e. FreeBSD 7.2 >= FreeBSD 7.1 and RedHat 5.7 >= RedHat 5.7.

comment:8 in reply to: ↑ 7 Changed at 2012-03-16T17:30:45Z by zooko

Replying to davidsarah:

No, that wasn't what I was confused about. I was confused as to how it is possible for an entry in the first column, i.e. "excluded version of the operating system", to be greater than or equal to the "oldest supported version of the operating system" -- i.e. FreeBSD 7.2 >= FreeBSD 7.1 and RedHat 5.7 >= RedHat 5.7.

I'm still not sure I understand what you were/are confused about. "excluded version of the operating system" is supposed to be the newest release of the operating system that doesn't have Python ≥ 2.6. "oldest supported version of the operating system" is supposed to be the oldest release of the operating system that is supported *by the operating system distributors*, i.e. the oldest release of the operating system that still gets security patches. Does that help? I apologize for the confusing wording.

comment:9 in reply to: ↑ 6 Changed at 2012-03-16T17:40:29Z by zooko

Replying to gdt:

As far as I can tell the FreeBSD column doesn't quite make sense. I'm a little fuzzy, but I think python isn't part of the base system, but is instead in "ports", which is similar to NetBSD. I think the culture is to run relatively recent ports even on older base systems, so my guess is that dropping <=2.5 will not be a significant issue for FreeBSD users.

I guess I was following distrowatch on that. So you're saying that even users of FreeBSD older than 7.3, such as FreeBSD 7.2 or 7.1 (or even 6?) will be able to deploy a Python 2.6-requiring Tahoe-LAFS without installing a local Python?

The chart above doesn't have NetBSD, but python 2.6 has been default in pkgsrc for probably 2 years if not more, and the next quarterly release of pkgsrc (April) will have 2.7 as the default. So dropping support for <=2.5 is fine from that viewpoint also.

According to distrowatch, the NetBSD entry would look like this:

distro excluded version excluded version release date oldest supported version of the operating system oldest supported release date Python in oldest supported notes next version Python in next version
NetBSD 5.0 2009-04-29 4.0 2007-12-19 2.5.2 pkgsrc? NetBSD-CURRENT? will be 6.0? 2.7.x

But apparently distrowatch, or my way of reading it, is inaccurate. Could you, gdt, please update the table to reflect NetBSD? Thanks!

comment:10 Changed at 2012-03-16T17:44:50Z by davidsarah

  • Description modified (diff)

sqlite bindings are in Python 2.5, and so is hashlib.sha256. When we've had buildslave failures for unsupported syntax, they've usually been for 2.4.x and not 2.5. So is the simplejson dependency the only reason to drop support for 2.5?

If so, note that the json module in the stdlib is not identical to simplejson. According to http://stackoverflow.com/questions/712791/json-and-simplejson-module-differences-in-python, the stdlib version is less frequently updated and slower (although I would be more concerned about security-relevant bugfixes than speed).

I agree we should definitely drop support for 2.4.x.

zooko wrote:

"oldest supported version of the operating system" is supposed to be the oldest release of the operating system that is supported *by the operating system distributors*, i.e. the oldest release of the operating system that still gets security patches.

Oh, ok, that wasn't at all obvious.

comment:11 Changed at 2012-03-16T18:05:22Z by gdt

distrowatch is confused; NetBSD simply does not include python (at all, and never has). In NetBSD culture, one use pkgsrc to get extras from among the 6000 or so other things that are not part of the operating system and are in pkgsrc. FreeBSD is basically the same way, except I can't speak as authoritatively about it.

On NetBSD, one basically needs a version of pkgsrc (stable ones are released quarterly) that supports the version of the OS one is running (meaning that most packages will build on that OS version), and then it's easy to use whatever is in that pkgsrc version. So people on NetBSD can have python 2.6 or 2.7 very easily, and 2.6 is what's been normal for a while. In NetBSD, at least the stable branch of the last two releases are generally supported by pkgsrc. 6.0 isn't out, so anyone with 4.0, 5.0, 5.1. etc. will be able to have python 2.6 or 2.7 without difficulty, from the current 2011Q4 stable branch, or the April 2012Q1 branch.

But, the notion that "NetBSD 4.0 has python 2.5.2" is confused. What they might mean is that a stable version of pkgsrc associated in time with the release of NetBSD 4.0 had python 2.5.2, but that doesn't matter now.

On FreeBSD, I believe the culture is that one upgrades from 7.x to 7.x+1 quite easily, and you're running old/not-really-maintained-code if you don't. So sure, people need to build python, but that's done automatically when building tha tahoe-lafs port (which either does exist or ought to).

Also, on FreeBSD (or Dragonfly, OpenBSD, Mac, Linux, Solaris, AIX, etc) one can use pkgsrc.

I suspect the problem is that distrowatch either doesn't get it that all the world does not work like GNU/Linux, or is trying to map the world into that model.

But all in all, my belief is that dropping 2.5 is a non-event for the BSD world.

I haven't updated the table, because I don't think it makes sense for other than Linux-style distributions and I don't know how to do it sensibly. There's no Windows row, which would have the same issues, as Windows doesn't include python either.

comment:12 Changed at 2012-03-26T04:19:24Z by zooko

  • Description modified (diff)

comment:13 Changed at 2012-03-29T19:12:17Z by davidsarah

  • Priority changed from major to normal

comment:14 Changed at 2012-04-01T00:21:56Z by davidsarah

  • Milestone changed from 1.11.0 to 1.10.0

comment:15 Changed at 2012-05-15T16:58:26Z by zooko

  • Type changed from defect to enhancement

comment:16 Changed at 2012-05-15T18:06:57Z by warner

I found some of my old notes for what we could clean up if we required a newer python version. If we dropped 2.4 and required >=2.5, we could:

  • replace DictOfSets with collections.defaultdict(set)
  • replace pycryptopp.hash.SHA256 with hashlib.sha256
  • replace pysqlite2|sqlite3 with stdlib sqlite3
  • switch to PEP328 relative imports

And then >=2.6 would get us stdlib json, and could replace our LRU cache (unless we already got rid of it?) with collections.deque(maxlen=). It might even be possible to get a codebase that works with py2.6 and py3.3 simultaneously, since 2.6 adds b"", print-as-function (guarded by __future__), and "except FooError as e:" (although, of course, we kind of have to wait for our dependencies to move to py3 first).

Last edited at 2015-05-02T18:44:40Z by daira (previous) (diff)

Changed at 2012-05-16T02:51:44Z by davidsarah

Require Python 2.5. Includes simplifications resulting from being able to use sqlite3 from the standard library. This also drops sqlite3 from the set of versions and paths we report.

comment:17 Changed at 2012-05-16T02:55:51Z by davidsarah

  • Keywords review-needed added
  • Owner changed from somebody to zooko

comment:18 Changed at 2012-05-16T02:57:05Z by davidsarah

Note that it's only dropping support for Python 2.4.x that is in milestone 1.10.

Changed at 2012-05-16T21:42:56Z by davidsarah

Since we now require Python 2.5, we can use os.SEEK_END. (Apply on top of previous patch.)

comment:19 Changed at 2012-05-18T23:26:29Z by davidsarah

  • Resolution set to fixed
  • Status changed from new to closed
  • Summary changed from drop support for Python < 2.6 to drop support for Python < 2.5

Zooko read these patches and approved of them, so I committed them (only on trunk, not for 1.9.2): 0fc196ea5fa4ad2f, a1a1b5bf8a6e1a09, 4ddcde309454179a

Please open separate tickets for any other simplifications that can be made, or if we want to also drop 2.5. (From comment:16, only the sqlite change has been done.)

comment:20 follow-up: Changed at 2012-05-18T23:28:45Z by davidsarah

By the way, dropping 2.6 would not allow eliminating the dependency on simplejson; that was added to the stdlib in 2.7.

Version 0, edited at 2012-05-18T23:28:45Z by davidsarah (next)

comment:21 Changed at 2012-05-18T23:34:39Z by davidsarah

  • Milestone changed from 1.10.0 to undecided
  • Resolution fixed deleted
  • Status changed from closed to reopened
  • Summary changed from drop support for Python < 2.5 to drop support for Python < 2.6

Oh, I'd missed comment:4 and http://twistedmatrix.com/trac/ticket/5553. So there is still a strong incentive to drop support for 2.5, so that we don't have to impose a conditional "Twisted <= 12.1" requirement in _auto_deps.py for Python 2.5.

comment:22 in reply to: ↑ 20 Changed at 2012-05-20T02:12:23Z by warner

Replying to davidsarah:

By the way, requiring 2.6 would not allow eliminating the dependency on simplejson; that was added to the stdlib in 2.7.

http://docs.python.org/library/json.html#module-json says that json was added in python2.6 . Some minor features were added in the py2.7 version, but I don't think they'd be useful to us.

comment:23 Changed at 2012-05-20T04:43:02Z by zooko

According to my notes from the original description in this ticket, plus gdt's comments in comment:11, there aren't many people who require Python 2.5. We should post to the tahoe-dev list asking someone to speak up if they use Python 2.5. I'll bet nobody does!

comment:24 Changed at 2012-06-11T03:53:48Z by david-sarah@…

In b82cddef7b7f5eee:

setup.py and bin/tahoe-script.template: the error when we try to use Python 3 should give the correct minimum Python version (now 2.5). refs #1658

comment:25 Changed at 2012-06-11T15:05:08Z by david-sarah <david-sarah@…>

In b82cddef7b7f5eee:

setup.py and bin/tahoe-script.template: the error when we try to use Python 3 should give the correct minimum Python version (now 2.5). refs #1658

comment:26 Changed at 2012-06-18T20:34:22Z by davidsarah

In src/allmydata/test/test_upload.py:

# copied from python docs because itertools.combinations was added in
# python 2.6 and we support >= 2.4.
def combinations(iterable, r):

comment:27 Changed at 2012-07-20T02:20:06Z by david-sarah@…

In 5923/cloud-backend:

[rebased for cloud-backend] setup.py and bin/tahoe-script.template: the error when we try to use Python 3 should give the correct minimum Python version (now 2.5). refs #1658

comment:28 Changed at 2013-03-15T05:19:26Z by davidsarah

  • Milestone changed from undecided to 1.10.0
  • Resolution set to fixed
  • Status changed from reopened to closed

Fixed in 7008ffa5.

I realized that the patch fixing #1775 depends on a construct added in Python 2.6 ("except Exception as e"). I think we had already decided to drop Python 2.5 for Tahoe-LAFS v1.10.

Note: See TracTickets for help on using tickets.