[tahoe-lafs-trac-stream] [Tahoe-LAFS] #2739: windows vs len("pycryptopp-0.7.1.869544967005693312591928092448767568728501330214")

[Tahoe-LAFS]

#2739: windows vs
len("pycryptopp-0.7.1.869544967005693312591928092448767568728501330214")
---------------------------+-----------------------
     Reporter:  warner     |      Owner:  daira
         Type:  defect     |     Status:  new
     Priority:  normal     |  Milestone:  undecided
    Component:  packaging  |    Version:  1.10.2
   Resolution:             |   Keywords:
Launchpad Bug:             |
---------------------------+-----------------------

Comment (by warner):

 I did another experiment: modifying spake2 to depend on pycryptopp, then
 doing a `pip install .` from the spake2 directory underneath variously-
 lengthed $AAAA parent directories. This causes pip to build (and install)
 a pycryptopp wheel in the same way that `pip install tahoe-lafs` ought to
 do. I found that it worked at least as far as $AAAA=150, which means that
 the pycryptopp build is *not* happening inside the current working
 directory.

 I eventually found the tempdir that is hosting the build:
 `C:\Users\USERNAME\AppData\Local\Temp\2\pip-build-j4xbzr\pycryptopp`. This
 directory does include the username, but does *not* include the CWD copy
 of the version string (it unpacks into `pycryptopp` rather than
 `pycryptopp-0.7.1.869544967005693312591928092448767568728501330214`, which
 must be pip or wheel controlling the target directory of
 `TarFile.extract_all()`). So I think that means each byte of the version
 string only counts against the ultimate path-length limit once, not twice.

 Since the original error (on Daira's box) was using a username of just
 "Daira", I think something else is going on here. I do notice that Daira's
 log in comment:27:ticket:1582 shows the pycryptopp version string getting
 included *twice*:

 {{{
   Copying pycryptopp.egg-info to build\bdist.win-
 amd64\wheel\pycryptopp-0.7.1.86954496700569331259192809244876
 7568728501330214.data\..\pycryptopp-0.7.1.869544967005693312591928092448767568728501330214-py2.7
 .egg-info
   error: [Errno 2] No such file or directory: 'build\\bdist.win-
 amd64\\wheel\\pycryptopp-0.7.1.869544967005693
 312591928092448767568728501330214.data\\..\\pycryptopp-0.7.1.869544967005693312591928092448767568728501330214-
 py2.7.egg-info\\dependency_links.txt'
 }}}

 For some reason it's reaching the egg-info directory as a sibling (up-and-
 over) relative to the `.data` directory, both of which have pycryptopp's
 long version string. It didn't do that in my tests.

 Daira: what version of setuptools and 'wheel' do you have on your box?
 I've got setuptools-20.2.2 and wheel-0.29.0 .

 I see one potentially-relevant note in the wheel changelog: version 0.27.0
 fixed [https://bitbucket.org/pypa/wheel/issues/91/cannot-create-a-file-
 when-that-file issue 91] "Don’t attempt to (recursively) create a build
 directory ending with .. (invalid on all platforms, but code was only
 executed on Windows)". It looks like the patch involves adding an
 `os.path.normpath()` call, which might flatten out that extra directory,
 and might reduce the severity of this issue.

 So maybe Daira is using wheel 0.26.0 or older, and it'd be sufficient to
 just upgrade that to 0.27.0 or newer. That'd give us an extra 65
 characters or headroom (maybe 70, since the `.data` suffix is removed
 too).

 If that helps, what would the new maximum version-string length be?
 Suppose we have $USERNAME, $PKGNAME, and $VERSION. The longest filename
 written into egg-info is `dependency_links.txt`. So I think we're talking:
 `C:\Users\$USERNAME\AppData\Local\Temp\2\pip-build-
 XXXXXX\$PKGNAME\build\bdist.win-amd64\wheel\$PKGNAME-$VERSION-py2.7.egg-
 info\dependency_links.txt`. That's a fully-qualified file name of 114 +
 len($USERNAME) + 2*len($PKGNAME) + len($VERSION) bytes (which must be less
 than 260 characters). The directory name is
 93+len($USERNAME)+2*len($PKGNAME)+len($VERSION), and must be less than 248
 characters, so those are nearly equal constraints.

 So for a 10-char $USERNAME and 10-char $PKGNAME, we can tolerate a
 $VERSION of up to 116 characters. Zooko's SHA1-as-decimal string is
 pycryptopp is 48 (49 max). So we should be able to build the current
 pycryptopp on systems with up to 78-character usernames. If git moves to
 SHA256 and Zooko keeps the same versioning scheme (SHA256-as-decimal would
 be 78 characters), then we can tolerate usernames of up to 48 characters.

--
Ticket URL: <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2739#comment:1>
Tahoe-LAFS <https://Tahoe-LAFS.org>
secure decentralized storage