#953 closed defect (fixed)

test and support the dependency on pycrypto

Reported by: zooko Owned by: terrell
Priority: major Milestone: 1.9.0
Component: packaging Version: 1.6.0
Keywords: pycrypto-lib test sftp manhole Cc: bewst
Launchpad Bug:

Description

We added a dependency on pycrypto for the SFTP implementation in 4143cd1786c38406, but there aren't tests of this dependency, and Terrell reported on IRC that it failed to compile on his OSX 10.5. We're going to remove the dependency in order to release v1.6.1, then put it back with proper tests and support for v1.7.

Change History (27)

comment:1 Changed at 2010-02-15T05:16:40Z by zooko

Commented-out the dependency in 784b208708f5e515.

comment:2 follow-up: Changed at 2010-02-15T19:22:44Z by davidsarah

  • Keywords pycrypto-lib test sftp manhole added

Fixing this entails fixing #775 and #552. See also #774.

comment:3 in reply to: ↑ 2 Changed at 2010-02-16T04:34:54Z by davidsarah

Replying to davidsarah:

Fixing this entails fixing #775 and #552.

That reference was supposed to be #522 (mac buildslave produces builds lacking pycrypto).

comment:4 Changed at 2010-04-12T17:34:01Z by zooko

  • Owner changed from somebody to zooko
  • Status changed from new to assigned

comment:5 Changed at 2010-04-12T17:42:17Z by zooko

  • Owner changed from zooko to somebody
  • Status changed from assigned to new

You know what? I'm not really motivated to work on this ticket right now. I don't currently use FTP or SFTP and, while I really want Tahoe-LAFS to have first-class support for SFTP, I'm not the right person to do this testing and packaging. Could someone who really cares about SFTP please take this ticket?

comment:6 Changed at 2010-04-12T22:56:08Z by davidsarah

  • Milestone changed from 1.7.0 to eventually

I care about SFTP, but don't have Mac OS X. Someone who does should first investigate #522.

comment:7 Changed at 2010-04-12T23:12:33Z by davidsarah

  • Milestone changed from eventually to 1.7.0
  • Owner changed from somebody to davidsarah
  • Status changed from new to assigned

We do need this for 1.7 if we want SFTP to be ready for that release.

comment:8 follow-up: Changed at 2010-04-13T17:16:59Z by davidsarah

OK, #522 is a red herring.

Terrell reported on IRC that [pycrypto] failed to compile on his OSX 10.5

Does someone with OS X want to reapply the change in 4143cd1786c38406 and test this?

comment:9 in reply to: ↑ 8 Changed at 2010-04-13T18:34:06Z by davidsarah

  • Owner changed from davidsarah to terrell
  • Status changed from assigned to new

Replying to davidsarah:

Terrell reported on IRC that [pycrypto] failed to compile on his OSX 10.5

Does someone with OS X want to reapply the change in 4143cd1786c38406 and test this?

Assigning to terrell to test this, and post the build output if it fails.

comment:10 Changed at 2010-04-14T03:39:03Z by terrell

I have uncommented 784b208708f5e515 which reinstates 4143cd1786c38406.

This time, I had no trouble. Still on 10.5.8. I have recovered from a hard drive crash since the original bug was filed (via Time Machine).

I cannot reproduce the original compile failure. However, I did see some signedness complaints and have pasted them below.

before:

[11:05:25:trel:~/Desktop/tahoestuff/tahoefromdarcs] tahoe -V
allmydata-tahoe: 1.6.1-r4264, foolscap: 0.4.2, pycryptopp: 0.5.17-r683, zfec: 1.4.5, Twisted: 2.5.0, Nevow: 0.9.33-rUnknown, zope.interface: 3.3.0, python: 2.5.1, platform: Darwin-9.8.0-i386-32bit, sqlite: 3.4.0, simplejson: 2.0.9, argparse: 0.9.1, pyOpenSSL: 0.6, pyutil: 1.3.34, zbase32: 1.1.1, setuptools: 0.6c12dev, pysqlite: 2.3.2

the edit and recompile:

[11:05:34:trel:~/Desktop/tahoestuff/tahoefromdarcs] nano _auto_deps.py
[11:06:20:trel:~/Desktop/tahoestuff/tahoefromdarcs] make
python setup.py build
running darcsver
running develop
Not found: tahoe-deps
Not found: ../tahoe-deps
running egg_info
writing requirements to src/allmydata_tahoe.egg-info/requires.txt
writing src/allmydata_tahoe.egg-info/PKG-INFO
writing top-level names to src/allmydata_tahoe.egg-info/top_level.txt
writing dependency_links to src/allmydata_tahoe.egg-info/dependency_links.txt
writing entry points to src/allmydata_tahoe.egg-info/entry_points.txt
writing manifest file 'src/allmydata_tahoe.egg-info/SOURCES.txt'
running build_ext
Creating /Users/trel/Desktop/tahoestuff/tahoefromdarcs/support/lib/python2.5/site-packages/allmydata-tahoe.egg-link (link to src)
allmydata-tahoe 1.6.1-r4264 is already the active version in easy-install.pth
Installing tahoe script to support/bin

Installed /Users/trel/Desktop/tahoestuff/tahoefromdarcs/src
Processing dependencies for allmydata-tahoe==1.6.1-r4264
Searching for pycrypto>=2.0.1
Reading http://allmydata.org/source/tahoe/deps/tahoe-dep-sdists/
Reading http://allmydata.org/source/tahoe/deps/tahoe-dep-eggs/
Reading http://pypi.python.org/simple/pycrypto/
Reading http://pycrypto.sourceforge.net
Reading http://www.amk.ca/python/code/crypto
Best match: pycrypto 2.0.1
Downloading http://www.amk.ca/files/python/crypto/pycrypto-2.0.1.tar.gz
Processing pycrypto-2.0.1.tar.gz
Running pycrypto-2.0.1/setup.py -q bdist_egg --dist-dir /var/folders/+s/+sWRqHcIGuCdNeu1OoUAx++++TI/-Tmp-/easy_install-UjRNRk/pycrypto-2.0.1/egg-dist-tmp-muIPlg
src/MD2.c: In function ‘hash_digest’:
src/MD2.c:115: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/MD2.c: In function ‘hash_digest’:
src/MD2.c:115: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
In file included from src/MD2.c:118:
src/hash_template.c: In function ‘ALG_hexdigest’:
src/hash_template.c:108: warning: pointer targets in assignment differ in signedness
src/hash_template.c:112: warning: pointer targets in assignment differ in signedness
In file included from src/MD2.c:118:
src/hash_template.c: In function ‘ALG_hexdigest’:
src/hash_template.c:108: warning: pointer targets in assignment differ in signedness
src/hash_template.c:112: warning: pointer targets in assignment differ in signedness
src/MD4.c: In function ‘hash_digest’:
src/MD4.c:200: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
In file included from src/MD4.c:203:
src/hash_template.c: In function ‘ALG_hexdigest’:
src/hash_template.c:108: warning: pointer targets in assignment differ in signedness
src/hash_template.c:112: warning: pointer targets in assignment differ in signedness
src/MD4.c: In function ‘hash_digest’:
src/MD4.c:200: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
In file included from src/MD4.c:203:
src/hash_template.c: In function ‘ALG_hexdigest’:
src/hash_template.c:108: warning: pointer targets in assignment differ in signedness
src/hash_template.c:112: warning: pointer targets in assignment differ in signedness
src/RIPEMD.c: In function ‘hash_digest’:
src/RIPEMD.c:218: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/RIPEMD.c: In function ‘hash_digest’:
src/RIPEMD.c:218: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
In file included from src/RIPEMD.c:507:
src/hash_template.c: In function ‘ALG_hexdigest’:
src/hash_template.c:108: warning: pointer targets in assignment differ in signedness
src/hash_template.c:112: warning: pointer targets in assignment differ in signedness
src/hash_template.c: In function ‘ALG_update’:
src/hash_template.c:139: warning: pointer targets in passing argument 2 of ‘hash_update’ differ in signedness
src/hash_template.c: In function ‘ALG_new’:
src/hash_template.c:211: warning: pointer targets in passing argument 2 of ‘hash_update’ differ in signedness
In file included from src/RIPEMD.c:507:
src/hash_template.c: In function ‘ALG_hexdigest’:
src/hash_template.c:108: warning: pointer targets in assignment differ in signedness
src/hash_template.c:112: warning: pointer targets in assignment differ in signedness
src/hash_template.c: In function ‘ALG_update’:
src/hash_template.c:139: warning: pointer targets in passing argument 2 of ‘hash_update’ differ in signedness
src/hash_template.c: In function ‘ALG_new’:
src/hash_template.c:211: warning: pointer targets in passing argument 2 of ‘hash_update’ differ in signedness
src/SHA256.c: In function ‘hash_digest’:
src/SHA256.c:197: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
In file included from src/SHA256.c:200:
src/hash_template.c: In function ‘ALG_hexdigest’:
src/hash_template.c:108: warning: pointer targets in assignment differ in signedness
src/hash_template.c:112: warning: pointer targets in assignment differ in signedness
src/SHA256.c: In function ‘hash_digest’:
src/SHA256.c:197: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
In file included from src/SHA256.c:200:
src/hash_template.c: In function ‘ALG_hexdigest’:
src/hash_template.c:108: warning: pointer targets in assignment differ in signedness
src/hash_template.c:112: warning: pointer targets in assignment differ in signedness
In file included from src/AES.c:1459:
src/block_template.c: In function ‘ALG_Encrypt’:
src/block_template.c:371: warning: pointer targets in passing argument 2 of ‘block_encrypt’ differ in signedness
src/block_template.c:388: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
In file included from src/AES.c:1459:
src/block_template.c: In function ‘ALG_Encrypt’:
src/block_template.c:371: warning: pointer targets in passing argument 2 of ‘block_encrypt’ differ in signedness
src/block_template.c:388: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/block_template.c: In function ‘ALG_Decrypt’:
src/block_template.c:561: warning: pointer targets in passing argument 2 of ‘block_encrypt’ differ in signedness
src/block_template.c:578: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/block_template.c: In function ‘ALG_Decrypt’:
src/block_template.c:561: warning: pointer targets in passing argument 2 of ‘block_encrypt’ differ in signedness
src/block_template.c:578: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/block_template.c: In function ‘ALGgetattr’:
src/block_template.c:680: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/block_template.c: In function ‘ALGgetattr’:
src/block_template.c:680: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
In file included from src/ARC2.c:185:
src/block_template.c: In function ‘ALG_Encrypt’:
src/block_template.c:371: warning: pointer targets in passing argument 2 of ‘block_encrypt’ differ in signedness
src/block_template.c:388: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/block_template.c: In function ‘ALG_Decrypt’:
src/block_template.c:561: warning: pointer targets in passing argument 2 of ‘block_encrypt’ differ in signedness
src/block_template.c:578: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
In file included from src/ARC2.c:185:
src/block_template.c: In function ‘ALG_Encrypt’:
src/block_template.c:371: warning: pointer targets in passing argument 2 of ‘block_encrypt’ differ in signedness
src/block_template.c:388: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/block_template.c: In function ‘ALGgetattr’:
src/block_template.c:680: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/block_template.c: In function ‘ALG_Decrypt’:
src/block_template.c:561: warning: pointer targets in passing argument 2 of ‘block_encrypt’ differ in signedness
src/block_template.c:578: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/block_template.c: In function ‘ALGgetattr’:
src/block_template.c:680: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
In file included from src/Blowfish.c:499:
src/block_template.c: In function ‘ALG_Encrypt’:
src/block_template.c:371: warning: pointer targets in passing argument 2 of ‘block_encrypt’ differ in signedness
src/block_template.c:388: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/block_template.c: In function ‘ALG_Decrypt’:
src/block_template.c:561: warning: pointer targets in passing argument 2 of ‘block_encrypt’ differ in signedness
src/block_template.c:578: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/block_template.c: In function ‘ALGgetattr’:
src/block_template.c:680: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
In file included from src/Blowfish.c:499:
src/block_template.c: In function ‘ALG_Encrypt’:
src/block_template.c:371: warning: pointer targets in passing argument 2 of ‘block_encrypt’ differ in signedness
src/block_template.c:388: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/block_template.c: In function ‘ALG_Decrypt’:
src/block_template.c:561: warning: pointer targets in passing argument 2 of ‘block_encrypt’ differ in signedness
src/block_template.c:578: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/block_template.c: In function ‘ALGgetattr’:
src/block_template.c:680: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
In file included from src/CAST.c:436:
src/block_template.c: In function ‘ALG_Encrypt’:
src/block_template.c:371: warning: pointer targets in passing argument 2 of ‘block_encrypt’ differ in signedness
src/block_template.c:388: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/block_template.c: In function ‘ALG_Decrypt’:
src/block_template.c:561: warning: pointer targets in passing argument 2 of ‘block_encrypt’ differ in signedness
src/block_template.c:578: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/block_template.c: In function ‘ALGgetattr’:
src/block_template.c:680: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
In file included from src/CAST.c:436:
src/block_template.c: In function ‘ALG_Encrypt’:
src/block_template.c:371: warning: pointer targets in passing argument 2 of ‘block_encrypt’ differ in signedness
src/block_template.c:388: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/block_template.c: In function ‘ALG_Decrypt’:
src/block_template.c:561: warning: pointer targets in passing argument 2 of ‘block_encrypt’ differ in signedness
src/block_template.c:578: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/block_template.c: In function ‘ALGgetattr’:
src/block_template.c:680: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
In file included from src/DES.c:665:
src/block_template.c: In function ‘ALG_Encrypt’:
src/block_template.c:371: warning: pointer targets in passing argument 2 of ‘block_encrypt’ differ in signedness
src/block_template.c:388: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
In file included from src/DES.c:665:
src/block_template.c: In function ‘ALG_Encrypt’:
src/block_template.c:371: warning: pointer targets in passing argument 2 of ‘block_encrypt’ differ in signedness
src/block_template.c:388: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/block_template.c: In function ‘ALG_Decrypt’:
src/block_template.c:561: warning: pointer targets in passing argument 2 of ‘block_encrypt’ differ in signedness
src/block_template.c:578: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/block_template.c: In function ‘ALG_Decrypt’:
src/block_template.c:561: warning: pointer targets in passing argument 2 of ‘block_encrypt’ differ in signedness
src/block_template.c:578: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/block_template.c: In function ‘ALGgetattr’:
src/block_template.c:680: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/block_template.c: In function ‘ALGgetattr’:
src/block_template.c:680: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
In file included from src/DES3.c:688:
src/block_template.c: In function ‘ALG_Encrypt’:
src/block_template.c:371: warning: pointer targets in passing argument 2 of ‘block_encrypt’ differ in signedness
src/block_template.c:388: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
In file included from src/DES3.c:688:
src/block_template.c: In function ‘ALG_Encrypt’:
src/block_template.c:371: warning: pointer targets in passing argument 2 of ‘block_encrypt’ differ in signedness
src/block_template.c:388: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/block_template.c: In function ‘ALG_Decrypt’:
src/block_template.c:561: warning: pointer targets in passing argument 2 of ‘block_encrypt’ differ in signedness
src/block_template.c:578: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/block_template.c: In function ‘ALG_Decrypt’:
src/block_template.c:561: warning: pointer targets in passing argument 2 of ‘block_encrypt’ differ in signedness
src/block_template.c:578: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/block_template.c: In function ‘ALGgetattr’:
src/block_template.c:680: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/block_template.c: In function ‘ALGgetattr’:
src/block_template.c:680: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
In file included from src/IDEA.c:196:
src/block_template.c: In function ‘ALG_Encrypt’:
src/block_template.c:371: warning: pointer targets in passing argument 2 of ‘block_encrypt’ differ in signedness
In file included from src/IDEA.c:196:
src/block_template.c: In function ‘ALG_Encrypt’:src/block_template.c:388: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness

src/block_template.c:371: warning: pointer targets in passing argument 2 of ‘block_encrypt’ differ in signedness
src/block_template.c:388: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/block_template.c: In function ‘ALG_Decrypt’:
src/block_template.c:561: warning: pointer targets in passing argument 2 of ‘block_encrypt’ differ in signedness
src/block_template.c:578: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/block_template.c: In function ‘ALGgetattr’:
src/block_template.c:680: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/block_template.c: In function ‘ALG_Decrypt’:
src/block_template.c:561: warning: pointer targets in passing argument 2 of ‘block_encrypt’ differ in signedness
src/block_template.c:578: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/block_template.c: In function ‘ALGgetattr’:
src/block_template.c:680: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
In file included from src/RC5.c:212:
src/block_template.c: In function ‘ALG_Encrypt’:
src/block_template.c:371: warning: pointer targets in passing argument 2 of ‘block_encrypt’ differ in signedness
In file included from src/RC5.c:212:
src/block_template.c: In function ‘ALG_Encrypt’:
src/block_template.c:371: warning: pointer targets in passing argument 2 of ‘block_encrypt’ differ in signedness
src/block_template.c:388: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/block_template.c:388: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/block_template.c: In function ‘ALG_Decrypt’:src/block_template.c: In function ‘ALG_Decrypt’:
src/block_template.c:561: warning: pointer targets in passing argument 2 of ‘block_encrypt’ differ in signedness
src/block_template.c:561: warning: pointer targets in passing argument 2 of ‘block_encrypt’ differ in signedness

src/block_template.c:578: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/block_template.c:578: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/block_template.c: In function ‘ALGgetattr’:
src/block_template.c:680: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/block_template.c: In function ‘ALGgetattr’:
src/block_template.c:680: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
In file included from src/ARC4.c:70:
src/stream_template.c: In function ‘ALG_Encrypt’:
src/stream_template.c:135: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/stream_template.c: In function ‘ALG_Decrypt’:
src/stream_template.c:164: warning: pointer targets in passing argument 2 of ‘stream_encrypt’ differ in signedness
In file included from src/ARC4.c:70:
src/stream_template.c: In function ‘ALG_Encrypt’:
src/stream_template.c:135: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/stream_template.c: In function ‘ALG_Decrypt’:
src/stream_template.c:164: warning: pointer targets in passing argument 2 of ‘stream_encrypt’ differ in signedness
In file included from src/XOR.c:52:
src/stream_template.c: In function ‘ALG_Encrypt’:
src/stream_template.c:135: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/stream_template.c: In function ‘ALG_Decrypt’:
src/stream_template.c:164: warning: pointer targets in passing argument 2 of ‘stream_encrypt’ differ in signedness
In file included from src/XOR.c:52:
src/stream_template.c: In function ‘ALG_Encrypt’:
src/stream_template.c:135: warning: pointer targets in passing argument 1 of ‘PyString_FromStringAndSize’ differ in signedness
src/stream_template.c: In function ‘ALG_Decrypt’:
src/stream_template.c:164: warning: pointer targets in passing argument 2 of ‘stream_encrypt’ differ in signedness
zip_safe flag not set; analyzing archive contents...
Adding pycrypto 2.0.1 to easy-install.pth file

Installed /Users/trel/Desktop/tahoestuff/tahoefromdarcs/support/lib/python2.5/site-packages/pycrypto-2.0.1-py2.5-macosx-10.5-i386.egg
Searching for pycryptopp==0.5.17-r683
Best match: pycryptopp 0.5.17-r683
Downloading http://allmydata.org/source/tahoe/deps/tahoe-dep-eggs/pycryptopp-0.5.17_r683-py2.5-macosx-10.5-i386.egg
Processing pycryptopp-0.5.17_r683-py2.5-macosx-10.5-i386.egg
removing '/Users/trel/Desktop/tahoestuff/tahoefromdarcs/support/lib/python2.5/site-packages/pycryptopp-0.5.17_r683-py2.5-macosx-10.5-i386.egg' (and everything under it)
creating /Users/trel/Desktop/tahoestuff/tahoefromdarcs/support/lib/python2.5/site-packages/pycryptopp-0.5.17_r683-py2.5-macosx-10.5-i386.egg
Extracting pycryptopp-0.5.17_r683-py2.5-macosx-10.5-i386.egg to /Users/trel/Desktop/tahoestuff/tahoefromdarcs/support/lib/python2.5/site-packages
pycryptopp 0.5.17-r683 is already the active version in easy-install.pth

Installed /Users/trel/Desktop/tahoestuff/tahoefromdarcs/support/lib/python2.5/site-packages/pycryptopp-0.5.17_r683-py2.5-macosx-10.5-i386.egg
Searching for Nevow==0.9.33-rUnknown
Best match: Nevow 0.9.33-rUnknown
Processing Nevow-0.9.33_rUnknown-py2.5.egg
Nevow 0.9.33-rUnknown is already the active version in easy-install.pth
Installing nevow-xmlgettext script to support/bin
Installing nit script to support/bin

Using /Users/trel/Desktop/tahoestuff/tahoefromdarcs/support/lib/python2.5/site-packages/Nevow-0.9.33_rUnknown-py2.5.egg
Searching for foolscap==0.4.2
Best match: foolscap 0.4.2
Processing foolscap-0.4.2-py2.5.egg
foolscap 0.4.2 is already the active version in easy-install.pth
Installing flappclient script to support/bin
Installing flappserver script to support/bin
Installing flogtool script to support/bin

Using /Users/trel/Desktop/tahoestuff/tahoefromdarcs/support/lib/python2.5/site-packages/foolscap-0.4.2-py2.5.egg
Searching for Twisted==2.5.0
Best match: Twisted 2.5.0
Twisted 2.5.0 is already the active version in easy-install.pth

Using /System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python
Searching for zope.interface==3.3.0
Best match: zope.interface 3.3.0
zope.interface 3.3.0 is already the active version in easy-install.pth

Using /System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python
Searching for simplejson==2.0.9
Best match: simplejson 2.0.9
Processing simplejson-2.0.9-py2.5-macosx-10.5-i386.egg
simplejson 2.0.9 is already the active version in easy-install.pth

Using /Users/trel/Desktop/tahoestuff/tahoefromdarcs/support/lib/python2.5/site-packages/simplejson-2.0.9-py2.5-macosx-10.5-i386.egg
Searching for zfec==1.4.5
Best match: zfec 1.4.5
Processing zfec-1.4.5-py2.5-macosx-10.5-i386.egg
zfec 1.4.5 is already the active version in easy-install.pth
Installing zfec script to support/bin
Installing zunfec script to support/bin

Using /Users/trel/Desktop/tahoestuff/tahoefromdarcs/support/lib/python2.5/site-packages/zfec-1.4.5-py2.5-macosx-10.5-i386.egg
Searching for setuptools==0.6c12dev
Best match: setuptools 0.6c12dev
Processing setuptools-0.6c12dev.egg
removing '/Users/trel/Desktop/tahoestuff/tahoefromdarcs/support/lib/python2.5/site-packages/setuptools-0.6c12dev.egg' (and everything under it)
creating /Users/trel/Desktop/tahoestuff/tahoefromdarcs/support/lib/python2.5/site-packages/setuptools-0.6c12dev.egg
Extracting setuptools-0.6c12dev.egg to /Users/trel/Desktop/tahoestuff/tahoefromdarcs/support/lib/python2.5/site-packages
Removing setuptools 0.6c7 from easy-install.pth file
setuptools 0.6c12dev is already the active version in easy-install.pth
Installing easy_install script to support/bin
Installing easy_install-2.5 script to support/bin

Installed /Users/trel/Desktop/tahoestuff/tahoefromdarcs/support/lib/python2.5/site-packages/setuptools-0.6c12dev.egg
Searching for pyOpenSSL==0.6
Best match: pyOpenSSL 0.6
Adding pyOpenSSL 0.6 to easy-install.pth file

Using /System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python
Searching for pyutil==1.3.34
Best match: pyutil 1.3.34
Processing pyutil-1.3.34-py2.5.egg
pyutil 1.3.34 is already the active version in easy-install.pth
Installing randfile script to support/bin
Installing adler32sum script to support/bin
Installing sha256dsum script to support/bin
Installing lines script to support/bin
Installing sha1sum_py script to support/bin
Installing unsort script to support/bin
Installing sha256sum_py script to support/bin
Installing randcookie script to support/bin
Installing sha256dcppsum script to support/bin
Installing tailx script to support/bin

Using /Users/trel/Desktop/tahoestuff/tahoefromdarcs/support/lib/python2.5/site-packages/pyutil-1.3.34-py2.5.egg
Searching for argparse==0.9.1
Best match: argparse 0.9.1
Processing argparse-0.9.1-py2.5.egg
argparse 0.9.1 is already the active version in easy-install.pth

Using /Users/trel/Desktop/tahoestuff/tahoefromdarcs/support/lib/python2.5/site-packages/argparse-0.9.1-py2.5.egg
Searching for zbase32==1.1.1
Best match: zbase32 1.1.1
Processing zbase32-1.1.1-py2.5.egg
zbase32 1.1.1 is already the active version in easy-install.pth

Using /Users/trel/Desktop/tahoestuff/tahoefromdarcs/support/lib/python2.5/site-packages/zbase32-1.1.1-py2.5.egg
Finished processing dependencies for allmydata-tahoe==1.6.1-r4264
running make_executable
running build
running build_py
copying src/allmydata/_auto_deps.py -> build/lib/allmydata
copying src/allmydata/_version.py -> build/lib/allmydata
touch .built

and after:

[11:07:47:trel:~/Desktop/tahoestuff/tahoefromdarcs] tahoe -V
allmydata-tahoe: 1.6.1-r4264, foolscap: 0.4.2, pycryptopp: 0.5.17-r683, zfec: 1.4.5, Twisted: 2.5.0, Nevow: 0.9.33-rUnknown, zope.interface: 3.3.0, python: 2.5.1, platform: Darwin-9.8.0-i386-32bit, sqlite: 3.4.0, simplejson: 2.0.9, argparse: 0.9.1, pycrypto: 2.0.1, pyOpenSSL: 0.6, pyutil: 1.3.34, zbase32: 1.1.1, setuptools: 0.6c12dev, pysqlite: 2.3.2

comment:11 Changed at 2010-05-11T15:41:11Z by zooko

This is a requirement for SFTP for v1.7.

comment:12 Changed at 2010-05-11T19:26:54Z by davidsarah

Signedness warnings are an indication of sloppy C coding, but rarely of an actual problem on a twos-complement machine (which all machines are these days).

The automated SFTP tests planned for 1.7 will not test that pycrypto actually works; only that it can be imported. This is because they are no-network, no-encryption tests. Manual testing of SFTP would however confirm that pycrypto works well enough for 1.7. We do need automated end-to-end tests of SFTP (i.e. over a localhost SSH connection), but I don't think that's going to happen for 1.7.

comment:13 follow-up: Changed at 2010-05-14T05:28:05Z by zooko

Okay, so it sounds like we're going to add back in the dependency on pycrypto, relying on our current unit tests and buildbots to let us know if pycrypto cannot be imported. In the future we will add tests that detect badly compiled pycryptos, pycryptos with bugs, etc..

comment:14 in reply to: ↑ 13 Changed at 2010-05-14T06:05:42Z by davidsarah

Replying to zooko:

Okay, so it sounds like we're going to add back in the dependency on pycrypto, relying on our current unit tests and buildbots to let us know if pycrypto cannot be imported.

Actually the current SFTP tests will be skipped, rather than failing, if pycrypto can't be imported. Maybe that is wrong.

In the future we will add tests that detect badly compiled pycryptos, pycryptos with bugs, etc..

Yes.

comment:15 Changed at 2010-05-26T14:19:14Z by zooko

The build of PyCrypto on NetBSD fails due to https://bugs.launchpad.net/pycrypto/+bug/518852 . Our options are:

  1. remove NetBSD from the list of platforms Supported for Tahoe-LAFS
  2. make the dependency on PyCrypto be conditioned on saying that you want the "SFTP" feature of Tahoe-LAFS, and users who are on NetBSD and have the option of installing without the "SFTP" extra feature
  3. delay Tahoe-LAFS v1.7.0 until there is a new release of PyCrypto which fixes this, and update our dependencies to require a version of PyCrypto at least that new; (We could also configure our dependencies to require PyCrypto >= that new version if you are building on NetBSD or on an older FreeBSD or require PyCrypto >= 2.0.1 if not.)
  4. leave NetBSD nominally listed in the "Supported" category but actually require all of our NetBSD users to manually patch their PyCrypto libraries before they can install Tahoe-LAFS
  5. ?

Are these all of the options? None of them seem that great. :-(

comment:16 Changed at 2010-05-26T14:20:06Z by zooko

  • Launchpad Bug set to 518852

comment:17 Changed at 2010-05-26T16:02:26Z by zooko

  • Cc bewst added

The build fails when we try the ticket1037 branch on David Abrahams's OpenSolaris builder: http://tahoe-lafs.org/buildbot/builders/David%20A.%20OpenSolaris%20i386/builds/196/steps/build/logs/stdio

It appears to fail due to the absence of gmp.h:

src/_fastmath.c:34:17: gmp.h: No such file or directory

Our options include:

  1. remove OpenSolaris from the list of Supported Platforms to run Tahoe-LAFS
  2. ask Dave Abrahams to manually install gmp.h and leave OpenSolaris listed as a Supported Platform
  3. remove PyCrypto from the standard set of requirements and create an "SFTP"-specific set of requirements which are installed only if users specify that they need the "SFTP" feature of Tahoe-LAFS
  4. try to help the PyCrypto maintainer, Dwayne Litzenberger patch his setup.py [*] and then wait for a new release of PyCrypto which has that patch.
  5. Do option 1 or 2 now while simultaneously working on option 4. People will not then be able to actually install Tahoe-LAFS on OpenSolaris machines that are configured the same way that Dave's is, but they will be able to install Tahoe-LAFS on OpenSolaris machines that are configured differently (with libgmp headers installed), and once a new version of PyCrypto comes out that fixes this problem then they will be able to install on any platform.
  6. ?

[*] It looks like to me that PyCrypto is intended to fallback to a pure-Python implementation (appropriately named "slowmath") when the _fastmath.c file can't be compiled. The fact that the build of PyCrypto fails on David's OpenSolaris machine appears to be a bug in PyCrypto's setup.py. I see from that file that it is checking whether there is a libgmp lib dir present and if so then relying on gmp.h. I infer that on David's OpenSolaris machine that there is a libgmp libdir but not the header files. This could be due to David having the library installed but not the headers installed, which is typical for a packaging system that offers a package of the library separate from the "-dev" package which has the headers. It could also just be a mistake on David's part or (more likely) on the part of the OpenSolaris maintainers. In any case, a more robust way to do this fall-back mechanism in your setup.py file is to attempt to compile the module that you need and then if you get an exception fallback to the pure-Python version. An example of that pattern is simplejson's setup.py. Perhaps the PyCrypto maintainers would like to switch to that pattern by copying the code from simplejson's setup.py. (By the way, this is an example of a more general pattern: Never look before you leap! Never check for good conditions and then proceed if conditions are good. Instead, attempt the thing and then handle failure if it fails. Never look before you leap! Instead just leap without looking and then handle failure gracefully. ;-) Okay so this rule applies better to build scripts than it does to, say, diving in a lake.) Here's a question: just how slow is the slowmath version? If it is too slow for the users of Tahoe-LAFS to rely on in the SFTP implementation, then the fact that PyCrypto does successfully fall back to slowmath on all other systems (that don't have libgmp installed) is a different issue for the Tahoe-LAFS packaging that we also need to deal with... I guess the next step is to write some simple benchmark of the relevant performance which we can run on the buildbots... I'll open a separate ticket for that issue.

comment:18 Changed at 2010-05-26T16:49:51Z by bewst

Suggestion: have the buildbots run a python script that prepares a virtualenv and installs all your dependencies. Then you can manage situations like this one without bugging slave maintainers. For an example, see http://github.com/ryppl/ryppl/blob/master/test/self_test.py

When I needed to fix bugs in dependencies, I just made them git submodules and did the fixes there until the patches appeared upstream. Not sure if this idea applies to this particular problem, but it worked for me.

comment:19 Changed at 2010-05-27T02:24:50Z by davidsarah

We could move FreeBSD, NetBSD, and OpenSolaris? into a new category of "Partially Supported" platforms, which require additional manual effort to get SFTP support working.

Ideally, Tahoe-LAFS should still build on those platforms even when that manual effort has not been expended. IOW, our setup.py should tolerate a failure to build the PyCrypto? dependency.

OTOH, if a new release of PyCrypto? can be cut quickly, then we might do options 3 and 4. Both of the fixes sound fairly straightforward.

comment:20 Changed at 2010-06-04T03:36:50Z by zooko

See also #1065 (host binary eggs of PyCrypto? for our supported platforms)

comment:21 Changed at 2010-06-18T23:32:01Z by zooko

  • Milestone changed from 1.7.0 to 1.7.1

I'm going to say that we can release 1.7, given that Tahoe-LAFS's use of PyCrypto does work on those platforms where PyCrypto can be installed, by hook or by crook.

This basically means that (once again) this Tahoe-LAFS is only really ready for people who are lucky on what platform they are on or how good they are at installing dependencies... :-/

Hopefully in Tahoe-LAFS v1.7.1 we can have real support for lots of platforms where the http://tahoe-lafs.org/source/tahoe-lafs/trunk/docs/quickstart.html Just Works for you (which means we've done a lot of work behind the scenes to provide binary packages for all the native-code dependencies: zfec, pycryptopp, pyOpenSSL, and PyCrypto).

comment:22 Changed at 2010-07-18T00:39:30Z by davidsarah

  • Milestone changed from 1.7.1 to 1.8.0

1.7.1 is now only for bugfixes, but I hope we'll have a really good bash at improving packaging for 1.8.

comment:23 Changed at 2010-08-14T06:45:45Z by zooko

  • Milestone changed from 1.8.0 to soon

comment:24 Changed at 2010-10-28T04:46:20Z by zooko

  • Summary changed from dependency on pycrypto needs to be tested and supported to test and support the dependency on pycrypto

I opened a bug report with FreeBSD ports asking for the to upgrade their port of PyCrypto from 2.2 to 2.3.

comment:25 Changed at 2010-10-28T05:07:05Z by zooko

Here's the FreeBSD ports issue ticket: http://www.freebsd.org/cgi/query-pr.cgi?pr=151791

comment:26 Changed at 2011-11-17T22:48:33Z by davidsarah

  • Launchpad Bug 518852 deleted

comment:27 Changed at 2012-10-17T21:26:22Z by zooko

  • Milestone changed from soon to 1.9.0
  • Resolution set to fixed
  • Status changed from new to closed

This is fixed! We have an explicit dependency on PyCrypto listed in the explicit dependencies in git/src/allmydata/_auto_deps.py, we have tests that will fail if PyCrypto is absent or raises an exception, and we have documentation, in such places as git/NEWS.rst and git/docs/frontends/FTP-and-SFTP.rst. (SFTP is the only reason we require PyCrypto.)

Note: See TracTickets for help on using tickets.