pycryptopp appears to have problems with amd64 linux systems (or gcc 4.1.3).

[faried]

On my 64bit Ubuntu 7.10 box (gcc 4.1.3), I am able to build tahoe from trunk, but it fails some of the tests related to pycryptopp. The errors look like

aes.Error: Precondition violation: key size is expected to be the default for AES, which is 16, but a key of size 16 was provided.

The error is from aesmodule.cpp:AES_init().

I examined allmydata.org/source/pycryptopp, and found that its AES_init() is slightly different. It has an updated format string (uses %zu to print size_t variables, instead of %d), and stores the DEFAULT_KEYSIZE as a local variable, instead of casting it inside the if() statement. I plugged the changes into tahoe's copy, and got a different error:

aes.Error: Precondition violation: key size is expected to be the default for AES, which is 16, but a key of size 47897475285008 was provided.

The standalone pycryptopp passes all its tests. What's probably needed is a small test case to locate the problem (is it in pycryptopp or libcrypto++?).

[faried]

make test TEST=allmydata.test.test_mutable

[faried]

make test TEST=allmydata.test.test_system

[faried]

make test TEST=allmydata.test.test_encode runs without errors. So, probably not pycryptopp?

[faried]

make test

[faried]

valgrind -v /usr/bin/python /usr/bin/trial --rterrors allmydata.test.test_mutable

[faried]

With today's pycryptopp, make test reveals just one error:

test_hexdigest (pycryptopp.test.test_sha256.SHA256) ... ERROR

====================================================================== ERROR: test_hexdigest (pycryptopp.test.test_sha256.SHA256) ---------------------------------------------------------------------- Traceback (most recent call last):

File "/t78/repos/pycryptopp/pycryptopp/test/test_sha256.py", line 40, in test_hexdigest

empty_hexdigest = sha256.SHA256().hexdigest()

AttributeError?: 'sha256.SHA256' object has no attribute 'hexdigest'

---------------------------------------------------------------------- Ran 27 tests in 1.386s

FAILED (errors=1)

If I build with "./setup.py build --debug", I get more errors; this may be due to Ubuntu or python bugs (I do have python-dbg installed). See attached file pycryptopptests for details.

[faried]

builds/tests of pycryptopp

[faried]

Two of the tests that fail with --debug are in test_aes. See val-test-aes-debug for the output of the test and the output of valgrind run. Also attached: val-test-aes for the same for pycryptopp built without --debug.

[faried]

python pycryptopp/test/test_aes.py, with and without valgrind

[faried]

python pycryptopp/test/test_aes.py (pycryptopp built with --debug), with and without valgrind

[zooko]

The thing about "no attribute: hexdigest" is actually not an error -- it is a "TODO" test. That is, it is a unit test which elicits a failure from the underlying pycryptopp code, but we know that it will fail because I haven't added a ".hexdigest()" method to the SHA256 class yet. Make sense?

It is unfortunate that people who are not already familiar with the twisted trial "TODO" / "SKIP" feature (and occasionally even people who are) perceive TODO or SKIP tests as revealing a bug. This issue has been irritating me recently in allmydata, too.

[zooko]

Heh. See these lines in your valgrind output?

==24079== Use of uninitialised value of size 8
==24079==    at 0x448B70: PyObject_Free (obmalloc.c:920)
==24079==    by 0x4CCB1A: code_dealloc (codeobject.c:260)
==24079==    by 0x4A0942: load_source_module (import.c:960)
==24079==    by 0x4A0FA0: import_submodule (import.c:2400)

I'll bet if you look into the Python suppressions file that came with your Linux distro's package of valgrind, you'll see that it suppresses those when the value is of size 4. This means that it successfully silences these bogus warnings on 32-bit architectures. :-) If you edit the .supp file (or cp it and edit the new one) to say "8" instead of "4" then I think all those warnings will be silenced.

(And of course, you might submit a patch to your Linux distro.)

[zooko]

Ah, that failing test that expects ".hexdigest()" wasn't actually set to TODO. So this is an example of the TODO formatting confusing someone who is familiar with it (me) rather than someone who isn't. ;-)

[warner]

FWIW, TODO items are *supposed* to cause some amount of unease, to encourage developers "TO DO" them sooner or later. I suppose there's a balance to be struck.. I find I don't use the .todo flag unless I expect to fix the issue shortly, and when the test is complicated enough that I want to record the thinking and effort that went into constructing the test. In other cases, I tend to put off writing the test until I've added the new functionality.

[zooko]

fixed by pycryptopp-0.3.0. Thanks!