#406 assigned defect

end-to-end encoding self-test

Reported by: zooko Owned by: zooko
Priority: major Milestone: eventually
Component: code-encoding Version: 1.0.0
Keywords: test pycryptopp integrity Cc:
Launchpad Bug:

Description

Figure out what is going on with Ben Laurie's Tahoe node on his FreeBSD-6 using the public Test Grid, make it reproducible if possible in the unit tests, fix it.

Change History (20)

comment:1 Changed at 2008-05-05T21:07:14Z by zooko

  • Milestone changed from undecided to 1.0.1

comment:2 Changed at 2008-05-05T21:07:17Z by zooko

  • Status changed from new to assigned

comment:3 Changed at 2008-05-05T21:08:36Z by zooko

  • Milestone changed from 1.0.1 to 1.1.0

Milestone 1.0.1 deleted

comment:5 Changed at 2008-05-06T17:28:16Z by warner

I'll read over the thread more closely and see if I can reproduce the problem locally.

I had first thought that it might just be the cap being corrupted, but that doesn't seem likely, given that the storage index is derived from the read-key.

comment:6 Changed at 2008-05-06T20:05:24Z by warner

ah, ok, pycryptopp vs the stack. nevermind.

comment:7 Changed at 2008-05-09T19:14:23Z by zooko

I'm working on adding "end to end" decryption, decoding, and parsing checks, i.e. in src/allmydata/test/test_encode.py upload some known-good, fixed ciphertext instead of encrypting, and likewise for SSK's and dirnodes.

comment:8 Changed at 2008-05-12T19:47:08Z by zooko

  • Summary changed from data corruption (integrity failure) bug on Ben Laurie's machine to end-to-end encoding self-test

comment:9 Changed at 2008-06-04T01:13:33Z by zooko

  • Milestone changed from 1.1.0 to 1.1.1

comment:10 Changed at 2009-03-28T19:38:40Z by zooko

The next step is to see what tahoe on Zandr's lenny-armel box does. The pycryptopp on that box seems to be giving incorrect decryption. Do the unit tests of tahoe on that box detect this misbehavior from pycryptopp?

comment:11 Changed at 2009-04-01T14:20:04Z by zooko

  • Owner changed from zooko to zandr
  • Status changed from assigned to new

Assigning this to Zandr in case he wants to look at it before the next time I circle around and log into his Linksys NAS box...

The hope is that when tahoe is used with a broken pycryptopp that gives wrong answers, that the tahoe unit tests fail.

comment:12 Changed at 2009-04-07T15:59:12Z by zooko

  • Milestone changed from 1.4.0 to 1.4.1

Some tests on Zandr's lenny-armel box do indeed fail, such as allmydata.test.test_system.SystemTest.test_upload_and_download_random_key. See the buildbot results: http://allmydata.org/buildbot/builders/Zandr%20Lenny-armel/builds/37/steps/test/logs/stdio . However, these are failing because encrypting a string and then decrypting the ciphertext doesn't yield the original string. A subtler bug (and one that we've had before in pycryptopp) is when encrypting the string and decrypting the ciphertext does yield the original string, but only when you use the same (buggy) code for both encryption and decryption. This ticket is to add a "test vector", for example a fixed ciphertext, computed with a known-good implementation of AES and then hardcoded into the test file as a string literal, and assert that when you decrypt that fixed ciphertext you get the correct plaintext.

comment:13 Changed at 2009-12-13T05:17:09Z by zooko

  • Milestone changed from 1.6.0 to eventually
  • Owner changed from zandr to zooko
  • Status changed from new to assigned

I'm making some progress on this, but it doesn't need to be done before Tahoe-LAFS v1.6.

comment:14 Changed at 2010-01-15T20:24:54Z by davidsarah

  • Keywords test pycryptopp integrity added

comment:15 Changed at 2010-02-02T00:29:33Z by davidsarah

  • Milestone changed from eventually to 1.7.0

Our tests are in general quite comprehensive, but this is a glaring omission.

comment:16 Changed at 2010-03-17T00:48:09Z by davidsarah

  • Priority changed from major to critical

comment:17 Changed at 2010-04-29T16:13:43Z by zooko

I think we might have some tests to detect this now. Here's how to find out: mock out the AES class from pycryptopp (which is used in the immutable upload, mutable publish, immutable download, and mutable retrieve).

comment:18 Changed at 2010-04-29T16:15:23Z by zooko

…and then set the mock to encrypt-decrypt wrongly but consistently and then see if any tests fail. Oh, and I guess this is also half of the job of writing the tests that we need! (If they aren't already there.) Shouldn't be too hard...

comment:19 Changed at 2010-05-16T23:39:51Z by zooko

  • Milestone changed from 1.7.0 to eventually

comment:20 Changed at 2012-03-29T16:01:59Z by zooko

  • Priority changed from critical to major

I'm demoting this from critical to major, since there are other urgent known-bugs (mostly in mutables) that are marked critical and this ticket, while certainly important, hasn't been touched in years.

Note: See TracTickets for help on using tickets.