[tahoe-lafs-trac-stream] [Tahoe-LAFS] #20: unit tests take too long

[Tahoe-LAFS]

#20: unit tests take too long
------------------------+------------------------------
     Reporter:  zooko   |      Owner:  warner
         Type:  defect  |     Status:  new
     Priority:  major   |  Milestone:  eventually
    Component:  code    |    Version:  unknown
   Resolution:          |   Keywords:  test performance
Launchpad Bug:          |
------------------------+------------------------------

Old description:

> The best way to fix this would be to optimize the code under test so that
> it passes the tests faster.
>
> The second best way would be to optimize the tests so that they test the
> same thing but faster.
>
> The third best way would be to reduce the "replications, redundancy,
> random stabs in the dark" and so forth, and the tests of scaling.  (This
> cries out for another category of test that reports on scaling
> behavior...)
>
> The fourth best way would be to judiciously prune tests which have been
> superceded by more comprehensive tests and are now redundant.

New description:

 The best way to fix this would be to optimize the code under test so that
 it passes the tests faster.

 The second best way would be to optimize the tests so that they test the
 same thing but faster.

 The third best way would be to reduce the "replications, redundancy,
 random stabs in the dark" and so forth, and the tests of scaling.  (This
 cries out for another category of test that reports on scaling
 behavior...)

 The fourth best way would be to judiciously prune tests which have been
 superceded by more comprehensive tests and are now redundant.

--

Comment (by warner):

 I noticed today that the full test suite takes 30 minutes on my (slow)
 linux box. Part of the problem is the size of the files created by the
 tests: the `_trial_temp` directory has 340MB of data at the end of the
 run, which is kind of excessive.

 The worst offender is `test_mutable`, which creates 132MB. `test_cli`
 makes 76MB. The next heaviest ones are `test_system`, `test_download`,
 `test_dirnode`, `test_upload`, and `test_repairer`, which create 12MB-14MB
 each. There's another 6 that produce 5-10MB, and the rest are 3MB or less.

 Note: now that we're on tox, the fastest cycle time for a specific test
 (when you're doing a tight edit-test-coverage-repeat loop) is something
 like:

 * `.tox/py27/bin/trial allmydata.test.test_foo.Bar.test_blah`

 Or:

 * `source .tox/py27/bin/activate`
 * `coverage 'which trial' allmydata.test.test_foo.Bar.test_blah`

 That won't do anything extra (installs, updates, etc). I'm no longer
 concerned about having a `make quicktest` or the like: virtualenvs are
 good enough for me now.

--
Ticket URL: <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/20#comment:28>
Tahoe-LAFS <https://Tahoe-LAFS.org>
secure decentralized storage