tests now run 40% faster
Brian Warner
warner at lothar.com
Sat Aug 6 18:11:40 UTC 2016
FYI, I just pushed a branch to master which speeds up the unit test
suite by roughly 40%. There were a couple of unnecessary things we were
doing:
* test_mutable was uploading around 900kB, I reduced that to 400kB
* test_mutable had some tests which exercised k=127/N=255, which took
forever and seemed unnecessary, so I removed them
* most tests that use GridTestMixin were changed to use k=N=1, instead
of the default k=3/N=10. (some tests exercise partially-uploaded
files, and thus require more than one share)
* we turned off the (new) control/log tubs in all GridTestMixin tests,
since they aren't used and their RSA key generation consumes
significant time
* I also split some large tests into multiple files, which doesn't
affect the speed but should make things more manageable in the future
The results are pretty cool:
* travis (linux) "trial" time went from 1711s to 1057s (a 40% speedup).
This is a 'tox -e coveraage' run, which includes branch coverage
checking, which is the most work that our tests can do. travis(OS-X),
which does a non-coverage test, had a similar speedup (from 2031s to
1369s, 32% speedup).
* our benchmark Ubuntu 15.10 buildslave (which runs on a pretty decent
Linode host) went from 1312s to 727s (45% speedup)
* The Raspberry Pi 3 buildslave, which couldn't even finish the test
suite before, now finishes in a mere 1.5 hours (5821s). A full 'tox -e
coverage' on the rpi3 (with all timeouts disabled) used to take 16412s
(4.5 hours). When I get a chance, I'll make a more direct comparison.
Some things we can do to further speed things up:
* make the mutable publish/update code accept a custom segment size. The
default 128KiB requires at least 384kB to exercise three-or-more
segments, but if tests could reduce that to e.g. 128 bytes, we could
exercise the same code paths with a tiny tiny amount of data. The
immutable code already does this, but mutable does not. (it's
non-trivial, but important, since mutations that modify just one
segment must obviously retain the uploaded file's segsize, and I'm not
sure we currently have code to modify mutable files with non-default
segsizes)
* test_system is still the longest single test: we should be able to
reduce its scope to make it faster
This all falls under ticket 20 [1].
cheers,
-Brian
[1]: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/20
More information about the tahoe-dev
mailing list