[tahoe-lafs-trac-stream] [Tahoe-LAFS] #1238: investigate using tox to run tests

[Tahoe-LAFS]

#1238: investigate using tox to run tests
---------------------------------+-----------------------------------------
     Reporter:  zooko            |      Owner:  somebody
         Type:  enhancement      |     Status:  new
     Priority:  major            |  Milestone:  undecided
    Component:  dev-             |    Version:  1.8.0
  infrastructure                 |
   Resolution:                   |   Keywords:  setuptools buildbot testing
Launchpad Bug:                   |
---------------------------------+-----------------------------------------

Old description:

> tox http://codespeak.net/tox/ offers:
>
>  * checking your package installs correctly with different Python
> versions and interpreters
>  * running your tests in each of the environments, configuring your test
> tool of choice
>  * acting as a frontend to Continous Integration servers, greatly
> reducing boilerplate and merging CI and shell-based testing.
>
> For one thing, we would like to test lots of different Python versions
> without necessarily maintaining a lot of different buildslaves, each of
> which is there solely to run a different Python version.

New description:

 tox http://codespeak.net/tox/ offers:

  * checking your package installs correctly with different Python versions
 and interpreters
  * running your tests in each of the environments, configuring your test
 tool of choice
  * acting as a frontend to Continous Integration servers, greatly reducing
 boilerplate and merging CI and shell-based testing.

 For one thing, we would like to test lots of different Python versions
 without necessarily maintaining a lot of different buildslaves, each of
 which is there solely to run a different Python version.

--

Comment (by warner):

 I've been playing with this, and I think it's the right way to go. In
 particular, we'll keep using trial for our tests (although meejah tells me
 that `py.test` knows how to handle Deferred-returning tests, and has some
 nice composable "fixtures", so maybe we should migrate towards that). But
 we'll use `tox` as the frontend, and our docs can say:

 {{{
 To run the unit test suite from your local source tree, run:

  tox
 }}}


 That will create a virtualenv (one for each supported version of python),
 install your local source (along with all dependencies) into the
 virtualenv, then run the tests. That means you don't need to create your
 own virtualenv for testing purposes (although you will if you want to
 actually run tahoe). The docs can provide more specialized instructions
 for running specific tests, installing development versions of
 dependencies (e.g. test current Tahoe trunk against current Twisted
 trunk), and getting code-coverage data.

 Something that I care about a lot: a very fast test-edit-repeat cycle is
 still possible, by populating the virtualenv with `--editable`, then
 running something like `.tox/py27/bin/trial allmydata.test.test_foo`.

--
Ticket URL: <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1238#comment:1>
Tahoe-LAFS <https://Tahoe-LAFS.org>
secure decentralized storage