[tahoe-lafs-trac-stream] [Tahoe-LAFS] #1698: the preferred cmdline to run tests under coverage is pretty huge, and comes in two flavors due to Debian renaming of the script

[Tahoe-LAFS]

#1698: the preferred cmdline to run tests under coverage is pretty huge, and comes
in two flavors due to Debian renaming of the script
----------------------------+-------------------------------------
     Reporter:  davidsarah  |      Owner:  daira
         Type:  defect      |     Status:  assigned
     Priority:  normal      |  Milestone:  1.12.0
    Component:  code        |    Version:  1.9.1
   Resolution:              |   Keywords:  usability test coverage
Launchpad Bug:              |
----------------------------+-------------------------------------

Comment (by warner):

 I've confirmed that running under the `coverage` executable gets us good
 data. When we first wrote this coverage code, there was no such
 executable, so we had to use that bwverbose-coverage plugin (and several
 related ones), to execute the `coverage.start()` and `.stop()` calls at
 the right times. In general, it's awesome that the coverage tool is so
 much easier to use now.

 But, if the executable doesn't have a predictable name, and if we want a
 "tahoe debug trial --coverage" to be responsible for locating it, then
 we're back to needing to run code at startup time, which locates the
 executable, and then shells out to the executable (which, we've learned,
 is usually a bad idea: One Process Is Best). It looks like `setup.py`
 `Trial` is currently doing that anyways, so maybe it wouldn't be too
 bad.

 What would the command be? We want to make sure we run tests with the
 same Python that was used to invoke `bin/tahoe`, which means we can't
 use `coverage` as the argv0. Hm, so the function would need to scan
 $PATH for "coverage" or "python-coverage", then use it as `argv[1]` in
 `["python", "coverage", ARGS..]`.

 I'm not super-comfortable about having one python program hunt around in
 $PATH to figure out which other python program to run.. it feels weird
 to do that at this level. Maybe this is something the Makefile should
 do? Maybe it could use `which coverage` and `which python-coverage` to
 decide upon the executable, then build up the bin/tahoe command to run
 underneath.

 BTW, why does your preferred command run `bin/tahoe @coverage run`
 instead of `coverage run bin/tahoe`? Are you assuming that `coverage`
 will be installed in `support/`, and therefore requires that `sys.path`
 and $PYTHONPATH and $PATH and whatnot be provided by a `bin/tahoe`
 parent process? If that's really a requirement, then
 `support/bin/coverage` will exist, and we can just use it instead of an
 OS-supplied coverage/python-coverage executable.

--
Ticket URL: <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1698#comment:10>
Tahoe-LAFS <https://Tahoe-LAFS.org>
secure decentralized storage