[tahoe-lafs-trac-stream] [Tahoe-LAFS] #1698: the preferred cmdline to run tests under coverage is pretty huge, and comes in two flavors due to Debian renaming of the script

[Tahoe-LAFS]

#1698: the preferred cmdline to run tests under coverage is pretty huge, and comes
in two flavors due to Debian renaming of the script
----------------------------+-------------------------------------
     Reporter:  davidsarah  |      Owner:  daira
         Type:  defect      |     Status:  assigned
     Priority:  normal      |  Milestone:  1.12.0
    Component:  code        |    Version:  1.9.1
   Resolution:              |   Keywords:  usability test coverage
Launchpad Bug:              |
----------------------------+-------------------------------------

Comment (by daira):

 The `Makefile` isn't usable (for most users) on Windows. Therefore, I
 would like to avoid adding new stuff to the `Makefile` if it needs to work
 on Windows, which this does.

 Note that the argument to "`coverage run`" is a Python script, not an
 arbitrary executable. It will always be run in-process, i.e. in the same
 Python interpreter instance as `coverage`.

 The reason why "`coverage run bin/tahoe debug trial`" does not work is
 that `bin/tahoe` runs the "`debug trial`" command in a subprocess, in
 order to pass the correct `PYTHONPATH`. (Running it directly wouldn't
 help, because `coverage` does not have the right `PYTHONPATH` and so
 neither does any script it runs.) So `src/allmydata/__init__.py` complains
 and exits, as it should.

 When `coverage` is run from `bin/tahoe` using `@`, it gets the correct
 `PYTHONPATH`, and so does the script that it runs. This was one of the
 main motivations for adding `@`. Importantly, `bin/tahoe` will expand
 `@tahoe` to the absolute path of the support script --not its own path--
 before invoking `coverage`. If it expanded `@tahoe` to its own absolute
 path, then the tests would be running in a subprocess of `coverage` which
 wouldn't work for the same reason as above.

 Which Python interpreter is used to run `coverage` (and therefore the
 tests suite) will be determined by the value of `sys.executable` in
 `bin/tahoe`.

 Note that tests that the test suite runs in a subprocess, e.g.
 `test_system.SystemTest.test_filesystem_with_cli_in_subprocess`, will not
 contribute to coverage. I don't know how to fix this. `trialcoverage` also
 had that problem.

--
Ticket URL: <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1698#comment:13>
Tahoe-LAFS <https://Tahoe-LAFS.org>
secure decentralized storage