[tahoe-lafs-trac-stream] [Tahoe-LAFS] #1698: the preferred cmdline to run tests under coverage is pretty huge, and comes in two flavors due to Debian renaming of the script

[Tahoe-LAFS]

#1698: the preferred cmdline to run tests under coverage is pretty huge, and comes
in two flavors due to Debian renaming of the script
----------------------------+-------------------------------------
     Reporter:  davidsarah  |      Owner:  davidsarah
         Type:  defect      |     Status:  assigned
     Priority:  major       |  Milestone:  eventually
    Component:  code        |    Version:  1.9.1
   Resolution:              |   Keywords:  usability test coverage
Launchpad Bug:              |
----------------------------+-------------------------------------

Old description:

> The preferred command line to run tests recording coverage information
> is:
> {{{
> bin/tahoe @coverage run --branch @tahoe debug trial [ARGS]
> }}}
> but Debian/Ubuntu renamed the "{{{coverage}}}" script to "{{{python-
> coverage}}}", in which case it is:
> {{{
> bin/tahoe @python-coverage run --branch @tahoe debug trial [ARGS]
> }}}
>
> The [source:Makefile] uses:
> {{{
> PYTHONPATH=. python bin/tahoe debug trial --reporter=bwverbose-coverage
> [ARGS]
> }}}
> or it is also possible to use:
> {{{
> bin/tahoe debug trial --coverage [ARGS]
> }}}
>
> However, these seem to cover fewer lines than the "coverage run"
> approach, in particular, top-level declarations in some files (for which
> their omission is clearly a bug... there's no way to import those files
> without executing those lines). Also, {{{--reporter=bwverbose-coverage}}}
> and {{{--coverage}}} do not allow passing additional options to
> 'coverage', therefore they do not support obtaining branch coverage,
> which is more useful.
>
> To fix this ticket, add and document a way to run tests with correct
> branch coverage that works on all platforms using a short command line.

New description:

 The preferred command line to run tests recording coverage information is:
 {{{
 bin/tahoe @coverage run --branch @tahoe debug trial [ARGS]
 }}}
 but Debian/Ubuntu renamed the "{{{coverage}}}" script to "{{{python-
 coverage}}}", in which case it is:
 {{{
 bin/tahoe @python-coverage run --branch @tahoe debug trial [ARGS]
 }}}

 The [source:Makefile] uses:
 {{{
 PYTHONPATH=. python bin/tahoe debug trial --reporter=bwverbose-coverage
 [ARGS]
 }}}
 or it is also possible to use:
 {{{
 bin/tahoe debug trial --coverage [ARGS]
 }}}

 However, these seem to cover fewer lines than the "coverage run" approach,
 in particular, top-level declarations in some files (for which their
 omission is clearly a bug... there's no way to import those files without
 executing those lines). Also, {{{--reporter=bwverbose-coverage}}} and
 {{{--coverage}}} do not allow passing additional options to 'coverage',
 therefore they do not support obtaining branch coverage, which is more
 useful.

 To fix this ticket, add and document a way to run tests with correct
 branch coverage that works on all platforms using a short command line.

--

Comment (by warner):

 also see #2289 ({{{--branch}}} coverage)

--
Ticket URL: <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1698#comment:5>
Tahoe-LAFS <https://Tahoe-LAFS.org>
secure decentralized storage