[tahoe-lafs-trac-stream] [Tahoe-LAFS] #3372: Ratchet based on test module, not individual test

[Tahoe-LAFS]

#3372: Ratchet based on test module, not individual test
--------------------------------+----------------------------------
 Reporter:  chadwhitacre        |          Owner:
     Type:  defect              |         Status:  new
 Priority:  normal              |      Milestone:  Support Python 3
Component:  dev-infrastructure  |        Version:  n/a
 Keywords:                      |  Launchpad Bug:
--------------------------------+----------------------------------
 The unit of work for the Python 3 [https://tahoe-lafs.org/trac/tahoe-
 lafs/wiki/Python3 port] is the module. Currently we ratchet up successful
 tests at the individual test level, which means that randomly successful
 tests in unrelated modules that we don't care about yet distract us from
 the module currently in focus. At the very least, they are noise during
 dev and code review. If they start failing again, the developer either has
 to manually remove them from the "binary-ish" `ratchet-passing`, which is
 an annoyance, or fix them, which is a rabbit hole.

 In #3369 there is a suggestion to not fail ratchet in CI if unexpected
 successes are discovered. But, since devs are expected to add expected
 successes to ratchet before pushing, this still places a burden on devs to
 then manually edit `ratchet-passing` and distinguish expected and
 unexpected successes.

 === Acceptance ===

 1. `tox -e py36` only runs the tests explicitly listed in
 `allmydata.utils._python3.PORTED_TEST_MODULES`.
 1. There's no automated "ratcheting" per se. CI semantics are the same as
 for any other env, just limited in scope to the specified set of test
 modules, which is manually ratcheted (as it were).

--
Ticket URL: <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3372>
Tahoe-LAFS <https://Tahoe-LAFS.org>
secure decentralized storage