Opened at 2019-07-01T14:57:18Z
Last modified at 2019-07-25T12:40:55Z
#3138 new defect
Run the test suite concurrency on CircleCI — at Initial Version
| Reported by: | exarkun | Owned by: | exarkun |
|---|---|---|---|
| Priority: | normal | Milestone: | undecided |
| Component: | dev-infrastructure | Version: | n/a |
| Keywords: | Cc: | ||
| Launchpad Bug: |
Description
CircleCI provides runners with two vCPUs. If a single CircleCI job can split its work across two Python processes we can take better advantage of the runner configuration and hopefully get a much faster CI run.
trial supports spreading test execution over multiple local processes with --jobs. We can already pass extra arguments to trial through tox thanks to #2894.
There are some problems with coverage collection's interactions with --jobs. There is a Twisted ticket to deal with this, https://twistedmatrix.com/trac/ticket/9663, but there is also https://pypi.org/project/coverage_enable_subprocess/ which basically solves the problem externally.
So, add coverage_enable_subprocess and --jobs 2 to our CircleCI configuration and get a ~50% reduction in CircleCI runtime.
See #2845 and #2894 for past ideas in this space. See for a Twisted ticket meant to support this effort.
