Opened at 2018-05-08T01:42:26Z
Closed at 2020-01-17T17:04:57Z
#2922 closed defect (cannot reproduce)
Integration test step goes green even if actual tests fail
| Reported by: | meejah | Owned by: | |
|---|---|---|---|
| Priority: | normal | Milestone: | undecided |
| Component: | unknown | Version: | 1.12.1 |
| Keywords: | Cc: | ||
| Launchpad Bug: |
Description
Is maybe the "tox" command not propagating its exit-code .. "or something"? See e.g. https://travis-ci.org/tahoe-lafs/tahoe-lafs/jobs/375959463
Change History (2)
comment:1 Changed at 2018-05-08T02:28:54Z by meejah
comment:2 Changed at 2020-01-17T17:04:57Z by exarkun
- Resolution set to cannot reproduce
- Status changed from new to closed
Hm. I think someone clicked the rebuild button on that travis build. Travis re-uses build numbers! So when you rebuild a failed job you lose all of the state from the failed run and it is replaced by the results of the new run. Fun stuff.
This sounds like a terrible bug sadly all of the information associated with the failure is gone.
We should re-open this if we observe the problem again and include all relevant information on the ticket instead of linking out to travis to avoid losing it again.
FWIW, fortunately or unfortunately, I've seen failed integration test runs on CI... so this doesn't *always* happen. :/
Locally at least (Debian stretch) tox -e integration does correctly return 1 (if e.g. the "Chutney" tests fail, in the case I was looking at).