[tahoe-lafs-trac-stream] [Tahoe-LAFS] #3568: codecov results for fork PRs are delivered before all coverage is submitted, claim massive loss of coverage

[Tahoe-LAFS]

#3568: codecov results for fork PRs are delivered before all coverage is
submitted, claim massive loss of coverage
---------------------+---------------------------
 Reporter:  exarkun  |          Owner:
     Type:  defect   |         Status:  new
 Priority:  normal   |      Milestone:  undecided
Component:  unknown  |        Version:  n/a
 Keywords:           |  Launchpad Bug:
---------------------+---------------------------
 If a PR is submitted from a GitHub fork of tahoe-lafs then codecov makes
 its report comment very early - before most builds have finished - and the
 report is ugly.  It often claims something like a 30% loss of coverage
 spread across the whole project.

 It seems like codecov is generating a report from the coverage data
 uploaded by the *first* job to finish.  It typically updates its comment
 later when more data comes in but the first report is obnoxious,
 misleading noise (it will often be emailed out in addition to showing up
 on the PR page).

 Also, the codecov status item is not always updated.  For example, right
 now https://github.com/tahoe-lafs/tahoe-
 lafs/pull/939/checks?check_run_id=1571210288 shows 63% coverage even
 though the report on codecov itself https://codecov.io/gh/tahoe-lafs
 /tahoe-lafs/pull/939 says 92% coverage.  These pages are totally out of my
 control, of course, so who knows what they'll look like after any amount
 of time passes.

--
Ticket URL: <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/3568>
Tahoe-LAFS <https://Tahoe-LAFS.org>
secure decentralized storage