Context
On the XWiki project, we've been pursuing a strategy of failing our Maven build automatically whenever the test coverage of each Maven module is below a threshold indicated in the pom.xml of that module. We're using Jacoco to measure this local coverage.
We've been doing this for over 6 years now and we've been generally happy about it. This has allowed us to raise the global test coverage of XWiki by a few percent every year.
Recently, I joined the STAMP European Research Project and one our KPIs is the global coverage, so I got curious and wanted to look at precisely how much we're winning every year.
This is when problems started 
I realized that, even though we've been generally increasing our global coverage, there are time where we actually reduce it or increase very little, even though at the local level all modules increase their local coverage...
Reporting
So I implemented a Jenkins pipeline script that is using Open Clover, that runs every day and that gets the raw Clover data and generates a report. This report shows how the global coverage evolves, Maven module by Maven module and the contribution of each module to the global coverage.
Here's a recent example report comparing global coverage from 2019-01-01 to 2019-01-08, i.e. just 8 days.
The lines in red are modules that have had changes lowering the global coverage (even though the local coverage for these modules didn't change or even increased!).
Why?
Reasons
Here are some reasons I analyzed that can cause a module to lower the global coverage even though its local coverage is stable or increases:
- TODO
Actions
TODO
