Generating test coverage reports for a single Maven project is simple. You can use the Clover maven plugin easily for that. For example:
Generating a report for several modules in the same Maven reactor (same build) is also easy since that's supported out of the box. For example:
However, generating a full coverage report for a multi-reactor project is much harder. Let's take the example of the XWiki project which has 4 separate Github repositories and thus 4 builds:
So the question is: How do we generate a single test coverage report for those 4 maven reactor builds. For example we want that tests that execute in the xwiki-enterprise repository generate coverage for source code located, say, in xwiki-commons.
The way to do this manually is to tell the Maven Clover plugin to use a single location for generating its data. Manually this can be achieved like this (more details can be found on the XWiki Test page):
mvn clean clover:setup install -Dmaven.clover.cloverDatabase=/path/to/clover/data/clover.db
...
# In xwiki-enterprise:
mvn clean clover:setup install -Dmaven.clover.cloverDatabase=/path/to/clover/data/clover.db
# From xwiki-enterprise, generate the full Clover report:
mvn clover:clover -N -Dmaven.clover.cloverDatabase=/path/to/clover/data/clover.db
This is already pretty cool. However it's taking a lot of time and it would be nicer if it could be executed on the CI (on http://ci.xwiki.org in our case).
One important note is that Clover modifies the artifacts and thus you need to be careful to not push them into production or make sure they're not used in other builds (since they'd fail since they'd need to have the Clover runtime JAR at execution time).
So, I chose to use Jenkins 2 and the new Pipeline plugin and used the following script:
def mvnHome
def localRepository
def cloverDir
stage('Preparation') {
def workspace = pwd()
localRepository = "${workspace}/maven-repository"
// Make sure that the special Maven local repository for Clover exists
sh "mkdir -p ${localRepository}"
// Remove all XWiki artifacts from it
sh "rm -Rf ${localRepository}/org/xwiki"
sh "rm -Rf ${localRepository}/com/xpn"
// Make sure that the directory where clover will store its data exists in
// the workspace and that it's clean
cloverDir = "${workspace}/clover-data"
sh "rm -Rf ${cloverDir}"
sh "mkdir -p ${cloverDir}"
// Get the Maven tool.
// NOTE: Needs to be configured in the global configuration.
mvnHome = tool 'Maven'
}
// each() has problems in pipeline, thus using a standard for()
// See https://issues.jenkins-ci.org/browse/JENKINS-26481
for (String repoName : ["xwiki-commons", "xwiki-rendering", "xwiki-platform", "xwiki-enterprise"]) {
stage("Cloverify ${repoName}") {
dir (repoName) {
git "https://github.com/xwiki/${repoName}.git"
runCloverAndGenerateReport(mvnHome, localRepository, cloverDir)
}
}
}
stage("Publish Clover Reports") {
...
}
}
def runCloverAndGenerateReport(def mvnHome, def localRepository, def cloverDir) {
wrap([$class: 'Xvnc']) {
withEnv(["PATH+MAVEN=${mvnHome}/bin", 'MAVEN_OPTS=-Xmx2048m']) {
sh "mvn -Dmaven.repo.local='${localRepository}' clean clover:setup install -Pclover,integration-tests -Dmaven.clover.cloverDatabase=${cloverDir}/clover.db -Dmaven.test.failure.ignore=true -Dxwiki.revapi.skip=true"
sh "mvn -Dmaven.repo.local='${localRepository}' clover:clover -N -Dmaven.clover.cloverDatabase=${cloverDir}/clover.db"
}
}
}
Note that we use the "Xvnc" Jenkins plugin because we run Selenium2 functional tests which require a display.
When this Jenkins job is executed is results in:
Over 5 hours of build time... Now you understand why we want to have this running on the CI agent and not on my local machine 
And the generated reports can be seen on xwiki.org.
Good news, we have an overall coverage of 73.2% for the full XWiki Java codebase, that's not too bad (I thought it would be lower ).
