Why is Jenkins's Incremental Build feature not working

Version 3.2 by Vincent Massol on 2015/06/05 10:54

Jun 05 2015

On the XWiki project we have enabled Jenkins's Incremental Build feature:

jenkins.png

This seemed like a nice feature to speed up our CI when building our Maven jobs. Alas, it doesn't work!

The problem is that from time to time you'll get build failure such as:

Caused by: org.sonatype.aether.transfer.ArtifactNotFoundException: Could not find artifact org.xwiki.platform:xwiki-platform-store:pom:6.4.4-SNAPSHOT in local.central (xxx)

You'd think it's not possible, especially as we use Maven's -U flag and thus, even if the artifact is not present in the local repository it should be downloaded from the Maven Remote Repository that we use (and it's available there!).

The reason is because of a Maven bug: MNG-5542. What happens is that the Incremental Build feature will use the -pl Maven parameter to list all the Maven projects to build and when this feature is used, artifacts declared in the <parent> section of your POMs are just ignored and not downloaded...

The consequence is that your build will fail from time to time when one artifact declared in one of your <parent> is not present in your local repository... If you have decided to have one Maven repository per Job in Jenkins - which would seem a good idea to isolate your jobs and to be able to use the Parallel build feature of Jenkins - then you'll hit the problem very frequently...

So in the end you have to choose between Parallel builds and Incremental builds but you cannot have both at the same time!

Note that even with Parallel builds turned off, you build will fail from time to time, just less frequently...

One solution for the XWiki project would be to break our big job that builds the Platform (over 100 modules located in the same Git repo that we release together under a single version) into 100 jobs. But doing this manually is a no go so we would need to script this or wait for some Jenkins dev to implement this idea...

Big pain!

Created by Vincent Massol on 2015/06/05 10:50