This is a call to Jenkins developers and experts. I'm looking for a solution to the following problem.
Statement of the problem:
- On the XWiki project we have a Git repository with a lot of top level Maven modules (about 85 as of today)
- Each of these top level modules have several sub modules including modules that run functional tests (long to execute)
- Right now we build the full platform in a single job on Jenkins and this takes too long: about 1 hour
Note that we do not want to create many Git repositories like one repo per top level module since that makes it a lot harder to release all modules at once (we release them together with the same version) and it would mean creating lots of jobs manually (85!).
The ideal solution I can think of would be:
- Give to Jenkins the ability to create "virtual" jobs (one per top level module)
- Each such job would automatically have its dependencies on other jobs defined based on the Maven dependencies so that jobs wait automatically for jobs that need to be run before them
- This would allow to dispatch the full build to all the agents available
Doing this manually would be a real pain since it would mean creating 85 jobs and recreating the Maven dependencies with job triggers. Of course it should be possible to use the Scriptler plugin to automate this but parsing the Maven POMs to create the matching job hierarchy is not something trivial to do...
Does anyone have a solution for this? Do you agree it's a legitimate use case?
Thanks!
