Blog Archive
Blog - posts for February 2017
Feb 06 2017
Jenkins going the Gradle way
Just realized that with the new Jenkins Pipeline strategy, Jenkins is actually moving towards a strategy similar to Gradle.
Before Gradle we had Maven which is using a Build by Configuration strategy. The idea is for users to tell Maven how to configure the build but not what it should do.
Before Pipeline, Jenkins Jobs were exactly that: you configured each job to give Jenkins each plugin's config, similar to Maven.
With Pipeline you now code your job in Groovy, specifying the what the job should do.
So you gain a lot of power to more precisely configure your jobs and an easier solution to reuse actions/configs between jobs. But you loose some simplicity and the fact that you could go to any Jenkins instance and understand what each job was doing easily. You now need to read code to understand what it's doing and everyone is going to have a different way of coding their jobs.
FYI I'm currently working on XWiki's Jenkinsfile. It's still simple at the moment but it'll become more complex as time passes.
Future will tell us if it's good or bad. FTM, being a dev, I'm enjoying it! 
I especially like the perks that come with it (but which could have been implemented with a declarative job configuration too):
- Save the CI job in the SCM next to the code
- Ability to automatically add or remove jobs for SCM branches
See also my blog post about Jenkins GitHub Organization Jobs.
Feb 02 2017
Jenkins GitHub Organization Jobs
The Jenkins Pipeline plugin includes a very nice feature which is the "GitHub Organization" job type. This job type will scan a given GitHub organization repositories for Jenkinsfile files and when found will automatically create a pipeline job for them.
This has some nice advantages:
- You save your Jenkins job configuration in your SCM (git in our case, in the Jenkinsfile), next to your code. You can receive email diffs to show who made modifications to it, the reason and understand the changes.
- It supports branches: when you create a branch it's automatically discovered by Jenkins and the build is executed on it. And if the branch gets removed, it's removed automatically from Jenkins too. This point is awesome for us since we used to have groovy script to execute to copy jobs when there were new branches and when branches were removed.
So we started exploring this for the XWiki project, starting with Contrib Extensions.
Here's a screenshot of our Github Organization job for XWiki Contrib:
And here's an example of a pipeline job executing:
Now if you implement this you'll quickly find that you want to share pipeline scripts between Jenkinsfile, in order to not have duplicates.
FYI here's what the Jenkinsfile for the syntax-markdown pipeline job shown above looks like:
name = 'syntax-markdown'
}
Simple, isn't it? The trick is that we've configured Jenkins to automatically load a Global Pipeline Library (implicit load). You can do that by saving libraries at the root of SCM repositories and configure Jenkins to load them from the SCM sources (see this Jenkins doc for more details).
So we've created this GitHub repository and we've coded a vars/xwikiModule.groovy file. At the moment of writing this is its content (I expect it to be improved a lot in the near future):
// xwikiModule {
// name = 'application-faq'
// goals = 'clean install' (default is 'clean deploy')
// profiles = 'legacy,integration-tests,jetty,hsqldb,firefox' (default is 'quality,legacy,integration-tests')
// }
def call(body) {
// evaluate the body block, and collect configuration into the object
def config = [:]
body.resolveStrategy = Closure.DELEGATE_FIRST
body.delegate = config
body()
// Now build, based on the configuration provided, using the followong configuration:
// - config.name: the name of the module in git, e.g. "syntax-markdown"
node {
def mvnHome
stage('Preparation') {
// Get the Maven tool.
// NOTE: Needs to be configured in the global configuration.
mvnHome = tool 'Maven'
}
stage('Build') {
dir (config.name) {
checkout scm
// Execute the XVNC plugin (useful for integration-tests)
wrap([$class: 'Xvnc']) {
withEnv(["PATH+MAVEN=${mvnHome}/bin", 'MAVEN_OPTS=-Xmx1024m']) {
try {
def goals = config.goals ?: 'clean deploy'
def profiles = config.profiles ?: 'quality,legacy,integration-tests'
sh "mvn ${goals} jacoco:report -P${profiles} -U -e -Dmaven.test.failure.ignore"
currentBuild.result = 'SUCCESS'
} catch (Exception err) {
currentBuild.result = 'FAILURE'
notifyByMail(currentBuild.result)
throw e
}
}
}
}
}
stage('Post Build') {
// Archive the generated artifacts
archiveArtifacts artifacts: '**/target/*.jar', fingerprint: true
// Save the JUnit test report
junit testResults: '**/target/surefire-reports/TEST-*.xml'
}
}
}
def notifyByMail(String buildStatus) {
buildStatus = buildStatus ?: 'SUCCESSFUL'
def subject = "${buildStatus}: Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]'"
def summary = "${subject} (${env.BUILD_URL})"
def details = """<p>STARTED: Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]':</p>
<p>Check console output at "<a href='${env.BUILD_URL}'>${env.JOB_NAME} [${env.BUILD_NUMBER}]</a>"</p>"""
def to = emailextrecipients([
[$class: 'CulpritsRecipientProvider'],
[$class: 'DevelopersRecipientProvider'],
[$class: 'RequesterRecipientProvider']
])
if (to != null && !to.isEmpty()) {
mail to: to, subject: subject, body: details
}
}
Ideas of some next steps:
- Put back our support for false positives in the pipeline script
- Make xvnc optional with a xvnc = true | false parameter option
Right now there's one limitation I've found: It seems I need to manually click on "Re-scan Organization" in the Jenkins UI so that new Jenkinsfile added to repositories are taken into account. I hope that will get fixed soon. One workaround would be to add another Jenkins job to do that regularly but it's not perfect. Also note that you absolutely must authenticate against GitHub as otherwise you'll quickly reach the GitHub API request limit (when authenticated you are allowed 5000 requests per hour).
Anyway it's great and I love it.
