Jun 09 2019

Context

On the XWiki project we use Jenkinsfiles in our GitHub repositories, along with "Github Organization" type of jobs so that Jenkins handles automatically creating and destroying jobs based on git branches in these repositories. This is very convenient and we have a pretty elaborate Jenksinfile (using shared global libraries we developed) in which we execute about 14 different Maven builds, some in series and others in parallel to validate different things, including execution of functional tests.

We recently introduced functional tests that can be executed with different configurations (different databases, different servlet containers, different browsers, etc). Now that represents a lot of combinations and we can't run all of these every time there's a commit in GitHub. So we need to run some of them only once per day, others once per week and the rest once per month.

The problem is that Jenkins doesn't seem to support this feature out of the box when using a Jenkinsfile. In an ideal world, Jenkins would support several Jenkinsfile to achieve this. Right now the obvious solution is to create manually new jobs to run these configuration tests. However, doing this removes the benefits of the Jenkinsfile, the main one being the automatic creation and destruction of job for branches. We started with this and after a few months it became too painful to maintain. So we had to find a better solution...

The Solution

Let me start by saying that I find this solution suboptimal as it's complex and fraught with several problems.

Generally speaking the solution we implemented is based on the Parameterized Shcheduler Plugin but the devil is in the details.

  • Step 1: Make your job a parameterized job by defining a type variable that will hold what type of job you want to execute. In our case standard or docker-latest (to be executed daily), docker-all (to be executed weekly) and docker-unsupported (to be executed monthly). All the docker-* job types will execute our functional tests on various configurations. Also configure the parameterized scheduler plugin accordingly:
    private def getCustomJobProperties()
    {
     return [
        parameters([string(defaultValue: 'standard', description: 'Job type', name: 'type')]),
        pipelineTriggers([
          parameterizedCron('''@midnight %type=docker-latest
    @weekly %type=docker-all
    @monthly %type=docker-unsupported'''
    ),
          cron("@monthly")
       ])
     ]
    }

    You set this in the job with:

    properties(getCustomJobProperties())

    Important note: The job will need to be triggered once before the scheduler and the new parameter are effective!

  • Step 2: Based on the type parameter value, decide what to execute.For example:
    ...
    if (params.type && params.type == 'docker-latest') {
      buildDocker('docker-latest')
    }
    ...
  • Step 3: You may want to manually trigger your job using the Jenkins UI and decide what type of build to execute (this is useful to debug some test problems for example). You can do it this way:
    def choices = 'Standard\nDocker Latest\nDocker All\nDocker Unsupported'
    def selection = askUser(choices)
    if (selection == 'Standard') {
     ...
    } else of (selection == 'Docker Latest') {
     ...
    } else ...

    In our case askUSer is a custom pipeline library defined like this:

    def call(choices)
    {
       def selection

       // If a user is manually triggering this job, then ask what to build
       if (currentBuild.rawBuild.getCauses()[0].toString().contains('UserIdCause')) {
            echo "Build triggered by user, asking question..."
           try {
                timeout(time: 60, unit: 'SECONDS') {
                    selection = input(id: 'selection', message: 'Select what to build', parameters: [
                        choice(choices: choices, description: 'Choose which build to execute', name: 'build')
                   ])
               }
           } catch(err) {
               def user = err.getCauses()[0].getUser()
               if ('SYSTEM' == user.toString()) { // SYSTEM means timeout.
                   selection = 'Standard'
               } else {
                   // Aborted by user
                   throw err
               }
           }
       } else {
            echo "Build triggered automatically, building 'All'..."
            selection = 'Standard'
       }

       return selection
    }

Limitations

While this may sound like a nice solution, it has a drawback. Jenkins's build history gets messed up, because you're reusing the same job name but running different builds. For example, test failure age will get reset every time a different type of build is ran. Note that at least individual test history is kept.

Since different types of builds are executed in the same job, we also wanted the job history to visibly show when scheduled jobs are executed vs the standard jobs. Thus we added the following in our pipeline:

import com.cloudbees.groovy.cps.NonCPS
import com.jenkinsci.plugins.badge.action.BadgeAction
...
def badgeText = 'Docker Build'
def badgeFound = isBadgeFound(currentBuild.getRawBuild().getActions(BadgeAction.class), badgeText)
if (!badgeFound) {
    manager.addInfoBadge(badgeText)
    manager.createSummary('green.gif').appendText("<h1>${badgeText}</h1>", false, false, false, 'green')
    currentBuild.rawBuild.save()
}

@NonCPS
private def isBadgeFound(def badgeActionItems, def badgeText)
{
   def badgeFound = false
    badgeActionItems.each() {
       if (it.getText().contains(badgeText)) {
            badgeFound = true
           return
       }
   }
   return badgeFound
}

Visually this gives the following where you can see information icons for the configuration tests (and you can hover over the information icon with the mouse to see the text):

history.png

What's your solution to this problem? I'd be very eager to know if someone has found a better solution to implement this in Jenkins.

Tags:
Created by Vincent Massol on 2019/06/09 11:09
Tags:
Created by Vincent Massol on 2019/06/09 11:09
This wiki is licensed under a Creative Commons 2.0 license