Jenkins Pipeline: Attach failing test screenshot

Version 1.1 by Vincent Massol on 2017/06/06 11:41

Jun 06 2017

Warning
This blog post is not published yet.

On the XWiki project we've started moving to Jenkins 2.0 and to using the Pipeline feature through Jenkinsfiles.

When we run our functional tests (we use Selenium2/Webdriver), we record a screenshot when a test fails. Previously we had a Groovy Scriptler script (written by Eduard Moraru, an XWiki committer) to automatically change the description of a Jenkins test page to include the screenshot as on:

failing.png 

So we needed to port this script to a Jenkinsfile. Here's the solution I came up with:

import hudson.FilePath
import hudson.tasks.junit.TestResultAction
import hudson.util.IOUtils
import javax.xml.bind.DatatypeConverter

def attachScreenshotToFailingTests() {
   def testResults = manager.build.getAction(TestResultAction.class)
   if (testResults == null) {
       // No tests were run in this build, nothing left to do.
       return
    }

   // Go through each failed test in the current build.
   def failedTests = testResults.getFailedTests()
   for (def failedTest : failedTests) {
       // Compute the test's screenshot file name.
       def testClass = failedTest.getClassName()
       def testSimpleClass = failedTest.getSimpleName()
       def testExample = failedTest.getName()

       // Example of value for suiteResultFile (it's a String):
       //   /Users/vmassol/.jenkins/workspace/blog/application-blog-test/application-blog-test-tests/target/
       //     surefire-reports/TEST-org.xwiki.blog.test.ui.AllTests.xml
       def suiteResultFile = failedTest.getSuiteResult().getFile()
       if (suiteResultFile == null) {
           // No results available. Go to the next test.
           continue
        }

       // Compute the screenshot's location on the build agent.
       // Example of target folder path:
       //   /Users/vmassol/.jenkins/workspace/blog/application-blog-test/application-blog-test-tests/target
       def targetFolderPath = createFilePath(suiteResultFile).getParent().getParent()
       // The screenshot can have 2 possible file names and locations, we have to look for both.
       // Selenium 1 test screenshots.
       def imageAbsolutePath1 = new FilePath(targetFolderPath, "selenium-screenshots/${testClass}-${testExample}.png")
       // Selenium 2 test screenshots.
       def imageAbsolutePath2 = new FilePath(targetFolderPath, "screenshots/${testSimpleClass}-${testExample}.png")
       // If screenshotDirectory system property is not defined we save screenshots in the tmp dir so we must also
       // support this.
       def imageAbsolutePath3 =
            new FilePath(createFilePath(System.getProperty("java.io.tmpdir")), "${testSimpleClass}-${testExample}.png")

       // Determine which one exists, if any.
        echo "Image path 1 (selenium 1) [${imageAbsolutePath1}], Exists: [${imageAbsolutePath1.exists()}]"
        echo "Image path 2 (selenium 2) [${imageAbsolutePath2}], Exists: [${imageAbsolutePath2.exists()}]"
        echo "Image path 3 (tmp) [${imageAbsolutePath3}], Exists: [${imageAbsolutePath3.exists()}]"
       def imageAbsolutePath = imageAbsolutePath1.exists() ?
            imageAbsolutePath1 : (imageAbsolutePath2.exists() ? imageAbsolutePath2 :
                (imageAbsolutePath3.exists() ? imageAbsolutePath3 : null))

        echo "Attaching screenshot to description: [${imageAbsolutePath}]"

       // If the screenshot exists...
       if (imageAbsolutePath != null) {
           // Build a base64 string of the image's content.
           def imageDataStream = imageAbsolutePath.read()
            byte[] imageData = IOUtils.toByteArray(imageDataStream)
           def imageDataString = "data:image/png;base64," + DatatypeConverter.printBase64Binary(imageData)

           def testResultAction = failedTest.getParentAction()

           // Build a description HTML to be set for the failing test that includes the image in Data URI format.
           def description = """<h3>Screenshot</h3><a href="${imageDataString}"><img style="width: 800px" src="${imageDataString}" /></a>"""

           // Set the description to the failing test and save it to disk.
            testResultAction.setDescription(failedTest, description)
            currentBuild.rawBuild.save()
        }
    }
}

Note that for this to work you need to:

  • Install the Groovy Postbuild plugin. This exposes the manager variable needed by the script.
  • Add the required security exceptions to http://<jenkins server ip>/scriptApproval/ if need be
  • Install the Pegdown Formatter plugin and set the description syntax to be Pegdown in the Global Security configuration (http://<jenkins server ip>/configureSecurity). Without this you won't be able to display HTML (and the default safe HTML option will strip out the datauri content).

Enjoy!

Tags: STAMP
Created by Vincent Massol on 2017/06/06 11:41