Jenkins Pipeline: Attach failing test screenshot

Last modified by Vincent Massol on 2019/06/11 10:11

Jun 06 2017

On the XWiki project we've started moving to Jenkins 2.0 and to using the Pipeline feature through Jenkinsfiles.

When we run our functional tests (we use Selenium2/Webdriver), we record a screenshot when a test fails. Previously we had a Groovy Scriptler script (written by Eduard Moraru, an XWiki committer) to automatically change the description of a Jenkins test page to include the screenshot as on:

failing.png 

So we needed to port this script to a Jenkinsfile. Here's the solution I came up with:

import hudson.FilePath
import hudson.tasks.junit.TestResultAction
import hudson.util.IOUtils
import javax.xml.bind.DatatypeConverter

def attachScreenshotToFailingTests() {
   def testResults = manager.build.getAction(TestResultAction.class)
   if (testResults == null) {
       // No tests were run in this build, nothing left to do.
       return
    }

   // Go through each failed test in the current build.
   def failedTests = testResults.getFailedTests()
   for (def failedTest : failedTests) {
       // Compute the test's screenshot file name.
       def testClass = failedTest.getClassName()
       def testSimpleClass = failedTest.getSimpleName()
       def testExample = failedTest.getName()

       // Example of value for suiteResultFile (it's a String):
       //   /Users/vmassol/.jenkins/workspace/blog/application-blog-test/application-blog-test-tests/target/
       //     surefire-reports/TEST-org.xwiki.blog.test.ui.AllTests.xml
       def suiteResultFile = failedTest.getSuiteResult().getFile()
       if (suiteResultFile == null) {
           // No results available. Go to the next test.
           continue
        }

       // Compute the screenshot's location on the build agent.
       // Example of target folder path:
       //   /Users/vmassol/.jenkins/workspace/blog/application-blog-test/application-blog-test-tests/target
       def targetFolderPath = createFilePath(suiteResultFile).getParent().getParent()
       // The screenshot can have 2 possible file names and locations, we have to look for both.
       // Selenium 1 test screenshots.
       def imageAbsolutePath1 = new FilePath(targetFolderPath, "selenium-screenshots/${testClass}-${testExample}.png")
       // Selenium 2 test screenshots.
       def imageAbsolutePath2 = new FilePath(targetFolderPath, "screenshots/${testSimpleClass}-${testExample}.png")
       // If screenshotDirectory system property is not defined we save screenshots in the tmp dir so we must also
       // support this.
       def imageAbsolutePath3 =
            new FilePath(createFilePath(System.getProperty("java.io.tmpdir")), "${testSimpleClass}-${testExample}.png")

       // Determine which one exists, if any.
        echo "Image path 1 (selenium 1) [${imageAbsolutePath1}], Exists: [${imageAbsolutePath1.exists()}]"
        echo "Image path 2 (selenium 2) [${imageAbsolutePath2}], Exists: [${imageAbsolutePath2.exists()}]"
        echo "Image path 3 (tmp) [${imageAbsolutePath3}], Exists: [${imageAbsolutePath3.exists()}]"
       def imageAbsolutePath = imageAbsolutePath1.exists() ?
            imageAbsolutePath1 : (imageAbsolutePath2.exists() ? imageAbsolutePath2 :
                (imageAbsolutePath3.exists() ? imageAbsolutePath3 : null))

        echo "Attaching screenshot to description: [${imageAbsolutePath}]"

       // If the screenshot exists...
       if (imageAbsolutePath != null) {
           // Build a base64 string of the image's content.
           def imageDataStream = imageAbsolutePath.read()
            byte[] imageData = IOUtils.toByteArray(imageDataStream)
           def imageDataString = "data:image/png;base64," + DatatypeConverter.printBase64Binary(imageData)

           def testResultAction = failedTest.getParentAction()

           // Build a description HTML to be set for the failing test that includes the image in Data URI format.
           def description = """<h3>Screenshot</h3><a href="${imageDataString}"><img style="width: 800px" src="${imageDataString}" /></a>"""

           // Set the description to the failing test and save it to disk.
            testResultAction.setDescription(failedTest, description)
            currentBuild.rawBuild.save()
        }
    }
}

Note that for this to work you need to:

  • Install the Groovy Postbuild plugin. This exposes the manager variable needed by the script.
  • Add the required security exceptions to http://<jenkins server ip>/scriptApproval/ if need be
  • Install the Pegdown Formatter plugin and set the description syntax to be Pegdown in the Global Security configuration (http://<jenkins server ip>/configureSecurity). Without this you won't be able to display HTML (and the default safe HTML option will strip out the datauri content).

Enjoy!

Tags: STAMP
Created by Vincent Massol on 2017/06/06 11:41