Wiki Home » On software quality, collaborative development, wikis and Open Source...

On software quality, collaborative development, wikis and Open Source...

Last modified by Vincent Massol on 2013/10/22 14:37
Comments (0) · Annotations (0) · History

May 15 2014

Internals of an open source company: XWiki SAS

I had the pleasure to present at Human Talks. I presented how an open source company works internally and the challenges of handling potentially conflicting interests between business interests and open source interests. My experience is based on the XWiki SAS company and the XWiki open source project.

It was a fun event, held at Mozilla Paris in a very nice room (fun fact: on this picture, which was taken on the day Mozilla opened the office, you can see Ludovic Dubost - with the blue polo -, creator of the XWiki open source project and founder of the XWiki SAS company emoticon_wink).

Thanks Human Talks for the invite!

Feb 04 2014

FOSDEM 2014

Another year @ FOSDEM with the XWiki gang: Ludovic, Marius, Anca, Oana and Fabio!

This year though we succeeded in getting dev room (yeah!), a wiki dev room, that I co-organized with Quim Gil from Wikimedia and Jean-Macr Libs from tiki.org.

The XWiki project was lucky to have 6 talks:

It was a nice FOSDEM. We enjoyed Belgian waffles and French fries as usual (although I heard some enjoyed it a bit less than usual since they were on a low-carb diet emoticon_wink).

At the content level, the conference was slightly too low-tech for me, a Java developer. Lots of C/C++ guys and lots of stuff close to the hardware emoticon_wink Not that I don't think it's nice to do that, but rather that I can't participate much. There were some other tracks more of interest to me like the Java dev room (but I was stuck in the wiki dev room at the same time so couldn't join) and the Javascript dev room but this one was so full that it was near impossible to get in...

I'd personally love to see some more room/space given to open source in businesses for the future editions of FOSDEM.

With over 8000 participants it seems it was, once again, a very successful FOSDEM.

See you next year maybe!

Jan 27 2014

Calling a Maven plugin from another plugin

Normally you're not supposed to call a Maven plugin's mojo from another mojo but it may happen times when you have this need. In my case I've developed a plugin and I needed to call the Maven License Plugin from within my mojo.

I succeeded in achieving this through the Maven Executor plugin.

Without further ado here's the code that worked for me:

[...]
* @requiresProject
* @requiresDependencyResolution compile
* @threadSafe
*/
public class FormatMojo extends AbstractVerifyMojo
{
   /**
     * The project currently being build.
     *
     * @parameter expression="${project}"
     * @required
     * @readonly
     */

   private MavenProject mavenProject;

   /**
     * The current Maven session.
     *
     * @parameter expression="${session}"
     * @required
     * @readonly
     */

   private MavenSession mavenSession;

   /**
     * The Maven BuildPluginManager component.
     *
     * @component
     * @required
     */

   private BuildPluginManager pluginManager;

[...]

   @Override
   public void execute() throws MojoExecutionException, MojoFailureException
   {
[...]
       // I needed to add a plugin dependency where my license.txt file is located.
       Dependency dep = new Dependency();
        dep.setGroupId("org.xwiki.commons");
        dep.setArtifactId("xwiki-commons-tool-verification-resources");
        dep.setVersion("5.4");

        Plugin licensePlugin = plugin(
            groupId("com.mycila"),
            artifactId("license-maven-plugin"),
            version("2.6"));
        licensePlugin.setDependencies(Collections.singletonList(dep));

        executeMojo(
            licensePlugin,
            goal("format"),
            configuration(
                element(name("header"), "license.txt"),
                element(name("strictCheck"), "true"),
                element(name("headerDefinitions"),
                    element(name("headerDefinition"), "license-xml-definition.xml")),
                element(name("includes"),
                    element(name("include"), "src/main/resources/**/*.xml"))
           ),
            executionEnvironment(
               this.project,
               this.mavenSession,
               this.pluginManager
           )
       );
   }

[...]
}

And boom it worked! emoticon_smile

Hope this can be useful to someone.

Sep 25 2013

Debugging the Maven Deploy Plugin, sort of

You may have been hit by some error while calling mvn deploy and found unable to debug the issue. For example a common error you can get is "Cannot connect. Reason: Auth fail".

The problem is that the Deploy plugin doesn't offer any debugging option. Researching the topic you'll find that it actually uses the JSCH library and researching how to debug this library you'll discover that it provides its own logging facade interface and doesn't provide any logging implementation, leaving it to the user to implement its interface to get any log.

Thanks this blog post I found a SLF4J implementation of the JSCH logging interface and added it to the wagon-ssh artifact.

In short to get debug logs, do the following:

  • Install this version of wagon-ssh in your local repository using this POM file, as follows:
    mvn install:install-file -Dfile=wagon-ssh-2.6-20130924-debug.jar -DpomFile=pom.xml
  • Force using this Wagon SSH modified version by modifying your project's POM to add:
    ...
    <build>
     <plugins>
    ...
         <plugin>
           <groupId>org.apache.maven.plugins</groupId>
           <artifactId>maven-deploy-plugin</artifactId>
           <dependencies>
             <dependency>
               <groupId>org.apache.maven.wagon</groupId>
               <artifactId>wagon-ssh</artifactId>
               <version>2.6-20130924-debug</version>
             </dependency>
             <dependency>
               <groupId>ch.qos.logback</groupId>
               <artifactId>logback-classic</artifactId>
               <version>1.0.13</version>
             </dependency>
           </dependencies>
         </plugin>
    ...
     </plugins>
    </build>
    ...

Now when you do a mvn deploy you'll get debug logs such as:

[INFO] --- maven-deploy-plugin:2.7:deploy (default-deploy) @ xxx ---
Downloading: scp://xxx
19:03:00.666 [pool-3-thread-1] INFO  com.jcraft.jsch - Connecting to xxx port 22
19:03:00.717 [pool-3-thread-1] INFO  com.jcraft.jsch - Connection established
19:03:00.760 [pool-3-thread-1] INFO  com.jcraft.jsch - Remote version string: SSH-2.0-OpenSSH_5.6
19:03:00.760 [pool-3-thread-1] INFO  com.jcraft.jsch - Local version string: SSH-2.0-JSCH-0.1.44
19:03:00.760 [pool-3-thread-1] INFO  com.jcraft.jsch - CheckCiphers: aes256-xxx
19:03:00.873 [pool-3-thread-1] INFO  com.jcraft.jsch - SSH_MSG_KEXINIT sent
19:03:00.874 [pool-3-thread-1] INFO  com.jcraft.jsch - SSH_MSG_KEXINIT received
19:03:00.874 [pool-3-thread-1] INFO  com.jcraft.jsch - kex: server->client aes128-xxx
19:03:00.874 [pool-3-thread-1] INFO  com.jcraft.jsch - kex: client->server aes128-xxx
19:03:00.884 [pool-3-thread-1] INFO  com.jcraft.jsch - SSH_MSG_KEXDH_INIT sent
19:03:00.884 [pool-3-thread-1] INFO  com.jcraft.jsch - expecting SSH_MSG_KEXDH_REPLY
19:03:00.938 [pool-3-thread-1] INFO  com.jcraft.jsch - ssh_rsa_verify: signature true
19:03:00.939 [pool-3-thread-1] INFO  com.jcraft.jsch - Host 'xxx' is known and mathces the RSA host key
19:03:00.939 [pool-3-thread-1] INFO  com.jcraft.jsch - SSH_MSG_NEWKEYS sent
19:03:00.939 [pool-3-thread-1] INFO  com.jcraft.jsch - SSH_MSG_NEWKEYS received
19:03:00.940 [pool-3-thread-1] INFO  com.jcraft.jsch - SSH_MSG_SERVICE_REQUEST sent
19:03:00.999 [pool-3-thread-1] INFO  com.jcraft.jsch - SSH_MSG_SERVICE_ACCEPT received
19:03:01.055 [pool-3-thread-1] INFO  com.jcraft.jsch - Authentications that can continue: publickey,keyboard-interactive,password
19:03:01.055 [pool-3-thread-1] INFO  com.jcraft.jsch - Next authentication method: publickey
19:03:01.107 [pool-3-thread-1] INFO  com.jcraft.jsch - Disconnecting from xxx port 22
[WARNING] Could not transfer metadata xxx from/to xxx): Cannot connect. Reason: Auth fail

Unfortunately, as you can see, it doesn't really add a lot of information to help you debug the real issue. In this case I had just modified my username to be a wrong one and I don't see how I can guess this from this log... 

I'm just reporting this in case it helps someone emoticon_smile

Note that a better solution is to check the SSH logs on the server side. In my case when I had the problem I got the following:

Sep 25 14:55:54 xxx sshd[801]: reverse mapping checking getaddrinfo for xxx [xxxx] failed - POSSIBLE BREAK-IN ATTEMPT!

This means that the PTR of that IP must match the hostname.

Strangely it worked fine when using SSH directly. I'm assuming this is because this message is a warning and the standard SSH client continue whereas JSCH chokes on it and stops.

Jun 29 2013

Issue: Jenkins and large Maven projects

This is a call to Jenkins developers and experts. I'm looking for a solution to the following problem.

Statement of the problem:

  • On the XWiki project we have a Git repository with a lot of top level Maven modules (about 85 as of today)
  • Each of these top level modules have several sub modules including modules that run functional tests (long to execute)
  • Right now we build the full platform in a single job on Jenkins and this takes too long: about 1 hour

Note that we do not want to create many Git repositories like one repo per top level module since that makes it a lot harder to release all modules at once (we release them together with the same version) and it would mean creating lots of jobs manually (85!).

The ideal solution I can think of would be:

  • Give to Jenkins the ability to create "virtual" jobs (one per top level module)
  • Each such job would automatically have its dependencies on other jobs defined based on the Maven dependencies so that jobs wait automatically for jobs that need to be run before them
  • This would allow to dispatch the full build to all the agents available

Doing this manually would be a real pain since it would mean creating 85 jobs and recreating the Maven dependencies with job triggers. Of course it should be possible to use the Scriptler plugin to automate this but parsing the Maven POMs to create the matching job hierarchy is not something trivial to do...

Does anyone have a solution for this? Do you agree it's a legitimate use case?

Thanks!

Apr 20 2013

CodeCamp Iasi 2013

I'm in Iasi, Romania for 1 week visiting the XWiki SAS Romanian office and I was invited to talk at CodeCamp 2013. It was my first time I spoke in Romania and it was fun!

The conference is nice (about 300 participants I'd say); the only issue for me being that all sessions are in ... Romanian... emoticon_smile

I gave the talk I did at Devoxx France 2013 last month. It's a technical talk and I wasn't sure how it would get appreciated since it requires some prerequisite knowledge of Maven, Jenkins and Software factories in general... Seems people like it overall.

Here are the slides:

Mar 31 2013

Devoxx France 2013

I was very happy to present at Devoxx France again! Big kudo to the organizers, it was a great event. As usual it was great to meet all my Java friends.

This year I was happy to be on 4 events:

Here are the slides of my quality on Java projects:

In the slides I mention some Scriptler script to automated modifying all jobs to prevent false positive emails.

A big thank you to Frédéric Bouquet who volunteered during the Hackergarten to work on participating to XWiki. We ended up coding a new CRaSH integration. It's still in progress but I hope we can release a 1.0 version real soon.

See you next year!

Jan 11 2013

Tip: Find out max Clover TPC for Maven modules

On the XWiki project we have a Jenkins Job that runs every night and checks that the quality of Maven modules have not decreased.

This is done by using the Maven Clover plugin and failing the build if the TPC is under a specified threshold for each module.

It's defined in a quality profile since it takes some time to execute (this is why we execute it once per night ATM).

Here's the setup:

   <!-- Profile for QA verifications that takes time -->
   <profile>
     <id>quality</id>
     <build>
       <plugins>
         <!-- Fail the build if the test coverage is below a given value. -->
         <plugin>
           <groupId>com.atlassian.maven.plugins</groupId>
           <artifactId>maven-clover2-plugin</artifactId>
           <configuration>
             <targetPercentage>${xwiki.clover.targetPercentage}</targetPercentage>
           </configuration>
           <executions>
             <execution>
               <id>clover-check</id>
               <phase>verify</phase>
               <goals>
                 <goal>instrument-test</goal>
                 <goal>check</goal>
               </goals>
             </execution>
           </executions>
         </plugin>
       </plugins>
     </build>
   </profile>

Then in each Maven, we have (for example):

...
 <properties>
   <xwiki.clover.targetPercentage>74.6%</xwiki.clover.targetPercentage>
 </properties>
...

Now that's fine and it allows to find out when someone adds code in an existing module and doesn't add enough unit tests to keep the TPC above the current threshold.

There remains an issue. Imagine that I add some code with unit tests. I also need to not forget to update the TPC value in the pom.xml file.

So here's a quick command line tip to find out the current TPC max threshold for all modules located under the directory when you run it:

mvn clean install -Pquality -Dxwiki.clover.targetPercentage=100%
-Dmaven.clover.failOnViolation=false 2>&1 |
awk '/Checking for coverage of/ { module = $9; }
/^Total/ { split(module, array, "/"); print array[length(array)-3],$4 }'

For example when run in xwiki-commons/xwiki-commons-core, it gives:

xwiki-commons-test-simple 0%
xwiki-commons-text 93.5%
xwiki-commons-component-api 22.7%
xwiki-commons-classloader-api 0%
xwiki-commons-classloader-protocol-jar 0%
xwiki-commons-observation-api 15.9%
xwiki-commons-component-observation 76.2%
xwiki-commons-component-default 74.6%
xwiki-commons-context 76.7%
xwiki-commons-script 0%
xwiki-commons-configuration-api 0%
xwiki-commons-test-component 0%
xwiki-commons-environment-api -100%
xwiki-commons-environment-common 0%
xwiki-commons-environment-standard 67.3%
xwiki-commons-environment-servlet 84.6%
xwiki-commons-properties 76.6%
xwiki-commons-logging-api 29.5%
xwiki-commons-observation-local 90.8%
xwiki-commons-job 36.1%
xwiki-commons-logging-logback 91.8%

Now the next step is to write a script that will automatically change the pom.xml files with the max TPC threshold values.

UPDATE 2013-01-31: To do the same with Jacoco you would use:

mvn clean install -Pquality -Dxwiki.jacoco.instructionRatio=100
-Djacoco.haltOnFailure=false 2>&1 |
awk '/jacoco-check/ { module = $6 } /Insufficient/ { print module, $7 }'

UPDATE 2013-07-09: Starting with Jacoco 0.6.4 we need to use:

mvn clean install -Pquality -Dxwiki.jacoco.instructionRatio=2.00
-Djacoco.haltOnFailure=false 2>&1 |
awk '/jacoco-check/ { module = $6 } /instructions covered ratio is/ { print module, $(NF-5) }'

Dec 22 2012

Jenkins: Don't send emails for false positives

Having a CI tool set up is great. However if your CI starts sending a lot of build failure emails and a portion of these are false positives (i.e. for example CI infrastructure issues or flickering tests) then what will happen is that the developers are going to trust less and less your CI and builds will stay failing more and more till your release day when you're going to spend several days trying to stabilize your build and your tests before you can release. Thus delaying the release and starting on a downward slope...

It's thus very important that your CI only sends real build failures to the developers.

I've been looking for a way to do with Jenkins and I've found a solution that works (although I would have liked something simpler - If Jenkins experts read this and there's a better please let me know! emoticon_smile).

So my solution needs 2 Jenkins plugins:

  • The Mail Ext plugin
  • The Groovy Postbuild plugin
  • The Scriptler plugin, for automation.

Without further ado, here's the script that you can run in Scriptler and which will modify the email notification of all your jobs:

import jenkins.model.*
import hudson.plugins.emailext.*
import hudson.plugins.emailext.plugins.trigger.*
import hudson.tasks.*
import hudson.maven.reporters.*
import org.jvnet.hudson.plugins.groovypostbuild.*
 
def instance = jenkins.model.Jenkins.instance

def jobNames = [
 "xwiki-commons.*",
 "xwiki-rendering.*",
 "xwiki-platform.*",
 "xwiki-enterprise.*",
 "xwiki-manager.*"
]
 
def deleteEmailConfiguration(def item, def publishers)
{
    // Remove the Maven Mailer if any since we want to use the Mail Ext plugin exclusively
    def reporters = item.reporters
    def mavenMailer = reporters.get(MavenMailer.class)
    if (mavenMailer) {
      println "  - Removing default mailer reporter with recipients [${mavenMailer.recipients}]"
      reporters.remove(mavenMailer)
    }

    // Remove any default Mailer Publisher
    def mailer = publishers.get(Mailer.class)
    if (mailer) {
      println "  - Removing ${Mailer.class.name} publisher"
      publishers.remove(Mailer.class)
    }

    // Remove any default ExtendedEmailPublisher Publisher
    def extMailer = publishers.get(ExtendedEmailPublisher.class)
    if (extMailer) {
      println "  - Removing ${ExtendedEmailPublisher.class.name} publisher"
      publishers.remove(ExtendedEmailPublisher.class)
    }
   
    // Remove any Groovy Postbuild Plugin
    def groovyPostbuild = publishers.get(GroovyPostbuildRecorder.class)
    if (groovyPostbuild) {
      println "  - Removing ${groovyPostbuild.class.name} publisher"
      publishers.remove(GroovyPostbuildRecorder.class)
    }    
}

 
instance.items.each() { item ->
  println "Checking ${item.name}..."

  def match = false;
  jobNames.each() {
    if (item.name.matches(it)) {
      match = true;
    }
  }

  if (match) {
    if (configureEmailForMatching.equals("true")) {
      println "  - Modifying ${item.name}..."
      
      def publishers = item.publishersList
      deleteEmailConfiguration(item, publishers)

      // Add the Groovy Postbuild Plugin
      def script = '''
import hudson.model.*

def messages = [
 [".*A fatal error has been detected by the Java Runtime Environment.*", "JVM Crash", "A JVM crash happened!"],
 [".*Error: cannot open display: :1.0.*", "VNC not running", "VNC connection issue!"],
 [".*java.lang.NoClassDefFoundError: Could not initialize class sun.awt.X11GraphicsEnvironment.*", "VNC issue", "VNC connection issue!"],
 [".hudson.plugins.git.GitException: Could not fetch from any repository.*", "Git issue", "Git fetching issue!"],
 [".*Error communicating with the remote browser. It may have died..*", "Browser issue", "Connection to Browser has died!"],
 [".*Failed to start XWiki in .* seconds.*", "XWiki Start", "Failed to start XWiki fast enough!"],
 [".*Failed to transfer file.*nexus.*Return code is:.*ReasonPhrase:Service Temporarily Unavailable.", "Nexus down", "Nexus is down!"]
]

def shouldSendEmail = true
messages.each() { message ->
  if (manager.logContains(message.get(0))) {
    manager.addWarningBadge(message.get(1))
    manager.createSummary("warning.gif").appendText("<h1>${message.get(2)}</h1>", false, false, false, "red")
    manager.buildUnstable()
    shouldSendEmail = false
  }
}

if (!shouldSendEmail) {
  def pa = new ParametersAction([
   new BooleanParameterValue("noEmail", true)
 ])
  manager.build.addAction(pa)
}
    '''

      def gp = new GroovyPostbuildRecorder(script, [], 0)
      println("  - Adding new ${gp.class.name} publisher")
      publishers.replace(gp);      
     
      // Add an Email Ext Publisher and configure it
      def ep = new ExtendedEmailPublisher()
      ep.defaultSubject = "\$DEFAULT_SUBJECT"
      ep.defaultContent = "\$DEFAULT_CONTENT"      
      ep.recipientList = "notifications@xwiki.org"
   
      // Add script to not send email for false positives
      ep.presendScript = '''
import hudson.model.*

build.actions.each() { action ->
  if (action instanceof ParametersAction) {
    if (action.getParameter("noEmail")) {
      cancel = true
    }
  }
}                                         
    '''
     
      ep.configuredTriggers.add(new FailureTrigger(
        email : new EmailType(sendToRecipientList : true,
          body : ExtendedEmailPublisher.PROJECT_DEFAULT_BODY_TEXT,
          subject : ExtendedEmailPublisher.PROJECT_DEFAULT_SUBJECT_TEXT)))

      // We don't want en email fro fixed builds now since it means build that have failed because of infra reasons will generate an email when they work again
      /*
      ep.configuredTriggers.add(new FixedTrigger(
        email : new EmailType(sendToRecipientList : true,
          body : ExtendedEmailPublisher.PROJECT_DEFAULT_BODY_TEXT,
          subject : ExtendedEmailPublisher.PROJECT_DEFAULT_SUBJECT_TEXT)))
      */
      println("  - Adding new ${ep.class.name} publisher")
      publishers.replace(ep);      
     
      item.save()
    } else {
      println "  - Skipping ${item.name}"
    }
  } else {
    if (removeEmailForNonMatching.equals("true")) {
      println "  - Removing email configuration for ${item.name}..."
      def publishers = item.publishersList
      deleteEmailConfiguration(item, publishers)
      item.save()      
    } else {
      println "  - Skipping ${item.name}"
    }
  }
}

After you run this script, if you check a job's configuration you'll find a Groovy postbuild step with the following script, which will set up a variable to tell the Email Ext plugin whether to skip sending an email or not:

import hudson.model.*

def messages = [
 [".*A fatal error has been detected by the Java Runtime Environment.*", "JVM Crash", "A JVM crash happened!"],
 [".*Error: cannot open display: :1.0.*", "VNC not running", "VNC connection issue!"],
 [".*java.lang.NoClassDefFoundError: Could not initialize class sun.awt.X11GraphicsEnvironment.*", "VNC issue", "VNC connection issue!"],
 [".hudson.plugins.git.GitException: Could not fetch from any repository.*", "Git issue", "Git fetching issue!"],
 [".*Error communicating with the remote browser. It may have died..*", "Browser issue", "Connection to Browser has died!"],
 [".*Failed to start XWiki in .* seconds.*", "XWiki Start", "Failed to start XWiki fast enough!"],
 [".*Failed to transfer file.*nexus.*Return code is:.*ReasonPhrase:Service Temporarily Unavailable.", "Nexus down", "Nexus is down!"]
]

def shouldSendEmail = true
messages.each() { message ->
  if (manager.logContains(message.get(0))) {
    manager.addWarningBadge(message.get(1))
    manager.createSummary("warning.gif").appendText("<h1>${message.get(2)}</h1>", false, false, false, "red")
    manager.buildUnstable()
    shouldSendEmail = false
  }
}

if (!shouldSendEmail) {
  def pa = new ParametersAction([
   new BooleanParameterValue("noEmail", true)
 ])
  manager.build.addAction(pa)
}

And the Email Ext plugin will also have been configured and it will contain a prerun script that verifies if it should send an email or not.

import hudson.model.*

build.actions.each() { action ->
 if (action instanceof ParametersAction) {
   if (action.getParameter("noEmail")) {
      cancel = true
    }
  }
}                                         

Of course you should adapt the messages the script is looking for in your build log and adapt it to your cases.

For our need we catch the following:

  • Check if the JVM has crashed
  • Check if VNC is down (we run Selenium tests)
  • Check for Browser crash (we run Selenium tests)
  • Check for Git connection issue
  • Check for machine slowness (if XWiki cannot start under 2 minutes then it means the machine has some problems)
  • Check if Nexus is up (we deploy our artifacts to a Nexus reporitory)

Enjoy!
...

Dec 20 2012

Devoxx 2012 Belgium

It was a long since I last attended Devoxx Belgium. It was a pleasure to be there again and meet all my friends. I was also happy to be able to present XWiki even though it was only for a quickie (15 minutes).

I presented the ability to quickly create applications within a wiki with the "Application Within Minute" feature of XWiki.

Here's the video:


...

Tags:
Created by Admin on 2013/10/22 14:34

This wiki is licensed under a Creative Commons 2.0 license
XWiki Enterprise 6.0.1-NODE1 - Documentation