Gitflow, Maven, and CI Done Right: Part 1 – Teaching Maven New Tricks

Bryan Varner - Featured Image

Bryan Varner, Author from E-gineering.com

A Note from Bryan Varner: Hi, I’m Bryan. This is my first blog post from E-gineering. I want it to be memorable. I want it to be epic. I want it to be informative. I want to convey how much time I’ve spent over the years looking for an elegant and functional method of creating a single Jenkins CI build job for a gitflow managed Maven project that would do the right thing for each environment without any of the technical tradeoffs I consider unsavory.

I wrote a lot. I wrote too much. What you see here is the terse version in which I’ve nixed all the academic insight, most of the witty banter, and the several pages worth of prologue. This intro paragraph is already too long, so without further ado…

We at E-gineering would like to introduce the gitflow-helper-maven-plugin.

 

What the gitflow-helper-maven-plugin is:

It’s a plugin for Maven >= 3.1 that teaches Maven to behave well when building every branch of a gitflow project on a CI server with a single Job definition. It also addresses some rather unsavory trade-offs and side-effects of the Maven repository system (only snapshot and release repositories? Yick!) and resolves the epic argument of ‘you should never build the master branch’ by promoting artifacts that were already built and deployed from other branches, preserving the checksum hashes and providing provable provenance of artifacts from staged test release to production release.

At E-g, we are using this plugin for real work with real clients with real results. We’ve also put it out there for the rest of the world to use, too. It’s published at central repository, and we’ve done all the development out in the open in our gitflow-helper-maven-plugin github repository. Since this is all open, we’d like to extend an invitation for your participation by reporting issues, requesting features, contributing code, and especially in trying it out and kicking the tires.

There’s more technical information and configuration documentation in the README of the current release. Yes, it works with CI servers other than Jenkins, and it ought to work with Repository managers other than Nexus OSS.

 

What the gitflow-helper-maven-plugin does:

Here’s a run-down of the plugin’s executable goals and what each goal will do when the project is being built from a specific type of gitflow branch.

gitflow-helper-maven-plugin goals
Type of Branch enforce-versions retarget-deploy promote-master tag-master
master Requires your project version to be a release version.
(non-SNAPSHOT)
Set repository:
release
Copies already deployed artifacts from stage to release.
Build extension ensures only our extension, and maven-deploy-plugin remain in the build.
creates tag
development N/A Set repository:
snapshot
Attaches a catalog of artifacts N/A
release non-SNAPSHOT, and branch name pattern matches pom.xml version Set repository:
stage
Attaches a catalog of artifacts N/A
bugfix N/A maven.deploy.skip=true N/A N/A
hotfix non-SNAPSHOT, and branch name pattern matches pom.xml version Set repository:
stage
Attaches a catalog of artifacts N/A
(others) N/A maven.deploy.skip=true N/A N/A
local builds
(undefined branch)
N/A maven.deploy.skip=true N/A N/A

For the purpose of this description I’m using the word “deploy” to refer to the maven-deploy-plugin definition — deploying project artifacts to a repository.

 

How the gitflow-helper-maven-plugin is used:

You can add the plugin to your own Maven project by including the following plugin definition:

<plugin>
<groupId>com.e-gineering</groupId>
<artifactId>gitflow-helper-maven-plugin</artifactId>
<version>1.2.2</version>
<configuration>
<releaseDeploymentRepository>${release.repository}</releaseDeploymentRepository>
<stageDeploymentRepository>${stage.repository}</stageDeploymentRepository>
<snapshotDeploymentRepository>${snapshot.repository}</snapshotDeploymentRepository>
<tag>${project.artifactId}-${project.version}</tag>
</configuration>
<extensions>true</extensions>
<executions>
<execution>
<goals>
<goal>enforce-versions</goal>
<goal>retarget-deploy</goal>
<goal>tag-master</goal>
<goal>promote-master</goal>
</goals>
</execution>
</executions>
</plugin>

You may have noticed the ‘stageDeploymentRepository’ property. If you didn’t, you should. If you’re using Nexus OSS, you can setup a repository with the ‘release’ policy, but configured to ‘allow redeployment’. When configured like this, hotfix and release branches will update the artifact in the repository with a new artifact each time they’re built, so you can test & fix gitflow release branch without doing really unsavory things like including the build number in the artifact.

To setup your CI server, create a new job for every branch in the repository, and have it invoke:

mvn clean deploy

When the CI job runs, if it’s building a

  • development branch
    • The project version must end in -SNAPSHOT or the build fails.
    • Artifacts are deployed to the ‘snapshot’ repository.
  • hotfix or release branch
    • The project version must be a release version.
    • Artifacts are deployed to the the ‘stage’ repository.
  • master branch
    • Only goals from the maven-gitflow-helper plugin, or the maven-deploy-plugin are executed. No Artifacts are built. No Tests Run. No Code Compiled.
    • The project version must be a release version.
    • Artifacts are downloaded from the ‘stage’ repository, and attached to the build.
    • Artifacts are uploaded to the ‘release’ repository, using the maven-deploy-plugin.
    • This implements a promote, not rebuild policy when on the master branch.
  • bugfix branch
    • The version number must be a release version.
    • Deployment is skipped.
  • feature branch (or any other branch on the CI server)
    • The version number must be a -SNAPSHOT version.
    • Deployment is skipped.
  • local build on your workstation (not on the CI server)
    • Deployment is skipped.

One CI Job does all that. Pretty awesome, huh?

But wait, there’s more! A new plugin goal that’s not bound to a lifecycle phase by default.

gitflow-helper:attach-deployed

Attach-deployed will first clean the project, then resolve artifacts from the same repository an ‘mvn deploy’ would target for the current branch. After artifacts are resolved they are copied into the build output directory and attached to the current maven build. This happens in the ‘package’ lifecycle phase leaving your build ready to run integration tests, verify, install, and deploy the artifacts, without having to recompile, test, and repackage.

This enables a short-circuit in the build process if you’d like to do completely sensible things like separating deployment to your artifact repository from deployment to your execution environment.

&nbsp

Putting it all together:

Let’s play pretend for a short example. Assume there is a Jenkins job that runs `mvn clean deploy` on every branch in a project. However it’s not desired for that “deploy” to actually put the artifacts into an execution environment. (Let’s agree to talk about that decision another time, Ok? Ok!) The job we have defined will make sure we get snapshots into the snapshot repo, testable releases (from release / hotfix branches) into the stage repo, and when we merge to master the staged artifacts will be copied to the release repo and the git repo tagged appropriately. Awesome.

But, what about getting those artifacts into an execution environment?

This is where the attach-deployed goal can come in handy.

One approach might be to create a job that lets you pick a branch to build, and have it run:

mvn gitflow-helper:attach-deployed jboss-as:deploy

This would download the already built (by your other job) artifacts and deploy them to your JBoss server. If you pick a:

  • development branch, it pulls the latest version from ‘snapshot’
  • hotfix or release branch, it pulls the latest version from ‘stage’
  • the master branch, it pulls the latest version from ‘release’

I think it’s a pretty nifty plugin, this gitflow-helper-maven-plugin. It solves a lot of issues I’ve run into over the years of doing CI with Maven. It alleviates CI job bloat, eliminates complex shell scripts to handle per-environment goal selection and property settings for branch-based builds. This plugin is also is the first solution I’ve found that makes a release into master as simple as a pull request merge, without rebuilding the already tested artifacts, and automatic tagging.


So that’s my first post. Awesome. Now that that’s out of the way, part two will be a step-by-step guide of how to make this work with BitBucket Server (Formerly Stash), Jenkins CI, and Nexus OSS complete with screenshots.

The following two tabs change content below.
Bryan Varner is a software developer and consultant at E-gineering, LLC in Indianapolis. Bryan and has been professionally developing software since 2002, and has a history of contributing to Free and Open Source Software. When not at work Bryan is a husband and father to three children and enjoys spending time making things (wooden furniture) you can actually see and touch.
Authors

Related posts

8 Comments

  1. CC said:

    Just wondering how this can be used in case of multimodule maven projects. In that case I need to update version in every child project before creating release or hotfix branch.

  2. Johan said:

    I’m intrigued by this statement about “the epic argument of ‘you should never build the master branch’ […]”.
    Do you have any reference material as to why this is such a good practice?

  3. Robert Andersson said:

    Awesome plugin, thanks!

    Unfortunately our corporate IT has upgraded to Nexus Repository Manager Pro release 3.

    This does not support staging.

    Any suggestions how to setup the plugin to get most of it working even without a staging repository?

    • Robert Andersson said:

      Ah, looks like this is answered in part 2 of this article.

      I will make a new nexus repository named “stage” with “Release” version policy and “Allow redeploy” deployment policy.

*

Top