Continuous Versioning with Git and Gradle

Semantic Versaioning

Everybody knows semantic versioning. I think it's still good for software sold in boxes, regardless if the real paper ones or as downloads. But for continuous deployment it does not seem to be good enough.

Looking at the semantic version number does not tell you much. It is barely more than "hey, something was changed and the change is/maybe/should-not be disturbing". Not mentioning that the developers tend to forget to change the version number. And even when they do, it is unnecessarily  difficult to find what exact changes, i.e. commits, are in the semantically versioned release.

Continuous Versioning

Here comes what I call continuous versioning. It's more principle than exact versioning pattern, although I am going to suggest this one:

${semantic_version}-${commit_timestamp}-${commit_id}.

As you see the semantic version is still there, mainly because people like to see something familiar when you change things :). The main point is to add information about the last commit the change release contains.  The commit timestamp is there to give the version numbers nice chronological ordering. For developers is the most useful the last part - id of the commit, for which we use short hash of Git commit.

As you see there is no big demand for developers to increment the semantic version - it's nice if they do that, but each artifact still gets unique version number if they do not. What's even better, all the information can be gathered during build and used for various artifacts the build can produce - nowadays often executable package and Docker image.

The example how to achieve the described versioning with Gradle and use it to tag a Docker image artifact is below. To get information from Git we use both com.palantir.git-version Gradle plugin and Git command line, because the plugin does not provide timestamp info.

plugins {
    id 'com.palantir.git-version' version '0.12.3'
    id 'com.bmuschko.docker-remote-api' version '6.1.2'
}

def semanticVersion = '1.0.0'
def gitTimestamp = { ->
    def stdout = new ByteArrayOutputStream()
    exec {
        commandLine 'git', 'show', '-s', '--format=%ct', 'HEAD'
        standardOutput = stdout
    }
    return stdout.toString().trim()
}
def gitVersion = details.gitHash

version = semanticVersion + "-" + gitTimestamp() + "-" + gitVersion

// ...

import com.bmuschko.gradle.docker.tasks.image.DockerBuildImage


task buildDockerImage(type: DockerBuildImage) {
    // custom task to prepare files ind build/docker directory:
    // dependsOn  "prepareFilesForDockerBuild"
    inputDir = file('build/docker')
    images.add("mycooldockerrepo.com/${project.name}:${version}")
}

import com.bmuschko.gradle.docker.tasks.image.DockerPushImage

task pushDockerImage(type: DockerPushImage) {  
    dependsOn "buildDockerImage"

    images.add("mycooldockerrepo.com/${project.name}:${version}")
    
    registryCredentials {
       url = "mycooldockerrepo.com"
       username = System.getenv('docker_user') ?: "${docker_user}"
       password = System.getenv('docker_password') ?: "${docker_password}"
    } 
}

To package the version info inside the Docker image, we can define custom task prepareFilesForDockerBuild and uncomment the dependsOn line of buildDockerImage().
To save the version number into a file, e.g. named docker/version.txt, this should be inside that task:

new File("docker/version.txt").text = "${version}"

I hope this article helps somebody to look at versioning schemes from a newer point of view. I will add Maven version when I have one but my recent projects seem to be Gradle-only, so it coudl take time - feel free to post yours to share.

Git with Attlasian Stash via SSH, on Windows

I do not know why Atlassian decided to name the thing Stash when there was  Git stash, probably some "clever" marketing.

I used Git for some time but just recently I had to use the SSH protocol on Windows  to authenticate to Stash. Integrating Linux-born software to Windows environment is sometimes easy but that was not this case - to prevent more people to curse Git because of that, I made the following HOWTO.

Install Git

First of all you need to install Git and configure access using SSH protocol.  I decided to not try anything fancy and downloaded the standard git client installation file. There is several dialogs during installation, I recommend to use "Run Git from Windows Command Prompt" in the "Adjusting PATH Environment"second dialog. The most important option is "Choosing SSH executable" - select "Use OpenSSH" - the Git will then use OpenSSH included in GIT installation.

I installed Git into directory C:\git, feel free to choose installation path that suites your needs. I would recommend to avoid using path wit space though.

When finished, open powershell console and go to directory to C:\git\bin.

Generate SSH Keys 

Following command generates new pair of keys (private will stay with you and public will be given later to Stash for you authentication).

C:\git\bin> .\ssh-keygen.exe -t rsa -C your.email@address.com

Set basic key file name to C:\git\bin\id_rsa when asked for the keys. To make things simple, use empty pass-phrase (you can fortify your setup later when you get more familiar with OpenSSH). The conversation with ssh-keygen should look like this:

Generating public/private rsa key pair.
Enter file in which to save the key (//.ssh/id_rsa): C:\git\bin\id_rsa
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in C:\git\bin\id_rsa.
Your public key has been saved in C:\git\bin\id_rsa.pub.

Configure Bundled OpenSSH

Create directory C:\git\.ssh and move the newly created files id_rsa (private key) and id_rsa.pub (public key) into it. Then create file C:\git\.ssh\config with following content:

Host stash.atlassian.com
    HostName stash.atlassian.com
    PreferredAuthentications publickey
    IdentityFile /.ssh/id_rsa

In case your company has its own Stash instance running, adapt the Host and HostName options accordingly.

Submit Public Key to Stash

Login to your Stash account.  In the top right corner, from select from drop-down menu "Manage account".  On left side of the page you should now see several links, click on "SSH Keys".

Put content of your public key (id_rsa.pub) into the text of the provided form. Do not edit it the content at all. Removing, regardless if deliberate or accidental, of some  white space characters, namely line breaks does not prevent Stash to accept the key but the authentication will fail.

Test Connection

First test the connection using only OpenSSH:

C:\git\bin> .\ssh.exe -v -T git@stash.atlassian.com -p 7999

If everything works well, the output you get ends with sentences "Authentication succeeded." and probably also "Entering interactive session." If so, you can move C:\git\bin\.ssh directory to C:\your_windows_login\ directory and retest that the connection still gets authenticated with the files in new location:

C:\git\bin\ssh.exe -v -T git@stash.atlassian.com -p 7999

Clone

If everything worked as described so far you should be now able clone your Stash-hosted repository using instructions available from Stash profile or project page.

Fighting Crawling Maven Dependency Changes With Git

You know the situation well - everything goes fine, continuous integration does what is used to do and new releases are produced on regular basis.

Then, out of the blue, integration tests fail with Java complaining about two different versions of the same library on classpath. You can be less lucky and use different version than you think, or different implementation of an API, or your JAR/WAR/whatever-package just grows without reason. Lot of bad things can happen when dependecies get out of control.

It does not depend on the source - one of your colleagues changing something without double-check or changes in some already used library. To fight these problems I tried to use Git pre-commit hook you can see below. It is not perfect but it works. It depends on Maven, if you use Ivy, Gradle or something else, you will have to adapt it.

Just put it in you projects' /git/hooks named pre-commit. When dependency changes are detected, they are printed and commit is aborted - you will have to use diff tool on deps.txt (old dependencies) and deps_tmp.txt (new dependencies) and update deps.txt if the dependency change is desired. If the change is unwanted, use mvn dependency:tree to find the source of problems.

It has only one more constraint -- all people using it must use the same version of Maven. Because of changes in dependency resolution in Maven 3,  Maven dependency plug-in produces slightly different list of changes in versions 2 and 3.

#!/bin/sh

mvn dependency:list -DincludeScope=compile | sort | grep jar \
| cut -c 11- | cut -d":" -f 1-4 | uniq >deps_tmp.txt

diff -a -B -b -u deps.txt deps_tmp.txt >deps_diff.txt

lines=`cat deps_diff.txt | wc -l`

if [ $lines -ne 0 ]; then
   echo "Dependency change detected - check 'mvn dependency:tree' and update deps.txt."
   cat deps_diff.txt
   if [ -a deps_diff.txt ]; then rm -f deps_diff.txt; fi
   exit 1
fi