Deploying from multiple Maven repositories with Chef

Introduction

A typical Git branching model will somehow link the contents of the ‘master’ branch with released-to-manufacturing code. For a project deploying builds to a binary repository such as Archiva, Artifactory, or similar, the branch-to-version map for a project using feature branches, Git Flow, or something else entirely might look similar to the following:

Because active development happens in feature branches, it’s a good idea to make such branches build SNAPSHOT builds, and to have a separate repository for snapshots to which they are deployed. This makes it possible to enforce rules on how long these builds persist in your binary repository independently of any released code (which, presumably, should be kept for an extended period of time, if not forever).

The community Maven cookbook makes it trivial to deploy code out of a binary repository and either onto a running machine, or into an image. Since the project has a repository both for releases and snapshots, it might be tempting to declare a resource similar to the following, which declares both repositories and pulls in the version through an attribute:

maven 'project' do
        artifact_id 'project'
        group_id 'com.companyname.projectgroup'
        version node['cookbook']['project_version']
        dest '/home/vagrant'
        repositories [
                'http://192.168.56.2:8080/repository/internal/',
                'http://192.168.56.2:8080/repository/snapshots/'
        ]
        action :put
end

This will make Maven error out with a seemingly cryptic error scenario - it finds the maven-metadata.xml from the “snapshots” repository, then decides to go look in “internal” for the artifact anyway. This fails, since no snapshots are in “internal”.

[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Maven Stub Project (No POM) 1
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-dependency-plugin:2.4:get (default-cli) @ standalone-pom ---
Downloading: http://192.168.56.2:8080/repository/internal/com/companyname/projectgroup/project/1.0.0-SNAPSHOT/maven-metadata.xml
Downloading: http://192.168.56.2:8080/repository/snapshots/com/companyname/projectgroup/project/1.0.0-SNAPSHOT/maven-metadata.xml
Downloaded: http://192.168.56.2:8080/repository/snapshots/com/companyname/projectgroup/project/1.0.0-SNAPSHOT/maven-metadata.xml (785 B at 3.4 KB/sec)
Downloading: http://192.168.56.2:8080/repository/internal/com/companyname/projectgroup/project/1.0.0-SNAPSHOT/project-1.0.0-20150719.022508-2.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4.550s
[INFO] Finished at: Sun Jul 19 02:29:26 UTC 2015
[INFO] Final Memory: 8M/20M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-dependency-plugin:2.4:get (default-cli) on project standalone-pom: Couldn't download artifact: Could not find artifact com.companyname.projectgroup:project:jar:1.0.0-20150719.022508-2 in temp (http://192.168.56.2:8080/repository/internal/)
[ERROR]
[ERROR] Try downloading the file manually from the project website.
[ERROR]
[ERROR] Then, install it using the command:
[ERROR] mvn install:install-file -DgroupId=com.companyname.projectgroup -DartifactId=project -Dversion=1.0.0-SNAPSHOT -Dpackaging=jar -Dfile=/path/to/file
[ERROR]
[ERROR] Alternatively, if you host your own repository you can deploy the file there:
[ERROR] mvn deploy:deploy-file -DgroupId=com.companyname.projectgroup -DartifactId=project -Dversion=1.0.0-SNAPSHOT -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]
[ERROR]
[ERROR]
[ERROR] com.companyname.projectgroup:project:jar:1.0.0-SNAPSHOT
[ERROR]
[ERROR] from the specified remote repositories:
[ERROR] central (http://repo.maven.apache.org/maven2, releases=true, snapshots=false),
[ERROR] temp (http://192.168.56.2:8080/repository/internal/, releases=true, snapshots=true),
[ERROR] temp (http://192.168.56.2:8080/repository/snapshots/, releases=true, snapshots=true)
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

Here is a test sandbox consisting of an Archiva VM, Java project to be deployed to the VM, and a Chef cookbook and corresponding VM to deploy to. A working Berkshelf and Vagrant environment is required to spin it up.

What’s the fix?

Thankfully, the fix here is simple - qualify all of the repositories passed to Maven.

maven 'project' do
        artifact_id 'project'
        group_id 'com.companyname.projectgroup'
        version node['cookbook']['project_version']
        dest '/home/vagrant'
        repositories [
                'snapshots::::http://192.168.56.2:8080/repository/internal/',
                'internal::::http://192.168.56.2:8080/repository/snapshots/'
        ]
        action :put
end

But why?

Under the hood, Chef is calling Maven in a similar fashion to the following:

 mvn org.apache.maven.plugins:maven-dependency-plugin:2.4:get \
        -DgroupId=com.companyname.projectgroup -DartifactId=project -Dversion=1.0.0-SNAPSHOT -Dpackaging=jar \
        -Ddest=/tmp/chef_maven_lwrp20150719-4894-1qssv5c/project.jar \
        -DremoteRepositories=http://localhost:8080/repository/internal/,http://localhost:8080/repository/snapshots/ \
        -Dtransitive=false

Though the documentation for the Maven dependency plugin isn’t terribly clear on this, it’s actually appending the qualifier “temp” to each comma-delimited repository that’s getting passed in. Maven interprets this as each repository passed in being a mirror of each other. When “maven-metadata.xml” is picked up from the “internal” repository (known to Maven as “temp”), it then makes the assumption that it will be able to download the artifact itself from “snapshots”, since the contents of “snapshots” and “internal” are assumed to be the same.

For additional edification, the format above that the “repositories” directive is expecting is: “id::[layout]::url”, where “id” is some unique identifier to Maven for the repository, “[layout]” is an optional parameter for repository layout, and “url” is the URL of the repository itself - what was being used on its own prior.

There must be a better way!

A better-yet solution is to use the “Repository Groups” feature of Archiva or “Virtual Repositories” feature of Artifactory to create a deployment repository which combines both. This ensures that both release and snapshot code will be available from a single URL.