Using Nexus 3 as a Private Docker Registry

Stephan Hochdoerfer - Featured Author

Stephan Hochdoerfer, Author

For a long time I was planning to use Docker at my company bitExpert AG but realized that if we wanted to make use of Docker we needed a private registry to store (and share) the different containers needed for our projects. Unfortunately most Docker registry implementations I came across are rather simple and thus have no support for authentication of any kind.

For us this was a no go. Thus I was very excited when I found the news that the next major release of Nexus will come with full support for a Docker registry.

A Brief History

bitExpert has used Nexus for more than 5 years now. In the past we used Nexus only for our Java teams where Nexus mainly acts as a Maven proxy as well as a Maven repository for some of our own packages that we cannot share publicly. When Milestone 7 of Nexus 3 was available, I set it up and started playing with it. Follow the next steps to figure out how we have set up the Docker registry with Nexus 3.

Steps to Setup the Docker Registry

Currently we use the hosted docker repository feature, Your configuration might need to be different if you also want Nexus to act like a proxy for docker hub.

In our case we run Nexus 3 as a docker container itself and use haproxy to route requests the respective docker instances based on the hostnames of the current request. haproxy is also responsible for SSL termination which also solved a problem I run into during the docker repository configuration with Nexus 3.

To create a new hosted Docker repository open the repositories view in the administration module of Nexus 3. Click the “Create repository” button and select the “docker (hosted)” recipe.
Click to view the full sized image: Create a private Docker registry

 

In the next screen you need to define a name of the repository e.g. use “docker”. For the repository connector choose “HTTP” and define a port where the registry will listen on. Since the default Nexus 3 instance is running on port 8081, I simply used 8082 for the docker registry.

Last but not least, you need to configure the blob store where Nexus will store the docker images. I tried to also configure an HTTPS port but received an error in the Nexus logs. Since the error was pretty generic I could not find a solution, thus I switched to HTTP only and let the haproxy instance handle the SSL termination.

Personal Requirements

One of my requirements was to not use a different port or a different subdomain to expose the docker registry. I felt this would make things to complicated to communicate since we will start using Nexus also to host and proxy npm and Bower packages.

Luckily the docker command line client defines a custom user agent which seemed the best way to distinguish between „normal“ Nexus requests and the specific docker requests. Luckily haproxy is pretty flexible, find below the configuration of the haproxy frontends and backends for our setup. A haproxy frontend defines how requests should be forwarded to the haproxy backends. We define frontends for both the 80 and 443 ports we want to serve Nexus as well as the Docker registry from.

<frontend haproxy-http>
<bind 192.168.65.1:80>
<acl host_is_nexus hdr(host) -i nexus.loc>
<acl client_is_docker hdr_sub(user-agent) -i docker>

<mode http>
<reqadd X-Forwarded-Proto:\ http>
<use_backend docker-backend if host_is_nexus client_is_docker>
<use_backend nexus-backend if host_is_nexus !client_is_docker>

<frontend haproxy-https>
<bind 192.168.65.1:443 ssl crt /etc/ssl/private/nexus.loc.pem>
<acl host_is_nexus hdr(host) -i nexus.loc>
<acl client_is_docker hdr_sub(user-agent) -i docker>

<mode http>
<reqadd X-Forwarded-Proto:\ https>
<use_backend docker-backend if host_is_nexus client_is_docker>
<use_backend nexus-backend if host_is_nexus !client_is_docker>

As you can see we define 2 checks: One for the hostname (nexus.loc in this example) and one for the user agent. Requests without the docker client specific user agent will get forwarded to the regular „nexus-backend“ which will serve the default Nexus site. Requests containing the docker client specific user agent will get forwarded to the „docker-backend“ which will connect to the Docker registry running in Nexus.

The haproxy backend configuration looks like this:

<backend nexus-backend>
<balance leastconn>
<cookie JSESSIONID prefix>
<mode http>
<option httpclose>
<option forwardfor>
<redirect scheme https if !{ ssl_fc }>
<server node1 127.0.0.1:8081>

<backend docker-backend>
<balance leastconn>
<cookie JSESSIONID prefix>
<mode http>
<option httpclose>
<option forwardfor>
<redirect scheme https if !{ ssl_fc }>
<server node1 127.0.0.1:8082>

The only difference of both configuration sections is that the the „nexus-backend“ redirects to port 8081 (the default port Nexus 3 is running on) while the „docker-backend“ configuration forwards the requests to port 8082 which is the port we configured in the docker hosted repository configuration. After restarting haproxy you are able to connect to.

Login to the Private Docker Registry

To login to the private docker registry use the following command and then pass your Nexus 3 user credentials. In our setup Nexus 3 is configured to read the users from our internal LDAP server giving our users the convenience to use the same set of credentials as they use to access other applications.

<docker login nexus.loc>

In case the command above gives you a “client is newer than server” error, try to force the docker client to run with a different api version. I got this error after installing the M7 release as well as after upgrading to the final release. Error was gone after I rebooted my system. As a temorary fix you can „force“ the client to use a specific API version by setting the  DOCKER_API_VERSION environment variable. In my case the server was running version 1.21 thus all I needed to do was this:

<DOCKER_API_VERSION=1.21 docker login nexus.loc>
The following two tabs change content below.
Stephan Hochdörfer currently holds the position of Head of Technology at bitExpert AG, a company specializing in software and mobile development. His primary focus is everything related to web development as well as automation techniques ranging from code generation to deployment automation.

Latest posts by Stephan Hochdörfer (see all)

Related posts

3 Comments

*

Top