Dockerised DropWizard

What is it?

It started as an exercise that was supposed to satisfy few, artificial requirements for a technical interview.

I have never attended the interview but most of the solution was ready and I thought I will keep going to learn new and solidify existing knowledge.

It incorporates few techniques and technologies and should be useful reference for projects using:

  • Maven to orchestrate the following:
    • build code
    • unit test
    • package
    • build Docker images
    • execute integration tests against Docker container
    • execute e2e tests against all containers (just a demo)
    • push Docker images to registry
      Simple dependency mechanism is used to chain the above steps. It increases number of modules but makes each module manageable and enables concurrent execution.
  • Fabric8 Docker Maven Plugin is a key plugin making it much easier to build, run and push Docker containers
  • Usage of Maven Wrapper to ensure there are no output deviations because of different version of Maven is used.
  • Docker (building and pushing images, running containers for testing purpose)
  • DropWizard - my first encounter with DropWizard
  • JDBI as an alternative to Hibernate and JPA Repositories
  • Liquibase - I haven't used Liquibase before but did use Flyway and wanted to get a feel for both
  • running parametrised jUnit tests with JUnitParams
  • externalisation of sensitive details into files

The functional aspect of the code itself is not that relevant as the project became just an experimentation ground.
Originally it was supposed to be a REST API that allows for creating user and storing its details in MySQL database.

The build process (and code) was tested on Ubuntu (native Docker) and Windows 7 (docker machine with default boot2docker guest). Soon I am expecting to verify it on Windows 10 with native Linux subsystem support.

Code available on BitBucket.

Contents

General prerequisites

The following steps are not specific to this project and you may already have all of these in working order.

On Windows use Bash command-line interpreter

I'm using MINGW as it was readily available as part of Git for Windows installation. Few scripts and commands are leveraging commands that are normally unavailable in Windows Command Prompt.

Docker Machine

If using Windows without support for native linux containers (here Windows 7)

Install Docker Toolbox (not Docker for Windows) from Docker site.
I have found it's best to first uninstall VirtualBox (if that's your choice of virtualisation platform).
Once that's done install Docker Toolbox with the optional VirtualBox package selected.
Trying to reuse existing installation of VirtualBox worked for me but was incredibly slow (I haven't figured out what went wrong yet).

If you want to try reusing existing VirtualBox installation, create virtual machine with:

1docker-machine create \
2    --driver virtualbox \
3    --virtualbox-cpu-count "2" \
4    --virtualbox-memory "8000" \
5    --virtualbox-disk-size "20000" \
6    default

Docker Toolbox contains docker and docker-compose cmd-line tools. You may also be tempted to install docker host on any Linux and configure all manually but that's a bit of extra work.

If you are running under Linux host, then you only need to install docker-ce and docker-compose.
Have a look here for docker-compose and here for docker-ce (for Ubuntu).

If you are running on a Windows OS host with support for native linux containers install Docker for Windows. More info about this feature and fairly generic (non-Docker related) video introduction.

Prepare Docker Registry

You can use Docker hub or local registry, amongst other options. These steps are for AWS ECR as I am consuming the Docker images from within AWS.

1. Install AWS-CLI if you didn't already

It will be used to configure credentials for AWS services and will generate docker login command. This command will enable your docker instance to authenticate within ECR.
AWS-CLI download and installation instructions at https://aws.amazon.com/cli

2. Create AWS user

Go to AWS IAM service at https://aws.amazon.com/iam and create a new user or amend existing one. For this activity you should only need programmatic access (AWS console access optional).
You need to add permissions described by policy AmazonEC2ContainerRegistryFullAccess (within IAM).
You can fine-tune the policy details to make it more tight. Make a note of your AWS Access Key ID and AWS Secret Access Key.

3. Configure credentials for AWS

If you run the following command you will be asked for your AWS Access Key ID, AWS Secret Access Key and default region. You should have all of these available from previous steps.

1aws configure

4. Verify you can authenticate docker with AWS ECR

1$(aws ecr get-login --no-include-email --region=<aws_ecr_region>)

Initial, one-time, project specific setup

Clone the code

git clone git@bitbucket.org:outo/dockerised-dropwizard.git

Setup Docker Registry

1. Create empty ECR repositories.

Go to https://aws.amazon.com/ecr/ and log into your AWS account. Select region where the registry is going to sit.
Create the following repositories:

  • dockerised-dropwizard/mysql-migrated
  • dockerised-dropwizard/user-service

2. Provide domain of Registry URI in project's config file

Make a note of the Repository URI shown on the top of two newly created repositories.
You need to provide registry's domain (same for both repositories) in
<project_top>/config/sensitive/tooling.properties.
It is done under the following key, format for reference:
DOCKER_REGISTRY=<aws_acc_no>.dkr.ecr.<aws_ecr_region>.amazonaws.com

3. Provide configuration for mysql docker container

In config/sensitive/container.db.properties supply:

1DB_ROOT_PASSWORD=<db root password>
2DB_USERSERVICE_USERNAME=<username for userservice db user>
3DB_USERSERVICE_PASSWORD=<password for userservice db user>
4DB_USERSERVICE_NAME=<database name for userservice>

4. Provide configuration for userservice REST endpoint:

In config/sensitive/container.userservice.properties supply:

1USERSERVICE_DB_USERNAME=<username for userservice db user>
2USERSERVICE_DB_PASSWORD=<password for userservice db user>
3USERSERVICE_DB_NAME=<database name for userservice>
4USERSERVICE_APP_PORT=8080
5USERSERVICE_ADMIN_PORT=9080

Repetitive steps

1. On Windows (without native linux support) start docker machine

For docker commands to work start docker machine if it is not running (docker-machine ls to check)

1docker-machine start

Also, few environment variables need to be populated, all done with:

1eval $(docker-machine env)

On Linux host this step is not needed.

2. Docker has to be logged into registry (here ECR)

1$(aws ecr get-login --no-include-email --region=<aws_ecr_region>)

This step needs to be done for any host OS.

aws ecr get-login will generate docker login command which we just evaluate here.

Session lasts few hours.

Note: Fabric8's DMP can authenticate you automatically. I just didn't find time to try it. More info here.

3. Managing the version of the maven artifacts/docker images

There is no automatic mechanism in place [yet] to auto-increment the version.
Manually it can be done with:

1mvn versions:set -DnewVersion=1.0.5
2# followed by
3mvn versions:commit 
4# if you are happy with the result, or revert version change with 
5mvn versions:revert 

4. Building the project

To build, run unit, integration and e2e tests execute:

1./mvnw clean install

Note:
Don't use maven command directly (mvn) on your machine as the version of Maven may not be what's required. For example the top level pom file requires property maven.multiModuleProjectDirector which has been introduced in version 3.3.1.
Maven wrapper (<project_top>/mvnw) ensures the correct maven version is used for the build and the results are less of a lottery.

This will also build docker images and store them locally.
To list the images use:

1docker images

If you also want to push the images to ECR (conditional on all tests passing) run:

1./mvnw clean install -DdockerPush

Note: You could move away from a cmd-line property and use profiles for this. I might even do so if I find enough motivation.

5. Running

If you want to spin all containers locally for debugging/experimentation purposes run:

1./docker-compose-up.sh

To run code directly from IDE against some of the containers run

1./docker-compose-up.sh db

which will only spin up container db (running MySQL)

It will allow you to run any code, and more importantly - debug it, from your IDE.

Additionally, when running against docker machine, for the above case to work you also have to provide docker machine ip output for dockerHostIp property in top-level pom. On Linux it can be left as default (localhost) if default network setup is used.

Gotchas

Cannot connect to MySQL

I've spent way too much effort trying to find a problem with connection to MySQL Docker container.

At the time, I didn't know if it was caused by the configuration of MySQL, usage of Docker machine, firewall, choice of network within Docker machine, network model within Docker or just the way I referred to the MySQL container host.

There are two things to bear in mind, depending on your network setup and use case:

  • bind MySQL to network interface that you will be connecting via (the most inviting, but insecure is 0.0.0.0). Have a look at bind-address entry in <project_top>/db/src/main/docker/my.cnf and corresponding changes in <project_top>/db/src/main/docker/assembly.xml and <project_top>/db/src/main/docker/Dockerfile.
    Keep in mind that different MySQL version than used here may have different default value of bind-address and may accept different values (e.g. list from MySQL 8.0.13 onwards).
    The way this project is configured (MySQL version and Docker network) worked fine with default VirtualBox network setup (NAT + Host Only). You may even delete my.cnf but it is better to be aware where this setting is and how to change it (for security reasons).

  • bind any container to correct network interface. That came out during Fabric8 DMP container run (integration and e2e testing). Have a look at apis or e2e pom files. Under Fabric8 DMP configuration the ports/port value binds the containers to "0.0.0.0". It is as permissive as it gets but not necessary secure enough for you, depending on usage case.

Fabric8 DMP prematurely reports MySQL container as "ready"

This sounds like but is no fault of guys at Fabric8.
This plugin provides a variety of good mechanisms to report container as ready to use. One of them scans the standard out of a Docker command and - should a pattern match - allows for depending containers to proceed. In case of MySQL I went for a piece of text somewhat similar to:
/usr/sbin/mysqld: ready for connections. and it would work, if not the fact that MySQL outputs similar line twice

1/usr/sbin/mysqld: ready for connections. Version: '8.0.11'  socket: '/var/run/mysqld/mysqld.sock'  port: 0  MySQL Community Server - GPL.
2...
3/usr/sbin/mysqld: ready for connections. Version: '8.0.11'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  MySQL Community Server - GPL.

{: class="skipnos"}

Just match it on the specific port:

1<log>(?s)mysqld: ready for connections.*  port: 3306  MySQL</log>

Docker terminates early with error code 127

The error code 127 usually indicates that a command couldn't be found (if anything). It was cryptic as I couldn't recollect any change that would be related.

One of my aims for this project was to make this build process work on Linux and Windows. I was moving frequently between the two systems, making changes while using both of them. That caused the line endings to be saved as CRLF on Windows and LF on Linux. All containers in this example are running on Linux derived images as well.

It came down simply to line endings which corrupted configuration files and scripts.

1git config --global core.autocrlf true

Externalised config for interpolation in "docker-compose.yml"

This is relating to interpolation of external config file with key/value pairs within "docker-compose.yml" (not for the runtime of containers themselves).

I've already had few externalised property files in a specific location (which is Git ignored). It would be great to just point docker-compose to it but I couldn't due to the imposed limitations:

  • it's name as it has to be ".env"
  • located at same level as "docker-compose.yml"
  • the above implies it has to be a single file

So I decided to merge the few properties files into one ".env" and place it where I didn't want to:

1awk '/.*=.*/{print $0}' ./config/sensitive/*.properties > .env

So I've just moved sensitive config into another place in the directory tree. To mitigate the risk of this being left behind I run docker-compose in detached mode -d. As soon as the containers are being started the sensitive file could be removed.

But could it?

It appears that docker-compose logs complains about the lack of properties, although I cannot see a good reason for it. After experimentation I realised it won't throw abuse at me if I leave the ".env" files with empty keys only, hence:

1awk -F'=' '/.*=.*/{print $1 "="}' ./config/sensitive/*.properties > .env

Which put me at ease.

Liquibase baseline and change log configuration

Liquibase was insisting for me to configure MySQL connection url in order to generate the SQL. I have made a decision to generate a complete SQL migration script first (without a baseline), put it into a MySQL Docker container and, then, I wanted to start it for the first time. It wasn't very obvious (at first) how to work around the mandatory connection uri. In the end it works with offline url which barely states the driver type and version.

1url=offline:mysql?version=8.0.11&changeLogFile=db/target/changeLogFile.csv

The changeLogFile parameter places the change log within the target directory to get rid of it on each mvn clean. I did it as I don't want to have long-lived instance for this demo. I am happy with the schema (not the data) to be the only thing imported into the container at this stage. Data import has to be more controlled anyway for sensitive applications.

JDBI unable to map non-primitive parameter or return value

The interface <project_top>/apis/src/main/java/com/sebwalak/db/UserDAO.java has query findAll, which returns a collection of User objects. I have overlooked a need for custom mapper to translate the result-set into a User. It transpired that I need to define a mapper, an example at <project_top>/apis/src/main/java/com/sebwalak/db/UserRowMapper.java.

Running code within IDE against a set/all of containers

Use docker-compose-up.sh to spin all or required containers, e.g.:

1./docker-compose-up.sh db app
2# in this case is equivalent to running the above command without the two extra parameters
3./docker-compose-up.sh

If you are running against Docker machine then put its IP address as value of dockerHostIp Maven property in top pom file.

If the test code requires maven properties or environment variables which are missing, PropertiesAndEnvironmentVariablesAsserter will fail the test and point out what is missing.

Non-sensitive values can be provided via surefire plugin configuration in the appropriate pom. Sensitive values are better provided via IDE, so they are not committed.

Credits

DropWizard

Maven - Introduction to the lifecycle

Fabric8 Docker Maven Plugin

Liquibase

Maven Assembly Plugin

Maven Properties Plugin

StackOverflow - negating property value in pom.xml

DZone - working with JUnitParams

AVAJAVA - displaying a value of property

RestAssured