Testcontainers, Bringing Sanity to Integration Testing

Writing and maintaining integration tests can be a difficult and frustrating experience, filled with a veritable minefield of things that could go wrong. For integration tests that connect to a remote resource you have issues of: the resource being down, datasets being changed or deleted, or heavy load causing tests to run slow. For integration tests that connect to a local resource you have the initial install and configuration of the resource on your local machine and the overhead of keeping your local instance in-sync with what the production instance looks like, otherwise you might run into this situation:Source: Minesweeper – The Movie

No application operates in isolation. Applications, even “monoliths”, depend on remote resources be it: databases, logging services, caches, or other applications to function. Just like the application we are maintaining will change over time as business needs and client demands change, so too will the resources it depends on. This necessitates a need to continually verify that our application can communicate with its dependent resources.

So to maintain development velocity and while having confidence our applications will function properly in production we need to write automated integration tests, however we need our integration tests to be:

• Reliable – Test failures should only happen because a change occurred in either our application or the resource, not because the resource is down or misconfigured.
• Portable – The tests should be able to run anywhere with minimal setup.
• Accurate – The resource being used in the integration test should be an accurate representation of what exists in production.

How do we accomplish these requirements?

Introducing Testcontainers

Testcontainers is a Java library that integrates with JUnit to provide support for starting up and tearing down a Docker container within the lifecycle of a test or test class. Testcontainers is a project that was started about four years ago, and I first learned about back in 2017 when I was putting together a Pluralsight video on automated testing.

I have noticed an uptick in interest in Testcontainers in my twitter outline recently, and it doesn’t seem long ago that Testcontainers passed the 1K stars mark on their github repo, which now sits at 2.2K. If you haven’t started familiarizing yourself with Testcontainers now would definitely be a good time.

This rapid increase in popularity is likely the result of Testconainers being easy to use, and the flexibility of Docker containers, allowing Testcontainers to address a lot of integration testing use cases. In this article we are going to look at two approaches of how to use Testcontainers for running an integration test against a database. The code examples will be using JUnit 5, if you want to get familiar with JUnit 5, I have written a lot about it, you should also check out the JUnit 5 user docs.

Launching a Testcontainer via JDBC URL

In the example we will be writing an integration test for connecting to a Postgresql database, Testcontainers does offer support for a number of other databases. The first step will be brining in the appropriate dependencies. For this example we will only need to add the Postgresql Testcontainers dependency, to our maven build file (which in turns brings in the Testcontainers JDBC and core libraries).

Full maven build file for this project can be found here.

With the appropriate dependencies imported, let’s look at how to use Testcontainers to write a database integration test.

Full class, including imports, here.

There is quite a bit going on, let’s breakdown what is happening in this class into more easily digestible bites.

@TestPropertySource("classpath:application.properties")

This isn’t really related to using Testcontainers, but since ApplicationContextInitializer (javadoc) isn’t super well known, but can also be really helpful when writing automated tests, I wanted to take a moment to show how to make it easier to work with when used in test classes.

Here I am telling the test class to bring in the properties defined in /src/test/main/application.properties (source). By bringing in the properties defined in application.properties, instead of having to define every property needed for connecting to the Testcontainers database, only the properties that are different for the tests in this class need to be overwritten. This reduces maintenance needs and helps with overall test accuracy as it is easier to keep a single properties file in-sync with what production looks like.

public static class Initializer implements ApplicationContextInitializer {
   @Override
   public void initialize(ConfigurableApplicationContext applicationContext) {
      TestPropertyValues.of("spring.datasource.url=jdbc:tc:postgresql:11.2://arbitrary/arbitrary", //
      "spring.datasource.username=arbitrary", //
      "spring.datasource.password=arbitrary", //
      "spring.datasource.driver-class-name=org.testcontainers.jdbc.ContainerDatabaseDriver")//
      .applyTo(applicationContext);
   }
}

Within Initializer four properties are being defined (overwritten), and a few of them have somewhat odd looking values, let’s take a closer look. When initializing  Testcontainers via the JDBC URL, Testcontainers will set the username, password, hostname, and database name to what ever values you pass it. Strictly speaking spring.datasource.username and password don’t need to be included as they are defined in application.properties.  For spring.datasource.url, the JDBC URL must start with jdbc:tc:. The 11.2 refers to the specific image tag of postgres to be used, this however is optional and would default to 9.6.8 if left out. Lastly, spring.datasource.driver must be set to org.testcontainers.jdbc.ContainerDatabaseDriver. ContainerDatabaseDriver is Testcontainers’ “hook” into this test class. After starting up the container, ContainerDatabaseDriver will be substituted with the standard database driver, in this case org.postgresql.Driver. While in this example I am using the base postgres image in this example, you can use a custom image, so long as the database within the container is postgres (or of the type of database you have brought in a dependency for).

The rest of the test class is comparatively simple and straightforward. Simple read and writes are being performed to ensure fields are being properly mapped and the generated id matches the expected pattern.

Using Testcontainers as a Class Field

Above we looked at how to use Testcontainers via the JDBC URL hook. This can be a great when your use case is pretty simple, however the complexities of applications in the real world often mean a need for greater control and customization in behavior.

First step would be to bring in the Testcontainers junit-jupiter library.

There are a lot of similarities with the previous code example, so lets focuses only on the differences.

At the top of the test class, is the @TestContainers annotation. This brings in the Testcontainers extension into the class which scans for fields annotated with @Container such as in this case PostgreSQLContainer container. A @Container field can be either static or an instance field. Static containers are started only once and are shared between test methods, instances containers are started and stopped for each test method.

@Container
private static PostgreSQLContainer container = new PostgreSQLContainer("storm_tracker_db:latest");

Here the container that will be used in this test class is defined. Like with the JDBC URL method, you are not required to use a base postgresql image, in this case the customer image “storm_tracker_db” is being used (the Dockerfile for this image is here). As long as the database within the container is postgres, you are fine. While not much additional customization is being done to the container in this class. Testcontainers does offer a number of options such as: executing commands, setting a volume mapping, or accessing container logs, among others. Be sure to check the documentation under features and modules for what is available, as well as the javadoc (v1.11.1).

These additional features provided when using a Testcontainer as a class field allow for flexibility in putting the container within a specific state for a test, easily switching the datasets to be used in a test, or being able to view the internals of container to verify expected behavior.

An additional benefit of using a Testcontainer as a class field is the ability to reference values from the container in use. In Initializer I am using container to populate the JDBC URL (container<span class="pl-k">.</span>getJdbcUrl()), username, and password properties for the Spring test application context. By default when using PostgreSQLContainer the username and password are both “test”, so we don’t really need to pull these values from the container, however the JDBC URL is dynamic. Being able to pull values from a container and pass them in to the application context for a Spring test, helps to increase the flexibility when using Testcontainers. Without this, you might have to use pre-defined ports, IPs, or other values, which might run into trouble when the tests are being executed on a build server.

Conclusions

I’m excited to see how much Testcontainers has grown both as a project and in interest from the community from when I first started using it. I have often struggled when writing integration tests, having to deal with either flickering tests, or the overhead of install and maintain a local resource. Neither are pleasant experiences. Testcontainers brought sanity in my life to the difficult task of writing integration tests.

The code used in this article can be found here.