Spring Cloud Contract, Completing the Consumer Driven Contracts Story

A new release of Spring Cloud, Hoxton, is gathering steam. Part of Milestone 2 release is a feature I have been looking forward to for Spring Cloud Contract (SCC), the ability to generate stubs directly off of contracts. In this article we are going to take a look at what this means and how to use this new feature.

Consumer Driven Contracts in a Polyglot World

Last year SCC implemented the functionality that allowed clients to write contracts without having to be familiar with the Java tech stack. This was done by supporting a YAML DSL for defining contracts and providing two Docker images, one that could consume contracts and validates a service is meeting the contracts and the other that can consume stubs so it could act as a mock of the service the contracts were written for. For more information on this, check out this blog article by Marcin Grzejszczak.

A common approach in many Java shops is to have a backend written in Java (i.e. the producer service), but the frontend UI (i.e. the consumer) written in JavaScript. By removing the reliance on Java specific technologies when writing contracts, this made writing contracts more accessible to frontend developers. By allowing the consumers to write the contracts you are able to do a work flow called Consumer Driven Contract Development. The consumers write a contract, telling a producer what they need in a format that can be validated programmatically, and then a producer developer can write, or update, the producer service to meet the requirements of the contract.

Consumer Driven Contract Development is a desirable development paradigm because it is a “pull” rather than “push” concept. Instead of producers “guessing” what a consumer might need, you allow the consumers to define the behavior they need from the producer. This helps to producers avoid building APIs consumers don’t need, or building APIs that are difficult for consumers to, well, consume.

We Keep Waiting on the Service to Change

While SCC’s changes allowed consumers to write contracts with technologies they were more familiar with, the consumer driven contract story was still a bit hollow. SCC had a Docker images that could act as a mock of the service, however it relied upon the availability of stubs. These stubs were only generated after the contracts were validated against the producer service confirming the producer service fulfilled the contracts.

A developer building a consumer could write a contract, but until the producer service actually implemented the behavior defined in the contract, the consumer wouldn’t be able to use the mock service. This could stall development on the consumer side for days or even longer depending on the workload and priorities of the developers of the producer service. If we are going to have consumer driven contracts, we need to allow consumers to be able to develop independently of producers. The contracts are still there, and since both parties are developing based on the contract, they should still arrive at the same point.

Running a Mock Service from Contracts

With the M2 Spring Hoxton release, SCC added the functionality to run a mock service directly from contracts. I have been presenting this year about Spring Cloud Contract, and how it can be used to do Collaborative Contract Driven Development, here is my presentation from Spring I/O in Barcelona back in May. I’m happy that now with the ability to run the mock service directly from contracts, this better completes the story in my presentation.

I have created a GitHub project for demonstrating how to use Spring Cloud Contract. The project has a repo for the producer, client (consumer), and a repo for the contracts. If you clone the client repo, you can see SCC’s new functionality in action by executing the run-produce-service-mock-no-stubs.sh script. The script contains the following:

docker run --rm \
-e "STUBRUNNER_IDS=com.ibm.developer:produce-service:+:stubs:9876" \
-e "STUBRUNNER_REPOSITORY_ROOT=git://https://github.com/wkorando/produce-contracts.git" \
-e "STUBRUNNER_STUBS_MODE=REMOTE" \
-e "STUBRUNNER_GENERATE_STUBS=TRUE" \
-p "8083:8083" \
-p "9876:9876" \
springcloud/spring-cloud-contract-stub-runner:2.2.0.BUILD-SNAPSHOT

This script is running a docker image and passing in some environment variables. Let’s take a look at some of the key values being passed in:

  • ​​​​​STUBRUNNER_REPOSITORY_ROOT This is telling the Docker image where the contracts reside
  • STUBRUNNER_STUBS_MODE This is telling the docker image the contracts are in a remote location (in this case a GitHub repo)
  • STUBRUNNER_GENERATE_STUBS This is variable that was added that tells the Docker image to build stubs directly off the contracts we sent
  • -p "9876:9876" This is telling the Docker image to have the mock service running on port 9876

Executing the script will have a mock service running on http://localhost:9876. If you look at the documentation that we generate from the contracts, you can see some of what the mock service is capable of doing. If you call the url http://localhost:9876/api/v1/produce you will get back a JSON list of produce items. You can also see how this works when working with a real UI by cloning this repository and running <span class="s1">npm run start:contract.

Conclusion

The GA release of Spring Cloud Hoxton should be available later this fall. The ability to generate stubs straight from contracts is a significant step forward in completing the Consumer Driven story. It’s a feature I have been looking forward to since I first started with with Spring Cloud Contract a few years back. If you are attending Spring One Platform next week and are interested in learning more about contract driven development, be sure to swing by my presentation on Thursday Collaborative Contract Driven Development.

What’s New in JUnit 5.5

Screen Shot 2019-06-07 at 2.50.27 PM

A new minor version of JUnit 5, 5.5, was released on June 30th. That means it’s time for yet another article looking at what’s new in this release of JUnit 5! The JUnit team keeps up their blistering pace of a new minor release roughly every four months, and while this release might not quite have a “killer” new feature like previous releases have had, the aggregate of all the tweaks and enhancements have actually made this a pretty big release, far larger than I will be able to cover here. Let’s dive in and take a look at a some of the key new features and changes introduced in JUnit 5.5.

Declarative Timeouts

Ideally automated tests should execute quickly, however some tests; integration, feature, end-to-end tests, etc., might require complicated setup and/or interact with remote resources. This behavior might occasionally lead to situations where a test can run excessively long. To address this concern @Timeout was introduced to provide declarative timeout support for test cases and lifecycle methods. Here is a code example showing some of the ways @Timeout can be used. I’ll go into more detail on its usage below:

@Timeout isn’t limited to being placed on test cases themselves. Along with the aforementioned test cases, @Timeout can also be placed at the type level, where it will provide a default timeout for all test cases declared in the class, this can be overridden by adding a @Timeout to a test case. @Timeout can also be added to lifecycle methods, @BeforeAll, @BeforeEach​, @AfterEach, @AfterAll​.

@Timeout also provides flexibility in setting the unit for timeout length. By default @Timeout uses seconds for its unit of measurement, however @Timeout also has the unit field which can take a value of TimeUnit. Hopefully you’ll never need to declare a TimeUnit value larger than seconds, but having the flexibility to use milliseconds, microseconds, or even nanoseconds, can be helpful when testing race conditions and following the principle of “fail fast” even in the automated testing world (even a unit as small as a second can start to add up in a large enough test suite).

Timeouts can also be declared as system properties. Like with @Timeout the default unit is seconds, but can be changed by adding the following labels:

ns|μs|ms|s|m|h|d

More specific properties override less specific ones (i.e. setting timeout.beforeall, would override the value of timeout.lifecycle). This provides a way of setting sensible defaults for how long tests should run for that can be easily overridden if needed. Below is the full list of timeout system properties:

junit.jupiter.execution.timeout.default
junit.jupiter.execution.timeout.testable.method.default
junit.jupiter.execution.timeout.test.method.default
junit.jupiter.execution.timeout.testtemplate.method.default
junit.jupiter.execution.timeout.testfactory.method.default
junit.jupiter.execution.timeout.lifecycle.method.default
junit.jupiter.execution.timeout.beforeall.method.default
junit.jupiter.execution.timeout.beforeeach.method.default
junit.jupiter.execution.timeout.aftereach.method.default
junit.jupiter.execution.timeout.afterall.method.default

@RegisterExtension Updates

@RegisterExtension was added in JUnit 5.1 to support registering extensions programmatically. Since 5.1 @RegisterExtension has mostly remained unchanged, save for the addition of being able to declare their execution order which was added in 5.4. With JUnit 5.5 @RegisterExtension has a number of small but helpful changes made to it, let’s take a look at them.

@RegisterExtension Gets Loud 📣

There are several constraints around how @RegisterExtension can be used. Two of them are the field annotated with @RegisterExtension must not be private and the assigned value must implement an Extension interface. With JUnit 5.5 violating these constraints will cause an exception to be thrown, whereas previously fields that violated these constraints would be silently ignored. The below gif demonstrates the change in behavior.

@RegisterExtension Gets Secretive

In previous releases of JUnit, the declared field of @RegisterExtension must also implement an Extension interface. The JUnit team relaxed this constraint that only the assigned value of the field must implement Extension. This is helpful for Extension designers to hide fields that users of an extension should not be changing and, generally make an API that is cleaner and easier to use.

TestExecutionListener Fails More Quietly

While JUnit has made misusage of @RegisterExtension “noisier”, with JUnit 5.5 exceptions thrown by TestExecutionListeners have gotten quieter. In JUnit 5.4 and before when an exception was thrown by a TestExecutionListener this caused the entire test run to terminate. This behavior isn’t really desirable. Along with listeners existing within the world of reporting, so not necessarily “critical”, even a test throwing an exception would terminate a test run.

With JUnit 5.5 when a TestExecutionListener throws an exception, the stacktrace is instead printed to the console at the WARN level. This allows the test suite execution to continue, but for the information about an error occurring not being lost, and also being clearly indicated.

MethodOrder Random Seed Print Out

In JUnit 5.4 method ordering was added. One of the default implementation of method ordering is random method ordering. Random method ordering could be useful for checking that tests don’t have any unnecessary relationships between them. By running tests in a random order, this can validate that testCaseD doesn’t depend upon/is impacted by behavior occurring in testCaseC.

Note: This is generally only referring to unit tests. It can be appropriate for other types of automated tests to have dependencies upon one another.

If a seed isn’t supplied via the junit.jupiter.execution.order.random.seed property, JUnit will generate a seed for you. With JUnit 5.5, when this happens JUnit will also print log statement displaying the seed value used. This allows a test run to be recreated if a problem was discovered. The below gif demonstrates the behavior:

Execute Classes in Parallel

The JUnit team has made further improvements to their support of parallel test execution. In JUnit 5.5 the configuration parameter: ​junit.jupiter.execution.parallel.mode.classes.default has been added, that allows support for defining how parallel test execution should interact at the class level. Like with ​​junit.jupiter.execution.parallel.mode.default, there are two default values accepted SAME_THREAD and CONCURRENT.  Additionally junit.jupiter.execution.parallel.mode.classes.default will default to the value that junit.jupiter.execution.parallel.mode.default is set to, which is SAME_THREAD by default.

The goal of this change was to make it easier to configure parallel test execution in your test suite. This behavior previously could had been implemented by manually adding @Execution(MODE) to the declaration of every class. The problem with that though is in the description, it required manual work. The addition junit.jupiter.execution.parallel.mode.classes.default allows a default behavior that can be applied to an entire test suite from a single location that can be overridden if needed. Below is a gif showing how a test suite would execute using the different settings:

Default Test Discovery Implementations

A core characteristic of JUnit 5 is its extensibility. Between the needs of library developers and the needs of individual organizations, the JUnit framework cannot reasonably natively support every use case. To address this concern, the JUnit team has worked to make JUnit 5 a foundation that is exceptionally easy to build off of. This characteristic continues with the addition of default implementations for test discovery for both Jupiter and Vintage JUnit tests.

The default test discovery implementations will make it easier to write custom TestEngine implementations. Generally this isn’t something most developers will need to do, but if you have an unusual use case or a need not quite being met by existing TestEngines, then have default discovery implementations will make writing your own TestEngine a little easier.

I would recommend checking out the associated GitHub issues behind this change for more in-depth information: 1739 and 1798.

Deprecation of EnableIf/DisabledIf

Introduced with the release of JUnit 5 was the ability to conditionally disable tests. This feature has gradually been enhanced over time with the JUnit team providing several sensible defaults like disabling (or enabling) a test based upon OS, Java version, and by system property.

However with JUnit 5.5 the JUnit team will be deprecating @EnabledIf and @DisabledIf for eventual removal (currently slated to be removed in 5.6). @EnabledIf/@DisabliedIf provided a mechanism for writing out a script (JavaScript, Groovy, et al.) to evaluate if a test should be executed. Ultimately though this method provided little benefit over writing an implementation of DisabledCondition, while having the drawbacks of creating a maintenance headache as the script would have to be copy and pasted if it was to be reused and also being slower to execute.

Even setting aside the above concerns, the expiration date of using script based conditions was nigh. Nashhorn, Java’s JavaScript engine, was deprecated with Java 11 deprecated and schedule for removal, and script based conditions don’t play nicely when using the module path. If you are using @EnabledIfor @DisabliedIf in your projects, it would be a good idea to start migrating away from them now.

Conclusion

Because there were so many changes in this release, I was only able to cover a small portion of them (smaller than even normal) to see all the changes included in this release be sure to check out the release notes. And always be sure to check out the JUnit user guides for more in-depth information on how to write automated tests with JUnit 5.

To view the code used in this article, check out my github repo on JUnit5.

Keeping Dependencies Up-to-Date with Automated Testing

One of my big passions in software development is automated testing, which can be seen by how often I write on the subject. This passion stems from having spent years working with organizations that struggled delivering to production in a timely manner and the anxiety I would experience during deploys. Too often when a release was pushed to production I would soon hear back that a bug has made it to production and the commit logs would show “bkorando” for when the bug was introduced. I would feel ashamed that a change I made caused a problem in production, and felt that it was because I wasn’t being careful enough.

Of course I wasn’t the only developer accidentally introducing bugs into production and the organizations I worked with, and many others, would respond to this problem by introducing new processes. Processes like; having manager sign off on a release, a tester manually run through a test script, or requiring extensive justification for why a change should be made. These additional processes would do little to resolve the issue of bugs getting to production, they did however result in two things:

  1. Production pushes became more painful, which lead to them happening less often
  2. Encouraged developers to limit the scope of their changes to reduce risk

In this article we are going to look at how relying on manual processes hinder an organization’s ability to keep their dependencies up-to-date, why this is a problem, and look at a use case of how an automated test can catch a subtle breaking change that occurred from upgrading a dependency.

Coding On the Shoulders of Giants

As modern software developers, we owe a lot to developers that came before us. Margret Hamilton pioneered error handling and recovery techniques while developing the software for the Apollo flight control computer. If you are performing distributed transactions, Hector Garcia-Molina and Kenneth Salem paved the way with the development of the SAGA pattern on how to rollback disturbed transactions when errors inevitably occur.

Our foremothers and forefathers, have allowed our field to advance by pushing on the boundaries of what was possible and introducing new concepts and patterns. Often though the way we benefit from these pioneering advances is through the use of libraries and frameworks that implement or provide abstractions into those concepts and patterns. While we labor all day building and maintaining applications for our organizations, the reality is the code we write is only a small fraction of the code actually running in production.

Graph depicting the we write resting at the peak of a pyramid of code (dependencies) it runs on

Just like how the applications we maintain are constantly changing, so too are the libraries and frameworks we depend upon. New features are being added, bugs fixed, performance improved, security holes plugged. If you want the hair to stand on the back of your neck, I would recommend checking out shodan.io, which can be a great resource for showing not only the vulnerabilities of a website, but how to exploit them!

Frozen in Time

Image result for mosquito in amber
Photo by George Poinar, Jr.

In the introduction, I talked about how additional process would encourage developers to minimize their changes. An easy way of keeping a change set small is by not updating an application’s dependencies. I have quite frequently worked on applications whose dependencies were years old! At times working on these applications made me feel like I was a paleontologist, examining applications frozen in amber from a time long ago.

From the perspective of an organization that depends upon manual process, the decision to freeze dependencies is understandable. Updating a dependency, especially if it’s a framework dependency like Spring Boot, could impact an entire application. If an entire application might be impacted from a change, then it would need a full regression test, and when that is manual, that is a time consuming and expensive effort. However in the attempt to resolve one issue, prevent bugs from being introduced into production, this created another, deploying applications with significantly out-of-date dependencies which might contain critical security vulnerabilities (CVEs).

Focusing on the Right Issues

Image result for asking the right question iRobot gif

Manual regression testing, along with being very time consuming, isn’t a very good way to verify the correctness of an application. Issues with manual regression testing include:

  1. Difficult to impossible to recreate some scenarios (e.g. system failure cases)
  2. Test cases not being executed correctly leading to false positives or false negatives
  3. Test cases not being executed at all

Automated testing can address these issues as automated tests are: executed much more rapidly, have very little manpower requirement to execute, are much more granular and flexible, and auditable to ensure they are being executed and testing the intended behavior.

While it is nice to sing the praises of automated testing in the abstract, it would be good to see an example of how they can catch a non-trivial, non-obvious bug that occurred from updating a dependency. As luck would have it I just so happen to have an example.

Catching Upgrade

When I was working on a recent article, wiring multiple datasources, I ran into a bit of a frustrating issue. To distill what happened to the essential points. While checking my work with existing articles out there on the same subject, I tried implementing a version of a solution another author demonstrated. However this solution didn’t work for me, even though the code looked the same. The gif below demonstrates the issue I ran into, as well as how to write a test that would catch the issue:

Code can be found on my GitHub. Note the code from the gif above are in branches off master.

A Perfect and Imperfect Use Case

Most any use case is going to run into relevancy. Problems with this use case, in showing the value of automated testing when upgrading dependencies, is that this problem would show up at start up time. Upgrading to Spring Boot 2.x and not updating the properties to use *.jdbc-url would cause the application to fail at start up as it is not able to properly configure an EntityManager. So the effect of this change would be fairly obvious.

On the other hand, this is a perfect use case for demonstrating the usefulness of automated testing for several reasons. Let’s quickly step through some of them:

  1. The test is easy to write – The test case as can be seen in the above gif is pretty simple to write. Recreating the failure scenario is easy and can be done with an in-memory database, so the test is highly portable.
  2. The test covers other concerns – The test isn’t strictly about recreating a narrow error case, i.e. you’d have to know about the problem before writing the test case. Database communication is important, and is something that would require a test regardless. If there were other subtle breaking changes in that area as well resulting from a dependency upgrade, they might be caught with a test case like this one as well.
  3. An automated test suite can find multiple issues simultaneously – This issue was found at startup. In a more substantial application it is likely there would be other issues that come from doing a major version upgrade of a framework dependency like Spring Boot. Even if all issues were at startup, which wouldn’t always be the case, it would be a grueling process to sequentially go through fixing each issue as it comes up having to restart the application every time. A test suite could catch multiple issues in a single execution, allowing them to be fixed simultaneously.

Resolving issues that come up when upgrading underlying dependencies, even with a good automated test suite can be a difficult process. It took me awhile to track down the solution to my issue, which would be the case regardless of how the issue was discovered.

Safe to Fail

To begin summarizing this article, I was listening to a recent episode of A Bootiful Podcast and this quote really stuck out to me (lightly edited):

You become safe to fail, you accept that no matter what you do there will be bugs… you’re just able to find them quickly and push the fix quickly

@olga_maciaszek

An important takeaway is that no automated test suite is going to entirely eliminate the risk of a bug making it to production. What an automated test suite does gives you though is a safety net and ability to rapidly respond to bugs when they do occur. It gives you confidence that generally it will find bugs in your code, and when a bug inevitably slips through you can respond to it rapidly and with confidence you will not be introducing yet another bug into production. This protection applies to changes made within your code, as well as changes made to dependencies your code depends upon as well.

Conclusion

Awareness and importance of automation, and particularly automated testing, has been slowly increasing in the software development industry over recent years. I am pleasantly surprised when I ask during presentations how many people are working at organizations that do continuous delivery, usually between 25-50% of the room raise their hands.

Still that leaves half or more of organizations which haven’t yet turned the corner on automation. I often find the best way to encourage change at organizations is to lay out the cost of staying with the status quo, and how the proposed changes resolve these problems. If you’re at an organization that still relies heavily on manual processes, and been wanting to change this, hopefully this article might help build your case for your organization to start embracing automation and automated testing.

An Extended Discussion on Customization in JUnit 5

Inspiration often comes in twos for me. While reviewing a recent blog article, What’s New in JUnit 5.4, a colleague suggested I go into more depth on the usage of extensions in JUnit 5. Then in my twitter timeline I saw this from one of the core committers of the JUnit 5 framework:Screen Shot 2019-03-27 at 5.34.07 PM

Source: Twitter.

Later on in the thread, what was trying to be done with the “hack” could had been accomplished by creating a custom extension that’s available to the public.

The above tells me two things; there is a need for a deep dive on the JUnit 5 extension model and a need to explain the extendability aspect of the JUnit 5 framework. When I talk extendability, I’m specifically referring to the quality of being able to build on top of the existing framework that the JUnit team has provided in JUnit 5. Whereas hacks have often been the modus operandi for getting around the limits of frameworks (rather those limits were intentional or not!), the JUnit team went to great strides to make JUnit 5 extendable, and we’ll see in this series how to take advantage of that quality.

JUnit 5’s extension model and extensibility are by no means trivial subjects, so to make it more digestible, this will be a three part blog series. The subject of each blog article will look like this:

  1. Introduction to and using the JUnit 5 extension model
  2. The JUnit 5 extension lifecycle and building a custom extension
  3. Understanding and using extensibility in JUnit 5

In this article, we will take a high level overview of the extensions model from the perspective of a user of extensions; well learn why the extension model was introduced, how this improves upon what was in JUnit 4, the different ways to register an extension, and how to define the order of extension execution. 

The JUnit 5 Extension Model

JUnit 5 was a fundamental re-write and re-design of the JUnit framework. Some areas largely remained the same, though with a few enhancements, like assertions. Other areas were completely overhauled, which includes runner (@RunWith), MethodRule (@Rule), and TestRule (@ClassRule), being rolled into the new extension model.

The benefits of this overhaul can be experienced in a number of ways. A pretty obvious one is you can now declare multiple extensions at the class level whereas before you could only declare a single @RunWith:

@ExtendWith(SpringExtension.class)
@ExtendWith(MockitoExtension.class)
public class TestSomeStuff{...
}

A bit more subtle, parameterized tests and normal tests can now co-exist in the same class:

public class ParameterizedAndNormalTestsLivingTogether{

   @Test
   pubic void aNormalTest(){
      ...
   }

   @ParameterizedTest
   @ValueSource(strings = { "val1", "val2" })
   public void aParameterizedTest(String val) {
   ...
   }
}

Note: @ParameterizedTest is built using the extension model

If you haven’t run into the constraints imposed by the previous Runner and Rule architecture, I can assure you it’s quite the painful experience when you do! So being able to register multiple extensions in the same test class or locate a parameterized test and normal test in the same test class are reasons to celebrate. But this is only just scratching the surface of the extension model, so let’s start going deeper.

Registering Extensions

There are three different ways to register an extension in JUnit 5: declaratively, programmatically, and automatically. Each way of registering an extension comes with specific rules, constraints, and benefits. Let’s step through the different types of ways to register extensions and understand when and why you might prefer using one method of the other.

Declaratively Registering Extensions

Extensions can be registered declaratively with an annotation at the class, method, or test interface level, and even with a composed annotation (which will be covered in-depth in the article on extendability). The code samples above are examples of registering extensions declaratively.

Declarative registering of extensions is probably the easiest way of registering an extension, which can be made even easier with a composed annotation. For example it is easier to remember how to write @ParameterizedTest when you want to declare a parameterized test than @ExtendWith(ParameterizedTestExtension.class).

As you are using an annotation to register an extension all the constraints with using annotations are there, such as only being able to pass static values to the extension. Also a test class cannot easily reference extensions that have been registered declaratively.

Programmatically Registering Extensions

Extensions can be registered programmatically with @RegisterExtension. There are a few rules regarding programmatically registered extensions. First, an extension field cannot be private. Second, the extension field cannot be null at time of evaluation. Finally an extension can either a static or instance field. A static extension has access to the BeforeAll, AfterAll, and TestIntancePostProcessor steps of the extension life cycle.

Registering a programmatic extension would look like this:

@RegisterExtension
SomeExtension extension = new SomeExtension();

Test classes have a much greater degree of freedom when interacting with a programmatically registered extension as they are just another field within the test class. This can be great for retrieving values out of an extension to verify expected behavior, passing values into the extension to manipulate its state at runtime, as well as other uses.

Automatically Registering Extensions

The final way to register an extension is with the Java Service Loader. The Java Service Loader can best be described as arcane, at least I generally get blank stares or looks of confusion when I bring it up. Though like many arcane things, it can be very powerful for both good and ill!

The Java Service Loader can be used to automatically register extensions within a test suite. This can be helpful as it allows certain behaviors to happen automatically when executing tests. The flip side to this, depending on the type of work that is occurring within the extension, this could have a non-trivial impact on your test suite runtime, it could also interfere in a non-obvious way with how a test behaves (the person executing the test might not realize the extension is being executed because it wasn’t registered locally). So to quote Uncle Ben:

remember-with-great-power-comes-great-responsibility

Registering an Automatic Extension

Registering an automatic extension is a more involved process than the other two ways, let’s quickly walk through the steps:

  1. Create a folder named META-INF on the base of your classpath
  2. Create a folder named services under META-INF
  3. Create a file named org.junit.jupiter.api.extension.Extension under services
  4. In org.junit.jupiter.api.extension.Extension add the fully qualified name of the extension you want registered, for example: my.really.cool.Extension
  5. Pass in -Djunit.jupiter.extensions.autodetection.enabled=true as a JVM argument (how to do this will vary based on your IDE)
    1. Configure your build file to automatically pass in the above argument. Here is an example using Surefire in maven:
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <configuration>
        <properties>
            <configurationParameters>
               junit.jupiter.extensions.autodetection.enabled=true
            </configurationParameters>
        </properties>
    </configuration>
</plugin>

You can see a full example of the above here. Note META-INF is located under /src/test/resources.

Doing these steps every time you would want to use an automatic extension in a project is a bit involved, in the article on extendability we’ll take a look at how to make automatic extensions more practical to work with.

Ordering Extension Execution

As of JUnit 5.4 there are two ways to order how extensions are instantiated during the test cycle. Ordering extension execution could be useful in the realm of higher level tests; that is test above the unit test level. Integration tests, functional tests, feature tests, and other such similar tests might require complex setup and tear down steps.

Even for test code, it is still important to follow principles like single responsibility. If for example you have a feature test that verifies the behavior for when your application interacts with a database and cache, it would be better to locate the logic for setting up and tearing down the database in one extension and similar behavior for the cache in a separate extension, instead of putting all that behavior in a single extension. This allows for great reusability of each extension as well as making them easier to comprehend.

For all ways of ordering extension execution, the order of execution is inverted for “after steps”. So if you have three extensions named A, B, and C, each implementing the BeforeEach and AfterEach behavior, then going into a test method the execution order would be A -> B -> C, while the execution order leaving the test method would be C -> B -> A.

Order of Declaration

When registering an extension declaratively, the order of declaration within the test class is the order in which the extensions are registered and executed. Take the below example:

@ExtendWith(FirstExtension.class)
@ExtendWith(SecondExtension.class)
public class TestExtensionExecutionOrdering(){

   @ExtendWith(ThirdExtension.class)
   public void testExtensions(){
   }
}

When executing the test method testExtensions() the execution order going in would be FirstExtension -> SecondExtension -> ThirdExtension and going out of testExtensions() it would be ThirdExtension -> SecondExtension -> FirstExtension.

I haven’t personally used this feature a whole lot. I have a lot of confidence that, from a framework perspective, this feature behaves as designed. What I worry about however is the extent this feature would be understood by most developers and test engineers. In my experience, the order in which annotations are declared in a class or on a method is not something that developers and test engineers often think about or interact with. If this concern is ever surfaced, it’s often for stylistic reasons, for example; the shortest annotation should be declared first.

The good news is, is through the enhancements to extendibility that I mentioned in the introduction to this article, a custom annotation could be created and shared that includes the declaration of multiple extensions in their proper order. We will take a deeper look at custom annotations, and other examples of extensibility later in this series.

Ordering Programmatically Registered Extensions

Ordering extension registration and execution by order of declaration has been a feature of JUnit 5 since its initial release. With JUnit 5.4 programmatically registered extensions can also be executed in a manually defined order (programmatically registered extensions have always been executed in a consistent order, but it is “intentionally non-obvious”).

To define the execution order of a programmatically registered extension the annotation @Order(n) needs to be added to the declaration of the extension field. You do not need to add an annotation at class level like you would for ordering test methods to enable this behavior. However like when ordering test method execution, you do not need to order every extension. Extensions that do not have a defined execution order are executed after all extensions that do, following the “consistent, but intentionally non-obvious” process mentioned above. So in the below example:

public class TestClass{
   @RegisterExtension
   @Order(1)
   BaseExtension baseExtension = new BaseExtension();

   @RegisterExtension
   @Order(2)
   SecondaryExtension secondaryExtension = new SecondaryExtension();

   @RegisterExtension
   AuxillaryExtension secondaryExtension = new AuxillaryExtension();

   public void testExtensions(){
   }
}

BaseExtension is executed first, SecondaryExtension second, and AuxillaryExtension, and any other extension, executed after.

Also note that programmatically registered extensions will be executed after all extensions that have been registered declaratively and automatically. So aa programmatically registered extension with an annotated with @Order(1) may not be the first extension to be executed when running the test. So keep that in mind!

Conclusion

The new extensions model added a lot of much needed (and appreciated!) flexibility when it replaced the runner and rules architecture from JUnit 4. In the next article in the series we will take an in-depth look at the lifecycle of an extension and build our own custom extension!

The code used in this article, and series, can be found here.

Testcontainers, Bringing Sanity to Integration Testing

Writing and maintaining integration tests can be a difficult and frustrating experience, filled with a veritable minefield of things that could go wrong. For integration tests that connect to a remote resource you have issues of: the resource being down, datasets being changed or deleted, or heavy load causing tests to run slow. For integration tests that connect to a local resource you have the initial install and configuration of the resource on your local machine and the overhead of keeping your local instance in-sync with what the production instance looks like, otherwise you might run into this situation:Screen Shot 2019-03-25 at 8.12.34 AMSource: Minesweeper – The Movie

No application operates in isolation. Applications, even “monoliths”, depend on remote resources be it: databases, logging services, caches, or other applications to function. Just like the application we are maintaining will change over time as business needs and client demands change, so too will the resources it depends on. This necessitates a need to continually verify that our application can communicate with its dependent resources.

So to maintain development velocity and while having confidence our applications will function properly in production we need to write automated integration tests, however we need our integration tests to be:

  • Reliable – Test failures should only happen because a change occurred in either our application or the resource, not because the resource is down or misconfigured.
  • Portable – The tests should be able to run anywhere with minimal setup.
  • Accurate – The resource being used in the integration test should be an accurate representation of what exists in production.

How do we accomplish these requirements?

Introducing Testcontainers

Testcontainers is a Java library that integrates with JUnit to provide support for starting up and tearing down a Docker container within the lifecycle of a test or test class. Testcontainers is a project that was started about four years ago, and I first learned about back in 2017 when I was putting together a Pluralsight video on automated testing.

I have noticed an uptick in interest in Testcontainers in my twitter outline recently, and it doesn’t seem long ago that Testcontainers passed the 1K stars mark on their github repo, which now sits at 2.2K. If you haven’t started familiarizing yourself with Testcontainers now would definitely be a good time.

This rapid increase in popularity is likely the result of Testconainers being easy to use, and the flexibility of Docker containers, allowing Testcontainers to address a lot of integration testing use cases. In this article we are going to look at two approaches of how to use Testcontainers for running an integration test against a database. The code examples will be using JUnit 5, if you want to get familiar with JUnit 5, I have written a lot about it, you should also check out the JUnit 5 user docs.

Launching a Testcontainer via JDBC URL

In the example we will be writing an integration test for connecting to a Postgresql database, Testcontainers does offer support for a number of other databases. The first step will be brining in the appropriate dependencies. For this example we will only need to add the Postgresql Testcontainers dependency, to our maven build file (which in turns brings in the Testcontainers JDBC and core libraries).

Full maven build file for this project can be found here.

With the appropriate dependencies imported, let’s look at how to use Testcontainers to write a database integration test.

Full class, including imports, here.

There is quite a bit going on, let’s breakdown what is happening in this class into more easily digestible bites.

@TestPropertySource("classpath:application.properties")

This isn’t really related to using Testcontainers, but since ApplicationContextInitializer (javadoc) isn’t super well known, but can also be really helpful when writing automated tests, I wanted to take a moment to show how to make it easier to work with when used in test classes.

Here I am telling the test class to bring in the properties defined in /src/test/main/application.properties (source). By bringing in the properties defined in application.properties, instead of having to define every property needed for connecting to the Testcontainers database, only the properties that are different for the tests in this class need to be overwritten. This reduces maintenance needs and helps with overall test accuracy as it is easier to keep a single properties file in-sync with what production looks like.

public static class Initializer implements ApplicationContextInitializer {
   @Override
   public void initialize(ConfigurableApplicationContext applicationContext) {
      TestPropertyValues.of("spring.datasource.url=jdbc:tc:postgresql:11.2://arbitrary/arbitrary", //
      "spring.datasource.username=arbitrary", //
      "spring.datasource.password=arbitrary", //
      "spring.datasource.driver-class-name=org.testcontainers.jdbc.ContainerDatabaseDriver")//
      .applyTo(applicationContext);
   }
}

Within Initializer four properties are being defined (overwritten), and a few of them have somewhat odd looking values, let’s take a closer look. When initializing  Testcontainers via the JDBC URL, Testcontainers will set the username, password, hostname, and database name to what ever values you pass it. Strictly speaking spring.datasource.username and password don’t need to be included as they are defined in application.properties.  For spring.datasource.url, the JDBC URL must start with jdbc:tc:. The 11.2 refers to the specific image tag of postgres to be used, this however is optional and would default to 9.6.8 if left out. Lastly, spring.datasource.driver must be set to org.testcontainers.jdbc.ContainerDatabaseDriver. ContainerDatabaseDriver is Testcontainers’ “hook” into this test class. After starting up the container, ContainerDatabaseDriver will be substituted with the standard database driver, in this case org.postgresql.Driver. While in this example I am using the base postgres image in this example, you can use a custom image, so long as the database within the container is postgres (or of the type of database you have brought in a dependency for).

The rest of the test class is comparatively simple and straightforward. Simple read and writes are being performed to ensure fields are being properly mapped and the generated id matches the expected pattern.

Using Testcontainers as a Class Field

Above we looked at how to use Testcontainers via the JDBC URL hook. This can be a great when your use case is pretty simple, however the complexities of applications in the real world often mean a need for greater control and customization in behavior.

First step would be to bring in the Testcontainers junit-jupiter library.

There are a lot of similarities with the previous code example, so lets focuses only on the differences.

At the top of the test class, is the @TestContainers annotation. This brings in the Testcontainers extension into the class which scans for fields annotated with @Container such as in this case PostgreSQLContainer container. A @Container field can be either static or an instance field. Static containers are started only once and are shared between test methods, instances containers are started and stopped for each test method.

@Container
private static PostgreSQLContainer container = new PostgreSQLContainer("storm_tracker_db:latest");

Here the container that will be used in this test class is defined. Like with the JDBC URL method, you are not required to use a base postgresql image, in this case the customer image “storm_tracker_db” is being used (the Dockerfile for this image is here). As long as the database within the container is postgres, you are fine. While not much additional customization is being done to the container in this class. Testcontainers does offer a number of options such as: executing commands, setting a volume mapping, or accessing container logs, among others. Be sure to check the documentation under features and modules for what is available, as well as the javadoc (v1.11.1).

These additional features provided when using a Testcontainer as a class field allow for flexibility in putting the container within a specific state for a test, easily switching the datasets to be used in a test, or being able to view the internals of container to verify expected behavior.

An additional benefit of using a Testcontainer as a class field is the ability to reference values from the container in use. In Initializer I am using container to populate the JDBC URL (container<span class="pl-k">.</span>getJdbcUrl()), username, and password properties for the Spring test application context. By default when using PostgreSQLContainer the username and password are both “test”, so we don’t really need to pull these values from the container, however the JDBC URL is dynamic. Being able to pull values from a container and pass them in to the application context for a Spring test, helps to increase the flexibility when using Testcontainers. Without this, you might have to use pre-defined ports, IPs, or other values, which might run into trouble when the tests are being executed on a build server.

Conclusions

I’m excited to see how much Testcontainers has grown both as a project and in interest from the community from when I first started using it. I have often struggled when writing integration tests, having to deal with either flickering tests, or the overhead of install and maintain a local resource. Neither are pleasant experiences. Testcontainers brought sanity in my life to the difficult task of writing integration tests.

The code used in this article can be found here.