It’s a new year and with that comes another release of the JUnit 5 framework! In this article we will look at some of the big new features released in JUnit 5.4.
Ordering Test Case Execution
I have been personally looking forward to this feature for sometime now. While unit tests by definition should be isolated from one another, JUnit covers a space larger than “just” unit testing. In my case, I have been wanting to be able to explicitly define test execution order to resolve an issue around an integration test scenario in a project demonstrating JUnit 5.
The goal of the integration test is to validate that the application can communicate with a Postgres database. In the test class, which is making use of TestContainers, three behaviors are being verified, reading, mapping, and writing to a database. For reading from the database, a simple count of the number of records is being used, which would obviously be impacted by writing a new record to the database. While tests in JUnit 5 are executed in a consistent order, it is “intentionally nonobvious” how that order is determined. With JUnit 5.4, we can finally define an explicit test execution order.
Let’s take a look at how to order test cases in a class (full class here):
To enable ordering tests cases in a class, the class must be annotated with the
@TestMethodOrder extension and an ordering type of either
Random must be provided.
Alphanumericorders test execution based on the method name* of the test case.
OrderAnnotationallows for a custom defined execution order using
@Orderlike shown above.
Randomorders test cases pseudo-randomly, the random seed can be defined by setting the property
junit.jupiter.execution.order.random.seedin your build file.
- You can also create your own custom method orderer by implementing the interface
*A test case’s
@DisplayName, if defined, will not be used to determine ordering.
Order Only the Tests that Matter
OrderAnnotation you should note, and this can be seen in the code example above, you don’t have to define an execution order for every test case in a class. In the example above only one test has an explicit execution order,
testCountNumberOfCustomersInDB, as that is the only test case that will be impacted by a change in state. By default JUnit will execute any tests without a defined execution order after all tests that do have a defined execution order. If you have multiple unordered tests, as is the case above, they will be executed in the default deterministic, but “nonobvious” execution order that JUnit 5 typically uses.
This design decision is not only helpful for the obvious reason of requiring less work, but it also helps prevent polluting tests with superfluous information. Adding an execution order to a test that does not need it, it could lead to confusion. If a test begins to fail, a developer or test automation specialist might spend time fiddling with execution order when the cause of the failure is unrelated to execution order. By leaving a test without a defined execution order it is stating this test is not impacted by state change. In short, it should be actively encouraged to omit
@Order on test cases that do not require it.
The new ordering functionality isn’t limited to just ordering the execution of test cases. You can also order how programmatically registered extensions, i.e. extensions registered with
@RegisterExentsion, are executed. This can be useful when a test(s) has complex setup/teardown behavior and that setup/teardown has separate domains. For example testing the behavior of how a cache and database are used.
While extensions by default execute in a consistent order, like test cases, that order is “intentionally nonobvious”. With
@Order an explicit and consistent extension execution order can be defined. In the below example a simple extension is defined which prints out the value passed into its constructor:
Here is the console output from executing the above test class:
Executing beforeAll with value:C Executing beforeAll with value:B Executing beforeAll with value:A Executing beforeEach with value:C Executing beforeEach with value:B Executing beforeEach with value:A Executing afterEach with value:A Executing afterEach with value:B Executing afterEach with value:C Executing beforeEach with value:C Executing beforeEach with value:B Executing beforeEach with value:A Executing afterEach with value:A Executing afterEach with value:B Executing afterEach with value:C Executing afterAll with value:A Executing afterAll with value:B Executing afterAll with value:C
A frequent question/concern I have heard when presenting on JUnit 5 has been the large number of dependencies that are required when using JUnit 5. With the 5.4 release the JUnit team will now start providing the junit-jupiteraggregate artifact. JUnit-Jupiter bundles junit-jupiter-api, junit-jupiter-params, so collectively this artifact should cover most of the needs when using JUnit 5. This change should help slim down the maven and gradle files of projects using JUnit 5, as well as make JUnit 5 easier to use in general. Below shows the “slimming” effect of the new aggregate artifact:
@TempDir began its life originally as part of the JUnit-Pioneer third-party library. With the release of 5.4,
@TempDir has been added as a native feature of the JUnit framework.
@TempDir makes the process of validating some file I/O behavior easier by handling the setup and teardown of a temporary directory within the lifecycle of a test class.
@TempDir can be injected in two ways, as a method argument or as a class field and must be used with either a
@TempDir cannot be injected as a constructor argument. Let’s take a look at
@TempDir in action:
Note: The same directory is shared across a test class even if you inject a
@TempDir in multiple locations.
TestKit was added in 5.4 as a way to perform meta-analysis on a test suite. TestKit can be used to check the number of; executed tests, passed tests, failed tests, skipped tests, as well as a few other behaviors. Let’s take a look at how you can check for tests being skipped when executing a test suite.
To use TestKit you will need to add the
junit-platform-testkit dependency to your build file.
But That’s not All…
Another new feature added with 5.4 is the new Display Name Generator. Lee Turner already wrote a great article on the new display name generator, so rather than re-explaining it this article, check his: https://leeturner.me/blog/2019/02/building-a-camel-case-junit5-displaynamegenerator.html
This is only a highlight of some of the new features in JUnit 5.4, to view all the new features, changes, and bug fixes, checkout the release notes the JUnit team maintains: https://junit.org/junit5/docs/current/release-notes/
Also be sure to check out the JUnit 5 user guides for examples on how to use all the features in JUnit 5: https://junit.org/junit5/docs/current/user-guide/index.html
I have been continually impressed by the JUnit team’s steady work improving the JUnit 5 framework. In a little under a year and a half we have now seen four minor releases. As someone who has come to deeply appreciate and advocate for automated testing over the past couple of years, I am happy to see the JUnit team aggressively adding new features to JUnit 5 and taking in feedback from the community and other testing frameworks like Spock, TestNG, and others.
To view the code used in this article check out my project github page here: https://github.com/wkorando/WelcomeToJunit5