The JUnit team continues to make steady work refining and improving on JUnit 5. In this article we will take a look at some of the new features and changes that are in JUnit 5.3 which was released on September 3rd. Also in this article we will look at some recent changes and improvements to libraries and tools that are often used in conjunction with JUnit.
Parallel Test Execution
I covered parallel test execution in a previous blog article. If you want an technical and in-depth look at the parallel execution features in JUnit 5.3, give that article a read. Here is a high level overview of parallel test execution in JUnit 5.3:
- Parallel test execution is enabled and configured through the build file (i.e. pom.xml, build.gradle)
- Parallel test execution behavior can be modified at the method and class level with the annotations:
@Executionand@ResourceLock.
Be sure to check out the user guide for more detailed information on configuring and using parallel test execution in JUnit 5.
AssertThrows Enhancements
Note: This feature has been reverted in 5.3.1
T̶h̶e̶ ̶a̶s̶s̶e̶r̶t̶T̶h̶r̶o̶w̶s̶ ̶a̶s̶s̶e̶r̶t̶i̶o̶n̶ ̶h̶a̶s̶ ̶b̶e̶e̶n̶ ̶u̶p̶d̶a̶t̶e̶d̶ ̶t̶o̶ ̶d̶i̶s̶p̶l̶a̶y̶ ̶t̶h̶e̶ ̶v̶a̶l̶u̶e̶ ̶r̶e̶t̶u̶r̶n̶e̶d̶ ̶f̶r̶o̶m̶ ̶a̶ ̶m̶e̶t̶h̶o̶d̶,̶ ̶i̶f̶ ̶a̶n̶y̶,̶ ̶w̶h̶e̶n̶ ̶a̶n̶ ̶e̶x̶c̶e̶p̶t̶i̶o̶n̶ ̶i̶s̶ ̶n̶o̶t̶ ̶t̶h̶r̶o̶w̶n̶.̶ ̶T̶h̶i̶s̶ ̶c̶a̶n̶ ̶b̶e̶ ̶h̶e̶l̶p̶f̶u̶l̶ ̶w̶h̶e̶n̶ ̶d̶e̶b̶u̶g̶g̶i̶n̶g̶ ̶a̶ ̶t̶e̶s̶t̶ ̶c̶a̶s̶e̶ ̶t̶h̶a̶t̶ ̶h̶a̶s̶ ̶s̶t̶a̶r̶t̶e̶d̶ ̶f̶a̶i̶l̶i̶n̶g̶.̶ ̶H̶e̶r̶e̶ ̶i̶s̶ ̶a̶ ̶s̶c̶r̶e̶e̶n̶s̶h̶o̶t̶ ̶o̶f̶ ̶w̶h̶a̶t̶ ̶t̶h̶i̶s̶ ̶d̶i̶s̶p̶l̶a̶y̶ ̶l̶o̶o̶k̶s̶ ̶l̶i̶k̶e̶ ̶i̶n̶ ̶E̶c̶l̶i̶p̶s̶e̶.̶ ̶I̶n̶ ̶t̶h̶i̶s̶ ̶e̶x̶a̶m̶p̶l̶e̶ ̶a̶n̶ ̶e̶m̶p̶t̶y̶ ̶L̶i̶s̶t̶ ̶i̶s̶ ̶b̶e̶i̶n̶g̶ ̶r̶e̶t̶u̶r̶n̶e̶d̶ ̶i̶n̶s̶t̶e̶a̶d̶ ̶o̶f̶ ̶t̶h̶e̶ ̶e̶x̶p̶e̶c̶t̶e̶d̶ ̶e̶x̶c̶e̶p̶t̶i̶o̶n̶.
Capturing Console Output
With JUnit 5.3 it is possible to capture System.out and System.err output using a TestExecutionListener. Capturing console output isn’t a frequent need, but might be necessary if working on more legacy code bases where statements are frequently being written to console and it would be nice to see what is being written in a report.
To start capturing console output it requires two steps:
1. Implementing a TestExecutionListener
2. Configuring surefire to start capturing console output.
Here is an example of each:
TestExecutionListener
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| public class ConsoleOutputListener implements TestExecutionListener { | |
| @Override | |
| public void reportingEntryPublished(TestIdentifier testIdentifier, ReportEntry entry) { | |
| entry.getKeyValuePairs().values().stream().forEach(c -> System.out.println("Captured output: " + c)); | |
| TestExecutionListener.super.reportingEntryPublished(testIdentifier, entry); | |
| } | |
| } |
Surefire Configuration
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <plugin> | |
| <groupId>org.apache.maven.plugins</groupId> | |
| <artifactId>maven-surefire-plugin</artifactId> | |
| <version>2.22.0</version> | |
| <configuration> | |
| <properties> | |
| <configurationParameters> | |
| junit.platform.output.capture.stdout=true | |
| junit.platform.output.capture.stderr=true | |
| </configurationParameters> | |
| </properties> | |
| </configuration> | |
| </plugin> |
Library and Tool Updates
As mentioned in the introduction of this article, we will be taking a look at updates to some other projects often used with JUnit. Most of these changes are unrelated too (i.e. not dependent upon) JUnit 5.3, but I wanted to cover them here as they happened relatively recently.
Surefire Native Support
Back in June the maven surefire team released version 2.22.0 of their plugin. With this release the surefire (and failsafe 2.22.0) plugin offer native support for JUnit 5. This is a nice, because prior to the 2.22.0 release, using JUnit 5 required a lot of additional configuration which was frankly a bit of an eyesore. Prior to 2.22.0, to run JUnit 5 with surefire this configuration was required:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <plugin> | |
| <groupId>org.apache.maven.plugins</groupId> | |
| <artifactId>maven-surefire-plugin</artifactId> | |
| <version>2.21.0</version> | |
| <dependencies> | |
| <dependency> | |
| <groupId>org.junit.platform</groupId> | |
| <artifactId>junit-platform-surefire-provider</artifactId> | |
| <version>${junit-platform.version}</version> | |
| </dependency> | |
| <!– Vintage engine only needed if JUnit 3/4 tests present –> | |
| <dependency> | |
| <groupId>org.junit.vintage</groupId> | |
| <artifactId>junit-vintage-engine</artifactId> | |
| <version>${junit-jupiter.version}</version> | |
| </dependency> | |
| <dependency> | |
| <groupId>org.junit.jupiter</groupId> | |
| <artifactId>junit-jupiter-engine</artifactId> | |
| <version>${junit-jupiter.version}</version> | |
| </dependency> | |
| </dependencies> | |
| </plugin> |
With 2.22.0 the default surefire configuration can look like this:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <plugin> | |
| <groupId>org.apache.maven.plugins</groupId> | |
| <artifactId>maven-surefire-plugin</artifactId> | |
| <version>2.22.0</version> | |
| </plugin> |
Note: For spring boot projects, you need only add maven-surefire-plugin.version to your properties, like this.
Note 2: JUnit 5.3 has a bug that prevents tests from being found when the reuseForks tag is set to false. Read more here.
Filtering Changes with Native Support
However with native support comes changes with how tag filtering behaves. When using the JUnit surefire-provider in 2.21.0 filtering by tags looked like this:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <plugin> | |
| <groupId>org.apache.maven.plugins</groupId> | |
| <artifactId>maven-surefire-plugin</artifactId> | |
| <version>2.21.0</version> | |
| <configuration> | |
| <properties> | |
| <excludeTags>excludeTag</excludeTags> | |
| <includeTags>includeTag</includeTags> | |
| </properties> | |
| </configuration> | |
| <!– dependency stuff goes here –> | |
| </plugin> |
However this does not work when running maven-surefire-plugin 2.22.0. To pass to surefire which tags you want to filter on you must use the groups, for which tests to execute, and excludeGroups, for which tests should not be executed, tags. Here is the 2.22.0 equivalent example of the above:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <plugin> | |
| <groupId>org.apache.maven.plugins</groupId> | |
| <artifactId>maven-surefire-plugin</artifactId> | |
| <version>2.22.0</version> | |
| <configuration> | |
| <groups>includeTag</groups> | |
| <excludedGroups>excludeTag</excludedGroups> | |
| </configuration> | |
| </plugin> |
Enhanced Mockito Dependency Injection Support
As the tweet from @sam_brannen states, you can now directly inject mockito mocks via constructor or method parameters. This can make help make tests more declarative as to what they depend upon and/or a little cleaner. These changes were added in 2.21.0¹, here is an example of those changes in action:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| @ExtendWith(MockitoExtension.class) | |
| public class TestMockitoInjection { | |
| private BoringService service; | |
| public TestMockitoInjection(@Mock BoringService service) { | |
| this.service = service; | |
| } | |
| @Test | |
| public void testConstructorInjectedValue() { | |
| when(service.returnNumber()).thenReturn(2); | |
| assertEquals(2, service.returnNumber()); | |
| } | |
| @Test | |
| public void testMethodInjection(@Mock BoringService service) { | |
| when(service.returnNumber()).thenReturn(3); | |
| assertEquals(3, service.returnNumber()); | |
| } | |
| public class BoringService { | |
| public int returnNumber() { | |
| return 1; | |
| } | |
| } | |
| } |
AssertJ Soft Assertions and opentest4J
I have covered AssertJ in a previous blog post. With 3.11.1 AssertJ will now use the MultipleFailuresError in the opentest4j library, if it’s available. opentest4j is a library created by the JUnit 5 team to help make the testing experience more consistent across projects for both users and library developers.
If you are not familiar with soft assertions, they are a way of asserting multiple values in a single test and instead of the test failing on the first assertion, all the failed assertions are instead collected and reported at the end of the test. This can help avoid the irritating loop of a test failing, fixing the failure, running the test again, and hitting another failure. Here’s a look at soft assertions in AssertJ:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| public class TestAssertJSoftAssertions { | |
| @Test | |
| public void testSoftAssertions() { | |
| Person person = new Person("John", "Doe"); | |
| SoftAssertions softly = new SoftAssertions(); | |
| softly.assertThat(person.getFName()).isEqualTo("Anonymous"); | |
| softly.assertThat(person.getLName()).isEqualTo("Person"); | |
| softly.assertAll(); | |
| } | |
| public class Person { | |
| private String fName; | |
| private String lName; | |
| public Person(String fName, String lName) { | |
| this.fName = fName; | |
| this.lName = lName; | |
| } | |
| public String getFName() { | |
| return fName; | |
| } | |
| public String getLName() { | |
| return lName; | |
| } | |
| } | |
| } |
Running this test will return both assertion failures, instead of just reporting that the first assert failed.
Conclusion
The JUnit team continues to make great progress on JUnit 5 adding new features, improving on existing ones, and fixing bugs as they are reported. This is only a highlight of some of the new features and changes coming in JUnit 5.3. For a full list of all the changes and improvements coming besure to check out the release notes and the JUnit 5 user guide on how to use the features as well.
1: Technically the changes were added in 2.20.3, but that is only available from the mockito’s maven repository, not maven central.
Billy Korando 