How to Better Captain of the Enterprise with Apache Maven

Apache Maven is popular tool for project management and defining how a project should be built. It’s flexible, extendable, and, because of it’s aforementioned popularity, most Java developers have at least some experience with it, creating a virtuous cycle of its usage.

When an enterprise adopts Maven, a frequent goal is to use it’s features and mechanisms as a way to encourage, or even enforce, enterprise standards and to make project initialization and maintenance easier. Over my career I have worked at a number of organizations, both as a developer consuming POMs defined by architects and technical leads, and as an architect defining POMs that would be consumed across an enterprise.

From this experience I have learned some of the pain points and common mistakes enterprises make when trying to use Maven. Often these pain points and mistakes are borne from a lack of knowledge of Maven’s feature set and/or incorrect usage of features. In this article we will step through how to make Maven more flexible and easier to manage in an enterprise setting.

picard

Being a Better Parent (POM)

One of the most common ways an enterprise utilizes Maven is through the creation of an enterprise wide parent, or super, POM. Parent POMs can be a great tool for encouraging developers to follow enterprise standards and reducing bloat in child POMs be defining commonly used dependencies, plugins, reporting etc.. Parent POMs can also become a huge burden slowing down build times, bloating the size of artifacts, and causing other problems in projects that consume the parent.

Maven provides features though when defining a parent POMs that allow it to be more flexible; offering help when and where it’s needed, but also quietly stepping back when a project might have unique needs and requirements. Let’s take a look at some of these key features.

Dependency Management

When defining a parent POM a common mistake is to add a lot dependencies that are believed to be universally required, but simply are not. Downstream this can cause problems because while it was thought every project would need dependencies X, Y, and Z, in reality some projects don’t need dependency Y, and for other projects; dependency Z causes an error when present on classpath.

Maven does provide a feature that gives a nice compromise between offering assistance with which dependencies should be used, particularly which version of a dependency should be used, and also keeping child parent POMs “clean”, and that is with Dependency Management. Dependency management is easy to use, just define dependencies as normal, but enclose them dependencyManagement tags. Here is an example below of using dependency management:


<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.my.developer</groupId>
<artifactId>my-org-starter-security</artifactId>
<version>0.0.1-SNAPSHOT</version>
</dependency>
</dependencies>
</dependencyManagement>
</project>

In the child pom we need only provide the groupId and artifactId of a dependency and all the additional information: version, type, scope, is pulled down transitively.

Screen Shot 2020-01-07 at 12.34.49 PM

Dependency management strikes a nice balance for both downstream developers and the architects and technical leads who are responsible for parent POMs.

Downstream developers benefit by not having to worry as much about which version of a dependency to use, what it’s scope should be, or their projects becoming bloated because a parent POM is adding extra dependencies they do not need. Downstream developers also benefit as the POMs for their projects are “cleaner” as only the artifactId and groupId for a dependency needs to be defined.

Tech leads and architects benefit by still having a centralized location from which to manage dependencies for downstream projects, but not have to deal with the issues that come from imposing the usage of dependencies, had the dependencies been defined normally within a parent POM.

Plugin Management

The same problems with dependencies, can also happen with plugins, and for that Maven offers a similar solution with Plugin Management. When you have a frequently, but not universally, used plugin, defining the plugin with plugin management is a good option.

A common example of this would be the maven-failsafe-plugin. Every project should be using the maven-surefire-plugin to run unit tests, however not every project will need the failsafe plugin which is commonly used for executing integration tests (an example here would be a shared library which might not need to communicate with any remote systems).

By defining the maven failsafe plugin using plugin management, when a project needs to execute integration tests, they need only add groupId and artifactId for the failsafe plugin in their POM, and then all the configuration defined within the pluginManagement of the parent POM. Here is an example of using pluginManagement in a parent POM:


<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
<includes>
<include>**/ITest*.java</include>
<include>**/*ITest.java</include>
<include>**/*ITests.java</include>
<include>**/*ITestCase.java</include>
</includes>
</configuration>
</plugin>
</plugins>
</pluginManagement>
</build>
</project>

And then a child POM pulling down the configuration of the failsafe plugin:

Screen Shot 2020-01-07 at 12.35.21 PM

Properties

Maven POMs also have section for properties. Like with any feature in Maven, and most programming languages, the most local definition of a property overrides the value that’s pulled down from a parent. Importantly though this overriding also applies to how that property is used within a parent POM. In the below example I have defined the property my-org.version, in my parent POM and set it to 0.0.1-SNAPSHOT. Under my dependency management section I reference that property in the version field of ​my-org-starter-security. This means by default the version of ​my-org-starter-security will be set to 0.0.1-SNAPSHOT in any child POM. Here is the code snippet below:


<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<properties>
<my-org.version>0.0.1-SNAPSHOT</my-org.version>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.my.developer</groupId>
<artifactId>my-org-starter-security</artifactId>
<version>${my-org.version}</version>
</dependency>
</dependencies>
</dependencyManagement>
</project>

view raw

properties.xml

hosted with ❤ by GitHub

However in a child POMs if the property of my-org.version is defined and set to 0.0.2-SNAPSHOT, then that project will use version 0.0.2-SNAPSHOT of my-org-starter-security even though the version field is not of that dependency is not defined.Screen Shot 2020-01-07 at 1.36.52 PM.png

This behavior would apply to anywhere where properties are used, not just to dependencies. Using properties can allow an easy way for downstream projects to change the version of dependencies they are using, as seen in the above example, or change the behavior of a plugin or reporting tool. As you are defining dependencies, plugins, etc. in parent POMs, strongly consider using properties instead of hardcoded values for fields downstream projects might want to change.

Looser Coupling with BOMs

For many enterprises parent POMs alone will be enough for handling dependency and project management concerns. However for larger enterprises, or an enterprise that works in a number of different domains, they might struggle using parent POMs alone. I experienced this at an organization where we created a whole tree of parent POMs that looked something like this chart:

Screen Shot 2020-01-08 at 2.14.09 PM

There were two major problems here:

  1. The tree was so top heavy that a change in the top POM kicked off a lot of downstream builds
  2. The development cycles between the organization wide parent POM and downstream POMs were not in-sync. So this created a lot of confusion in versioning. If the “Super Parent” had a major version change, should that require a major version change in downstream POMs? If a downstream POM had a lot of changes should it also increase it’s major version or should that only be tied to upstream changes?

This confusions led to  frustration for both the architecture team that I was a part of and the downstream developers as they were constantly finding issues and having to update parent POM versions.

In situations like this, it might be advantageous to use BOM’s, or Bill of Materials, so there is a looser relationship to an organization’s parent POM. A real world example of this is in the Spring framework, the Spring Cloud team publishes a BOM for managing the libraries that are associated with that project as they have a different release cadence from the Spring Boot and Spring Framework projects. You can view the BOM the Spring Cloud team produces here.

Going back the earlier example, instead of a complex tree of POMs and child POMs, a single “super parent” POM could be used and a pair of BOMs that cover the Web and Batch domains could be created. This approach likely would had been less work for the architect team that I was on, and a more pleasant experience for downstream developers as there would be fewer changes to consume.

Screen Shot 2020-01-08 at 3.25.01 PM

Shared Library POMs

In a recent article I stepped through how to create a custom Spring Boot starter. When creating a library that will be shared within your enterprise it’s important to take care when defining the library’s POM. A developer’s perception of a shared library can sour quickly if using it brings in extra dependencies, causes classpath conflicts, or leads to other problems. Let’s look at some strategies for when creating a shared library’s POM.

Scopes

Maven has several scopes that can be used to define when and under which circumstances a dependency should be included. The scopes are; compile, provided, runtime, and test.

  • Compile: is the default scope and means that a dependency must be available when a project is being built and packages in its artifact.
  • Provided: the dependency must be present when the project is being built, but will be provided by the system that will be executing the artifact.
  • Runtime: the dependency isn’t required to build the artifact but must be present when running the artifact.
  • Test: the dependency is only used as part of the test cycle, and not required at runtime.

Using these scopes correctly can help avoid unnecessary dependencies being brought and give developers consuming a library more control on what dependencies are being brought in.

Optionals

Some dependencies might be required only in certain circumstances. In my Spring Boot starter example,  my-org-starter-security, the starter could be used in a web context and a console context. To build the project the spring-boot-starter-web and spring-security-web dependencies needed to present, but only needed if the project consuming my-org-starter-security is a web project. By setting spring-boot-starter-web and spring-security-web as optional console developers don’t have to worry about a bunch of extra web libraries being brought in. Below is an example of setting some dependencies as optional in a POM file:


<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
<version>${spring.boot.version}</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-aop</artifactId>
<version>${spring.framework.version}</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-config</artifactId>
<version>${spring.framework.version}</version>
<scope>compile</scope>
</dependency>
<!– Optional Dependencies –>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<version>${spring.boot.version}</version>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-web</artifactId>
<version>${spring.framework.version}</version>
<scope>compile</scope>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-configuration-processor</artifactId>
<version>${spring.boot.version}</version>
<optional>true</optional>
</dependency>
</dependencies>
</project>

Examples at Scale

Small scale demonstrations on how to use the features highlighted in this article is one thing, but often new problems occur at scale or once business requirements are introduced. To see examples of these concepts applied at scale I would recommend checking out how the Spring team has structured their POMs which can be found on their project GitHub repos:

https://github.com/spring-projects/spring-boot

https://github.com/spring-cloud

Conclusion

Image of star trek next generation enterprise flying in space

Proper management of parent POMs and shared library POMs can make a world of difference in how smoothly an enterprise operates. Define them well and you have happy developers and architects who can explore more important questions. Define them poorly and you will find yourself stranded while you try desperately to fix problems.

This article focused on a few key features to make managing maven easier in most enterprise settings, it only scratches the surface of all that Maven can do however. If you want to learn more be sure to check out the official documentation, as well as this great article which steps through a full featured maven pom.

You can view the code for this article on my GitHub Repo.

Spring Cloud Contract, Completing the Consumer Driven Contracts Story

A new release of Spring Cloud, Hoxton, is gathering steam. Part of Milestone 2 release is a feature I have been looking forward to for Spring Cloud Contract (SCC), the ability to generate stubs directly off of contracts. In this article we are going to take a look at what this means and how to use this new feature.

Consumer Driven Contracts in a Polyglot World

Last year SCC implemented the functionality that allowed clients to write contracts without having to be familiar with the Java tech stack. This was done by supporting a YAML DSL for defining contracts and providing two Docker images, one that could consume contracts and validates a service is meeting the contracts and the other that can consume stubs so it could act as a mock of the service the contracts were written for. For more information on this, check out this blog article by Marcin Grzejszczak.

A common approach in many Java shops is to have a backend written in Java (i.e. the producer service), but the frontend UI (i.e. the consumer) written in JavaScript. By removing the reliance on Java specific technologies when writing contracts, this made writing contracts more accessible to frontend developers. By allowing the consumers to write the contracts you are able to do a work flow called Consumer Driven Contract Development. The consumers write a contract, telling a producer what they need in a format that can be validated programmatically, and then a producer developer can write, or update, the producer service to meet the requirements of the contract.

Consumer Driven Contract Development is a desirable development paradigm because it is a “pull” rather than “push” concept. Instead of producers “guessing” what a consumer might need, you allow the consumers to define the behavior they need from the producer. This helps to producers avoid building APIs consumers don’t need, or building APIs that are difficult for consumers to, well, consume.

We Keep Waiting on the Service to Change

While SCC’s changes allowed consumers to write contracts with technologies they were more familiar with, the consumer driven contract story was still a bit hollow. SCC had a Docker images that could act as a mock of the service, however it relied upon the availability of stubs. These stubs were only generated after the contracts were validated against the producer service confirming the producer service fulfilled the contracts.

A developer building a consumer could write a contract, but until the producer service actually implemented the behavior defined in the contract, the consumer wouldn’t be able to use the mock service. This could stall development on the consumer side for days or even longer depending on the workload and priorities of the developers of the producer service. If we are going to have consumer driven contracts, we need to allow consumers to be able to develop independently of producers. The contracts are still there, and since both parties are developing based on the contract, they should still arrive at the same point.

Running a Mock Service from Contracts

With the M2 Spring Hoxton release, SCC added the functionality to run a mock service directly from contracts. I have been presenting this year about Spring Cloud Contract, and how it can be used to do Collaborative Contract Driven Development, here is my presentation from Spring I/O in Barcelona back in May. I’m happy that now with the ability to run the mock service directly from contracts, this better completes the story in my presentation.

I have created a GitHub project for demonstrating how to use Spring Cloud Contract. The project has a repo for the producer, client (consumer), and a repo for the contracts. If you clone the client repo, you can see SCC’s new functionality in action by executing the run-produce-service-mock-no-stubs.sh script. The script contains the following:

docker run --rm \
-e "STUBRUNNER_IDS=com.ibm.developer:produce-service:+:stubs:9876" \
-e "STUBRUNNER_REPOSITORY_ROOT=git://https://github.com/wkorando/produce-contracts.git" \
-e "STUBRUNNER_STUBS_MODE=REMOTE" \
-e "STUBRUNNER_GENERATE_STUBS=TRUE" \
-p "8083:8083" \
-p "9876:9876" \
springcloud/spring-cloud-contract-stub-runner:2.2.0.BUILD-SNAPSHOT

This script is running a docker image and passing in some environment variables. Let’s take a look at some of the key values being passed in:

  • ​​​​​STUBRUNNER_REPOSITORY_ROOT This is telling the Docker image where the contracts reside
  • STUBRUNNER_STUBS_MODE This is telling the docker image the contracts are in a remote location (in this case a GitHub repo)
  • STUBRUNNER_GENERATE_STUBS This is variable that was added that tells the Docker image to build stubs directly off the contracts we sent
  • -p "9876:9876" This is telling the Docker image to have the mock service running on port 9876

Executing the script will have a mock service running on http://localhost:9876. If you look at the documentation that we generate from the contracts, you can see some of what the mock service is capable of doing. If you call the url http://localhost:9876/api/v1/produce you will get back a JSON list of produce items. You can also see how this works when working with a real UI by cloning this repository and running <span class="s1">npm run start:contract.

Conclusion

The GA release of Spring Cloud Hoxton should be available later this fall. The ability to generate stubs straight from contracts is a significant step forward in completing the Consumer Driven story. It’s a feature I have been looking forward to since I first started with with Spring Cloud Contract a few years back. If you are attending Spring One Platform next week and are interested in learning more about contract driven development, be sure to swing by my presentation on Thursday Collaborative Contract Driven Development.