Pact: Contract Testing
Written by
Gilles Van Gestel
Contract Testing is a way to efficiently coordinate API communication between multiple parties. Where before we needed to notify all stakeholders of any changes to the API, with Contract Testing we can see when we might cause problems and for what parties. As a result we only have to inform these last parties instead of all parties. In this blog post we discuss an implementation of Contract Testing with Pact using Java and Spring. The goal is to provide those interested with practical background information for when they wish to start Contract Testing themselves.
Category:
Development
The big picture
When developing an API between two parties, communication has to be on point to prevent mishaps in the shared understanding of what the API messages look like. Contract testing is a way to prevent this kind of misunderstandings. Pact is a form of contract testing that allows the consumer to express its expectations of what these messages should look like. Thus, when the consumer accepts messages from a provider with which it has a valid Pact, it can rest assured that the messages meet the expectations it had set out for them.
The consumer expresses its expectations by writing a set of Unit tests. When running these tests a Pact is generated under the form of a JSON-file. These files are shared between consumers and providers through a Pact Broker. A Pact Broker is an entity hosted in an environment reachable by both parties, often through a public URL, with or without credentials.
After the consumer publishes the Pacts to the Broker the provider can use these Pacts to verify whether or not it complies with the consumer's expectations. The provider verifies the contract in an analogous manner by composing Unit tests which can be run against the provided Pacts. When successfully validating the Pact the provider can publish its results to the Broker. This way all parties are aware of which consumer-provider version pairs have a valid Pact between them.
Now, what is the impact of all this?
Traditionally, when a provider wishes to make changes to the API they have to sufficiently communicate their intentions with all stakeholders. Meaning they need to notify at least all consumers of their API that possible breaking changes are coming their way. With Pact, when a provider makes changes to the API, they can use the Pacts on the Pact Broker to verify whether or not they made any breaking changes to the API and for which consumers these changes were breaking. But as long as no Pacts are broken by the provider's changes, they can modify the API as much as they want without having to necessarily notify any of their consumers. This concept adheres nicely with Postel's Law.
Maven dependencies and plugins
To efficiently write the required Unit tests we use Pact in combination with JUnit 5 for both the consumer and the provider. We combine these dependencies with a Maven plugin.
At the time of writing (July 2023) we are using the following pom.xml properties.
Consumer dependency
Provider dependency
Maven plugins
This setup works for both consumers and providers, with the distinction that the <serviceProviders> part is specific to the provider. When configuring a consumer this part should be left out.
The <configuration> part that comes after <serviceProviders> contains some example configurations, more options can be found here.
Unit tests
To clarify, contract testing is not like functional API testing. It only tests for compliance with the shape of a message. Meaning that changes to a message which will break a consumer can be detected even before deploying the changes.
The consumer composes Unit tests to provide specifications of what it expects the messages to look like. JUnit 5 allows us to use a couple of useful annotations. The PactConsumerTest extension allows us to write Pact consumer tests with JUnit 5 and needs to be declared on Pact classes.
For every Pact test we need to define a separate method with the @Pact annotation. This method specifies the interactions for the test and what the Pact will look like.
As a final step, we define the test method with the @PactTestFor annotation. This method specifies how the Pact extension should set up the Pact test. This annotation can also be used on the test class.
There is no need for a Pact extension on the provider side. We simply specify the provider by using @Provider, the Pact Broker by using @PactBroker and if wanted any extra configuration, e.g. by using @IgnoreNoPactsToVerify. This last annotation makes sure that our tests don't fail in the absence of Pacts on the Broker. These annotations are to be declared on the test class.
For each Unit test on the provider side we need to declare a name of the request or event we will be verifying. This way the plugin knows what Pact specification to look for on the Broker. We do this by using @PactVerifyProvider.
Request-Response API
Pact was originally designed for request-response interactions. We mainly use it for event-driven interactions. For completeness, a short request-response example will be provided.
The consumer provides a specification of what the request looks like and what is expected from the response. In this case the consumer is an Authorization Service which requests permissions from a Permission Service, acting as the provider. The implementation shows a method getPermissions which specifies what the Pact will look like and what is to be expected from the interaction. It does this by utilizing a DSL builder (Domain Specific Language). We specify that we expect to receive a JSON object with one string in it, "permissions". This specification then serves as the input for a mock server. Finally, the response of this mock server is checked against a predefined dummy response in the Test method getPermissions_whenObjectWithId10Exists.
When all goes well a JSON file is generated containing a Pact for the interaction between the Authorization Service and the Permission Service. This way the Pact is only generated when the assertion that the Pact specification matches the expected object, succeeds. One could also choose not to use a JSON object to compare against but could compare an expected POJO against a mapping of the mocked JSON object to a POJO of the same class. This way the Pact is compared against the actual Java object that will be used in the consumer. When this assertion succeeds we know for sure that the consumer will be able to map the API response as long as the provider plays by the rules of the Pact.
With a simple Permission Service class.
After generating the Pact from the consumer specification and publishing said Pact to the Pact Broker, the provider is able to use this Pact as a way to verify if its response fits the desired specification. The provider side code is more compact. We simply provide a test URL to a (dummy) Permission Service and the required test State(s).
The Pact plugin does the heavy lifting for us. It verifies our Permission Service against the Pacts on the Pact Broker and publishes the results as well. More on the Pact Broker and publishing of results below.
For more details on Pact with request-response based API interactions, please refer to the official Pact documentation. Also, this tutorial provides valuable insights on the subject. More specifically the part about the consumer and the part about the provider.
Event-driven API
An event-driven Pact setup looks a lot like its request-response based counterpart but has some clear and important distinctions. In the case of an event-driven API it is the provider who publishes a message (e.g. an event) which in turn is picked up by the consumer. As before, it is the consumer who expresses what this message should look like and the provider who verifies if its messages comply with this specification. There is no request so one difference with the previous case is that the consumer has no need to specify what the request looks like.
As before, the consumer defines two methods, one being the @Pact method lunchEventReceivedPact which specifies what the message should look like and how it will be defined in the Pact. The difference with the request-response case here is that this method works with a MessagePact builder. This builder allows us to specify what kind of event we expect to receive and what the content of said event should look like. We use Lambda DSL to do this. It states that we expect a JSON body with some strings and an object with again some strings in it. The DSL allows us to specify rather complex structures. For example we can model an array with a minimum amount of objects and how many examples should be mocked for the test's assertion with .minArrayLike("menuOptions", 0, 1, menuOption -> { ... }. Then between the curly braces we can specify what the objects in the array should look like.
Just like with the request-response based Pact, we define a Test method verifyLunchEventReceivedPact to check the validity of the Pact's consumer side by comparing the proposed Pact against a dummy object that would be used by the consumer after receiving the event. After successful assertion the Pact's JSON file is generated and we can upload it to the Pact Broker.
The provider verifies this Pact through usage of the Maven plugin. It uses a dummy object we provided in the @PactVerifyProvider method verifyLunchEvent and compares it to the specification it found in the Pact. The plugin notifies us of the verification results and publishes them to the Pact Broker as well.
Pact Broker
The Pact Broker is a useful tool to share Pacts between consumers and providers. The Broker looks something like this:
It shows consumer and provider combinations together with when the Pacts were last published and last verified. The symbol(s) in the middle allow us to inspect the contents of the Pact and its version history.
For testing purposes you can deploy a Pact Broker using this docker-compose file.
Together with this postgres-service file.
Can-I-Deploy
The Pact Maven plugin offers another useful tool, can-i-deploy. This allows us to query whether or not the version we're about to deploy is compatible with the versions of the other apps that use our API. For example, we can execute can-i-deploy for one of the Pacts displayed in the Pact Broker image above.
This is answered with either positive feedback;
or negative feedback;
The can-i-deploy check is especially useful in automated actions like GitHub Actions. You can read more about can-i-deploy in general here.
That's it for this short introduction to Pact: Contract Testing. We'll finish up with some useful links:
- Pact documentation: https://docs.pact.io/
- Consumer dependency: https://docs.pact.io/implementation_guides/jvm/consumer/junit5
- Provider dependency: https://docs.pact.io/implementation_guides/jvm/provider/junit5
- Maven plugin: https://docs.pact.io/implementation_guides/jvm/provider/maven
- Maven plugin configuration properties: https://docs.pact.io/implementation_guides/jvm/docs/system-properties
- Request-response Pact tutorial: https://github.com/pact-foundation/pact-workshop-jvm-spring
- Can-I-Deploy: https://docs.pact.io/pact_broker/can_i_deploy
Let us talk
Contact us
Let us talk
Contact us
Let us talk
Contact us
Let us talk
Contact us
Discover us
Follow us
part of the
family