Testcontainers with OpenAPI

Testcontainers is an open-source multi-language framework that has revolutionised integration testing by leveraging containerisation.

The concept is simple yet powerful: developers define and spin up Docker containers directly from the test code. These containers serve as replicas of the dependencies utilised by the application, creating an effective environment for running integration tests.

Testcontainers offers an extensive selection of pre-configured containers, such as databases, message queues, and web servers. Additionally, it offers the flexibility to create new containers on the fly. This is a remarkable feature: developers can mock any dependency they require, including APIs under development.

And this is how I created a Testcontainer OpenAPI extension.

In this article: create a Testcontainer from an OpenAPI file, execute the integration testing leveraging the API contract, follow a step-by-step guide on how to test your API

Check out Contract Testing with OpenAPI to understand the challenges and solutions of Contract Testing with the OpenAPI standard.

Photo by Glen Carrie on Unsplash

Intro to Testcontainers

If you haven’t done it yet, head to the Quickstart example and try it out.

Testcontainers seamlessly integrate with the unit testing framework (JUnit for Java, Testing package for Go, etc..) of your choice. During the execution of the tests, the dependencies (i.e. database, cache, queues) required by your application are loaded by Testcontainers.

Testcontainers require a Docker runtime.

Testing with Testcontainers — image by author

Contract testing with OpenAPI

In my previous blog, I have discussed the challenges (and options) of implementing contract testing with OpenAPI.

In a nutshell, the idea is to define the request-response interactions inside the OpenAPI file, the contract of your API. OpenAPI does not offer (yet) a native method for establishing those interactions, therefore we need to find a workaround: defining custom tags or adopting strict naming conventions.

Once this is solved, we can create an API mock server that implements those expectations. This mock server is going to be our Testcontainer.

Use cases

An API Tescontainers module can serve 2 use cases.

API SDK development

API SDKs are valuable tools that help developers integrate APIs without the need to implement the REST calls and all other aspects of the protocol. Adyen, for example, provides open-source libraries to build better integrations.

If you’re familiar with SDK development, you know that integration testing is critical to verify the following:

• the SDK code invokes the API endpoints

• request payloads are built correctly

• JSON responses are properly deserialized

• headers are set accordingly

Writing integration tests that invoke directly the API (on a test server) is always an option, but far from ideal. It is slow, and unpredictable (think of a network problem, server maintenance or a re-deployment of the service) and might have side effects (i.e. modify data during a POST request).

Mock 3rd party API

In many projects, developers consume 3rd party APIs. However, directly invoking these APIs during integration testing can be tricky. Factors such as latency, availability, and changes in behaviour can lead to unreliable integration tests.

OpenAPI Testcontainers to the rescue

In both scenarios Testcontainers takes the pain away: the API runs in a Docker container, either locally or in a CI, creating an accurate replica of the production environment where integration tests are fast(er) and reliable.

It also gives you control over the test scenario: unlike the “real” API, it is possible to define specific scenarios and simulate various responses, error conditions, or edge cases.

OpenAPI mock server container

The generation of the mock server is accomplished using the OpenAPI generator. This popular open-source project allows the generation of code, documentation and tools starting from an OpenAPI file.

Leveraging the OpenAPI generator and creating a new specific generator (that can determine the request-response interactions — as mentioned above) the API server-side code is generated and packaged in a Docker image.

Photo by Daniel K Cheung on Unsplash

Getting started

Check out the OpenAPI Testcontainers demo. It includes a sample Java project that shows all you need to do:

• create a multi-stage Dockerfile

• write the integration tests

The project is a sample SDK for using a User Management API: the UserService class provides a few methods that wrap the API calls (i.e. GET https://api.example.com/svc/users/{id}). The integration tests start a container hosting the API mock server and validate that the SDK methods can invoke the server-side API.

Docker file

The Docker file defines the container that the integration tests will interact with. This approach takes advantage of the “Creating images on the fly” feature of Testcontainers.

There are 3 steps to consider:

1. Setup: install the OpenAPI generator

2. Make: generate the API server-side implementation

3. Package: build a minimal Docker image

Using a Docker multi-stage build the process becomes more efficient, resulting in a smaller footprint for the final image.

Setup

The first step is already provided in the base image (gcatanese/openapi-native-mock-server) by the OpenAPI Native Mock Server. This includes everything you need (JDK, OpenAPI Generator client) to run the OpenAPI generator.

Onto task 2, the most interesting one.

Make

This step generates the server-side code of the API. Using the OpenAPI Generator it creates a Go server that implements all endpoints of the API contract.

Note: the Go server runs in Docker therefore it doesn’t matter which language or stack you are using (as long as it is supported by Testcontainers). The demo application, for example, is in Java.

By leveraging the predefined request-response interactions, the API makes sure that for any given request, the expected response will be returned. In cases where a request does not match any of the predefined responses, a default JSON payload is generated to ensure that there is always a response available.

During this step, the OpenAPI file must be available and passed to the Docker build.

Package

The last step is the creation of a minimal Docker image, starting from a “scratch” image.

In the context of Docker, a “scratch” image refers to an empty base image. It is a special image, provided by Docker, that contains no files. When using the scratch image, you cannot rely on any pre-existing operating system or runtime to execute your application.

By leveraging Go static binary compilation, we can build a self-contained binary executable and copy it into the scratch image. The result is an extremely lightweight image (~15MB in this Sample app) that contains just the API server, highly convenient in the context of integration testing.

FROM gcatanese/openapi-native-mock-server

# pass OpenAPI file
ADD openapi.yaml /openapi/openapi.yaml
# codegen
RUN java -cp /openapi/bin/openapi-native-mock-server.jar:/openapi/bin/openapi-generator-cli.jar \
  org.openapitools.codegen.OpenAPIGenerator generate -g com.tweesky.cloudtools.codegen.NativeMockServerCodegen \
   -i /openapi/openapi.yaml -o /openapi/go-server

# build Go executable
FROM golang:1.19-alpine3.15
COPY --from=0 /openapi/go-server ./go-server

WORKDIR /go/go-server
RUN go mod tidy

RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o /build .

# build minimal image
FROM scratch AS runtime
# copy executable from previous stage
COPY --from=1 /build .

EXPOSE 8080
ENTRYPOINT ["./build"]

Write the tests

Import the Testcontainers dependency (Maven or Gradle) and mark the tests with the @Testcontainers annotation.

@Testcontainers
public class UserServiceTest {

Next, define the @ClassRule : this is the JUnit rule that manages the container at class-level and therefore shared across all tests in the class. The ClassRule starts the container before any test in the class is executed and stops it when they are all completed.

Here is where you define the OpenAPI file that will be used to generate the API server-side code.

private final static String OPENAPI_FILE = "src/test/resources/openapi.yaml";    
private final static boolean DELETE_ON_EXIT = false;

@ClassRule
public static GenericContainer container = new GenericContainer(
   new ImageFromDockerfile("my-test-cont", DELETE_ON_EXIT)
      .withFileFromFile("openapi.yaml", new File(OPENAPI_FILE))
      .withFileFromFile("Dockerfile", new File("Dockerfile"))
    )
    .withExposedPorts(8080);

A few important remarks:

• the OpenAPI file must be in the Dockerfile build context: a good place is with the test resources (src/test/resources)

• setting the deleteOnExit flag to false doesn’t delete the container when the JVM exits, so that it is already built and ready when we run the tests the next time

• only when the OpenAPI file changes the container is rebuilt

POST tests

Finally, we write our tests. No surprises here but it is interesting to look at the scenario we test. In this first test, we verify a successful GET user.

// test successful get user
@Test
public void getUser() throws Exception {
  User user = userService.get("usr0001");

  assertEquals("usr0001", user.getId());
  assertEquals("Alan", user.getFirstName());
}

The UserService.get method, given the id usr0001 , returns a User object with the expected ID and first name. This expected response is defined in the OpenAPI file.

In the second test, we check the scenario when the user is not found.

@Test(expected = RuntimeException.class)
public void userNotFound() throws Exception {
  User user = userService.get("usr9999");
}

Again, the OpenAPI file defines this interaction (return 404 when id is usr9999 ) and the test confirms that the SDK throws an exception.

Conclusion

Testcontainers make integration testing easy.

Using the OpenAPI extension I have shown how to create a container starting from an OpenAPI specification. The API server-side code is generated with OpenAPI Generator and packaged in a lightweight Docker image.

The Getting Started guide presents a step-by-step tutorial covering the setup, including the creation of the Docker file, and writing and executing the tests.

A sample demo also is provided to show Testcontainers with OpenAPI in action.