Integration tests
Designing a strategy of how to perform integration tests of your microservice is a key decision.
More often than not your microservice will need other components to perform its tasks: a database, another microservice, some files, etc…
When you need to design how are you going to test your microservice in an integration test, you need to choose between two different approaches:
- Mock your client connector
- Start another service with test data and make real calls
Mocking your connectors is the easiest way. However, with this approach you are leaving some code untested. If you have a bug inside your mapping logic or the parameters sent you are not going to realize during this kind of integration tests.
On the other hand, starting a mock server/database that will accept incoming connections is slower and more complex.
You need to strike a balance between them and make a decision, based on your needs.
In this post we are going to focus on how to perform an integration test over the entire stack, by staring a mock server.
Technology and use case
We are going to use Spring Boot Webflux and WireMock to peroform an integration test.
Our business logic will consist of building a microservice that returns data about SpaceX rockets.
In order to obtain that data, our microservice will need to call an external API and perform some basic data transformations.
The external API is this one: SpaceX Data.
Finally, we are going to use the endpoint that has to do with rockets.
Let’s move forward!
Set up the project
We are going to use Spring Initializr to create our project form scratch:
We are just setting our preferences to Maven, the last stable version of Spring Boot and Java 11.
Inside dependency section we will select Webflux and Lombok:
With these options we can generate our project.
After downloading the ZIP file, we can extract it and load it into our favorite IDE.
Now we can start coding!
TDD approach step by step
As usually, following a TDD approach, we are going to start writing a test.
Let’s start with the simplest one we can imagin:
package om.jfcorugedo.spacedatademo;
import org.junit.jupiter.api.Test;
import org.springframework.boot.test.context.SpringBootTest;
@SpringBootTest
class RocketsResourceTest {
@Test
void getValidRocket() {
}
}
We are using @SpringBootTest annotation to tell Spring this is an integration test, so the server should be started.
Since Spring 5 uses JUnint 5, we need to mark our tests with @Test annotation from jupiter.
Let’s code some assertions inside the test.
We are using Webflux Spring flavour, so we need to use a client that know how to deal with reactive return types.
We are going to use WebTestClient to make the calls to our API.
Following the documentation, we add a new WebTestClient instance to our test class and perform the initialization:
private WebTestClient client;
@BeforeEach
public void setUp(ApplicationContext context){
client=WebTestClient.bindToApplicationContext(context).build();
}
We are going to use a valid rocket name from SpaceX starship fleet: falcon9.
We can perform a call to the external API just to be sure the name is valid:
curl --request GET 'https://api.spacexdata.com/v3/rockets/falcon9'
This command will return a lot of data about this rocket.
Let’s go back to our integration test.
Now we are going to make a call to our endpoint and check the status code returned:
this.client.get()
.uri("/api/v1/rockets/{name}","falcon9")
.accept(APPLICATION_JSON)
.exchange()
.expectStatus().isOk()
.expectBody();
Quite simple test, but in TDD is always interesting to start easy.
If we run the test, it will give us a failure:
ERROR 8612 --- [ main] o.s.t.w.reactive.server.ExchangeResult : Request details for assertion failure:
> GET /api/v1/rockets/falcon9
> WebTestClient-Request-Id: [1]
> Accept: [application/json]
No content
< 404 NOT_FOUND Not Found
< Content-Type: [application/json]
< Content-Length: [149]
{"timestamp":"2021-05-03T13:17:33.435+00:00","path":"/api/v1/rockets/falcon9","status":404,"error":"Not Found","message":null,"requestId":"28a48515"}
java.lang.AssertionError: Status expected:<200 OK> but was:<404 NOT_FOUND>
Expected :200 OK
Actual :404 NOT_FOUND
Good. It fails as expected. Instead of a 200 OK status code we are getting a 404 Not Found code.
No surprises here, because we have not created our endpoint yet.
Let’s make the test pass by creating a Resource (or Controller) inside our applicaiton.
Since it is quite simple, we are going to use TDD Obvious implementation approach instead of Fake it:
package om.jfcorugedo.spacedatademo.rockets.resources;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RestController;
@RestController
public class RocketsResource {
@GetMapping("/api/v1/rockets/{name}")
public Mono<String> getRocket(@PathVariable String name) {
return Mono.just(String.format("%s Rocket", name));
}
}
We are using Mono as the return type, because Spring Webflux uses RxJava types by default.
Also, we are returning a String object because our test does not need any data yet.
Remember TDD mindset: Make it pass as soon as you can.
The implementation is pretty simple (even a bit dummy), but it is good enough, as it makes our test pass:
Now it is time to refactor!
No many things to do right now. Let’s just extract the common path of our resource to a class annotation:
@RestController
@RequestMapping("/api/v1/rockets")
public class RocketsResource {
@GetMapping("{name}")
public Mono<String> getRocket(@PathVariable String name) {
return Mono.just(String.format("%s Rocket", name));
}
}
Ok, let’s move on!
Now we are going to check for some fields in the response:
@Test
void getValidRocket(){
this.client.get()
.uri("/api/v1/rockets/{name}", "falcon9")
.accept(APPLICATION_JSON)
.exchange()
.expectStatus().isOk()
.expectBody()
.jsonPath("$.rocket_name").isEqualTo("Falcon 9")
.jsonPath("$.cost_per_launch").isEqualTo(50000000)
.jsonPath("$.success_rate_pct").isEqualTo(97);
}
In addition to the status code, now we are checking tree fields: rocket_name, cost_per_launch and success_rate_pct.
We are using JSON Path expressions, because they are pretty simple and powerful.
Obviously, when we run the test again, it fails due to the lack of these new checks:
java.lang.AssertionError: No value at JSON path "$.rocket_name"
Ok, let’s make it pass using TDD Fake it approach.
@RestController
@RequestMapping("/api/v1/rockets")
public class RocketsResource {
@GetMapping("{name}")
public Mono<Rocket> getRocket(@PathVariable String name) {
return Mono.just(new Rocket("Falcon 9", 50000000, 97));
}
}
This new code does not compile. There isn’t any Rocket class in our application yet.
Let’s create one:
package om.jfcorugedo.spacedatademo.rockets.dto;
import lombok.*;
@Getter
@AllArgsConstructor
@NoArgsConstructor
@ToString
public class Rocket {
private String rocket_name;
private int cost_per_launch;
private double success_rate_pct;
}
Now with all of these small changes the test is green again.
Let’s start refactoring.
The name of the fields does not look quite smart. Let’s change them to camel case:
package om.jfcorugedo.spacedatademo.rockets.dto;
import com.fasterxml.jackson.annotation.JsonProperty;
import lombok.*;
@Getter
@AllArgsConstructor
@NoArgsConstructor
@ToString
public class Rocket {
@JsonProperty("rocket_name")
private String rocketName;
@JsonProperty("cost_per_launch")
private int costPerLaunch;
@JsonProperty("success_rate_pct")
private double successRatePct;
}
We need to use JsonProperty anotation since we don’t want to change our API definition.
Now lets focus on the RocketsResource controller.
We are not really doing our job. We are just returning what the method is expecting. So let’s remove duplicates by calling the real API:
@GetMapping("{name}")
public Mono<Rocket> getRocket(@PathVariable String name) {
WebClient client = WebClient.create("https://api.spacexdata.com/v3/rockets/".concat(name));
return Mono.just(new Rocket("Falcon 9", 50000000, 97));
}
We are going to use WebClient connector. It uses Netty behind the scenes to make a call reactively.
That is the reason it does not return the result, but a RxJava type.
Remember, we are building a reactive microservice, so blocking calls are not an option.
Run the test again and check it is still in green, since we are still returning a Mono.just(...) instance.
Let’s change it:
@GetMapping("{name}")
public Mono<Rocket> getRocket(@PathVariable String name) {
WebClient client = WebClient.create("https://api.spacexdata.com/v3/rockets/".concat(name));
Mono<Rocket> result = client.get()
.accept(MediaType.APPLICATION_JSON)
.retrieve()
.bodyToMono(Rocket.class);
return result;
}
The fluent api of a WebClient instance is very expressive and easy to use. If you have time, play a little with its API.
If we run the test now, we will check that it is still in green.
Ok, let’s continue with the refactors.
It is not a good thing to have a URL hardcoded in our controller, so we are going to move the URL to our application.property file:
spacexdata.url=https://api.spacexdata.com/v3/rockets/
Now we need to modify RocketsResource to use this property instead of the hardcoded value:
@RestController
@RequestMapping("/api/v1/rockets")
public class RocketsResource {
@Value("${spacexdata.url}")
private String baseUrl;
@GetMapping("{name}")
public Mono<Rocket> getRocket(@PathVariable String name) {
WebClient client = WebClient.create(baseUrl.concat(name));
Mono<Rocket> result = client.get()
.accept(MediaType.APPLICATION_JSON)
.retrieve()
.bodyToMono(Rocket.class);
return result;
}
}
We have added a new class attribute (baseUrl) annotated with Spring @Value annotation to inject the value from the property file.
Finally, instead of using the hardcoded value, we are now using this class attribute to build the URL we are using to call the external API.
If we run the test, everything is in green.
Great!
However, there is a problem. This test needs an internet connection to pass.
And what worse, it depends on an external API. If something fails in the network or in the API, the test will fail.
That is not acceptable. We want our test to be as much isolated as we can from the external environment.
Here is when WireMock comes to the rescue.
With WireMock, we can start a server that simulates responses from our external APIs so our test no longer needs an internet connection or a third-party applicaiton up and running.
To use WireMock, the first thing to be done is to add WireMock dependency to our pom.xml:
<dependency>
<groupId>com.github.tomakehurst</groupId>
<artifactId>wiremock-jre8</artifactId>
<version>2.27.2</version>
<scope>test</scope>
</dependency>
Then we need to create and configure an instance of the server inside our Test class:
package om.jfcorugedo.spacedatademo;
import com.github.tomakehurst.wiremock.WireMockServer;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.context.ApplicationContext;
import org.springframework.test.web.reactive.server.WebTestClient;
import static com.github.tomakehurst.wiremock.core.WireMockConfiguration.options;
import static org.springframework.http.MediaType.APPLICATION_JSON;
@SpringBootTest
class RocketsResourceTest {
public static WireMockServer wireMockServer = new WireMockServer(options().dynamicPort());
private WebTestClient client;
@BeforeEach
public void setUp(ApplicationContext context) {
client = WebTestClient.bindToApplicationContext(context).build();
}
@Test
void getValidRocket(){
this.client.get()
.uri("/api/v1/rockets/{name}", "falcon9")
.accept(APPLICATION_JSON)
.exchange()
.expectStatus().isOk()
.expectBody()
.jsonPath("$.rocket_name").isEqualTo("Falcon 9")
.jsonPath("$.cost_per_launch").isEqualTo(50000000)
.jsonPath("$.success_rate_pct").isEqualTo(97);;
}
}
The field needs to be static, since we are telling WireMock to use a dynamic port, so we will need to change our configuration before the execution of any test.
Next, start the server after all the tests:
@BeforeAll
static void startWireMock() {
wireMockServer.start();
}
Finally, we need to change Spring configuration to point at the WireMock server instead of the real API during tests execution.
To do so, we need to use Spring @ContextConfiguration annotation.
@ContextConfiguration defines class-level metadata that is used to determine how to load and configure an ApplicationContext for integration tests.
We are going to use an inner class to change application.properties on the fly to point at the WireMock server we have just started.
@SpringBootTest
@ContextConfiguration(initializers = RocketsResourceTest.ConfigInitializer.class)
class RocketsResourceTest {
....
static class ConfigInitializer implements ApplicationContextInitializer<ConfigurableApplicationContext> {
@Override
public void initialize(ConfigurableApplicationContext applicationContext) {
TestPropertyValues.of(
String.format("spacexdata.url=http://localhost:%d/", wireMockServer.port())
).applyTo(applicationContext);
}
}
}
As you can see, we are asking WireMock about the port it is using at this execution to change spacexdata.url property before all the tests.
If we run the test now, it will give us a failure:
org.springframework.web.reactive.function.client.WebClientResponseException$NotFound: 404 Not Found from GET http://localhost:52352/falcon9
It makes sense. We did not tell WireMock what to do when someone ask him about rockets.
So, to finish the integration, we need to tell WireMock which response we expect to receive from a falcon9 call.
In order to do that, we need to do two things:
Add a JSON file with the expected response to src/test/resources/__files/rockets/falcon9.json file.
You can call the external API and store the respone in that file or you have an example here.
Finally, we need to tell WireMock to use this file inside our Test method:
@Test
void getValidRocket(){
wireMockServer.givenThat(
get(urlEqualTo("/falcon9"))
.willReturn(
aResponse()
.withStatus(200)
.withHeader(CONTENT_TYPE, APPLICATION_JSON.toString())
.withBodyFile("rockets/falcon9.json")
)
);
this.client.get()
.uri("/api/v1/rockets/{name}", "falcon9")
.accept(APPLICATION_JSON)
.exchange()
.expectStatus().isOk()
.expectBody()
.jsonPath("$.rocket_name").isEqualTo("Falcon 9")
.jsonPath("$.cost_per_launch").isEqualTo(50000000)
.jsonPath("$.success_rate_pct").isEqualTo(97);;
}
With all these changes, the test will be in green again, and without any external dependency!
To leave everything in order, two more things need to be done inside the test: stopping wiremock server after all the tests and removing any mock response after each test.
Stop wiremock server after all the tests:
@AfterAll
static void tearDownWireMock() {
wireMockServer.stop();
}
Clear all the mocks after each test:
@BeforeEach
public void setUp(ApplicationContext context) {
client = WebTestClient.bindToApplicationContext(context).build();
wireMockServer.resetMappings();
}
The test class should look like this one:
package om.jfcorugedo.spacedatademo;
import com.github.tomakehurst.wiremock.WireMockServer;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.util.TestPropertyValues;
import org.springframework.context.ApplicationContext;
import org.springframework.context.ApplicationContextInitializer;
import org.springframework.context.ConfigurableApplicationContext;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.web.reactive.server.WebTestClient;
import static com.github.tomakehurst.wiremock.client.WireMock.*;
import static com.github.tomakehurst.wiremock.core.WireMockConfiguration.options;
import static org.springframework.http.HttpHeaders.CONTENT_TYPE;
import static org.springframework.http.MediaType.APPLICATION_JSON;
@SpringBootTest
@ContextConfiguration(initializers = RocketsResourceTest.ConfigInitializer.class)
class RocketsResourceTest {
public static WireMockServer wireMockServer = new WireMockServer(options().dynamicPort());
private WebTestClient client;
@BeforeAll
static void startWireMock() {
wireMockServer.start();
}
@AfterAll
static void tearDownWireMock() {
wireMockServer.stop();
}
@BeforeEach
public void setUp(ApplicationContext context) {
client = WebTestClient.bindToApplicationContext(context).build();
wireMockServer.resetMappings();
}
@Test
void getValidRocket(){
wireMockServer.givenThat(
get(urlEqualTo("/falcon9"))
.willReturn(
aResponse()
.withStatus(200)
.withHeader(CONTENT_TYPE, APPLICATION_JSON.toString())
.withBodyFile("rockets/falcon9.json")
)
);
this.client.get()
.uri("/api/v1/rockets/{name}", "falcon9")
.accept(APPLICATION_JSON)
.exchange()
.expectStatus().isOk()
.expectBody()
.jsonPath("$.rocket_name").isEqualTo("Falcon 9")
.jsonPath("$.cost_per_launch").isEqualTo(50000000)
.jsonPath("$.success_rate_pct").isEqualTo(97);;
}
static class ConfigInitializer implements ApplicationContextInitializer<ConfigurableApplicationContext> {
@Override
public void initialize(ConfigurableApplicationContext applicationContext) {
TestPropertyValues.of(
String.format("spacexdata.url=http://localhost:%d/", wireMockServer.port())
).applyTo(applicationContext);
}
}
}
Conclusion
Now you have a way of testing external dependencies in your integration tests using all the real components involved in your application.
As I told you at the begining, this is not a silver bullet. You need to make a decision whether or not you want to test your entire stack or just mocking the connectors.
You can see all the code of this example in this repository: spacedata.
