Integration testing: You keep using that word, I do not think it means what you think it means

A good test harness is an essential safety net in any code base. Tests save us from ourselves – from writing bad functions, regressing existing features, and creating user journeys with dead ends. Most importantly, they give us a sense of trust that what we release to the world, while never perfect, is at least functional and getting better and better all the time.

The test pyramid: the size of each block roughly represents the ratio of the types of tests that should comprise your automated test suite.

The test pyramid: the size of each block roughly represents the ratio of the types of tests that should comprise your automated test suite.

Unit tests, the foundation of any reliable test suite, are usually the easiest to write, and the most obviously valuable. The (hopefully small and possibly pure) function outputs what was expected and the test passes. A solid layer of these – at least one per unit of logic – forms the base of the test pyramid. Similarly, we can immediately see the value of functional (or UI) tests. Simulate a user's journey through the product and verify that elements are present and interactive on screen. Because these tests tend to be very slow to run we should write as few as possible; these make up the top block of the test pyramid.

Unit tests are limited in scope but writing too many or too detailed functional tests will leave our test harness slow, brittle, and ultimately untrustworthy. So in addition, a thorough, well-written set of integration tests holds a strong test suite together. This is the essential middle block of the test pyramid.

In reality, very few projects have a robust integration test layer because no one really knows what they are. In fact, there is no such thing as an integration test. Integrate and test what exactly? It is a deliberately vague term we use to encapsulate that mushy bit of testing in between unit and UI testing that no one can really nail down. What we actually mean by that term is any test with a wider scope than a single nugget of logic but without the overhead and distance from the code of a functional test. And that's not just one type of test, but many. The most important part of building an integration test layer is determining which types of tests make sense for your codebase and team.

"You keep using that word. I do not think it means what you think it means."

For example, imagine you're building a retail website that interacts with an inventory API. A thorough test suite will inspect many different bits of the page: that it's displaying the correct information, any same-page user interactions (like adding something to the shopping basket) do what they should, navigating to other pages, and interacting with the external API. This is in addition to, not instead of, unit and functional UI tests (although there will definitely be overlap).

Different tech stacks have different testing limitations so take this into account when picking your toolkit. Using a framework like Enzyme for ReactJS applications, you can fully render a single page of an application with dummy input data and quickly test the generated DOM. Because this is so much quicker than spinning up your app and interrogating the browser for that information, you can very thoroughly test what should exist on the page at a level of detail that would be devastatingly slow in a UI test.

Testing an integration with an external API calls for contract tests. Make calls to the API that mimic those used in the application, and investigate the response. This is a massive waste of time if the API is well documented and widely used – for instance, don't bother writing contract tests for the Google Maps API. On the other hand, definitely write contract tests if the API is internal to your company or was written by an offshore team or has existed for 20 years and no one remembers how it works anymore (looking at you, every giant retail project ever).

Although it veers into the world of functional testing, running UI tests against mocked versions of your data sources and external APIs can make for valuable integration tests. If there are many different pathways through a site, writing functional tests to be run in isolation, while maintaining a set of golden-path non-mocked UI tests, allows you to test more of these paths with less overhead. Depending on the complexity and speed of the application's dependencies, you could potentially have all your functional tests run in a mocked environment if you're comfortable with the tradeoffs.

You might not use any of these testing strategies – you have to decide with your team what integration testing means for your specific product and codebase. Defining the term is the first step to building a layer of tests that will check the interactions between different parts of your codebase and actually finish running before the heat death of the universe. After all, what is the point of tests you're afraid to run or that you know doesn't catch bugs?

Unit tests will always be the rock of a good test suite, and functional golden-path journeys are essential smoke checks of your final output but so many developers are missing out on this powerful and efficient middle layer of tests. For a speedy but still robust test harness, keep writing unit tests, stop writing so many damn UI tests and rely more on integration testing techniques to build confidence in your code.


Want to learn more? Check out Graham's blog on testing React with Enzyme, and come talk to a Badger about testing at one of our events!