Automated testing is integral to software engineering, ensuring that a product meets the desired specifications. Many types of automated tests are available depending on the design of an application. Choosing a test type often requires coordination between product owners and QA/software engineers to decide what and how to test. In this article, we will discuss the purpose of different types of automated tests and when to use them.
Top 3 Most Impactful Test Types
Unit Tests
Unit tests are the most basic type of automated testing used to test individual functions or components. They can help identify low-level bugs before an application is released into production. Unit tests should be used to verify that all code works as expected and that new code does not introduce any unexpected behavior. Developers often do this testing during the development process to ensure that their code meets its design requirements.
Unit tests are most valuable when used on small pieces of code, such as functions or classes. Testing smaller units of code help ensure these composable units continue to work as expected before they are integrated into the larger application. Test Driven Development (TDD) is often used at this level, where tests are written first, and code is written to fulfill the test.
Below is a Javascript example function add that accepts and adds two values. Using this example, we can write assertions as unit tests against the add function and test any scenarios that make sense in this context.
// unit of code to test
function add(a, b) {
return a + b;
}
// unit test #1
test("should add 2 postive numbers") {
expect(add(1,2)).toEqual(3);
}
// unit test #2
test("should add 2 negative numbers") {
expect(add(-5,-1)).toEqual(-6);
}Integration Tests
Integration tests are used to test how different parts of a system interact with each other. For example, if a software application includes a user interface, an API, and a database, integration tests can be used to ensure certain parts work together correctly. Integration tests are beneficial when adding new features or changing existing features to verify that related systems are functioning correctly after the changes have been introduced.
Integration tests are most valuable when testing for expectations between two or more systems that communicate with each other. Writing tests against this help ensure that contracts and expectations between systems don’t unexpectedly change. Integration tests are often utilized when testing APIs for available URLs, acceptable parameters, and the response meets specific criteria.
Let’s assume we have an API with three services, Customer, Orders, and Profiles. The Customer and Orders services are accessible individually, but the Profile endpoint joins both services to display the customer data and the most recent ten orders.
// Example service definition
function getCustomer(id: string) {
return db.getCustomer(id);
}
function getRecentOrders(id: string) {
return db.getRecentOrdersByCustomer(id);
}
function getProfile(id: string) {
return {
customer: getCustomer(id),
orders: getRecentOrders(id)
}Let’s assume a Profile API accepts an id at https://www.example.com/profile/:id. Three integration tests could look like the following in a Node environment:
test("/profile/:id should exist") {
supertest(app).get("/profile/" + customer.id)
.expect(200)
}
test("/profile/:id should return the customer by id") {
supertest(app).get("/profile/" + customer.id)
.expect(200)
.then((response) => {
expect(response.body.customer.id).toBe(customer.id);
});
}
test("/profile/:id should return the customer orders") {
supertest(app).get("/profile/" + customer.id)
.expect(200)
.then((response) => {
expect(response.body.orders.length).toBe(10);
});
}These may look similar to a unit test, but during execution, they rely on much more code and services than what’s contained in the example service definition. In our example, we’re testing the router, both services, databases, and any other mapping/filtering/sorting logic contained within each of those services.
Additionally, if the underlying Customer or Orders implementation changes, our Profile integration tests may fail. This failure would be beneficial and allow us to inspect the differences and determine what needs to be addressed instead of introducing a possible regression to the Profile API because of a seemingly unrelated change to the Customer or Orders APIs.
End-to-End (e2e) Tests
End-to-end (e2e) testing tests the entire system from start to finish. It involves running generated scenarios through the system and testing for errors or expected outcomes. This type of regression test isn’t necessarily different from other regression tests, but it covers more significant portions of the system than unit or integration tests. End-to-end regression tests can be beneficial when changes have been made to multiple components within a system; they ensure that all parts are working together as expected.
End-to-end tests are essential for ensuring that an entire system performs as expected. They can catch errors or regression issues from individual components and are excellent for catching integration bugs between different system parts, which might not be detected by unit testing alone.
Let’s assume our Todo application is accessible at https://todo.app.com, and the functionality we want to test is the ability to add todos in our application. Before we start, let’s compare how this might be tested with the previous methods.
With a unit test, we could test the todo title and description parsing before it’s added to the database. With an integration test, we could test that the API saves a todo to the database and retrieves the most current todos. Those methods are valuable but don’t exercise the whole experience of adding a todo, and that’s where end-to-end tests excel.
The following test assumes we’re using a library like Cypress to handle the setup, navigation, and assertions. With it, we could load the application by providing a URL, find the todo input, add and submit two todos, and finally assert they’re displayed in the application.
test('adds todos', () => {
cy.visit('https://todo.app.com')
cy.get('[data-testid="new-todo"]')
.type('write code{enter}')
.type('write tests{enter}')
// confirm the application is showing two items
cy.get('[data-testid="todos"]').should('have.length', 2)
})This test provides an end-to-end test of our application’s routing, UI components, API, and database. With test runners like Cypress, we can easily simulate and inspect our tests.
Other Test Types
What About Functional Tests?
Functional tests test how well a system performs its intended function. These tests can include checking for errors in data entry, verifying functionality across different browsers or devices, or testing user workflows such as signing up for an account or making a purchase on an e-commerce site. Functional tests can be used before releasing a product into production to ensure that it meets its desired specifications and behaves as expected for users. Often, testing an application’s functionality requires you to test a system in its entirety which is why Functional tests are most closely related to end-to-end tests and sometimes are used interchangeably.
Integration Tests vs. e2e Tests
Integration tests are designed to test how two or more different parts of a system interact with each other. Unlike end-to-end tests, integration tests typically do not involve user interaction and instead focus on testing how specific parts of an application interact. This may look like testing API contracts or multiple UI components that rely on each other.
End-to-end testing ensures that all parts of an application or website function as intended when used by real users. End-to-end tests are often performed to ensure that all features behave as expected. Unlike integration tests, end-to-end tests typically do not test isolated components within an application and instead focus on the application or system as a whole. This can provide more surface value but may increase the likelihood of broken tests as the application evolves.
UI Component Tests
UI component tests strike a balance between integration tests and end-to-end tests. Cypress provides a UI component test framework to simplify this kind of testing. Like integration tests, they test one or more components within a smaller development environment. Like end-to-end tests, you can simulate user interactions with fewer dependencies and faster test execution since the entire application isn’t loaded.
What Are Smoke Tests?
Smoke tests, also known as build verification tests, are regression tests that involve a small selection of tests to determine whether the system is stable enough for more thorough testing. These tests are typically used as pre-production checks and can help diagnose any significant issues with a software product before it is released. Smoke tests are often added as the first set of end-to-end tests to ensure a minimum level of functionality exists until developers can devote more time to additional tests.
Why Are Regression Tests Necessary?
Regression tests verify that changes to an existing system do not disrupt its functionality. By running regression tests after making changes to a system, it is possible to detect unintended consequences of the new code and address them before release. Catching unintended changes can be especially important for safety-related products, where regression testing can help catch errors before they result in catastrophic outcomes. Therefore regression tests are not necessarily a unique test type but are used to describe tests that run after each change to a system or codebase, typically before introducing code changes into a branch or application.
Automated tests also serve as documentation by describing how an application is intended to function. As an application evolves, tests can inform new and old contributors of the intended design and functionality and prevent unintended regressions.
Conclusion
Automated regression testing is crucial in software development and maintenance, helping teams build robust, predictable applications while reducing manual effort. We covered three main types of testing. Unit tests are used to test small, often composable, units of code. Integration tests check how different parts of a system interact. End-to-end tests check how well the entire system performs its intended function. All these types of automated testing help create stable and reliable applications.