Acceptance Tests Layer in .NET Core

Posted by

This article continues the testing layers series, covering in detail a proposal on how to test a modern cloud application. The previous post in the series focused on the first and largest of the testing layers: unit tests. The next layer up, acceptance tests, validates the cloud app in a way that is equally as important as unit tests.

These types of tests don’t get nearly the attention that unit tests do and often end up as afterthoughts. But they should be part of any project that wants to achieve Continuous Delivery or Deployment. Without further ado, here’s the what and how of acceptance tests in .NET Core 2.

All code samples below can be found in more detail here.

What Are Acceptance Tests?

We’ll start by defining what an Acceptance Test is because there is inevitably more than one term and definition for everything in software development. You’ve probably seen these types of tests called integration tests. Although they do test the application in an integrated way, the requirements they satisfy go well beyond a normal integration test. They are also commonly known as Contract Tests.

The purpose of Acceptance Tests are to:

  • Verify the requirements of the system are satisfied, typically based on acceptance criteria in an Agile environment.
  • Ensure that the application fulfills those requirements once deployed, whether it be development, staging or production.
  • Ensure that no breaking changes are introduced to the customer-facing contract.
TestingLayersV2
Layers of testing

There should be as many acceptance tests as are needed to fully validate that the deployed cloud application is running as expected.

Most cloud applications today get deployed in a semi-automated way (typically triggered via a tool like Octopus Deploy) followed by a team member performing sanity checks to ensure that everything is working correctly. This approach is slowly changing as more organizations adopt Continuous Delivery and Deployment. Deployments in both these models happen on a highly frequent basis (sometimes even on every commit), so you need a quick, thorough, and efficient way to validate the deployments from a business and operational perspective. Acceptance tests provide you with all this.

In other words, automated acceptance tests enable us to build faster and safer with the confidence that our deployed application is running as expected.

How To Write Acceptance Tests

You can find the full source code for the examples below in the Supermarket application source code, which can be found here.

The first step to writing good acceptance tests is to have them live alongside the production code they are meant to test. To that end, the Supermarket sample splits the production code from the test code via the src and test folders. The test folder is then further subdivided into unit and deployment folders to make the deliniation as clear as possible.

Solution Explorer Acceptance Tests

Acceptance tests should test the behaviours that the cloud application exposes. That means testing happy paths and the most common edge cases. The tests should be exhaustive enough so that a succesful run makes you confident that the code is running as expected.

Let’s look at what an acceptance tests look like. There are a total of 5 tests in the Acceptance folder of the Supermarket.Deployment.Tests project. These tests validate that:

  • A request with missing parameters is not accepted
  • A valid postal code results in a successful response
  • The edge cases of tax calculation are handled correctly

Each test has the same basic structure that is similar to that of a unit test. The test is set up by retrieving an access token on the environment on which it is running. The request is then sent to server and the response is checked for the correct result. The needed details, such as URL and credentials to use, should be provided to the tests via environment variables.

[Theory]
[InlineData("J4Z 2B9")]
[InlineData("H3Y 0J3")]
public async Task GivenPostalCode_ThenHasCorrectResponseItems(string postalCode)
{
    var token = AuthenticationHelper.GetToken(_clientApiUrl);

    var response = await HttpRequestFactory.Post(_clientApiUrl, _checkoutResourceUrlSegment, new { postalCode });

    dynamic responseContent = response.ContentAsDynamic();
    Assert.NotNull(responseContent.preTax);
    Assert.NotNull(responseContent.postTax);
}

And that’s all there is to it. Having these tests run succesfully on every deployment tells us that the application is in an expected state and can act as a “first alert” when something goes wrong with a deployment.

Some of the code in the acceptance tests layer was sourced from elsewhere on the web. I used Tahir Naushad’s HttpClientFactory implementation to perform the API calls and the implementation of token-based authentication in .NET Core is based on a blog post by Anuraj from dotnetthoughts.net. Thanks to both of you!

Using Visual Studio or Code To Write Acceptance Tests

A cloud application should be in a working state by the time you’re writing acceptance tests. It should already be thoroughly unit tested [LINK] and there’s been some manual testing of the application.

You can run acceptance tests locally, albeit with a small caveat: you can’t debug both the tests and the production code because Visual Studio doesn’t support launching the WebApi and running the tests simultaneously. Follow these steps to run the tests locally:

  1. Build the WebAPI project in Visual Studio
  2. Open a command prompt and dotnet run the WebAPI project
  3. Execute the acceptance tests from Visual Studio

Another approach could be to have two separate SLN files at the root of the project. The first for the production code, and the other for the acceptance tests project. This would allow you to have two separate instances of Studio running that let you debug both the acceptance tests and WebApi project simultaneously.

Use test playlists in Visual Studio to configure which tests to run together. You can set up a “Unit Tests” and “Acceptance Tests” playlist which will allow you to easily run or the other set of tests. For those who are Visual Studio Code inclined, I’d suggest creating some tasks that run dotnet test with filter attributes to run the test sets.

The real power of acceptance tests is running them against deployed environments. The image below shows how a CI/CD server executes the tests as the application is deployed across environments, ensuring that everything is running expected from feature development to production.

download

The Big Picture

Acceptance tests need to run on every environment to which you deploy the application. The best way to ensure that happens is to integrate them as part of your CI/CD pipeline. When a developer commits code, it gets compiled, unit tested, deployed, and then acceptance tested. Rince and repeat for every environment that the code gets deployed to.

Above all else, the tests need to be reliable. They need to run consistently, over and over with little to no false positives. This may seem obvious but is really hard to achieve. Network issues, downstream service failures and configuration problems can result in all sorts of false-positives. Spend the time needed to fix any brittle tests or you’ll quickly lose the value that they provide.

Acceptance tests should not change once they are stable. It’s possible to add on to them but a change to an existing acceptance test implies introducing a breaking change. Without even trying, they protect you from breaking downstream clients.

Writing acceptance tests in this way ensures that your application is deployed succesfully, fulfills its requirements, and that you don’t accidently break a contract. Most importantly, it will enable you to build higher quality cloud applications.

One comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s