Writing Great Acceptance Tests

Posted by

A few weeks ago I described the layers of testing that go into building a modern cloud service. This article focuses on the Acceptance Tests layer by looking at what it’s for, how to build it, and how to use it to deliver a quality experience.

What Are Acceptance Tests?

Let’s start by defining what an Acceptance Test is since there is inevitably more than one term and definition for everything in software development. The purpose of Acceptance are:

  • Verify the requirements of the system are satisfied.
  • Ensure that the service is running correctly on a given environment (typically development, staging and production).
Screen Shot 2017-09-05 at 8.17.06 PM
Layers of Testing

As can be seen on the above diagram, they should be the second largest layer of tests after unit tests. Unfortunately, they don’t get nearly the attention that unit tests do and are often after thoughts. This mindset is slowly changing as more organizations adopt Continuous Delivery and Deployment.

The reality of most applications is that they get deployed and then a developer performs some quick sanity checks to ensure that everything is working. It becomes impractical to do manual sanity checks every time a service is deployed as we move towards deploying on a continuous basis. Automated acceptance tests enable us to build faster and safer with the confidence that our service is running as expected.

How To Write Acceptance Tests

So how do you go about building these tests? For starters, make sure the tests live closely alongside the code they are meant to test. This makes the tests easier to find and modify as needed.

If you already have an automated test suite, then it’s a logical extension to build your acceptance tests in the same language. Otherwise, NodeJs is one of the most efficient way to write them because of the low barrier to entry, quick setup and massive adoption of JavaScript in the last few years.

Acceptance tests should test the behaviours that the service takes on. That means testing edge cases and happy paths. You want them to be exhaustive enough so that a succesful run makes you confident that the service is up and running as expected.

These tests need to run on every environment to which you deploy the service. The best way to ensure that happens is to integrate them as part of your deployment pipeline. When a developer commits some code, it gets compiled, unit tested, deployed, and then acceptance tested. Rince and repeat for every environment that the code gets deployed to.

Above all else, the tests need to be reliable. They need to be able to run consistently, over and over with little to no false positives. This may sound simple enough but is really hard to achieve. Network issues, downstream service failures and configuration problems can result in all sorts of false-positive failures. Spend the time needed to fix any brittle tests or you’ll quickly lose the value that they provide.

Acceptance tests should not change once they are stable. It’s possible to add on to them but a change to an existing acceptance test implies a breaking change. At that point, you’ll want to version your service and the tests along with it.

Practical Examples

Let’s use a very basic bank account management API to look at what this would look like. In this fictional scenario, I’m using an HTTP-based service but it could very well be something other than HTTP and the concepts would be the same.

Start by having the tests sit alongside the other tests in the service. This way they are easy to find and hard to forget about.

Root Project Folder
|
tests
--unit
--acceptance
----account
----auth
src
--services
----(supporting services go here)
--controllers
----account
----auth

Within the tests/acceptance/account folder, organize all related tests within a single file using a BDD framework. BDD frameworks logically group tests making them easier to understand.

describe('get account', function () {
    describe('non-existing account', function () {

        it('should return 400 Bad Request', function () {
            let actual = getNonExistingAccount();
            expect(actual.statusCode).to.be.equal(400);
        });
        it('has correct error message', function () {
            let actual = getNonExistingAccount();
            expect(actual.error).to.be.equal('account does not exist');
        });
    });
    describe('existing account', function () {

        it('should return 200 Ok', function () {
            let actual = getExistingAccount();
            expect(actual.statusCode).to.be.equal(200);            
        });

        it('returns the correct amount for the account', function () {
            let actual = getExistingAccount();
            expect(actual.result.amount).to.be.equal(200);
        });
    });
});

These sample tests are very simple. A real test will require a lot more setup and teardown but the underlying concept remains the same.

Writing acceptance tests in this way ensures that your application matches its requirements. It will also give you a reasonable degree of confidence that the system is deployed correctly to every environment it’s running on. And most importantly, it will allow you to build faster and more efficiently.

One comment

  1. I encourage to have a look into the BDD literature as well. It’s full of tricks as to how to write acceptance tests that rely on the business domain. It’s a gateway to increases discussion with domain experts and it makes the tests more stable !

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s