Testing Serverless Functions in .NET

Testing has changed a lot in the last fifteen years. It used to be a manual, time-consuming affair that would take hours to complete by dedicated testers every time you wanted to release a build. The only automated tests to be seen were unit tests, and even they were often missing in action.

All that changed once the concept of Continuous Delivery/Deployment became mainstream. The release pipeline, and therefore the testing, had to become completely automated so that deployments could occur once, twice, or even a hundred times a day. And while most projects aren’t yet using a CD pipeline, more and more development teams have adopted some kind of automated test suite to help with validating a release.

Serverless is a great candidate for automated testing, but there aren’t many published best practices as to how to approach it. This is my attempt to shed some light on the strategies that I’ve seen work well.

Terminology

I’d like to start by defining a few terms so that we’re all on the same page. A function (lowercase-f) is the agnostic term I use to refer to a single serverless function, be it an Azure Function or AWS Lambda.

A serverless application, on the other hand, is a grouping of many functions within a single business domain. An order processing application, for example, could be composed of many functions to handle orders from the moment they are added to a cart to the time they are delivered on your doorstep.

The Big Picture

Testing a serverless application has to be approached from multiple angles to ensure that its requirements are met from both a functional and non-functional perspective. To achieve that goal, I like to use multiple layers of testing:

Each layer serves a different purpose but all have the same underlying goal of making sure the application is running to specification. The build process for this type of testing approach looks like follows:

• Build the serverless functions.
• Run the unit tests.
• Deploy the functions to a development environment.
• Run the acceptance and security tests against the deployed functions.
• Deploy the functions to the next environment, typically staging.
• Run the acceptance and security tests against this latest environment.
• Repeat the previous two steps until you’ve deployed all the way to production.

Running the tests in this way ensures that each component works correctly in isolation and that the individual functions play nicely together and form a complete serverless application.

Let’s look at each of the testing layers to understand them better.

Unit Tests

The reasons for unit testing have been written and re-written hundreds of times so it should go without saying that the place to start testing a serverless app is unit testing.

Unit tests are especially valuable when it comes to testing business logic, such as calculating the cost of a cart, or finding the most appropriate seats on a plane for a family of five. There can be many different rules, edge cases and error conditions that are best expressed through unit tests.

Unit tests are less useful when there is little to no business logic. Take the example of a function that receives an image, uploads it to storage, and returns its name. The code will have few execution paths and need only a couple of tests to achieve 100% code coverage. The unit tests are more of a formality than anything else in a case like this.

I won’t show you step-by-step how to write unit tests since you likely already know how to do that. But there is one step that can be taken to make unit testing a function a bit less painful, and that’s to move as much logic as possible out of the function entry point.

A function trigger should do nothing more than receive an event and invoke a service that performs the actual work. Structuring the code like this means that all the business logic lives outside the function trigger. That makes it easier to write and run the unit tests since you aren’t dependent on the function’s runtime to execute the tests.

Helpful Resources

I highly recommend Ben Morris’ article on unit testing and mocking functions based on the Azure Functions V1 triggers. Microsoft have created some helpers to help test V2 Trigger functions as well.

On the AWS Lambda side of things, take a look at Ken Halbert’s sample .NET Core implementation to have an idea of how to write Lambda unit tests.

Acceptance Tests

The strength of unit tests is also their biggest flaw: every component is tested in isolation, so there is no way to know if all the components of the function work together in a cohesive manner. That’s where acceptance tests come in.

Acceptance tests are a twist on integration tests. They run against a deployed serverless application and test the happy and failure paths that the code is expected to produce. The acceptance tests provide immediate feedback on the state of the deployed application. They are a must-have if you’re aiming for Continuous Deployment or Delivery.

Let’s look at some examples by way of a fictional serverless application that accepts airplane telemetry and analyzes the data for potential flaws. This imaginary application is broken into two serverless functions: the first to receive the telemetry via an API, and the second, which is triggered by the first via queue message, to analyze the dataset for that airplane.

Each function is validated separately with its own set of happy and failure path tests. With that in mind, you’d want to write the following tests for the HTTP API function:

• When valid telemetry data it sent / Then it is added to the data store
• When valid telemetry data is sent / Then a message with the airplane’s id is put on a queue to signal that it’s time to analyze the data for that plane
• When invalid telemetry data is sent / Then a 400 Bad Request is sent to the caller
• When a request is made anonymously / Then an 401 Unauthorized is sent to the caller

The tests for the background processing of the airplane’s data would look something like this:

• When no flaws are detected / Then the function completes successfully
• When a potential flaw is detected / Then an email is sent to the maintenance staff
• When the airplane can’t be found in the data store / Then the message goes to the dead-letter queue.

Since we’re hitting real endpoints with these tests, it becomes necessary to have a fake airplane that can be used to put the application through its paces, even on the production environment.

Have a look at the code samples on Microsoft Docs to get a feel for what an acceptance test should look like. You can also see my examples in the deep-dive article that I wrote on acceptance testing a cloud application.

Acceptance tests could very well be the most important layer in testing a serverless application. The next layer, however, is gaining in importance on a daily basis.

Security Tests

No serverless testing suite can be considered complete until there’s a minimum amount of security testing that’s been done. The serverless infrastructure is secured by the cloud provider, but a serverless function’s code is just as vulnerable to attack as any other hosting platform.

Security tests can be seen as a specialization of acceptance tests, seeing as they are run in the same way: the application is first deployed, and then the tests are run against the live application. But testing for security isn’t as easy as writing acceptance tests. It requires out of the box thinking to find and prevent weaknesses.

Going back to the airplane maintenance application, there are some low hanging fruit that can easily be automated as part of the test suite:

• Validating that the input to each telemetry field is of the proper type and length.
• Ensuring that unauthenticated and unauthorized requests are rejected.
• Ensuring that SQL injection is not possible when inserting the telemetry data to the data store.

I encourage you to read my in-depth post on security tests for some code samples of typical security tests.

Security is often left until later because it’s seen as being unnecessary from the start. The problem is that later usually means never, or at least until it’s too late. Writing these tests helps keep you that wee bit safer without costing much more in terms of development time.

What’s Missing?

Back in early 2018, I wrote a series of articles that focused on the process of testing a cloud application. That cloud app was fairly generic and was based on a typical Web API project. That project had a few additional layers, most notably load and performance testing.

I eliminated load testing from the equation since scale is handled out of the box by serverless platforms. Any change in demand automatically adjusts the number of instances running a function. That makes it practically impossible to out-scale the serverless platform.

Performance tests could be needed for serverless functions that are highly dependent on response times, such as HTTP-based serverless APIs. Cold-start times for C# functions aren’t great on AWS or Azure, so having some tests that systematically check the response time of a function could come in handy.

I hope this article has shed some light on testing serverless applications. Putting this down on paper has definitely helped me think of serverless testing differently. Feel free to reach out with any questions or suggestions you might have to improve this testing strategy.