Building a Worker in AWS with .NET Framework and .NET Core

About a week ago I wrote about the different technologies that can be used to build a .NET API on AWS. Many of the technologies that have emerged over the last few years support the .NET Framework, .NET Core, or both, providing us with more options than ever as to how to run our applications.

While AWS has adopted .NET Core as a first-class citizen, giving us a multitude of choice for performing background work, the same can’t be said about the .NET Framework. Workers in the .NET Framework are typically hosted by a Windows Service. This obviously limits the choices to Windows-based technologies. Making matters worse, some technologies that support Windows, like Beanstalk, don’t support worker environments with .NET.

.NET Framework: EC2 or Bust

In the first years that AWS became available, the only solution to run a background process was to have a Windows EC2 instance on which you would install a Windows Service or Console Application. Years later, not much has changed.

There’s nothing wrong with provisioning an EC2 instance to run some background processing. There are probably lots of customers doing that very thing right now. It’s also the option that involves the least amount of operations work. For that reason alone, stick with EC2 unless you have more advanced needs.

Hangfire is a cool option for simple background tasks that run on a schedule or on demand. It runs within an ASP.NET application and can fire off jobs on a worker thread.

A more robust option presented itself in late December 2017. EC2 Container Service (ECS) added support for Windows containers, bringing a whole new dimension to running background processes.

Containers will give you the scalability, resource optimization, and cost effectiveness that a high quality application will need. But don’t underestimate the amount of operations work that needs to be done to set up a production-grade container environment. Even with managed services like ECS and Fargate, there are many hurdles to jump to ensure that the service is rugged enough to be used in production.

.NET Core: Serverless First, Containers Second.

From serverless to containers and everything in between, there’s no lack of options to building a worker with .NET Core. This flexibility has somewhat ironically made it more difficult to create workers via a Windows Service.

Serverless

My personal preference for running short-lived (less than 5 minutes) background processes remains Lambda. The advantages are clear. A worker that has nothing to do waits for requests to come in without incurring any cost. Lambda can also scale to ridiculous levels without batting an eye. Not to mention the operational and deployment model are simple to streamline through the Console or CLI.

The only potential roadblock to using Lambda is that it can’t be triggered via an SQS message. One alternative is to use SNS, which would typically be used for publish/subscribe, but can substitute for SQS in a pinch. Another option is to trigger a Lambda on a schedule to look for messages in the queue. Either one works but hopefully AWS will address this shortcoming at some point.

Containers

Containers are becoming more and more popular and using them to run a background process with .NET Core is a great way to leverage their power. As I mentioned in the .NET Framework section, lots of operations work has to go into building a decent container. For that reason I would only recommend using containers if you’ve already got the infrastructure in place.

Another potential reason to look at containers is for long-running jobs. Lambda’s 5 minute max running time puts limitations on what you can do. Containers give you the flexibility to run a process as long as needed. Many containerized background jobs can also run on the same host, making it a viable and affordable solution for running lots of background processes.