Cloud-Native Design Techniques - Scale

Designing systems for the cloud, or to be cloud-native, is all the rage right now. There are numerous articles written about this, but most of them come across like partial cooking instructions where the reader is expected to know the recipe already. I want to talk about cloud-native design, but not make any assumptions about prior knowledge.

For the past 14 years, I've worked on a system that processes electronic payment transactions. When we built this system, we specifically designed the applications to handle scale. We were betting the business on being able to scale the software linearly with business growth. What exactly does this mean and how is it done?

First, let's define what it means for an application to "scale." In the simplest terms, an application is able to "scale" when it can handle more requests or load without changing the application itself, without failing, and generally without significant changes to the infrastructure that executes it. Let's take for example an order handling system. The code below loosely defines this system:

We can see in the code that three activities occur for a new order: 1) the order is validated, 2) the order is billed, and 3) the order is shipped. In this example, the order validation must occur for the other two activities to happen, but not all of the activities are chained together. We can see that the process of billing an order interacts with a database. Database interactions generally require requests over a network and can be constrained by the resources available to the database server. This operation could be slow or contentious. If orders occur infrequently, this approach is fine and will work well enough. If orders are streaming in rapidly, this approach will either cause significant delays in processing orders, or it will crash the entire system as resources get overwhelmed.

There are three ways that we could easily modify this system to handle scale. The first approach would be to upgrade the server it runs on, or to run it on multiple servers. A hardware-based approach is the easiest for developers because they don't really have to do anything, but it may not be the best long-term solution. Upgrading the existing server may allow for more orders to be handled for a while, but the system will fail again as the number of orders increases. Running this system on multiple servers may work, or it may cause more contention in shared resources, like a database, that causes the system to fail even sooner. If this application is running on a virtual machine in a public cloud, modifying the underlying hardware is an expensive approach to making it scale.

A second approach would be to run the HandleOrder method in a new thread for each order. This approach isn't terrible, but it isn't as easy as it sounds. Simply creating a new thread for each order may work, but that assumes that everything in our HandleOrder method is thread-safe. Most database operations are not thread-safe, so the order billing operation will be an issue. In my experience, a lot of code is only thread-safe by accident, not because it was intentionally written that way. Using threading to scale this application may be a valid approach, but it requires diligence and understanding to get it right. The other limitation to this approach is that at some point, the resources of the server will be exhausted and the app will no longer scale. When this happens, a hardware approach will be necessary.

The third approach to making this system scale would be to break up the operations and handle them independently. We see that the order validation is the only required step for billing or shipping an order. The system can be broken up into three pieces, two of which can be called independently. The code below illustrates how the separation could occur:

The code above separates the billing and shipping calls and allows them to run simultaneously since they aren't dependent on each other. (This code is overly simplistic to illustrate my point.) Breaking the system into constituent parts allows us to scale the BillOrder function separately from the ShipOrder function. As our load of orders grows, we can run the BillOrder function separately from the ShipOrder function, using the HandleOrder function as an orchestrator for the work. In a public cloud environment, this approach would allow us to run the billing and shipping functions as either serverless functions or independent Docker containers. (Either serverless functions or independent containers are cheaper options than running full virtual machines in public cloud environments.)

Designing for scale requires an architect to see the divisions in an application and to separate the system along these divisions. Once that separation is properly done, the system is better prepared to take advantage of cloud techologies to handle scale in a more cost-effective manner.

F# and Azure Functions

Lately I've been creating tools for my wife to use with her classroom to help with distance learning. I've implemented these tools using Azure Functions. Azure Functions are the serverless computing option on the Azure platform. I picked these because I'm familiar with Azure and they are a quick, cheap option for standing up an API.

Another reason I picked Azure Functions is because they support creating functions in F#. I have been able to create a function project and function in F#, and at times run it locally, but I have struggled to get an F# function working in the cloud. I assumed that this issue was a combination of spotty support for F# in some areas, and my lack of understanding about how to get F# to play nicely as an Azure Function.

I have a new project I want to start as a series of Azure Functions, so I wanted to give F# another try. In my reading, I found this step-by-step guide by Luis Quintanilla for manually creating Azure Functions in F#. Luis does a great job walking the reader through creating a function in F#, without relying on the built-in templates provided by Microsoft. I found this guide fantastic because it gave me a better understanding of the composition of an Azure Function, AND it got me closer to successfully deploying an F# function.

The guide isn't perfect. It's missing some steps around adding necessary packages to the project that are referenced in the function itself (easy to do). This "gap" helped me discover some version discrepancies between by installed version of the .NET Core SDK and the Azure Function CLI tool. At first it appeared to be an issue with my project templates being out of date, but updating the templates didn't seem to fix anything. Ultimately, I updated my version of the .NET Core SDK to the latest version, and I upgraded my version of the Azure Function CLI.

At this point, my function that I created by following the guide worked locally and in the cloud! I went back and re-created my function using the command-line tools, and that worked as well. Using version 3.1.300 of the .NET Core SDK and version 3.0.2534 of the Azure CLI tools (which are targetting the Azure Function Runtime version 3.0.133353.0), the following steps will successfully create an Azure Function in F# that will work in the cloud:

mkdir HelloWorld
cd HelloWorld
dotnet new func --language F#
dotnet new http --language F# --name PrintHello
dotnet build
func start

The logging displayed in the console from the func start command will provide the URL for the local function. Pointing your browser to that URL will activate the locally running function.

I'm now ready to start creating larger applications in F#!

F# Records

My previous post F# - Refactoring to Functional - Part 1 garnered some attention for my use of the struct data type.

Tobias Burger pointed me at the correct implementation of F# records for my data passing. He suggested using ordinary records as my data type instead of structs since they are a more idiomatic approach to F#. It's taken me a little while to get back to the code, but I modified my data types to be records. The code is below:

I have to say "Thank you" to Tobias. His criticism helped me better understand F# records, and it made my code shorter and cleaner!

Microsoft's Project Tye

I just read through a blog post about Microsoft's Project Tye. This is an experimental project that helps developers run multiple apps under a single command. It also has some service discovery elements and a deployment capability for Kubernetes.

I'm putting this link here so that I can find it again later. The ability to coordinate the running of multiple projects (either in a microservices world or just with a frontend and backend components) is compelling enough. The service discovery and Kubernetes deployment capabilities make this really worth understanding.

F# - Refactoring to a Functional Design - Part 1

Shortly after the coronavirus pandemic started, I started trying to create web-based tools for my wife (who is a preschool teacher) to evaluate the progress her students are making with distance learning. I reached for Azure Functions first since they are quick for implementing a publicly visible API. As I created and worked with this new API, I needed a way to test the end points to make sure they worked the way I needed them to work. I decided to create a tool in F# to do this testing.

(I know tools like Postman exist and are great. As I was working on the application in question, I was working on a brand new laptop, and I didn't want to go nuts installing software. Also, as much as I like Postman, it has a few drawbacks. The most notable two for me are the fact that changes to tests don't appear in a text file until the user exports the tests. This disconnect results in me forgetting to update tests from Postman with my source code. The other drawback is that additional third-party tools are needed to run Postman tests from some popular CI/CD tools. I recently ran into an issue with one of these tools where the wrapper for the CI/CD pipeline was incompatible with an NPM module needed to run Postman tests. This incompatibility forced me to remove the tests from my build pipeline. That shouldn't happen.)

My first pass at the code in F# is like what I have below: This code is alright, but you can see that there is a lot of duplication for each endpoint to call. This code is also more difficult to read and maintain. I would like an easy way to define an API endpoint to exercise, and just have it tested.

The first refactoring I will do is to remove the duplication of the HTTP call. In the code above, the HTTP calls are not terribly onerous. These calls are a single line of code for each endpoint, and that's not bad. What I don't like about this code is that the HTTP dependencies are mixed with the rest of my tests, and I have to include the HTTP method calls with my definition of the end point to test.

The first step for me was to create an abstraction for the endpoint itself. I want to use an F# type for the endpoint, but I don't want this type to be a class or to contain any logic. To achieve this purpose, I used the F# struct type. This element allows me to define value fields in my type, along with a constructor. My Endpoint struct is below: This "type" gives me a Name so that I can identify my test externally, a URL to hit, the HTTP verb to use in the call, and a "body" for sending in data when necessary. (For this post, I won't be using the Body property.)

Now that I have an endpoint abstraction in place, I need a method to make my HTTP call. Since this is a single line of code in my original file, it should be a pretty short and straightforward method. The method looks like:

let GetEndpoint (endPoint: Endpoint) =
    Http.Request(endPoint.Url, httpMethod = endPoint.Verb)

This method takes in an endpoint. I wanted to ensure that this method stuck to my endpoint abstraction, so I specified a type with the ": type" syntax. Since every line of code in F# is an expression, this method returns the result of the Http.Request method, which is an HttpResponse object.

With my endpoint abstraction and my function to make HTTP calls, I can modify my original testing code to look like this:

This code doesn't look a lot better than the original code, but it is a little easier to see the endpoints that are being tested. This code also shows me that I still have a good bit of "boilerplate" stuff with my testing of the response data. The testing code will be the subject of my next post.

Quick Intro to Azure Functions

I've been working on a small web application for my wife to help her evaluate her preschool students. I created this application as a single page application (SPA) and backing APIs. For the SPA, I used VueJS. For the backing API, I used Azure Functions.

Azure Functions are Microsoft's serverless computing offering. They allow a developer to create functionality based on several triggers. Triggers can be HTTP, queues, Azure Storage, and so on. One of the most commonly used triggers is HTTP. A function based on an HTTP trigger fires for a web request. I think about this like a Model View Controller (MVC) application with the models and controllers and none of the boilerplate code that goes with a full API application.

An Azure Function application consists of a minimum set of files. Those files are:

host.json
local.settings.json
FunctionApp.csproj
Function.cs

Where "FunctionApp" and "Function" are the names you give your Azure Function.

The host.json file contains runtime information, including the version of Azure Functions being used and things like logging configurations. The local.settings.json file contains application-specific settings including connection strings. The CSPROJ file is a fairly typical C# project file that specifies any package dependencies and things like that. The CS file is the "function" itself. It looks like a typcial static C# class except for an annotation and a specific method signature.

The "function" that serves as a trigger for your function app is annotated with [FunctionName("MyFunction")]. This annotation specifies the publicly visible name that your function will have. The function signature is specific to the type of trigger for your function. Since I used an HTTP trigger, my function signature looked like:

public static async Task Run([HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)] HttpRequest req, ILogger log)

This signature tells the runtime that I am using an HTTP trigger that responds to HTTP GET and POST requests. I don't specify a specific route. The HttpRequest and a default logger are supplied using the dependency injection that is baked into the service.

When you generate your first function, a default "Hello World" app is created. This application illustrates how to work with the HttpRequest object passed into the function. It also shows how to create the IActionResult to pass output from the function.

For my specific Azure Function app, I put just enough code into the function itself to handle the request and return a response. I created service and repository classes for handling the business logic and data movement for the calls. This approach worked very well for me!

This approach to API development allowed me to get something created and deployed very quickly. I was able to utilize some basic tooling from Microsoft to build, test, and deploy my API very quickly.

F# Active Patterns

My company deals with file parsing quite a bit, and I've been playing around with F# to see if I could write a better file parsing library using it instead of what we currently use in the C# world. This experimentation led me to F# Active Patterns.

F#, along with other languages, has a concept of Pattern Matching. Pattern matching allows a programmer to transform data by matching patterns in the shape of the data automatically, without a lot of if-then-else branching logic.

Pattern matching, using the match keyword, functions a lot like a switch statement in C#. A typical pattern matching statement in F# would look something like:

let filter1 x =
    match x
    | 1 -> printfn "The value is 1."
    | _ -> printfn "The value is not 1."

filter1 5.   // The value is not 1.
filter1 1.   // The value is 1.

This code defines a function, filter1, that takes an integer. It checks to see if the integer is a constant value of 1. If it is, it prints a message telling us it is 1. If it isn't, it tells us the value is not 1. This code illustrates a constant pattern, and their are many additional options. The other options are described here.

The example above, and the other basic pattern matching, is great, but not enough for some of my file parsing needs. Many of the files that we need to parse are fixed-width ASCII text files. Parsing the data requires defining fields in a particular order with a specified size. This aspect is unavoidable, but I wanted to find an elegant, way to approach this problem.

This is where I found Active Patterns. Active Patterns allow you to define a named pattern and apply it in a match statement. This pattern will match and parse the data if defined correctly. It is also the mechanism for matching with regular expressions, which lend themselves well to my file parsing problem. An Active Pattern looks like this:

let (|EmailMatchActivePattern|_|) input =
    let m = Regex.match(input, "(.*)@(.*)")
    if m.Success then Some m.Groups.[2].Value else None

This construct creates an Active Pattern named EmailMatchActivePattern and it takes in input to work against. Inside the definition, I'm using a regex for a basic email pattern (.*)@(.*). If the regex matches successfully, I return the second grouping. (The indexing starts at 0, but the 0 index contains the entire string that was tested with the regex. The following indices contain the matched classes from the regex.)

Now that I have a named pattern defined, I can use it in a normal pattern matching scenario to match a line from a file and parse the data in that line. I do that with code like:

let parseLine line =
    match line with
    | EmailMatchActivePattern (domainName) -> printfn "This is the domain in the email %s" domainName
    | _ -> printfn "Not an email addres."

This code parses a line of text to find an email address. If the line matches, it parses the domain portion of the email address into domainName. I can then reference domainName in the subsequent function call (a print statement). This code matches and parses the data at the same time.

I took this concept further to explore the file parsing that I described at the beginning. In this exploration, I read lines from a file, match the first two characters of the line to determine a record type, and then parse the line into tuples with individual values. This construct allowed me to define my file format as a couple of predefined regular expressions and to match the records and parse them at the same time. The source code for this is below.

Active Patterns is an F# construct that can be used to dynamically match and parse data. This is a powerful construct that can be used to build tools to parse files and other data structures with a minimal amount of code.