F# for the Cloud worker

In part 2 we started to build a simple script to show information about virtual machines in AWS (EC2 instances). There was also a focus on developer workflow and work integrated with the editor and the F# Interactive REPL.

In this part we are going explore building functions as a service (FaaS) components, more specifically with AWS Lambda. The following text will assume that you are familiar with what AWS Lambda is and have some experience with it. AWS Lambda is “funtion-as-a-service”, i.e. you write some piece of code which can be represented as a function (small or large) - it is triggered by some input and produces some kind of result and/or side-effect. AWS provides some documentation about what AWS Lambda is here.

AWS provides some templates and tools to make deploying .NET stuff as lambda easier and that includes F#. When we start to work with these things we will go beyond simple scripts through and start going into projects. If you do not already use an IDE that provides support for handling projects and solutions I suggest you pick one up, like Jetbrains Rider, Visual Studio or Visual Studio Code.

Set-up for AWS Lambda

As with a lot of functionality, we will use the dotnet command-line tool to set up the templates and components we need. To install the AWS Lambda templates provided for .NET, we simply run the command:

1
dotnet new -i Amazon.Lambda.Templates

See also AWS documentation here. The documentation refers to C#, but the template and tool installation is the same regardless of .NET language used. We can now see what we have available for use with F#:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
~ ❯❯❯ dotnet new --language "F#" --list
Templates                                        Short Name                              Language      Tags                 
-------------------------------------------      ----------------------------------      --------      ---------------------
Lambda Custom Runtime Function                   lambda.CustomRuntimeFunction            F#            AWS/Lambda/Function  
Lambda Detect Image Labels                       lambda.DetectImageLabels                F#            AWS/Lambda/Function  
Lambda Empty Function                            lambda.EmptyFunction                    F#            AWS/Lambda/Function  
Lambda Simple DynamoDB Function                  lambda.DynamoDB                         F#            AWS/Lambda/Function  
Lambda Simple Kinesis Function                   lambda.Kinesis                          F#            AWS/Lambda/Function  
Lambda Simple S3 Function                        lambda.S3                               F#            AWS/Lambda/Function  
Lambda ASP.NET Core Web API                      serverless.AspNetCoreWebAPI             F#            AWS/Lambda/Serverless
Serverless Detect Image Labels                   serverless.DetectImageLabels            F#            AWS/Lambda/Serverless
Lambda Empty Serverless                          serverless.EmptyServerless              F#            AWS/Lambda/Serverless
Lambda Giraffe Web App                           serverless.Giraffe                      F#            AWS/Lambda/Serverless
Serverless Simple S3 Function                    serverless.S3                           F#            AWS/Lambda/Serverless
Step Functions Hello World                       serverless.StepFunctionsHelloWorld      F#            AWS/Lambda/Serverless
Console Application                              console                                 F#            Common/Console       
Class library                                    classlib                                F#            Common/Library       
Unit Test Project                                mstest                                  F#            Test/MSTest          
NUnit 3 Test Project                             nunit                                   F#            Test/NUnit           
NUnit 3 Test Item                                nunit-test                              F#            Test/NUnit           
xUnit Test Project                               xunit                                   F#            Test/xUnit           
ASP.NET Core Empty                               web                                     F#            Web/Empty            
ASP.NET Core Web App (Model-View-Control...      mvc                                     F#            Web/MVC              
ASP.NET Core Web API                             webapi                                  F#            Web/WebAPI           

There are two types of AWS Lambda templates here - those tagged with AWS/Lambda/Function and those tagged with AWS/Lambda/Serverless. The former group are for Lambda functions that do not contain anything else, it is just the function itself. The latter group of templates uses AWS Serverless Application Model (SAM) and those also will have an AWS Cloudformation template which is used during deployment.

Setting up Lambda function with a template

We are going to start with something equivalent to Hello Cloud, just a very simple lambda function to start with. For this, we are starting with just a plain empty AWS Lambda function, calling it HelloLambda.

So we start with creating a directory for HelloLambda and then initiate it with an empty Lambda function template:

HelloLambda init

There are quite a few files created here and it gets a bit more complicated than just a single script file. We will initially look at the four files in the src/HelloLambda directory.

Sorting out dependencies

Function.fs is the file where the actual code resides. Although the template says empty function, it is not empty. It implements a simple function that takes a string input and returns an uppercase version of that string.

This is what the file looks like in VS Code and in Jetbrains Rider, their indications are pretty similar. What the problem is here that the code is dependent on packages which have not been downloaded and installed (from NuGet) so they are available for us to use.

Initial errors in VS Code

Initial errors in Jetbrains Rider

Before going into the code itself, let us sort out these dependency issues.

What is required here is that the state of these packages gets restored, which in effect is that they get downloaded and made available. Fortunately, a log of the project-oriented commands of the dotnet command-line tool will trigger this restore as part of their execution.

So one way to get these references sorted out is to go to the project directory and run the dotnet build command:

Run dotnet build in the project directory

The information of what to restore is picked up from the project file, which is the file with the file extension .fsproj. In our case, it is called HelloLambda.fsproj. This is a project file and a project is essentially a collection of information (source code and other things) that forms some kind of deployable component - an application, a library etc. The project may contain information that describes how to build and optionally also deploy what the project represents.

The file uses XML to describe the build and deployment information. Fortunately, it is not necessary to understand the details of its content for us now and leave that for the future. However, if you wish to dive more into the project file format there is some Microsoft documentation to look at.

If you are working in VS Code with the Ionide plugin for F#, then you can also perform project-level actions by selecting the F# solution explorer view and then right-click on the specific project to get a context menu with actions, see picture below.

Access project actions via VS Code F# solution explorer

Examining the function itself

Let us have a look at what our “empty” function does. Open the file Function.fs and we see this content:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
namespace HelloLambda


open Amazon.Lambda.Core

open System


// Assembly attribute to enable the Lambda function's JSON input to be converted into a .NET class.
[<assembly: LambdaSerializer(typeof<Amazon.Lambda.Serialization.SystemTextJson.DefaultLambdaJsonSerializer>)>]
()


type Function() =
    /// <summary>
    /// A simple function that takes a string and does a ToUpper
    /// </summary>
    /// <param name="input"></param>
    /// <param name="context"></param>
    /// <returns></returns>
    member __.FunctionHandler (input: string) (_: ILambdaContext) =
        match input with
        | null -> String.Empty
        | _ -> input.ToUpper()

The file starts with a namespace declaration of HelloLambda. A namespace in F# is a grouping construct, which can be used to group Types and Modules (not functions). Contents of multiple files can be part of the same namespace, even multiple libraries (assemblies) can be part fo the same namespace.

The namespace declaration is declared at the beginning of the source code file and normally it is used as a top-level declaration (although possible to have nested namespace declarations). Modules are also a grouping construct, which can contain all sorts of F# entities (except namespaces) - values, functions, modules, types.

The next new thing is the attribute, the things that are surrounded by [< and >]. Attributes are a way to add extra information, metadata, to a construct in a program. There are different types of attributes that are possible to use and it is a .NET thing. For F# there is some information about attributes here. In this case, the attribute is of type assembly and as the comment says, helps with translating the JSON structure provided as input to the Lambda function to F# (.NET) classes.

After this, we start to get to the nitty-gritty, the definition of the code itself. AWS Lambda does not separate between F# or C# (or VB.Net) when it comes to invoking Lambda functions, so the declaration needs to be in a way that works for all languages. With C# being the main language for the .NET platform, this also means that the interfaces work with is more suited to the more object-oriented nature of C#.

Thus, the Lambda function is declared as a member function of a class. This is fine to declare in F#, which supports object programming but is functional-first. A class in F# is just another type, but one which has a constructor and possibly member functions.

The way to define a constructor to a type in F# (i.e. a class) is to include a parameter list after the type name surrounded by parenthesis. The parenthesis is mandatory if declaring a class. Then expressions can be written after the equal sign, indented if on separate lines.

Member functions are declared just like regular functions, but with the addition of the member keyword, plus that they need to have a reference to the object itself as the prefix. In other languages, this self-reference may be called this or self. In F# it can essentially be given any name, but if the name to use has not been declared explicitly and is not otherwise used in the code, the traditional name to use has been the double underscore (single underscore also possible from F# 4.7).

So, in this case, the member function is called FunctionHandler and takes two parameters, the input string and a lambda context value. AWS Lambda functions always have an event input and a context value, so this is business as usual for AWS Lambda. The body of the code uses a simple pattern matching on the input string - if the input is null (no value), then return an empty string. If there is an actual string in the input, call the ToUpper function on the string to convert to a string with uppercase letters and return that. And that is it - this is all that is needed.

A note about the comments above the function declaration. Normally a single line comment starts with //, but in this case, it starts with ///. This is to indicate that this is an XML comment. Within XML comments there can be XML elements included that provide code documentation. In our case, there is some documentation about the member function, our lambda function implementation. There is more information about XML documentation in comments here.

Now that we have looked at what the code does, let’s try to deploy it!

Preparing for the deployment of Lambda function

AWS provides a tool that can be used with the dotnet command for deploying Lambda functions. This can be installed in this way:

Install dotnet lambda tool

After installation we can get some idea of what the tool can do by using the --help option for it:

dotnet lambda –help

The sub-command we want to use here is deploy-function. Many options can be specified to this sub-command, but the ones we will need to provide are:

  • The name of the Lambda function, what we want to call it.
  • The AWS profile to use for deploying it (if there is a named profile not called default, or environment variables already set)
  • The region to deploy to (unless the default for the AWS profile is used)
  • The IAM permissions for the Lambda function.

Except for the AWS profile, any missing parameters will be prompted for if they have not been specified already. The basic IAM permissions for Lambda, if the function does need to access any specific AWS services (besides Cloudwatch logs) is named basic_lambda_exec and thus we can deploy the function like this (using AWS profile erik) with the name HelloLambda:

1
dotnet lambda deploy-function --profile erik --region eu-north-1 --function-role basic_lambda_exec HelloLambda

Deploy lambda function with dotnet CLI

Setting default for deployment

While it is certainly can be ok to specify all the settings every time a deployment is made, it may be more convenient to have these settings saved somewhere. This is where the file aws-lambda-tools-defaults.json in the project directory comes into play. This is simply a file where some deployment parameters can be specified and the dotnet lambda command will pick up these settings, if available. By default the content of this file looks like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
{
  "Information": [
    "This file provides default values for the deployment wizard inside Visual Studio and the AWS Lambda commands added to the .NET Core CLI.",
    "To learn more about the Lambda commands with the .NET Core CLI execute the following command at the command line in the project root directory.",
    "dotnet lambda help",
    "All the command-line options for the Lambda command can be specified in this file."
  ],
  "profile": "",
  "region": "",
  "configuration": "Release",
  "framework": "netcoreapp3.1",
  "function-runtime": "dotnetcore3.1",
  "function-memory-size": 256,
  "function-timeout": 30,
  "function-handler": "HelloLambda::HelloLambda.Function::FunctionHandler"
}

It is a pretty simple format and all the command-line options available can be specified as fields in the JSON structure here. So if we enter the same parameters as specified in the previous deploy step, the only thing left to specify is the name of the function in AWS:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
{
  "profile": "erik",
  "region": "eu-north-1",
  "configuration": "Release",
  "framework": "netcoreapp3.1",
  "function-runtime": "dotnetcore3.1",
  "function-memory-size": 256,
  "function-timeout": 10,
  "function-role": "basic_lambda_exec",
  "function-handler": "HelloLambda::HelloLambda.Function::FunctionHandler"
}

and thus our deployment command will simply be:

1
dotnet lambda deploy-function HelloLambda

If you are working on some code which shared among multiple developers, it is probably not a good idea to enter the profile field, since that name may be different for each developer. Only keep the settings that are common regardless of the developer.

Running our Lambda function

After a successful deploy, let us have a look in the AWS Console. The function can be found there, as it should be.

Deployed function in AWS Console

We can use the AWS Console to test the function, byt creating a test event and run it. The result looks as expected.

Test event

Logs from test

However, it is a bit cumbersome to login to the AWS Console to test the function and see the result. Fortunately, we can run this from the dotnet CLI as well:

Invoke a function from dotnet CLI

Yet another option is to use the AWS Toolkit IDE extension (available for Visual Studio, Intellij and VS Code). For example, in the case of VS Code, we can go to extensions and search for the AWS Toolkit and then install it. One installed, we can get access to a number of AWS resource directly from the IDE. This includes invoking the Lambda function and getting a result back.

Install AWS Toolkit extension

Invoke AWS Lambda function

Send input and see the result

Useless Lambda

While it may be enjoyable to test out a simple Lambda function like this as a starting point, it is not particularly useful. If you use a Function-as-a-service, it is rarely a standalone piece - it needs to be triggered by something. In most cases, this is an external event of some kind. It also needs to interact with other services and resources typically. For this, the function needs to be put into a bigger package of components and resources - most likely some other AWS services are involved as well, in case of AWS Lambda.

This may be done via Cloudformation or some other Infrastructure-as-code solution. Since the dotnet lambda and the AWS Toolkit supports AWS SAM (Serverless Application Model), which integrates Cloudformation with deployment of Lambda function, we will have a look at that as well in part 4.

We will also go even further and look at some other tools, which allows describing the AWS resources as F# code also. This includes the AWS Cloud Development Kit and Pulumi.

We also have not looked into the test project which was generated as well and running unit tests in F#. This is also the subject for another blog post later.

Summary

This post was a bit focused on the tooling itself, rather than a lot of F# code. For AWS Lambda there is a bit of a bump to get started and get a somewhat reasonable workflow in place and there are a few options to choose from. This part has focused on the basics using the dotnet CLI and to some extent the AWS Toolkit.

Source code

The source code in the blog posts in this series is posted to this Github repository:

https://github.com/elzcloud/fsharp-for-cloud-worker