Fun with Functions – OCI Serverless Part 1: Environment Setup

What you should know: I am currently working for Orbit Cloud Solutions as Cloud Advisor, but any posts on this blog reflect my own views and opinions only.

I’ve been pretty busy lately, doing certifications for OCI and Azure and having to deal with daily business, so there was no post on this blog for quite some time. As you might have guessed, i had as well spent some time digging into the Oracle serverless offering which is based on open source project fnproject.io. So i think i now got some interesting things that i will describe in a few blog posts.

So let us start from the beginning: When talking about serverless, we usually speak of functions as a service, i.e. small applications that are hosted on some kind of runtime that you do not have to think much about. This runtime will take care of spinning up the required instances of your application and stop those instances when no usage is seen. This idea more or less started for the mainstream with AWS Lambda and every major cloud vendor has some kind of service like this in its offering. The nice thing is that you are promised to pay only for the function calls and resources actually used – so no need to pay for some VM idling 95% of the time.

So what are typical use cases for serverless applications? First of all, there are functions that can be executed pretty isolated and that are not called with a regular pattern, think microservice. So that could be your tiny app that that you still do not know the usage pattern of. Another good use for serverless functions is in the field of cloud automation. My example discussed in this post further below will be a simple function that generates thumbnails from images uploaded to an object storage bucket.

In case of Oracle OCI it is Oracle Functions which more or less is a subset of fnproject.io running on Oracle infrastructure. Now what is it from a technical point of view? With fnproject.io you get a simple server that will queue requests and dispatch them to docker images running your applications. The server handles as well the management of runtime slots that will be filled by the instances of those docker images. If there are more instances needed and still slots free, it will spin up another instance. And if there is no need for keeping an instance alive or the slot is needed for another application image, the instance is shut down.

So far nothing magical, you will have to deal with the same issues you get when running applications on top of docker.

Automation with Functions and Events

To make it easier to get an understanding how serverless is working in OCI, i will discuss some of my learnings in this post. The goal is to show a simple serverless function implemented in Go that gets triggered by a cloud event and generates thumbnails from images uploaded to a object storage bucket.

This post will discuss the setup in OCI up to invoking a dummy hello-world function. Another post will cover the implementation of the function in Go.

In the infrastructure directory you will find a template env.sh for preapring the environment to run the terraform scripts.

export TF_VAR_oci_tenancy_ocid="ocid1.tenancy.oc1.. "
export TF_VAR_oci_user_ocid="ocid1.user.oc1.. "
export TF_VAR_oci_compartment_ocid="ocid1.compartment.oc1.."
export TF_VAR_oci_fingerprint=...

export TF_VAR_ssh_public_key=$(cat ~/.ssh/id_rsa.pub)

export TF_VAR_oci_region=eu-frankfurt-1

export TF_VAR_oci_cidr_vcn="10.0.0.0/16"
export TF_VAR_oci_cidr_private_subnet="10.0.1.0/24"
export TF_VAR_oci_cidr_public_subnet="10.0.2.0/24"

Edit the file to match your environment and simply source the script.

. ./env.sh

The terraform scripts provided will do the whole OCI setup described in the next section for you.

As you will be working with OCI, you as well should have a working setup ready. Make sure that your OCI CLI setup is fine and you can connect to your tenancy.

Preparing the Environment for Functions

When starting with Oracle Functions, you first have to set up the environment. In the beginning you either should have tenancy admin rights yourself or have access to someone who will do the setup of some parts for you.

Oracle Faas Policy

For reasons i really do not understand, Oracle requires you to manually create a policy on tenancy level to be able to run Oracle Functions.

allow service FaaS to read repos in tenancy,
allow service FaaS to use virtual-network-family in tenancy,

My terraform code will create such a policy for you, just think of removing that part if you already have created such a policy yourself.

Users and user groups

The next thing i recommend to do is to create a user group for developers and admins that will be working with functions and attach a policy having relevant rights. Unlike my example code, always use a personalized user account so the attribution of actions can be done more easily. Some kind of best practice, you know.

allow group fn-usr-grp to manage repos in tenancy
allow group fn-usr-grp to use virtual-network-family in tenancy
allow group fn-usr-grp to manage functions-family in compartment id ${var.oci_compartment_ocid}
allow group fn-usr-grp to read metrics in compartment id ${var.oci_compartment_ocid}
allow group fn-usr-grp to read objectstorage-namespaces in compartment id ${var.oci_compartment_ocid}
allow group fn-usr-grp to use cloud-shell in compartment id ${var.oci_compartment_ocid}

Dynamic Groups

As we do not want to store any credentials (i.e. keys, passwords, …) in a serverless function docker image, we will use resource principals instead. This will allow us to provide the rights needed to a function via standard IAM mechanisms, i.e. policies.

For this to work we create a dynamic group that will match all functions in our compartment.

ALL {resource.type = 'fnfunc', resource.compartment.id = '${var.oci_compartment_ocid}'}"

For this dynamic group a policy is added containing rights on that compartment.

allow dynamic-group fn-dyn-grp to manage object-family in compartment id ${var.oci_compartment_ocid}
allow dynamic-group fn-dyn-grp to use all-resources in compartment id ${var.oci_compartment_ocid}

This allows the function to perform all tasks on object storage and use of all other resources in that compartment. If you want to work with such broad definitions of rights, you will want to make sure that you do not share the compartment with other non-related applications.

Network

For the function we will need some VCN and subnet with internet access (internet gateway or NAT). Setting up those components is pretty straightforward. The terraform scripts will take care of creating a VCN app-vcn and the two subnets privsubnet and pubsubnet. As well a Internet Gateway is added.

Creating the Oracle Functions Application

In OCI, serverless functions are organized as applications that provide as container that will hold common configuration variables, the definition of log mechanisms and the network settings. As well, using different applications will guarantee the isolation of runtimes between the functions contained in the different applications – so one app will not hurt the other app if something goes terribly wrong.

With terraform the app automation-app is created and attached to the two subnets privsubnet and pubsubnet.

fn CLI

One important thing you still need is the fnproject CLI. Just follow the instructions in the Oracle cloud documentation to get it running. When you are done, you always can do the basic check if the cli is installed:

fn version

Further fn cli setup requires some steps described in the documentation. To make life easier for me, i created a shell script fn_context.sh for these steps and put it in the functions directory. Parts of the configuration we already know from the terraform setup is written into a script oci_env.sh, so that only minimal adaptions to fn_context.sh is needed.

So if you are using a different OCI config profile than DEFAULT, you can change that in fn_context.sh. And if you do not want to call your OCIR repo to be named fn-repo, this is the place to change it as well. Once you are done, simply run the script from the scripts directory.

. ./fn_context.sh

The next thing that needs to be done is to log in to your OCI docker repository. Login is done with an OCI auth token. You can generate one yourself in OCI console or cli or simply use my script fn_docker_login.sh. This will generate an auth token, print it out and then try to do a docker login – simply copy the auth token from the screen and paste it on the prompt and you should be good to go.

Note: There is a limit of 2 auth tokens per user in OCI. If you exceed this limit when trying to generate a new token with the script, you either need to use one of your previously generated tokens or remove one of them.

Setting up the Development Environment.

One thing that makes development for Oracle Functions a bit hard is that there is only bare bones development support. In a real life environment, so one that will not allow all aspects to be easily tested by piping some input to the function from the command line, you will have a rather painful time debugging your function. What you absolutely will need, is access to some log target to have OCI send logs to. Don’t even try to use the logs sent to object storage for development and debugging – the latency between executing the function and logs arriving in object storage will drive you insane.

I use papertrail (https://papertrailapp.com/) and am pretty happy with their offering. And the log volume available with their free account seems to be perfectly sufficient for the limited development tasks i am using it for. And if you are working with Visual Studio Code, i as well wrote a minimal papertrail integration extension that will stream the logs to the output console. Get it from https://marketplace.visualstudio.com/items?itemName=maxian-io.papertrail

When you got your log target set up, you now can enable sending logs from OCI functions to that location. In contrast to what the docs tell you, use tls. So the target would be for example tcp+tls://logs99.papertrailapp.com:12345 . You honestly do not want your debugging information flying around without any encryption whatsoever.

If you decide to go for a syslog service, you can add the target url to the app just like this:

fn update app automation-app --syslog-url tcp+tls://logs99.papertrailapp.com:12345

Deploying the hello-world Function

The final step in this part of my posts on Oracle Functions will be to create the dummy hello-world function that comes with the fnproject cli.

Create a new directory, then run the following commands in this directory to generate the dummy function and deploy it to the automation-app that has been created before with terraform.

fn init -runtime go
fn -v deploy -app automation-app

Then just invoke the function to check if everything works.

This concludes the first part. In the next post i will discuss the function to generate thumbnails and hook it up to events emitted by an object storage bucket.