lambdr: Create a Runtime for Serving Containerised R Functions on AWS Lambda

Description

This package provides an R runtime for the AWS Lambda serverless compute service. It is intended to be used to create containers that can run on AWS Lambda. lambdr provides the necessary functionality for handling the various endpoints required for accepting new input and sending responses.

This package is unofficial. Its creators are not affiliated with Amazon Web Services, nor is its content endorsed by Amazon Web Services. Lambda, API Gateway, EventBridge, CloudWatch, and SNS are services of Amazon Web Services.

To see an example of how to use this package to create a runtime, refer to vignette("lambda-runtime-in-container", package = "lambdr").

The default behaviour is to convert the body of the received event from JSON into arguments for the handler function using the jsonlite package. For example, a raw event body of {"number": 9} will be converted to list(number = 9). The handler function will then receive the arguments directly after unlisting, eg. number = 9. This works for direct invocations, as well as situations where the user wishes to implement behaviour specific to a trigger.

Some invocation types have their own logic for converting the event body into an R object. This is useful for say, using an R function in a Lambda behind an API Gateway, so that the R function does not need to deal with the HTML elements of the invocation. The below invocation types have custom logic implemented. Refer to the vignettes or the package website for more information.

Alternatively, user-defined functions can be provided for parsing event content and serialising results. The user can also use the identity function as a deserialiser to pass the raw event content — as a string — to the handler function. Refer to lambda_config for more information.

Direct invocations

REST API Gateway invocations

vignette("api-gateway-invocations", package = "lambdr")

HTML API Gateway invocations

vignette("api-gateway-invocations", package = "lambdr")

EventBridge invocations

vignette("eventbridge-and-sns-invocations", package = "lambdr")

SNS invocations

vignette("eventbridge-and-sns-invocations", package = "lambdr")

Author(s)

Maintainer: David Neuzerling david@neuzerling.com

Other contributors:

• James Goldie me@jamesgoldie.dev [contributor]

See Also

Useful links:

• https://lambdr.mdneuzerling.com/

• https://github.com/mdneuzerling/lambdr

• Report bugs at https://github.com/mdneuzerling/lambdr/issues

Convert an object to JSON

Description

This function effectively wraps toJSON with two hardcoded arguments:

Usage

as_json(x, ...)

Arguments

Details

• auto_unbox is set to TRUE, such that singleton values are not represented as lists.

• NULL values are represented as JSON nulls.

Value

character of class "json"

Convert an R object to stringified JSON matching AWS Lambda conventions

Description

Stringified JSON is a string which can be parsed as a JSON. While a standard JSON interpretation of list(number = 9) would be {"number":9}, a stringified JSON representation would be ⁠"{\"number\":9}"⁠.

This function will convert NULL values to JSON "nulls", to match the convention used by Lambda event inputs, and values are automatically unboxed.

Usage

as_stringified_json(x, ...)

Arguments

Value

character

Examples

as_stringified_json(list(number = 9))
"{\"number\":9}"

Check that the status code shows a success, and error otherwise

Description

Check that the status code shows a success, and error otherwise

Usage

assert_status_code_is_good(status_code)

Arguments

Value

TRUE

Classify an event based on how it is invoked

Description

Events need to be handled differently depending on how the Lambda is invoked. For example, an event via an API Gateway needs to be parsed and handled differently to that of an event received via direct invocation. This function attempts to detect the method of invocation and returns a character vector which can be used to assign an S3 class to the event. The last element of the vector is always "event".

Usage

classify_event(event_content)

Arguments

Value

character vector, the last element of which is always "event"

Define a condition (like an error) with support for HTTP status codes

Description

For more information on conditions and errors see http://adv-r.had.co.nz/Exceptions-Debugging.html. See also conditions.

Usage

condition(subclass, ..., code = 500L, request_id = NULL, call = sys.call(-1))

Arguments

Convert a config to a runtime API if necessary

Description

Endpoint functions need to accept either a config (as created by lambda_config) or a runtime API. This function will accept either and ensure that the runtime API is returned.

Usage

config_or_runtime_api(config)

Arguments

Decode the body of event content coming via an API Gateway

Description

Decode the body of event content coming via an API Gateway

Usage

decode_html_body(body, config, base64_encoded = FALSE)

Arguments

Value

Either a list or, if the body is Base64 and the configuration demands that Base64 values are not decoded, a Base64 value as a character

AWS Lambda endpoints

Description

These endpoints are configured based on the "AWS_LAMBDA_RUNTIME_API" environment variable set by AWS Lambda during initialisation. They generally won't be available locally. The "AWS_LAMBDA_RUNTIME_API" environment variable (accessed through lambda_config) is used in the following functions:

• get_next_invocation_endpoint returns the endpoint which R must query for the next input. R must send a GET request to this endpoint and will wait until either a response is received or the Lambda instance is shut down for inactivity. When Lambda receives an input from, say, an API Gateway, it will respond to the pending request with details of the input.

• get_initialisation_error_endpoint returns the endpoint to which an error should be sent if the error occurs when setting up the runtime. This is distinct from errors that occur during handling of an event.

• get_response_endpoint returns the endpoint to which an event response should be sent. It is unique for each event.

• get_invocation_error_endpoint returns the endpoint to which errors that occur during event handling should be sent. It is unique for each event.

The values returned by get_next_invocation_endpoint and get_initialisation_error_endpoint are unique in each Lambda instance. That is, the runtime only needs to retrieve their values once. The values returned by get_response_endpoint and get_invocation_error_endpoint are determined by the request_id argument that these functions require, and so need to be recalculated for each event. The request ID is given in the "lambda-runtime-aws-request-id" header in the event.

Usage

get_next_invocation_endpoint(config, runtime_api)

get_initialisation_error_endpoint(config, runtime_api)

get_response_endpoint(config, request_id)

get_invocation_error_endpoint(config, request_id)

Arguments

Value

character

Combine class-specific context with general context for an event

Description

The extract_context function dispatches on the class of an event to extract context specific to that class. By default, it returns an empty list. This function takes that class-specific context and combined it with the context that is applicable for all classes, contained in both event headers and environment variables.

Usage

extract_and_augment_context(event, config, ...)

Arguments

Value

list

Event context

Context is metadata associated with each invocation. If the handler function accepts a context argument then it will automatically receive at runtime a named list consisting of these values along with the arguments in the body (if any). For example, a function such as my_func(x, context) will receive the context argument automatically. The context argument must be named (... will not work).

Refer to vignette("lambda-runtime-in-container", package = "lambdr") for details.

Extract the context of a Lambda invocation from the headers of an event

Description

Extract the context of a Lambda invocation from the headers of an event

Usage

extract_context(event, config, ...)

Arguments

Value

list

Event context

Context is metadata associated with each invocation. If the handler function accepts a context argument then it will automatically receive at runtime a named list consisting of these values along with the arguments in the body (if any). For example, a function such as my_func(x, context) will receive the context argument automatically. The context argument must be named (... will not work).

Refer to vignette("lambda-runtime-in-container", package = "lambdr") for details.

Extract context from environment variables

Description

This function is intended to provide ambient configuration that makes up part of the context returned by extract_context. These are the components of the context formed by environment variables. To speed things up a little we call the environment variables only once on runtime initialisation and store them in the config.

Usage

extract_context_from_environment()

Value

list

Extract the headers from a Lambda event

Description

This function is largely equivalent to headers, which it wraps. The only difference is that the names of the headers returned are converted to lower-case (these are meant to be case-insensitive) and the headers are logged at the DEBUG level.

Usage

extract_event_headers(event)

Arguments

Decode a Base64 encoded value to a string

Description

Events coming via an API Gateway can have content with bodies encoded as Base64. This is especially true for HTML API Gateways (as opposed to REST API Gateways).

This function propagates NULLs. That is, from_base64(NULL) returns NULL.

Usage

from_base64(x)

Arguments

Value

character

Examples

from_base64("eyJudW1iZXIiOjd9")

Determine if a function accepts a context argument

Description

The context of a Lambda is the metadata associated with each request, such as the ARN. In other languages, a function used in a Lambda must accept the context as an argument. We allow here for functions that disregard it, since it's not necessary.

The purpose of functions_accepts_context then is to determine if the arguments of the function defined by the handler includes context, in which case we pass the context as an argument whenever the Lambda is invoked. The context argument must be named (... won't be recognised). Primitive functions will always return FALSE.

Usage

function_accepts_context(func)

Arguments

Value

logical

Parse the content of an event and pass it through the handler function

Description

Parse the content of an event and pass it through the handler function

Usage

generate_result(event, config = lambda_config(), deserialiser = NULL)

Arguments

Value

An object of class the same as event. The object contains a result value, and the result_calculated attribute is set to TRUE.

Determine the function referred to by the "_HANDLER" environment variable

Description

This function will try to identify the function referred to by the "_HANDLER" environment variable. This environment variable is configured by AWS Lambda, either through the CMD of the Dockerfile containing the runtime or through the AWS Lambda console (which takes priority). This function also performs some checks, making sure that the environment variable is defined and that it exists in the given environment, and that it's a function.

Usage

get_handler_function_from_env_var(environ)

Arguments

Retrieve a Lambda environment variable if available, and error otherwise

Description

This function is provided to return a specific error if an environment variable is not defined. This is used by lambda_config to ensure that the environment variables that are expected to be defined by AWS are present.

If the environment variable is undefined but a default value is provided, then that default value will be returned. However, the environment variable will always take precedence.

Usage

get_lambda_environment_variable(env_var, default = NULL)

Arguments

Value

character

Process the input of an event, and submit the result to Lambda

Description

If the handler function accepts a named context argument then the Lambda invocation context will be included as an argument. See the section below for more details.

Usage

handle_event(event, config = lambda_config())

Arguments

Invocations via an API Gateway

Events coming from an API Gateway need to be treated a little differently, both in parsing the event content and in posting the results. Refer to vignette("api-gateway-invocations", package = "lambdr") for details.

Event context

Context is metadata associated with each invocation. If the handler function accepts a context argument then it will automatically receive at runtime a named list consisting of these values along with the arguments in the body (if any). For example, a function such as my_func(x, context) will receive the context argument automatically. The context argument must be named (... will not work).

Refer to vignette("lambda-runtime-in-container", package = "lambdr") for details.

Generate a handling function for an invocation error

Description

An error caught during event handling must be handled in a special way. An error message must be logged and posted to the invocation error endpoint, and the the runtime must continue — an invocation error is a problem for the invocation, not the runtime.

The handle_event_error function accepts an event and generates a function. The generated function accepts error caught by tryCatch, logs it, and then submits it to the invocation error endpoint. Importantly it does not stop the kernel — the intention is that the runtime moves onto the next event.

This function may need to be implemented differently depending on the source of an event. As such, handle_event_error is an S3 generic that can dispatch on the event class as returned by classify_event.

tryCatch(
    handle_event(...),
    error = handle_invocation_error(event) # returns a function(e)
 )

Usage

handle_event_error(event, config, ...)

Value

A function that accepts an error e as caught by tryCatch

Prepare a HTML response for a Lambda behind an API Gateway

Description

Lambdas behind API Gateways need to send specially formatted responses that look like this:

{
  "statusCode": 200,
  "headers": {
    "Content-Type": "application/json"
  },
  "isBase64Encoded": false,
  "body": "{\"best_animal\": \"corgi\"}"
}

For basic applications where the handler function is returning a simple result, lambdr will do its best to automatically return a result compatible with API Gateways. It will do this whenever an event is detected as having come via an API Gateway. For most purposes this is sufficient, and allows users to focus on the handler function rather than the specifics of how AWS Lambda works.

For more complicated applications, such as when the Lambda needs to return a specific content type or specific headers, may require a bespoke response. This function will take any R object and format it in style of the above example, allowing for customisation.

When the handler function returns a html_response the formatted result will be returned to the API Gateway without further serialisation.

Usage

html_response(
  body,
  is_base64 = FALSE,
  status_code = 200L,
  content_type = NULL,
  headers = list()
)

Arguments

Value

A stringified JSON response for an API Gateway, with the "already_serialised" attribute marked as TRUE. This will stop serialise_result from attempting to serialise the result again.

Examples

html_response("abc")
html_response("YWJj", is_base64 = TRUE)
html_response("abc", headers = list(x = "a"))
html_response(
  "<html><body>Hello World!</body></html>",
  content_type = "text/html"
)

Determine if a Lambda event is coming from a EventBridge event (CloudWatch events)

Description

See https://docs.aws.amazon.com/lambda/latest/dg/services-cloudwatchevents.html for more information.

Usage

is_eventbridge_event_content(event_content)

Arguments

Value

logical

Determine if a Lambda event is coming via an API Gateway

Description

Determine if a Lambda event is coming via an API Gateway

Usage

is_from_html_api_gateway(event_content)

Arguments

Value

logical

Invocations via an API Gateway

Events coming from an API Gateway need to be treated a little differently, both in parsing the event content and in posting the results. Refer to vignette("api-gateway-invocations", package = "lambdr") for details.

Determine if a Lambda event is coming via a REST API Gateway

Description

Determine if a Lambda event is coming via a REST API Gateway

Usage

is_from_rest_api_gateway(event_content)

Arguments

Value

logical

Invocations via an API Gateway

Events coming from an API Gateway need to be treated a little differently, both in parsing the event content and in posting the results. Refer to vignette("api-gateway-invocations", package = "lambdr") for details.

Determine if a Lambda event is coming from SNS

Description

See https://docs.aws.amazon.com/lambda/latest/dg/with-sns.html for more information.

Usage

is_sns_event_content(event_content)

Arguments

Value

logical

Set up endpoints, variables, and configuration for AWS Lambda

Description

This function provides a configuration object that can be passed to start_lambda. By default it will use the environment variables configured by AWS Lambda and so will often work without arguments.

The most important configuration variable is the handler function which processes invocations of the Lambda. This is configured in any of the three below ways, in order of decreasing priority:

1. configured directly through the AWS Lambda console

2. configured as the CMD argument of the Docker container holding the runtime

3. passed as a value to the handler argument of lambda_config

In the first two options, the handler will be made available to the runtime through the "_HANDLER" environment variable. This function will search for the function in the given environment.

If the handler accepts a context argument then it will receive a list of suitable event context for every invocation. This argument must be named (... will not work), and the configuration may be different for each invocation type. See the section below for more details.

Usage

lambda_config(
  handler = NULL,
  runtime_api = NULL,
  task_root = NULL,
  deserialiser = NULL,
  serialiser = NULL,
  decode_base64 = TRUE,
  environ = parent.frame()
)

Arguments

Details

As a rule of thumb, it takes longer to retrieve a value from an environment variable than it does to retrieve a value from R. This is because retrieving an environment variable requires a system call. Since the environment variables do not change in a Lambda instance, we fetch them once and store them in a configuration object which is passed to the various internal functions.

AWS Lambda variables

The lambda_config function obtains the configuration values for the Lambda runtime configures the R session for Lambda based on environment variables made available by Lambda. The following environment variables are available:

• Lambda Runtime API, available as the "AWS_LAMBDA_RUNTIME_API" environment variable, is the host of the various HTTP endpoints through which the runtime interacts with Lambda.

• Lambda Task Root, available as the "LAMBDA_TASK_ROOT" environment variable, defines the path to the Lambda function code. It isn't used in container environments with a custom runtime, as that runtime is responsible for finding and sourcing the function code. Hence, a missing task root is ignored by this package.

• The handler, available as the "_HANDLER" environment variable, is interpreted by R as the function that is executed when the Lambda is called. This value could be anything, as the interpretation is solely up to the runtime, so requiring it to be a function is a standard imposed by this package.

These handler, runtime_api and task_root arguments to the lambda_config function can also provide values to these configuration options, although the environment variables will always be used if available. While it may be sensible to provide the handler function directly, the other two configuration options are only provided for debugging and testing purposes.

Event context

Context is metadata associated with each invocation. If the handler function accepts a context argument then it will automatically receive at runtime a named list consisting of these values along with the arguments in the body (if any). For example, a function such as my_func(x, context) will receive the context argument automatically. The context argument must be named (... will not work).

Refer to vignette("lambda-runtime-in-container", package = "lambdr") for details.

Give a value the "already_serialised = TRUE" attribute

Description

Give a value the "already_serialised = TRUE" attribute

Usage

mark_as_already_serialised(x)

Arguments

Value

x with "already_serialised" attribute TRUE

Parse the body of the Lambda event

Description

Parse the body of the Lambda event

Usage

parse_event_content(event, config)

Arguments

Value

A list containing the arguments to be passed to the handler function

Invocations via an API Gateway

Events coming from an API Gateway need to be treated a little differently, both in parsing the event content and in posting the results. Refer to vignette("api-gateway-invocations", package = "lambdr") for details.

Parse a JSON, but force a NULL or empty string to be interpreted as an empty list

Description

Since jsonlite::fromJSON(NULL) and jsonlite::fromJSON("") return errors, this function will force a NULL or empty string to be interpreted as list(). Otherwise, the output of this function is identical to fromJSON.

Usage

parse_json_or_empty(json, ...)

Arguments

Value

list

Post an error to an endpoint with the format expected by AWS Lambda

Description

According to the AWS Lambda Developer Guide an error posted to the initialisation or invocation error endpoints must be of content type application/vnd.aws.lambda.error+json and of the format:

{
  "errorMessage": "...",
  "errorType": "...",
  "stackTrace": [],
}

Here the errorMessage and errorType are strings, with the stackTrace a list of strings.

This function accepts an error as caught by tryCatch and posts it to the given endpoint, ensuring that the correct formatting is adhered to. This function does not stop the runtime as in the case of invocation errors it's desirable to continue.

The stacktrace is not currently reported. This functionality is yet to be implemented. See the GitHub issue for more details.

Usage

post_lambda_error(e, endpoint)

Arguments

Post an event with a result to the response endpoint

Description

This function will first serialise the event result according to its class by dispatching through serialise_result.The result of that serialisation is posted as is to the response endpoint; this function will not perform any JSON serialisation, for example.

Usage

post_result(event, config)

Arguments

Invocations via an API Gateway

Events coming from an API Gateway need to be treated a little differently, both in parsing the event content and in posting the results. Refer to vignette("api-gateway-invocations", package = "lambdr") for details.

Convert a list to a single character, preserving names

Description

Convert a list to a single character, preserving names

Usage

prettify_list(x)

Arguments

Value

character

Examples

prettify_list(list(a = 1, b = 2, c = 3))
# "a=1, b=2, c=3"

Serialise a result

Description

This result is posted as is, and so all JSON serialisation, etc. must be performed here.

Usage

serialise_result(event, config)

Arguments

Value

character.

Invocations via an API Gateway

Events coming from an API Gateway need to be treated a little differently, both in parsing the event content and in posting the results. Refer to vignette("api-gateway-invocations", package = "lambdr") for details.

Start the Lambda runtime

Description

This is the main function of the package, responsible for starting the infinite loop of listening for new invocations. It relies on configuration provided to the config argument and produced by the lambda_config function.

Usage

start_lambda(
  config = lambda_config(environ = parent.frame()),
  timeout_seconds = NULL
)

Arguments

Details

See vignette("lambda-runtime-in-container", package = "lambdr") for an example of how to use this function to place an R Lambda Runtime in a container.

This package uses the logger package for logging. Debug log entries can be enabled with logger::log_threshold(logger::DEBUG). This will log additional information such as raw event bodies.

Event context

Context is metadata associated with each invocation. If the handler function accepts a context argument then it will automatically receive at runtime a named list consisting of these values along with the arguments in the body (if any). For example, a function such as my_func(x, context) will receive the context argument automatically. The context argument must be named (... will not work).

Refer to vignette("lambda-runtime-in-container", package = "lambdr") for details.

AWS Lambda variables

The lambda_config function obtains the configuration values for the Lambda runtime configures the R session for Lambda based on environment variables made available by Lambda. The following environment variables are available:

• Lambda Runtime API, available as the "AWS_LAMBDA_RUNTIME_API" environment variable, is the host of the various HTTP endpoints through which the runtime interacts with Lambda.

• Lambda Task Root, available as the "LAMBDA_TASK_ROOT" environment variable, defines the path to the Lambda function code. It isn't used in container environments with a custom runtime, as that runtime is responsible for finding and sourcing the function code. Hence, a missing task root is ignored by this package.

• The handler, available as the "_HANDLER" environment variable, is interpreted by R as the function that is executed when the Lambda is called. This value could be anything, as the interpretation is solely up to the runtime, so requiring it to be a function is a standard imposed by this package.

These handler, runtime_api and task_root arguments to the lambda_config function can also provide values to these configuration options, although the environment variables will always be used if available. While it may be sensible to provide the handler function directly, the other two configuration options are only provided for debugging and testing purposes.

Examples

## Not run: 
# A general usage pattern involves sourcing necessary functions and running
# this `start_lambda` in a `runtime.R` file which is then executed to start
# the runtime. In the following example, the function handler can be set to
# "lambda" either as the container `CMD`, or configured through AWS Lambda.

parity <- function(number) {
  list(parity = if (as.integer(number) %% 2 == 0) "even" else "odd")
}

start_lambda()

# Alternatively, it can be passed as an argument `handler = parity` to
# the lambda configuration. If the handler is configured through other means
# then this will be ignored:

start_lambda(config = lambda_config(handler = parity))

## End(Not run)

Start listening for events, and process them as they come

Description

Start listening for events, and process them as they come

Usage

start_listening(config = lambda_config(), timeout_seconds = NULL)

Arguments

Raise an error with a request ID if known

Description

During decomposition from an invocation into an event, it is possible for an error to occur when a request ID is known but the event hasn't been fully realised yet. In order to handle these, an error must be posted to the invocation error endpoint.

stop_decomposition raises an error with an optional request_id. The error can then be processed by handle_decomposition_error. If a request_id is present then the error can be posted to the invocation error endpoint. Otherwise, the error is simply logged. In either case the error does not stop the kernel, and the runtime can move onto the next event.

handle_decomposition_error accepts a config object, created through the lambda_config function. It returns a function that accepts an error e, which means that handle_decomposition_error(event) can be passed as a value to the tryCatch error argument.

Usage

stop_decomposition(..., request_id = NULL)

handle_decomposition_error(config)

Arguments

Raise an error with an optional HTML status code for API Gateways

Description

This variation of stop can be used to raise an error with a specific error code. This is provided to the API Gateway to return an appropriate response. It had no use outside of invocations via an API Gateway.

If a status code is not provided, a generic "500" internal server error will be used.

Usage

stop_html(..., code = 500L)

Arguments

Examples

## Not run: 
stop_html("Resource doesn't exist", code = 404L)

## End(Not run)

Validate a Lambda config object

Description

This function only verifies that an object has a "lambda_config" S3 class.

Usage

validate_lambda_config(config)

Arguments

Value

TRUE

Wait for and handle event

Description

Combines wait_for_event and handle_event along with

Usage

wait_for_and_handle_event(config)

Arguments

Value

NULL

Query the next invocation endpoint to get the next input

Description

The query will receive a response when an input is queued up. If there is no input waiting, the Lambda instance will be shut down after a period of inactivity.

Usage

wait_for_event(config = lambda_config())

Arguments

Details

The Request ID is unique for each input of a Lambda. It is carried by the "lambda-runtime-aws-request-id" header of the response from the next invocation endpoint (see endpoints).

If an error occurs when extracting the Request ID it is impossible to post it to the invocation error endpoint as that is determined by the Request ID. We log the error and move on.