Testing Locally with Lambda Managed Instances

AWS has just launched the new Lambda Managed Instances feature, and at LocalStack we’ve collaborated with the AWS Lambda team to provide local testing support. This new feature gives flexibility on how compute capacity is allocated, allowing you to run functions on your own EC2 instances. Defining a Capacity Provider lets you specify the quantity and class of EC2 instances to use when invoking Lambda functions; only those servers will be used.

Similar to the ECS Managed Instances feature launched earlier this year, Lambda Managed Instances gives you much tighter control over where your Lambda functions are executed. Given that EC2 prices are very predictable, this makes Lambda costs easier to control. You can reduce your costs with EC2 Reserved Instances, and can request specific instance types to enhance your Lambda functions. A significant benefit for serious Lambda users!

LocalStack provides local testing support for Lambda Managed Instances. We support all of the new APIs, as well as the CloudFormation and Terraform resource types. All the same configuration is therefore possible on your local machine, without needing to modify your infrastructure as code (IaC). However, no emulated EC2 instances will be created inside LocalStack, since your functions will continue to run inside containers on your local machine.

Using Lambda Managed Instances

At a high level, you should follow these steps to execute a Lambda function using Lambda Managed Instances:

1. Add a new Capacity Provider resource to specify what type of EC2 instances to use, how many you need, and in which VPC they should be deployed. You can define scaling parameters to specify the maximum number of vCPUs, and declare a metric to trigger auto-scaling.
2. Define your Lambda Function in the usual way, but add the Capacity Provider as a parameter to CreateFunction.
3. Publish a version of the function, using PublishVersion. This informs the Lambda service to deploy that specific version of your function onto the available EC2 instances.
4. Invoke the Lambda function in the usual way, and it’ll be executed on your EC2 instances.
5. Unlike regular Lambda functions, there’s a fair amount of clean-up required to decommission the Capacity Provider.

We’ll now walk through these steps in detail, using a CloudFormation-based IaC template.

Step 1 - Define the Capacity Provider

The first step is to define the CapacityProvider resource in your CloudFormation template to optionally specify what type of EC2 instances to deploy (size and architecture), which VPC they’ll be attached to, and what the auto-scaling characteristics should be. For Terraform users, there’s an equivalent Terraform resource type.

ExampleCapacityProvider:
  Type: AWS::Lambda::CapacityProvider
  Properties:
    CapacityProviderName: "ExampleCapacityProvider"
    InstanceRequirements:
      Architectures:
        - x86_64
      AllowedInstanceTypes:
        - m5.large
    PermissionsConfig:
      CapacityProviderOperatorRoleArn: !GetAtt ExampleOperatorRole.Arn
    VpcConfig:
      SubnetIds:
        - !Ref ExampleSubnetA
        - !Ref ExampleSubnetB
      SecurityGroupIds: []
    CapacityProviderScalingConfig:
      MaxVCpuCount: 50
      ScalingMode: "Auto"

Most of this configuration is self-explanatory, except for the new Operator Role referenced in the CapacityProviderOperatorRoleArn field. This new IAM role is required for the Lambda service to create, manage, and delete your EC2 instances on your behalf. You should refer to the AWS documentation for exact details of what to include in this role.

As we’ll discuss later, LocalStack uses containers on your local machine to run Lambda functions, so no EC2 instances are actually deployed.

Step 2 - Define the Lambda Function

The next step is to specify the Capacity Provider when defining the Lambda function:

ExampleFunction:
  Type: AWS::Lambda::Function
  Properties:
    FunctionName: "ExampleFunction"
    Code:
      ZipFile: |
        exports.lambdaHandler = async (event) => {
            return {
                statusCode: 200,
                body: JSON.stringify('Hello, World!!!'),
            };
        };
    Handler: index.lambdaHandler
    Runtime: nodejs22.x
    Role: !GetAtt ExampleFunctionExecutionRole.Arn
    CapacityProviderConfig:
      LambdaManagedInstancesCapacityProviderConfig:
        CapacityProviderArn: !GetAtt ExampleCapacityProvider.Arn
        PerExecutionEnvironmentMaxConcurrency: 64
        ExecutionEnvironmentMemoryGiBPerVCpu: 2

The new CapacityProviderConfig section specifies the Capacity Provider to use when invoking this function, along with a couple of scaling parameters. The most interesting is the newly-introduced concept of multi-concurrency (see PerExecutionEnvironmentMaxConcurrency), providing for higher levels of scalability. By packing more Lambda invocations onto your EC2 instances, the overall cost per Lambda execution is lower.

Until now there’s been a guarantee that only one Lambda invocation will run inside the environment at any given time, although the environment can be reused for future invocations. With the new multi-concurrency feature, you gain much higher levels of scalability but with the trade-off of requiring your Lambda code to be thread-safe. This might involve rewriting parts of your Lambda function code!

Step 3 - Publish a new version

With Lambda Managed Instances, you must publish a new version of your function to specify exactly what should be deployed to your EC2 instances. If you use CloudFormation this will be done automatically for you, but to publish a version explicitly use:

$ aws lambda publish-version --function-name ExampleFunction

If you’re deploying to real AWS, launching the underlying EC2 instances will take a few minutes, so plan on a delay before invoking your Lambda function. When developing locally using LocalStack, everything is executed inside a container, so you won’t experience a delay.

Step 4 - Invoke the Lambda function

Invoking a Lambda function is done in the usual way, although with the restriction that the $LATEST qualifier can’t be used. This ensures the function code has already been deployed to the Capacity Provider, and the invocation will be immediate

$ aws lambda invoke --function-name ExampleFunction --qualifier 1 output.txt

Step 5 - Clean up

Cleaning up resources involves additional steps, because it’s necessary to delete all the function versions (so they’re no longer deployed to the Capacity Provider), then delete the Capacity Provider itself. The recommended approach is to simply delete the CloudFormation stack, but use the following steps if you need to do it manually:

$ aws lambda delete-function --function-name ExampleFunction --qualifier 1
 ... repeat for other versions ...
$ aws lambda delete-capacity-provider \
     --capacity-provider-name ExampleCapacityProvider

As you’d expect, deleting the Capacity Provider takes a few minutes while the EC2 instances are decommissioned. When developing locally with LocalStack, the deletion is immediate.

Limitations of Local Emulation

LocalStack provides support for testing your application locally, before deploying to the AWS cloud. The focus is on correctness, but you won’t experience the same scalability or security as with the real cloud. The following aspects of this feature aren’t currently supported by LocalStack.

• No EC2 instances are created - Unlike the AWS cloud, LocalStack won’t create any emulated EC2 instances when you specify a capacity provider. We continue to run Lambda functions inside containers running on your local machine. This means that querying the EC2 service won’t show any new EC2 instances, and your Lambda functions won’t run within the specified VPC. We have not implemented these features, as we don’t believe they’re useful for local testing.
• Scaling configuration - Any parameters you specify to set the sizing/scaling parameters are stored by LocalStack, and can be retrieved by querying APIs. However, they have no effect in scaling the number of EC2 instances (there are none!). We simply run Lambda functions inside containers on your local machine.
• IAM Permissions are not enforced - A Capacity Provider is configured with an Operator Role, but the permissions are not enforced by LocalStack. It’s important you check the AWS documentation to ensure you’ve configured this role correctly.
• Multi-concurrency - As mentioned earlier, one of the key benefits of Lambda Managed Instances is enabling the multi-concurrency execution model, allowing for multiple function executions to be running within the same Lambda environment. This moves away from the one-thread-per-lambda assumption, to instead requiring that code be thread-safe. We plan to support multi-concurrency at a later time.

Summary

LocalStack has collaborated with the AWS Lambda team to provide local testing support for the new Lambda Managed Instances feature. If you make extensive use of Lambda functions and would like better control over your costs, you should consider this execution model. By defining an EC2 cluster with specific hardware requirements, along with auto-scaling needs, your Lambda functions will execute on a fixed set of EC2 instances.

With LocalStack support, you can adopt this new execution model, while continuing to test your application before deploying to the AWS cloud. The same APIs are supported, as are the same CloudFormation resources. This feature will be available in LocalStack version 4.12, so early adopters should use the latest release available in DockerHub (the latest tag).