SOCI lazy loading: start Lambda containers faster

Lambda Managed Instances (LMI) are a new way to run Lambda functions using EC2 instances in your own account, rather than on a shared Lambda fleet. Key differences between LMI and regular Lambda include multiple invocations per environment, container-based isolation on Nitro instances, and billing that combines EC2 price with a 15% management fee. Scaling is asynchronous based on CPU utilization, down to zero, making LMI more suitable for high and predictable traffic workloads.

The article provides a detailed walkthrough of implementing SOCI (Seekable OCI) lazy loading in LMI, which allows Lambda containers to start faster by fetching only the required files from a large image. A small example service is built to illustrate the concept - a book recommendation system using language models, the SOCI index, and CDK stack. The SOCI index reads a table of contents to fetch only the needed files from the image, resulting in up to 70% faster startup times for large images.

To implement SOCI lazy loading, five main steps are outlined:

1. Set up prerequisites: AWS account, AWS CLI, Node.js, Docker, AWS CDK, and Linux machine with containerd.

2. Create the SOCI index using nerdctl and push it to ECR.

3. Set up a capacity provider and function using AWS CDK in TypeScript.

4. Measure startup times with and without SOCI.

5. Analyze results and optimize code to ensure consistent initialization.

The core insight for developers is that while SOCI does not modify the original image, it creates a separate index that enables Lambda to fetch and unpack only the necessary parts of the image during initialization. This approach significantly reduces the time taken for startup, especially when dealing with large container images containing numerous models and libraries that may not be required for each function execution.