Streaming Claude Tokens with Fastify on Lambda: A Step‑By‑Step Guide Explained Simply

Developers often believe they require powerful frameworks to manage streaming responses from language models, but Fastify’s minimalistic core can efficiently proxy Claude’s token stream with minimal overhead. This guide demonstrates how to transform a Fastify service into a real-time chat endpoint that transmits tokens directly to the browser.

Fastify outperforms Express in streaming LLM responses. When you prompt a language model like Claude, it generates text in token form, one token at a time. To create a "live-typing" experience, each token must be forwarded to the browser immediately upon arrival. Express, a widely-used Node.js framework, incorporates numerous middleware layers to enhance flexibility.

While these layers are beneficial, each additional function call introduces minuscule latency. For a service handling dozens of tokens per second, this cumulative latency becomes noticeable. Conversely, Fastify prioritizes performance. Its core is streamlined, comprising solely the HTTP server and routing logic. It natively supports async/await, eliminating the necessity for callbacks that could block the event loop.

Leveraging the ultra-fast ultra-http library under the hood, Fastify facilitates rapid transmission of data from the model to the client, akin to a high-speed conveyor belt. In contrast, Express operates like a congested hallway where the same package (the token) must navigate through multiple doors before proceeding. This disparity is palpable when implementing a streaming chat endpoint.

In simple terms, Fastify’s compact design minimizes server-side waiting, enabling users to perceive tokens almost as soon as Claude produces them.

The code snippet below illustrates a minimal Fastify server setup:

```javascript

import Fastify from 'fastify';

// Create a Fastify instance with the smallest possible options

const fastify = Fastify({ logger: true });

fastify.get('/', async (request, reply) => {

// A simple health-check endpoint

return { status: 'ok' };

});

export default fastify;

```

Fastify initializes the server. The `logger: true` option logs request details, which proves invaluable during development. No extra middleware is loaded, thereby maintaining negligible overhead per request. Configuring Fastify with Node.js 22’s inherent `fetch` function, which mirrors browser behavior, allows seamless communication with AWS Bedrock, the service hosting Claude.

Utilizing native `fetch` reduces bundle size, crucial for Lambda, where every byte impacts cold-start time. Avoiding extra dependencies is vital because each additional library must be zipped, uploaded, and unpacked during Lambda initialization. Native `fetch` circumvents this overhead, thereby accelerating cold starts.

The following code snippet configures a Fastify route to invoke Claude via Bedrock and stream SSE data:

```javascript

import { FastifyInstance, FastifyReply, FastifyRequest } from 'fastify';

export async function chatRoute(fastify: FastifyInstance) {

// Register a POST /chat endpoint

fastify.post('/chat', async (request: FastifyRequest, reply: FastifyReply) => {

// Parse the incoming JSON body

const { prompt } = request.body as { prompt: string };

if (!prompt) {

reply.status(400).send({ error: 'Prompt is required' });

return;

}

// Call the helper function to interact with Bedrock

const stream = await invokeClaudeStream(prompt);

// Set up Server-Sent Events (SSE) headers

reply.raw.writeHead(200, {

'Content-Type': 'text/event-stream',

'Cache-Control': 'no-cache',

Connection: 'keep-alive',

});

// Stream each token from Bedrock to the client

for await (const token of stream) {

reply.raw.write(`data: ${JSON.stringify({ token })}\n\n`);

}

// Close the SSE stream when Bedrock completes

reply.raw.end();

});

}

```

This setup reads the JSON payload the browser transmits, dispatches the prompt to Bedrock, and streams each token as an SSE line. The `reply.raw` variable provides access to the underlying Node `http.ServerResponse`, enabling direct manipulation of raw SSE data. It is advisable to encapsulate fetch calls and response handling within the route to minimize code execution during each request, thus enhancing cold-start performance.