From Playground to Caching: Learning Amazon Bedrock

This post is a part of the author's notes from Formação AWS, a course led by Henrylle Maia. The focus of this particular class is Amazon Bedrock, a fully managed AWS service that provides API access to multiple foundation models. The author starts by exploring the Bedrock Playground to gain familiarity with how Bedrock works, before moving on to scripting to learn how to use the Bedrock API.

The author creates a simple chatbot that functions like a clerk selling tickets for an AWS course. Initially, the chatbot operates as a single Bash script that sends one message and prints the response. The author then enhances the chatbot to be interactive, allowing for genuine conversation. Next, the chatbot is moved to Node.js, enabling streaming of responses as they arise.

Lastly, the author incorporates a cache checkpoint mechanism to prevent the same System Prompt from being billed at full price on every message.

The post begins with an introduction to Amazon Bedrock, a managed AWS service that offers API access to various foundation models. The author chooses to utilize Anthropic's Claude Sonnet 4.5 model for their project. When attempting to send a prompt in the Playground, the author encounters a ThrottlingException error, which indicates that the Cross-Region Quota for Anthropic Claude Sonnet 4.5 has been exceeded.

The Cross-Region Quota limits the combined input and output tokens that can be sent to the model per minute through the cross-region inference profile across all Bedrock APIs, including Converse, ConverseStream, InvokeModel, and InvokeModelWithResponseStream.

To resolve this issue, the author navigates to the Service Quotas console, locates the appropriate quota for Claude Sonnet 4.5, and requests an increase to 5,000,000 tokens per minute. Once the request is approved, the author can proceed with testing the model in the Playground.

The Playground is where the author tests the model's responses directly in the AWS console and adjusts parameters such as Temperature, Top P, and Top K. Temperature ranges from 0 to 1, with 0 resulting in a more mechanical response and 1 promoting more creative and potentially hallucinatory output. Top P and Top K control the model's vocabulary selection, with higher values broadening the pool of words for each prediction.

The author sets the temperature to 0.7, which provides a balanced response that is neither too mechanical nor overly creative.

The author also explores the use of a System Prompt, which provides the model with context to generate appropriate responses. By testing two different System Prompts, the author demonstrates how a detailed prompt can significantly improve the model's focus and relevance to the task at hand. When the author runs the chatbot, the responses become more tailored to the requested functionality, such as accurately describing the pricing and features of the AWS Training Course tickets.