{
  "id": 19366232,
  "title": "Amazon Bedrock tokenomics data lake",
  "url": "https://urgent.news/2026/11/10/amazon-bedrock-tokenomics-data-lake",
  "topic": "ai",
  "section": "AI",
  "published": "2026-11-10T16:56:36.000Z",
  "source": {
    "name": "AWS Machine Learning",
    "slug": "aws-machine-learning",
    "url": "https://aws.amazon.com/blogs/machine-learning/amazon-bedrock-tokenomics-data-lake/"
  },
  "original_language": "en",
  "account": "Amazon Bedrock offers model invocation logging, which provides a full record of requests and responses to Amazon Simple Storage Service (Amazon S3) or Amazon CloudWatch. However, the data is locked inside individual accounts, making it difficult to gain a single, fast, queryable view of every invocation across the whole organization. This is where the Amazon Bedrock observability data lake comes in, providing a centralized solution for all Bedrock invocations.\n\nThe data lake is built by pulling raw model invocation logs from linked accounts into Amazon CloudWatch and then forwarding those records to your central account through subscription log filters. The central account processes the data through Amazon Data Firehose and exposes it through Amazon Athena as one clean table that can be queried across the entire organization.\n\nPer invocation, the data lake captures various information such as the prompt sent, the model response received, input and output token counts, the model used, the stop reason (like max_tokens), the account and timestamp for attribution and time-slicing, and the IAM role used for each request. This provides valuable visibility for platform and ML engineers to debug spikes, spot wasteful prompts, and track usage across teams.\n\nThe design of the data lake involves five stages. First, Amazon Bedrock writes an invocation log for every application call. Second, CloudWatch forwards the logs to a central account using subscription filters. Third, a small Lambda function tidies each record by unwrapping batches, removing noise, and flattening an awkward field. Fourth, Amazon Data Firehose converts the records to Parquet format and writes them to an Amazon S3 bucket with date-based paths. Finally, AWS Glue creates an Athena-compatible table for querying the data using standard SQL. With this infrastructure, you can quickly see which users are using the most tokens, which models are being selected, and what prompts are being used.",
  "summary": "Running Amazon Bedrock across many accounts shows you the bill but not what happened inside each call. This post builds a centralized observability data lake that aggregates Bedrock invocation logs from every linked account into one queryable table, so you can see prompts, token counts, model choice, and per-invocation cost across your organization.",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}