Introducing usage-based pricing for Amazon SageMaker AI bidirectional streaming
AWS Marketplace now supports usage-based metering for Amazon SageMaker bidirectional streaming inference, so model providers can charge for what customers actually consume during a live stream. Deepgram is the launch partner, bringing usage-based speech-to-text and text-to-speech billing to AWS Marketplace.
The new usage-based metering for Amazon SageMaker bidirectional streaming enables model providers to bill customers based on actual usage during live streaming sessions. Originally, software metering for real-time request-response inference could only report a fixed quantity in an HTTP response header, which worked well for APIs with known billable quantities.
However, bidirectional streaming involves continuous audio streaming and transcription, with usage accumulating throughout a session that can last from seconds to minutes. To address this, a second WebSocket connection, the metadata connection, was introduced to carry metering data separately from the inference stream. Model servers opt-in by returning a specific header on the WebSocket upgrade response, establishing a correlation between metering records and the active inference session.
Metering data, sent as JSON text frames, reports usage in the inference.count dimension with the consumed units. This approach provides more accurate and transparent billing for production real-time AI workloads, aligning charges with customer consumption and consolidating AWS billing with AWS Marketplace offers.
Written by urgent.news from AWS Machine Learning's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
