Urgent.News

What's breaking now, across thousands of outlets.

AI

API rate limits when thousands of devices share one LLM account

Rate limits belong to your account or project, so every device shares one pool. On Anthropic's published Claude Sonnet 5.5 limits as of October 31, 2026, 5,000 devices split to between 0.8 and 2.6 requests per device per minute, depending on tier. Bursts can trip limits earlier, so stagger wake-ups, back off with jitter and queue centrally. A device fleet looks like many users, but a model API…

The single pool of API rate limits applies to all devices sharing an account or project, regardless of how many individual devices are accessing the service. For Anthropic's Claude Sonnet 5.5 model as of October 31, 2026, the limits range from 0.8 to 2.6 requests per device per minute, depending on the tier, when 5,000 devices are sharing the account.

Bursts can trigger rate limit errors earlier, so it is recommended to stagger wake-ups, incorporate jitter, and queue requests centrally to avoid exceeding limits. Google's Gemini API follows similar principles, with limits scoped at the organization or project level rather than individual devices. OpenAI does not specify per-device limits, but rather overall limits for the organization or project.

To determine the effect on individual devices, divide the account limit by the number of devices in the fleet to calculate the per-device allowance per minute and monthly budget per device. Compare the calculated demand against the provider's limits, and if the first three calculations exceed the tier limits or the monthly spend exceeds the cap, the chosen tier cannot support the fleet size.

On launch day, unexpected high traffic from a firmware issue or inadequate wake-up design can deplete the shared rate limit pool faster than anticipated, leading to potential rate limit errors for all devices in the fleet.

Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.

Read the original at dev.to →

More in AI

More from Tuesday 17 November →