Guides
Rate limits
Understand RPM, TPM, quota windows, and retry behavior.
Overview
Rate limits protect upstream providers, account budgets, and service stability. Limits may apply by account, key, project, model, provider, subscription, or upstream route.
Common limit types
| Limit | Meaning |
|---|---|
| RPM | Requests per minute. |
| TPM | Tokens per minute. |
| Daily / weekly / monthly quota | Subscription or workspace usage windows. |
| Balance | Prepaid or pay-as-you-go balance. |
| Provider limit | Upstream provider rate or capacity limit. |
429 responses
A `429` response means the request should not be retried immediately. If a `Retry-After` header is present, wait at least that long.
const retryAfter = Number(response.headers.get("retry-after") ?? "1");
await new Promise((resolve) => setTimeout(resolve, retryAfter * 1000));Reducing 429s
- Use exponential backoff with jitter.
- Spread traffic across time instead of bursting.
- Set `max_completion_tokens` to control output size.
- Use separate keys for different environments and workloads.
- Review `/app/usage` and `/app/logs` by model, key, and workspace.
When to upgrade
Upgrade quota or contact support if legitimate production traffic frequently hits limits even after backoff and batching.