The most-misread AI error is a 429. It arrives during a load spike, it stops your app cold, and it looks exactly like the provider going down — but it's the opposite. A 429 means the service is healthy enough to enforce its limits and is deliberately refusing your requests. Fixing it is on your side, and it's usually straightforward once you know which limit you hit.
What a 429 actually is ¶
HTTP 429 Too Many Requests is the standard "you've sent too many requests in a window" signal. For LLM APIs it almost always carries a Retry-After header (or provider-specific reset headers) telling you when to try again. The general mechanics — and why it's not a server error — are covered in our 429 deep-dive; this post is the AI-specific version.
The limits that trigger it ¶
LLM providers meter more than one dimension at once, and you get a 429 the moment you cross any of them:
- RPM — requests per minute. Too many calls, regardless of size.
- TPM — tokens per minute. The one that surprises people: a handful of huge-context requests can blow the token budget while your request count looks tiny.
- Concurrent requests. Too many in-flight at once, common with unbounded parallelism.
- Daily / monthly quotas. Especially on free and low tiers.
- Per-model limits. Newer or larger models often have tighter caps than the workhorse ones.
How to back off correctly ¶
The wrong response to a 429 is to immediately retry — that's a retry storm, and it keeps you pinned against the limit. The right response:
- Honor
Retry-After. If the provider tells you when the window resets, wait that long. Don't guess. - Exponential backoff with jitter. If there's no header, wait 1s, then 2s, 4s, 8s… with a little randomness so parallel workers don't all retry in lockstep.
- Cap retries. After a few attempts, fail gracefully to the user rather than looping forever.
- Queue, don't fan out. A small client-side rate limiter that paces requests under your RPM/TPM ceiling prevents most 429s before they happen.
Structural fixes when 429s are constant ¶
If you're throttled all the time, backoff is a band-aid — change the shape of your usage:
- Batch and cache. Deduplicate identical prompts and cache results; many "requests" are the same call repeated.
- Trim context. TPM limits are about tokens — shorter prompts and smaller context windows stretch the same budget much further.
- Request a higher tier. Limits scale with usage tier / spend; if your workload is real, raise the ceiling.
- Spread load. Smooth bursts into a steady rate instead of firing everything at the top of the minute.
429 vs. a real outage ¶
If you're unsure whether it's a limit or an incident, the code and a quick probe settle it: a 429 is a rate limit (up); a 500/503 is a server error (down). Confirm the provider itself is healthy with a multi-region check and the AI status board — see how to check OpenAI for the full routine. If the provider is green and you're getting 429s, the fix is entirely on your side.
FAQ ¶
Does a 429 mean the AI provider is down?
No — it's the opposite. A 429 means the service is up and enforcing rate limits by refusing your excess requests. A genuine outage returns 5xx status codes or fails to connect. If you're getting 429s while the provider's status is healthy, the throttling is coming from your own request volume.
Why do I get a 429 when my request count is low?
Almost always the tokens-per-minute (TPM) limit rather than requests-per-minute. A few large-context calls can exhaust the token budget even though the number of requests is small. Trim your prompts and context, or move to a tier with a higher TPM ceiling.
What's the right way to retry after a 429?
Honor the Retry-After header if present; otherwise use exponential backoff with jitter (1s, 2s, 4s…) and a retry cap. Never retry immediately in a tight loop — that keeps you pinned against the limit and slows recovery. Better still, pace requests under your limit with a client-side rate limiter so 429s rarely fire.