Rate limits

1 · What it is

A rate limit caps how much you can use an AI service in a set stretch of time. Providers use them to stop floods of requests. They also help share the service fairly, so it stays smooth for everyone.

Services usually count several things at once: requests each minute (RPM), tokens each minute (TPM) and requests each day (RPD). Tokens are the small chunks of text a model reads and writes. Token limits cap how much text you send, and some providers also cap how much comes back. Go over any single limit and you get an error. Picture a limit of 20 requests a minute. A 21st request inside that minute fails, even if each request was tiny.

The error is the web status code 429, which means too many requests in a short window. The reply may carry a retry-after header. A header is a short note attached to the reply, and this one says how long to wait. Its value can be a calendar date or a count of seconds. The usual response is to wait at least that long, then retry. If it fails again, wait a little longer each time, and add a short random pause so many apps do not all retry at once. Resending instantly in a loop does not work, because failed requests still count toward the per-minute limit.

For engineers: OpenAI replies carry headers such as x-ratelimit-remaining-requests, which show how many requests you have left. Anthropic’s API uses a token bucket. Think of a bucket that slowly refills up to the top, instead of being emptied and refilled at fixed times. A cache keeps text the model has already read, so it can be reused. On most Claude models, input tokens served from the cache are left out of the input-token limit. On the Gemini API, the daily request quota starts fresh at 12 a.m. Pacific time.