Azure OpenAI 429 errors: why you're throttled below quota, and how to fix it

A 429 "Too Many Requests" from Azure OpenAI in Microsoft Foundry is the most misread error on the platform. The usage charts say you're well under quota, yet calls fail, so people request more quota and nothing changes. The rate limit and the usage metrics measure different things, and there are four different reasons for a 429 that each need a different fix. This guide shows how to tell them apart and what to do for each.

Read the error before you act

Every response carries rate-limit headers. On a 429, read them before you change anything:

Then match the message to its cause:

Microsoft says it plainly: many customers treat capacity-related 429s as quota problems and request increases that can't help.

Why you are throttled below quota

The usage metrics in Azure Monitor show billed tokens from successful requests. The rate limit is applied to requests as they arrive, with an estimate, and that estimate is what fills the bucket:

• The token estimate includes max_tokens (and best_of), not just the prompt. A request that expects a 200-token answer with max_tokens set to 4,000 reserves 4,000 against your limit, even though you're billed for the real output. The estimate is also based partly on character count, so it's approximate and can trip earlier than an exact count would.
• Requests per minute are checked in short windows, typically 1 or 10 seconds. A 600 RPM deployment allows about 10 requests a second; a burst of 30 in one second is throttled even if the minute's total is fine.
• Rejected and failed requests still count. A request refused for being too long (a 400) can count against the limit without appearing in billed usage. Retrying without backing off makes it worse.
• Shared capacity can be reduced temporarily. Standard, pay-per-token deployments share a pool. When demand nears its limits, the service lowers a deployment's effective rate for a while. You can see it: the header's limit drops below your configured TPM.
• Quota must sit on the deployment that gets the traffic. Approved subscription quota does nothing until it's allocated to that deployment, and enforcement across a distributed service isn't perfectly precise either.

Retry properly

Honour retry-after-ms when it's present; otherwise back off exponentially with random jitter, and cap the attempts. The openai Python package already does this for 429s, with two retries by default; raise it on the client rather than writing your own loop:

from openai import AzureOpenAI

client = AzureOpenAI(
    azure_endpoint="https://<your-resource>.openai.azure.com/",
    api_key="<your-api-key>",
    api_version="2024-10-21",
    max_retries=5,  # exponential backoff with jitter, honours retry-after
)

If you use a retry library such as tenacity or Polly instead, set the SDK's max_retries to 0 first. Otherwise every attempt of yours can trigger up to two SDK retries of its own, and the extra requests count against the same limit.

Then shrink what each request reserves:

• Set max_tokens to the smallest value your scenario needs. This is the single most common fix.
• Leave best_of at 1 unless you really need several completions; each increment multiplies the token count.
• Shorten prompts where you can.
• Spread traffic. Ramp new workloads up gradually; avoid sharp spikes; split across deployments or regions when one deployment can't carry the load; queue work that doesn't need an immediate answer.

Check and rebalance quota

Quota is tokens per minute per subscription, region, model and deployment type, and each deployment takes a slice of it.

1. Open Manage > Quota in the Foundry portal. The Token per minute tab shows each deployment's allocation and usage; the pencil in Affiliated deployments using shared quota moves TPM between deployments. Changes take up to 15 minutes.
2. To see it you need the Cognitive Services Usages Reader role at the subscription scope, not on the resource group or the resource.
3. For a script or an alert, the Usages API returns every quota line in a region with currentValue and limit; the Model Capacities API says where a model can still be deployed and how much capacity is free.
4. When the whole subscription is short, submit Microsoft's quota increase form. Requests are handled in order, and priority goes to subscriptions that actively use what they have.

When to escalate or move to provisioned throughput

Provisioned deployments measure utilisation differently and don't share the pool, so they're the answer for work that needs the same throughput at the same latency every minute. For everything else, the fixes above are usually enough.

Sources

• Manage Azure OpenAI in Microsoft Foundry Models quota, Microsoft Learn (rate limits, headers, the four 429 causes, retry guidance)
• Troubleshoot common HTTP errors for Azure OpenAI, Microsoft Learn
• What is provisioned throughput for Foundry Models?, Microsoft Learn (spillover)
• Role-based access control for Microsoft Foundry, Microsoft Learn (the Usages Reader role)