Skip to main content
OpenPAI applies multi-dimensional rate limits per user / token / model at the gateway, ensuring stability and fair use.

​
Limit dimensions

​
Default limits

Different user groups have different ceilings; a common configuration: The live limits shown in Console → account home are authoritative.

​
Response headers

Every API response carries: The client can use these for intelligent throttling.

​
Hitting the limit

When you exceed the ceiling, you get 429 rate_limit_exceeded:
The Retry-After: 12 header gives the suggested retry time.

Exponential backoff

1s → 2s → 4s → 8s → 16s, capped at 30s.

Client-side throttling

Maintain a leaky / token bucket locally to control QPS proactively.

Batching

Merge short messages into one request to reduce RPM pressure.

Prompt caching

Route repeated prefixes through the cache to lower actual TPM usage.

​
Raising the limits

If your workload genuinely needs more, you can increase limits by:
  • Topping up to VIP / SVIP: automatically gets more generous quotas.
  • Enterprise custom quota: apply for dedicated RPM / TPM / concurrency via the business channel.
  • Dedicated channels: bind an isolated upstream account to specific models for capacity isolation.
Submit the request in the console.