Skip to main content
OpenPAI applies multi-dimensional rate limits per user / token / model at the gateway, ensuring stability and fair use.

​
Limit dimensions

DimensionMeaning
RPM (Requests Per Minute)Maximum requests per minute
TPM (Tokens Per Minute)Maximum tokens per minute (prompt + completion)
ConcurrencyThe maximum number of outstanding requests at one time
Daily quota (optional)Maximum daily spend (optionally enabled for enterprise accounts)

​
Default limits

Different user groups have different ceilings; a common configuration:
GroupRPMTPMConcurrency
default200200,00016
vip1,0001,000,00064
svip5,0005,000,000256
enterpriseNegotiatedNegotiatedNegotiated
The live limits shown in Console → account home are authoritative.

​
Response headers

Every API response carries:
HeaderMeaning
X-RateLimit-Limit-RequestsCurrent RPM ceiling
X-RateLimit-Remaining-RequestsRequests remaining this minute
X-RateLimit-Reset-RequestsSeconds until reset
X-RateLimit-Limit-TokensCurrent TPM ceiling
X-RateLimit-Remaining-TokensTokens remaining this minute
X-RateLimit-Reset-TokensSeconds until reset
The client can use these for intelligent throttling.

​
Hitting the limit

When you exceed the ceiling, you get 429 rate_limit_exceeded: 
{
  "error": {
    "type": "rate_limit_error",
    "code": "rate_limit_exceeded",
    "message": "Rate limit reached. Please retry after 12 seconds."
  }
}
The Retry-After: 12 header gives the suggested retry time.

Exponential backoff

1s → 2s → 4s → 8s → 16s, capped at 30s.

Client-side throttling

Maintain a leaky / token bucket locally to control QPS proactively.

Batching

Merge short messages into one request to reduce RPM pressure.

Prompt caching

Route repeated prefixes through the cache to lower actual TPM usage.

​
Raising the limits

If your workload genuinely needs more, you can increase limits by:
  • Topping up to VIP / SVIP: automatically gets more generous quotas.
  • Enterprise custom quota: apply for dedicated RPM / TPM / concurrency via the business channel.
  • Dedicated channels: bind an isolated upstream account to specific models for capacity isolation.
Submit the request in the console.