Skip to main content
The major vendors (OpenAI, Anthropic, DeepSeek, Google) all support prompt caching — mark a long system prompt / RAG context, and subsequent requests with the same prefix can hit the cache and be settled at a lower unit price. OpenPAI passes through all cache-related fields and bills the cache-hit portion at a lower multiplier, transparently to the client.

​
OpenAI automatic caching

Since GPT-4o, OpenAI automatically caches prefixes ≥1024 tokens, with no client configuration needed.
  • The cache-hit price is typically 0.5× of the input price (GPT-4o) or 0.1× (o3 / GPT-5).
  • The response usage.prompt_tokens_details.cached_tokens field shows the number of hits.
OpenPAI passes this field through and bills at the corresponding multiplier.

​
Claude cache_control

Claude marks cache segments explicitly with cache_control: 
{
  "model": "claude-opus-4-8",
  "system": [
    {
      "type": "text",
      "text": "<long system prompt, 50000 chars>",
      "cache_control": {"type": "ephemeral"}
    }
  ],
  "messages": [{"role": "user", "content": "..."}]
}
  • Write: creating the cache the first time costs 1.25× the input price (25% more).
  • Hit: the same prefix within the next 5 minutes costs 0.1× the input price.
  • TTL: 5 minutes (extendable via ttl: "1h" in cache_control, if the corresponding product supports it).

​
DeepSeek automatic caching

DeepSeek-V3 / R1 automatically cache long prefixes; cache hits are billed at the very low unit price the vendor publishes (typically about 0.1× of the original).
  • The number of hits is shown in the usage.prompt_cache_hit_tokens field.
  • OpenPAI passes it through and bills accordingly.

​
Google Gemini Context Caching

The Gemini 2.5 series supports the explicit cachedContents API: 
cache = client.caches.create(
    model="gemini-3.1-pro-preview",
    contents=["<long context>"],
    ttl="1h",
)

resp = client.models.generate_content(
    model="gemini-3.1-pro-preview",
    contents="Answer the question based on the context...",
    cached_content=cache.name,
)
OpenPAI passes through the cached_content reference, and the hit portion is settled at Google’s published cache price.

​
Console display

In the call log you can see:
FieldMeaning
Input tokensThe raw input sent by the client
Cache-hit tokensThe portion that hit the cache
Cache multiplierThe multiplier billed for the hit portion
Cache-write tokensClaude only, the extra higher-priced first-write portion

​
Best practices

Put invariant content first

System prompts, RAG results, few-shot examples → place them at the front of the messages array, then append the user’s question.

Share a prefix across batched requests

Ask different questions under the same long context to maximize cache-hit benefit.

Control variability

Avoid inserting timestamps or random numbers into the prefix, or the prefix won’t match and won’t hit.

Measure the benefit

Watch the cached_tokens ratio in the logs to optimize your prompt structure.

​
Caveats

  • The cache-hit rate depends on actual upstream availability; OpenPAI does not fabricate cache hits.
  • Cross-channel-routed requests may switch between two upstream accounts, invalidating the cache — for cache-dependent scenarios, bind a fixed channel to the model in the console.
  • Cached content is stored encrypted by the upstream vendor; OpenPAI does not store any prompt content.