Prompt Caching

Prompt caching is provided by the upstream. Hits depend on the model, prompt prefix, and TTL. GPROXY can add markers through cache breakpoint rules or magic strings in prompts. Markers do not guarantee a hit or add caching support to an upstream that lacks it.

Add a cache breakpoint in the console

Add a cache breakpoint rule on the provider’s rewrite page. Select the upstream protocol and a global, system, or message target. Claude also supports tools. For a local target, positive indices start at 1, negative indices count from the end, and a blank index selects the last available position.

Claude offers default, 5-minute, and 1-hour TTLs. OpenAI local breakpoints offer default, 30-minute, and 12-hour TTLs; global rules offer default and 30-minute TTLs. Actual behavior depends on the upstream. Select the protocol used after conversion; Gemini is not supported by this rule type.

For API configuration, use action: "cache_breakpoint" and a JSON-encoded configuration string in replacement, for example:

{"dialect":"claude","target":"message","index":-1,"ttl":"5m"}

{"dialect":"openai","target":"message","index":-1,"ttl":"12h"}

This rule runs on the request body in the configured rewrite order. Clients that can set native cache fields may also use the upstream’s supported fields directly.

Use magic strings

Clients without cache controls can include the following strings in prompt text and enable the provider’s matching magic-cache option.

The three strings

GPROXY_MAGIC_STRING_TRIGGER_CACHING_CREATE_7D9ASD7A98SD7A9S8D79ASC98A7FNKJBVV80SCMSHDSIUCH
GPROXY_MAGIC_STRING_TRIGGER_CACHING_CREATE_49VA1S5V19GR4G89W2V695G9W9GV52W95V198WV5W2FC9DF
GPROXY_MAGIC_STRING_TRIGGER_CACHING_CREATE_1FAS5GV9R5H29T5Y2J9584K6O95M2NBVW52C95CX984FRJY

String	On a Claude target	On an OpenAI target
…7D9ASD…	cache_control with the API’s default TTL	an explicit breakpoint
…49VA1S…	cache_control with "ttl": "5m"	prompt_cache_breakpoint ("ttl": "30m")
…1FAS5G…	cache_control with "ttl": "1h"	prompt_cache_breakpoint ("ttl": "12h")

The same strings select short and long retention by target dialect: Claude 5m / 1h maps to OpenAI 30m / 12h. The default string leaves TTL unset. Local rules write TTL on the selected breakpoint, without changing the request-wide cache options.

Native cache fields use the same bidirectional mapping in Claude ↔ OpenAI Chat/Responses conversions. Marked system, message, image/document and tool-result blocks retain their boundaries. A Claude automatic cache policy becomes an explicit breakpoint at the corresponding final block when that block has a target representation. Per-tool definition markers without an OpenAI equivalent remain listed in the conversion report.

When converting OpenAI to Claude, only an explicitly supplied prompt_cache_options.mode: "implicit" creates top-level cache_control. Omitting the options object or its mode, including an empty object or TTL alone, does not enable Claude automatic caching. Explicit block markers still convert independently.

They are frozen: they are part of the client-to-proxy protocol and do not change between releases.

The Rule That Matters Most

A token is always stripped, whether or not caching is enabled. A leaked marker sitting in somebody’s prompt is never wanted.

What the switch decides is only whether a breakpoint is placed where the token was. With magic cache off, the tokens come out and nothing else is touched: the client’s own cache_control or prompt_cache_breakpoint survives untouched, and a body with no token is forwarded byte for byte, never parsed and re-serialized.

The bare strip pass is also what claudeweb applies, since a browser session has no cache control at all.

Enabling It

Two provider config switches, one per family:

{ "config": { "enable_claude_magic_cache": true,
              "enable_openai_magic_cache": true } }

Switch	Channels that offer it
enable_claude_magic_cache	aws_bedrock, azure, claudeapi, claudecode, custom, opencodego, opencodezen, openrouter
enable_openai_magic_cache	aws_bedrock, azure, openai, codex, custom, opencodego, opencodezen, openrouter

The switch is read against the target dialect, not the client’s. A request from an OpenAI-shaped client routed to a Claude provider is converted to Claude Messages first, so the Claude switch is the one that applies and the marker written is cache_control. The client keeps receiving its own format.

Where a Marker Lands

Claude Messages

The body is canonicalized first: string content becomes block arrays, empty text blocks are dropped, and a cache_control sitting on a dropped block moves to the nearest cacheable block — which is what makes a lone token on its own line work. Thinking blocks are kept only on assistant turns.

Then system is walked before messages, so the budget is spent in prompt order rather than in JSON key order. Every text block carrying a token is marked:

{ "cache_control": { "type": "ephemeral", "ttl": "1h" } }

A block that already has cache_control is left alone.

OpenAI Chat Completions

Every message except the function role can be marked. A plain-string content is split into a marked text part, because a marker needs somewhere to sit; an array content has its parts marked in place.

{ "type": "text", "text": "…", "prompt_cache_breakpoint": { "mode": "explicit" } }

OpenAI Responses

Three places, in order:

1. instructions. A token there is stripped and a marked developer message is prepended in front of input — instructions is a string and cannot carry a marker itself.
2. prompt.variables, marked like Chat parts.
3. input, whether it is a bare string (which becomes one marked user message) or an array of items, whose content and output are marked with the input_text / output_text kinds.

Gemini

No breakpoints. Tokens are stripped and nothing else happens.

The Cap of Four

At most four breakpoints leave the proxy, the client’s own included. The budget is computed by counting the markers already in the body, so a client that placed three of its own leaves room for one.

Tokens beyond the cap are still stripped — they just do not become markers. That is the same rule as everywhere else: the string never reaches an upstream.

Usage and Pricing

Cached tokens are their own metric, and settlement prices them from your rate rows:

What an upstream reports	The metric
cache read	cached_input_tokens
cache write, by retention	cache_creation_tokens + ttl_seconds
cache write, unknown retention	cache_creation_tokens

Cache writes use cache_creation_tokens. A rate condition such as {"ttl_seconds":43200} selects a 12-hour write price; omit this condition for the fallback price. Each reported write bucket is priced separately. Cache reads remain a single cached_input_tokens count, with no TTL. Tier overrides use cacheCreationPerMillion: [{"ttlSeconds":43200,"price":"5"}]; an entry with ttlSeconds: null is the fallback. Prices are per million tokens, not per token-second.

OpenAI Responses and Chat explicit breakpoints accept prompt_cache_breakpoint: {"mode":"explicit","ttl":"12h"} (or "30m"). The TTL survives Chat ↔ Responses conversion. The current OpenAPI schema still limits request-wide prompt_cache_options.ttl to "30m"; prompt_cache_retention is a separate legacy policy. The per-call usage schema reports only a total cache_write_tokens, so GPROXY records its TTL as unknown rather than guessing a split for mixed breakpoints.

cached_input_tokens is capped at input_tokens: the uncached remainder is priced at the input rate and the cached part at the cache-read rate, which falls back to the input rate when no cache-read rate exists. Cache writes with no rate row of their own settle at zero. See Pricing & Tiers.

A token count is optional on a record — an upstream that did not report a field did not measure zero — while every total is a plain number, because a sum of “nothing reported” really is zero.

Keeping a Hit

Rewrite rules run after conversion and before the channel, so a rule that edits prompt text can turn an expected hit into a miss. Put the stable content first, the marker after it, and everything per-request after the boundary — and check that no rewrite rule with a broad paths selector is editing the prefix.

See Rewrite Rules & Operation Overrides.