Build
Prompt caching.
Every request is cached where its model can cache, with nothing to turn on. A prompt that starts as an earlier one did reads that start from the cache, and every answer says how many tokens were read from the cache and written to it.
On this page
A model's provider keeps the start of a recent prompt for a few minutes. When the next prompt starts the same way, that start is read from the cache instead of being processed again. Every cached read and every cache write is billed at the rate the price sheet lists for it.
The gateway sends your prompt in the order you sent it, translated the same way every time, and adds nothing that changes from one request to the next, so a repeated start stays cacheable. To make the most of it:
- Put what stays the same first: the system prompt, the tools, long documents and examples. Put what changes, such as the latest question, last.
- Keep the repeated part identical, byte for byte. A changed word early in the prompt starts a new cache from there.
- A prompt shorter than the model's minimum is not cached, and costs nothing extra. The minimum is 1,024 tokens or more, depending on the model.
Many models cache a repeated start with nothing in the request. GET /v1/models/{author}/{slug}/endpoints marks every route that caches each prompt, by itself or with the automatic breakpoint below, and bills cached reads at a rate of their own:
A route that caches
"supports_implicit_caching": true
prompt_cache_key, a string of up to 256 characters, is passed on to the models whose API takes it, where it only steers which cache a prompt lands in. It costs nothing.
Some models cache only up to a breakpoint that marks where the cached part ends. Their price lists a cache write rate. On those models every request carries an automatic breakpoint on the last block of the prompt that can be cached, and it moves forward as a conversation grows, so a conversation is cached with no change on your side.
Your own breakpoints are kept: "cache_control": {"type": "ephemeral"} on a system block, a message, a content part or a tool.
A breakpoint after a long system prompt
curl https://api.usdf.fi/v1/chat/completions \ -H "Authorization: Bearer $USDF_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "claude-sonnet-5-5", "messages": [ {"role": "system", "content": [ {"type": "text", "text": "<a long document or instructions>", "cache_control": {"type": "ephemeral"}} ]}, {"role": "user", "content": "Summarise the second section."} ], "max_tokens": 256 }'
| Rule | What happens |
|---|---|
| How many | Four breakpoints in a request at most, the automatic one among them. Of yours, the last three are kept. |
| How long | Five minutes. "ttl": "1h" asks for an hour on models whose price lists a one-hour write. |
| The automatic one | A cache_ at the top level of the request body sets the automatic breakpoint's lifetime. A breakpoint of yours on the prompt's last block becomes the automatic one. |
| Other models | On a model with no cache write rate, breakpoints are removed before the request is sent, and the model caches by itself where it can. |
Writing to the cache is billed at the write rate the sheet lists, a one-hour write at its own rate; reading it back is billed at the cached rate.
A prompt the same key sent in the last five minutes goes back to the route that served it, so it reads that cache rather than starting a cold one elsewhere. A prompt counts as the same when the model, the tools and every message up to the first user message match, and that part is 1,024 tokens or more.
The repeat goes back only when reading the cache there costs no more than the first route in the order would cost without it. Only the default order moves: a request that sorts, sets a provider order or asks for a tier keeps its own.
With "model": "auto", a conversation also stays on the model first chosen for it while its cache is warm, five minutes from its last request. The conversation is the key with its prompt_cache_key, or with its system prompt and first user message. See Routing.
Every answer counts the tokens read from the cache and written to it, both among the prompt tokens:
| API | Read from the cache | Written to the cache |
|---|---|---|
Chat completions, / | usage. | usage. |
Responses, / | usage. | Not reported |
Messages, / | usage. | usage.; input_ counts neither |
| The receipt | receipt., receipt. | receipt., receipt. |
Chat completion usage
"usage": { "prompt_tokens": …, "completion_tokens": …, "prompt_tokens_details": { "cached_tokens": …, "cache_write_tokens": … } }
The receipt's prompt_cache is there when the request read from the cache or wrote to it. It states, at the rates the request was charged:
Receipt, trimmed
"receipt": { "usage": { "prompt_tokens": …, "completion_tokens": …, "cached_tokens": …, "cache_write_tokens": … }, "prompt_cache": { "read_tokens": …, "read_saving": { "units": "<units>", "usd": "<usd>", "usdf": "<usdf>" }, "write_tokens": …, "write_premium": { "units": "<units>", "usd": "<usd>", "usdf": "<usdf>" } } }
read_: the tokens read from the cache at the input rate, less the same tokens at the cached rate.saving write_: the tokens written to the cache at their write rate, less the same tokens at the input rate.premium
A stream reports the same usage in its last chunk when it asks for it with "stream_options": {"include_usage": true}, and its receipt as the last comment line.
Read with your key, the price sheet lists each model's cache rates beside its input and output rates, where the model has them:
| Rate | GET /v1/pricing, per 1M tokens | GET /v1/models, per token |
|---|---|---|
| A cached read | price. | pricing. |
| A cache write, five minutes | price. | pricing. |
| A cache write, one hour | price. | pricing. |
A model with no cached rate bills every prompt token at its input rate.