Prompt Caching

Overview

Concentrate supports two provider-native prompt-caching interfaces:

Direct OpenAI GPT-5.6 Sol, Terra, and Luna use prompt_cache_options and content-block prompt_cache_breakpoint.
Anthropic and AWS Bedrock Claude models use cache_control.

The interfaces have different modes and TTLs, so use the fields documented for the provider you want to route to. If a request carrying one interface’s markers routes to the other provider family, Concentrate converts them to the routed provider’s native form. See Cross-provider conversion.

OpenAI GPT-5.6 explicit caching

OpenAI GPT-5.6 supports a request-wide caching mode and TTL:

string

"implicit" allows OpenAI to choose cache locations. "explicit" writes only at content blocks marked with prompt_cache_breakpoint.

string

One of "5m", "30m", or "1h". OpenAI supports "30m" natively; the other values are accepted for cross-provider compatibility and snap to the nearest TTL the routed provider supports. TTLs are minimum lifetimes, and providers may retain cached prefixes longer.

string

Set this to "explicit" to end an explicitly cached prefix after that text, image, or file block.

Responses API example

{
  "model": "openai/gpt-5.6-sol",
  "prompt_cache_options": {
    "mode": "explicit",
    "ttl": "30m"
  },
  "input": [
    {
      "role": "developer",
      "content": [
        {
          "type": "input_text",
          "text": "Long-lived application instructions...",
          "prompt_cache_breakpoint": {
            "mode": "explicit"
          }
        }
      ]
    },
    {
      "role": "user",
      "content": "Answer this request."
    }
  ]
}

Put stable content first. Any change before a breakpoint changes that cached prefix. OpenAI only caches prompt prefixes of 1,024 tokens or more. A single request can create at most four cache writes: when a request contains more markers, OpenAI writes the latest breakpoints: up to the latest three explicit breakpoints in implicit mode, or the latest four in explicit mode. prompt_cache_retention remains available for earlier OpenAI models. GPT-5.6 and later use prompt_cache_options.ttl instead.

Chat Completions example

The same controls are available on the Chat Completions API. Set prompt_cache_breakpoint on text, image_url, or file content parts of developer, system, user, tool, or assistant messages:

{
  "model": "openai/gpt-5.6-sol",
  "prompt_cache_options": {
    "mode": "explicit",
    "ttl": "30m"
  },
  "messages": [
    {
      "role": "developer",
      "content": [
        {
          "type": "text",
          "text": "Long-lived application instructions...",
          "prompt_cache_breakpoint": {
            "mode": "explicit"
          }
        }
      ]
    },
    {
      "role": "user",
      "content": "Answer this request."
    }
  ]
}

Chat Completions usage reports cache activity in usage.prompt_tokens_details.cached_tokens and usage.prompt_tokens_details.cache_write_tokens. Breakpoints on assistant message parts are accepted and forwarded, but OpenAI does not currently write cache entries for them. Only developer, system, user, and tool content is cached.

Anthropic and AWS Bedrock

Use cache_control for Claude models served directly by Anthropic or through AWS Bedrock:

{
  "model": "anthropic/claude-sonnet-4-6",
  "input": [
    {
      "role": "developer",
      "content": [
        {
          "type": "input_text",
          "text": "Long-lived application instructions...",
          "cache_control": {
            "type": "ephemeral",
            "ttl": "1h"
          }
        }
      ]
    },
    {
      "role": "user",
      "content": "Answer this request."
    }
  ]
}

Anthropic accepts at most four cache_control markers per request. When a request routed to Anthropic contains more, Concentrate keeps the last four markers in prompt order (the ones closest to the end of the prompt) and strips the rest. A request-wide cache setting (top-level cache_control, prompt_cache_options, or prompt_cache_retention) places a marker on the last cacheable block, so it occupies one of the four slots. Prompt cache TTLs are rounded up according to what the provider supports. For example, sending prompt_cache_options with a TTL of 30 minutes to Anthropic results in a 1 hour TTL being set, as Anthropic does not support 30 minute cache TTLs. Anthropic/Bedrock cache_control TTL options are "5m" and "1h". prompt_cache_options.ttl accepts the same values for compatibility, but on OpenAI routes they snap to "30m". Anthropic also requires "1h" markers to appear before "5m" markers in the prompt. When a kept "5m" marker (an omitted ttl defaults to "5m") precedes a "1h" marker, Concentrate will lower the later marker to "5m" for you.

Cross-provider conversion

You can keep one interface in your request and still route across provider families, and Concentrate converts markers to the routed provider’s native form:

On Anthropic and Bedrock routes, a content block’s prompt_cache_breakpoint converts to a cache marker after that block. The converted marker’s TTL comes from prompt_cache_options.ttl: "5m" stays five minutes; any other value or none becomes "1h".
On OpenAI GPT-5.6 routes, a content block’s cache_control converts to prompt_cache_breakpoint: { "mode": "explicit" }. Block-level TTLs do not carry over; the request-wide prompt_cache_options.ttl applies and snaps to "30m".
When a block carries both fields, the routed provider’s native field takes priority and the other is converted only in its absence.

Converted markers behave like native ones: on Anthropic routes they count toward the four-marker limit and participate in TTL ordering.

Usage and cost

Inspect these response usage fields:

input_tokens_details.cached_tokens is the number of cache-read tokens.
input_tokens_details.cache_write_tokens is the number of prompt tokens written to cache, reported for every provider.
input_tokens_details.cached_tokens_created is deprecated and will be removed in a future release. Use cache_write_tokens instead. It reports the same cache-write count and remains populated for now to give integrations time to switch.

For GPT-5.6, cache writes cost 1.25 times the applicable uncached input rate. Cache reads use the model’s cache-read rate. input_tokens already includes cache reads and writes; do not add the detail counters to it.

Cache isolation

Concentrate supplies a stable provider cache key derived from your API key when you do not send prompt_cache_key. Cached prefixes are isolated between API keys.

Troubleshooting

The cache is not being reused

Check that the model/provider supports the fields you sent, content before the breakpoint is byte-for-byte stable, and the TTL has not expired. Verify reuse with cached_tokens.

The first request costs more

An explicit cache miss can create a write billed at 1.25 times input. Check cache_write_tokens; later matching requests should report cache reads instead.

Azure ignores explicit cache fields

Concentrate does not advertise prompt_cache_options or prompt_cache_breakpoint for Azure OpenAI because Azure has not documented those GPT-5.6 controls. Use a direct OpenAI route for this feature.

Request Parameters

Complete request parameter reference

Create Response

Responses API endpoint documentation

Responses

Chat Completions (Beta)

Messages (Beta)

Models API

Features

Utilities

Reference

Catalog

Overview

OpenAI GPT-5.6 explicit caching

Responses API example

Chat Completions example

Anthropic and AWS Bedrock

Cross-provider conversion

Usage and cost

Cache isolation

Troubleshooting

Request Parameters

Create Response

​Overview

​OpenAI GPT-5.6 explicit caching

​Responses API example

​Chat Completions example

​Anthropic and AWS Bedrock

​Cross-provider conversion

​Usage and cost

​Cache isolation

​Troubleshooting

​Related documentation

Request Parameters

Create Response

Overview

OpenAI GPT-5.6 explicit caching

Responses API example

Chat Completions example

Anthropic and AWS Bedrock

Cross-provider conversion

Usage and cost

Cache isolation

Troubleshooting

Related documentation