Incident Report: Prompt Cache Invalidation for Claude Code on Bedrock Invoke

LiteLLM incident report. Published 2026-07-13.

Date: July 4 to July 10, 2026 Affected versions: v1.91.0 and v1.91.1 Severity: Medium (silent cost regression; no correctness impact) Status: Resolved in v1.91.2

Note: If you run Claude Code against Amazon Bedrock through LiteLLM on either v1.91.0 or v1.91.1, upgrade to v1.91.2 or higher.

Summary

Between July 4 and July 10, proxies running v1.91.0 or v1.91.1 silently broke Anthropic prompt caching for Claude Code sessions routed through Amazon Bedrock’s Invoke API. For the customers who reported it, warm-session cache hit rates dropped from roughly 90% to 25-45% and team daily spend rose 2-3x for the same usage. Requests kept returning 200s with correct completions; the only symptoms were the cache miss rate and the bill.

The cause: PR #31364 moved every role: "system" entry in messages into the top-level system field on the Invoke path, which invalidates every cache breakpoint past the tool definitions and system prompt. The fix shipped July 10 in v1.91.2 (#32578, #32831, #32882), with regression tests that fail on pre-fix code.

We own this outcome entirely. The trigger was a poorly documented change in how new Claude models and Claude Code use system messages, but customers run a gateway precisely so they do not have to track provider quirks. Translating requests faithfully, including their caching semantics, is our core job and here we fell short. This post explains exactly what happened, why our testing and review failed to catch it, and what we have changed so this class of regression does not ship again.

Background

Three facts set up the incident:

1. Claude prompt caching is prefix based:

2. the pricing of tokens in increasing cost is:

1. cache read (0.1x)

2. normal write (1x)
3. cache write (1.25x for 5m ttl, 2x for 1h ttl)

2. when Claude Code makes a new request, the Bedrock provider checks if any previous request was a truncated prefix of the current request. If so, it reads from the cache only up to that point.

2. Mid-conversation system messages are new:

3. On May 28, 2026, Claude Opus 4.8 shipped as the first model accepting role: "system" entries inside messages (docs). This was documented on the Claude API docs on the same day.

4. Claude Code (v2.1.154) began emitting them on May 28, 2026, with no mention in its changelog.

5. Bedrock documented the same support by June 9, 2026 at the latest (the first archive.org capture; the page may have appeared as early as May 28).

6. Bedrock has two Anthropic APIs with different rules:

7. Converse requires all system content in a top-level field; LiteLLM has hoisted it there since December 2024 (#7037).

8. Invoke takes the native Anthropic Messages format, where models older than Opus 4.8 reject mid-conversation system entries with a 400 and newer models accept them.

What went wrong

1. After May 28, Claude Code sessions on Bedrock Invoke began failing with 400s mid-session when two things were true:

2. the model was older than Opus 4.8

3. the model is served under an alias

1. Claude Code detects capabilities by looking for version substrings like `opus-4-7` or `sonnet-4-6` in the model name.

2. For example, an alias like `bedrock-claude` contains none, so Claude Code assumes the newest feature set and always emits mid-conversation system messages.

2. An enterprise customer worked around the 400s with a local patch that hoisted every system entry from messages into top-level system, mirroring the Converse behavior, and asked us to upstream it. We shipped it as #31364 in v1.91.0 on July 4. 3. The hoist fixed the 400s but by always hoisting the system entries in messages, that would invalidate the entire messages cache whenever a new mid-conversation system message was written by Claude Code, which we measured to happen every ~3 turns on average. 4. This greatly decreased the cache hit rate and increased the cache write rate, increasing spend.

Detection and response

On July 8, an affected customer gave a detailed bug report of this regression. We reproduced it ourselves the same day end-to-end.

Three PRs fixed it, all released July 10 in v1.91.2 after extensive end-to-end testing, with regression tests that fail on pre-fix code:

1. #32578 disables mid-conversation system message hoisting on Invoke.
2. #32831 re-enables hoisting on models below Opus 4.8.
3. #32882 disables hoisting on Sonnet 5 and Fable 5, too.