Incident Report: Encrypted Content Failures in Multi-Region Responses API Load Balancing

LiteLLM incident report. Published 2026-02-24.

Date: Feb 24, 2026 Duration: Ongoing (until fix deployed) Severity: High (for users load balancing Responses API across different API keys) Status: Resolved

Summary

When load balancing OpenAI’s Responses API across deployments with different API keys (e.g., different Azure regions or OpenAI organizations), follow-up requests containing encrypted content items (like rs_... reasoning items) would fail with:

{
  "error": {
    "message": "The encrypted content for item rs_0d09d6e56879e76500699d6feee41c8197bd268aae76141f87 could not be verified. Reason: Encrypted content organization_id did not match the target organization.",
    "type": "invalid_request_error",
    "code": "invalid_encrypted_content"
  }
}

Encrypted content items are cryptographically tied to the API key’s organization that created them. When the router load balanced a follow-up request to a deployment with a different API key, decryption failed.

• Responses API calls with encrypted content: Complete failure when routed to wrong deployment
• Initial requests: Unaffected. Only follow-up requests containing encrypted items failed
• Other API endpoints: No impact. Chat completions, embeddings, etc. functioned normally

Background

OpenAI’s Responses API can return encrypted “reasoning items” (with IDs like rs_...) that contain intermediate reasoning steps. These items are encrypted with the organization’s key and can only be decrypted by the same organization’s API key.

When load balancing across deployments with different API keys, the existing affinity mechanisms were insufficient:

• responses_api_deployment_check: Requires previous_response_id which some clients (like Codex) don’t provide
• deployment_affinity: Too broad. Pins all requests from a user to one deployment, reducing effective quota by the number of users
• session_affinity: Requires explicit session IDs and still reduces quota

Root Cause

LiteLLM’s router had no mechanism to track which deployment created specific encrypted content items and route follow-up requests accordingly. The router treated all deployments as interchangeable, leading to decryption failures when encrypted content crossed organizational boundaries.

The Problem Flow:

1. User calls router.aresponses() with model gpt-5.1-codex
2. Router load balances to Deployment A (Azure East US, API Key 1)
3. Response contains encrypted reasoning item rs_abc123 (encrypted with Org 1’s key)
4. User makes follow-up request with rs_abc123 in the input
5. Router load balances to Deployment B (Azure West Europe, API Key 2)
6. Deployment B tries to decrypt rs_abc123 with Org 2’s key → fails

Why Existing Solutions Didn’t Work:

• previous_response_id: Not provided by all clients (e.g., Codex)
• deployment_affinity: Pins all user requests to one deployment → reduces quota to 1/N where N = number of deployments
• session_affinity: Requires explicit session management and still reduces quota

Timeline:

1. Users configured multi-region Responses API load balancing with different API keys
2. Initial requests succeeded, but follow-up requests with encrypted content failed intermittently
3. Error rate correlated with number of deployments (more deployments = higher chance of routing to wrong one)
4. Investigation revealed encrypted content was organization-bound
5. Existing affinity mechanisms deemed unsuitable (quota reduction, missing previous_response_id)
6. New solution designed and implemented: encrypted_content_affinity

The Fix

Implemented a new encrypted_content_affinity pre-call check that intelligently tracks encrypted content and routes follow-up requests only when necessary.

Implementation

1. Encoding model_id into output items (responses/utils.py)

The same approach used for previous_response_id affinity, with no cache needed. When a response contains output items with encrypted_content, LiteLLM encodes the originating deployment’s model_id in two places for redundancy:

1. Into the item ID (if present): rs_abc123 → encitem_{base64("litellm:model_id:{model_id};item_id:rs_abc123")}
2. Into the encrypted_content itself: Wraps the content with litellm_enc:{base64("model_id:{model_id}")};{original_encrypted_content}

# Encoding item IDs (when present)
def _build_encrypted_item_id(model_id: str, item_id: str) -> str:
    assembled = f"litellm:model_id:{model_id};item_id:{item_id}"
    encoded = base64.b64encode(assembled.encode("utf-8")).decode("utf-8")
    return f"encitem_{encoded}"

# Wrapping encrypted_content (always, for redundancy)
def _wrap_encrypted_content_with_model_id(encrypted_content: str, model_id: str) -> str:
    metadata = f"model_id:{model_id}"
    encoded_metadata = base64.b64encode(metadata.encode("utf-8")).decode("utf-8")
    return f"litellm_enc:{encoded_metadata};{encrypted_content}"

Why wrap encrypted_content directly? Some clients (like Codex) don’t consistently send item IDs in follow-up requests, but they always send the encrypted_content itself. By embedding model_id into the content, affinity works even when IDs are missing.

Streaming responses: The wrapping logic is applied to both:

• Final response objects (non-streaming)
• Individual streaming events (response.output_item.added, response.output_item.done)

This ensures clients receiving streaming responses get wrapped content they can send back.

Before forwarding to the upstream provider, LiteLLM restores the original item IDs and unwraps encrypted_content so the provider never sees the encoded form:

# In responses/main.py — before calling the handler
input = ResponsesAPIRequestUtils._restore_encrypted_content_item_ids_in_input(input)

2. EncryptedContentAffinityCheck, routing only (encrypted_content_affinity_check.py)

No async_log_success_event or cache lookups; the model_id is decoded directly from the item ID or encrypted_content:

class EncryptedContentAffinityCheck(CustomLogger):
    async def async_filter_deployments(self, model, healthy_deployments, ...):
        """Extract model_id from input items (ID or encrypted_content) and pin to that deployment."""
        for item in request_kwargs.get("input", []):
            # Try to extract model_id from two sources:
            model_id = self._extract_model_id_from_input(item)

            if model_id:
                deployment = self._find_deployment_by_model_id(
                    healthy_deployments, model_id
                )
                if deployment:
                    request_kwargs["_encrypted_content_affinity_pinned"] = True
                    return [deployment]
        return healthy_deployments

    def _extract_model_id_from_input(self, item: dict) -> Optional[str]:
        """Extract model_id from either encoded ID or wrapped encrypted_content."""
        # 1. Try decoding from item ID (if present)
        item_id = item.get("id", "")
        if item_id:
            decoded = ResponsesAPIRequestUtils._decode_encrypted_item_id(item_id)
            if decoded:
                return decoded["model_id"]

        # 2. Try unwrapping from encrypted_content (fallback for clients that omit IDs)
        encrypted_content = item.get("encrypted_content", "")
        if encrypted_content and encrypted_content.startswith("litellm_enc:"):
            model_id, _ = ResponsesAPIRequestUtils._unwrap_encrypted_content_with_model_id(
                encrypted_content
            )
            return model_id

        return None

3. Rate Limit Bypass (router.py)

When encrypted content requires a specific deployment, RPM/TPM limits are bypassed (the request would fail on any other deployment anyway):

# In async_get_available_deployment, after filtering healthy deployments:
if (
    request_kwargs.get("_encrypted_content_affinity_pinned")
    and len(healthy_deployments) == 1
):
    return healthy_deployments[0]  # Bypass routing strategy (RPM/TPM checks)

3. Configuration

router_settings:
  routing_strategy: usage-based-routing-v2
  enable_pre_call_checks: true
  optional_pre_call_checks:
    - encrypted_content_affinity
  deployment_affinity_ttl_seconds: 86400  # 24 hours

Key Benefits

✅ No quota reduction: Only pins requests containing encrypted items ✅ Bypasses rate limits: When encrypted content requires a specific deployment, RPM/TPM limits don’t block it ✅ No previous_response_id required: Works by encoding model_id directly into the item ID ✅ No cache required: model_id is decoded on-the-fly from the item ID, so no Redis and no TTL ✅ Globally safe: Can be enabled for all models; non-Responses-API calls are unaffected ✅ Surgical precision: Normal requests continue to load balance freely

Remediation