Cross-request KV cache exposure in the GLM-5.3 Workers AI model (155 responses)

Who Should Read

Backend and infra developers who self-host LLMs with open-source inference engines like SGLang, or use multi-tenant LLM APIs

Core Mechanics

• 155 responses exposed other customers' data
  • Between Aug 27 and Sep 9, 2026, 155 GLM-5.3 (@cf/zai-org/glm-5.3) responses were affected.
  • All were for prompts longer than 681,280 tokens.
  • No other Workers AI model was affected.
• Context window exceeded the KV pool
  • The model was deployed with a 1,048,576-token context window.
  • The prefill GPUs' KV cache pool held only 681,280 tokens.
  • Cloudflare knew about the gap before launch but assumed the engine would return an error.
• Rejected requests flowed through as normal
  • SGLang rejected prompts that didn't fit and swapped them for a one-token placeholder.
  • In disaggregated prefill/decode mode, admission queues checked only finished(), so rejected requests were admitted.
  • On a single node the client correctly got an error.
• Decode read stale cache pages
  • Decode reserved 8 pages, but prefill wrote and sent only 1.
  • Pages P1–P7 still held earlier users' prompts and partial generations.
  • The client got HTTP 200 and a coherent reply instead of HTTP 400.
• KV pages are not zeroed between requests
  • The next request just overwrites them, which is good for performance.
  • The engine must therefore track page ownership correctly.
• Tests only checked HTTP status codes
  • Pre-launch long-context tests checked success codes and performance.
  • They did not validate output coherence.

Community Reactions

• Praise for the RCA and fast response
  • One commenter liked this RCA more than most for explaining why testing missed the bug.
  • They also liked that the model was pulled quickly.
• Question about advertised context length
  • One commenter asked whether it was advertised as 1,048,576 tokens when only 681,280 fit.
• Experience of unrelated responses elsewhere
  • One commenter said they had occasionally received completely unrelated responses from some provider's API, which this might explain.
• Shared tenancy is inherently risky
  • One commenter said tokens aren't magically isolated on shared tenants.
  • They argued enterprises with sensitive data should use non-shared hosts.

How to Apply

• Check and upgrade your SGLang version
  • Today, check the SGLang version on your servers.
  • Upgrade if it is below v0.5.22.
• Compare KV pool size with context length before deploy
  • Read the KV pool size from startup logs or the exposed metric.
  • Block the deployment if it is smaller than the model's context length.
• Add canary markers to long-context tests
  • Send limit-length prompts and check response content, not just status codes.
  • Verify that unique strings planted in other requests never appear in responses.
• Review isolation level for sensitive data
  • Check whether your LLM API runs on shared hosts.
  • Consider dedicated hosts for sensitive data.

Try It

# Paste into Claude or ChatGPT — a simplified pre-deploy checklist based on the incident

We serve {{model name}} with SGLang in disaggregated prefill/decode mode.
Context length: {{context length in tokens}}
Prefill KV pool size: {{KV pool tokens from startup log}}

Using the incident below, build a pre-deploy checklist.
- The context window was larger than the KV pool, and requests that didn't fit should have been rejected but were passed to decode as normal, reading earlier requests' cache pages.
- Tests missed it because they only checked HTTP status codes.

Include: comparing pool size to context length, how to validate response content for limit-length requests, and a canary-string leakage test across requests.

Next step: ask it to turn the checklist into a CI script.

Terminology

Related Papers

• Microsoft Decision-1: 9B Qwen Model Launch Explained

  Microsoft's 9B Qwen-based model returns only per-option probabilities for routing and classification.

• Cerebras Deploys Trillion-Parameter AI Models

  Cerebras fits Kimi K3 (3T) across 59 wafers in a pipeline, serving 1000-2000+ tokens per second

• A practical guide to AI image generation, from diffusion to agents

  Testing image APIs showed you pay per canvas, and camera-style prompts beat describing the view

• DeepInfer: High-performance inference framework for DeepSeek models

  DeepSeek open-sourced DeepInfer, its production inference engine, under Apache-2.0.

• Mercury 3: Diffusion at scale

  Scaling diffusion LLM 40x gives Mercury 3: 4–8x faster than speed-tier models, similar quality

• DeepSeek banned from U.S. federal civilian agencies under new DHS order

  DHS order bans DeepSeek from U.S. federal civilian agency IT systems

• Timestral: Mistral's time-series foundation models

  A 209M time-series model ranks first on three benchmarks and runs on CPU.

• DeepSeek V4.1 Flash 1017 (Max)

  DeepSeek V4.1 Flash 1017 Max scores 45 on the Intelligence Index at 211 tok/s and $0.30 input.

Related Resources

• https://blog.cloudflare.com/kv-cache-exposure-remediation/