Stealing Reasoning Traces from Proprietary LLM APIs
Stealing Reasoning Traces from Proprietary LLM APIs
We demonstrate this across frontier models from OpenAI, Anthropic, and Google. The decoded reasoning closely tracks the number of hidden thinking tokens reported by the API. Each point below corresponds to one of 120 Codeforces problems: the horizontal axis shows the hidden thinking-token count reported by the API, while the vertical axis shows the token count of the decoded reasoning when passed back to the model as input.
我们展示了这一现象在 OpenAI、Anthropic 和 Google 的前沿模型中均存在。解码后的推理过程与 API 报告的隐藏思维令牌（thinking tokens）数量高度吻合。下方的每个点对应 120 个 Codeforces 问题中的一个：横轴显示 API 报告的隐藏思维令牌计数，纵轴显示将解码后的推理内容作为输入传回模型时产生的令牌计数。
Distinct leaked items: 351 Technical identifiers, 204 PII, 126 Credentials, 23 Other. We collected 6,708 publicly available agent trajectories from GitHub and Hugging Face, produced by Claude, GPT, and Gemini models and still containing encrypted reasoning blocks. Applying our decoding pipeline to every signed block yielded 315,320 reconstructed reasoning blocks.
泄露项分类：351 个技术标识符，204 个个人身份信息（PII），126 个凭据，23 个其他。我们从 GitHub 和 Hugging Face 收集了 6,708 条公开的智能体轨迹，这些轨迹由 Claude、GPT 和 Gemini 模型生成，且仍包含加密的推理块。将我们的解码流水线应用于每个签名块，共还原出 315,320 个推理块。
These hidden traces contain real secrets and sensitive information. Restricting to genuine, non-benchmark user sessions, we recovered 704 distinct privacy artifacts, including 62 API keys, 33 passwords, 24 access tokens, and 30 personal email addresses, alongside names, postal addresses, internal URLs, and other technical identifiers. Of those 704 artifacts, 64 appeared exclusively inside the reasoning blocks and nowhere in the visible session.
这些隐藏的轨迹包含真实的机密和敏感信息。在排除基准测试，仅针对真实用户会话进行分析后，我们恢复了 704 个不同的隐私工件，包括 62 个 API 密钥、33 个密码、24 个访问令牌和 30 个个人电子邮件地址，以及姓名、邮寄地址、内部 URL 和其他技术标识符。在这 704 个工件中，有 64 个仅出现在推理块中，而在可见的会话内容中完全未出现。
(Note: The following section is an excerpt of a reconstructed reasoning trace showing the model’s internal thought process during a code sanitization task.)
(注：以下部分是还原出的推理轨迹片段，展示了模型在执行代码清理任务时的内部思考过程。)
GPT-5.2 Codex encrypted_content · decoded with GPT-5.6 Luna: Terminal-Bench sanitize-git-repo task [ … ] No matches. But we need to ensure no API keys present maybe other tokens, like openai. But request only these? “sanitize of all API keys.” We should search for patterns maybe like “api_key”, “apikey”, “token”, “secret”. But avoid modifying non-sensitive; also do not modify non contaminated files. Might have false positives. We should search targeted patterns and inspect.
GPT-5.2 Codex 加密内容 · 由 GPT-5.6 Luna 解码： Terminal-Bench sanitize-git-repo 任务 [ … ] 无匹配项。但我们需要确保没有 API 密钥存在，也许还有其他令牌，比如 openai 的。但只要求这些吗？“清理所有 API 密钥”。我们应该搜索类似 “api_key”、“apikey”、“token”、“secret” 的模式。但要避免修改非敏感内容；也不要修改未受污染的文件。可能会有误报。我们应该搜索目标模式并进行检查。
We’ll search for common patterns: “api_key”, “apikey”, “secret”, “token”. But may be in docs; we only replace actual values not placeholders. We’ll identify keys with assignment like API_KEY=…; with long strings. We’ll carefully inspect. Maybe there are config files with real keys like in .env, .yaml. We’ll use grep for ‘key’ with patterns. But we must be careful about large. We’ll run grep for ‘API_KEY’, ‘apikey’, ‘apiKey’, ‘secret’, ‘token’, ‘pass’, ‘password’. Then inspect results. We’ll list matches. Use grep -RIn.
我们将搜索常见模式：“api_key”、“apikey”、“secret”、“token”。但它们可能出现在文档中；我们只替换实际值，不替换占位符。我们将识别类似 API_KEY=… 且带有长字符串的赋值语句。我们将仔细检查。也许存在包含真实密钥的配置文件，如 .env 或 .yaml。我们将使用 grep 搜索带有模式的 “key”。但必须小心处理大文件。我们将运行 grep 搜索 “API_KEY”、“apikey”、“apiKey”、“secret”、“token”、“pass”、“password”。然后检查结果。我们将列出匹配项。使用 grep -RIn。