How AI Reasoning Traces Can Be Extracted: A Two-Step Distillation and API Security Case Study

Introduction

AI model distillation has been one of the most sensitive topics in the LLM industry. When a new model suddenly becomes much stronger, or its answers begin to resemble the behavior of a leading model, researchers naturally ask whether some form of knowledge or reasoning transfer is taking place.

A recent 116-page research paper has brought a different kind of evidence into the discussion. The researchers report that hidden reasoning state exposed through certain API workflows could, under the conditions they studied, be recovered with the help of another model. The same research also raises a broader security concern: reasoning traces and agent trajectories can contain sensitive information if they are stored, shared, or replayed without sufficiently strict isolation.

The findings should be read as a research result rather than a final verdict on AI distillation. The authors did not claim that every similarity between models proves distillation. Instead, their experiments provide new evidence about how reasoning traces may be transferred, reproduced, and exposed.

Peeling Back an Opus Reasoning Process with Two API Calls

Researchers from MATS Research, the ELLIS Institute Tübingen, and other institutions investigated whether reasoning that is normally hidden behind an API could be recovered.

The reported setup used a stronger model to generate the reasoning state and a smaller model from the same model family to interpret it. In the example described by the source article, Opus 4.8 generated the reasoning while Haiku 4.5 was used to expose the hidden process.

The same general idea was also reported across other model ecosystems, including OpenAI and Google models. The important point is not that a smaller model is inherently capable of reading every stronger model's internal state, but that shared API infrastructure and compatible reasoning-state formats can create unexpected interaction surfaces.

The procedure described in the source article is surprisingly simple.

Step 1: Ask the stronger model to solve a problem

The first request asks Opus 4.8 to solve a problem, such as finding the largest prime factor of 8,139,881.

According to the source article, the API response contains two relevant pieces of information: a processed reasoning summary and a long string of apparently unreadable data. The article identifies the latter as an encrypted representation of the model's complete chain of thought.

Step 2: Replay the reasoning block to another model

The reported attack then places that encrypted block into a new request and switches the model to Claude Haiku 4.5.

The prompt asks the smaller model to continue and reproduce the reasoning associated with the supplied state inside a <thinking-copy> tag.

According to the reported experiment, Haiku was able to reproduce the hidden reasoning. The recovered trace reportedly included operations such as trial division, elimination, factorization, and verification.

The reported token counts were also striking. The source article says that the number of extracted reasoning tokens was close to the number of thinking tokens actually billed by the API, suggesting that the recovered material was not merely a short summary.

Why Could a Smaller Model Read the Stronger Model's Hidden State?

This is the central technical question.

Reasoning models often need to perform multi-step work. During an agentic task, a model may search the web, execute code, call tools, inspect results, and then continue reasoning from the resulting state.

That state has to survive between API calls.

One possible architecture is to keep the state on the client side rather than storing everything on the provider's servers. The model's reasoning state can be packaged into an encrypted block, returned to the client, and then supplied again in a later request. The server can decrypt the block and allow the model to continue from where it stopped.

This design can reduce server-side storage requirements and can support privacy-oriented workflows. But it also creates a security boundary that must be enforced carefully.

The reported issue was that the encrypted reasoning blocks were not always sufficiently bound to their original context. In the conditions studied by the researchers, this created three forms of interoperability:

1. Cross-session: A reasoning block from an earlier session could be reused in a new session.
2. Cross-user: A publicly exposed reasoning block could be submitted by another user.
3. Cross-model: A reasoning block produced by a stronger model could be processed by another model from the same provider.

These behaviors may have legitimate engineering motivations. Model switching, fallback systems, context compression, and long-running agent workflows all benefit from reusable state.

The problem is that the same interoperability can become an attack surface if the state is not tightly scoped to the original user, session, model, and authorization context.

A Low-Cost Decoder Can Change the Economics

Another important finding concerns cost.

The source article estimates that, using the cited Haiku 4.5 API pricing assumptions, decoding 10,000 reasoning trajectories could have a nominal cost of about $720 when each trajectory contains roughly 12,000 input and output tokens.

That changes the economics of reasoning-trace extraction. An attack that requires expensive access to a frontier model may become much more practical if the expensive model is only used to generate the original state and a cheaper model performs the decoding.

This is particularly relevant for model providers because reasoning traces can represent a substantial part of the value created during inference. If those traces can be recovered and reused outside their intended boundary, they may become a new form of model IP leakage.

The Distillation Mystery Gets Its First Pieces of Evidence

Once the researchers had access to more complete reasoning traces, they used them to investigate a broader question: can model-specific reasoning patterns reveal evidence of distillation?

The researchers ran two experiments.

Experiment 1: Reproducing Consecutive Reasoning Tokens

In the first experiment, the researchers selected 16 consecutive tokens from an Opus reasoning trace and asked several models to continue the same problem.

The idea was straightforward: if a model can reproduce the exact continuation much more easily than another model, that may indicate that it has learned something unusually close to the source model's reasoning patterns.

The reported differences were large.

One model would theoretically need around 10 billion attempts to reproduce the selected Opus phrase by chance. Two other models were estimated to require roughly 100 trillion and 10 quadrillion attempts, respectively.

In other words, one model reproduced the same Opus reasoning pattern up to about one million times more easily than the alternatives in the reported comparison.

Experiment 2: Seeding Only the Beginning of the Reasoning

The second experiment provided another model with only the beginning of an Opus reasoning trace and then observed how its response developed.

In one example, only five tokens were supplied.

Even this very small amount of information reportedly shifted the target model's wording, answer, and problem-solving rhythm toward the Opus trace. The reported similarity score increased from 0.17 to 0.33.

The researchers then expanded the test to 30 problems. According to the source article, 29 of those problems showed the same general effect: giving the model an Opus-style starting point made the later response more similar to Opus than when another model's starting point was used.

Does This Prove Distillation?

Not by itself.

The source article emphasizes that the researchers did not declare the distillation question settled. Similarity in outputs or reasoning patterns is evidence worth investigating, but it is not automatically proof that one model was trained directly on another model's private reasoning data.

What the experiments do provide is a more concrete way to study model-specific reasoning signatures now that longer reasoning traces can be analyzed.

GitHub and Public Agent Traces Create Another Security Problem

The issue is not limited to model companies.

The researchers reportedly collected 6,708 publicly available agent trajectories from GitHub and Hugging Face and reconstructed 315,320 reasoning blocks.

Among those trajectories, 328 contained at least one sensitive item, representing about 4.9% of the collected traces.

The source article reports that the researchers found:

• 62 API keys
• 33 passwords
• 24 access tokens
• 7 private keys
• 30 personal email addresses
• 130 personal names
• 36 postal addresses

This is a practical warning for developers building agent systems.

A trajectory may look like ordinary debugging output, but it can contain tool arguments, environment variables, credentials, personal information, internal URLs, or other data that should never become part of a public training set or repository.

If an agent trace is going to be logged, shared, or published, developers should treat it as potentially sensitive data rather than ordinary application logs.

常见问题

What are AI reasoning traces?

AI reasoning traces are intermediate states or reasoning-related outputs generated while a model works through a task. Depending on the API design, users may receive a summary, structured reasoning state, or an encrypted representation rather than the model's raw internal computation.

What is AI model distillation?

Model distillation is a technique in which a smaller or different model learns from the behavior of a larger or stronger model. It is widely used to transfer useful capabilities into models that are cheaper or faster to run.

Does recovering a reasoning trace prove that a model was distilled?

No. Recovering a reasoning trace can provide evidence about model similarity, but it does not by itself establish how a model was trained. Training-data provenance, controlled experiments, and additional evidence are needed to make a stronger attribution.

Why can encrypted reasoning state become a security risk?

Encryption protects the contents of a state block, but it does not automatically guarantee that the block is bound to the correct user, session, model, or authorization context. If those boundaries are weak, a valid encrypted state may potentially be replayed in an unintended context.

Why are agent trajectories sensitive?

Agent trajectories can include tool calls, prompts, environment variables, API credentials, personal data, internal file paths, and other information collected during execution. Publishing them without proper sanitization can therefore expose secrets even when the original application appears harmless.

Should developers publish raw AI agent traces on GitHub?

They should avoid publishing raw traces unless they have been carefully reviewed and sanitized. Secrets, tokens, credentials, personal information, private URLs, and proprietary data should be removed before traces are shared.

Is a smaller model always able to decode a larger model's reasoning?

No. The reported result depends on the specific API behavior, model family, reasoning-state format, and experimental conditions. It should not be interpreted as a general rule that smaller models can universally recover the reasoning of larger models.

相关工具

• Anthropic API Documentation: Official documentation for Claude APIs and model-related development.
• OpenAI Platform Documentation: Official documentation for OpenAI APIs and model integration.
• Google Gemini API Documentation: Official documentation for building with Gemini models.
• GitHub: A common platform for inspecting and sharing source code and agent traces.
• Hugging Face: A major platform for models, datasets, and machine learning research resources.

Related Links

• Anthropic Documentation: Official Claude API and developer documentation.
• OpenAI Documentation: Official OpenAI API documentation.
• Google AI for Developers: Official Gemini and Google AI developer resources.
• GitHub: Source-code hosting and collaboration platform relevant to public agent traces.
• Hugging Face: Model and dataset hub referenced by the source article.
• Source Reference on X: The reference post cited by the original article.

Summary

The reported research highlights a subtle but important security boundary around AI reasoning state. If reasoning blocks can be replayed across sessions, users, or models without strict authorization binding, a feature designed to support efficient state management can become an unexpected extraction channel.

The work also offers a new way to investigate model similarity and possible distillation by examining longer reasoning traces rather than relying only on final answers. At the same time, the findings are not a definitive verdict that any particular model was trained through unauthorized distillation.

For developers, the most immediate lesson is practical: treat reasoning state and agent trajectories as sensitive data, and enforce strict identity, session, model, and access boundaries before storing or sharing them.