Anthropic's hidden reasoning might be this
Disclaimer: I'm LLM curious/hobbyist and developer, not a researcher.
TL;DR: I think the hidden reasoning might be a diffusion step and/or a specialized model.
I'll explain why.
Pi displays the model's reasoning, and out of curiosity, I often read the reasoning trace when trying out new models - and most of it is "rambling" more than "reasoning".
When using Claude models like Opus 5.5 or Fable, I noticed it's visible reasoning trace is actually very calm and grounded - it's more like a summary of it's reasoning, rather than the actual reasoning.
I later learned that's exactly what it is - that the internal protocol includes base64 encrypted blobs between the summaries, which (for obvious reasons) Pi can't display.
So I've been wondering what exactly it might be doing between those steps, and I made two observations:
(1) when it's writing the visible summaries, it writes at the same tokens per second rate as the rest of the response, suggesting it is the main model doing the writing, and
(2) the pauses between the summaries are very short, and almost seem to have a fixed duration, maybe around a second - it's too short for the main model to write more than a few words at it's working rate.
If the main model was doing the reasoning at it's native tokens/second rate, there is no way it would have time to draft code or write any substantial amount of reasoning.
So what are they doing in what appears to be a fixed time step between the summaries?
And my guess is they trained a diffusion model to predict a fixed-length reasoning block - that is, you start with the main model doing the reasoning, the traditional approach, but then you train a smaller diffusion model to predict the reasoning traces, which can be done in practically a fixed time slot (fixed number of diffusion steps) e.g. around one second.
Once you have that working, whenever the main model emits a reasoning start token, you switch to the fast diffusion model - so instead of spending expensive output tokens on "reason rambling", you parse the diffusion model's output as input tokens, and the main model only uses expensive output tokens to write a summary. (the summaries are likely there not only to provide end user feedback, but to help ground the model between noisy reasoning steps.)
It might not be a diffusion step - that part is a guess, mostly based on observed timing and output token counts. It could be something else, like a draft model, a smaller specialized reasoning model, a heavily quantized model, who knows. Diffusion is my guess based on the observed timing and the reported output_tokens in the response, which couldn't have been written by the main model in the short intervals between the summaries.
Diffusion would also make it extremely easy to predictably scale reasoning depth, e.g. generate a larger block, take more diffusion steps, or both. Reasoning effort, from what I've seen, is extremely unpredictable in models where one model writes the response end to end - it's probability based afaik? e.g. the effort setting adjusts the probability of emitting the end reasoning token, which, if you've read the reasoning traces, often means the model spends an obscene amount of tokens reasoning in circles, repeating itself, debating itself about stopping, etc.
What do you think, is it plausible? :-)
I know about draft models, but are there any open models that literally switch to a second model during reasoning specifically? Or are most open models still based on the traditional "one model" approach? Is there any published research in that direction? Do LLM runtimes even support doing that, or are they all built around the idea of a single model?
As said, I'm not a researcher, just a curious hobbyist - if I'm obviously wrong, please feel free to educate me!
Cheers :-)
