Key takeaways

Latent reasoning means the model does not write its intermediate thoughts as tokens. It feeds the last-layer hidden state back in as the next input and keeps thinking internally. CoT (chain of thought) leaves readable text behind, while latent reasoning can use fewer tokens but is harder to inspect.

This post explains what latent reasoning is, how it differs from CoT, and how a model that does not think in text can still call tools. It was prompted by the paper “Hybrid Reasoning Agents”, discussed on Hacker News on 2026-10-19 (KST). Claims specific to the paper are marked as “the paper claims”, and no numbers or benchmarks are covered.

What is latent reasoning?

One-line answer: The model skips writing intermediate thoughts as tokens and instead feeds the last-layer hidden state back in as the next input, continuing to think in an internal vector space.

A hidden state is the vector the last Transformer layer outputs for one position. The paper describes two ways of producing the next input. In text mode, the model takes the last-layer hidden state, applies a softmax over the vocabulary to pick one token, and that token’s embedding becomes the next input. In latent mode, no token is picked and the hidden state itself is reused as the next input. That is standard latent reasoning, and Coconut-style methods are prior work in this line.

If you want a refresher on tokens, see this post on tokens and context windows.

Text mode vs latent mode loop

How is it different from CoT?

One-line answer: CoT leaves thoughts as readable tokens but spends output tokens on every step, while latent reasoning can use fewer tokens but is harder to read and audit.

CoT (chain of thought) writes intermediate steps out as tokens. People can read those steps, but each step costs output tokens. Latent reasoning does not spell out intermediate steps, so it can use fewer tokens, yet those steps are not directly readable text, which makes supervision and auditing harder. How to write CoT prompts is not covered here; see the AI Day4 prompt engineering post.

AspectCoTLatent reasoning
Form the thinking takesTokens (text)Hidden states (vectors)
Readable by peopleYesHard to read directly
Token cost (qualitative)Output tokens spent on every stepCan use fewer, since intermediate steps are not written out
Tool callsCan be generated directly as textHard, because tool interfaces are text

The table lists general facts. Claims specific to the paper are separated below with “the paper claims”.

How can tools be called if the model does not write its thoughts?

One-line answer: Tools are called through text interfaces, so the paper claims its HybridAgent thinks in latent space and acts (calls tools) through text.

Because tools are invoked through text interfaces, a purely latent reasoner has trouble calling them. The paper’s abstract makes the same point: today’s LLM agents reason and act through text, and latent reasoning is an efficient internal medium but lacks native tool access and transparency. I checked this against the Hacker News thread (posted 2026-10-19 KST) and the arXiv paper.

The paper’s HybridAgent thinks in latent space and acts through text, so tool calls such as running Python or searching are generated as text.

To move between modes, the paper adds a lightweight adapter called HybridLink. The paper claims it maps the last-layer hidden state back into the input embedding space so the model can switch modes. Special anchor tokens mark where latent mode starts and ends.

Multi-turn context is carried forward as KV entries, which the paper calls working memory. The KV cache itself is covered in the AI Day3 LLM inference and KV cache post.

Training runs in two stages, HybridSFT and then HybridRL. HybridSFT is a supervised fine-tuning stage, and the paper claims HybridRL uses reinforcement learning to learn when to think in latent space and when to act through text. For what SFT and RL mean, see the post on pretraining, fine-tuning and RLHF.

Latent thinking and text action timeline

Can latent thoughts be inspected?

One-line answer: The paper claims each latent step can be read as pseudo-text by mapping it to its nearest token embedding, but in general latent thoughts are hard to read directly.

The paper claims that mapping each latent step to its nearest token embedding lets you read the step as pseudo-text. That is the paper’s claim. In general, latent steps leave no written trace, so they are harder to supervise and audit. Research on reading model internals more broadly is covered in the mechanistic interpretability post.

What are the frequently asked questions?

Does latent reasoning remove the need for CoT prompts?

No, they sit at different layers. A CoT prompt is about how you write the input, while latent reasoning is about how the model generates internally. For the prompting side, see the prompt engineering post.

Does the chatbot I use today do latent reasoning?

It cannot be known from the outside. The paper discussed here is a research-stage proposal.

What is a hidden state?

It is the vector the last Transformer layer outputs for a position.

What are the sources?