The thinking you never see

You ask a local model something simple on an M4 with LM Studio and nothing happens. A spinner. Maybe the fans spin up. Thirty seconds, sometimes a minute. Then an answer appears that a dumber model would have given you in three. Your instinct is that something is broken — the model is stuck, the queue is jammed, the machine is swapping. Almost always it is none of those. The model is thinking: generating text you cannot see, and the pause is that text being written.

Three definitions before anything else.

Token. The unit a language model reads and writes in — a chunk of text, usually a word or part of a word. You are billed per token, rate-limited per token, and the model's speed is measured in tokens per second. It is not a character and not a word, and it is the only unit any number in this lecture uses.

Chain-of-thought (CoT). A series of intermediate steps written before an answer: the packs hold three pens each, twelve pens is four packs, four times seven is twenty-eight. Written down, those steps are just more tokens. Most explanations make this sound mystical; it is not. A chain of thought is output. It costs what output costs because it is output.

Reasoning model. A model trained to always produce a chain of thought before its answer, and to keep that chain in a separate channel from the answer. The channel has different names in different products, which is half the confusion:

I verified the first four families' docs this run; the Grok row is named from the product and I did not fetch xAI's documentation, so it carries no link. Treat it as a name, not a citation.

The contrast that matters is with a plain model — one that answers directly, in a single pass, with no separate thinking channel. Ask it the pens question and it emits the answer immediately, or a wrong answer immediately. It has no scratch phase to hedge in. That is not a defect; it is a different shape of computation, and for most of what an agent stack does all day — classify this, extract that, summarise this page — the direct shape is correct and cheap.

Imagine a person doing long division. They could do it in their head and say the answer, or reach for scratch paper, work the columns, cross something out, start again, then say the answer. The paper makes them likelier to be right — not because paper is intelligent, but because the steps are somewhere they can be looked at.

A reasoning model's thinking tokens are that scratch paper. That is a metaphor and it stops here: it does not tell you the paper is billed, that it can be thrown away before you read it, or that the model will sometimes skip the paper entirely. Those are separate facts, and they are below.

The mechanism, without the metaphor: the model is a next-token predictor, and each token is a forward pass with a fixed amount of computation — the same amount whether the next token is “the” or “therefore the answer is”. A model asked to produce a correct answer in one token is being asked to compress arbitrary arithmetic into one fixed-size step. A chain of thought spreads that arithmetic over hundreds of cheap steps. Reasoning models are the ones trained to do this always, in a channel that is not the answer.

Two ways a model ends up thinking

This is the distinction most people collapse, and it matters for cost.

Prompted chain of thought. Wei et al. (2022) showed that putting worked examples in the prompt — question, then steps, then answer — makes the model copy the pattern, and accuracy on arithmetic and symbolic reasoning jumps; eight exemplars took a 540B-parameter model to state of the art on GSM8K [2]. Kojima et al. (2022) showed you do not even need exemplars: appending “Let's think step by step” is enough [3]. This is prompting, and the steps land in the visible answer, because there is nowhere else for them to land.

Native reasoning. A reasoning model is trained to think first. DeepSeek's R1 is the cleanest public account: large-scale reinforcement learning on a base model, with no supervised examples of reasoning at all, rewarded for verifiable correctness — and long chains of thought emerged, along with self-verification, reflection and strategy switching [1]. That first model, R1-Zero, was unreadable and mixed languages; the shipped R1 adds a cold-start stage to fix readability. OpenAI describes the same mechanism: through reinforcement learning, o1 “learns to hone its chain of thought and refine the strategies it uses… it learns to recognize and correct its mistakes” [7].

The practical difference: prompted steps are in the answer — you read them, you pay for them as output, and the model has no separate budget for them. Native thinking is in its own channel — it can be hidden, summarised, truncated, encrypted, carried across turns or budgeted, and it is billed even when none of it reaches you (§05).

“Reasoning model” tells you nothing about whether you will see the reasoning. There are four states, and the same model can be in different ones depending on the client.

1. Visible, in tags. Open-weight reasoning models often emit thinking inside <think>…</think>. LM Studio's write-up on DeepSeek R1 describes it: the response opens with a think token, then “a long stream of consciousness, where the model is working through the problem step by step”, then a closing token, then the answer [13]. If your client does not special-case the tags, the scratch paper gets printed into your chat.

2. Exposed as a separate field. Ollama returns chat with message.thinking and message.content as separate fields, and generate with thinking and response [10]. LM Studio 0.3.9 added an experimental API that returns DeepSeek's reasoning in a reasoning_content field, enabled in App Settings → Developer [14]. That field name is the one you meet across providers.

3. Summarised. Anthropic's API returns thinking blocks ahead of the text blocks, but what is in them “is never the raw chain of thought: the text in a thinking block is a summary of Claude's reasoning” [8]. Each block also carries a signature, an encrypted copy of the full reasoning to pass back unchanged. A display setting decides whether the summary appears at all: "summarized" returns it, "omitted" returns blocks with an empty thinking field — and “either way the block is billed the same”.

4. Omitted entirely. OpenAI shows no raw chain of thought. Their stated reasons, in the o1 announcement, are the industry's actual argument: they want the freedom to monitor the chain of thought for signs of manipulation, which requires letting the model think in unaltered form; they do not want an unaligned chain visible to users; and they weigh “user experience, competitive advantage, and the option to pursue the chain of thought monitoring” [7]. What they show instead is a model-generated summary, and they “strive to partially make up for it by teaching the model to reproduce any useful ideas from the chain of thought in the answer.”

Notice what is not on that list: cost. Hiding the reasoning is not a billing trick. You are billed either way — OpenAI's docs say reasoning tokens “are not visible via the API” and “are billed as output tokens” [6]. Hiding is about readability, distillation (a visible chain is training data for a competitor), and monitoring.

Everything below ran on this box.

The setup

$ ollama --version
ollama version is 0.32.1
$ ollama pull qwen3:0.6b
$ ollama list
NAME          ID              SIZE      MODIFIED
qwen3:0.6b    7df6b6e09427    522 MB    Less than a second ago

Confirm it is a thinking model before you time it. Ollama reports capabilities rather than asking you to guess:

$ ollama show qwen3:0.6b
  Model
    architecture        qwen3
    parameters          751.63M
    context length      40960
    quantization        Q4_K_M
  Capabilities
    completion
    tools
    thinking

That thinking line is the answer to “is this a reasoning model?” — one command, one word. (751.63M is the parameter count; Q4_K_M is a 4-bit quantization, a compressed numeric format that trades a little quality for a lot of memory. Another lecture's topic; here it only explains the speed.)

The request is a one-line change, think: true or think: false:

curl http://localhost:11434/api/chat -d '{
  "model": "qwen3:0.6b",
  "messages": [{"role":"user","content":"A shop sells pens in packs of 3 for 7 dollars. How much do 12 pens cost? Reply with just the number."}],
  "think": true,
  "stream": false
}'

The measurement

Same prompt, temperature 0, seed 42, three warm runs each (the model is loaded before timing starts, so load time is not in these numbers).

7.9× the output tokens. 9.3× the wall clock. The same number.

Two honest caveats. First, the answers are not byte-identical — the thinking run restated the arithmetic (“determine how many packs… four packs… 4 × 7”) while the direct run answered in two lines, so some extra tokens are a wordier answer rather than thinking. Second, thinking length varied between runs of the same prompt: 1,111 characters in the timed runs, 2,506 in the streaming run below. A reasoning model's cost is not a fixed property of the question. It is a distribution, and that distribution is why thinking budgets exist (§07).

The part that looks like a hang

Stream the same request and timestamp the chunks. A chunk carrying message.thinking is invisible to a client that renders only message.content.

first thinking chunk at 0.58s
first ANSWER chunk at 50.51s   (i.e. 49.9s of visible silence)
thinking chars: 2506 | answer chars: 2
total wall: 50.60s

The model started working half a second after enter, worked for fifty seconds, and your chat window showed nothing the whole time — because the only thing it knows how to display is the answer, and the answer did not start until second 50.5. That is the entire “why is it hanging” mystery, and it is not a bug: the thinking channel is not the answer, and the client renders the answer.

On an M4 with a Q8 9B model the arithmetic is worse. A 9B model has more than ten times the parameters of this 0.6B one, and Q8 uses twice the bits of Q4, so per-token generation is several times slower — before counting that a better model may also think longer. The pause you asked about was all three effects at once.

Two sentences are the whole economics of this lecture.

“While reasoning tokens are not visible via the API, they still occupy space in the model's context window and are billed as output tokens.”

OpenAI · Reasoning models guide §“Reasoning effort” · fetched 2026-11-01

“Thinking has a cost: the tokens Claude spends reasoning are billed as output tokens, even when the thinking text isn't returned to you, and they count toward max_tokens alongside the response text.”

Anthropic · Thinking documentation · fetched 2026-11-01

Three consequences are packed into those two sentences.

1. Price. Thinking is billed at the output rate — the expensive rate, typically several times the input rate. (I am not quoting prices; they change monthly. The ratio is what matters and it is stable.)
2. Latency. Thinking tokens are generated one at a time at the model's tokens-per-second rate, before the answer starts. §04 measured 9.3×.
3. The ceiling. Thinking counts against max_tokens / max_output_tokens — the same ceiling as the answer. A big enough think can eat the whole allowance.

The failure mode nobody warns you about

That third consequence has a specific, ugly shape, and OpenAI documents it:

“If the generated tokens reach the context window limit or the max_output_tokens value you've set, you'll receive a response with a status of incomplete … This might occur before any visible output tokens are produced, meaning you could incur costs for input and reasoning tokens without receiving a visible response.”

OpenAI · Reasoning models guide · fetched 2026-11-01

You paid for the thinking, the model ran out of room before writing the answer, and you got nothing. Their recommendation: reserve “at least 25,000 tokens for reasoning and outputs” when experimenting. The habit: when a reasoning call returns an empty or half-finished answer, check status before blaming the model.

The second cost: thinking occupies context

Reasoning tokens are not just billed; they take up room — they “occupy space in the model's context window”. Anthropic goes further for multi-turn work: Claude Opus 4.5 “and models numbered 4.6 and higher keep prior turns' thinking blocks in context and bill them as input”, where earlier models stripped them [9]. So on a long agent run, thinking costs twice: once as output when generated, again as input every time the conversation is re-sent. If you have read 301-02 (prompt caching) this is the same mechanism from the other side.

The bill, as a formula

bill = (input tokens × input rate)
     + (visible answer tokens × output rate)
     + (thinking tokens × output rate)

with visible answer + thinking fitting under max_tokens. For the pens question, ignore input (39 vs 45 prompt tokens, near-identical) and compare output only. At any per-output-token rate R:

• direct answer: 66 × R
• thinking answer: 522 × R = 7.9× the cost for the same $28

What matters to someone running a stack is not the multiplier on one call. It is that a scheduled job making 200 such calls pays the multiplier 200 times, and the visible output — the thing in the log — looks identical.

Training-time compute is what a lab spends once: GPUs, weeks, a fixed model at the end. Inference-time compute is what you spend each call — normally one forward pass per output token, the same cost for an easy question and a hard one. A reasoning model breaks that symmetry on purpose, spending more inference compute on hard questions by generating more thinking tokens. That is test-time compute (“test time” is the research name for the moment you use the model). The claim behind every reasoning model is: for some problems, more thinking beats a bigger model.

OpenAI's o1 announcement states the empirical version: performance “consistently improves with more reinforcement learning (train-time compute) and with more time spent thinking (test-time compute)”, with accuracy rising smoothly as test-time compute increases [7].

The obvious next thought is “then think maximally, always”. That is wrong. Snell et al. (2024) found the effectiveness of different ways of spending a fixed amount of inference compute “critically varies depending on the difficulty of the prompt”, which motivates a compute-optimal strategy — allocate thinking where it pays [4]. On easy prompts extra thinking is waste; on hard ones it is the whole gain. That is the justification for an effort dial defaulting to something in the middle rather than maximum.

The s1 paper (2025) shows the opposite lever, “budget forcing”: you can force a model to stop thinking early, or to think longer by appending “Wait” when it tries to end, which makes it double-check and often fixes a wrong step [5]. Their 32B model went from 50% to 57% on AIME24 purely by forcing more thinking. Thinking length is a dial you can turn from outside the model.

One more fact that keeps this honest. DeepSeek's R1 card reports that the series “tend to bypass thinking pattern (i.e., outputting \n\n) when responding to certain queries, which can adversely affect the model's performance”, and recommends forcing a newline at the start so the model actually thinks [12]. A trained-in thinking habit is a strong tendency, not a guarantee. If you are measuring cost, sample more than once.

Every provider that ships reasoning also ships a way to turn it down — an admission that thinking is a cost knob.

OpenAI: reasoning.effort

Their guide describes it as guiding “how much to think when performing a task”, with model-dependent values that can include none, minimal, low, medium, high, xhigh, max [6]. Their own table is the clearest statement of the trade:

Two details worth carrying: none is rejected with HTTP 400 on some models (GPT-6 Astra does not support it), and defaults are per-model rather than universal (gpt-5.5 defaults to medium). “Set effort to low” is not a portable instruction. Also, for perceived latency they suggest asking the model to emit a short preamble before deep reasoning — the same trick a chat UI uses when it streams thinking.

Anthropic: budget, then effort

Older Claude models took an explicit budget:

thinking: {type: "enabled", budget_tokens: 10000}

with rules that are the shape of every budget knob: minimum 1,024 tokens (the API rejects less), must be less than max_tokens because thinking counts toward it, and the budget “is a target rather than a strict cap” — Claude may stop well before it, and max_tokens is the hard ceiling [9]. Their tuning advice is the compute-optimal argument in practice: start near 1,024 for simple tasks, 16,000+ for complex ones, expect diminishing returns.

Newer models moved to adaptive thinking plus output_config.effort; budget_tokens is deprecated on 4.6 and returns a 400 on 4.7-and-later. The behavioural difference is the interesting part: with a fixed budget “Claude thinks on every request”, whereas with adaptive thinking it decides whether to think at all and at low effort “might skip thinking entirely on easy inputs”.

One trap nobody expects, from the same page: changing the thinking configuration between requests invalidates cache breakpoints, because the budget value is rendered into the prompt. If you tune the dial per request, you pay to re-warm the cache every time.

Ollama and local models: think

Locally the dial is cruder and more honest: think: true, think: false, or a named level, with /api/show reporting what a model supports ({"thinking": {"values": ["low","medium","high"], "default": "medium"}}) [10]. The docs are explicit about edge cases: values: [false] means the model does not support thinking, and if the thinking object is omitted entirely the model “might conduct thinking based on its existing behavior”. That is the local-stack version of DeepSeek's bypass note: absence of a knob is not evidence of absence of thinking.

delegation:
  backend: tiers
  backends:
    tiers:
      tiers:
        - {name: reasoner, provider: openrouter, model: x-ai/grok-4.7, reasoning_effort: high}

One file, four tiers, and the design decision this lecture is about.

~/.hermes/plugins/delegation-tiers/__init__.py defines:

DEFAULT_TIERS = [
    {"name": "scout",    "model": "openai/gpt-4.1-nano",        "description": "Cheap bulk retrieval, parsing, classification, vision/OCR."},
    {"name": "reader",   "model": "anthropic/claude-haiku-4.5", "description": "Document reading, long-context extraction, structured summaries."},
    {"name": "reasoner", "model": "x-ai/grok-4.7",              "description": "Critical analysis, thesis stress-testing, invalidation logic."},
    {"name": "writer",   "model": "anthropic/claude-sonnet-4.6","description": "Prose, memos, journal entries in Mo's voice."},
]

_ROUTE_KEYS = ("provider", "model", "base_url", "api_key", "api_mode",
               "reasoning_effort", "service_tier", "request_overrides")

Two things to notice.

The tier names are task shapes, not sizes. scout is bulk retrieval, reader extraction, writer prose in Mo's voice, reasoner “critical analysis, thesis stress-testing, invalidation logic”. Only the last is a task whose answer quality depends on intermediate work — precisely the condition under which test-time compute pays (§06). The other three are tasks where more thinking buys nothing and costs everything.

reasoning_effort is a route key, sitting next to model and provider — evidence that effort is a routing decision, like which model to use. A reasoner tier running grok-4.7 at low is a different thing from the same tier at high, and the config format can express that.

Confirming what is actually running

$ grep -n "reasoning_effort" ~/.hermes/config.yaml
$ grep -n "backends:" ~/.hermes/config.yaml        # no output = plugin defaults apply
$ hermes config get model.default

On this box right now: config.yaml line 2-3 is model.default: deepseek-flash / provider: deepseek; line 63 is agent.reasoning_effort: medium; line 378 is delegation.model: deepseek-flash; line 386 is delegation.reasoning_effort: ''; line 391 is delegation.backend: tiers. And the backends: grep returns nothing, so DEFAULT_TIERS — including x-ai/grok-4.7 for reasoner — is what delegations actually use.

The honest disagreement

The audit brief says “deepseek-v4-pro is the reasoning-heavy default”. config.yaml line 2 says the default model is deepseek-flash. Both are true, about different things:

• The audit describes which model Mo reaches for when reasoning matters — this week's market brief ran its orchestrator on deepseek-v4-pro. The same audit records why the global default moved: deepseek-v4-pro advertises input_modalities: ["text"], deepseek-flash advertises ["text","image"], and the default was flipped to flash for native vision. That is 301-04's lecture.
• config.yaml describes the line the runtime reads at startup. Confirm it with hermes config get model.default; /models shows what each model advertises about itself.

The lesson generalises: “the reasoning model” is not a property of the stack. It is a property of this call, with this model, at this effort. When debugging cost, read the config, not the intention.

They share a mechanism — spreading computation over more tokens beats compressing it into one — but differ in control: only native reasoning gives you a knob that does not pollute the answer.

Two corollaries from the sources:

• Prompting a reasoning model to “think step by step” can backfire. DeepSeek's R1 card recommends no system prompt at all — all instructions in the user message — and reserves “Please reason step by step” for maths problems specifically [12]. Stacking a prompt-level CoT instruction on a model that already reasons natively adds tokens and can fight its trained format.
• Prompted CoT is still the right tool sometimes. On a cheap non-reasoning model, “show your work” costs a few hundred output tokens and often fixes the answer: a poor man's reasoning pass on a model with no reasoning channel.

1. Two runs, same answer, different bill. A cron job that produces a one-paragraph summary produces the same paragraph whether the model thought for 66 tokens or 522. The artifact, the log line, the diff — identical. The bill is 7.9× different, and nothing in the output says so. Cost accounting for agents has to include a number the user never sees: log eval_count (or usage.output_tokens_details), not just the answer.

2. The “hang” that was never a bug. Fifty seconds of silence on a local model is not a stall, a crash or a swap storm; it is 2,500 characters of scratch paper at 18 tokens a second. Two behaviour changes follow: you stop killing runs that are working, and you set timeouts from measured thinking time rather than vibes. A 600-second child_timeout_seconds is fine for a tier that thinks for 30 seconds and a coin flip for one that thinks for 300.

3. The separate reasoner tier exists because of this. Not because reasoning models are better — because they are differently expensive, and the multiplier is large enough to deserve an explicit, named, opt-in slot rather than a default. “Scout / reader / reasoner / writer” is a statement about where thinking is worth buying.

4. Hidden reasoning is a trust question, not just a cost question. If the chain of thought is where the model decides what to do and you cannot see it, then your audit surface is the answer plus the tool call. That is OpenAI's own argument for keeping monitoring access internally while showing users a summary. The practical version for an agent stack: verify from effects, not from reasoning. You can read the answer; you cannot read the scratch paper. Check what it did.

A. See thinking tokens on your own machine (Ollama)

# 1. Is this model a reasoning model? Look for "thinking" under Capabilities.
ollama show <model>

# 2. What does this model's thinking knob support?
curl -s http://localhost:11434/api/show -d '{"model":"<model>"}' | grep -A4 '"thinking"'

# 3. Ask the same question twice, thinking on and off.
#    Use arithmetic or a multi-step question — a yes/no question thinks for 3 tokens.

Compare four fields from the two responses: message.thinking (empty when off), message.content, eval_count, eval_duration. If eval_count is dramatically higher with thinking on and the answers agree, you have reproduced §04 on your own hardware.

The script that produced this lecture's numbers warms the model first, so load_duration does not contaminate the timing:

import json, urllib.request

def call(think):
    body = json.dumps({
        "model": "qwen3:0.6b",
        "messages": [{"role": "user", "content": "A shop sells pens in packs of 3 for 7 dollars. "
                                                 "How much do 12 pens cost? Reply with just the number."}],
        "think": think, "stream": False,
        "options": {"temperature": 0, "seed": 42},
    }).encode()
    req = urllib.request.Request("http://localhost:11434/api/chat", data=body,
                                 headers={"Content-Type": "application/json"})
    with urllib.request.urlopen(req, timeout=900) as r:
        return json.load(r)

call(True)          # warm-up: load the model before measuring

for think in (True, False):
    d = call(think)
    print(think, "out_tokens=", d["eval_count"],
          "eval_s=", round(d["eval_duration"]/1e9, 2),
          "thinking_chars=", len(d["message"].get("thinking") or ""))

Expected: output tokens roughly 5–10× higher with thinking on, eval time roughly 8–15× higher on a small CPU or Apple-silicon box, same final answer. No difference at all means the model is not a thinking model, or the client is stripping the field — re-check step 1.

Measure the pause. Stream the same request, timestamp the two kinds of chunk, and take the gap between the first message.thinking and the first message.content. That gap is the empty screen a user sees. On this box: 0.58 s vs 50.5 s.

B. See it in LM Studio (the M4 path)

1. Load a reasoning model (any R1 distill, Qwen thinking model, or gpt-oss variant).
2. Ask something that requires arithmetic reasoning.
3. The reasoning text appears in a collapsible Thoughts block above the answer. It is the thinking channel rendered by the UI, not part of the answer. (0.3.7 printed it inline; 0.3.9 introduced the collapsible Thinking blocks; 0.3.10 made the tags configurable in My Models for models that label reasoning differently.)
4. To see it as data instead of UI: App Settings → Developer → enable the experimental reasoning_content field, then call the local OpenAI-compatible endpoint and read choices[0].message.reasoning_content.
5. Do not send reasoning_content back in the next request. LM Studio's release notes say so explicitly, following DeepSeek's guidance — the scratch paper is not part of the conversation.

If the Thoughts block never appears, the model's reasoning tags do not match what LM Studio expects. That is a configuration issue, not proof the model is not thinking.

C. Read the count from a hosted API

OpenAI splits it in the usage object — this example is from their docs:

"usage": {
  "input_tokens": 75,
  "output_tokens": 1186,
  "output_tokens_details": { "reasoning_tokens": 1024 },
  "total_tokens": 1261
}

1,024 of 1,186 output tokens were thinking; the visible answer was ~160 tokens.

reasoning_tokens: 1024  —  the 1,024 tokens you were billed for and could not read,
folded inside output_tokens: 1186, of which only ~160 were the answer.

1,024 of 1,186 output tokens were thinking, and the visible answer was roughly 160 tokens. Also check status: incomplete with incomplete_details.reason of max_output_tokens means you paid for thinking and got no answer.

Anthropic's equivalent is usage.output_tokens_details.thinking_tokens, which “reports how many of the billed output tokens were internal reasoning”; on streaming it appears only on the final message_delta event [9].

DeepSeek / OpenRouter / most OpenAI-compatible servers expose reasoning_content alongside content, or usage.completion_tokens_details.reasoning_tokens, depending on the server. If neither exists, the provider is folding thinking into completion_tokens and hiding the split — legal, but it means you can only measure the tax by comparing two runs (§11A).

D. The checklist

1. ollama show <model> → is thinking under Capabilities?
2. /api/show → does it expose thinking.values / thinking.default? (values: [false] = no thinking.)
3. LM Studio model page → does it carry a Reasoning tag?
4. Hermes /models → what does the model advertise? (301-04's material: input_modalities and the capability fields — reference it, do not re-derive it here.)
5. First call of the day: is eval_count far larger than the answer's length suggests? If yes, you are paying for tokens you cannot see.
6. Any reasoning call that returns empty: check status / finish_reason before debugging anything else.

Thinking tokens are output tokens with the display turned off. They are generated one at a time at output speed, billed at output rates, counted against the same max_tokens ceiling as the answer, and — on multi-turn work — often re-billed as input when the conversation is re-sent. They explain the pause that looks like a hang, they explain why a reasoner slot costs several times a direct answer for an identical visible result, and they are why every serious provider ships an effort dial instead of letting you think maximally by default.

The habit to take away: when you compare two agent runs, compare eval_count, not the answer. The answer is the part you already knew how to read. The count is where the reasoning tax shows up.