Chain of Thought Prompting: What Still Works With Reasoning Models

Chain of Thought (CoT) prompting is the idea that a model answers better if it works through a problem before committing to a conclusion. In 2022 that was a prompting trick: you added "think step by step" and accuracy jumped. In 2026 most frontier models do it on their own, before they write a single visible word.

So this lesson covers both halves. You'll learn what chain of thought is and why it works, then how the advice changes for reasoning models that already think.

The core idea

A language model produces its answer one token at a time. Each token it writes becomes context for the next one. If it jumps straight to an answer, it's pattern-matching. If it writes out the intermediate steps first, every later step can build on the earlier ones.

Without CoT (on a model with no built-in reasoning):

Prompt: If a train travels 60mph for 2.5 hours then 80mph for 1.5 hours,
what's the total distance?

Response: 260 miles   ← wrong

With CoT:

Prompt: If a train travels 60mph for 2.5 hours then 80mph for 1.5 hours,
what's the total distance? Think step by step.

Response:
Step 1: First leg = 60 × 2.5 = 150 miles
Step 2: Second leg = 80 × 1.5 = 120 miles
Step 3: Total = 150 + 120 = 270 miles
Answer: 270 miles   ← correct

The paper that named the technique (Wei et al., 2022) showed chain-of-thought prompting taking PaLM 540B from roughly 18% to 57% on the GSM8K math benchmark. No new model, just a different prompt.

What changed: reasoning models think on their own

Current flagship models run a hidden reasoning phase before they answer. OpenAI calls these reasoning tokens, Anthropic calls it thinking, and Google calls it thinking too. In every case:

• The model decides how to break the problem down. You don't need to trigger it.
• The raw reasoning is hidden. None of the three vendors returns it. You can ask for a summary of the reasoning, but not the reasoning itself.
• You still pay for it. Reasoning tokens count as output tokens and take up context window.
• You control depth with a setting, not a phrase. Each API has an effort or thinking-level parameter.

OpenAI's reasoning guide is blunt about what this means for prompts. It recommends you "give the model the task, constraints, and desired output format" and avoid prescribing intermediate steps. A step-by-step recipe written for a 2023 model can make a 2026 reasoning model worse, because it swaps the model's own plan for yours.

How to prompt a reasoning model

Instead of asking the model to show every thought, tell it what a good answer looks like and how to check itself.

Old habit:

Evaluate this startup idea. Think step by step. First identify the market,
then analyze competitors, then assess risks, then give a score.
Show all your reasoning.

Better for a reasoning model:

Evaluate this startup idea for a seed investor.

<idea>
An app that connects homeowners with vetted freelance contractors
for small home repair jobs, with upfront pricing.
</idea>

What I need:
- A score from 1 to 10
- The two biggest risks, each in one sentence
- A short rationale (under 80 words) a partner could skim

Constraints: be specific to this market; say "unknown" rather than
inventing market-size numbers.

The second prompt never mentions steps, but it's more demanding. It sets the audience, the output shape, a length limit, and a rule against making up numbers. The model plans its own path to that answer.

Three habits that carry over from classic CoT and still matter:

1. Ask for a short rationale, not a transcript. "Explain your answer in two sentences" gives you something to check without paying for a long visible monologue on top of the hidden reasoning.
2. Ask it to verify. "Check the arithmetic before answering" or "confirm each claim is supported by the document" still improves results.
3. Separate the answer from the explanation. XML tags like <answer> and <rationale> make the output easy to parse.

What to drop: <thinking> tags you ask the model to fill in. On reasoning models the real thinking already happened in a hidden channel, so a visible <thinking> block is a second, performed version you pay for twice.

When classic chain of thought still helps

"Think step by step" isn't dead. It's the right tool whenever the model doesn't have a reasoning phase, or you've turned it down:

• Small and fast models such as fast-tier API models run at low effort, where you've traded thinking for speed
• Open-weight models running locally that weren't trained to reason internally
• Reasoning set to off or minimal, for example reasoning.effort: "none" on OpenAI or thinking_level: "minimal" on models that support it
• When the visible working is the product, like a tutoring app that has to show a student each step

In those cases the three classic patterns still work:

Level 1: The simple trigger

What's the best way to structure a Series A pitch deck?
Think step by step.

Level 2: Guided steps

Tell the model which steps to take when you know the structure better than it does:

Evaluate this startup idea:
1. Identify the target market and estimate its size
2. Analyze the main competitors
3. Assess the key risks (market, technical, execution)
4. Give an overall score from 1-10 with a one-sentence justification

Level 3: Reasoning and answer in separate tags

Think through this proposal in a <reasoning> block, covering effort vs. impact,
user demand, technical risk, and opportunity cost. Then give your final
recommendation in a <recommendation> block with a clear yes/no.

Your code can then strip the <reasoning> block and show only the recommendation.

CoT plus few-shot

Worked examples that include the reasoning still teach a small model the pattern you want:

Review: "The battery life is incredible but the camera is mediocre."
Reasoning: One strong positive (battery), one negative (camera). No overall lean.
Sentiment: Mixed

Review: "Does what it says on the tin, nothing more."
Reasoning:

Quick decision guide

Key takeaway

Chain of thought taught the whole field that models reason better when they get room to work. Reasoning models took that lesson and built it in. Your job has shifted from making the model think to directing its thinking: a precise task, clear constraints, the right effort setting, and a way to check the result.

Go deeper: Prompting reasoning models covers effort settings and cost trade-offs in detail. Current models and prices are on the models page.

Next: Learn Avoiding Hallucinations: how to keep AI models grounded in facts and prevent confident fabrication.