How do I use chain of thought prompting?

The simplest way is to add 'Let's think step by step' to your prompt. For more control, provide worked examples that demonstrate the reasoning format you want the model to follow (few-shot CoT). The model will then apply the same step-by-step approach to new inputs.

When should I use chain of thought prompting?

Use CoT for tasks requiring multi-step reasoning: math problems, logical deductions, code debugging, comparing multiple options, planning sequences of actions, or any task where the answer depends on intermediate calculations or decisions.

What is the difference between zero-shot and few-shot chain of thought?

Zero-shot CoT uses a simple trigger phrase like 'Let's think step by step' without providing any examples. Few-shot CoT includes 2–5 worked examples that demonstrate the reasoning process. Few-shot CoT typically produces more consistent and formatted output; zero-shot is faster to implement.

Does chain of thought prompting work with all AI models?

CoT prompting is most effective with larger, more capable models like GPT-4, Claude 3, and Gemini Advanced. Smaller or older models may not benefit as much because they have less capacity for complex multi-step reasoning, regardless of how the prompt is structured.

Chain of Thought Prompting: The Complete Guide

Q: What is chain of thought prompting?

Chain of thought (CoT) prompting is a technique where you ask an AI model to show its reasoning step-by-step before giving a final answer. This improves accuracy on complex tasks like math, logic, and multi-step planning because the model works through the problem rather than guessing.

Chain of Thought (CoT) prompting is one of the most powerful techniques in a prompt engineer's toolkit. It's the difference between asking an AI to guess an answer and asking it to work through the problem.

What is Chain of Thought Prompting?

Chain of Thought prompting is a technique where you instruct the AI to reason step-by-step before giving a final answer. Instead of jumping directly to a conclusion, the model shows its work.

Without CoT:

Prompt: A store has 24 apples. They sell 1/3 in the morning and 1/4 of the remainder in the afternoon. How many are left?

Response: 12 apples (often wrong)

With CoT:

Prompt: A store has 24 apples. They sell 1/3 in the morning and 1/4 of the remainder in the afternoon. How many are left? Think step by step.

Response:

Start: 24 apples

Sell 1/3 in morning: 24 × (1/3) = 8 sold → 16 remaining

Sell 1/4 of remainder: 16 × (1/4) = 4 sold → 12 remaining

Answer: 12 apples (correct, and verifiable)

Why Does It Work?

Large language models predict the next token. When forced to reason out loud, each step becomes context for the next — meaning the model builds on correct intermediate steps rather than pattern-matching to a potentially wrong answer.

Research shows CoT significantly improves performance on:

Math word problems
Multi-step logic puzzles
Planning tasks
Code debugging
Any task requiring sequential reasoning

Three Levels of Chain of Thought

Level 1: Basic — "Think step by step"

The simplest implementation. Just add a phrase that signals you want reasoning.

What is 17% of 250? Think step by step.

Level 2: Guided CoT — Tell it the steps

Provide a framework for how the model should reason.

You are a financial analyst reviewing a Q4 report.

Here is the report: [paste report]

Analyze this by:
1. Identify the key revenue metrics and YoY changes
2. Assess the cost structure and margins
3. Identify 2-3 risks mentioned
4. Give a one-sentence investment outlook

Be thorough in each step before moving to the next.

Level 3: Structured CoT with XML tags

The most powerful implementation — especially for Claude.

You are a legal assistant reviewing a contract clause.

<clause>
The licensor may terminate this agreement with 30 days written notice if licensee
fails to cure any material breach within 15 business days of receiving notice.
</clause>

First think through the implications in a <thinking> block,
then provide your final analysis in an <analysis> block.

The <thinking> tag gives the model dedicated space to reason freely without polluting the final output.

When to Use Chain of Thought

Use CoT when:

The task involves multiple steps
There's a right answer that can be reasoned to
You need to verify the logic, not just the output
The task involves math, analysis, or planning

Don't use CoT when:

The task is simple and direct (translating a phrase, writing a subject line)
Speed and low token count matter more than perfect reasoning
The task is creative rather than logical

Key CoT Phrases to Use

Add any of these to activate step-by-step reasoning:

"Think step by step."
"Let's work through this carefully."
"Reason through this before giving your answer."
"Walk me through your reasoning."
For XML: Use <thinking> tags (especially effective with Claude)

Key Takeaway

Chain of Thought prompting is one of the highest-impact techniques with almost zero cost — just a few extra words. For any task involving reasoning, analysis, math, or multi-step logic, always add a CoT instruction. The improvement in accuracy is often dramatic.