Chain-of-thought (CoT) prompting asks a language model to write out its intermediate reasoning steps before giving a final answer. On multi-step problems such as arithmetic, logic, planning and multi-condition questions, this tends to improve accuracy because each written step becomes context for the next, and it lets you see where a wrong answer went wrong. With today’s reasoning models, which already think internally, the technique has changed: you often get better results by turning on the model’s built-in thinking than by writing “think step by step”.
This article covers where the technique came from, the three ways to prompt for it, a worked example with a flawed step you can catch, how it works with reasoning models in 2026, and the cases where it wastes tokens or misleads you.
Where chain-of-thought comes from
The term comes from a 2022 Google paper by Wei and colleagues, Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. The authors put a few worked examples in the prompt, each showing the reasoning steps that lead to an answer, and found large improvements on arithmetic, commonsense and symbolic reasoning tasks. With eight such examples, their largest model reached state-of-the-art results at the time on GSM8K, a benchmark of grade-school math word problems. They also reported that the benefit appeared only in sufficiently large models.
Shortly afterwards, other researchers showed that a single instruction such as “Let’s think step by step”, with no examples, also triggered step-by-step reasoning. That version is usually called zero-shot chain-of-thought, and it is the one most people know.
Why writing the steps changes the answer
A language model produces text one token at a time, and each token is conditioned on everything written before it. When a model answers a multi-step problem directly, it has to land on the result in one jump. When it writes the steps first, the intermediate results are on the page, and the final answer can build on them instead of on a guess. That is the practical intuition, and it explains both when CoT helps (problems with real intermediate steps) and when it does not (questions where there is nothing to work out).
The second benefit is for you. Visible steps turn a wrong answer from “the model is wrong” into “step 3 is wrong”, which is something you can fix with a targeted follow-up.
Three ways to prompt for chain-of-thought
There are three levels, from least to most control. Pick the lightest one that works for your task.
1. Zero-shot: a single instruction
Solve this step by step, showing your reasoning at each step
before giving the final answer:
[your question]Cheap and quick to add. The weakness is that you do not control what the steps are, so the model may skip the step that matters or spend its effort on trivial ones.
2. Structured: name the stages
Work through the problem below in four stages. Do not skip to the answer.
1. Restate the question in one sentence.
2. List the given information and any assumptions you are making.
3. Solve step by step, showing the arithmetic or logic for each step
and the intermediate result.
4. Give the final answer, plus one quick way to check it.
Put your working inside <reasoning> tags and the final answer
inside <answer> tags.
<problem>
[your problem]
</problem>Why each part exists: restating the question catches misreadings early; listing assumptions surfaces the ones you might disagree with; showing intermediate results makes errors findable; the check in stage 4 asks the model to test its own answer; and the tags let you, or a script, pull out just the answer. This structure tends to beat the bare instruction on harder problems because it stops the model from jumping ahead.
3. Few-shot: show worked examples
<example>
Problem: A job runs every 15 minutes and takes 4 minutes. How many
minutes per day is the job running?
Reasoning: There are 24 * 60 = 1440 minutes in a day.
1440 / 15 = 96 runs per day. 96 * 4 = 384 minutes.
Answer: 384 minutes
</example>
Now solve the next problem the same way.
Problem: [your problem]This is the original form from the Wei paper. Use it when you want the reasoning in a particular shape, for example always computing totals before rates. Anthropic’s prompting best practices note that worked examples also shape how Claude reasons in its own thinking, and suggest framing each one as a problem, the method to apply and the expected answer. For how to choose good examples, see zero-shot vs. few-shot prompting.
Worked example: catching a flawed step
The following is an illustrative example, written to show the technique rather than taken from a real model run. You ask a capacity question using the structured prompt above:
A service handles 1,200 requests per second at peak. Each instance
handles 150 requests per second at 70% CPU, and we want to stay at or
below 70% CPU with one instance able to fail at any time.
How many instances do we need?Suppose the reasoning comes back like this:
Stage 2 - Given: peak 1,200 rps; 150 rps per instance at 70% CPU;
N+1 redundancy. Assumption: load is spread evenly.
Stage 3 - Step 1: 1,200 / 150 = 8 instances at 70% CPU.
Step 2: Add one for failure tolerance: 8 + 1 = 9.
Step 3: Add 20% headroom: 9 * 1.2 = 10.8, round up to 11.
Stage 4 - Answer: 11 instances. Check: 11 * 150 = 1,650 rps > 1,200.Because the steps are visible, the error is easy to locate: step 3 adds headroom that the question never asked for. The 150 rps figure is already measured at the 70% CPU target, so 9 instances meets the stated requirement. Without the reasoning, you would have seen “11” and either accepted an over-provisioned answer or rejected the whole thing. With it, the follow-up is precise:
Re-check step 3 specifically. Does the question require extra headroom
beyond the 70% CPU target and one spare instance? If not, recompute.Note what the visible steps did not do: they did not guarantee the right answer. They made the wrong step inspectable. That distinction matters for the next section.
Chain-of-thought with reasoning models in 2026
Reasoning models generate their own internal chain of thought before answering, so asking them to “think step by step” is often redundant. Vendors now document this directly:
- OpenAI‘s reasoning best practices tell developers to keep prompts simple and direct and to skip instructions such as “think step by step” or “explain your reasoning”, which it says are unnecessary and can sometimes hurt performance.
- Anthropic‘s best practices recommend adaptive thinking on current Claude models, where the model decides how much to think, controlled by an effort setting. The same page says a general instruction like “think thoroughly” often produces better reasoning than a hand-written step-by-step plan. It also notes that on its newest models, a prompt asking the model to write out reasoning in tags may be declined, and recommends relying on built-in thinking, at a lower effort if cost matters.
So the practical rule depends on the model you call:
| Situation | What to do |
|---|---|
| Reasoning model with thinking on | Do not add “think step by step”. State the goal, constraints and success criteria clearly; raise the effort or thinking setting for harder problems. |
| Non-reasoning or fast model, or thinking turned off | Use manual CoT: the structured prompt above, with the answer in separate tags. |
| You need the reasoning as part of the deliverable (a calculation someone will audit) | Ask for a written derivation in the answer itself, as an explanation for the reader, rather than asking the model to show its internal thinking. |
| You want a specific method applied | Give one or two worked examples showing the method. |
The parameter names for thinking differ by provider; budget_tokens, reasoning_effort and thinking_level compared maps them side by side, and AI reasoning models and extended thinking explains how these models differ from standard ones.
The reasoning you see is not always the reasoning used
A written chain of thought is useful for debugging, but it is not a reliable record of how the model reached its answer. Anthropic’s April 2025 study Reasoning models don’t always say what they think slipped hints about the answer into evaluation questions and checked whether models that used a hint admitted it in their reasoning. Claude 3.7 Sonnet mentioned the hint about a quarter of the time and DeepSeek R1 about 39% of the time. The unfaithful explanations were, on average, longer than the faithful ones.
The lesson for prompt writers: use the reasoning to find steps you can check independently (a calculation, a quoted fact, a stated assumption), and check them. Do not treat a tidy explanation as proof the answer is right, or as proof of how the model decided.
When chain-of-thought does not help
Chain-of-thought costs output tokens and time, and on some tasks it buys nothing. Skip it for:
- Simple factual lookups. If the model does not know a fact, reasoning longer will not produce it; it can even dress a guess in convincing steps. Supply the source instead, as covered in reducing AI hallucinations with better prompts.
- Creative and subjective writing. A tagline does not have intermediate steps to get right.
- High-volume, simple classification. Writing reasoning for every item multiplies cost; use clear categories and examples instead.
- Latency-sensitive features. Autocomplete or chat replies where users wait for each response.
- Reasoning models with thinking already on, as described above.
When it backfires
A few failure patterns show up often enough to plan for:
- Confident wrong reasoning. Plausible steps make a wrong answer look more trustworthy. The fix is checking, not more reasoning.
- Early errors compound. A misread number in step 1 carries through every step. Restating the question and the given facts first (stage 1 and 2 in the structured prompt) catches many of these.
- Parsing breaks. If code expects a bare answer and gets paragraphs of reasoning, your integration fails. Put the answer in its own tags and extract only that.
- Over-prescribed steps. A rigid step list written for one kind of problem can force a bad method on a different one. For capable models, describe the goal and checks, not every move.
- Reasoning leaked to users. Intermediate working may include assumptions or internal details you would not show a customer. Keep it in a separate field and decide deliberately what to display.
Related techniques
Chain-of-thought is the base that several other techniques build on. Least-to-most prompting first splits a problem into subproblems, then solves them in order. Tree-of-thought explores several reasoning paths and compares them instead of committing to one. Self-verification methods add a checking pass after the reasoning. When the steps are large enough to deserve separate calls you can inspect, you are into prompt chaining rather than chain-of-thought.
