Zero-shot, few-shot and chain-of-thought are three prompting strategies that differ in how much guidance you put into the prompt itself. Zero-shot gives the model only an instruction, few-shot adds worked examples, and chain-of-thought asks the model to write out intermediate reasoning before answering. They trade prompt length and cost against control and reliability, and the right choice depends on task difficulty, how much the output format matters, and whether the reasoning is the hard part.
What Prompt Engineering Really Controls
A language model generates its output one token at a time — a token being a chunk of text, roughly a word or part of one — by predicting what comes next given everything in its context window, the fixed budget of text it can see at once. Your prompt is the opening portion of that context, so prompt engineering is the practice of shaping that opening so the model's continuation lands where you want. It does not change the model's weights or add new knowledge; it changes the conditional probability of the tokens it produces.
In practice, a prompt controls four things: the task framing (what the model believes it is being asked to do), the output format (length, structure, tone, whether it answers directly or shows work), the evidence the model conditions on (your instructions, examples, and any data you paste in), and the stopping behavior (when it decides the answer is complete). Ambiguity in any of these leaks into the output. That is why "be specific" is the most common — and most underrated — prompting advice.
It helps to separate what prompting can and cannot do. It can steer format, decompose a problem into steps, constrain the scope of an answer, and make implicit assumptions explicit. It cannot reliably supply facts the model never learned, guarantee correctness on tasks beyond its capability, or make a weak model behave like a strong one. Prompting is steering, not upgrading.
Zero-Shot Prompting: Instructions Without Examples
Zero-shot prompting means giving the model an instruction with no worked examples: "Classify this support ticket as billing, technical, or account" followed by the ticket text. The name comes from machine learning, where "zero-shot" means performing a task with no task-specific training examples; here it means no examples in the prompt. You are relying entirely on what the model already learned during training, plus whatever the instruction conveys.
Zero-shot is the default and the right starting point. It is cheapest per call because the prompt is short, easiest to maintain, and often sufficient for common tasks: summarizing, rewriting, translating, extracting obvious fields, and answering straightforward questions. Modern instruction-tuned models — models trained to follow directions — handle these well. If a clear zero-shot prompt works, adding examples only adds tokens and cost.
Zero-shot fails in predictable ways. It struggles when the task has a specific output format you care about, when the category boundaries are subtle or idiosyncratic to your domain, or when the model must apply a convention it cannot infer from the instruction alone. The symptoms are inconsistent formatting, label drift (the model invents categories), and answers that are plausible but not what you meant. Before reaching for examples, try tightening the instruction: define each label, state the output format explicitly, and specify what to do with edge cases.
Few-Shot Prompting: Teaching by Demonstration
Few-shot prompting includes a small number of input–output examples in the prompt before the actual query. Typically you provide three to five demonstrations, though the useful range depends on task complexity and context budget. The model infers the pattern — format, label set, tone, level of detail — from the demonstrations and applies it to the new input. This is in-context learning: adapting behavior from examples in the prompt without any weight updates.
Examples are most valuable when the format is hard to describe but easy to show, when labels are domain-specific (your internal ticket taxonomy, say), or when you need consistent structure across many calls. A well-chosen set of demonstrations often fixes label drift and formatting issues in one step. The demos also act as a specification: they reveal edge-case handling you would otherwise have to write out in prose.
The costs are real. Every example consumes tokens on every call, so a five-example prompt multiplies your input cost compared to zero-shot — the exact multiplier depends on example length relative to the instruction, but it is easy to raise input tokens several-fold. Examples also bias the model: if all your demos are short, outputs skew short; if they share a superficial pattern, the model may latch onto it. Choose demonstrations that are representative, cover the tricky cases, and are formatted identically to how you want the real output. Keep the ordering consistent, and verify that the examples themselves are correct — a wrong demonstration teaches the wrong pattern.
Chain-of-Thought: Making Reasoning Explicit
Chain-of-thought (CoT) prompting asks the model to produce intermediate reasoning steps before giving a final answer — for example, "Work through this step by step, then state the result." Because the model generates tokens sequentially, writing out intermediate steps puts that reasoning into the context, and each subsequent token can condition on it. This measurably helps on multi-step problems like arithmetic word problems, logical deduction, and planning, where jumping straight to an answer invites errors.
The mechanic matters: CoT does not give the model extra computation in the abstract; it gives it more tokens to think in. That is why it helps most when the task genuinely requires several dependent steps and least when the task is a single lookup or classification, where the reasoning is padding. A useful pattern is to separate reasoning from output — let the model reason, then extract only the final answer for downstream use, so verbose reasoning does not pollute your data.
CoT has failure modes. The written reasoning is not guaranteed to reflect what actually drove the answer, and a confident-looking chain can rationalize a wrong conclusion, so treat it as a aid to accuracy, not a proof of correctness. It also increases output tokens and latency, sometimes substantially. And on simple tasks it can hurt by inviting the model to second-guess a correct instinct. Use it selectively, and check accuracy with and without it on your own task rather than assuming it helps.
Few-Shot Prompting: Teaching by Demonstration
Few-shot prompting means putting a small number of worked examples inside the prompt, before the actual input you want the model to handle. Each example shows an input and the output you expect. The model then continues the pattern for the new input.
The term "shot" refers to an example pair. Zero-shot means no examples; one-shot means exactly one; few-shot usually means somewhere between two and roughly ten, though there is no hard rule.
A concrete example for sentiment tagging:
Review: "The battery dies in two hours." -> negative
Review: "Arrived a day early, works fine." -> positive
Review: "It is a toaster." -> neutral
Review: "Support replied within minutes." ->
The model should answer positive, because the examples have fixed the label set, the output format, and the mapping between review language and label.
Why this works: examples communicate things that are awkward to state in prose — the exact output shape, edge-case handling, tone, and the level of detail you want. They act as a specification by demonstration.
Practical rules:
- Keep examples consistent. Mixed formats teach the model to be mixed.
- Order matters somewhat; put the most representative example last, closest to the real query.
- Cover the boundaries, not just the easy middle. If your task has a "neutral" or "refuse" category, show it.
- Don't include examples that contradict your written instruction. The instruction and the examples should tell the same story.
Chain-of-Thought: Making Reasoning Explicit
Chain-of-thought (CoT) prompting asks the model to write out intermediate reasoning steps before giving a final answer, instead of jumping straight to the result. The classic trigger phrase is "Let's think step by step," or you simply show an example whose answer includes a short derivation.
Why it helps: multi-step tasks — arithmetic, logic, multi-constraint planning — require the model to hold intermediate values. Writing them down puts them in the context window, where the model can condition on them. Skipping straight to an answer forces the model to compress several steps into one token prediction, which is where errors creep in.
Worked example with simple numbers. Suppose a task asks: a subscription costs 120 per year, billed monthly with a 10% annual discount applied to the total. What is the monthly charge? Assume the discount applies to the yearly total.
Step 1: yearly total before discount = 120
Step 2: discount = 10% of 120 = 12
Step 3: discounted yearly total = 120 - 12 = 108
Step 4: monthly charge = 108 / 12 = 9
Answer: 9 per month
Without the steps, a model may answer 10.8 (applying the discount to the monthly figure as 12 - 1.2) or 11 (rounding somewhere). The explicit chain makes each arithmetic move checkable.
Important caveats:
- CoT exposes reasoning, but does not guarantee it is correct. A fluent chain can still contain a wrong step. Verification is your job.
- It costs tokens: longer outputs mean higher latency and higher API cost.
- For simple classification or extraction tasks, CoT usually adds cost without accuracy gains.
Checklist before choosing a technique:
[ ] Task needs examples to pin down format? -> add few-shot
[ ] Task needs multiple reasoning steps? -> add chain-of-thought
[ ] Task is simple, well-specified, one step? -> start zero-shot
[ ] Output must be short/structured? -> CoT in a scratch field, answer separately
[ ] Accuracy matters? -> test all variants, measure, don't assume
## Frequently Asked Questions
### What is the difference between zero-shot, few-shot and chain-of-thought prompting?
Zero-shot prompting gives the model only an instruction, with no worked examples, and relies on what it learned during training. Few-shot prompting adds a handful of input-output examples in the prompt so the model can imitate the pattern you want. Chain-of-thought prompting asks the model to write out its intermediate reasoning steps before giving a final answer, which you can combine with either zero-shot or few-shot setups.
### When should I use few-shot prompting instead of zero-shot?
Use few-shot when the task has a specific format, tone, or labeling convention that is hard to describe in words but easy to show, or when zero-shot output keeps drifting from what you need. It is also useful for classification with custom categories, structured extraction, and consistent house style. If a clear instruction already produces correct output, zero-shot is cheaper and simpler, so try it first and add examples only when results fall short.
### Does chain-of-thought prompting always improve accuracy?
No. Chain-of-thought tends to help on multi-step tasks like arithmetic, logic, and planning, where the model benefits from working through intermediate values. On simple factual recall or straightforward classification it can add noise, latency, and tokens without improving the answer, and a flawed reasoning chain can still lead to a wrong result. Treat it as a tool for the right task, and verify outputs rather than assuming the visible reasoning is correct.
### How do examples in a prompt affect token usage and API cost?
Every token in your prompt counts toward input cost, and examples are usually the largest part of a prompt, so each added example raises input tokens for every single call. Output tokens cost more than input tokens on most APIs, which is why chain-of-thought is expensive: the reasoning text is generated and billed on each request. As a rough estimate, a few-shot prompt with several examples can use several times the input tokens of a bare instruction, and longer reasoning can multiply output tokens, so measure your actual usage.
### Can prompt engineering fix a model that lacks domain knowledge?
Rarely. Prompting can shape how a model uses knowledge it already has, and examples can teach format and conventions, but it cannot reliably supply facts the model never learned. For missing domain knowledge, the practical fixes are retrieval, where you paste relevant source documents into the prompt, or fine-tuning on your own data. Prompt engineering then helps the model use that supplied context well, but it is not a substitute for the knowledge itself.