There is no universal minimum, but for a narrow task with clean, consistent examples, a few hundred to a few thousand can be enough to see a real improvement; most practitioners land somewhere between 500 and 10,000 examples for instruction-style tuning, while projects that need to teach new facts or a new language typically need tens of thousands. The honest answer is that dataset size should be derived from your task, your method, and your quality bar — not picked as a starting number.
Why Dataset Size Is the Wrong First Question
Asking "how much data do I need?" before defining the task is like asking how long a piece of string should be. The number depends entirely on what behavior you want the model to learn. Fine-tuning a model to always respond in a fixed JSON schema is a different problem from teaching it your company's internal jargon, and the two need wildly different data volumes. The first is a formatting habit the model can pick up from a few hundred consistent examples; the second requires enough coverage that each new term appears in several contexts.
The more useful first question is: what exactly is the model getting wrong today? If you can articulate that as a specific, observable behavior — tone, structure, classification decisions, domain vocabulary — you can then ask how many examples it takes to demonstrate that behavior unambiguously. Dataset size is an output of that analysis, not an input.
There is also a cost dimension people often skip. Every example you add increases training time, token costs, and the effort of cleaning and reviewing. Overshooting is not free. Many teams get a better result from 800 carefully curated examples than from 8,000 scraped ones, because inconsistent data teaches the model inconsistent behavior.
Task Type and the Data You Actually Need
Different tasks have very different data appetites. Roughly, they fall into a few buckets:
- Style and format adaptation — teaching the model a response structure, tone, or output schema. This needs the least data because the underlying capability already exists; a few hundred to a couple thousand examples usually suffice.
- Classification and extraction — mapping inputs to labels or pulling fields out of text. Needs enough examples to cover every label and the natural variation within each. A workable starting point is often a few hundred per label, more if classes are subtle or imbalanced.
- Domain vocabulary and reasoning patterns — the model must learn terms, conventions, or multi-step procedures specific to your field. This needs broader coverage: expect low thousands to tens of thousands of examples, because each concept needs multiple appearances.
- New factual knowledge — teaching facts the model does not have. This is the hungriest category and often the wrong tool; retrieval (looking facts up at inference time) is frequently more reliable and cheaper to maintain than baking facts into weights.
A practical habit: sort your task into one of these buckets, then start at the low end of the corresponding range, train, and evaluate. Add data only where the evaluation shows failures. This iterative loop beats guessing a large number up front.
Rules of Thumb by Fine-Tuning Method
The method you choose changes the data budget more than any other factor. Here is a working range for each, with the reasoning behind it.
Prompt engineering with a few examples (in-context learning). You are not training anything; you are placing examples in the prompt. A handful of well-chosen examples (often 3–10) is typical. More than that and you burn context window — the maximum number of tokens the model can read at once — for diminishing returns.
Parameter-efficient fine-tuning (PEFT), including LoRA. LoRA (Low-Rank Adaptation) freezes the base model and trains small adapter matrices, so it needs far less data than full fine-tuning. A practical starting range is roughly 100–1,000 examples for a narrow task (classification, extraction, a fixed response style). For broader behavior changes, plan on 1,000–10,000. Below ~100 examples, results are unstable; above ~10,000 for a single narrow task, you often hit diminishing returns and should check quality before adding more.
Full fine-tuning. Updating all weights needs more data to avoid overfitting (memorizing your examples instead of learning the pattern). A reasonable floor is a few thousand examples; 10,000–100,000+ is common for robust behavior change. This is expensive and rarely necessary for narrow tasks.
Worked example. Suppose you want a support-reply assistant and target 1,000 LoRA examples. Assume each example averages 300 tokens (input plus output). Total training tokens:
1,000 examples × 300 tokens = 300,000 tokens per epoch
If you train for 3 epochs, that is 300,000 × 3 = 900,000 training tokens. At an illustrative rate of $0.008 per 1,000 tokens (an estimate, not a quote — check current provider pricing), cost is:
900,000 / 1,000 × $0.008 = $7.20
Double-check: 900 × 0.008 = 7.2. Correct. The point is not the exact price but that token count × epochs drives cost, so estimate tokens first.
Quick formula:
training_tokens = num_examples × avg_tokens_per_example × epochs
estimated_cost = (training_tokens / 1000) × price_per_1k_tokens
Quality Beats Quantity: Deduplication and Diversity
A thousand near-identical examples teach a model almost nothing new. Two things matter more than raw count:
- Deduplication: remove exact and near-duplicate rows. Duplicates skew training toward whatever they repeat and waste tokens. Near-duplicates (same answer, slightly reworded) are the sneaky ones — check for them, not just exact matches.
- Diversity: cover the range of inputs the model will actually see — edge cases, different phrasings, varying lengths, and the failure modes you care about. A dataset of 500 varied, clean examples usually beats 5,000 repetitive ones.
Pre-training checklist:
- Every example has a correct, consistent target output.
- Exact duplicates removed; near-duplicates reviewed.
- Inputs cover common cases and known edge cases.
- Format is uniform (same schema, same instruction style).
- A held-out validation set (e.g., 10–20%) is set aside before training.
- Token counts estimated so cost is known up front.
Frequently Asked Questions
How many examples do I need to fine-tune an LLM with LoRA?
There is no universal number, but a practical starting range is a few hundred to a few thousand high-quality examples for LoRA (Low-Rank Adaptation, a fine-tuning method that trains a small set of added weights while keeping the base model frozen). Simple, narrow tasks such as a fixed output format or a single classification can often move with roughly 200–500 clean examples, while broader behavior changes or multi-step reasoning tasks typically need 1,000–5,000 or more. Treat these as estimates: validate on a held-out set and add data only where the model keeps failing.
Can I fine-tune a model with only 100 examples?
Yes, you can run a fine-tuning job with 100 examples, but expect narrow results. At that size the model can pick up a specific style, tone, or rigid output template, yet it will likely overfit (memorize the training examples instead of generalizing) and may degrade on inputs that differ from your samples. If 100 is all you have, prefer parameter-efficient methods like LoRA, keep training short, hold out 10–20 examples for evaluation, and supplement with prompt engineering.
Does fine-tuning need more data than prompt engineering?
Not necessarily in raw volume, but fine-tuning is far more sensitive to data quality and consistency. Prompt engineering needs zero training examples, just well-crafted instructions and a few in-context examples, so it is the cheaper first experiment; fine-tuning becomes worthwhile when you have repeatable input-output pairs and need consistent behavior at scale. A few hundred curated examples often beat thousands of sloppy ones, so start with prompting and fine-tune only when examples accumulate.
How do I estimate token counts for my fine-tuning dataset?
Token counts depend on the tokenizer, but a rough English estimate is about 1 token per 0.75 words, or roughly 4 characters per token. For a dataset of N examples with average prompt length P and completion length C (in tokens), total training tokens are approximately:
total_tokens ≈ N × (P + C)
Add overhead for special tokens and, if you train for multiple epochs, multiply by the epoch count. Count exactly with the model's tokenizer before committing, since estimates can be off by 20–30% on code or non-English text.
What happens if I fine-tune on too little or too much data?
Too little data causes overfitting: the model memorizes your examples, loses generality, and may become brittle or repetitive on new inputs. Too much data, or many epochs over the same set, causes diminishing returns at best and overfitting or catastrophic forgetting (losing prior capabilities) at worst, while raising cost and training time. The practical fix is the same in both directions: watch validation loss on a held-out split and stop when it stops improving.