Counting tokens in a chat conversation means tokenizing each message's text, then adding the structural tokens the chat format inserts: role tags and separators per message, plus a small fixed priming overhead, plus 3 tokens per reply for the assistant's turn. For a typical three-message exchange, estimate the text tokens, add roughly 4 tokens per message for the role and separators, add about 3 tokens for the priming, and add 3 tokens, giving the count you would send before the model's answer.
Why Chat Token Counts Differ from Plain Text Counts
A token is the unit a language model actually reads: a word, part of a word, or a punctuation mark, produced by a tokenizer (the algorithm that splits text into those units). If you tokenize a paragraph of plain text, you get a number, and that number is what most tokenizer demos show you. A chat request is not a paragraph. It is a structured list of messages, each with a role (system, user, assistant) and content, and the API serializes that structure into a single token sequence before the model sees it.
That serialization is the source of the gap. The model does not receive "user said X." It receives delimiter tokens that mark where a message begins, which role is speaking, where the content ends, and where the next turn starts. Those markers are real tokens and they land in the same context window as your prose. So the count for a chat is always text tokens plus structure tokens, and structure scales with the number of messages, not the length of the text.
There is also a fixed component. Chat formats typically open with a priming sequence that tells the model a conversation is beginning, and they may close the assistant's turn with an end-of-turn marker. These are small, on the order of a handful of tokens, but they are constant per request, so they matter most when your messages are tiny — a one-line question pays the same structural tax as a long one.
Finally, the count you care about depends on what you are measuring. Input tokens are what you send; output tokens are what the model generates and are counted separately, though they share the same context budget. Practitioners often conflate the two when estimating cost or fit.
The Hidden Per-Message Overhead and Role Tags
For the widely used ChatML-style format, each message is wrapped in markers that look conceptually like this:
<|im_start|>role
content<|im_end|>
Each wrapper contributes tokens that are not in your text: the start marker, the role name, a separator, and the end marker. In practice, tokenizers treat these as a small number of tokens per message — commonly estimated at around 4 tokens per message for the role and delimiters combined, though the exact figure depends on the tokenizer and how the role name is encoded. Some formats use a slightly different scheme (for example, an end-of-text token after each turn plus a role prefix), which lands in a similar range.
The practical consequence is that overhead is a per-message constant. Ten short messages cost more structure than one long message with the same total text. If you are trimming a prompt to fit a budget and you can merge turns without losing meaning, merging saves tokens twice: once in text and once in structure. Role tags themselves — the literal words "system," "user," "assistant" — do count, because they are encoded and sent like any other text; they are not metadata the model infers.
Two caveats keep this honest. First, exact overhead varies by model family and tokenizer version, so treat any fixed number as an estimate and verify with the provider's token-counting endpoint when precision matters. Second, special tokens may be counted differently by different tools: some tokenizers add them only when explicitly requested, so a naive encode(text) call can undercount a chat by the entire structural layer.
Worked Example: Counting a 3-Message Conversation
Assume a tokenizer in which one token is roughly four English characters, and assume the per-message overhead is 4 tokens (3 for the role and formatting tags, plus 1 separator). These are round, illustrative numbers — treat them as a stated assumption, not a measurement of any specific model.
The conversation:
system: You are a concise assistant. (30 characters)
user: How do I count chat tokens? (28 characters)
assistant: Add per-message overhead first. (34 characters)
Step 1 — estimate tokens per text by dividing characters by 4 (ceil so partial tokens round up):
system: ceil(30 / 4) = 8 tokens
user: ceil(28 / 4) = 7 tokens
assistant: ceil(34 / 4) = 9 tokens
Step 2 — add overhead for each message (3 messages × 4 tokens):
overhead total = 3 * 4 = 12 tokens
Step 3 — add a fixed reply primer, since the model continues the thread (assume 3 tokens):
primer = 3 tokens
Step 4 — total:
8 + 7 + 9 + 12 + 3 = 39 tokens
So the visible text alone is 24 tokens, but the request actually consumes about 39 — roughly 60% more. That gap is the point: text-only estimates undercount real chat requests.
A short code idea for repeatable counting:
import math
def chat_tokens(messages, overhead=4, primer=3):
text = sum(math.ceil(len(m["content"]) / 4) for m in messages)
return text + overhead * len(messages) + primer
A Pre-Call Token Budgeting Checklist
- Know your context limit. The context window is the maximum tokens a model can read and write in one call; input plus output must fit inside it.
- Reserve output tokens first. Decide the longest reply you will accept, subtract it from the limit, and treat the remainder as your input budget.
- Count every message's text, then add overhead. Use
overhead × number of messages; don't forget the system prompt, which is a message too. - Include history deliberately. Multi-turn chats resend the whole thread each call, so old turns keep costing tokens until you trim, summarize, or drop them.
- Account for tool or function definitions if present. These are injected text and consume tokens even though you never typed them.
- Leave a safety margin. Character-based estimates drift; keep roughly 10–20% headroom on long inputs.
- Log after the call. Compare your estimate with the API's reported token usage and adjust your overhead assumption so future budgets are tighter.
Quick formula:
input_budget = context_limit - max_output_tokens
estimated_input = text_tokens + (overhead * messages) + primer
assert estimated_input <= input_budget * 0.9
If the assertion fails, trim the oldest turns or shorten the system prompt before sending.
Frequently Asked Questions
How many tokens does each chat message add beyond its text?
There is no single universal number, because the overhead is defined by the model's chat format rather than by the tokenizer alone. As a practical planning estimate, assume roughly 3 to 5 tokens per message for role markers and structural delimiters, plus a small fixed cost once per conversation for the reply priming that tells the model it is the assistant's turn. Treat these as estimates derived from how chat templates typically wrap messages, not as exact figures: the reliable way to get the true number is to count with the tokenizer and chat template of the specific model you are calling.
Do role tags like system and user count as tokens?
Yes. The role labels, turn delimiters, and any special control tokens inserted by the chat template are part of the input sequence, so they are tokenized and billed exactly like ordinary text. This is why the same sentence costs more inside a chat request than as raw text: the wrapper around the message is invisible in your code but present in the sequence the model reads. Count the serialized prompt, not just the message contents.
Can I estimate chat tokens without calling the API?
Yes, provided you use the right tokenizer and the right chat template. Load the tokenizer that ships with your model, apply its chat template to your full message list, and count the resulting token IDs; most tokenizer libraries expose a helper that applies the template for you. Pure text heuristics such as characters divided by four give a rough ballpark for prose but ignore per-message overhead, so they drift as the number of turns grows. You can also call a token-counting endpoint where one is offered, which avoids downloading tokenizer files.
Why does my token count differ between models and tokenizers?
Different models use different tokenizers: some split text into subword pieces with different vocabularies and merge rules, so the same string can map to different token counts. On top of that, each model family wraps messages in its own chat format, adding different role tags and delimiters, and some count image or tool payloads differently. A count from one model is therefore only an approximation for another; always re-count with the tokenizer and template of the model you will actually call.
How do I budget tokens for a long multi-turn conversation?
Add up three components: the tokens of every message's text, the per-message overhead multiplied by the number of messages, and the tokens you are reserving for the model's reply. Subtract that total from the model's context limit to see how much room remains, and plan to trim the oldest turns or summarize them once you approach it. Reserve the reply budget deliberately, since output tokens consume the same context window, and re-run the count after every truncation because removing messages also removes their overhead.