If you've ever used an AI API, you've met the word "token" within about five minutes. Pricing pages list it. Error messages mention it. Dashboards chart it. Yet a lot of people use AI tools for months without really knowing what a token is, which makes it hard to understand why a bill looks the way it does.

This guide fixes that. By the end you'll know what tokens are, how they're counted, what "token usage metrics" actually measure, and what you can do to keep usage under control.

So, what is a token?

A token is a small chunk of text that a language model treats as a single unit. It's not exactly a word and not exactly a character. It sits somewhere in between.

Models don't read text the way you do. Before a model sees your prompt, a piece of software called a tokenizer chops it into pieces and converts each piece into a number. The model works with those numbers, produces more numbers, and the tokenizer turns them back into text.

A rough rule for English: one token is about four characters, or about three quarters of a word. So 100 tokens is roughly 75 words. That's a ballpark, not a law.

How the splitting works

Most modern models use some form of subword tokenization, with Byte Pair Encoding (BPE) being a common approach. The idea is simple. Start with individual characters or bytes, then repeatedly merge the pairs that appear together most often in a large body of training text. Over time, common words become single tokens, while rare words get broken into several.

That's why you'll see patterns like these:

Short, common words ("the", "and", "water") are usually one token.
Longer or rarer words get split. A word like "unbelievably" might become two or three tokens.
Spaces and punctuation often attach to neighboring pieces.
Numbers can be split in surprising ways, which is one reason models sometimes stumble on arithmetic.
Code, URLs and emojis can use many more tokens than you'd expect.

Languages matter too. Tokenizers built mostly on English text tend to split other languages into more pieces. Turkish, Arabic, Hindi or Thai text may use noticeably more tokens than the same message in English, which means it can cost more and fill the context window faster. Newer tokenizers have improved this, but the gap hasn't fully disappeared.

One more important point: every model family has its own tokenizer. The same paragraph can be 120 tokens for one model and 135 for another. If you're comparing costs across providers, don't assume the counts match.

Input tokens, output tokens, and the context window

Token usage is usually reported in at least two numbers.

Input tokens are everything you send to the model: your question, the system prompt, earlier messages in the conversation, documents you attach, and tool definitions.

Output tokens are what the model writes back.

Many providers charge different prices for each, and output tokens typically cost more than input tokens. That makes sense when you think about how generation works. Reading your prompt can be done in parallel, but writing a response happens one token at a time.

Then there's the context window, the maximum number of tokens the model can handle in a single request, input and output combined. If a model has a 200,000-token window, your prompt plus its answer have to fit inside that. Go past it and you'll get an error, or the oldest content gets cut off.

Some newer models also produce reasoning tokens, the hidden "thinking" the model does before answering. Depending on the provider, these are billed like output tokens even if you never see them. This is one of the most common reasons a request costs more than people expected.

Why chats get expensive without you noticing

Here's something that surprises beginners. In a typical chat, the model doesn't remember your earlier messages on its own. Each time you send a new message, the whole conversation so far is sent again as input.

So a conversation with ten turns doesn't use ten small requests' worth of tokens. Turn one might use 500 tokens. Turn ten might send 6,000 tokens of history plus your new message. The cost grows with every turn.

Add a long system prompt, a few attached files, and tool outputs, and the input side balloons fast. For agents that call tools in a loop, a single user request can trigger dozens of model calls, each carrying the full history.

What "token usage metrics" actually are

When people talk about token usage metrics, they usually mean the numbers a team or company tracks to understand how AI is being used. Common ones include:

Total tokens per day, week or month. The simplest measure of overall volume.
Input versus output split. Useful because the two are priced differently.
Tokens per request. Shows whether prompts are bloated.
Tokens per user, team or feature. Helps you see who or what is driving cost.
Cost per task. Tokens translated into money for a specific job, such as summarizing a ticket.
Cache hit rate. How often repeated content is served from cache at a lower price.
Tokens per second. A speed measure rather than a cost one, but it's usually reported alongside usage.

Some organizations have also begun treating token consumption as a rough sign of AI adoption, asking how much each team is actually using these tools. That can be informative, but it has an obvious flaw: high usage isn't the same as high value. A team burning millions of tokens on poorly designed prompts may be getting less done than one using a fraction of that. Raw consumption is a starting point, not a verdict.

How to measure your own usage

Most providers return token counts in every API response, typically with fields for input tokens, output tokens, and sometimes cached or reasoning tokens. Log those numbers. It's the most reliable way to know what you're spending.

If you want to estimate before sending, many providers offer a token-counting tool or library for their tokenizer. Use the one that matches your model, since counts differ between families.

For teams, the useful habit is tagging. Attach labels to each request, such as feature name, customer, or environment, so you can break down usage later. Without tags, you'll have a bill and no explanation.

Practical ways to reduce token costs

You rarely need to sacrifice quality to spend less. Most savings come from removing waste.

Trim your prompts. Long, rambling system prompts get paid for on every call. Rewrite them tighter and cut examples that don't help.

Don't resend what you don't need. In long conversations, summarize older turns instead of carrying them all forward word for word.

Use prompt caching where available. If the start of your prompt is the same every time, such as instructions or a big reference document, many providers let you cache that portion and charge a lot less for repeated reads.

Pick the right model size. A smaller, cheaper model can handle classification, extraction and simple rewriting just fine. Save the large model for hard problems.

Cap the output. Set a maximum length and ask for concise answers when that's all you need. Remember that output is usually the pricier side.

Retrieve, don't dump. Instead of pasting a whole document, search for the relevant passages and send only those.

Batch non-urgent work. Some providers offer lower prices for jobs that can wait.

Watch reasoning settings. If a model lets you control how much it thinks, don't leave it at the maximum for easy tasks.

Common misunderstandings

A few quick corrections, since these come up constantly.

Tokens aren't words. Word counts are a decent estimate in English and a poor one elsewhere.

A bigger context window doesn't mean you should fill it. Stuffing in more text costs more, and models can get worse at using information buried in very long inputs.

Output length isn't the only cost driver. Hidden reasoning, tool calls and repeated history often matter more.

And a cheaper per-token price doesn't automatically mean a cheaper system. A model that needs three retries to get the right answer may cost more than a pricier one that gets it right the first time.

The takeaway

Tokens are the basic currency of language models. They determine what fits in a prompt, how fast things run and what you pay. Once you understand how they're counted and where they pile up, usage metrics stop being a confusing dashboard and start telling you something useful: where your AI spending goes, and where it's going to waste.

Start simple. Log your counts, tag your requests, and look at the biggest line item first. That alone will teach you more about your system than any pricing page.