Why Non-English Text Costs More Tokens: Turkish, Chinese, Arabic

You pay for tokens, not words. That sounds neutral until you ship a product in more than one language and discover that the same sentence costs different amounts depending on the language it is written in. A support bot that costs you a few dollars a month in English can quietly cost double in Turkish, and nothing in your logs will tell you why unless you look at token counts.

This guide explains where the gap comes from, how to measure it on your own text, and what to change in your prompts and budget so a multilingual product does not surprise you at the end of the month.

Why the same meaning costs different amounts

A tokenizer learns its vocabulary from a large body of training text, and most of that text is English. In , frequent character sequences get merged into single tokens, round after round. Sequences the tokenizer saw often end up as one token. Sequences it saw rarely stay in smaller pieces, sometimes down to single characters or even single bytes.

The consequence is simple. A language that was well represented in the training data tends to need fewer tokens per idea. A language that was underrepresented needs more. The words are not harder, the tokenizer is just less familiar with them.

This is a property of the tokenizer, not of the language model's intelligence. A model can understand Turkish perfectly well and still charge you more to read it, because the bill is calculated before any understanding happens. If you want the full picture of how a token becomes a cost, start with .

Language by language

Each language has its own reason for producing extra tokens.

Turkish is agglutinative. A single word can carry a stem plus several suffixes that English would express with separate words. Take evlerimizden, which means "from our houses". A tokenizer trained mostly on English may cut it into the stem and a handful of suffix fragments, so one Turkish word costs several tokens where the English phrase costs three or four short, common ones.

Chinese has no spaces between words. Very common characters and character pairs are often single tokens, but rarer characters can fall back to byte-level pieces. Each Chinese character takes three bytes in UTF-8, so an unfamiliar character can cost up to three tokens by itself.

Arabic combines a non-Latin script, rich word morphology and optional diacritics. Letters take two bytes in UTF-8, and attached prefixes and suffixes create many surface forms of the same root. Each of these can fragment text.

Newer tokenizers with larger vocabularies narrow the gap, but they rarely close it, and the gap changes between versions. That drift is covered in .

Measure the gap yourself

Do not rely on a rule of thumb. Measure it on text that looks like your real traffic:

1. Take one realistic prompt, and a faithful translation of it in every language you support.
2. Paste each version into the . It runs entirely in your browser, so nothing you type is uploaded.
3. Record the results in a table like the one below.

The last column is your language multiplier. Do this for input text and for typical model answers separately, because they can differ. Use the multiplier to scale any English-based estimate, including the ones in . English ratios do not transfer to other languages, so a "one token is about three quarters of a word" rule will mislead you here.

What it does to your bill

Suppose, purely as an illustration, that your English prompt is 800 tokens and the Turkish version measures 1,600. The multiplier is 2.0x. At 10,000 requests per month and a hypothetical input price of $1 per million tokens:

• English: 800 x 10,000 = 8 million tokens, about $8
• Turkish: 1,600 x 10,000 = 16 million tokens, about $16

The difference looks small at this scale, but it scales linearly with traffic. At 10 million requests the same ratio is the difference between $8,000 and $16,000. Output tokens are usually priced higher than input tokens, so a language that also inflates the answer length hurts even more.

Plug your measured numbers into the . It compares every priced model side by side, which matters because models do not all share one tokenizer, and the cheapest model for English may not be the cheapest for Turkish.

It also shrinks your context window

Cost is only half of the story. A context window is measured in tokens, so a language with a 2x multiplier fits roughly half as much text into the same limit. That affects:

• How many chat turns fit before you have to truncate or summarize
• How many retrieved documents fit into a retrieval prompt
• How long an input document can be before it must be split

If your retrieval pipeline was tuned on English documents, retest your chunk sizes in the target language. explains how chunk size, overlap and token budgets interact.

How to keep multilingual costs down

• Keep instructions in English. Put the system prompt, rules and output format in English, and leave only the user content and the answer in the native language. Test answer quality first, because the effect varies by model and task.
• Cut boilerplate. Repeated text is multiplied by your language multiplier on every request. A 600-token disclaimer in Turkish may really be 1,200 tokens.
• Measure with the right tokenizer. Use the tokenizer of the model you actually call. A count from one family of models can be off by a noticeable margin for another.
• Compare models on your own text. Benchmarks are mostly English. A cheaper model with a weaker tokenizer for your language can end up costing more per request than a pricier one.
• Budget in tokens, not words or characters. See .
• Mind rate limits. Token-based limits are hit sooner with a high multiplier. shows the arithmetic.

Common questions

Does a higher token count mean worse quality? Not by itself. It means a higher price and less context room. Quality is a separate question, so test it separately.

Should I translate prompts to English and back? Sometimes it saves money, but translation adds its own cost, latency and errors. Measure both paths on real examples before deciding.

Do emojis and symbols matter? Yes, they can add many tokens. See .

Checklist

• Measured a real prompt and a real answer in every target language
• Wrote down the language multiplier for input and for output
• Re-ran the monthly cost estimate with the multiplier applied
• Checked answer quality when instructions are in English and content is native
• Re-tested chunk sizes and context limits for each language

If you want to see the mechanism behind all of this from the ground up, work through .