AI Tokens: What They Are and Why They Affect Costs and Response Length

Tokens are the units a language model uses to read and write. It does not see sentences or pages the way we do: it sees a sequence of small pieces of text, and each piece is a token. The more text goes in and the more text comes out, the more tokens are consumed.

The definition, in the Claude glossary, is clear: tokens are the smallest individual units of a language model and can correspond to words, subwords, characters or even bytes in the case of Unicode. They usually stay hidden, because we write in the form of text. They become important when you look at the exact inputs and outputs, that is, when you talk about limits, response length and costs.

This article first explains the idea, then the mechanism: how text becomes tokens, what goes into the context window, why a chat that is too long gets worse, and where tokens touch the budget of a small business. For a more introductory take on counting and costs you can also read what AI tokens are and how they are counted; here we focus on the link with limits and response length.

What are AI tokens, in simple terms?

Think of a language model as someone who reads only through tiles. Before answering, your text is broken into tiles, the model works on those and produces the answer in the same way, one tile after another. This is our own analogy, not a definition from the sources, but it helps to remember the point: the model does not count words, it counts tiles.

The glossary describes exactly this step: when Claude is given a text to evaluate, that text, made of a series of characters, is encoded into a series of tokens that the model processes. A common word can be a single token, while a long or rare word can be split into several parts.

For someone running a business, the practical consequence is simple. When a tool talks about "length", "limits" or "consumption", it is almost always talking about tokens and not about pages, lines or words.

How many characters fit in a token?

There is no single answer. According to the Claude glossary, for Claude the number of characters represented by a token depends on the model and the language. The same document reports that Claude 4.7 and later models and Claude Mythos Preview use a newer tokenizer, which produces about 30 percent more tokens for the same text compared with earlier models, in which a token represents about 3.5 English characters.

Two things not to mix up. The estimate of about 3.5 characters concerns English and the earlier models. For Italian, the sources we consulted give no official figure, so we do not derive a rule of thumb from them. And the same text can cost more tokens simply by changing model.

The route indicated by the documentation is practical: to get exact counts you use token counting with the model you intend to use, as the glossary. To estimate a request before sending it, there is the token counting API, described in the page on context windows.

What is the context window and what goes into it?

The context window is all the text the model can consult when generating a response, response included. So says the documentation on context windows. An analogy, again our own: it is the desk the model works on. What is not on the desk, it does not see.

More ends up on the desk than you might think. According to the same page, everything present in the request is counted: the system prompt, every message (including tool results, images and documents) and the tool definitions. The output generated for that turn also counts, including extended thinking.

The same principle applies to the chat app. The Anthropic Help Center explains that length limits refer to the context window, that is, the amount of information Claude can work with in a single chat. A portion is always reserved for the response, so the longest possible conversation is slightly smaller than the full window.

How big is the context window?

It depends on the model. For the app, the Help Center indicates that on paid plans the most recent models support a window of up to 1M tokens, while others support 500K or 200K. The exact value for your case should be checked against the model and plan you use, because it changes over time.

If you want to understand how much this size really matters when choosing a model, the article on the context window compared puts the question in perspective.

Why does a very long chat answer worse?

Because more context is not automatically better. The documentation on context windows observes that a larger window allows longer and more complex prompts to be handled, but that as the number of tokens grows, accuracy and recall degrade. The phenomenon is called context rot, a deterioration of the context.

In practice: pasting everything "just to be safe" is not a good strategy. If the chat has accumulated discarded drafts, attachments that are no longer useful and detours, the model has to work in the middle of that noise. It is often better to open a new conversation with a clean summary of what is needed.

How do tokens affect usage limits and costs?

Here it is worth distinguishing two types of limit. Again according to the Anthropic Help Center, usage limits concern the amount over time, while length limits concern the depth and complexity of individual conversations. They are two different counters.

Several factors weigh on consumption: the length and complexity of conversations, the features used, the chosen model and the selected effort level. A higher effort level uses more tokens. Tools and connectors require many tokens, so managing them helps both to maximize the available window and to optimize limits.

For those using the API there is a billing detail. Each response reports consumption in the usage field, and the page on context windows specifies that thinking tokens are billed as output tokens only once, when they are generated. On models that keep previous thinking blocks, those blocks then enter the input of subsequent requests and are billed as input tokens, like the rest of the history.

Per-token prices are not in the sources we used: they should be read on the provider's official pricing page. To understand how they translate into a budget there is price per million tokens explained and, for Claude models, the price list compared with the cost of labor.

What does the problem look like in a real small business?

Here are two hypothetical examples, built to explain the mechanism. They are not client cases.

An accounting firm. A team member opens a chat in the morning, pastes in a contract, then a circular, then a series of emails, and keeps asking questions all day. Toward evening the answers become less precise and the usage limit gets closer. The mechanism is the one described above: all the material stays in the window and the deterioration of the context makes itself felt. The remedy is to work file by file: one conversation per document, with a short summary carried into the next.

A craft online shop. The owner connects several tools and connectors to the assistant "for convenience", even though one would be enough to describe the products. Every definition and every tool result takes up context and consumes tokens. Turning off what is not needed frees up space for the real work. If the shop then used the API, the same reasoning would carry over to the cost per request.

For an idea of how another model is evaluated with the same grid, you can read how reasoning and practical limits show up in Kimi. To see how the same topic appears inside a developer tool, there is a dedicated piece on reading the context window in Claude Code.

What mistakes should you avoid with tokens?

Thinking in pages instead of tokens

It happens because pages and words are the units we know. The problem is that the model does not use them, and the count changes with model and language. To avoid it, count with the model you will actually use.

Assuming a fixed rule for Italian

You sometimes hear round estimates, but the sources do not provide one for Italian. You avoid the mistake by measuring a sample of your texts with the official counting tool.

Keeping a single endless chat

It is convenient, but the context fills up and quality drops. Open new conversations when the task changes and bring along only what is needed.

Leaving unnecessary tools and connectors active

It is done out of habit or for fear of not having something at hand. Yet, as the Help Centerreminds us, they require many tokens. Turn on only those useful for the task.

Confusing usage limit and length limit

They look like the same block but they are not: one measures the amount over time, the other the depth of a conversation. Understanding which of the two is stopping you tells you whether you need to wait or open a new chat.

How do you apply this in a business, step by step?

Before starting

• You know which model and which plan you will use.
• You have the provider's official pricing page at hand.
• You have chosen three or four typical texts from your work to use as a sample.
• You know which tools and connectors are really needed.

The steps

1. Take the sample texts and measure them with the token counting of the chosen model, without trusting generic estimates.
2. Look at the usage field in the responses, if you use the API, to see what a real request consumes.
3. Ask for responses of the right length: a response longer than necessary consumes tokens without adding value.
4. Turn off unnecessary tools and connectors and open a new conversation when the chat gets long.
5. Only at this point estimate a budget, with prices read on the provider's site, and review it after a few weeks of use.

What can go wrong: using a sample that is not representative, forgetting that a more recent model may count more tokens for the same text, or estimating costs with outdated prices. For a broader calculation method, how to calculate API costs completes the picture.

Where to start?

Start from measurement, not intuition: a few real texts, the chosen model and the official data are enough to remove most of the surprises. If you want to talk it over with someone who knows your processes, get in touch on our AI consulting page.