Tokens: definition, cost and meaning
Definition
Tokens are the small units into which a language model splits text to process it. A token is often a word part, a short word or a punctuation mark, in English roughly a syllable to a word. Models compute and are usually billed per token, separately for input and output. Whoever understands tokens can estimate the cost and limits of AI applications realistically.
Why the Term Matters
Tokens are like the counting unit on a taxi meter; every processed unit costs, there and back. The term matters because running AI cost almost always hangs on the number of tokens, not on the number of requests alone. As a neutral advisor we cost use cases in tokens before a budget is released, so there is no later surprise in operating cost.
When It Is Relevant
- The running cost of an AI application should be estimated realistically in advance.
- Long inputs or answers drive up the effort per request.
- A model comparison should include cost per token, not just performance.
- Prompts should be shortened to lower operating cost.
When Not
For a first feasibility question no one needs to count tokens; what counts is whether the case brings value at all. And for very small, rare applications token cost is often negligible. Tokens become important once volume and continuous operation come into play, not with every test.
Industries That Use It
- E-commerce: convert high support request volumes precisely into token cost per case.
- Document-heavy law firms: long documents burden the budget, shorter inputs cut cost.
- Software companies: compare models by cost per token for continuous operation.
- Marketing: plan mass text generation by token use instead of gut feeling.
Typical Use Cases
- The cost of a use case is projected from estimated tokens.
- A model comparison weighs cost per token against performance.
- Prompts are shortened to cut token use per request.
- Long documents are split to stay within limits.
- A budget is planned and monitored based on expected token volumes.
Decision Guide
| Signal | Token view matters | Alternative |
|---|---|---|
| High request volumes in continuous use | cost per token calculation | a flat estimate is too coarse |
| Long inputs and outputs | shorten prompts and texts | uncut gets expensive |
| Only a first feasibility question | secondary | clarify value first |
| Very rare use | often negligible | no fine calculation needed |
Example
An online retailer wants to introduce AI reply drafts in support and asks about cost. We estimate the average tokens per case for input and output, project them onto the monthly case count and compare two models by cost per token. This puts the operating budget in place before the start. This describes the method, not an asserted client metric.
Service Connection
The cost per use case we calculate in the AI potential analysis. The path from idea to economical delivery is guided by AI implementation, the overall frame by AI consulting for mid-market.
Sources
This framing describes common practice in cost calculation for language models. Concrete prices per token and how text is split into tokens differ by vendor and language and should be checked case by case against current vendor documentation.