Introduction to tokens
A token is the smallest unit of text a language model reads and writes. A token can be as short as a single character, like "a", or as long as a whole word, like "apple" — long or unusual words are often split across several tokens.
Think of tokens as the building blocks of a conversation. Just as a Lego structure is made of individual bricks, any piece of text is made of individual tokens. As a rough guide, 1,000 tokens is about 750 words of English.
How LLMs use tokens
The graphic's claim that a model reads a prompt "one token at a time" is inaccurate. The spotlight illustrates a token, not the order of prompt processing.
Language models process text as tokens and generate responses by predicting successive tokens. Processing the prompt is not the same as generating the answer token by token.
Every model also has a ceiling on how many tokens it can hold at once — its context window. Early models were small: GPT-3 originally topped out at 2,048 tokens, roughly three or four pages. Today's models are far larger, typically from a couple of hundred thousand tokens up to a million or more — the length of several novels.
Understanding token count
Token count shapes everything a model does, from processing a prompt to writing a reply. The conversation and attached material sent in a request, together with the model's generated response, must fit within the model's limits.
This is why token count matters in practice: a long conversation with many uploaded documents consumes the window much faster than a short exchange, even though both feel like "one chat".
Flint summarizes long attachments and older conversation history, so each request does not necessarily contain every original message or full document. Model capacity is also different from Flint's message and upload limits. See What is a context window?.
Input tokens and output tokens
Tokens are counted in two directions:
| Term | What it means |
|---|---|
| Input tokens (also called prompt tokens) | Tokens sent to the model, including messages, attached material, system instructions and tool definitions |
| Output tokens (also called completion or sample tokens) | Tokens the model generates; some models also count internal reasoning that is not shown in the answer |
For example, if you ask "What is the weather today?", the tokens in your question are input tokens. The model's reply is made of output tokens.
Key points to remember
- Tokens are the smallest units of text a model can process — roughly 1,000 tokens to 750 words.
- Models process text as tokens and generate responses by predicting successive tokens.
- Input and generated output must fit within the model's limits; Flint can send summaries instead of full older messages or documents.
- Input tokens are what goes in; output tokens are what the model generates, including hidden reasoning on some models.