Token Usage Optimization

A token is the smallest unit of text a language model reads and processes, not quite a word and not quite a character, but something in between.

Before a model does anything with text, it breaks it into these units: a common word often maps to a single token, while longer or rarer words split into several pieces, and even spaces and punctuation count.

In English, a token averages about three to four characters, or roughly three-quarters of a word, so 1,000 tokens come out to around 750 words (this varies by model and language). Tokens matter because they are the unit everything is measured in: cost, context limits, and usage are all counted in tokens.

Token usage falls into two categories.

Input tokens are everything sent to the model: the prompt typed, attached files, etc.

Output tokens are the words the model generates in response.

Much of the input is invisible, since system instructions and earlier turns ride along behind every new message even when they aren’t retyped. The distinction between the two is the foundation for everything else, because input and output are counted, and often priced, separately.

In a multi-turn conversation, every new message includes the full conversation history as context. Each earlier turn, both the questions asked and the responses given, is resent in full on top of whatever is newly typed, since the model processes the whole thread fresh on every request.

Token consumption therefore accumulates with each exchange: even a short follow-up drags the entire prior thread along, so the cost of a single turn keeps climbing as the session grows.

On reasoning models, effort settings control how many tokens a model spends on internal reasoning before it produces a response.

This reasoning is what the model does privately to plan or think through a problem, separate from the visible answer that follows, so a higher effort setting means more hidden reasoning tokens, not a longer reply.

Those reasoning tokens are billed like output (on the expensive side), so an uncapped thinking pass can quietly inflate the cost of a single answer. The practical move is to match effort to the task: low for simple lookups and formatting, higher only for genuinely hard problems.

Context compression is the mechanism by which a platform summarizes or discards older conversation history as a session nears its context limit. Generating that summary costs tokens up front, but it replaces a long history with a compact stand-in, so every later message resends far fewer tokens and the session grows cheaper from there.

The trade-off is fidelity: afterward, earlier messages survive only as an approximation, no longer verbatim, so a precise instruction or constraint set early can quietly degrade.

Images and PDFs are the heaviest inputs a model can take. Each one is converted into tokens before its content can be read, a step plain text skips entirely.

The magnitudes are large: one image can run a few hundred to a couple thousand tokens, and a multi-page PDF (each page parsed as an image) can reach ten thousand or more.

So when the content is really just text, pasting it costs a fraction of uploading the file. Reserve images and PDFs for when the visual itself is the point—a chart or a UI bug—not a default container for words.

Across providers, output tokens cost meaningfully more than input, commonly three to five times more per token and sometimes higher. That makes the length of the response, not the prompt, the bigger driver of most bills. The practical consequence is where to spend effort: trimming a verbose prompt saves on the cheap side, while shaping the output (a shorter reply, no padding, only what’s actually needed) saves on the expensive one.

Caching is the mechanism by which a platform stores a snapshot of content it has already processed (a long system prompt, an uploaded document, the earlier turns) and reuses it on the next request instead of re-reading from scratch.

When supported, reuse is often automatic and billed at a steep discount, often a tenth of the normal input rate. But the snapshot only lasts a short, provider-specific window: miss it, and the full context is reprocessed from scratch at standard rates.

Verbose, filler-heavy prompts spend tokens without improving the result. Politeness padding, repetition, and overexplained context increase token counts without adding signal, so the same request stated plainly often costs far less while producing the same output.

Structure also enables useful trimming. Instead of narrated transitions, we can use an explicit label (a ## Context header or <context> tags in Claude) to communicate the same information in fewer tokens while removing ambiguity about where each section begins and ends. The gains from structure are usually smaller than the gains from cutting filler, but both help move the prompt toward a higher signal-to-token ratio.