tokens

The fundamental units that LLMs process. Roughly a word or subword. Count matters for context window limits and API billing.

Syntax

prompt-engineering

1 token ≈ 4 characters ≈ 0.75 words (English)

Example

prompt-engineering

"Hello, World!" = 4 tokens
"Hello", ",", " World", "!"

"ChatGPT" = 3 tokens: "Chat", "G", "PT"

// Context window: how many tokens fit in one request
// Claude Sonnet 4: 200k token context
// 1k tokens ≈ 750 words ≈ 1.5 pages

// Prompt tokens + completion tokens = total usage