Time to first token (TTFT) is how long you wait after sending a prompt before the first piece of the AI model's answer arrives.
In plain words
When a chat reply streams in word by word, the pause before anything appears is the time to first token. It covers waiting in a queue, the model reading your whole prompt (the prefill step), and network travel. Longer prompts usually mean a longer wait. Total response time is TTFT plus the time to produce the rest of the answer.
A quick example
Paste a very long document into a chatbot and ask a question. The gap before the first word appears is usually longer than for a short question, because the model first processes the whole document.
Where you'll meet it
- Measuring how responsive a chat app feels.
- Benchmarking LLM serving systems.
Related
Sources
- DocsLLM benchmarking metrics, NVIDIA NIM docs · read 8 Oct 2026
- DocsReducing latency, Claude Docs (Anthropic) · read 29 Sept 2026
Checked by WhatIsAI against the sources above · 8 Oct 2026. Spotted an error? Report it.