Concepts
Time to first token
Reference entryTime to first token
Time to first token is how long a model takes to start its reply after receiving a request, a key measure of how responsive an AI app feels.
Also known as TTFT.
Where it sits
Time to first token is how long a model takes to start its reply after receiving a request, a key measure of how responsive an AI app feels.
Also known as TTFT.