Building with AI
Semantic caching
Reference entrySemantic caching
Semantic caching reuses earlier responses when a new request is sufficiently similar in meaning, reducing latency and model cost.
Where it sits
Semantic caching reuses earlier responses when a new request is sufficiently similar in meaning, reducing latency and model cost.