A chatbot can appear responsive while leaving the visitor waiting for the information they need. An agency demonstration may show fast streaming text but hide slow retrieval or an external lookup. Define performance around complete user tasks and separate feedback from an answer that is actually ready to use.
Related service: Custom AI chatbot development- 01Record request start
- 02Measure retrieval and tools
- 03Measure useful answer arrival
- 04Review slow cases
SECTION 01
Agree what the timer includes
Specify whether measurements start at the send click, server receipt or model request. Record network conditions, device type, input length and whether caches are warm. For U.S. and remote global users, test the actual geographic access pattern rather than assuming the agency's development location represents everyone.
Microsoft's RAG architecture separates retrieval and generation. Ask for timing around these stages and any external business tool so a slow catalog lookup is not blamed on the model. Report a distribution of observations and slow-tail cases, not only the fastest successful demonstration or a small average.
SECTION 02
Illustrative example: a product comparison question
A visitor asks which of two products fits a specific requirement. The bot immediately displays a working indicator, retrieves the current specifications and begins a short answer. A live inventory check takes longer. The interface distinguishes the supported comparison from the still-pending availability lookup instead of mixing an unverified stock claim into the early text.
Acceptance records when the comparison becomes useful and when availability is confirmed. If inventory times out, the answer retains the verified comparison and explains the missing live result. This example shows why one response-time number does not describe every part of a multi-step conversation.
SECTION 03
Test the experience while waiting
Include long questions, empty retrieval results, expired sessions, parallel visitors and slow connected systems. Make cancellation and retry behavior understandable. W3C's status-message guidance is relevant to informing assistive-technology users without forcing focus away from their current activity.
Ask the agency to deliver the measurement method, baseline, agreed targets and known limits. Targets are project acceptance criteria, not universal promises. Compare reducing unnecessary tool calls or excessive answer length with changing the model. Keep source checks and access control intact, and rerun correctness tests after performance changes to catch shortcuts that damage the answer.
SECTION 04
Frequently asked questions
Is time to first text enough?
No. It can describe responsiveness, but measure when the visitor receives a useful answer and when any required action or lookup finishes.
Should every answer stream?
Not necessarily. Some outputs need validation before display. Choose streaming where partial content is understandable and does not imply unverified results.
Can we promise the same speed worldwide?
Not without suitable evidence and operating conditions. Network, provider and integration delays vary; define measurable project targets and limitations.
PRIMARY REFERENCES
Sources and further reading
These references cover the standards, platforms, or published prices discussed in the guide. Worked examples and checklists are our editorial guidance.
- Design and evaluate a RAG solutionMicrosoft Learn
- Accessible status messagesW3C WAI
EDITORIAL METHOD
About this guide
We use AI to assist with drafting and editing. Catapult AI Work is responsible for the published content. Examples illustrate possible approaches; they are not client case studies unless identified as such.
Budget examples are not Catapult package prices. Check linked provider pages for current fees and plan limits before making a purchase.
Read the editorial policy