Your support team has 40 real conversations to review before lunch. Some ask about pricing. Others need account access, a refund decision, or a human who can look at the customer record. A demo chat that answers one easy question tells you almost nothing.
To learn how to evaluate an AI chatbot, test it against the support workload you plan to launch, then make a pass or fail decision by category. Do not reduce launch readiness to one average score. A chatbot that handles setup questions well but invents refund rules is not ready for customer traffic.
Start with the support workload, not the chatbot
Write down what the agent must handle on day one. Name the support team responsible for review, the customer segment asking questions, the channels in scope, and the workload the agent must cover. Keep the launch boundary narrow enough to test properly.
Group real customer questions into categories such as:
- Pre-sales questions about plans, features, and fit
- Setup and onboarding questions
- Billing, refunds, and policy questions
- Account access and account-specific requests
- Questions with no approved answer in the knowledge base
Each category needs a pass condition. For example, the agent may need to answer approved product questions, collect a lead for a sales request, and hand billing disputes to a person. Account-specific questions may require verified identity or human review.
Write these conditions before you inspect the tool. Otherwise, a polished interface or a convincing demo can quietly change your standard.
Separate launch requirements from later additions. A multilingual widget, a new API action, or another support channel may matter later. It should not hide a failure in the core workload. Your first decision is narrower: can this agent handle the questions you are willing to put in front of customers now?
Build a test set from real customer questions
A useful test set starts with your existing language. Collect questions from support conversations, sales calls, contact forms, and website search data. Remove personal information, keep the intent and wording, and record the category for each question.
Include the awkward versions customers actually send:
- Short prompts such as “price?”
- Misspellings and missing punctuation
- Vague requests such as “I need help with my account”
- Follow-up questions that depend on the previous answer
- Messages that combine two intents, such as a plan question and a refund request
For every test case, record the approved answer and its source. A source might be a product page, an internal policy, or a support article. Use exact Q&A pairs for wording that must not be paraphrased, such as refund terms or eligibility policies. AssistLoop supports website crawling, uploaded files, pasted text, and exact Q&A pairs as training sources.
Add cases where the correct behavior is to refuse, ask for more information, or hand off. Include account-specific requests, billing disputes, and questions for which the knowledge base has no approved answer. A confident answer to an unknown question is a failed test, even if the writing sounds helpful.
Keep the test set stable. Run the same questions against every agent revision and save the results. That gives your team a real regression check when you change training content, the language model, or an instruction. AssistLoop lets you pick the right AI model, so model changes belong in the same review process as knowledge-base changes.
Technical teams can use a dataset and repeated evaluation workflow such as the one described in LangSmith’s chatbot evaluation tutorial. For broader metric ideas, see Microsoft’s guide to evaluating LLM-based chatbots. Your support acceptance test still needs to reflect your own customer questions and launch rules.
Test answer quality in a way your team can audit
Score each answer against the same fields. A reviewer should be able to see why a case passed or failed without relying on memory.
| Field | What to check |
|---|---|
| Factual accuracy | Does the answer match the approved product or policy information? |
| Source support | Can the reviewer identify the approved source behind the answer? |
| Specificity | Does it answer this customer’s question rather than a nearby one? |
| Customer action | Does the customer know what to do next? |
| Tone | Is the answer appropriate without masking an accuracy problem? |
| Reviewer decision | Pass, fail, or handoff required, with a short reason |
Record the source used, expected answer, actual answer, and reviewer decision for every case. Keep the original response. Replacing it with a corrected version destroys the evidence you need to diagnose the problem.
Test what happens after the customer challenges the first answer. Say, “That does not match what I was told,” or provide a detail that changes the situation. This exposes whether the agent checks its source, asks a useful question, or repeats an unsupported claim.
Review tone separately from accuracy. A polite answer can still be wrong. A correct answer can still create more work if it buries the next step or sends the customer to a page that does not answer the question.
When a failure repeats, fix the source rather than rewriting isolated responses. With training on your own data, you can update the agent through a website crawl, uploaded files, pasted text, or exact Q&A pairs. Use exact pairs where a policy answer needs controlled wording. Then rerun the failed cases and the surrounding category, not only the one question you edited.
Test handoff and actions with failure cases
A support agent should hand off when the answer needs account access, human judgment, a billing decision, or information missing from the knowledge base. Put those cases in the test set deliberately. Do not treat handoff as a failure of the AI. Treat an unmarked need for human help as a failure of the launch design.
Check the handoff after several turns, not only as the first message in a chat. Test an upset customer, a customer who changes topics, and a customer who asks for a person after receiving an answer. The reviewer should confirm that the team receives the full conversation history and enough context to continue without asking the customer to start over.
AssistLoop’s human handoff sends the thread to a shared inbox with its history. The team can reply from the dashboard or the iOS and Android apps. The handoff view also includes country, browser, and device details, which gives the reviewer more context when a conversation needs investigation.
Actions need their own failure tests. If the agent books a meeting, captures a lead, checks order status, or calls your API, test all of these cases:
- A required field is missing
- A value is invalid
- The API returns an error
- The customer repeats the request
- The customer cancels halfway through
The agent must not claim that an action succeeded when it did not. It should explain what happened and hand off or ask for the missing information. When the required information is available, confirm that lead capture, meeting booking, or an account lookup finishes inside the conversation. Agent Actions can call REST endpoints with GET, POST, PUT, PATCH, or DELETE requests, so your test should cover the exact action your team plans to use.
Check the evidence your support team will use after launch
A launch review does not end when the test set passes. Your team needs evidence from real conversations after traffic starts. Review conversation logs for unanswered questions, repeated customer phrasing, incorrect answers, and handoffs that lack useful context.
For each problem, identify its source:
- Missing training content
- A weak or unclear answer
- A failed action
- A poor escalation decision
Define the first post-launch review period before you publish the widget. Choose the conversation categories your team will inspect and the reviewer responsible for each one. Do not judge the agent from a single demo chat. Review a set of conversations that includes successful answers, handoffs, and unanswered questions.
Keep the operating measures small. Track answer accuracy on the test set, correct handoff rate, action completion rate, unanswered-question count, and message credits used. These measures tell you where to fix the agent. A single blended score does not.
Check message-credit capacity against the workload you defined. Each user message and each AI response counts as one credit, and credits do not roll over. AssistLoop’s Free plan has 150 message credits per month. Paid plans and current allowances are listed on AssistLoop pricing and message credits.
Last verified: July 2026.
Make a pass or fail launch decision
Create one launch table with a row for each support category. Use the conditions you wrote before testing.
| Support category | Answer quality | Source support | Handoff behavior | Action behavior | Reviewer decision |
|---|---|---|---|---|---|
| Pre-sales | Meets stated condition | Approved source found | Lead capture tested where needed | Booking tested where needed | Pass or fail |
| Setup | Meets stated condition | Approved source found | Human route tested for unknowns | Not required or tested | Pass or fail |
| Billing and policy | Meets stated condition | Exact policy source found | Dispute goes to a person | No unsupported action claim | Pass or fail |
| Account access | No unverified account answer | Identity or approved source checked | Full context reaches a person | Lookup failure tested | Pass or fail |
| Unknown questions | No invented answer | Missing source recorded | Handoff or refusal works | No action claimed | Pass or fail |
Pass only when every launch-critical category meets its stated condition. A strong average score cannot hide a failure in billing, account access, or another high-risk area.
Fail the launch if the agent invents policy, claims an action succeeded when it did not, loses context during handoff, or answers account-specific questions without verified identity. Fix the source, action, or routing rule, then rerun the affected cases and the full regression set.
You can also launch with a narrower workload. If the agent passes setup and approved product questions but fails account access, state that boundary clearly. Let the agent handle the passing categories and keep the failed questions with the human team until they pass their own acceptance tests.
After the final test, create your AI agent and use the approved test set as the first acceptance test before adding more traffic. The launch decision should come from evidence tied to your workload, not from how good the demo looked.
FAQ
What should be included in an AI chatbot evaluation?
Use real customer questions grouped by support category, then mark the approved answer, source, and required behavior for each case. A useful test set also includes vague prompts, follow-ups, misspellings, unknown questions, and requests that require a human.
Is one overall chatbot score enough for a launch decision?
No. A chatbot should pass each launch-critical category, even if its average score is high. A failure involving billing policy, account access, identity, or a customer action can make the launch unsafe.
How do you test AI chatbot handoff quality?
Test the handoff after several turns, during an upset conversation, and after a topic change. Confirm that the human receives the full thread and enough context to continue without asking the customer to repeat everything.
How should message credits affect chatbot evaluation?
Count the user messages and AI responses in the workload you defined. In AssistLoop, each user message and each AI response uses one message credit, and unused credits do not roll over.
Written by