A support lead approves a chatbot on Friday because it answered five clean demo questions. On Monday, a customer asks about a disputed charge, the agent guesses, and nobody owns the handoff. This chatbot support launch checklist prevents that approval. Every gate needs a pass, a named owner, and recorded evidence before the agent reaches customers.
Set one launch decision before testing
A successful demo does not justify a production launch. Approve the AI support agent only when every launch gate passes and the result can be checked later.
Create one launch record before testing starts. Keep it with the test evidence, not in a separate project thread that disappears after launch.
| Launch record field | What to record |
|---|---|
| Agent version | The training version or content set used in testing |
| Test date | The date and time of the test run |
| Reviewer | The person who ran or reviewed each case |
| Selected plan | The AssistLoop plan active for the release |
| Open defects | Failed cases, severity, and current status |
| Follow-up owner | A named person for each unresolved item |
| Final decision | Go or no-go, with the approving names |
Use pass or fail for every case. Do not mark a case as passed because the answer sounded plausible. A pass means the answer matches approved content, the behavior fits the support workflow, and the evidence is attached.
Your single decision is simple: can this agent answer from approved sources, handle failure safely, reach a person when needed, work in the website widget, and receive daily review during its first week? If one of those conditions is unproven, the decision is no-go.
Gate 1: approve the knowledge base before testing answers
Answer testing is worthless if the source material is stale. Start by listing every approved training source. Remove expired pricing, old policies, retired product details, and internal notes that customers should never see.
Assign a source owner to each document or page. The owner confirms that the content is current and approved for customer-facing answers. Include the questions your support team receives most often. Add exact Q&A pairs for wording that must not be paraphrased, such as refund rules, eligibility requirements, or legal wording.
AssistLoop supports four training sources: files, website crawling, pasted text, and exact Q&A pairs. Train the agent on your data before you judge its answers. The source choice matters. A refund rule that needs exact wording belongs in a controlled Q&A pair, not only in a long page where the agent may need to find and restate it.
Test a small sample against the source of truth. For each case, record the passage that supports the expected answer. An answer fails when it is correct by accident but cannot be traced to approved content.
Evidence required before this gate passes:
- Approved source list and source owner for each item
- Test questions and written expected answers
- Actual agent answers, including any missing or extra claims
- Source passage supporting each passed answer
- Unresolved content gaps and their owners
Do not move to broad answer testing while the source list still contains retired material. The agent cannot compensate for an approval process that has no current source of truth.
Gate 2: test normal, incomplete, and difficult questions
Build test cases from real support conversations. Clean questions written by the person who configured the agent test the configuration, not the customer experience.
Include common questions, misspellings, incomplete requests, follow-ups that rely on earlier context, and questions that combine two policies. Test the same intent with different wording. A strong answer should not depend on one phrase that appeared in the training data.
For each case, define the expected behavior before sending the message. The result may be an answer, a request for missing information, or a human review. Record which one occurred.
A useful test row looks like this:
| Test case | Expected result | Actual result | Pass condition |
|---|---|---|---|
| Customer asks about a current plan | Accurate answer from an approved source | Agent response and source | Facts match the approved plan content |
| Customer omits an order or account detail | Agent asks for the missing detail | Follow-up message | No invented account-specific answer |
| Customer sends a misspelled product term | Agent identifies the intended topic or asks for clarification | Full exchange | The conversation moves toward a supported answer |
| Customer follows up with “What about the other one?” | Agent uses earlier context or asks which item they mean | Full exchange | Context is preserved without a guess |
| Customer combines refund and eligibility rules | Agent handles both rules or sends it for review | Full exchange | Neither policy is dropped or blended incorrectly |
Inspect conversation logs after the run. A screenshot of the final answer is weak evidence because it hides the prompt, earlier context, and any failed handoff. The record should show the full exchange.
Gate 3: prove safe fallback behavior
The most important pre-launch test is what happens when the agent cannot support an answer. Test unsupported product questions, requests outside the approved sources, sensitive personal details, complaints about charges, and requests that require a policy decision.
A pass has three parts. The agent says what it cannot confirm. It avoids inventing an answer. It follows the defined next step. A fail includes a confident guess, an unapproved policy, or repeated wording that leaves the customer in the same place.
Write the fallback response before testing. Then test it in direct wording, indirect wording, and an agitated customer message. The wording should be clear enough that the customer knows what happens next. “I can’t confirm that from the information available here” is useful only when the conversation then reaches the right support path.
Some topics must reach a person even when the agent knows the factual answer. A disputed charge, serious complaint, or policy exception can require judgment and care. Accuracy alone does not make the interaction suitable for automation.
Record these stop-ship cases:
- The agent invents a price, policy, product detail, or account fact
- The agent presents an unapproved source as an answer
- The agent repeats a failed response after the customer clarifies the request
- The defined next step does not work or has no owner
- A sensitive request receives a specific answer without the required verification or review
Evidence required: sensitive-prompt test cases, expected fallback behavior, captured conversations, and the list of stop-ship failures. Link your test criteria to the OWASP Top 10 for Large Language Model Applications when reviewing risks such as prompt manipulation or information disclosure.
Gate 4: verify human handoff and ownership
Run a live handoff from the website widget. Confirm that a team member receives the conversation with its history and can reply without asking the customer to repeat the issue. Test the handoff after a normal answer and after a failed answer. The path should work in both situations.
Name an owner for incoming handoffs, an expected coverage window, and a backup owner. “The support team” is not an owner. The launch record should identify the person or queue responsible for the next reply.
Test at least these handoff paths:
- An unanswered product question
- An angry customer message
- A billing issue or disputed charge
- A conversation that begins with the agent and needs a person later
AssistLoop human handoff is available on paid plans. The Free plan does not include human handoff, so a production launch that depends on takeover cannot pass on Free. Last verified: August 2026. Check AssistLoop human handoff and AssistLoop pricing when choosing the plan for the release.
Evidence required: completed takeover test, recipient confirmation, reply test, ownership schedule, backup owner, and plan check. If the first human reply has no clear owner, the gate fails even when the technical handoff works.
Gate 5: inspect the website widget as a customer
Test the widget on the published or staging site. Use the production theme, the pages where support questions begin, desktop layouts, and mobile layouts. A correct answer does not matter if the customer cannot open the widget or the widget blocks checkout controls.
Check that the widget opens, accepts a message, displays the response, preserves the conversation, and shows the handoff path. Test a slow or interrupted session. Record what the customer sees when a response or takeover does not complete.
Review the greeting, name, avatar, brand color, light or dark theme, and suggested replies. Remove any suggested reply that leads to content the agent cannot answer. Each visible prompt creates an expectation, so it belongs in the test set.
Use the AssistLoop widget customization settings to match the site without hiding the controls customers need. Also check keyboard access, focus order, readable contrast, and mobile touch targets against the WCAG standards and guidelines.
Evidence required:
- Staging URL or release candidate
- Browser and device list
- Screenshots of the open widget, active conversation, and handoff state
- Results from a slow or interrupted session
- List of visual and interaction defects
A widget that works in one browser on the installer’s laptop has not passed. Test the places and devices your customers actually use.
Gate 6: run the release controls and sign-off
Use a staging environment or restricted release before sending every visitor to the agent. Limit exposure while the team confirms the widget, answers, logs, and handoff path.
Before publishing, check the production Agent ID, approved domain or embed configuration, greeting, training version, and team notification path. Compare each item with the launch record. Do not paste an Agent ID from a test agent into the production release.
Set the launch owner and a rollback or disable procedure. The team must know who can turn off the AI agent when a bad answer pattern appears. Write the procedure where the support owner can find it during an active incident.
Require sign-off from the content owner, support owner, and person who tested the site. Each signer approves the evidence. Signing a statement that “testing was completed” is not the same decision.
Stop the launch when:
- Source accuracy is unproven
- Unsupported questions produce confident claims
- Human handoff cannot be completed
- No one owns incoming conversations
- The production widget has not been tested
The final record should show the decision, date, signers, open defects, and the person authorized to disable the agent. If the record only says “ready,” it is not a release control.
Watch the first week and reopen failed gates
A launch record is still active after publication. Review conversation logs every day during the first week. Tag unanswered questions, repeated customer confusion, handoff requests, incorrect answers, and questions that need new Q&A pairs.
Track the measures that fit the support workflow: deflection rate, handoff volume, first response time after takeover, and CSAT when available. Label any current team or AssistLoop measurement with its review date. Last verified: August 2026. Do not publish a number without checking the underlying logs and recording when it was measured.
Turn repeated failures into a source update or exact Q&A pair. Then rerun the original test case. Do not close a defect because the answer improved in one conversation. The original wording, the alternate wording, and the full conversation log belong in the retest evidence.
Hold a day-one review and a first-week review with named owners. Keep the launch record open until stop-ship defects are closed or the agent is disabled. If your team has passed the checks, create an AI agent and use the same record to approve the first production release.
FAQ
What should be tested before launching a customer support chatbot?
Test real support questions alongside incomplete requests, misspellings, follow-ups, sensitive prompts, and unsupported topics. Review the full conversation logs and confirm that the widget, fallback path, handoff process, and ownership schedule all work together.
How do you know if a support chatbot is ready to go live?
It is ready when every defined gate has a recorded pass, a named reviewer, and no stop-ship defect. The approval should cover source accuracy, safe failure behavior, human handoff, website behavior, release controls, and first-week monitoring.
What is the most important chatbot pre-launch test?
Test the agent with a question it cannot answer from approved content. A passing result states the limit clearly, avoids an invented claim, and sends the customer to the right next step when human review is required.
What should a support team monitor during the first week after chatbot launch?
Review conversation logs each day for incorrect answers, unanswered questions, repeated confusion, and handoff requests. Compare those findings with deflection, takeover response time, and CSAT when those measures are available.
Does AssistLoop support human handoff for customer support chats?
Yes, AssistLoop includes human handoff on paid plans. The Free plan does not include it, so teams that require takeover should confirm the selected plan before approving production.
Written by