A customer asks, “Can I cancel after the trial?” Your agent finds three answers: one in the help center, one on the pricing page, and one in an internal note. It picks the wrong one. That is why chatbot knowledge base setup should start with failure modes, not file uploads. Build around real questions, clean up conflicting sources, test difficult cases, and decide when a person must take over.
Start with the questions your customers already ask
Do not begin by connecting every help article you have. Begin with recent support conversations.
Export a sample of conversations from your inbox or chat tool. Group them by the issue the customer had, not by the internal team that handled the conversation. “Where is my order?” belongs with other order-status questions, even if sales, support, and operations each answered it.
Mark each question with one of four labels:
- Repetitive: the same question appears often and has a stable answer.
- Account-specific: the answer depends on a customer record, subscription, order, or private detail.
- Policy-sensitive: a wrong sentence could create a refund, legal, security, or compliance problem.
- Unsuitable for automation: the customer needs judgment, negotiation, or a person who can take responsibility.
Build the first knowledge base around repetitive questions with stable answers. Leave account-specific and unsuitable questions out until you have a safe way to handle them. Policy-sensitive questions can belong in the knowledge base when the source is current and the wording is precise. They also need a clear handoff path.
Remove outdated articles, duplicates, internal notes, and pages written for employees. An agent cannot tell that “internal escalation procedure” was never meant for customers unless you keep it out of the source material.
Audit every source before you connect it
Every source needs an owner, a last review date, a product version or scope, and a clear audience. If an article has none of these, treat it as unapproved content until someone checks it.
Look for conflicts between the places customers might find an answer:
- Pricing pages and plan descriptions
- Refund and cancellation policies
- Setup documentation
- In-app labels and onboarding copy
- Security or privacy commitments
- Sales pages and support articles
Resolve the conflict before training the agent. Do not expect the model to choose the newest page by itself. Pick one approved answer, archive the others, and record why the approved version wins.
Separate public answers from internal procedures. A customer can be told that a refund request needs review. They should not see the internal approval steps, staff notes, or private links used to process it.
For wording that must stay exact, write a question and answer pair instead of relying on a broad article. Refund terms, cancellation windows, security commitments, and eligibility rules are good candidates. AssistLoop supports uploaded PDF, DOCX, and TXT files, website crawling, pasted text, and exact Q&A pairs. Its train on your data feature gives you several ways to supply approved content.
The Intercom setup guide and Zendesk guide to knowledge base chatbots provide useful background. Your audit still needs to happen inside your own support content. Generic guidance cannot resolve a conflict between your current refund page and an old article.
Write content that gives retrieval a clean target
Give each article one job. “How billing works” is too broad if it contains invoices, failed payments, upgrades, refunds, and tax details. Split it into focused pages or sections with a clear question at the top.
Put the direct answer first. Add conditions, exceptions, and steps below it. A useful structure looks like this:
Question: Can I cancel my subscription at any time?
Answer: You can cancel from the billing page. Your access remains available until the end of the current billing period.
Conditions: Cancellation does not automatically create a refund. Refund requests follow the refund policy.
Next step: If you cannot access the billing page, ask for human help.
Use the product names and terms customers use in chat. If your team calls a feature “workspace provisioning” but customers call it “adding a team,” include the customer term. Do not switch between five internal labels for the same thing.
Break broad pages into focused sections. Each section should answer one question without pulling in unrelated instructions. This helps the agent find the relevant passage and reduces answers that combine two policies into one confusing reply.
Write the failure states. Explain what happens when a feature is unavailable, a plan does not qualify, a payment fails, an order cannot be found, or an account lacks permission. Customers ask about the exception as soon as the normal path fails.
A knowledge base should also state its limits. “We do not support this action in chat” is more useful than a confident answer that sends a customer through the wrong process.
Build a test set before you publish
A test set is a small collection of questions with approved answers. Build it from real conversations, not from questions that sound neat in a planning document.
Include misspellings, short messages, follow-ups, and missing context:
- “where is my orde”
- “Can I change it?” after a question about a subscription
- “refund?” with no account details
- “How do I add someone” when the product has more than one type of user
Add questions that expose source problems. Test conflicting policy pages, incomplete setup instructions, and questions where the answer is not present. The agent should admit uncertainty or hand off instead of filling the gap with a guess.
Score each answer on four points:
- Factual accuracy: Does it match the approved source?
- Source fit: Did it use the right policy or product instruction?
- Tone: Is it clear and appropriate for a customer?
- Next step: Does the customer know what to do now?
Keep the test set after launch. Retest after a product release, pricing change, documentation rewrite, or policy update. Review conversation logs as well. New questions, repeated corrections, and handoff requests show you what to add next.
Decide where a person must take over
Some questions should go to a person even when the agent knows the policy. Send billing disputes, account-specific requests, complaints, security concerns, and unclear policy questions to your team.
The handoff should preserve the full conversation. A customer should not have to repeat the problem, explain which plan they have, or paste the answer they already received. AssistLoop’s human handoff feature passes the thread and its history to the team on paid plans.
Write the handoff message around the next step:
“I can’t verify that account detail here. I’m sending this conversation to the support team so they can review it. You won’t need to repeat the details already shared.”
Do not tell the customer the issue is solved when the agent has only created a handoff. The message should set an honest expectation and make the transition clear.
AssistLoop can also be used as a live-chat inbox with the AI turned off. That makes human handling available when automation is the wrong choice for a particular workflow.
Connect actions only after the answers are reliable
An action changes the risk. A wrong answer wastes time. A wrong API call can change an order, create a lead, or book a meeting the customer did not intend to book. Add actions after the knowledge base passes its test set.
Start with one narrow outcome. Booking a meeting, capturing a lead, or checking order status through an API are easier to test than a general instruction to “manage customer accounts.”
Before an action runs, define the fields the agent must collect. For an appointment, that might include the customer’s name, email, meeting type, and preferred time. For order status, it might require an order number and a verified user identity.
Write the failure path before connecting the endpoint. The agent needs instructions for three cases:
- The action fails.
- The endpoint returns no result.
- The request needs human review.
It should say what happened, avoid exposing technical details, and give the customer a useful next step. AssistLoop Agent Actions can call REST endpoints using GET, POST, PUT, PATCH, or DELETE. The public API gives developers a way to connect the surrounding product and control access.
If the customer only needs a static explanation, keep the interaction informational. Use an action when the customer needs work completed inside the conversation.
Assign ownership before quality drops
A knowledge base without ownership becomes a storage bin. Give each source a named owner who reviews changes and removes stale guidance.
Set review triggers for product releases, pricing changes, policy updates, and repeated incorrect answers. Do not wait for a quarterly calendar review when a refund policy changes today.
Track three signals in conversation logs: unanswered questions, handoff requests, and correction patterns. If customers keep asking for the same missing detail, add it to the source. If the agent keeps combining two policies, split the content or add an exact Q&A pair.
Keep a change record for high-risk answers. Record what changed, who approved it, and why. This gives the team a way to trace an answer when a customer questions it later.
AssistLoop’s conversation logs let you review what customers asked and where the agent needed help. The useful review question is simple: what did the customer need that the current source did not provide?
The operating playbook to use next week
Use this five-day plan for a first production-ready pass:
- Day 1: Collect real questions. Remove unsafe, stale, duplicate, and internal sources.
- Day 2: Rewrite the highest-volume answers. Add exact Q&A pairs for sensitive wording.
- Day 3: Run the test set. Fix weak answers and define the handoff rules.
- Day 4: Add one narrow action. Test success, failure, no-result, and human-review paths.
- Day 5: Publish with a named owner, review triggers, and a written list of questions the agent must hand off.
The most common setup mistake is publishing the first version and treating the work as finished. Your conversation logs become the next source of test questions. Use them to correct the knowledge base before customers find the same gap again.
AssistLoop is a poor fit if you need a ticket queue or a shipped multi-channel support system today. It is built for an AI support agent on your site, with human handoff and actions when those workflows fit your support operation.
When your sources are ready, create an AssistLoop AI agent and test it against your own questions before inviting customers in.
FAQ
What should I add to a chatbot knowledge base first?
Start with recent support conversations, then group questions by customer issue. Add the questions that occur often and have stable answers, while removing stale, duplicate, and internal content.
Should I use FAQs or full help articles?
Use exact question-and-answer pairs for wording that must remain precise, such as refund terms, cancellation rules, and security commitments. Broad articles work better for general explanations and procedures.
How do I test a chatbot knowledge base?
Test misspellings, short messages, follow-up questions, missing context, and questions with no approved answer. Score accuracy, source fit, tone, and whether the response gives the customer a useful next step.
When should a chatbot hand off to a human?
Send billing disputes, account-specific requests, complaints, security concerns, and unclear policy questions to a person. The handoff should include the full conversation so the customer does not need to start over.
Written by