Before launching a business chatbot, test what it says, what it does and what happens when it cannot help. Use approved answers, realistic conversations and observable records in your CRM or support inbox. A polished reply is not proof that a lead was saved or a human received a handoff.
This guide proposes a practical acceptance process for a website assistant that answers service questions and captures enquiries. Adapt it to your own risks, languages, tools and support hours. The examples below are hypothetical test scenarios, not WaveINO customer results or a guarantee that a chatbot will never fail.
Define the launch scope before testing
Write a short statement of what the assistant is allowed to do. For example: explain published services, ask for the minimum information needed for an enquiry, and route unresolved questions to the support team. List the actions it must not perform, such as promising a discount that has not been approved or confirming a booking without a successful booking-system response.
Name a business owner for policy and answer approval, a technical owner for integration failures, and a support owner for escalations. Agree the actual support hours and the message visitors should receive outside them. Avoid promising an immediate human response when no one is available.
Build test cases with expected outcomes
Start with questions drawn from real sales and support work, after removing personal information. Include straightforward questions, vague requests, spelling errors, changes of mind and follow-up questions that depend on earlier context. If customers use more than one language, test each supported language explicitly. Do not assume that a successful English conversation proves equivalent performance in Hindi or Odia.
For every case, write the input, the relevant approved source, acceptable behaviour and the evidence required to pass. Focus on meaning and actions rather than insisting on identical wording. Google Cloud's test-case documentation shows how saved conversations can carry expectations about responses, tool use and conversational flow for later regression checks. Google Cloud test cases.
Keep a separate set of evaluation conversations that were not used to tune the assistant. Otherwise, a team may optimize for its familiar examples and miss the ways real customers phrase the same need.
Use this starter acceptance table
| Test scenario | Expected behaviour | Evidence to inspect |
|---|---|---|
| Ask about a published service | Answer stays within the approved service description | Reply and current source |
| Ask for an unpublished price | Explain the limitation and offer the agreed next step | No invented price or unsupported promise |
| Change the requested service midway | Update the enquiry context without mixing old and new details | Final summary and saved enquiry |
| Provide an invalid contact field | Ask for correction without claiming submission succeeded | Validation result and no false success message |
| Submit the same enquiry twice | Follow the agreed duplicate-handling rule | CRM records and submission identifiers |
| Ask for a person | Offer the correct handoff route | Assigned ticket or inbox record, not just a reply |
| Ask outside support hours | State the actual next step without an invented response time | After-hours routing and visitor message |
| Simulate an unavailable integration | Explain that completion failed and offer a recovery route | Failure record and absence of a false confirmation |
| Ask a question with no approved answer | Acknowledge the gap or route it | No unsupported factual claim |
| Attempt to access another customer's record | Do not disclose unrelated information | Access-control result in an authorized test environment |
These cases are a starting point. A bot that takes payments, changes bookings or accesses customer accounts needs additional acceptance criteria specific to those actions. Use test accounts and sandbox integrations so testing does not contact real prospects or alter real orders.
Check answers against current sources
Give the reviewer the exact service page, policy or knowledge-base entry that should support an answer. Test stale and conflicting source material deliberately: if the website and an old document disagree, the team needs an explicit source-priority rule rather than hoping the assistant chooses correctly.
Include a question where the source is incomplete. The expected outcome may be a clarifying question, a limited answer or a handoff. Repeating a confident answer is not a useful substitute for evidence. Record unsupported statements as failures even if the wording sounds professional.
Review factual accuracy separately from tone. A response can be polite but wrong, or factually correct but fail to tell the visitor what to do next. Separate ratings make the corrective action clearer.
Verify lead capture beyond the chat window
Submit a synthetic enquiry and inspect the actual destination record. Check service interest, contact details, source, timestamps and assignment. Make sure the information in the record matches the visitor's final answers, including corrections made during the conversation.
Then test interruptions: the visitor closes the widget, loses the connection, submits twice or returns later. Decide which cases should create a new enquiry and which should update an existing one. Document that decision so the team can recognize a failure rather than debating expected behaviour after launch.
For a service business, response ownership is part of the product. Use the CRM lead follow-up blueprint to plan what happens after capture. Do not measure success solely by the number of times the assistant says that an enquiry was received.
Test human handoff as an end-to-end workflow
A handoff should deliver enough context for the next person to act: the visitor's request, the relevant conversation summary, unresolved questions and the agreed contact route. Share only the information required by your approved process. Have the receiving team open the record and confirm that it is understandable.
Test a direct request for a human, repeated misunderstanding and a question outside scope. Check which queue receives each case and whether responsibility is clear. Test what happens when the destination is unavailable. A promise to connect someone should never conceal a failed transfer.
The visitor should understand whether they are entering live chat, sending a message for later review or being directed to a form. These are different service experiences and should have different confirmation messages.
Test failure recovery, access and usability
Ask your implementation team to run authorized checks for cross-user data exposure, unexpected tool actions and instructions embedded in untrusted source material. A few successful prompts do not prove that the system is secure. Record the tested boundary, the expected restriction and the observed result.
Check the actual website widget on desktop and mobile. Verify that the keyboard does not hide the input, long answers remain readable, links work, focus is visible and the widget can be closed. Try keyboard-only navigation and test the supported browsers. Include slow connections and a delayed response, with a clear way to recover or contact the team.
Agree operational thresholds before the test. Response time, failed submissions and unresolved handoffs need owners and review rules. There is no universal acceptable latency or pass percentage for every business; choose criteria that fit the action and its consequences.
Make the launch decision from evidence
Use a failure log with a test ID, observed behaviour, severity, owner, correction and retest result. Treat disclosure of unrelated customer information, unauthorized actions and false confirmations of important transactions as launch blockers in this proposed process. Do not hide them inside an average score across easier questions.
For less serious issues, document the limitation and decide whether a reduced launch scope is appropriate. For example, the assistant might launch for service explanations while an unreliable booking action remains disabled. The business owner should understand what is included and what is still excluded.
Automated evaluation can help repeat checks, but reviewers should inspect representative conversations and all critical failures. Google's playbook evaluation documentation describes test cases and reference responses used to assess several agent behaviours; platform metrics should support your acceptance decision, not replace it. Google Cloud playbook evaluations.
Plan the first week after launch
Assign someone to review failed answers, missing knowledge, abandoned conversations and handoff outcomes. Add real failure patterns to the test set after removing personal information. Rerun relevant cases whenever approved sources, prompts, integrations or model settings change.
Keep a documented fallback: who can disable an unreliable capability, where visitors should go instead, and how the team confirms recovery. A tested contact route is more useful than leaving customers inside a broken conversation.
Frequently asked questions
How many conversations should we test?
Choose coverage based on supported tasks and failure consequences. Include the main paths, common variations and important failure cases. A fixed number alone does not establish readiness.
Is an accurate FAQ answer enough to launch?
No. Verify the full visitor journey, including capture, error handling, routing and usability, for every capability included in the launch scope.
Can we test with real customer records?
Prefer synthetic records and approved test accounts. Follow your organization's data-handling rules and avoid sending real messages or changing real orders during acceptance testing.
What should a business receive at handover?
Request the approved scope, source inventory, test cases, failure log, retest evidence, ownership list and fallback procedure. These make future changes easier to evaluate.
Discuss the workflow and acceptance criteria with WaveINO, or explore AI automation services. A useful launch brief should define what the assistant can do, how success will be verified and who acts when it cannot complete the task.
Reviewed 23 September 2026. Test cases are illustrative and should be adapted to the approved business scope.