Once your agent is set to Live, it answers real students. This article gives you a repeatable way to test it through the same channels students use, and to keep checking it after launch.
Confirm with CollegeVine
Before testing starts, go through this checklist with your CollegeVine team.
Testers, timing, and channels. Share who is testing, when, and on which channels. Include the email addresses and phone numbers your testers will use, so we can tell test activity apart from student activity.
Audience. Confirm the agent's audience is correct. If a test contact is outside the Who should I be engaging with? segment, the agent will not respond to it.
Channels. Confirm all channels are configured. Open the workflow from Workflows and check that it shows Live and that the channel you are testing is selected. For example, if Phone is set to None, the agent will not answer calls.
Knowledge. Confirm all knowledge changes have been published. A live agent only uses published content.
Build a test group and establish testing protocols
Three or four people who know your topics (admissions, financial aid, registrar, IT) will catch more than one person testing alone. Add a student worker or two, because students phrase questions differently than staff do.
Test for volume
One of the most useful things you can do is test for volume. Pick a set of questions that covers the middle 75% of what students ask, the common questions your team answers every day, and ask each one a few times, in different words and on different channels. This tells you more than spending your time on the rare edge cases at either end of the curve.
Look for patterns, not one-off mistakes
Treat your agent like a person. A single odd answer does not mean something is broken, just as one slip from a new team member does not. Focus on patterns of behavior, such as the same kind of question answered wrongly more than once, and correct those. If you see a one-off mistake, ask the question a few more times. If it does not repeat, note it and move on.
Establish how you want to log feedback
Before testing starts, align with your testers on how you want to gather feedback. Agree on one shared place to log it, so every tester captures the same details and you can compare rounds of testing. For each question, record the expected answer and the web page or document it should come from. Here is a format that works well.
Test date | Tester name | Channel | Question or scenario tested | Feedback | Suggested changes or expected response | Knowledge update or behavior update? | Changes made? | Retest result | Notes |
Nov 12 | Tester 1 | Phone | When is the fall deposit deadline? | Agent gave last year's date | Date from the admissions deadlines page | Knowledge update | Yes | Pass | Old page removed from knowledge base |
Nov 12 | Tester 2 | Webchat | Can I talk to a person? | No handoff offered | Hand off to staff during office hours | Behavior update | No |
| Sent to CollegeVine |
Use Knowledge update when the answer is wrong or missing because of the content your agent has. Use Behavior update when the content is right but the agent's approach needs to change, such as tone, escalation, or how it asks follow-up questions. Check Changes made? once the fix is in, then retest and record the result. If you are not sure how to make a behavior change, reach out to your CollegeVine team and we will help.
Tip: To make this easy for your testers, create a short form (for example, a Google Form) with the same fields and send the responses to your sheet. Testers can submit feedback as they go without opening the sheet.
What to test
Knowledge accuracy
Ask your 20 to 30 most common student questions, including ones with dates, deadlines, fees, phone numbers, and links.
Ask the same question several ways: short, vague, misspelled, and as a follow-up to an earlier answer.
Ask questions the agent should not be able to answer. It should say it does not know and offer a next step instead of guessing.
If an answer is wrong, find the page or document it came from, fix the source, update your knowledge base, publish, and ask again. See Testing your knowledge assets.
Tone and sensitive topics
Check that the voice and name match what you chose in your agent persona.
Try sensitive situations, such as a student in distress, a safety concern, or financial hardship. Confirm the agent responds the way you want and routes the student to the right person.
Escalation and handoff
Ask for a person in several ways. On the phone, try "live person," "customer service," and saying "zero."
Test during your office hours and again after hours. The agent behaves differently in each case, so check both.
Test your call transfer setup. Check the transfer numbers and hours you configured, then call and ask for a person to confirm the transfer reaches the right line. Try it during your transfer hours and outside them, and check what the caller hears each time. See Configuring live call transfers.
Confirm a transferred call rings the right line, and check the caller ID your staff see.
Confirm escalation emails reach the right inbox. If you receive two emails for one escalation, tell your CollegeVine team so we can adjust your settings.
Phone
Call your dedicated number from a mobile phone. The agent should answer within a few rings.
Listen for pronunciation of names, buildings, and acronyms, and for awkward pauses or interruptions.
If you forward an existing number to the agent, call the existing number and confirm the forward works. Phone trees can stop a forwarded call from reaching a live person.
After each call, review the recording and transcript. Recordings are available to users with admin-level access.
Tip: If a voice does not sound right, preview other voices in your agent persona settings. Your CollegeVine team can also adjust the prompt for pronunciation. See the phone guide.
Send a question to the agent's email address from an address outside your institution, then reply to the answer to check that the conversation continues.
Check that replies arrive in the inbox and not in spam, and that the sender name looks right.
If you forward email to the agent, send a test through your forwarding rule. See Forwarding email to your agent.
Text messages
Text your dedicated number and check that replies are timely and read well on a phone.
Read the section on opt-outs below before testing with stop words.
Webchat
Test on a staging page first, in a private browser window, then on the live page. If you do not have a staging page on your own site, CollegeVine can provide one on request.
Webchat visitors are not verified, so a test chat creates its own webchat record. It does not attach to an existing record in your CRM.
Check the widget on desktop and on a phone.
If you need to meet accessibility standards, ask your accessibility team to review the widget, including how links display.
Avoid common testing problems
Opt-outs. Texting STOP or similar words can opt a number out of messages from all CollegeVine agents, and the agent can also decide to record an opt-out during a conversation. Use a dedicated test phone, avoid stop words unless you are testing opt-out on purpose, and tell your CollegeVine team if a tester stops receiving messages. We can restore the contact.
Test activity is real activity. Test conversations appear in conversation history and in your reports. Keep a list of your test contacts so you can account for them when you read the numbers. If you need a clean history at launch, tell your CollegeVine team early.
Do not test with real student records. Use test contacts so students do not get messages meant for a tester.
Allow time after changes. New knowledge base content needs to show a status of Ready and be published before the live agent uses it.
Keep checking after launch
Testing does not end at launch. A simple routine keeps the agent accurate as your website and policies change. For how to review conversations and make updates, see Maintenance best practices for post-launch agent management.
First week: read conversation transcripts and listen to call recordings every day. Check that escalations arrive and that someone on your team follows up.
After that: review on the cadence that fits your conversation volume, as described in the maintenance guide, and keep a list of questions the agent could not answer.
When your content changes: rerun your most common questions to catch outdated answers, then update and publish your knowledge base.
When you report a problem, include the date and time with time zone, the channel, the test contact, what you asked, and what the agent did. If a chat message gets no reply or gets two replies, send us the time it happened.
Quick checklist
Area | Check | It works when |
Setup | Workflow is Live, channel selected, test contacts in the audience | The agent responds to your test contact |
Knowledge | Top 20 to 30 questions, plus questions it should not answer | Answers match your sources, and unknowns are handled honestly |
Escalation | Request a person in several ways, in and out of office hours | The right person is reached, or the right message is sent |
Phone | Call from a mobile phone and through any forwarded number | The agent answers, sounds right, and the recording is available |
Send, reply, and check spam | Replies arrive in the inbox from the right sender | |
Text | Text the number from a test phone | Replies are timely and clear |
Webchat | Test on staging, then live, on desktop and phone | The widget loads and the agent answers |
Reporting | Share your test contacts with CollegeVine | You can separate test activity from student activity |