The test harness lets you safely simulate how your AI agent responds to various events under the scope of that workflow. By simulating emails, texts, or other events from actual constituents in your system, you can evaluate both the accuracy of the response and the agent’s behavior, all without impacting live communications.
We strongly recommend using the test harness to validate every workflow before you launch it.
What you can test with the test harness
The test harness allows you to:
Test a workflow even before launching it live
Simulate real constituent events, such as a constituent entering the segment, a document uploaded, or an inbound message
See how the agent reasons, what actions it takes, and why
Validate knowledge accuracy and behavioral decisions
Iterate confidently before real students ever interact with the agent
A workflow does not need to be live or launched to be tested. In fact, it’s best to start testing your workflow before it goes live!
How to access the test harness
Navigate to the Agent —> Workflows tab.
Select the workflow you want to test.
Open the workflow view and click Test.
If this is your first time testing, you will see a message indicating that you are starting a simulation.
Choose a constituent
Each simulation runs in the context of a real constituent, because your agent might behave differently based on the information it knows about the given constituent.
Search for a specific constituent by name, or
Select a random constituent
These options represent constituents already uploaded to your agent, ensuring realistic context during testing.
Once selected, click Start simulation.
Simulate an event
Next, choose the trigger event you want to simulate. You will only be able to select trigger events that have been configured for that workflow.
Common examples include:
Email received
SMS received
For most tests, we recommend starting with a common event you expect your agent to experience.
Example event
After entering the details, click Send.
What happens during the simulation
Once the event is simulated:
The agent reasons about the event, the context, and the behavioral guidance from the workflow
The agent decides what action(s) to take, such as whether to respond, escalate, or set reminders
The agent simulates the execution the appropriate action, such as sending an email
You will see a clear record of how the agent reasoned and what actions it would take.
Testing a follow-up scenario
The test harness is especially useful for multi-step scenarios.
Example workflow:
A first email is received from a constituent
The agent responds and sets a follow-up alarm
A second email arrives as a follow-up
The agent ends the original alarm
The agent sends a new response
A new follow-up alarm is created based on the latest message
This multi-step scenario allows you to validate that the workflow handles ongoing conversations and time-based follow-ups correctly.
How to evaluate the results
When reviewing a test run by examining the reasoning blocks and the corresponding actions, focus on two distinct areas.
1. Accuracy
Ask yourself:
Did the agent pull the correct information from the knowledge base?
Were deadlines, policies, and details accurate?
Did the response align with your institution’s official guidance?
Accuracy issues often point to gaps or improvements needed in your knowledge base.
2. Behavior
Consider:
Did it escalate when it should have?
Did it include unnecessary information?
Did it decide to follow up later?
What timeframe did it choose for a follow-up?
Behavioral changes can usually be adjusted by reconfiguring the workflow’s instructions, tools, or settings.
Best practices
Always test before launching a workflow
Review accuracy and behavior separately
Iterate based on test results, not assumptions
Re-run tests after making configuration or knowledge updates
Next steps
Once you are satisfied with how a workflow performs in the test harness, you can confidently move toward launching it live. If you need help interpreting test results or adjusting behavior, contact your CollegeVine partnerships team for guidance.
Testing early and often ensures your AI recruiter delivers accurate, helpful, and institution-aligned experiences from day one.