Skip to main content
Templates let you save evaluation configurations—including agent, test set, persona, and metrics—so you can launch simulations consistently with one click. You can also schedule recurring evaluations from any template.

​
Creating a Template

Navigate to Templates in the sidebar, then click New Template.

​
Configuration Steps

The template creation form walks you through each component: 1. Select Agent Choose the voice or chat agent you want to test. Your agent connection settings (phone number, websocket URL, etc.) are preserved from your agent configuration. 2. Select Persona(s) Choose how the simulated user should behave. You can select multiple personas—each persona will create a separate run, letting you compare performance across different user types. 3. Select Test Set Pick the test cases that define the conversation scenarios. These determine what the simulated user will say and do during the evaluation. 4. Set Iterations Define how many times each test case runs. With 2 test cases and 3 iterations, you’ll get 6 total conversations. 5. Set Concurrency Control how many simulations run in parallel. Higher concurrency speeds up evaluation but may hit rate limits on your agent infrastructure. 6. Select Metrics Choose which metrics to evaluate. These can be built-in metrics (latency, interruptions) or custom metrics you’ve created. 7. Select Mutations (Optional) If you’ve set up agent mutations, select which variants to test. Each mutation creates a separate run comparing the base agent against the mutated version. 8. Save Template Click Create Template to save. Your template now appears in the templates list.

​
Launching from a Template

From the Templates list:
  1. Find your template and click Run Now
  2. Review the pre-filled configuration
  3. Click Launch Evaluation to start immediately
Or from the Launch Evaluation page:
  1. Select Use Template
  2. Choose your saved template
  3. Customize any settings for this specific run
  4. Launch

​
Scheduling Recurring Evaluations

Templates can power scheduled, recurring evaluations—useful for continuous monitoring and regression detection.

​
Creating a Scheduled Run

  1. From the Templates list, click Schedule on your template
  2. Configure the schedule:
    • Name: Identify this scheduled job
    • Frequency: Hourly, daily, or weekly
    • Start/End dates: Optional window for the schedule
  3. Review the template configuration that will be used
  4. Click Create Schedule
The scheduled run inherits all template settings—agent, personas, test set, metrics, and mutations. Each time the schedule triggers, it launches a new evaluation with those exact parameters.

​
Managing Schedules

View all scheduled runs in the Scheduled tab:
  • Active: Schedules currently running on their cadence
  • Paused: Temporarily disabled schedules
  • Completed: Schedules that reached their end date
Click any schedule to see its run history, success rate, and trend metrics over time.

​
Best Practices

Template Organization
  • Create templates for each major workflow you test regularly
  • Name templates descriptively (e.g., “Disputes - Angry Customer Persona”)
  • Use folders or naming conventions to group related templates
Scheduled Runs
  • Start with daily schedules for active development
  • Use hourly only for high-traffic production monitoring
  • Set end dates for temporary testing periods
Mutation Testing
  • Create templates with mutations to validate prompt changes
  • Compare base vs. mutated results before deploying changes

​
Deprecated Features

The legacy “Scheduled Evaluations” feature has been removed. All recurring evaluations now use Templates with Scheduled Runs, which provides:
  • Better visibility into configuration
  • Consistent parameter inheritance
  • Centralized management in the Templates section