Skip to main content

​
Overview

Scheduled Runs let you run evaluations on a recurring cadence—hourly, daily, weekly, or on a custom interval. They’re built on top of Templates, which capture your full evaluation configuration (agent, test set, personas, metrics, and mutations). Each time the schedule fires, a new run is launched automatically with those exact parameters. Common use cases:
  • Regression detection: Catch when a new deployment breaks expected behaviors
  • Continuous quality monitoring: Track metric trends across agent versions
  • Health checks: Validate your agent is responsive and performing correctly at regular intervals

​
Prerequisites

Before creating a scheduled run, you need:
  1. An agent configured in Coval — see Agents
  2. A test set with the conversation scenarios to run — see Test Sets
  3. At least one metric selected for evaluation — see Metrics
  4. A saved template that ties these together — see Templates

​
Full Setup Flow

1

Create a Template

Navigate to Templates in the sidebar and click New Template.Configure your evaluation:
  • Agent: The voice or chat agent to test
  • Test Set: The conversation scenarios to run
  • Persona(s): How the simulated user should behave
  • Iterations: How many times each test case runs
  • Concurrency: How many simulations run in parallel
  • Metrics: Which metrics to evaluate against
Click Create Template to save. This template will be the source of truth for every scheduled run — all parameters are inherited automatically.
2

Schedule the Template

From the Templates list, click Schedule on your template. This opens the Schedule Evaluation dialog.Fill in the schedule configuration:Name Give the scheduled run a descriptive name (e.g., “Nightly Regression – Disputes Flow”). This appears in the Scheduled Runs list and in run history.Schedule TypeChoose between two scheduling modes:
Runs fire at a fixed cadence from when the schedule is created.Select a quick preset:
PresetInterval
15 minEvery 15 minutes
30 minEvery 30 minutes
1 hourEvery hour
6 hoursEvery 6 hours
12 hoursEvery 12 hours
DailyOnce per day
WeeklyOnce per week
MonthlyEvery 30 days
Or set a Custom Interval by entering a number and selecting minutes, hours, or days. The minimum is 15 minutes and the maximum is 30 days.
Click Schedule to create the scheduled run. It activates immediately.
3

Monitor Your Scheduled Runs

Navigate to Scheduled Runs in the sidebar to see all your configured schedules.The list shows:
  • Status: Active (running on schedule) or Disabled (paused)
  • Name: The label you gave the scheduled run
  • Agent: Which agent is being evaluated
  • Schedule: Human-readable frequency (e.g., “Daily at 9:00 AM”, “Every 6 hours”)
  • Template: The template powering this schedule (click to view its configuration)
  • Created: When the schedule was set up
Use the search bar to filter by name or ID, or filter the list by Active / Disabled status.Click any row to open the run history for that schedule — you’ll see every evaluation it has launched, with pass/fail results for each metric.

​
Managing Scheduled Runs

​
Enable and Disable

To pause a schedule without deleting it, open the actions menu (⋮) on any row and select Disable. Re-enable it the same way. You can also bulk-enable or bulk-disable: check multiple rows, then use the Enable Selected or Disable Selected buttons in the toolbar that appears.

​
Edit a Schedule

To change the name or timing of an existing schedule, open the actions menu and select Edit Schedule. You can update the display name, switch between interval and time-of-day modes, or adjust the frequency.
Editing a schedule does not affect the underlying template or evaluation parameters — only the timing changes. To update what gets evaluated (agent, metrics, test cases), edit the template directly.

​
Delete a Schedule

Scheduled runs must be disabled before they can be deleted. Once disabled, open the actions menu and select Delete. This action is permanent. To delete multiple schedules at once, disable them first, then select them and use Delete Selected.

​
Viewing Run History

Click any scheduled run to open its detail page. Here you can see:
  • Every run triggered by this schedule
  • The pass/fail result for each metric per run
  • Trend data showing how metric scores change over time
This is useful for spotting regressions: if a metric score drops across consecutive runs, something likely changed in your agent or its environment.

​
Tips

  • Start with daily schedules during active development. Hourly is better suited for production monitoring where you need fast feedback.
  • Name schedules clearly — include the agent name and what it tests (e.g., “Hourly – Billing Bot – Core Flows”).
  • Use the Template link in the Scheduled Runs list to quickly verify what configuration is being used before debugging a failing run.
  • Disable rather than delete schedules you might want to resume — deleted schedules and their history are gone permanently.