Documentation Index
Fetch the complete documentation index at: https://docs.coval.dev/llms.txt
Use this file to discover all available pages before exploring further.
List Metrics
coval metrics list [OPTIONS]
| Option | Type | Default | Description |
|---|
--filter | string | — | Filter expression (supports metric_type, metric_name, create_time) |
--page-size | number | 50 | Results per page (1-100) |
--order-by | string | — | Sort field, prefix with - for descending |
--include-builtin | flag | — | Include built-in metrics (e.g. Turn Count, Audio Duration) |
Get Metric
coval metrics get <metric_id>
| Argument | Type | Required | Description |
|---|
metric_id | string | Yes | The metric ID |
coval metrics get met_abc123
Create Metric
coval metrics create [OPTIONS]
| Option | Type | Required | Description |
|---|
--name | string | Yes | Metric display name |
--description | string | Yes | What this metric evaluates |
--type | string | Yes | Metric type (see below) |
--prompt | string | No | LLM evaluation prompt (required for llm-binary, categorical, numerical and their audio variants) |
--categories | string | No | Comma-separated categories (required for categorical, audio-categorical) |
--min-value | number | No | Minimum value (required for numerical, audio-numerical) |
--max-value | number | No | Maximum value (required for numerical, audio-numerical) |
--regex-pattern | string | No | Regex pattern to match (required for regex) |
--role | string | No | Transcript role to match against (optional for regex) |
--match-mode | string | No | presence (default) or absence — absence returns 1 if pattern NOT found |
--position | string | No | any (default), first, or last message of the role |
--case-insensitive | boolean | No | Enable case-insensitive matching |
--metadata-field-type | string | No | Metadata field type (required for metadata) |
--metadata-field-key | string | No | Metadata field key to extract (required for metadata) |
--min-pause-duration-seconds | number | No | Minimum pause duration threshold (required for pause) |
Metric Types
| Type | Description | Type-Specific Options |
|---|
llm-binary | Binary (yes/no) LLM judgment | --prompt |
categorical | Categorical LLM judgment with defined options | --prompt, --categories |
numerical | Numerical score from LLM judgment | --prompt, --min-value, --max-value |
audio-binary | Binary audio analysis | --prompt |
audio-categorical | Categorical audio analysis | --prompt, --categories |
audio-numerical | Numerical audio analysis | --prompt, --min-value, --max-value |
toolcall | Tool call success verification | — |
metadata | Extract metadata field value | --metadata-field-type, --metadata-field-key |
regex | Match transcript against a regex pattern | --regex-pattern, --role, --match-mode, --position, --case-insensitive |
pause | Analyze pause durations in audio | --min-pause-duration-seconds |
Examples
# LLM Binary
coval metrics create \
--name "Issue Resolved" \
--description "Did the agent resolve the customer issue?" \
--type llm-binary \
--prompt "Was the customer's issue fully resolved?"
# Categorical
coval metrics create \
--name "Sentiment" \
--description "Customer sentiment during the call" \
--type categorical \
--categories "positive,neutral,negative" \
--prompt "What was the customer's overall sentiment?"
# Numerical
coval metrics create \
--name "Professionalism Score" \
--description "Rate the agent's professionalism" \
--type numerical \
--min-value 1 \
--max-value 10 \
--prompt "Rate the agent's professionalism on a scale of 1-10"
# Audio Binary
coval metrics create \
--name "Background Noise" \
--description "Is there excessive background noise?" \
--type audio-binary \
--prompt "Is there excessive background noise in the audio?"
# Audio Numerical
coval metrics create \
--name "Audio Clarity" \
--description "Rate the audio clarity" \
--type audio-numerical \
--min-value 1 \
--max-value 5 \
--prompt "Rate the audio clarity on a scale of 1-5"
# Tool Call
coval metrics create \
--name "Tool Usage" \
--description "Did the agent use the correct tool?" \
--type toolcall
# Metadata
coval metrics create \
--name "Response Time" \
--description "Extract the response time from metadata" \
--type metadata \
--metadata-field-type "number" \
--metadata-field-key "response_time_ms"
# Regex — basic pattern match
coval metrics create \
--name "Greeting Check" \
--description "Did the agent greet the customer?" \
--type regex \
--regex-pattern "(hello|hi|welcome|good morning)" \
--role "agent" \
--case-insensitive
# Regex — compliance (absence mode)
coval metrics create \
--name "No Unauthorized Promises" \
--description "Agent must not make unauthorized promises" \
--type regex \
--regex-pattern "(guarantee|promise|definitely)" \
--role "agent" \
--match-mode "absence" \
--case-insensitive
# Regex — first message disclosure
coval metrics create \
--name "Recording Disclosure" \
--description "Agent must state recording disclosure in first message" \
--type regex \
--regex-pattern "this call may be recorded" \
--role "agent" \
--position "first" \
--case-insensitive
# Pause
coval metrics create \
--name "Long Pauses" \
--description "Detect pauses longer than 3 seconds" \
--type pause \
--min-pause-duration-seconds 3.0
Update Metric
coval metrics update <metric_id> [OPTIONS]
| Argument | Type | Required | Description |
|---|
metric_id | string | Yes | The metric ID to update |
| Option | Type | Description |
|---|
--name | string | New display name |
--description | string | New description |
--prompt | string | New evaluation prompt |
coval metrics update met_abc123 --prompt "Updated evaluation prompt"
Delete Metric
coval metrics delete <metric_id>
| Argument | Type | Required | Description |
|---|
metric_id | string | Yes | The metric ID to delete |
