Chat Simulations - Coval Documentation

Overview

If you have chat agents or want to simulate your voice agent conversations without calls, you can use Coval the same way it’s used for voice simulations by generating text conversations. Create test sets, define metrics, and configure templates to evaluate your chat agents automatically. Requirements: Provide a custom text endpoint that Coval can connect to

Quick Start

Minimum Required Configuration:

Chat Endpoint - The URL where your agent receives messages
Authorization Header - Authentication credentials for your API

That’s it! All other fields are optional and depend on your specific API requirements.

Core Configuration

Chat Endpoint (Required)

The primary URL where Coval sends conversation messages during simulation. Format: Full HTTPS URL

https://api.yourdomain.com/chat

Requirements:

Must use HTTPS (HTTP will be auto-upgraded)
Cannot use local/private IP addresses
Must return JSON responses

Example:

https://api.yourdomain.com/v1/chat

Authorization Header (Required)

Authentication credentials sent with every API request. Common Formats:

Bearer Token

Bearer your-secret-token-here

API Key

API-Key your-api-key-here

Custom Authorization

Custom-Auth your-custom-format

UI Tip: Use the dropdown to select your auth type, then paste your token/key. The system automatically formats the header correctly.

Standard Protocol

The standard integration for chat uses an HTTP, JSON-based API endpoint that you provide. When running a simulation, Coval’s simulator, acting as a user, will connect to the endpoint to get responses from your agent. We use OpenAI’s chat completions format, although we also support receiving responses in the OpenAI responses format. Query strings are not allowed in URLs.

Optional Configuration

Initialization Endpoint

Called once before the conversation starts to set up session state. When to use:

Your API requires session initialization
You need to obtain a session ID or auth token
You want to pre-configure conversation context

Format: Full HTTPS URL

https://api.yourdomain.com/init

Example Response:

{
  "sessionId": "abc-123-def",
  "userId": "user-456",
  "conversationId": "conv-789"
}

How it works:

Coval calls the initialization endpoint with your initialization payload
The response is captured and made available to subsequent chat requests via template variables like {{sessionId}} or {{init_response.conversationId}} in your input template and custom headers

The initialization endpoint is called before any chat messages are sent. The input template is only used for chat requests — it does not affect the initialization call.

Initialization Payload

JSON payload sent to the initialization endpoint. Format: Valid JSON

{
  "user_id": "test-user",
  "config": {
    "language": "en",
    "temperature": 0.7
  }
}

Template Variables Available:

Variable	Description	Example Value
`{{simulation_output_id}}`	Unique ID for this simulation	`"sim-abc-123"`
`{{persona.field}}`	Data from persona metadata	`{{persona.user_id}}`

Example with Variables:

{
  "session_id": "{{simulation_output_id}}",
  "user_context": {
    "user_id": "{{persona.user_id}}",
    "preferences": {}
  }
}

Persona Integration: To use {{persona.field}} variables, add initialization_parameters to your Persona’s metadata:

{
  "initialization_parameters": {
    "user_id": "customer-123",
    "account_type": "premium"
  }
}

Then reference in payload:

{
  "user_id": "{{persona.user_id}}",
  "account_type": "{{persona.account_type}}"
}

Custom Data

Additional JSON data included in every chat request (for APIs using standard payload format). Format: Valid JSON

{
  "metadata": {
    "source": "coval-evaluation",
    "version": "1.0"
  },
  "context": {
    "department": "sales"
  }
}

How it’s sent:

{
  "messages": [...],
  "customData": {
    "metadata": {...},
    "context": {...}
  }
}

Note: Only used when NOT using input_template. If you use input_template, reference custom data with {{custom_data.field}} instead.

Custom Headers

Additional HTTP headers sent with every chat request, with support for dynamic values from the initialization response. Format: Valid JSON object with string keys and values

{
  "X-Session-ID": "{{sessionId}}",
  "X-User-ID": "{{init_response.user.id}}",
  "X-Custom-Header": "static-value"
}

Template Variables Available:

Variable	Description	Example
`{{sessionId}}`	Session ID from init response	Extracted from `init_response.sessionId`
`{{simulation_output_id}}`	Unique simulation ID	Generated by Coval
`{{init_response.path}}`	Any nested field from init response	`{{init_response.auth.token}}`

Common Use Cases: Use Case 1: Session ID in Header

{
  "X-Session-ID": "{{sessionId}}"
}

Use Case 2: Nested Auth Token

{
  "X-Auth-Token": "{{init_response.auth.token}}"
}

Use Case 3: Mixed Static and Dynamic

{
  "X-Session-ID": "{{sessionId}}",
  "X-API-Version": "v2",
  "X-Simulation-ID": "{{simulation_output_id}}"
}

Field Size Limit: 16kB maximum

Chat Messages

Your chat endpoint should be an HTTPS URL that will respond to POST requests with a JSON body. If an Authorization token was provided, it will be included in the headers. Initial Request from Coval:

{
  "sessionId": "XXXX",
  "customData": {},
  "messages": [
    {
      "role": "user",
      "content": "Initial reach out text"
    }
  ]
}

Expected Response Format: Standard format:

{
  "messages": [
    {
      "role": "assistant",
      "content": "Response to the initial text"
    }
  ]
}

Or in the newer Responses format:

{
  "messages": [
    {
      "role": "assistant",
      "content": [
        {
          "type": "text_output",
          "text": "Thanks for contacting us"
        }
      ]
    }
  ]
}

Advanced Configuration

Response Format

Determines the format for tool call responses sent to your API. Options:

Chat Completions (Default)

Standard OpenAI format for tool responses:

{
  "role": "tool",
  "content": "result",
  "tool_call_id": "call_123"
}

Responses API

Alternative format for function call outputs:

{
  "type": "function_call_output",
  "call_id": "call_123",
  "output": "result"
}

Configure the response format by adding response_format to your model configuration:

{
  "response_format": "responses_api",
  "chat_endpoint": "https://api.your-company.com/chat",
  "authorization_header": "Bearer your-api-key"
}

Coval will respond with the entire chat history in the format specified:

Chat Completions (Default)
Responses API

{
  "sessionId": "XXXX",
  "customData": {},
  "messages": [
    {
      "role": "user",
      "content": "Initial reach out text"
    },
    {
      "role": "assistant",
      "content": "Thanks for contacting us"
    },
    {
      "role": "user",
      "content": "When will my order arrive?"
    }
  ]
}

{
  "sessionId": "XXXX",
  "customData": {},
  "messages": [
    {
      "role": "user",
      "content": "Initial reach out text"
    },
    {
      "role": "assistant",
      "content": [
        {
          "type": "text_output",
          "text": "Thanks for contacting us"
        }
      ]
    },
    {
      "role": "user",
      "content": "When will my order arrive?"
    }
  ]
}

When to use: Only change this if your API explicitly requires the Responses API format for tool calls. Most APIs use the default Chat Completions format.

Tool Calls

You can include tool/function calls in the Responses format:

{
  "messages": [
    {
      "type": "function_call",
      "name": "get_order_date",
      "arguments": "{\"shipment_id\": \"xx555\"}"
    },
    {
      "role": "assistant",
      "content": [
        {
          "type": "text_output",
          "text": "Your order should arrive next Tuesday"
        }
      ]
    }
  ]
}

Payload Wrapper

Wraps the entire payload in a specified field name. When to use: Your API requires all payloads nested under a specific key (e.g., data, request, body). Example: Without wrapper:

{
  "messages": [...],
  "customData": {...}
}

With wrapper set to "data":

{
  "data": {
    "messages": [...],
    "customData": {...}
  }
}

Common Values:

data
request
body
payload

Input Template

Completely customize the JSON payload sent to your chat endpoint on each conversation turn.

The input template is not used for the initialization endpoint — that call always uses the initialization payload. The simulation flow is:

Coval calls your initialization endpoint with the initialization payload
The init response is captured
For each chat turn, Coval uses the input template to build the request to your chat endpoint — and you can reference fields from the init response (e.g. {{init_response.conversation_id}})

When to use:

Your API expects a non-standard payload format
You need to include specific fields from initialization response
You want fine-grained control over the request structure

Format: JSON with template variable placeholders Available Template Variables:

Variable	Type	Description
`{{messages}}`	Array	Full conversation history
`{{latest_message}}`	String	Most recent user message content
`{{sessionId}}`	String	Session ID (from init response or simulation ID)
`{{simulation_output_id}}`	String	Unique simulation identifier
`{{custom_data}}`	Object	The custom data object
`{{custom_data.field}}`	Any	Specific field from custom data
`{{any.nested.path}}`	Any	Extract any field from init response using dot notation

Example Templates: Example 1: Simple Custom Format

{
  "user_input": "{{latest_message}}",
  "session_id": "{{sessionId}}",
  "context": {{custom_data}}
}

Example 2: Nested Init Response Fields

{
  "messages": {{messages}},
  "user_id": "{{init_response.user.id}}",
  "conversation_id": "{{init_response.conversation.id}}",
  "api_key": "{{init_response.auth.api_key}}"
}

Example 3: String Substitution

{
  "input": "User said: {{latest_message}}",
  "session": "{{sessionId}}",
  "metadata": {
    "source": "coval",
    "user": "{{custom_data.user_id}}"
  }
}

Note: When using input_template, the custom_data field is ignored. Reference custom data using {{custom_data}} or {{custom_data.field}} in your template instead.

Quoting rules for template variables:

Object/Array variables ({{messages}}, {{custom_data}}) substitute to valid JSON literals — do not wrap them in quotes.
String variables ({{sessionId}}, {{latest_message}}, {{init_response.*}}) substitute to plain text — you must wrap them in quotes.

For example, "conversation_id": {{init_response.conversation.id}} produces invalid JSON because the substituted value is not quoted. Use "conversation_id": "{{init_response.conversation.id}}" instead.

Response Message Path

Tells Coval where to find the assistant’s message in your API response using dot notation. When to use: Your API returns a non-standard response format. Default Behavior (when not set): Expects response in this format:

{
  "messages": [
    {
      "role": "assistant",
      "content": "The response text"
    }
  ]
}

Custom Path Examples: Example 1: Direct Field

Response Message Path: output_message

Extracts from:

{
  "output_message": "The assistant response",
  "metadata": {...}
}

Example 2: Nested Object

Response Message Path: data.response.text

Extracts from:

{
  "data": {
    "response": {
      "text": "The assistant response"
    }
  }
}

Example 3: Array Index

Response Message Path: choices.0.message.content

Extracts from:

{
  "choices": [
    {
      "message": {
        "content": "The assistant response"
      }
    }
  ]
}

Path Notation Rules:

Use . to navigate nested objects: data.response.text
Use numeric indices for arrays: choices.0.message
Combine for complex paths: data.results.0.output.text

Strip Message Timestamps

Removes timestamp fields from messages before sending to your API. When to use: Your API rejects requests containing timestamp fields. Default: Disabled (timestamps included) Example: With timestamps (default):

{
  "messages": [
    {
      "role": "user",
      "content": "Hello",
      "timestamp": "2025-01-15T10:30:00Z"
    }
  ]
}

With stripping enabled:

{
  "messages": [
    {
      "role": "user",
      "content": "Hello"
    }
  ]
}

Common Error Pattern:

{
  "message": ["messages.0.property timestamp should not exist"],
  "statusCode": 400
}

If you see this error, enable “Strip Message Timestamps”.

Ending the Chat

You can end the conversation by setting “status” to “ended” in your response:

{
  "status": "ended",
  "messages": [...]
}

Common Configuration Patterns

Pattern 1: OpenAI-Compatible API

Chat Endpoint: https://api.yourdomain.com/chat
Authorization: Bearer sk-your-key-here

(All other fields: leave empty/default)

Pattern 2: API with Session Initialization

Chat Endpoint: https://api.yourdomain.com/chat
Initialization Endpoint: https://api.yourdomain.com/init
Authorization: API-Key your-key-here
Initialization Payload:
{
  "user_id": "{{persona.user_id}}",
  "session_id": "{{simulation_output_id}}"
}

Pattern 3: Custom API Format with Template

Chat Endpoint: https://api.yourdomain.com/v1/message
Authorization: Bearer your-token
Input Template:
{
  "user_input": "{{latest_message}}",
  "session_id": "{{sessionId}}",
  "conversation_history": {{messages}}
}
Response Message Path: data.response.text

Pattern 4: API with Payload Wrapper

Chat Endpoint: https://api.yourdomain.com/chat
Authorization: Bearer your-token
Payload Wrapper: data

Pattern 5: Complex Custom Format

Chat Endpoint: https://api.yourdomain.com/chat
Initialization Endpoint: https://api.yourdomain.com/sessions/create
Authorization: Bearer static-token
Custom Headers:
{
  "X-Session-ID": "{{sessionId}}",
  "X-User-Context": "{{init_response.user.id}}"
}
Input Template:
{
  "messages": {{messages}},
  "user_id": "{{init_response.user.id}}",
  "api_version": "v2"
}
Response Message Path: response.text
Strip Message Timestamps: true

Troubleshooting

Error: “Failed to run simulation due to an unexpected error”

Problem: This generic error often indicates an issue with your agent configuration, most commonly an invalid input template. Common causes:

Unquoted string variables in your input template (e.g. {{init_response.id}} instead of "{{init_response.id}}")
Malformed JSON in your input template, initialization payload, or custom data
Invalid field references in template variables

Solution: Double-check your input template for valid JSON syntax. Make sure all string template variables are wrapped in quotes. See the quoting rules in the Input Template section.

Error: “Invalid JSON response from endpoint”

Problem: Your API returned non-JSON response Solution: Ensure your endpoint returns Content-Type: application/json

Error: “Could not extract message from path ‘X’ in response”

Problem: Response message path doesn’t match your API response structure Solution: Verify the path using dot notation matches your actual response structure

Error: “messages.0.property timestamp should not exist”

Problem: Your API rejects timestamp fields Solution: Enable “Strip Message Timestamps”

Tool calls not showing in transcript

Problem: Tool call extraction not configured Solution: Verify your API returns OpenAI-compatible format or contact support for custom tool call extraction configuration

Session ID not working in headers

Problem: Template variable not being substituted Solution: Verify initialization endpoint returns sessionId field, or check custom headers configuration

Best Practices

Start Simple: Begin with just Chat Endpoint and Authorization, add complexity as needed
Test Incrementally: Add one advanced feature at a time and test
Use Template Variables: Leverage {{sessionId}} and init response fields to maintain session state
Validate JSON: Always validate JSON fields before saving
Check API Logs: Use your API server logs to debug payload/response format issues
Document Custom Formats: Keep notes on your API’s expected format for future reference

Introduction

Configuration

Observability

Step-by-Step Guides

Use Cases

Connect & Collaborate

​Overview

​Quick Start

​Core Configuration

​Chat Endpoint (Required)

​Authorization Header (Required)

​Bearer Token

​API Key

​Custom Authorization

​Standard Protocol

​Optional Configuration

​Initialization Endpoint

​Initialization Payload

​Custom Data

​Custom Headers

​Chat Messages

​Advanced Configuration

​Response Format

​Chat Completions (Default)

​Responses API

​Tool Calls

​Payload Wrapper

​Input Template

​Response Message Path

​Strip Message Timestamps

​Ending the Chat

​Common Configuration Patterns

​Pattern 1: OpenAI-Compatible API

​Pattern 2: API with Session Initialization

​Pattern 3: Custom API Format with Template

​Pattern 4: API with Payload Wrapper

​Pattern 5: Complex Custom Format

​Troubleshooting

​Error: “Failed to run simulation due to an unexpected error”

​Error: “Invalid JSON response from endpoint”

​Error: “Could not extract message from path ‘X’ in response”

​Error: “messages.0.property timestamp should not exist”

​Tool calls not showing in transcript

​Session ID not working in headers

​Best Practices

Overview

Quick Start

Core Configuration

Chat Endpoint (Required)

Authorization Header (Required)

Bearer Token

API Key

Custom Authorization

Standard Protocol

Optional Configuration

Initialization Endpoint

Initialization Payload

Custom Data

Custom Headers

Chat Messages

Advanced Configuration

Response Format

Chat Completions (Default)

Responses API

Tool Calls

Payload Wrapper

Input Template

Response Message Path

Strip Message Timestamps

Ending the Chat

Common Configuration Patterns

Pattern 1: OpenAI-Compatible API

Pattern 2: API with Session Initialization

Pattern 3: Custom API Format with Template

Pattern 4: API with Payload Wrapper

Pattern 5: Complex Custom Format

Troubleshooting

Error: “Failed to run simulation due to an unexpected error”

Error: “Invalid JSON response from endpoint”

Error: “Could not extract message from path ‘X’ in response”

Error: “messages.0.property timestamp should not exist”

Tool calls not showing in transcript

Session ID not working in headers

Best Practices