Skip to main content

​
Overview

OpenAI Realtime agents are voice-to-voice agents where Coval connects directly to the OpenAI Realtime API on your behalf. You provide an API key, an agent prompt, a voice, and a model — Coval handles everything else. No webhook, no SIP trunk, and no self-hosted server required. This makes OpenAI Realtime the fastest way to evaluate an OpenAI-based agent configuration: tune the prompt, change the voice, bump the temperature, and re-run the same test set to see how the change affects behavior.
Use this connection when you want to evaluate an agent built directly on the OpenAI Realtime API. If your production agent runs on Pipecat, LiveKit, or a custom stack, connect it through the matching integration instead.

​
Configuration Requirements

The only required field for a MODEL_TYPE_OPENAI_REALTIME agent is metadata.openai_realtime_api_key. Every other field below is optional — if omitted, Coval applies the default shown.

​
OpenAI API Key

  • Field: openai_realtime_api_key
  • Type: String (required)
  • Purpose: Authenticates Coval’s connection to the OpenAI Realtime API
  • Format: A valid OpenAI API key (typically starts with sk-) with Realtime API access enabled
  • Security: Stored encrypted and handled securely

​
Agent System Instructions

  • Field: openai_realtime_instructions
  • Type: String (optional)
  • Default: "" (model default instructions)
  • Purpose: System prompt sent as session instructions to the Realtime model
  • Use Cases: Role definition, behavior guidelines, response formatting
  • Example: "You are a helpful customer support agent. Always be polite and provide accurate information."

​
Model

  • Field: openai_realtime_model
  • Type: String (optional)
  • Default: gpt-4o-realtime-preview (used when the field is omitted)
  • Purpose: Selects which OpenAI Realtime model powers the agent

​
Agent Voice

  • Field: openai_realtime_voice
  • Type: String (optional)
  • Default: alloy
  • Purpose: Prebuilt voice used by the agent
Available Voices: alloy, ash, ballad, coral, echo, sage, shimmer, verse

​
Temperature

  • Field: openai_realtime_temperature
  • Type: Number (optional)
  • Default: 0.8
  • Range: 0.0 (deterministic) to 2.0 (very random)
  • Purpose: Controls randomness in the agent’s responses

​
Simulation Timeout

  • Field: simulation_timeout_seconds
  • Type: Integer (optional)
  • Default: 900 (15 minutes)
  • Range: 1 to 1800 (30 minutes)
  • Purpose: Maximum duration for a single simulated conversation

​
Setup Instructions

1

Get an OpenAI API key

Create a key in your OpenAI dashboard and confirm it has Realtime API access.
2

Create the agent in Coval

Navigate to app.coval.dev/agents/create and select OpenAI Realtime under Voice-to-Voice.
3

Configure the agent

Paste your OpenAI API key, write your system instructions, pick a model and voice, and adjust temperature if needed.
4

Run a simulation

Create a test set, launch a simulation, and review the transcript and metric outputs.

​
Creating via the API

OpenAI Realtime agents can also be created with the v1 Agents API: 
curl -X POST https://api.coval.dev/v1/agents \
  -H "x-api-key: YOUR_COVAL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "OpenAI Realtime Voice Agent",
    "model_type": "MODEL_TYPE_OPENAI_REALTIME",
    "metadata": {
      "openai_realtime_api_key": "sk-...",
      "openai_realtime_instructions": "You are a helpful customer support agent...",
      "openai_realtime_model": "gpt-4o-realtime-preview",
      "openai_realtime_voice": "alloy",
      "openai_realtime_temperature": 0.8
    }
  }'

​
How Simulations Work

When you launch a simulation against an OpenAI Realtime agent, Coval:
  1. Opens a connection to the OpenAI Realtime API using your API key, model, voice, and system instructions
  2. Plays the simulated user’s turns into the live session
  3. Captures the agent’s audio responses and transcribes them
  4. Records the full conversation transcript
  5. Runs your configured metrics against the transcript

​
Troubleshooting

Invalid or missing API key
  • Confirm the key is valid in your OpenAI dashboard
  • Verify your OpenAI account has Realtime API access
Voice rejected on save
  • Voice must match one of the allow-listed values above. Other strings are rejected at save time.
Temperature validation error
  • Temperature must be between 0.0 and 2.0. Values outside that range are rejected at save time.
Agent is unresponsive or cuts off early
  • Increase Simulation Timeout if conversations are being truncated
  • Check that your system instructions don’t tell the agent to end calls immediately