Skip to main content
The Coval Wizard is in beta and under active development — more features coming soon! It uses an LLM to analyze and modify your code, so results may vary. Always review the proposed diff carefully before applying changes.
The Coval Wizard (@coval/wizard) reads your Python agent code, figures out exactly where to inject tracing, and writes the changes for you — including a diff preview, file backup, and connectivity validation. It works for Pipecat, LiveKit Agents, Vapi, and generic Python agents.

​
Quick Start

Run this from your agent’s project directory: 
npx @coval/wizard
The wizard will prompt you for your Coval API key if COVAL_API_KEY is not already set in your environment. 
# With API key pre-set
COVAL_API_KEY=your-key npx @coval/wizard

​
What It Does

1

Detects your project

Scans the directory for a Python project manifest (pyproject.toml, requirements.txt, Pipfile, or setup.py) and identifies your framework (Pipecat, LiveKit, Vapi, or generic Python).
2

Analyzes your code

Sends your entry point file to an LLM along with framework-specific injection rules to determine the minimal changes needed.
3

Shows a diff

Displays a colored diff of the proposed changes to your entry point and asks for confirmation before writing anything.
4

Writes the files

Creates coval_tracing.py — a self-contained OpenTelemetry module — and modifies your entry point. Your original entry point is backed up to <filename>.bak.
5

Validates connectivity

Sends a test span to api.coval.dev to confirm your API key is working and spans can reach Coval. If you don’t have an API key yet, go to Settings in the Coval platform and click Create Key.

​
Supported Frameworks

FrameworkDetectionWhat gets injected
Pipecatpipecat-ai in dependenciessetup_coval_tracing() before pipeline construction; simulation ID from args.body SIP headers; enable_metrics=True, enable_tracing=True on PipelineTask
LiveKit Agentslivekit-agents in dependenciessetup_coval_tracing() before AgentSession; simulation ID from SIP participant attributes; instrument_session(session) after await session.start()
VapiVapi webhook patterns in .py filessetup_coval_tracing() at module level; simulation ID extracted from assistantOverrides.variableValues["coval-simulation-id"] in webhook handler
Generic PythonAny Python projectsetup_coval_tracing() at module level; # TODO comment marking where to call set_simulation_id()

​
What It Sets Up

The wizard creates coval_tracing.py — a self-contained module you import in your agent. It provides three public functions: 
from coval_tracing import setup_coval_tracing, set_simulation_id, instrument_session

# Call once at startup (or at the start of each call session)
setup_coval_tracing(service_name="my-agent")

# Call when the Coval simulation ID arrives (e.g. from a SIP header)
set_simulation_id(simulation_id)

# LiveKit only — hooks session events to emit STT, LLM, and tool call spans
instrument_session(session)
Spans emitted before set_simulation_id() is called are buffered and flushed automatically once the ID arrives — no spans are lost even if tracing initializes before the call connects. The module also includes convenience helpers (create_llm_span, create_stt_span, create_tts_span, create_tool_call_span) for manually wrapping operations in spans if needed.

​
What It Doesn’t Set Up

The wizard installs the tracing infrastructure but does not add advanced span attributes automatically — yet. For richer observability, you will need to add these manually after running the wizard:
  • stt.confidence — ASR confidence score per utterance (0.0–1.0). Requires hooking into your STT provider’s result to extract the confidence value.
  • llm.finish_reason — Why the LLM stopped generating (stop, tool_calls, length). Requires observing the LLM response before emitting the span.
  • gen_ai.usage.input_tokens / gen_ai.usage.output_tokens — LLM token counts. Requires extracting token usage from your LLM response.
  • stt.provider.<name> sub-spans — Per-provider attempt spans for fallback chains. Requires wrapping each provider call individually.
See the voice agent examples for complete reference implementations that include all of these attributes.

​
Environment Variables

VariableRequiredDescription
COVAL_API_KEYYesYour Coval API key. Prompted interactively if not set.
WIZARD_LLM_KEYNoLLM API key for direct use.
WIZARD_LLM_PROVIDERNoanthropic, openai, or gemini
WIZARD_LLM_MODELNoOverride the default model (e.g. gpt-4o, gemini-2.5-flash)

​
Limitations

  • Python only — The wizard requires a Python project manifest (pyproject.toml, requirements.txt, Pipfile, or setup.py) in the target directory.
  • Single entry point — Only one file is analyzed and modified. The entry point must be named agent.py, main.py, bot.py, app.py, or server.py, or be the sole .py file in the project root. Entry points larger than 50 KB are not supported.
  • LLM-generated code — The wizard uses a language model to determine what to inject. Results are generally accurate for standard Pipecat, LiveKit, and Vapi patterns, but unusual project structures may require manual corrections. Always review the diff before confirming.
  • Generic Python wizard support is minimal — For projects that aren’t one of the three supported frameworks, the wizard adds setup_coval_tracing() and a # TODO comment. You will need to call set_simulation_id() manually and instrument spans yourself.
  • Validation is connectivity-only — The validation step confirms that your API key can reach api.coval.dev. It does not verify that spans are correctly wired to your agent’s call lifecycle.
  • No multi-file analysis — The wizard reads your entry point and dependency manifest only. It does not analyze helper modules, shared utilities, or subpackages.

​
Next Steps

After running the wizard:
  1. Deploy your updated agent and run a simulation in Coval.
  2. Open the result and look for the OTel Traces card — traces appear within a few seconds of the simulation completing.
  3. To add richer span attributes (stt.confidence, llm.finish_reason, provider sub-spans), see the OpenTelemetry guide and the voice agent examples.