Voice agent observability: logs, transcripts, and traces that explain a bad call

A voice agent handled a call badly. The customer was annoyed, the booking did not happen, and now someone forwards you the complaint with the world's least debuggable bug report: "the agent sounded confused."

Sounded confused where? Did it mishear the caller? Did the speech recognizer mangle a word? Did a tool call time out and leave the agent improvising? Did the model just reason poorly? "Sounded confused" cannot distinguish any of these — and each one has a completely different fix.

Observability is what turns "sounded confused" into "the availability webhook took 4 seconds, the agent filled the silence, and the caller talked over the delayed response." That is a bug you can fix. The vibe is not.

Why the recording is not enough

The recording is the obvious artifact, and teams reach for it first. It has a hard limit: it captures the symptom, never the cause.

A recording can tell you the agent paused awkwardly for four seconds. It cannot tell you that the pause was a getAvailability tool call waiting on a slow calendar API. It can tell you the agent gave a wrong answer. It cannot tell you whether the speech recognizer fed it the wrong words, or it got the right words and reasoned badly.

The recording is one layer. To debug, you need the layers underneath it.

The four layers to capture

For every call, capture enough to reconstruct it completely:

• Transcript. Both sides, turn by turn, with timestamps. This is what the agent heard and what it said — searchable and diffable in a way audio is not.
• Turn timings. For each turn, how long each stage took: speech recognition, model response, text-to-speech, plus any tool call fired mid-turn. This is where latency problems become visible.
• Tool-call trace. Every tool or webhook the agent invoked, with the exact inputs it passed, the outputs it got back, and the duration. This is where "the agent did something weird" becomes "the agent called the tool with a malformed date."
• Recording. The audio, as the final reference layer — for tone, for interruptions, for anything the transcript flattens.

Together, these four answer almost any "why did this call go wrong" question.

Per-turn timing is the highest-value layer

If you capture only one thing beyond the recording, capture timing.

A voice call is a latency game. The caller expects a response within a beat; exceed it and the conversation breaks — the caller talks over the agent, repeats themselves, or hangs up. "The agent felt slow" is one of the most common complaints, and on its own it is unactionable.

Per-turn timing makes it actionable. Break each turn into its stages and you can see exactly where the time went:

• Speech recognition slow? The audio pipeline or the recognizer.
• Model response slow? The LLM, or a prompt that is too long.
• Text-to-speech slow? The voice synthesis stage.
• A gap with none of the above? A tool call you fired mid-turn and waited on.

Each of those points to a different fix. Without the breakdown, you are changing things at random and hoping the vibe improves.

One call, one trace

The organizing principle: a single call is a single trace. Each turn is a span within it. Each tool call is a child span with its inputs, outputs, and duration.

You do not necessarily need a heavyweight distributed-tracing platform — but you do need this structure. Whether it lives in a tracing tool or in structured logs all keyed by a call ID, the requirement is the same: given any call ID, you can pull up the entire call and walk it end to end. Who said what, when, how long each stage took, what every tool returned.

If reconstructing a call means stitching together a recording, some scattered log lines, and guesswork — you do not have observability, you have archaeology.

From observability to fixes

Capturing the data is half the job. The point is what you do with it:

• Prompt fixes. Transcripts show recurring failure phrasings — the agent consistently misreads a question type, or gives an answer that confuses callers. That is a prompt change, validated against real transcripts.
• Tool fixes. The tool trace shows malformed inputs or slow responses. Maybe the agent passes dates in a format the tool mishandles, or a webhook is slow enough to break the conversation rhythm.
• Latency fixes. Turn timings show which stage to optimize — and just as importantly, which stages are already fine and not worth touching.

Observability turns debugging from opinion into evidence. "I think the agent is slow" becomes "the model stage is 800ms over budget on long prompts" — and now you know exactly what to change.

Getting started

Call2Me emits structured events for every stage of a call and exposes transcripts, recordings, and analytics per conversation. The events documentation covers the event types and payloads. Pipe them into whatever you already use — a tracing tool, a log store, a dashboard — keyed by call ID, and the next "the agent sounded confused" report becomes a five-minute investigation instead of a shrug.