Track customer service KPIs you can measure for a voice agent.

Why do classic support KPIs need adjusting?

Most customer service KPIs assume a ticket, a known customer and a person answering. A website voice agent works differently: callers are anonymous unless they say who they are, and the agent can’t see their account. A call reaches your team only if you’ve turned on human handover, which is an early release. First contact resolution needs repeat contacts matched to one customer, and CSAT needs a survey, so neither can be read from the agent’s records.

What you can measure is what happened in each conversation: whether it completed, how long it lasted, what the caller asked, whether the agent answered from your knowledge and whether the caller pressed That was helpful.

Which KPIs should you track?

These definitions use the fields each conversation records. Leave out dashboard tests, which the export marks, so your own calls don’t skew the numbers.

1. Work out the counts from the export.

On the Conversations tab, Export downloads your latest 100 conversations as JSON. Each record has its channel, topic, status, start time, length in seconds, whether it was a dashboard test, the summary and any feedback, but not the transcript. A status of ended is what the dashboard calls Completed. Escalated is a completed call where the caller then chose Talk to a human and a handover to your team started, so the script counts it as completed too. Incomplete is Interrupted: the call didn’t close normally, which includes calls refused as they start, such as after the free conversations are used. Failed and pending calls never connected.

The script below reads one export and prints the counts for each agent, using only Python’s standard library. Run it weekly with the date of your last run, and keep each file, because an export only ever holds the latest 100 conversations.

"""Voice agent KPIs from a conversations export.

Usage: python3 kpis.py export.json [YYYY-MM-DD]
The optional date keeps conversations started on or after that day.
"""
import json
import statistics
import sys
from collections import Counter

RATING = "Customer rating: helpful"

with open(sys.argv[1], encoding="utf-8") as file:
    calls = json.load(file)
since = sys.argv[2] if len(sys.argv) > 2 else ""
calls = [c for c in calls if not c.get("is_test") and c["created_at"] >= since]

for channel in ("support", "sales"):
    rows = [c for c in calls if c["channel"] == channel]
    if not rows:
        continue
    statuses = Counter(c["status"] for c in rows)
    completed = [c for c in rows if c["status"] in ("ended", "escalated")]
    notes = [c.get("feedback") or "" for c in rows]
    helpful = sum(RATING in note for note in notes)
    to_read = sum(bool(note.replace(RATING, "").strip()) for note in notes)
    print(f"{channel} conversations: {len(rows)} {dict(statuses)}")
    print(f"  completion rate: {len(completed) / len(rows):.0%}")
    if completed:
        median = statistics.median(c["seconds"] for c in completed)
        print(f"  median completed length: {median / 60:.1f} minutes")
    print(f"  marked helpful: {helpful}, feedback notes to read: {to_read}")

2. Measure answer, gap and referral rates from transcripts.

These three come from reading, because Kawl Now doesn’t mark answers as right or wrong. Each week, open a sample of completed conversations, or all of them while volumes are small, and give each one a single label: answered, partly answered, didn’t know, referred, wrong or misheard.

Answer rate is answered divided by the number you labelled. Gap rate is didn’t know divided by the same number, and each one points to an article to write. Keep the labels and the sample size the same each week, so the numbers stay comparable. The conversation review guide has a log to copy and a script that flags likely gaps.

Which classic KPIs need another source?

Some familiar metrics can’t come from the agent’s records alone.

• CSAT and NPS need a survey. The widget’s only rating is a That was helpful button, with no negative option, so treat helpful presses as a floor and never report them as satisfaction.
• First contact resolution needs repeat contacts matched to one customer. Callers aren’t identified, so track repeated topics instead: the same question several times a week means the answer is missing or hard to find.
• Deflection needs your help desk data. Compare ticket volume on the topics the agent covers before and after launch, and treat the result as a rough signal, because releases, seasons and pricing changes also move ticket volume.
• Cost per conversation can’t be worked out yet, because paid calls aren’t available. Track how many of the workspace’s 50 free conversations you’ve used on the Usage & billing tab instead.

3. Set targets from your own baseline.

Use your first few weeks as the baseline, and set targets from it. Benchmarks from contact centres measure phone queues and human agents, so they don’t transfer to a voice agent on a website.

Watch the direction as much as the level. A falling gap rate after you add articles suggests they’re being found. A rising share of interrupted calls is worth checking in the browsers your customers use, once you’ve ruled out calls refused because the free conversations ran out.

Common questions