Why do AI models hallucinate?
A language model generates likely next words given its training and its prompt. For well-documented facts that usually produces correct statements, but nothing in the process checks the output against reality. NIST’s Generative AI Profile describes confabulation as confidently stated but erroneous or false content, including output that contradicts the model’s own input or its earlier statements.
A 2025 research paper, “Why Language Models Hallucinate”, argues that training and evaluation reward guessing over admitting uncertainty. Like a student facing a hard exam question, a model that guesses scores better on most benchmarks than one that says “I don’t know”.
Ask a model with no access to your docs “Can I export reports to Excel?” and it may say yes, describe a menu that doesn’t exist and sound sure. The answer is plausible because many products work that way.
Why do hallucinations matter for a voice agent?
A spoken answer leaves nothing on screen to reread, and a calm voice is easy to believe. A support agent that invents a setting sends the customer hunting for it, and a sales agent that invents a discount creates a promise your team has to deal with.
Voice adds a second source of error: mishearing. If the agent hears the wrong question, it may answer that question correctly. Reading the transcript, with the caller’s words as they were transcribed, tells you which happened.
The usual defences are grounding the agent in checked sources, telling it to say when it doesn’t know, testing it with questions your sources don’t answer and reading transcripts.
How does Kawl Now reduce hallucinations?
Kawl Now answers product questions from knowledge your team approves. The agent is told to search that knowledge for product facts and use only supported facts. When the excerpts don’t answer the question, it is told to say it doesn’t know rather than guess, and to point the caller to other help, such as your website’s usual contact options.
Default instructions tell the support agent never to invent capabilities or claim it has changed an account, and the sales agent never to invent pricing, discounts, commitments or capabilities.
None of this guarantees a correct answer. Approval controls what the agent can use, not what it says, and the dashboard doesn’t show which source an answer came from. Every conversation is saved as a text transcript, so test with questions inside and outside your sources and read the replies.
Common questions
Can AI hallucinations be eliminated?
Not with current models. Grounding in checked sources, instructions to admit uncertainty and testing all reduce them, but a model can still state something false, so answers that matter need checking.
Is a wrong answer from an out-of-date source a hallucination?
Not strictly. An agent that repeats an approved but outdated source is grounded but wrong, while a hallucination is content the source doesn’t support. Both show up in transcripts, and the fixes differ: update the source, or tighten the instructions and test again.