What is a large language model (LLM)?

How does a large language model work?

An LLM reads and writes text as tokens, which can be whole words, parts of words or single characters. In training it processes a very large collection of text and adjusts its parameters, the numbers that shape its predictions, until it is good at predicting which token comes next, and most models are then tuned further to follow instructions. To answer a prompt, it predicts a token, adds it to the text and predicts again, one token at a time, until the reply is complete.

Most current LLMs use the Transformer architecture, introduced in the 2017 paper “Attention Is All You Need”. Its self-attention mechanism lets the model weigh how each token relates to the others in its context, which is how it keeps track of what a long question is about. Google’s machine learning glossary defines an LLM, at a minimum, as a language model with a very high number of parameters, and more informally as any Transformer-based language model.

Ask an LLM “How do I reset my password?” and it produces a likely, helpful-sounding reply. If it has never seen your product’s reset steps, that reply may describe a menu your product doesn’t have, because the model predicts what such answers usually say rather than checking yours.

In EU law, the AI Act gives large generative AI models as a typical example of a general-purpose AI model: one that displays significant generality and can competently perform a wide range of distinct tasks.

What does an LLM do inside a voice agent?

The LLM decides what the agent says. Around it, speech has to become text on the way in and audio on the way out, either through separate speech recognition and text-to-speech systems or through a speech-to-speech model that works on audio directly.

Two traits of LLMs shape a voice agent’s design. The model knows nothing about your product beyond what it is given, so the agent needs a way to look up your docs during the call. And it writes confident sentences whether or not they are true, so the agent needs instructions to say when it doesn’t know, and someone needs to read what it said.

How does Kawl Now use an LLM?

Kawl Now runs on OpenAI models. OpenAI’s GPT-Live voice service handles the spoken conversation: audio streams between the visitor’s browser and OpenAI over WebRTC, and OpenAI’s documentation, checked on 8 October 2026, describes GPT-Live as able to listen while it speaks. After calls, an OpenAI model also produces automatic insights, grouping recurring questions, reported issues and themes from what callers said, with exact quotes from the transcripts. Kawl Now doesn’t train or fine-tune a model on your content.

Product facts come from knowledge your team approves. During a call, the agent is told to search that knowledge before it answers product questions, and to say it doesn’t know when the excerpts don’t answer. It greets callers in English and is told to say it’s an AI.

These are instructions to the model, so they shape its answers rather than guarantee them. Every AI conversation is saved as a text transcript for your team to check, and no call audio is recorded.

Common questions