How does RAG work?
A RAG system has a retriever and a generator. When a question arrives, the retriever searches a collection of documents for passages that match it. The system adds the best passages to the prompt, usually with an instruction to answer from them, and the language model writes the reply.
The name comes from a 2020 paper by Patrick Lewis and colleagues, which paired a pre-trained generator with a dense vector index of Wikipedia. The term has since broadened. NIST’s definition covers any generative AI system paired with a separate information retrieval system or knowledge base, and notes that RAG lets a model’s knowledge change without retraining. The retriever can use vector search, keyword search or both.
A customer asks “Does the Team plan include single sign-on?” The retriever finds the plans page and the SSO article, and the model answers from those passages instead of guessing.
Why does RAG matter for a voice agent on a website?
A voice agent for your product needs facts that change: features, plans, setup steps. RAG lets it use your current docs during the call, and when the docs change, the next answer can change with them.
Voice puts pressure on retrieval. Callers won’t wait long, so the search has to be quick, and the agent should read a few focused passages rather than whole manuals. If retrieval misses the right passage, even a strong model answers without the facts it needs, which is why the way sources are written matters as much as the model.
How does Kawl Now use RAG?
Kawl Now uses a simple form of RAG with keyword search rather than embeddings or a vector database. Approved sources are stored as text. When the agent needs product facts, it calls a search function that matches the words of its query against the approved sources for that call’s channel and returns up to four excerpts of about 4,500 characters each.
Retrieval happens when the agent decides to search, not before every reply. It is told to search approved knowledge for product facts and, if the excerpts don’t answer the question, to say it doesn’t know. That makes searching an instruction the model follows rather than a step it can’t skip.
Keyword matching works best with short sources written in your customers’ words. Keep one topic in each source, include the terms people actually say and split long pages, because each excerpt starts just before the first matching word.
Common questions
Does RAG need a vector database?
No. Many RAG systems use embeddings and a vector database to find passages by meaning, but the retriever can also be keyword search or a mix of both. Kawl Now uses keyword search over a small set of approved sources.
Is RAG better than fine-tuning?
They do different jobs. Fine-tuning changes how a model behaves by training it further, while RAG gives it facts at the moment it answers. For product knowledge that changes often, RAG is easier to keep current, because you edit a document instead of retraining a model.