How does a context window work?
Models read text as tokens: words, parts of words or single characters. Google’s machine learning glossary defines the context window as the number of tokens a model can process in a given prompt, and notes that a larger window lets the model use more information to give coherent, consistent responses. Everything in a request shares the window: system instructions, earlier turns, tool results and the new question.
When a conversation outgrows the window, something has to go. Applications drop the oldest turns, summarise them or start a fresh session seeded with a summary. OpenAI’s documentation for its GPT-Live voice service, checked on 8 October 2026, gives a default context window of 128,000 tokens per session, including instructions, conversation text and audio tokens that don’t appear in the transcript, and says GPT-Live summarises older history in the background as a conversation grows.
A long product manual pasted into a prompt can crowd out the conversation or not fit at all. Retrieving the few passages that match the question uses a small part of the window and leaves the rest for the conversation.
Why does the context window matter for a voice agent?
In a speech model, audio counts towards the window as well as the words in the transcript. A voice agent also carries its instructions, the caller’s question and whatever it has looked up, all at once.
Focused instructions, short excerpts instead of whole documents and a cap on call length all leave room for the conversation itself. Facts that must survive a long call, such as what the caller has already tried, are safer kept by the application than left in the model’s context.
What fills the context window in Kawl Now?
Kawl Now doesn’t load your knowledge base into the model’s context. A workspace can hold up to 100 sources of up to 50,000 characters each, and each search returns at most four excerpts of about 4,500 characters from the approved sources for that call’s channel.
Every call does carry the agent’s instructions (up to 4,000 characters) and the fixed rules every agent gets, its greeting (up to 500), the topic the caller typed before starting (up to 1,000 characters, labelled as untrusted) and the conversation so far.
Calls stop at 10 minutes, which also limits how much conversation builds up. Long sources are a separate risk: each excerpt starts just before the first matching word, so an answer far down a long page can be cut off. Split long pages into focused sources.
Common questions
What happens when a conversation is longer than the context window?
The oldest material has to be dropped or summarised, so the model can lose details from early in the conversation. Applications usually keep important facts outside the model, for example in a database, and add them back when they’re needed.
Does a bigger context window remove the need for retrieval?
Not usually. A bigger window holds more documents, but a longer prompt is usually slower to process, and focused excerpts make it easier for the model to use the passage that matters. Retrieval also lets you change the knowledge without rewriting the prompt.