What Is a Context Window?

A context window is the model's working memory—finite and token-measured. Here's what it is, why it constrains your app, and how RAG works around its limits.

What a context window is

A context window is the maximum text—measured in tokens—that a language model can consider at once. It's the model's working memory for a single request.

What counts against it

Content	Uses context
System instructions	Yes
Conversation history	Yes
Retrieved documents	Yes
User's message	Yes

Everything the model uses to respond must fit within the window.

What happens when you exceed it

Content that doesn't fit must be dropped, truncated, or summarized—so the model loses access to it. This causes it to "forget" earlier conversation or miss information, a common source of poor answers.

How RAG works around it

Retrieval-augmented generation fetches only the most relevant chunks and places those in the window—instead of stuffing entire documents. This keeps the useful information in view within the token limit, the core of how to build a RAG system.

Design implications

Bigger windows help but cost more tokens and can dilute focus. Good design keeps the most relevant content in view—see context engineering.

Why FISTA

FISTA Solutions designs LLM systems that use the context window well—retrieval and prompt design that keep the right information in view—through AI enablement, backed by 150+ projects across 12+ countries.

Building context-aware LLM apps? Talk to FISTA.