I gave it more than enough room. It forgot the middle.

A huge context window opens up possibilities. The promise remains incomplete when input size is treated as proof that every detail will be used correctly.

Hypothetical scene.

The researcher has low vision. She has been writing her thesis by dictation for months. She speaks, reviews it with a screen reader, and dictates again. On a good afternoon, she decides to use an AI advertised as able to “read an entire book at once.”

She provides all five chapters and her supervisor’s notes. She asks for a specific check: is the definition of “participation” in chapter 3 consistent with the analysis in chapter 5?

The answer arrives quickly, politely and confidently. It comments on the introduction and praises the conclusion. Then it suggests a definition that contradicts the methodological decision recorded in chapter 3, developed through weeks of discussion.

No warning appeared. No bar turned red.

The thesis made it in. The important decision never reached the answer.

Fitting is only the beginning

Tokens are units representing the input: they may correspond to words, parts of words, punctuation and other elements. There is no fixed conversion from tokens to pages, especially across languages, code and multimodal material.

The context window specifies a sequence limit. The context of a particular run is the content actually made available for that answer. Instructions, messages, documents and tool results may occupy this space; how it relates to the output depends on the system’s limits.

There is another distinction the interface can hide. A document or conversation remaining visible in the app does not prove that it was sent to the model again in full. The product may select, summarize, retrieve or omit parts of the history.

Before concluding that the model “forgot,” we need to distinguish two situations: the information was absent from the call, or it was included but used incorrectly. Both can harm the person; the technical fix is different.

In the thesis scene, we assume that every chapter and the notes were sent in the call to the model. Even so, accepting the input does not guarantee applying the right definition.

What current windows allow

In the documentation consulted on October 24, 2026, the limits are already these:

Model and documented interface: GPT-6.1 Sol, API; Context or input limit: Context window of 1,050,000 tokens; Output limit: 128,000 tokens

Model and documented interface: Gemini 4 Argon, API; Context or input limit: Unified window of 1,048,576 tokens, shared between input and output; Output limit: 65,536 in the default synchronous mode; extension through long decode, within the shared window

Model and documented interface: Gemini 3.8 Flash, API; Context or input limit: Maximum input of 1,048,576 tokens; Output limit: 65,536 tokens

Sources: official documentation for GPT-6.1 Sol [1], Gemini 4 Argon [2] and Gemini 3.8 Flash [3].

The numbers need to identify the kind of limit. An input ceiling is not the same as a window shared with the answer. Neither tells us how much history an app delivers in each call.

These capabilities have value: they make it possible to work with documents, code and relationships that would not fit into a short input. The next question is how the model uses the material it receives.

The test needs to reach the task

Finding a hidden sentence is different from relating chapters, counting interpreted occurrences or incorporating a corrected decision.

RULER, published in 2024, was already expanding the “needle in a haystack” test with retrieval, relationship tracing and aggregation. Its effective length is the greatest tested length at which performance still meets the threshold adopted for the task set. It therefore depends on the tasks and the threshold chosen by the authors. RULER [4].

NoLiMa, from 2025, reduces literal overlap between the question and the information being sought: an association must be retrieved, rather than simply matching words. Its historical results help explain the problem; they do not measure GPT-6.1 or Gemini 4. NoLiMa, version 3 [5].

Oolong, whose version 2 was revised in October 2026, adds another requirement: analyzing multiple passages and aggregating those analyses. It includes classification, counting and temporal relationships in synthetic and conversational data. Here, much of the context can contribute to the answer instead of merely serving as a haystack. Oolong, version 2 [6].

None of these tests alone proves that an assistant will check the researcher’s thesis correctly. They show why evaluation needs to match the operation we expect it to perform.

What current data shows — and what is missing

There is current evidence of strong long-context performance. In the Gemini 4 Argon model card, Google reports these GraphWalks results, a task involving traversing graph relationships through breadth-first search, or BFS:

GraphWalks subset: Contexts up to 128K tokens; Number of problems: 650; Reported F1 for Argon: 99.7%

GraphWalks subset: Contexts between 256K and 1M tokens; Number of problems: 200; Reported F1 for Argon: 84.2%

F1 combines precision (the share of retrieved items that is correct) and recall (the share of expected items that was retrieved); an F1 of 84.2% does not mean that 84.2% of conversations are entirely correct. Google calculated these results. The subsets contain different problems: the difference between the rows does not isolate the effect of increasing length alone. Results [7] and methodology [8].

This result belongs in the discussion: long context can support complex work. It also leaves room to investigate errors, different tasks and variation within the window.

For GPT-6.1 Sol, I found the official limits but no equivalent evaluation of retrieval at different lengths in the OpenAI sources consulted. That gap does not demonstrate poor performance. Scores from earlier generations do not fill that gap, and window size alone does not establish reliability.

New models need new measurements. Published performance on one task helps us decide what to test next; it does not conclude the evaluation.

The middle can be a difficult region

The Lost in the Middle study, published in TACL in 2024, observed that changing the position of relevant information affected performance. In the tasks and models evaluated, information was often used better at the edges than in the middle. Paper [9].

The study is a historical reference about position, not an automatic diagnosis of 2026 models. It helps formulate a hypothesis for the thesis scene, but does not establish that every piece of information in the middle will be ignored. To find out whether the effect persists in a current model, we need to test its version, the task and the position of the evidence.

For the researcher, however, the consequence is concrete: she needs to discover that the analysis disregarded a decision already present in the material. The cost of checking returns to the person who asked for help.

Attention is not a single mental budget

In Transformers, attention mechanisms relate positions in a sequence. In dense causal attention, a position can attend to itself and to earlier positions allowed by the mask. The network also has other transformations, multiple layers and multiple attention heads. Original Transformer architecture [10].

Softmax turns scores into weights that sum to 1 within each attention distribution. That sum does not represent a single reserve of concentration divided across the entire model.

Ken M. Nakanishi’s Scalable-Softmax investigates how distributions flatten as length increases and proposes a change to normalization. The mathematical condition needs to accompany the metaphor: if score differences remain bounded while the number of positions grows, maintaining a dominant weight becomes harder. Study [11].

A simple example makes this visible. If one position has score Δ and all the others have score zero, its weight is:

p = \frac{e^{\Delta}}{e^{\Delta} + (n-1)}.

Keeping Δ fixed while increasing the number of positions n makes this weight decrease. In this example, keeping the same weight requires Δ to grow with the logarithm of n−1. If scores change with length, the conclusion requires examining that change. The sum of weights alone does not prove that a commercial model will lose a particular piece of information.

There are also attention sinks: initial positions may receive high weights even without carrying semantically important content. StreamingLLM studies this phenomenon and uses it to improve processing with a limited window. This does not mean retaining and retrieving the entire history of an indefinitely long sequence. StreamingLLM [12].

These are research clues about different mechanisms. Simply adding them together does not yield a complete causal explanation for every long-context error.

Conversation occupies machine memory too

In models using a KV cache, the system stores key and value states to reuse during generation. This cache is part of the inference state; it is not the product’s persistent personal memory.

An estimate for full-cache attention layers with equal dimensions is:

\text{bytes de KV} = B \times 2 \times L \times n \times H_{KV} \times d_h \times s.

Here, B is the number of sequences, 2 represents keys and values, L is the number of layers, n is the retained length, H_KV is the number of key/value heads, d_h is each head’s dimension and s is the number of bytes per element.

To see the order of magnitude without attributing the calculation to a proprietary model, let us use two illustrative attention configurations with a full cache: one sequence, eight KV heads per layer, a head dimension of 128 and two bytes per element, as in BF16.

Layers with a full cache: 32; KV per token: 128 KiB; KV for 128,000 tokens: 16.78 GB; KV for 1,000,000 tokens: 131.07 GB

Layers with a full cache: 80; KV per token: 320 KiB; KV for 128,000 tokens: 41.94 GB; KV for 1,000,000 tokens: 327.68 GB

The table applies arithmetic to the formula; it is not a hardware measurement or proof that any particular model supports those lengths. It uses decimal GB; GiB, used in many system tools, gives different numbers. It also excludes weights, activations, buffers and server costs.

GQA shares key/value heads across queries and reduces the cache compared with a configuration with more KV heads. Quantization can reduce it further: KIVI and KVQuant, both from 2024, are research references on low-bit representations. Actual savings include metadata and quality effects; moving from 16 to 2 bits does not divide the entire application by eight. KIVI [13], KVQuant [14].

This formula alone does not describe every current architecture. The Qwen3.8-Next report, from August 2026, describes a combination of Gated DeltaNet and attention, with one global attention layer every four layers; at a later stage, these layers use Qwen Sparse Attention. Recurrent components and sparse attention require their own calculations. Technical report [15].

The vLLM cache manager also distinguishes full-attention layers, which retain all positions, from sliding-window layers, which retain only a recent span. Memory depends on the layers and retention policy. You cannot apply the full-cache calculation to the entire model just because it accepts a million tokens. Hybrid cache documentation [16].

Prefill and generation have different costs

During prefill, the system processes the input and prepares states for continuation; this work can be split into chunks. In dense attention, the number of relationships between positions grows quadratically with length. All else equal, doubling the input brings that component close to four times the work.

Using explicit lengths, moving from 128,000 to 1,000,000 tokens multiplies the quadratic term by approximately 61. This describes that idealized component, not a guaranteed 61-fold increase in total response time.

In decoding with a KV cache, each new token can consult the previous state without recomputing the entire input. For dense attention and fixed dimensions, this component per token grows approximately linearly with the length already retained. The network’s other computations still exist.

FlashAttention improves use of the memory hierarchy and avoids materializing the entire attention matrix in the GPU’s main memory. Its dense variant preserves exact attention and does not eliminate the quadratic order of its operations. Paper [17].

Qwen3.8-Next, mentioned above, is a current example of why these approximations cannot be extended to every architecture. Observed cost also depends on kernels, hardware, work partitioning, request batching and cache policy. Length alone does not determine latency.

There is also prefix caching, which reuses states from an already processed prefix and can save prefill work. It does not increase the window by itself or demonstrate that the answer applied a decision correctly. The vLLM documentation distinguishes this benefit from the work of generating new tokens. Prefix caching [18].

How I would test the chapter 3 decision

A test close to the scene needs to request an answer that can be checked: which definition applies, where it was established and how it changes the analysis in chapter 5.

I would compare a short input containing the necessary evidence, the complete document and a version using passage retrieval. I would use the same model and answer budget, recording instructions, configuration and the content actually delivered in each call.

Then I would vary material length and evidence position without changing its meaning or the order of decisions. I would include questions phrased differently from the document and cases in which a later correction supersedes an old definition.

Checking would track decision accuracy, reference fidelity and introduced contradictions. Multiple cases and repetitions would allow variation and regressions to be recorded. This is a proposed evaluation design; I am not presenting results from an experiment already conducted.

The human cost does not appear in window size

Here is an accessibility hypothesis I want to investigate, without presenting it as a measured statistic.

A person working by dictation may develop an idea aloud, return to a topic and explain a decision again. For someone reviewing with a screen reader, finding a contradiction scattered across a long conversation may require another pass through substantial material.

Some neurodivergent people may also prefer to return to threads and reorganize their thinking during conversation. These experiences vary. There is no single way of using voice, having low vision or thinking with a tool.

The design problem arises when the system requires someone to repeatedly reconstruct a decision they have already communicated. More text can increase the cost of retrieving the right information; checking the failure may take even more work.

An accessibility evaluation needs to observe that cycle: what had to be repeated, how much work checking required and whether the tool managed to incorporate the correction.

Memory that survives the conversation

In the area-based memory proposal I have been designing, decisions, sources and relationships are organized outside the current conversation. For each request, the system retrieves what may help at that moment.

The aim is to preserve continuity without requiring someone to paste the entire history again. This needs to be demonstrated in tests; storing information does not guarantee retrieving it well, and retrieving a passage does not guarantee applying it correctly.

External memory and long context can work together. A summary may lose an exception; a search may return an old version; a graph may retain an incorrect connection. The architecture needs to preserve provenance, corrections and the possibility of review, as well as choose what enters the call.

For people using AI today, it helps to keep a short record of decisions and their complete sources, ask for references to the passages used and check answers that depend on important changes. Analyzing material in parts can be useful, provided the division preserves the relationships the task requires.

Placing decisive instructions where they are easy to retrieve is a strategy to test. It should not become an obligation to reorganize an entire book to compensate for each tool.

The researcher needed to check a decision from chapter 3. The system should help her find it, relate it to chapter 5 and show where its answer came from.

Window size matters. Continuity of work needs to show up in the result.

Talk to us

Have you ever had to repeat a decision already present in a document or conversation? Tell us privately what information was lost and what it took to resume. Avoid names, private documents and sensitive data. Publishing an account requires separate permission.

Questions, corrections and suggestions: [email protected].

Further reading

• RULER — tasks and effective length [4].

• NoLiMa — published study [5] and additional results from the authors [19].

• Lost in the Middle — position effects in the evaluations [9].

• Scalable-Softmax — normalization and length generalization [11].

• StreamingLLM — attention sinks [12].

• FlashAttention — exact attention and memory use [17].

• KIVI [13] and KVQuant [14] — KV cache compression.

• GPT-6.1 Sol — documented limits [1].

• Gemini 4 Argon — results [7] and GraphWalks methodology [8].

• Oolong v2 — aggregation and long-context reasoning [6].

• Qwen3.8-Next — hybrid architecture [15].

• vLLM — hybrid cache [16] and prefix reuse [18].