Long context
Also written: Large context window, Long-context tasks
Models can process and understand large amounts of input data, like long documents or code, in a single pass.
Long context refers to a large language model's ability to take in and understand a lot of information at once. Think of it like a person reading a whole book instead of just a single page.
This allows the model to answer questions or perform tasks based on extensive material. For example, you could give it an entire research paper and ask it to summarize the key findings or explain a complex section.
Long context refers to the maximum amount of input data, measured in tokens, that a large language model can process in a single forward pass. This enables models to maintain coherence and recall information across extended documents, codebases, or conversations.
Tradeoffs include increased computational cost and memory requirements during inference, though advancements in architectures like attention mechanisms and retrieval-augmented generation (RAG) are mitigating these challenges.
Google Gemini's recent announcements highlight advancements in processing large amounts of data and complex code. This focus on long context is driven by the need for AI agents to handle more intricate tasks and analyze extensive datasets efficiently. The push for longer context windows allows models to maintain coherence and recall information over extended interactions, enabling more sophisticated applications.
- spark Gemini 3.5 FlashGoogle · Gemini Models ·
- Gemini 2.5 ProGoogle · Gemini Models ·
- Gemini 2.5 ProGoogle · Gemini Models ·
- Gemini 2.5 ProGoogle · Gemini Models ·
- DeepMind Gemini 3 Lead: What Comes After "Infinite Data"Google · The MAD Podcast ·
- Building Gemini's Coding CapabilitiesGoogle · Google AI: Release Notes ·
- Developing Google DeepMind's Thinking ModelsGoogle · Google AI: Release Notes ·
- Google Gemini
- Google Gemini models like Gemini 2.5 Pro and Gemini 3 Flash are being developed with significantly larger context windows to analyze massive datasets and complex codebases, enabling faster and more efficient processing of long tasks.
- Context window
- The context window is the specific size limit of the long contex
The concept of long context has evolved with the development of transformer architectures and their ability to process sequential data. Specific advancements are often announced by leading AI labs.
Copy and paste a lengthy article or a chapter from a book into the chat interface of a capable AI model. Then, ask specific questions about details mentioned early in the text to see if the model can recall and use that information.
The Playbook: reusable how-tos, prompts, configs and tips pulled out of the news.
- PromptRole-play a student struggling with math for AI tutor simulationSimulate a student's math struggles and uncertainty for AI tutor evaluation. · 5 min
- TemplateReusable workflow templates for healthcare teamsTeams can create and share templates for common workflows like documentation or prior authorizations. · 15 min
- TemplateTemplate for building an AI tool tailored to your workA structured approach to defining and building a custom AI tool for specific business needs. · 15 min
- TemplateEvals CSV prompt list for agent skillsA CSV file template for listing prompts used in evaluating agent skills. · 20 min
- HeuristicDouble coding agent task completion with docs linksProviding coding agents with a documentation link significantly increases their task completion rate. · 2 min
Questions people ask
- What is a context window?
- A context window is the maximum number of tokens a language model can consider at one time when processing input and generating output.
- How does long context help?
- Long context allows models to understand and reason over much larger amounts of information, leading to more comprehensive and accurate responses for complex tasks.
First seen on Subvolts Sep 8, 2026. Updated Oct 15, 2026. Written by an AI model from the items above and reviewed against them weekly; report an error.