Compaction
Also written: Server-side compaction
AI models shorten their memory by keeping important data, allowing them to handle longer tasks and more information.
Compaction is a technique AI models use to manage their memory. Think of it like summarizing a long book. Instead of remembering every single word, the AI keeps the most important plot points and characters. This helps the AI stay focused and understand long conversations or documents without getting confused.
For example, if you're asking an AI to write a story based on a very long prompt, compaction helps it remember the beginning of the story while it's writing the end. It makes the AI more efficient and capable of handling complex, lengthy tasks.
Compaction refers to processes that reduce the size of an AI model's internal state or context window while preserving essential information. This is crucial for managing long sequences, such as extended conversations or large documents, preventing performance degradation and computational overhead. Techniques can include selective memory pruning, attention mechanism optimizations, or data summarization to retain coherence and relevance over extended inputs.
Frontier AI labs are focusing on compaction techniques to improve the efficiency and capabilities of their large language models. This is driven by the need to handle increasingly long contexts and complex tasks, as seen with OpenAI's GPT-6 and GPT-5.1-Codex-Max models and Anthropic's CliffCompaction research. By reducing data size and pruning historical context, these models can manage more information and maintain coherence in demanding applications like coding.
- A model guide for the GPT-6 familyOpenAI · OpenAI News ·
- CliffCompaction: Cost-Efficient Compaction for Long-Horizon Coding AgentsAnthropic · arXiv cs.AI ·
- API blog postsOpenAI · OpenAI Developers Blog ·
- Building more with GPT-5.1-Codex-MaxOpenAI · OpenAI News ·
- GPT-5.1-Codex-Max System CardOpenAI · OpenAI News ·
- Anthropic Claude
- Anthropic's CliffCompaction research specifically highlights a new AI technique for cost-efficient compaction, aiming to improve Claude's ability to work with large codebases.
- OpenAI ChatGPT
- OpenAI's recent GPT-6 family and GPT-5.1-Codex-Max models are noted for their improved efficiency and long-context handling, suggesting the use of compaction techniques to manage complex AI applications and coding tasks.
- [object Object]
- [object Object]
- [object Object]
The concept of compaction in AI is related to memory management and data pruning techniques developed over years of research in machine learning and artificial intelligence.
Ask an AI chatbot to summarize a very long article or document you provide. Observe how well it retains key details from the beginning of the text when answering questions about the end. If the AI struggles to recall earlier information, it might indicate limitations in its compaction or context management capabilities.
The Playbook: reusable how-tos, prompts, configs and tips pulled out of the news.
- Cost tipReduce AI coding costs with CliffCompactionCut AI coding costs up to 50% for long tasks by compacting context without rephrasing. · 15 min
- HeuristicCompaction for multi-context window operationUtilize compaction to enable models to operate coherently across millions of tokens in multiple context windows. · 5 min
- Cost tipManage GPT-6 context and cost with compactionUse compaction to reduce context size for longer conversations while preserving necessary state. · 5 min
- HeuristicCompaction enables long-horizon coherent workCompaction allows models to work coherently over millions of tokens by preserving essential context. · 5 min
- PromptRole-play a student struggling with math for AI tutor simulationSimulate a student's math struggles and uncertainty for AI tutor evaluation. · 5 min
Questions people ask
- Why is compaction important for AI?
- Compaction allows AI models to handle longer inputs and maintain coherence over extended interactions, making them more capable for complex tasks like writing long documents or engaging in lengthy conversations.
- Does compaction make AI 'forget' things?
- Compaction aims to intelligently discard less relevant information while retaining crucial details, so it's more about efficient memory management than outright forgetting.
First seen on Subvolts Nov 19, 2025. Updated Oct 12, 2026. Written by an AI model from the items above and reviewed against them weekly; report an error.