State Space Models
Also written: SSMs
AI models that track internal states to efficiently process sequences, like Mamba-2.
State space models (SSMs) are a way for AI to process sequences of data, like text or audio. Instead of looking at the whole sequence at once, they keep an internal "state" that summarizes what they've seen so far. As new data comes in, they update this state and use it to make predictions.
Think of it like reading a book. You don't re-read the whole book every time you read a new sentence. You keep track of the story in your mind (your state) and use that to understand the next sentence. SSMs work similarly, making them efficient for long sequences.
State space models (SSMs) represent system dynamics using a set of latent state variables, enabling efficient sequence modeling. They discretize continuous-time state space representations, allowing for recurrent computation during inference and parallelizable convolution during training. This duality offers a compelling alternative to traditional recurrent neural networks (RNNs) and transformers, particularly for long sequences where computational cost is a concern.
SSMs excel at capturing long-range dependencies due to their ability to maintain and update a compressed state representation. Recent advancements, such as Mamba, have further improved their performance and efficiency, making them competitive with or even superior to transformers on various sequence modeling tasks.
State space models are gaining prominence as researchers seek more efficient alternatives to transformers for processing long sequences. Google's recent work on Titans and MIRAS highlights the ongoing effort to imbue AI with long-term memory capabilities, a problem SSMs are well-suited to address.
- Titans + MIRAS: Helping AI have long-term memoryGoogle · Google Research Blog ·
- Google Gemini
- Google Gemini is exploring state space models, as evidenced by their work on Titans and MIRAS, which aim to provide AI with long-term memory.
- Recurrent Neural Networks (RNNs)
- While both process sequences, SSMs often offer
- Transformers
- Transformers use attention mechanisms to weigh the importance of d
- Convolutional Neural Networks (CNNs)
- CNNs excel at grid-like data (images) usin
The concept of state space models originates from control theory, developed in the 1960s to model dynamical systems. Its application to deep learning for sequence modeling has seen significant recent advancements. Source ↗
While direct experimentation with SSMs might require coding, you can observe their effects by comparing how different AI models handle very long texts. Try asking a chatbot to summarize a lengthy document or a long conversation transcript. Models that better retain context from the beginning of the text are likely using architectures that handle long sequences well, potentially including SSMs.
The Playbook: reusable how-tos, prompts, configs and tips pulled out of the news.
- PromptRole-play a student struggling with math for AI tutor simulationSimulate a student's math struggles and uncertainty for AI tutor evaluation. · 5 min
- TemplateReusable workflow templates for healthcare teamsTeams can create and share templates for common workflows like documentation or prior authorizations. · 15 min
- TemplateTemplate for building an AI tool tailored to your workA structured approach to defining and building a custom AI tool for specific business needs. · 15 min
- TemplateEvals CSV prompt list for agent skillsA CSV file template for listing prompts used in evaluating agent skills. · 20 min
- HeuristicDouble coding agent task completion with docs linksProviding coding agents with a documentation link significantly increases their task completion rate. · 2 min
Questions people ask
- Are state space models better than transformers?
- SSMs can be more efficient for very long sequences and may offer advantages in certain tasks. However, transformers are currently more widely adopted and perform exceptionally well across a broad range of applications.
- How do SSMs help AI remember things?
- SSMs maintain an internal 'state' that summarizes past information. This state is updated as new data arrives, allowing the model to retain context and 'remember' relevant details from earlier in a sequence.
First seen on Subvolts Dec 4, 2025. Updated Sep 15, 2026. Written by an AI model from the items above and reviewed against them weekly; report an error.