Top news
Investing in the public benefit OfficialAnthropic NewsOct 18Popular Musk Pledges SpaceX Shares to ‘Every Kid in America’ WebBloomberg TechnologyOct 19Popular Websites are closing their doors to AI agents WebThe Verge AIOct 17 This is the new ChatGPT Voice, powered by GPT Live OfficialOpenAI on YouTubeOct 18 How AI giants will reshape global philanthropy WebAxios AIOct 19 Gemini 4 Argon Really Surprised Me WebTwo Minute PapersOct 19 Google Gemini 4 agents just beat Minecraft with no human intervention WebEngadget AIOct 19 Who could it be? 🧐 OfficialGoogle on YouTubeOct 18 Anthropic gives outside AI watchdog “employee-like” access WebArs Technica AIOct 18 What are self-replicating ‘AI worms’, and why is Anthropic worried about them? WebThe Guardian AIOct 18
GlossaryModelsIntermediate

Mixture of expert

Also written: MoE, MoE models, Latent-MoE, Latent Mixture-of-Experts

A neural network architecture using multiple specialized subnetworks, called experts, guided by a router to process inputs efficiently.

In plain terms

Mixture of Experts (MoE) is a way to build large AI models. Instead of one giant network trying to do everything, MoE uses several smaller, specialized networks, or 'experts.' Think of it like a team of specialists where each expert is good at a particular type of task.

When the AI gets a request, a 'router' or 'gating network' decides which expert or experts are best suited to handle it. This allows the model to be very large and capable, but only activate the necessary parts for each specific task. This can make it faster and more efficient than a similarly sized model where all parts are always active.

For example, one expert might be good at writing code, another at summarizing text, and a third at answering factual questions. The router directs your prompt to the right expert(s) for the job.

The expert version

Mixture of Experts (MoE) is a model architecture that combines multiple feed-forward networks, termed 'experts.' A gating network dynamically selects which experts process each input token. This approach allows for models with a vast number of parameters, potentially improving performance and efficiency by activating only a subset of parameters per inference step.

The primary tradeoff involves increased complexity in training and routing, but it enables scaling to trillions of parameters while keeping inference costs manageable compared to dense models of equivalent size. Current large models often employ sparse MoE architectures.

Why it matters now

Mixture of Experts (MoE) models are gaining traction as frontier AI labs seek more efficient ways to scale up their large language models. This architecture allows models to activate only specific parts of their network for a given task, potentially reducing computational costs and improving performance. Recent mentions suggest MoE is a key area of development for companies aiming to push the boundaries of AI capabilities.

Where it shows up 21 items, newest first
How each company uses it
Anthropic Claude
Anthropic's Opus 5, Sonnet 5, and Fable 5 models are discussed in contexts of new releases and benchmark gains, suggesting that MoE might be an underlying architecture enabling these advancements or offering specialized capabilities.
Google Gemini
Google's Gemma 4 model family is noted as offering diverse open-source models for multimodal AI development, implying potential use or exploration of MoE architectures for specialized tasks within the family.
OpenAI ChatGPT
OpenAI is mentioned in relation to GPT-5.6, a model that could potentially incorporate MoE principles to manage its complexity and improve efficiency.
Related terms

TransformerLarge Language ModelNeural NetworkParameter EfficiencySparse ModelGating Network

Often confused with
Ensemble Learning
Ensemble methods combine multiple independent models, whereas
Modular Neural Networks
While both use distinct components, MoE specifically re
Multi-task Learning
Multi-task learning trains a single model on multiple tasks
Origin

The Mixture of Experts concept was introduced in the paper 'Explosive Expertise: An Efficient Learning Scheme for Artificial Neural Networks' by Jacobs, Jordan, Nowlan, and Hinton in 1991. Source ↗

Try this about 5 minutes

You can observe the effects of specialized AI by asking a general-purpose chatbot to perform very different tasks in sequence. For instance, ask it to write a poem about a cat, then immediately ask it to explain a complex scientific concept. Notice if there's any lag or if the quality shifts significantly, which could indicate different internal 'experts' being engaged.

Try it: Playbook entries that use thisPlaybook →

The Playbook: reusable how-tos, prompts, configs and tips pulled out of the news.

Questions people ask

Is Mixture of Experts a new idea?
The core concept of Mixture of Experts was introduced in the early 1990s, but it has seen a resurgence and significant advancements with modern deep learning techniques.
How does MoE make AI faster?
MoE models are faster because only a fraction of the total parameters are activated for any given input, making computation more efficient than dense models with the same total parameter count.
Are all large AI models MoE?
No, not all large AI models use the Mixture of Experts architecture. Many still use dense architectures, but MoE is a popular approach for scaling to very large parameter counts efficiently.

First seen on Subvolts Mar 16, 2026. Updated Oct 19, 2026. Written by an AI model from the items above and reviewed against them weekly; report an error.