Encoder-decoder
A neural network architecture with an encoder to understand input and a decoder to generate output.
An encoder-decoder model is a type of AI system that works in two main steps. First, the encoder part reads and understands the input data, like a sentence or an image, and compresses it into a compact summary. Then, the decoder part takes this summary and uses it to create the desired output, such as a translation or a description.
Think of it like a translator who first reads a sentence in one language, figures out its core meaning, and then writes that meaning in another language. This structure is very common in AI tasks that involve transforming one type of data into another.
An encoder-decoder architecture is a neural network design consisting of two main components. The encoder maps the input sequence to a fixed-length vector representation (often called a context vector or thought vector). The decoder then takes this vector and generates an output sequence, one element at a time. This approach is fundamental to sequence-to-sequence tasks like machine translation and text summarization. Tradeoffs include potential information loss in the fixed-length vector for very long sequences, though attention mechanisms have largely mitigated this.
Encoder-decoder models are central to many recent advancements in generative AI. The launch of T5Gemma by Google DeepMind highlights their continued importance as a foundational architecture for large language models. Their application in areas like 4D scene reconstruction shows their versatility beyond traditional text tasks.
- D4RT: Teaching AI to see the world in four dimensionsGoogle · Google DeepMind Blog ·
- T5Gemma: A new collection of encoder-decoder Gemma modelsGoogle · Google DeepMind Blog ·
- Google Gemini
- Google DeepMind has launched T5Gemma, a new collection of open-weights encoder-decoder LLMs, demonstrating their ongoing development and release of this model type.
- Autoencoder
- An autoencoder also uses an encoder and decoder but is primarily de
- Generative Adversarial Network (GAN)
- GANs involve two networks, a generator and
- Transformer
- While Transformers often use an encoder-decoder structure, the term
The encoder-decoder architecture was first proposed in the context of machine translation by Cho et al. in 2014, utilizing recurrent neural networks. Source ↗
Use a translation tool like Google Translate or DeepL. Input a sentence in one language, and the tool outputs the translated sentence in another. The underlying AI model likely uses an encoder-decoder structure to understand your input and generate the translation.
The Playbook: reusable how-tos, prompts, configs and tips pulled out of the news.
- PromptRole-play a student struggling with math for AI tutor simulationSimulate a student's math struggles and uncertainty for AI tutor evaluation. · 5 min
- TemplateReusable workflow templates for healthcare teamsTeams can create and share templates for common workflows like documentation or prior authorizations. · 15 min
- TemplateTemplate for building an AI tool tailored to your workA structured approach to defining and building a custom AI tool for specific business needs. · 15 min
- TemplateEvals CSV prompt list for agent skillsA CSV file template for listing prompts used in evaluating agent skills. · 20 min
- HeuristicDouble coding agent task completion with docs linksProviding coding agents with a documentation link significantly increases their task completion rate. · 2 min
Questions people ask
- What is the main purpose of an encoder-decoder model?
- The main purpose is to transform an input sequence into an output sequence, which is useful for tasks like translation, summarization, and question answering.
- How does the encoder-decoder model handle long inputs?
- Early models struggled with long inputs due to information loss in the fixed-length vector. Modern architectures often use attention mechanisms to allow the decoder to focus on relevant parts of the input at each step.
First seen on Subvolts Oct 25, 2025. Updated Sep 15, 2026. Written by an AI model from the items above and reviewed against them weekly; report an error.