Reinforcement learning (RL) from human feedback
Also written: RLHF
AI learns from human feedback, getting rewards for good actions and penalties for bad ones.
Reinforcement learning from human feedback, or RLHF, is a way to train AI models. Imagine teaching a dog a trick. You give it a treat when it does well and say 'no' when it makes a mistake. RLHF works similarly, but with AI.
AI systems try to complete a task. Humans then rate the AI's performance, essentially giving it a 'treat' for good outputs and a 'penalty' for bad ones. The AI learns from these ratings to improve its behavior over time, aiming to produce more of the outputs humans prefer.
Reinforcement learning from human feedback (RLHF) fine-tunes large language models by optimizing a reward model trained on human preferences. Human evaluators rank or rate model outputs, and this data is used to train a separate reward model. The LLM is then trained using reinforcement learning, with the reward model providing the signal to encourage preferred behaviors and discourage undesirable ones. This approach is crucial for aligning model behavior with human values and instructions, though it can be data-intensive and susceptible to reward hacking.
Reinforcement learning from human feedback is a core technique for aligning AI behavior with human preferences. As frontier AI labs like OpenAI continue to develop more capable models, ensuring their outputs are safe and helpful relies heavily on RLHF. Recent discussions around AI safety and governance, such as board appointments at OpenAI, highlight the ongoing importance of this training method.
- OpenAI adds a prominent AI doomer to its board of directorsOpenAI · TechCrunch AI ·
- OpenAI ChatGPT
- OpenAI has extensively used RLHF to train its models, including ChatGPT, to be more helpful, harmless, and honest by incorporating human feedback into the training loop.
- Supervised learning
- Supervised learning trains models on labeled input-output p
- Reinforcement learning
- Standard RL involves an agent learning from environmenta
- Direct Preference Optimization (DPO)
- DPO is a newer method that achieves simila
The concept of using human feedback to guide AI learning has roots in early AI research, but RLHF as a specific technique for LLMs gained prominence with work from OpenAI and others around 2020-2022. Source ↗
Ask ChatGPT to write a short story about a cat. Then, ask it to rewrite the story, but make the cat more mischievous. Observe how the model adjusts its output based on your follow-up instructions, which act as a form of feedback.
The Playbook: reusable how-tos, prompts, configs and tips pulled out of the news.
- PromptRole-play a student struggling with math for AI tutor simulationSimulate a student's math struggles and uncertainty for AI tutor evaluation. · 5 min
- TemplateReusable workflow templates for healthcare teamsTeams can create and share templates for common workflows like documentation or prior authorizations. · 15 min
- TemplateTemplate for building an AI tool tailored to your workA structured approach to defining and building a custom AI tool for specific business needs. · 15 min
- TemplateEvals CSV prompt list for agent skillsA CSV file template for listing prompts used in evaluating agent skills. · 20 min
- HeuristicDouble coding agent task completion with docs linksProviding coding agents with a documentation link significantly increases their task completion rate. · 2 min
Questions people ask
- What is the goal of RLHF?
- The main goal is to make AI models, especially language models, behave in ways that humans find helpful, honest, and harmless, aligning their outputs with human values and instructions.
- Is RLHF the only way to train AI?
- No, RLHF is one of several training methods. Other common methods include supervised learning and unsupervised learning, often used in combination.
First seen on Subvolts Sep 9, 2026. Updated Sep 16, 2026. Written by an AI model from the items above and reviewed against them weekly; report an error.