Unknown unknowns
Risks or problems that are not only unrecognized but whose existence is not even suspected.
Unknown unknowns are risks or problems that nobody has thought of yet. They are completely unexpected issues that can arise.
For example, imagine a new self-driving car system. We might test for common problems like bad weather or other cars. An unknown unknown would be if the car's sensors were confused by a specific type of bird that only appears in one small region, causing it to brake suddenly and unexpectedly. This is a problem we didn't even consider during testing.
Unknown unknowns represent a category of risks or failures that are not only unrecognized but whose existence is not even suspected. This contrasts with known unknowns, which are understood risks that have not yet been quantified or addressed. Identifying and mitigating unknown unknowns is a significant challenge in AI safety and system development, often requiring robust monitoring, diverse testing methodologies, and a culture that anticipates unforeseen failure modes.
As AI models become more complex and capable, they are more likely to exhibit emergent behaviors. Researchers are developing new methods, like Anthropic's proposed 'diff' tool, to uncover these unexpected behaviors, which could represent unknown unknowns.
- A “diff” tool for AI: Finding behavioral differences in new modelsAnthropic · Anthropic Research ·
- Anthropic Claude
- Anthropic's research on 'diff' tools aims to detect unexpected behavioral differences in AI models, which could help surface unknown unknowns.
- Known unknowns
- These are risks that are recognized but not yet fully understood
- Black swan events
- These are rare, high-impact events that are unpredictable and
The concept of 'unknown unknowns' was popularized by U.S. Secretary of Defense Donald Rumsfeld in 2002, referring to risks that are not only unrecognized but whose existence is not even suspected. Source ↗
Ask an AI chatbot to write a story about a specific historical event. Then, ask it to write a story about the same event but from the perspective of a fictional character who was present. Compare the two stories for any subtle, unexpected differences in how the event is portrayed, which might hint at unforeseen biases or interpretations.
The Playbook: reusable how-tos, prompts, configs and tips pulled out of the news.
- PromptRole-play a student struggling with math for AI tutor simulationSimulate a student's math struggles and uncertainty for AI tutor evaluation. · 5 min
- TemplateReusable workflow templates for healthcare teamsTeams can create and share templates for common workflows like documentation or prior authorizations. · 15 min
- TemplateTemplate for building an AI tool tailored to your workA structured approach to defining and building a custom AI tool for specific business needs. · 15 min
- TemplateEvals CSV prompt list for agent skillsA CSV file template for listing prompts used in evaluating agent skills. · 20 min
- HeuristicDouble coding agent task completion with docs linksProviding coding agents with a documentation link significantly increases their task completion rate. · 2 min
Questions people ask
- How are unknown unknowns different from known unknowns?
- Known unknowns are risks that we are aware of but don't fully understand or can't measure yet. Unknown unknowns are risks we don't even know exist or suspect could happen.
- Why is this concept important for AI?
- As AI systems become more powerful, they can behave in ways we haven't predicted. Identifying and preparing for unknown unknowns is crucial for ensuring AI safety and reliability.
First seen on Subvolts Apr 3, 2026. Updated Sep 15, 2026. Written by an AI model from the items above and reviewed against them weekly; report an error.