Existential Risk
Dangers that could end humanity or permanently limit our future.
Existential risk refers to dangers that could cause human extinction or permanently cripple humanity's future potential. These are not everyday risks but threats on a global, species-ending scale.
An example would be a future superintelligent AI that, in pursuing its goals, inadvertently causes a catastrophe that wipes out humanity. The concern is that such an AI might not understand or value human life, leading to unintended but irreversible consequences for our species.
Existential risk (x-risk) denotes threats that could cause the permanent destruction of humanity's potential or lead to extinction. These risks are characterized by their finality and scope, affecting all future generations. The focus is on scenarios where recovery or reversal is impossible, distinguishing them from other catastrophic events that humanity might survive and rebuild from.
Concerns about existential risk from AI are becoming more prominent as advanced AI systems develop. Recent discussions and warnings from researchers and within company filings highlight the potential for AI to pose a threat to humanity's future.
- Anthropic’s prospectus details losses, growth, and, yes, a warning that its AI could end humanityAnthropic · TechCrunch AI ·
- Anthropic's Existential Risk Warnings Hijack Larger AI DebateAnthropic · Bloomberg Technology ·
- OpenAI, Anthropic, Google have been in talks on AI safety for weeksAnthropic · TechCrunch AI ·
- Anthropic researcher says AI has more than 10% chance of 'killing all humans' after colleague quitsAnthropic · CNBC Technology ·
- Governance of superintelligenceOpenAI · OpenAI News ·
- Anthropic Claude
- Anthropic has explicitly detailed existential risk in its filings, with researchers warning about the potential for AI to cause human extinction.
- OpenAI ChatGPT
- OpenAI has proposed proactive governance strategies for superintelligence, indicating a focus on managing potential existential risks.
- Catastrophic Risk
- While existential risks are a subset of catastrophic risks, n
- Global Catastrophic Risk
- This term is broader and can include natural disasters
- Societal Collapse
- Societal collapse is a severe outcome, but existential risk i
The concept of existential risk was significantly popularized by philosopher Nick Bostrom in his work on future humanity and the potential dangers posed by advanced technologies. Source ↗
Ask an AI model like ChatGPT or Claude about the potential dangers of advanced AI. For example, ask 'What are the biggest risks associated with developing superintelligent AI?' and observe how it discusses potential negative outcomes, including those that could be considered existential.
The Playbook: reusable how-tos, prompts, configs and tips pulled out of the news.
- PromptRole-play a student struggling with math for AI tutor simulationSimulate a student's math struggles and uncertainty for AI tutor evaluation. · 5 min
- TemplateReusable workflow templates for healthcare teamsTeams can create and share templates for common workflows like documentation or prior authorizations. · 15 min
- TemplateTemplate for building an AI tool tailored to your workA structured approach to defining and building a custom AI tool for specific business needs. · 15 min
- TemplateEvals CSV prompt list for agent skillsA CSV file template for listing prompts used in evaluating agent skills. · 20 min
- HeuristicDouble coding agent task completion with docs linksProviding coding agents with a documentation link significantly increases their task completion rate. · 2 min
Questions people ask
- Is existential risk the same as AI safety?
- AI safety is a field focused on ensuring AI systems are beneficial and do not cause harm. Existential risk is a specific type of extreme risk that AI safety research aims to prevent.
- Are current AI models an existential risk?
- Most experts do not consider current AI models to be an existential risk. The concern is primarily focused on future, more advanced AI systems that could possess capabilities far beyond human control.
First seen on Subvolts May 22, 2023. Updated Oct 6, 2026. Written by an AI model from the items above and reviewed against them weekly; report an error.