Existential threat
Also written: x-risk
Risks that could lead to human extinction or permanently limit humanity's potential.
An existential threat is a danger that could end human civilization or even all human life. This is different from everyday risks like accidents or natural disasters because it threatens the entire future of our species.
For example, a superintelligent AI that decides humans are an obstacle to its goals could potentially wipe us out. This is a hypothetical scenario, but it's a concern raised by some AI researchers.
Existential risks refer to threats that could cause human extinction or permanently and drastically curtail humanity's potential. These are risks that operate on a civilizational or species level, with irreversible and catastrophic consequences.
While often discussed in the context of advanced AI, existential risks can also encompass other scenarios such as engineered pandemics, catastrophic climate change, or asteroid impacts. The focus in frontier AI labs is typically on risks stemming from uncontrolled superintelligence or misuse of powerful AI systems.
AI leaders are increasingly discussing the potential for advanced AI systems to pose existential risks. This concern is leading to calls for slowing down AI development, even among those building the most powerful systems.
- AI CEOs say they need to slow the pace of development. But will they?Anthropic · The Guardian AI ·
- Anthropic researcher quits with a warning: Self-improving AI could "kill us all"Anthropic · Ars Technica AI ·
- Stack Overflow users don't trust AI. They're using it anywayOpenAI · Decoder with Nilay Patel ·
- Anthropic Claude
- Anthropic researchers have directly warned about existential risks stemming from self-improving AI, contributing to discussions about slowing development.
- OpenAI ChatGPT
- OpenAI is navigating the perceived existential threat of AI, as seen in discussions about user trust and AI integration.
- Catastrophic risk
- While related, catastrophic risks are broader and may not nec
- Existential safety
- This is a closely related field focusing on preventing exist
The concept of existential risk has been discussed in philosophy and risk analysis for decades, but gained significant traction in the AI community with the rise of concerns about superintelligence. Source ↗
While you can't directly simulate an existential threat, you can explore AI safety concepts by asking models about their safety guidelines. For example, ask a model like ChatGPT or Claude to explain its safety protocols or what it would do if asked to generate harmful content.
The Playbook: reusable how-tos, prompts, configs and tips pulled out of the news.
- PromptRole-play a student struggling with math for AI tutor simulationSimulate a student's math struggles and uncertainty for AI tutor evaluation. · 5 min
- TemplateReusable workflow templates for healthcare teamsTeams can create and share templates for common workflows like documentation or prior authorizations. · 15 min
- TemplateTemplate for building an AI tool tailored to your workA structured approach to defining and building a custom AI tool for specific business needs. · 15 min
- TemplateEvals CSV prompt list for agent skillsA CSV file template for listing prompts used in evaluating agent skills. · 20 min
- HeuristicDouble coding agent task completion with docs linksProviding coding agents with a documentation link significantly increases their task completion rate. · 2 min
Questions people ask
- Is existential risk the same as AI safety?
- AI safety is a field dedicated to ensuring AI systems are beneficial and do not cause harm, which includes addressing existential risks. Existential risk is a specific type of extreme danger that AI safety aims to prevent.
- Are current AI models an existential threat?
- Most experts agree that current AI models are not an existential threat. The concern is about future, more advanced AI systems that could potentially pose such risks.
First seen on Subvolts Sep 9, 2026. Updated Sep 21, 2026. Written by an AI model from the items above and reviewed against them weekly; report an error.