Dangerous capabilities
AI capabilities that could be misused to cause widespread harm.
Dangerous capabilities are potential abilities of advanced AI systems that could be used to cause significant harm. Think of abilities that go beyond what current AI can do, and could be misused by bad actors or even accidentally cause problems.
For example, an AI that could rapidly design novel biological agents or cyberweapons would be considered a dangerous capability. The concern is that these powerful tools, if not carefully controlled, could pose a serious risk to public safety.
Dangerous capabilities refer to emergent or foreseeable abilities of AI systems that, if misused or uncontrolled, could lead to catastrophic societal harm. These capabilities often arise from scaling models to unprecedented sizes or training them on vast datasets, leading to emergent behaviors not explicitly programmed. Examples include the potential for AI to accelerate the development of novel pathogens, design sophisticated cyberattacks, or enable mass manipulation.
Identifying and mitigating these capabilities is a central challenge in AI safety research. It involves both predicting potential harms and developing technical and policy safeguards to prevent their realization.
OpenAI's recent proposal highlights the growing concern around dangerous capabilities as AI models become more powerful. The company suggests that as AI advances, so does its potential for misuse, necessitating proactive regulation and safety measures.
- Frontier AI regulation: Managing emerging risks to public safetyOpenAI · OpenAI News ·
- OpenAI ChatGPT
- OpenAI explicitly discusses the need to manage dangerous capabilities as AI systems advance, proposing regulatory frameworks to address these risks.
- AI ethics
- Concerns about fairness, bias, and societal impact of AI, which are d
- AI alignment
- The technical challenge of ensuring AI systems act in accordance w
- AI risk
- A broader term encompassing all potential negative outcomes of AI, incl
The term 'dangerous capabilities' is widely used in AI safety discourse, particularly by organizations like OpenAI, to describe the potential for advanced AI systems to cause severe harm. Source ↗
While directly demonstrating dangerous capabilities is not feasible or safe, you can explore AI's potential by prompting models like ChatGPT to brainstorm creative solutions to complex problems. Observe how the AI generates novel ideas, and consider how such generative power, if amplified, could be applied to less benign tasks.
The Playbook: reusable how-tos, prompts, configs and tips pulled out of the news.
- PromptRole-play a student struggling with math for AI tutor simulationSimulate a student's math struggles and uncertainty for AI tutor evaluation. · 5 min
- TemplateReusable workflow templates for healthcare teamsTeams can create and share templates for common workflows like documentation or prior authorizations. · 15 min
- TemplateTemplate for building an AI tool tailored to your workA structured approach to defining and building a custom AI tool for specific business needs. · 15 min
- TemplateEvals CSV prompt list for agent skillsA CSV file template for listing prompts used in evaluating agent skills. · 20 min
- HeuristicDouble coding agent task completion with docs linksProviding coding agents with a documentation link significantly increases their task completion rate. · 2 min
Questions people ask
- What makes a capability 'dangerous'?
- A capability is considered dangerous if it could be misused to cause widespread harm, such as enabling the creation of dangerous substances or facilitating large-scale cyberattacks.
- Are dangerous capabilities present in current AI models?
- While current models may exhibit precursors, the primary concern is about future, more advanced AI systems that could possess significantly more potent dangerous capabilities.
First seen on Subvolts Jul 6, 2023. Updated Sep 15, 2026. Written by an AI model from the items above and reviewed against them weekly; report an error.