Subvolts take AI control is possible; we've regulated dangerous tech before, argues a former Pentagon official. May 31, 2026 - Developers
- Understand the policy and legal arguments surrounding AI deployment, especially concerning accountability.
- Business
- Learn about the potential legal liabilities and regulatory challenges for AI products making critical decisions.
- Everyday users
- Grasp the risks of AI systems making judgments about individuals and the importance of accountability.
This is a solid interview with a former high-ranking government official on AI policy and control.
When AI Decides You're a Threat — Brad Carson
Audio streams from Machine Learning Street Talk (MLST). Your place is remembered on this device.
From Subvolts
Summary and analysis by Subvolts editorial, AI-assisted.
This Machine Learning Street Talk episode features Brad Carson, former Army General Counsel and Acting Under Secretary of Defense, discussing AI policy. Carson argues against the idea of unstoppable AI, citing historical precedents. The conversation explores the risks of AI decision-making in critical situations, like Palantir's heat maps, and the lack of accountability when AI models err.
This episode discusses if AI is too powerful to control. A former government official argues we can manage risky AI, like we did with other new technologies. It also covers how AI might make critical decisions, like identifying threats, and who is responsible when it makes mistakes.
This episode is for anyone concerned about AI's societal impact and governance. Carson, with deep government and defense experience, offers a grounded perspective on controlling advanced technology. It connects to current AI safety discussions, particularly around accountability and the potential for AI to make life-or-death decisions without human recourse. Listeners will understand the arguments for proactive AI regulation and the challenges in assigning responsibility when AI systems fail, a critical issue as frontier labs deploy increasingly autonomous systems.
- HostTim Scarfe
- GuestBrad CarsonFormer Army General Counsel, Americans for Responsible Innovation
- Guest: Brad Carson
- Title: When AI Decides You're a Threat
- Show: Machine Learning Street Talk
- Length: 81 minutes
- Carson's core claim: The genie is not out of the bottle; we have pulled dangerous tech back before.
- Example of AI risk: A Palantir heat map scores 0.73 on whether someone is a combatant, leading to strikes.
Brad Carson was the Army's General Counsel, served two terms in Congress and was Acting Under Secretary of Defense for Personnel and Readiness. He now heads Americans for Responsible Innovation, the AI-policy advocacy group he co-founded. Keith Duggar spends roughly eighty minutes pushing back.
SPONSOR:
---
Cyber Fund built the Monastery to help founders ship products that were impossible a year ago. Applications for Batch 1 are now open.
Apply now: https://cyber.fund
---
Carson's whole case rests on one line: the genie is not out of the bottle. We have pulled dangerous tech back before. Asilomar halted recombinant DNA in 1975, and the West still controls the chips AI runs on. Calling it unstoppable, he says, is the most dangerous idea in the room.
Then Keith drags him somewhere darker. A Palantir heat map scores you 0.73 on whether you are a combatant, and a strike follows. The model is wrong some accepted share of the time, and when it is, nobody answers for it. You cannot court-martial a model, and not even the interpretability researchers can say why it picked you.
—
Note: after recording, we learned that Americans for Responsible Innovation is backed by EA-aligned philanthropy (not sponsored)
---
TIMESTAMPS:
00:00:00 From the Pentagon to AI governance
00:04:52 Regulatory capture vs Silicon Valley networks
00:07:56 Transparency and the Claude tier changes
00:09:40 Tort liability when AI tools cause harm
00:13:40 AI is a product, not a person
00:16:01 Children, suicide, and the suicide business
00:19:59 Opaque neural nets and the law of war
00:25:54 Probabilistic targeting and the death of accountability
00:28:47 The arms race fallacy: Asilomar and restraint
00:34:02 Talking to China: track 2 talks and chip leverage
00:39:45 Air power never wins: capital for labour
00:43:29 Anthropic vs the Department of War
00:51:29 Concentration, open source, and brain drain
01:00:18 DeepSeek, Chinese culture, and AI as diplomacy
01:12:25 Upskilling Congress and why public trust matters
---
REFERENCES:
organization:
[00:02:45] ICRC position on autonomous weapons
https://www.icrc.org/en/law-and-policy/autonomous-weapons
[00:05:22] Americans for Responsible Innovation (ARI)
https://ari.us
[00:07:20] Andreessen Horowitz (a16z)
https://a16z.com/
[01:16:05] Office of Technology Assessment
https://en.wikipedia.org/wiki/Office_of_Technology_Assessment
other:
[00:03:35] Beneficial AGI 2019 Conference (Future of Life Institute, Puerto Rico)
https://futureoflife.org/event/beneficial-agi-2019/
[00:18:30] Section 230 of the Communications Decency Act
https://en.wikipedia.org/wiki/Section_230
[00:19:59] Lethal Autonomous Weapons (LAWS)
https://en.wikipedia.org/wiki/Lethal_autonomous_weapon
[00:31:35] Strategic Arms Limitation Talks (SALT)
https://en.wikipedia.org/wiki/Strategic_Arms_Limitation_Talks
[00:32:28] Asilomar Conference on Recombinant DNA (1975)
https://en.wikipedia.org/wiki/Asilomar_Conference_on_Recombinant_DNA
[00:39:45] The New Iron Triangle (ARI policy byte)
https://ari.us/policy-bytes/the-new-iron-triangle/
[00:48:05] Defense Production Act
https://en.wikipedia.org/wiki/Defense_Production_Act
person:
[00:03:35] Anthony Aguirre
https://en.wikipedia.org/wiki/Anthony_Aguirre
[00:06:48] Dean Ball — Hyperdimensional
https://www.hyperdimensional.co/
[00:23:13] Neel Nanda — mechanistic interpretability
https://www.neelnanda.io/
[00:36:02] Jack Clark (Anthropic) on Conversations with Tyler
https://conversationswithtyler.com/episodes/jack-clark/
[00:39:15] Robert Trager — Centre for the Governance of AI
ht
- recombinant DNA
- Brad Carson
- a type of genetic engineering
- tort liability
- Tim Scarfe
- legal responsibility for causing harm
- interpretability researchers
- Tim Scarfe
- experts who try to understand how AI models make decisions
48 minWhat Most People Get Wrong About Evolution | Akarsh Kumar
WithAkarsh Kumar
1 h 101 h 14Who Checks a Proof No Human Can Read? — Leo de Moura
WithLeo de Moura
44 min
1 h 16Alex Imas and Phil Trammell – What remains scarce after AGI?
WithAlex Imas, Phil Trammell
50 minBuilding Search for AI Agents with Exa CEO Will Bryk
WithWill Bryk
Questions people ask
- What is the main argument against AI being unstoppable?
- Brad Carson argues that humanity has successfully halted or controlled dangerous technologies in the past, suggesting AI is not inherently beyond our control.
- What are the risks of AI making critical decisions?
- The episode highlights the danger of AI systems like Palantir's heat maps incorrectly identifying individuals as threats, with no clear accountability for errors.
Page generated Oct 19, 2026. Summaries are Subvolts' own; the audio, artwork and notes belong to Machine Learning Street Talk (MLST).