Subvolts take AI models are vulnerable to social engineering; learn how to find and fix these flaws. Dec 2, 2025 - Developers
- Learn how to proactively test your AI agents for security flaws using conversational simulation techniques.
- Business
- Understand the critical "lethal trifecta" that leads to AI security incidents and how to prevent them.
This interview provides practical insights into AI security from a practitioner who builds tools to test AI systems.
Why Social Engineering Now Works on Machines
Audio streams from a16z. Your place is remembered on this device.
From Subvolts
Summary and analysis by Subvolts editorial, AI-assisted.
This AI + a16z podcast episode features Ian Webster discussing how social engineering tactics now work on AI models. Webster, founder of PromptFoo, explains how his tools identify vulnerabilities by simulating conversations, preventing security incidents before they impact users. The discussion highlights the "lethal trifecta" of untrusted input, sensitive data, and exfiltration channels leading to AI security breaches.
Imagine teaching a computer to be helpful, but it can still be tricked like a person. This episode explains how people can "talk" to AI in ways that make it reveal secrets or break its own rules. It's important because it helps make AI safer for everyone to use.
This episode is for AI developers and security professionals. It reveals how AI systems, like those developed by OpenAI and Google Gemini, can be tricked into revealing sensitive information or breaking security rules. Webster's approach of simulating user interactions offers a practical method for red-teaming AI agents. Understanding these vulnerabilities is crucial as frontier labs focus on AI safety and enterprise adoption.
- Hosta16z
- GuestIan Websterfounder, PromptFoo
- Guest: Ian Webster
- Title: Why Social Engineering Now Works on Machines
- Show: AI + a16z
- Length: 25 minutes
- PromptFoo's method: 30,000 conversations with a system
- Vulnerability combination: Untrusted input plus sensitive data plus an exfiltration channel
Ian Webster built PromptFoo after watching 200 million Discord users systematically dismantle his AI agent—now Fortune 10 companies pay him to break theirs before customers do. The "lethal trifecta" sounds academic until you realize it's already happening: untrusted input plus sensitive data plus an exfiltration channel equals the security incident that just cost a SaaS company its multi-tenancy guarantees. Webster's red-teaming agents don't use signatures—they have 30,000 conversations with your system, socially engineering their way past guardrails the same way a teenager with emojis convinced ChatGPT to leak data, except his tools find the vulnerability before your users become the pen testers.
Follow Ian Webster on X: https://x.com/iwebst
Follow Joel on LinkedIn: https://www.linkedin.com/in/3448827723723234/
Check out everything a16z is doing with artificial intelligence here, including articles, projects, and more podcasts.
Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.
- social engineering
- Ian Webster
- Tricking people or systems into revealing information or performing actions they shouldn't.
- AI agent
- Ian Webster
- A program that uses artificial intelligence to perform tasks autonomously.
- red-teaming
- Ian Webster
- Simulating attacks on a system to find security weaknesses before real attackers do.
- exfiltration channel
- Ian Webster
- A pathway used to secretly steal data from a system.
24 minHow AI Is Rewriting the Rules of Cybersecurity | Truffle Security & Socket
WithDylan Ayrey, Feross Aboukhadijeh
47 minInside vLLM: The Engine Powering Open-Source AI
WithSimon Mo
41 minBuilding Self-Accelerating AI with Mirendil
WithBehnam Neyshabur, Harsh Mehta
43 minIdeogram’s Open-Weights Image Model and the Future of AI Design
WithMohammad Norouzi
1 h 04
30 minScience, Innovation, and Economic Growth: OpenAI’s Ronnie Chatterji
WithRonnie Chatterji
1 h 28Pedro Domingos: Tensor Logic Unifies AI Paradigms
WithPedro Domingos
Questions people ask
- What is the "lethal trifecta"?
- It is a combination of untrusted input, sensitive data, and an exfiltration channel that creates a security incident.
- How does PromptFoo find AI vulnerabilities?
- PromptFoo agents simulate conversations, engaging in 30,000 interactions to socially engineer their way past guardrails and find weaknesses.
Page generated Oct 19, 2026. Summaries are Subvolts' own; the audio, artwork and notes belong to a16z.