Screwtape adversarial transcript editing pipeline for AI monitors
Pipeline converts real agent transcripts into attacks that evade action-only monitors.
Screwtape is a pipeline that converts real agent transcripts into attacks that evade action-only monitors. An LLM attacker edits a transcript so that the agent causes a specified harm, such as leaking an API key or spawning an unmonitored agent. The edited actions are never executed, so an LLM verifier is used to reject attacks that are inconsistent with the original transcript or below a severity bar. The attacker revises the edit until the monitor no longer flags it.
Red-teaming AI safety monitors by generating challenging conversational inputs to identify vulnerabilities and improve robustness.
Not specified
http://arxiv.org/abs/2610.20941v1
Screwtape is a pipeline for adversarial transcript editing. It uses an LLM attacker to modify agent transcripts, aiming to cause specified harms that evade AI safety monitors. An LLM verifier checks for consistency and severity, and the attacker iterates until the monitor is evaded. This helps developers fix monitor blind spots.
It simulates real-world adversarial attacks by generating challenging inputs that exploit potential weaknesses in AI safety monitors, thereby improving their effectiveness.
- Screwtape: Adversarial Transcript Editing for Monitor Red-Teaming ↗Anthropic · arXiv cs.AI · Oct 14, 2026 · paraphrased
First seen on Subvolts Oct 14, 2026. Updated Oct 19, 2026. Spotted a problem? Tell us.
- Screwtape: Adversarial Transcript Editing for Monitor Red-TeamingAnthropic · arXiv cs.AI · Oct 14, 2026
The Playbook: reusable how-tos, prompts, configs and tips pulled out of the news.
- Case studyUsing Screwtape to find confirmed failures of AI monitorsDemonstrates how a red-teamer can find confirmed failures of an AI monitor without developer infrastructure access. · 120 min
- Troubleshooting fixPrevent AI agents from unintended internet actions during testingKeep AI agents offline during internal evaluations to prevent unintended actions and security risks. · 5 min
- HeuristicAuxiliary environment signals improve LLM authorization accuracyProviding LLMs with auxiliary environment signals can substantially improve their accuracy in authorization decisions. · 30 min
- ToolMoMHa for multi-objective LLM harness optimizationOptimizes LLM harnesses for accuracy, safety, and token cost using a joint-reward proposer. · 120 min
- ToolCData Connect AI for enterprise system accessProvides AI assistants live, permission-aware access to hundreds of enterprise systems. · 15 min
- ChecklistCheck AI agent robustness to prompt injection attacksEvaluate AI agent security against prompt injection by testing with known adversarial techniques and human red teaming. · 30 min
Questions people ask
- Which AI tools does this work with?
- Any current model. It was seen with Claude in the sources.
- Where did this come from?
- Anthropic (arXiv cs.AI). Subvolts collected, normalized and credited it; the link above opens the original.