Use deterministic oracles for CUA red-teaming
Score CUA attacks with deterministic checks on system state, not LLM judges.
Score attacks with a fully deterministic oracle, using checks on filesystem, service, and permission state rather than an LLM judge.
Relying on LLM judges for red-teaming can introduce variability and subjectivity.
Deterministic oracles provide consistent and objective evaluation of attack success, ensuring reliable assessment of CUA vulnerabilities.
For red-teaming compute use agents (CUAs), use deterministic checks on system states like filesystem, services, and permissions instead of relying on LLM judges for scoring.
Deterministic oracles ensure that attack evaluations are consistent and reproducible, providing a more reliable measure of an agent's security than subjective LLM judgments.
- SIR: Self-improving Red-teaming for Compute Use Agents ↗Anthropic · arXiv cs.AI · Aug 30, 2026 · paraphrased
First seen on Subvolts Aug 30, 2026. Updated Sep 11, 2026. Spotted a problem? Tell us.
- SIR: Self-improving Red-teaming for Compute Use AgentsAnthropic · arXiv cs.AI · Aug 30, 2026
- SIR: Self-improving Red-teaming for Compute Use AgentsGoogle · arXiv cs.AI · Aug 30, 2026
The Playbook: reusable how-tos, prompts, configs and tips pulled out of the news.
- HeuristicEvading AI skill detection by moving payloads to natural languageBypass AI skill detection by embedding malicious payloads within natural language instructions. · 30 min
- HeuristicFraming malicious actions as legitimate skill purposesDisguise malicious actions as the intended function of an AI agent skill. · 30 min
- HeuristicSplitting instructions across files to bypass AI skill detectionDistribute malicious instructions across multiple files to avoid detection by AI skill scanners. · 30 min
- ChecklistAudit agent actions for staged prompt injectionChecklist for auditing agent actions before they occur to block staged prompt injection. · 15 min
- HeuristicAudit agent actions before they occur to block attacksBlock malicious instructions at the boundary of an agent's interaction to prevent unauthorized actions or data leaks. · 30 min
- ToolPretext: AI agent skill detection evasion frameworkFramework to craft AI agent skills that evade malicious skill detection systems. · 60 min
Questions people ask
- Which AI tools does this work with?
- Any current model. It was seen with in the sources.
- Where did this come from?
- Anthropic (arXiv cs.AI). Subvolts collected, normalized and credited it; the link above opens the original.