AI-resistant take-home evaluation challenge
A template for a take-home coding challenge designed to be resistant to AI solutions.
You are tasked with optimizing code for a simulated accelerator with specific characteristics: manually managed scratchpad memory, VLIW, SIMD, and multicore. The problem involves a parallel tree traversal, deliberately not deep learning flavored. Your goal is to achieve the best possible performance within the given time constraints, using the provided hot-reloading Perfetto trace for debugging and analysis. You are permitted to use AI tools as you would on the job, but your solution must demonstrate your own skills in performance optimization. Accelerator Features: - Manually managed scratchpad memory - VLIW (multiple execution units running in parallel each cycle) - SIMD (vector operations on many elements per instruction) - Multicore (distributing work across cores) Task: Optimize a parallel tree traversal algorithm for maximum performance on this simulated accelerator. Evaluation Criteria: - Performance achieved (e.g., execution time, throughput) - Code efficiency and clarity - Effective use of accelerator features - Approach to debugging and optimization
What it does and when to use it
This template outlines a take-home coding challenge focused on optimizing code for a simulated accelerator with complex features. It allows AI tool usage but requires candidates to demonstrate core performance engineering skills.
Why it works
The complexity of the simulated hardware and the nature of the optimization task make it difficult for current AI models to generate a fully optimized solution without significant human guidance and understanding.
Sources and credit
- Designing AI resistant technical evaluations ↗Anthropic Engineering · Anthropic Engineering · Jan 21, 2026 · quoted as shared
First seen on Subvolts Jan 21, 2026. Updated Sep 11, 2026. Spotted a problem? Tell us.
Where it shows up
- Designing AI resistant technical evaluationsAnthropic · Anthropic Engineering · Jan 21, 2026
Terms used
Related in the PlaybookPlaybook →
The Playbook: reusable how-tos, prompts, configs and tips pulled out of the news.
- TemplateInteractive tutorial notebook for hydrology model implementationA Python notebook to guide users through implementing the hydrology model code. · 30 min
- HeuristicUse issue trackers as control planes for AI agentsLeverage your issue tracker as a central control plane to manage and orchestrate AI agentic work. · 10 min
- Case studyUsing LLMs to build tools for iOS location data analysisResearchers used an LLM to build a tool that decoded iOS location data, overcoming LLM analysis limitations. · 60 min
- ToolExtractor tool to recover protobuf schemas from Apple binariesPython tool to reliably recover protobuf schemas from Apple binaries for analysis. · 30 min
- HeuristicAdversarial Loop for Robust Software PatchingUse an adversarial loop to ensure software patches are robust against varied exploits. · 60 min
- ToolP2Patch for Exploit-Guided Vulnerability RepairAn AI system that uses adversarial attacks to find and fix software vulnerabilities. · 120 min
Questions people ask
- Which AI tools does this work with?
- Any current model. It was seen with in the sources.
- Where did this come from?
- Anthropic Engineering (Anthropic Engineering). Subvolts collected, normalized and credited it; the link above opens the original.