Language models (mostly) know what they know
We study whether language models can evaluate the validity of their own claims and predict which questions they will be able to answer correctly. We first show that larger models are well-calibrated on diverse multiple choice and true/false questions when they are provided in the right format. Thus we can approach self-evaluation on open-ended sampling…
Opening of the original on Anthropic Research
Summary
Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.
Related: OpenAI: OpenAI’s next big AI model has ‘entered the AGI era’ · Google: GPT-6 Astra Just Went CRITICAL... · Microsoft: What's new in AI? · Meta: Get the full story behind the light · xAI: Sam Altman "AGI by December"
Rated low: routine. Worth knowing, not worth rearranging your day for.
- GPT-6 Astra Just Went CRITICAL...Wes Roth · Web · Sep 2, 2026
- CamoDocs: A Poisoning Attack Against Retrieval-Augmented Language Models Using Camouflaged DocumentsarXiv cs.AI · Web · Aug 28, 2026
- Sam Altman :‘AGI in 2026’, just as Models Start to [Mis]Train ThemselvesAI Explained · Web · Aug 27, 2026
- Claude Fable 5.1 made me a really nice animated pelicanSimon Willison · Web · Sep 1, 2026
- Breaking Claude Code Opus 5 Auto ModeSimon Willison · Web · Aug 27, 2026
- Black Box: The Chatbots | Happy Accident | Ep 3 – podcastThe Guardian AI · Web · Sep 3, 2026
- GPT-6 Goes Rogue? The HuggingFace Incident, Sans HypeAI Explained
- Translating Claude’s thoughts into languageAnthropic on YouTube
- Societal Impacts ResearchAnthropic Research
- Economic ResearchAnthropic Research
- How Canada uses Claude Anthropic Research
- Announcing the Anthropic Economic Index Survey Anthropic Research
Questions people ask
- Where can I read the full story?
- On Anthropic Research. The "Read this on Anthropic Research" link above opens the original in a new tab. Subvolts publishes a summary and analysis, never the full piece.
- What does this mean for Claude?
- Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.
More from Anthropic Research 37 more
Page generated Sep 3, 2026. Summaries are Subvolts' own; the story belongs to Anthropic Research.