LoCoBench for training and evaluating proof auto-formalization
A dataset for training and evaluating AI systems on formalizing mathematical proofs from diverse sources.
LoCoBench is a dataset comprising 58.9k instances from Mathlib, CSLib, Mizar Math Library, and a bounded-arithmetic textbook. It includes a 771-instance validation split with theorem-proof pairs that have no public Lean formalization.
Training and evaluating AI systems, such as AIProver, on the task of autonomous proof auto-formalization and proof synthesis.
Not specified
http://arxiv.org/abs/2610.05367v1
LoCoBench is a dataset designed for training and evaluating AI systems in mathematical proof formalization. It contains nearly 59,000 instances from various mathematical libraries and textbooks, including a challenging subset with no existing formalizations.
It provides a comprehensive and challenging benchmark for AI systems attempting to translate natural language mathematical theorems and proofs into formal languages, pushing the boundaries of current capabilities.
Not suitable for tasks outside of mathematical proof formalization.
- AIProver: Agentic Auto-Formalization of Mathematical Research via Certificate-Driven Evolving Harness ↗arXiv · arXiv cs.AI · Oct 4, 2026 · paraphrased
First seen on Subvolts Oct 4, 2026. Updated Oct 8, 2026. Spotted a problem? Tell us.
- AIProver: Agentic Auto-Formalization of Mathematical Research via Certificate-Driven Evolving HarnessAnthropic · arXiv cs.AI · Oct 4, 2026
The Playbook: reusable how-tos, prompts, configs and tips pulled out of the news.
- ToolAIProver for autonomous proof auto-formalization and synthesisAn agentic framework for automating mathematical proof formalization and synthesis using evolving harnesses. · 240 min
- HeuristicFocus memory use on evidence, not just relevanceShift LLM memory use from finding relevant past info to building specific evidence sets for queries. · 30 min
- HeuristicAnticipatory Directed Curation for LLM WorkspacesProactively reorganize unstructured corpora before inference to efficiently serve diverse future queries. · 30 min
- HeuristicSelf-Scaling Benchmarks for Coding AgentsBenchmarks should create tasks at evaluation time to prevent data leakage and adapt difficulty based on agent performance. · 5 min
- Anti-patternAgents discarding exploration trajectories and final answers onlyDon't discard agent exploration data; it's a rich source for learning and improving strategies. · 20 min
- HeuristicEnhance reinforcement learning with dense rewardsSupplement sparse reinforcement learning feedback with on-policy distillation and process reward models. · 5 min
Questions people ask
- Which AI tools does this work with?
- Any current model. It was seen with in the sources.
- Where did this come from?
- arXiv (arXiv cs.AI). Subvolts collected, normalized and credited it; the link above opens the original.