Elo
Also written: Elo-based tournament
Elo is a rating system that estimates the skill level of players in competitive games, now used to rank AI models.
Elo is a way to rank players in games like chess. It gives each player a score. When two players compete, the winner gains points and the loser loses points. The number of points exchanged depends on how big the difference in their scores was before the game. If a lower-ranked player beats a higher-ranked player, they gain more points than if they had beaten someone with a similar score.
AI labs are starting to use Elo to compare their models. Instead of just looking at benchmark scores, they can have models "play" against each other. The winner is the model that performs better on a given task. This gives a more dynamic ranking of AI capabilities.
The Elo rating system, originally developed by Arpad Elo for chess, is a method for calculating the relative skill levels of players in zero-sum games. It works by adjusting a player's rating based on the outcome of a match and the difference in ratings between the two players. A win against a higher-rated opponent yields more points than a win against a lower-rated opponent. This system is increasingly adopted by AI research labs to create dynamic leaderboards for large language models, where models are pitted against each other in head-to-head evaluations.
The Elo rating system is frequently used to benchmark AI models, particularly in head-to-head comparisons. Recent news shows Anthropic's Claude models climbing the Artificial Analysis Intelligence Index, which uses Elo to rank performance, highlighting its importance in competitive AI development.
- Anthropic has released Claude Haiku 5.5Anthropic · Artificial Analysis ·
- Claude Sonnet 5.5 reaches #2 on the Artificial Analysis Intelligence IndexAnthropic · Artificial Analysis ·
- Claude Opus 5.5 takes the top spot on the Artificial Analysis Intelligence IndexAnthropic · Artificial Analysis ·
- Benchmarking Grok 4.7xAI · Artificial Analysis ·
- Co-Scientist: A multi-agent AI partner to accelerate researchGoogle · Google DeepMind Blog ·
- Anthropic Claude
- Anthropic's Claude models are frequently ranked using Elo-based systems like the Artificial Analysis Intelligence Index, showing their performance relative to other leading models.
- Google Gemini
- While not explicitly stated in the provided summaries, Google's Gemini models are likely evaluated using Elo or similar comparative metrics in industry benchmarks.
- xAI Grok
- xAI Grok's performance is benchmarked, suggesting the use of comparative ranking systems like Elo to assess its capabilities against other AI models.
- Glicko rating system
- A more complex rating system that also accounts for the un
The Elo rating system was invented by Arpad Elo, a Hungarian-American physics professor, for the game of chess. It was first published in 1962 and adopted by the United States Chess Federation in 1960 and FIDE in 1970. Source ↗
You can't directly use Elo yourself, but you can observe its application. Look for leaderboards from AI labs that use Elo ratings. Compare how different models rank over time as they "compete" against each other in evaluations.
The Playbook: reusable how-tos, prompts, configs and tips pulled out of the news.
- PromptRole-play a student struggling with math for AI tutor simulationSimulate a student's math struggles and uncertainty for AI tutor evaluation. · 5 min
- TemplateReusable workflow templates for healthcare teamsTeams can create and share templates for common workflows like documentation or prior authorizations. · 15 min
- TemplateTemplate for building an AI tool tailored to your workA structured approach to defining and building a custom AI tool for specific business needs. · 15 min
- TemplateEvals CSV prompt list for agent skillsA CSV file template for listing prompts used in evaluating agent skills. · 20 min
- HeuristicDouble coding agent task completion with docs linksProviding coding agents with a documentation link significantly increases their task completion rate. · 2 min
Questions people ask
- How does Elo apply to AI models?
- AI models are made to compete on specific tasks. The model that performs better "wins" the round, and their Elo scores are adjusted accordingly, similar to how chess players' ratings change.
- Is Elo the only way to rank AI models?
- No, Elo is one method. Other methods include standardized benchmarks, human evaluations, and task-specific performance metrics.
First seen on Subvolts May 12, 2026. Updated Oct 13, 2026. Written by an AI model from the items above and reviewed against them weekly; report an error.