Precision
In classification, the proportion of predicted positives that are actually correct. How often the system is right when it flags something as bad.
Precision measures how accurate a system is when it predicts a positive outcome. Imagine a spam filter. If it flags 10 emails as spam, and 8 of them actually are spam, its precision is 80%. The other 2 were legitimate emails incorrectly marked as spam.
This is important because you don't want a system to be overly aggressive and flag too many correct things as wrong. For example, if a content moderation system has high precision, it means that when it flags a piece of content as harmful, it's very likely to actually be harmful. This helps build trust in the system's judgments.
Precision is a metric used in classification tasks, defined as the ratio of true positives (TP) to the sum of true positives and false positives (TP + FP). It answers the question: of all the instances the model predicted as positive, how many were actually positive? High precision means a low rate of false positives. This is crucial in applications where the cost of a false positive is high, such as medical diagnosis or spam detection.
Precision is a key metric for evaluating the safety and reliability of AI models, especially those involved in content generation and moderation. As frontier AI labs like OpenAI develop more sophisticated systems, ensuring high precision in identifying and mitigating harmful outputs becomes paramount for user trust and responsible deployment.
- DALL·E 2 pre-training mitigationsOpenAI · OpenAI News ·
- OpenAI ChatGPT
- OpenAI uses precision as a measure of performance and safety in its models, particularly when filtering harmful content or ensuring accurate image generation.
- Accuracy
- Accuracy measures the overall correctness of a model across all predic
- Recall
- Recall measures the proportion of actual positives that were correctly i
Precision is a standard statistical measure originating from the field of information retrieval and classification. It has been a fundamental metric in machine learning evaluation for decades.
Ask a generative AI model like ChatGPT to write a short story about a specific topic. Then, ask it to identify all the sentences that mention a particular keyword. Check how many of the sentences it identifies actually contain the keyword. This demonstrates how the model is precise in its identification.
The Playbook: reusable how-tos, prompts, configs and tips pulled out of the news.
- PromptRole-play a student struggling with math for AI tutor simulationSimulate a student's math struggles and uncertainty for AI tutor evaluation. · 5 min
- TemplateReusable workflow templates for healthcare teamsTeams can create and share templates for common workflows like documentation or prior authorizations. · 15 min
- TemplateTemplate for building an AI tool tailored to your workA structured approach to defining and building a custom AI tool for specific business needs. · 15 min
- TemplateEvals CSV prompt list for agent skillsA CSV file template for listing prompts used in evaluating agent skills. · 20 min
- HeuristicDouble coding agent task completion with docs linksProviding coding agents with a documentation link significantly increases their task completion rate. · 2 min
Questions people ask
- What is a false positive?
- A false positive occurs when a system incorrectly predicts a positive outcome. For example, a spam filter flagging a legitimate email as spam.
- Why is precision important for AI safety?
- High precision in safety systems means that when the AI flags something as dangerous or harmful, it is very likely to be correct. This reduces the chance of wrongly censoring safe content or misidentifying threats.
First seen on Subvolts Jun 28, 2022. Updated Sep 15, 2026. Written by an AI model from the items above and reviewed against them weekly; report an error.