Property-based testing
Testing code by checking if general rules about its behavior always hold true, rather than using specific examples.
Property-based testing is a way to check if your code works correctly. Instead of writing many specific examples of inputs and outputs, you define general rules or properties that your code should always follow.
For example, if you have a function that sorts a list, a property might be that the sorted list is always the same length as the original list, and that every element in the sorted list is less than or equal to the next element. The testing tool then generates many random inputs to see if these properties hold true. This helps find bugs that might be missed with just a few examples.
Property-based testing is a software testing methodology where test cases are generated automatically based on a set of properties or invariants that the code should satisfy. Instead of designing specific input-output pairs, developers define abstract properties that describe the expected behavior of the code. A test runner then generates numerous random inputs and checks if these properties hold for all of them. This approach is effective at uncovering edge cases and subtle bugs that might be missed by traditional example-based testing.
Tradeoffs include a steeper learning curve for defining properties and potential for generating trivial test cases if properties are not well-defined. However, it can significantly increase test coverage and reduce the manual effort of writing extensive test suites.
Frontier AI labs are increasingly using AI agents to assist in software development tasks, including testing. Property-based testing is a powerful technique for finding bugs, and integrating it with AI agents allows for more automated and comprehensive code verification.
- Finding bugs with Claude and property-based testingAnthropic · Anthropic Research ·
- Anthropic Claude
- Anthropic's AI agent has been used to find bugs in Python packages by applying property-based testing techniques.
- Example-based testing
- This method uses specific, hand-written input-output pair
- Fuzzing
- While both generate random inputs, fuzzing primarily aims to find crash
Property-based testing was popularized by the QuickCheck library, originally developed for the Haskell programming language by Koen Claessen and John Hughes. Source ↗
You can try property-based testing by using a library like Hypothesis in Python. Define a simple function, like one that adds two numbers, and then write a property that states the result should always be greater than or equal to either input. Run the tests to see how it generates various inputs to check your property.
The Playbook: reusable how-tos, prompts, configs and tips pulled out of the news.
- PromptRole-play a student struggling with math for AI tutor simulationSimulate a student's math struggles and uncertainty for AI tutor evaluation. · 5 min
- TemplateReusable workflow templates for healthcare teamsTeams can create and share templates for common workflows like documentation or prior authorizations. · 15 min
- TemplateTemplate for building an AI tool tailored to your workA structured approach to defining and building a custom AI tool for specific business needs. · 15 min
- TemplateEvals CSV prompt list for agent skillsA CSV file template for listing prompts used in evaluating agent skills. · 20 min
- HeuristicDouble coding agent task completion with docs linksProviding coding agents with a documentation link significantly increases their task completion rate. · 2 min
Questions people ask
- What is a property in property-based testing?
- A property is a general rule or invariant that your code should always satisfy, regardless of the specific inputs it receives.
- How does property-based testing find bugs?
- It works by generating many random inputs and checking if the defined properties hold true for all of them. If a property fails for any input, a bug is detected.
First seen on Subvolts Jun 17, 2026. Updated Sep 14, 2026. Written by an AI model from the items above and reviewed against them weekly; report an error.