Cross-Jurisdiction Substitution
AI models mixing up facts or rules from one place with those from another.
Cross-jurisdiction substitution is when an AI model incorrectly applies information or rules from one geographic area, legal system, or policy domain to another.
For example, if you ask an AI about the speed limit for a specific road in California, but it gives you the speed limit from Texas instead, that's cross-jurisdiction substitution. This can happen because the AI has learned from a vast amount of data that includes information from many different places, and it sometimes struggles to keep them separate.
Cross-jurisdiction substitution occurs when a large language model, trained on diverse datasets, fails to correctly attribute information or apply context specific to a particular jurisdiction (e.g., a state, country, or legal system). This can manifest as recalling incorrect laws, policies, or factual details relevant to a specific region. The phenomenon highlights challenges in grounding model outputs to precise contextual parameters, especially when dealing with nuanced, geographically or legally distinct information.
This issue is particularly relevant in applications requiring high factual accuracy and adherence to specific regulatory frameworks. It suggests limitations in the model's ability to perform fine-grained contextual retrieval and may necessitate specialized fine-tuning or retrieval-augmented generation (RAG) techniques to ensure jurisdictional fidelity.
This term is relevant as AI models are increasingly used for tasks involving specific geographic or legal contexts. Recent research highlights how large language models can incorrectly apply information from one jurisdiction to another, leading to factual errors in policy recall. This is a growing concern as AI systems are deployed in areas requiring precise, localized knowledge.
- Right Number, Wrong State? Measuring Cross-Jurisdiction Substitution in LLM Recall of State PolicyAnthropic · arXiv cs.AI ·
- Anthropic Claude
- Anthropic's research specifically investigates how their models exhibit cross-jurisdiction substitution when recalling US state policy information.
- Hallucination
- Hallucination is when an AI generates false information, whereas
- Data Contamination
- Data contamination occurs when training data inadvertently i
- Out-of-Distribution Generalization
- Out-of-distribution generalization refers to
The term describes a specific failure mode observed in large language models, particularly as they are applied to tasks requiring precise knowledge of different legal and policy domains. It is a concept emerging from empirical evaluation of LLM capabilities.
Ask an AI model to describe a specific law or regulation from one US state, then immediately ask it to describe the same law or regulation from a different US state. Compare the answers for any inconsistencies or incorrect details that might indicate it's confusing the two. For example, ask about California's specific vehicle emissions standards, then immediately ask about Texas's specific vehicle emissions standards.
The Playbook: reusable how-tos, prompts, configs and tips pulled out of the news.
- Eval methodCheck LLM state policy recall for cross-jurisdiction substitutionVerify if an LLM provides another state's policy value when asked about a specific state. · 60 min
- Troubleshooting fixCorrect LLM state policy errors by checking other states' valuesWhen an LLM gives a wrong state policy answer, check if it matches another state's correct value. · 10 min
- PromptRole-play a student struggling with math for AI tutor simulationSimulate a student's math struggles and uncertainty for AI tutor evaluation. · 5 min
- TemplateReusable workflow templates for healthcare teamsTeams can create and share templates for common workflows like documentation or prior authorizations. · 15 min
- TemplateTemplate for building an AI tool tailored to your workA structured approach to defining and building a custom AI tool for specific business needs. · 15 min
Questions people ask
- Why do AI models substitute information from different places?
- AI models learn from vast datasets that often mix information from various regions or contexts. They can sometimes struggle to perfectly distinguish and apply the correct context, leading to substitution errors.
- Is cross-jurisdiction substitution a type of hallucination?
- It's related but distinct. Hallucination is making up false information. Cross-jurisdiction substitution is applying correct information, but from the wrong place or context.
First seen on Subvolts Oct 7, 2026. Updated Oct 14, 2026. Written by an AI model from the items above and reviewed against them weekly; report an error.