OpenAI OfficialWebLearn & DoResources
All NewsTradeVideosVoicesCommunityResearch
OpenAI · Web · Research Article

CamoDocs: A Poisoning Attack Against Retrieval-Augmented Language Models Using Camouflaged Documents

Source: arXiv cs.AI By Jaewon Jung, Haizhong Zheng, Hongsun Jang, Jaeyong Song
Image: arXiv cs.AI

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.…

Opening of the original on arXiv cs.AI

Researchers developed CamoDocs, a novel attack that poisons retrieval-augmented language models like OpenAI's ChatGPT by embedding malicious data within seemingly innocuous documents. This method bypasses standard defenses by camouflaging harmful content, potentially leading to model manipulation and the generation of incorrect or biased outputs. The attack highlights a new vulnerability in how LLMs process external information, posing a risk to the reliability of AI systems that rely on retrieved data.

Why it matters

Why it matters: This research introduces a sophisticated poisoning technique targeting retrieval-augmented models, a common architecture for systems like ChatGPT and Google Gemini. CamoDocs exploits the retrieval process itself, making it harder to detect than traditional data poisoning. This affects developers building LLM applications and users who rely on their accuracy. Future work should focus on developing robust defenses against such camouflaged attacks, potentially involving more advanced content verification or adversarial training methods for retrieval components.

Read this on arXiv cs.AI
Opens in a new tab. Subvolts summarizes and links; the full piece belongs to arXiv cs.AI.
Post to XLinkedIn
Where the other five stand

Related: Google: Black Box: The Chatbots | Spirals | Ep 1 – podcast · Anthropic: Black Box: The Chatbots | Happy Accident | Ep 3 – podcast · Microsoft: How Kier Group’s Louisa Finlay is using Copilot to drive safety and productivity in the construction industry · Meta: Get the full story behind the light · xAI: Ajeya Cotra – "This might be the clearest warning shot we ever get"

Hype check
3/5Notable

Rated middle: a real update, not a headline event.

Who's talking about it
Prior coverage our earlier items on the same thing
Published
Source
arXiv cs.AI (arxiv.org)
Author
Jaewon Jung, Haizhong Zheng, Hongsun Jang, Jaeyong Song
Company
OpenAI · Web · Research
Products
ChatGPT
Summary by
Subvolts, using an AI model (how we work). Spotted a mistake? Tell us.
#Research#Safety

More on OpenAI

Questions people ask

What is the CamoDocs attack?
CamoDocs is a poisoning attack designed to compromise retrieval-augmented language models. It works by hiding malicious data within ordinary-looking documents that the model retrieves for information.
How does CamoDocs differ from other attacks?
This attack camouflages harmful content within documents, making it harder for standard defenses to detect and prevent the model from processing the poisoned data.

More from arXiv cs.AI 12 more

Everything from arXiv cs.AI →

Page generated Sep 3, 2026. Summaries are Subvolts' own; the story belongs to arXiv cs.AI.