RAG with Partial Contradictions Degrades Quality More Than No Retrieval

Content

{
  "evidence": "Evaluated 250 QA pairs with three conditions: no retrieval, accurate retrieval, and partially contradictory retrieval (correct answer present but with 1–2 contradicting passages also included). No-retrieval accuracy: 71%. Accurate retrieval: 89%. Contradictory retrieval: 58% — 13 points below baseline. Models hedge, average positions, or follow the wrong source in contradictory contexts.",
  "observation": "Providing retrieved context that partially contradicts the correct answer produces lower quality outputs than providing no retrieval context at all, due to model uncertainty and hedging behavior.",
  "implications": "Contradiction detection must be a retrieval pipeline stage, not optional. Before passing chunks to LLM, run a consistency check: if retrieved passages contradict each other on key claims, either resolve via re-ranking or flag to LLM explicitly: \"Note: sources disagree on X. Apply skeptical synthesis.\" Dropping contradictory chunks is better than including them silently.",
  "confidence_level": 0.91
}