When Your RAG Cites a Source That Doesn't Say That

The citation that points to the wrong thing

RAG earns trust with citations. The answer arrives with little superscripts linking each claim to a retrieved document, and that visible provenance is exactly what makes users, and auditors, believe it. Which is why a wrong citation is so much more dangerous than a missing one. A claim with no citation reads as opinion and gets scrutinized. A claim with a confident citation to source [3] reads as verified fact, and nobody opens source [3] to check that it actually says what the answer claims.

They should, because often it doesn’t. A substantial portion of generated text in RAG systems lacks proper support from its cited references, and industry studies report citation accuracy of only about 74% for popular generative search engines (Maxim, RAG Evaluation: A Complete Guide for 2025). One in four citations is a broken promise. In a casual search product that’s annoying. In a system that summarizes case law or drug interactions, it’s the kind of error that ends up in a complaint.

Correctness is not faithfulness

The subtle trap, and the one most teams miss, is that an answer can be true and still be badly cited. The 2025 paper “Correctness is not Faithfulness in Retrieval Augmented Generation Attributions” draws this line precisely: a model can produce a factually correct claim while attributing it to a passage that doesn’t actually contain it. The model knew the answer from its parametric memory, generated it, and then attached whatever retrieved chunk looked topically related as a citation. The fact is right; the attribution is fabricated.

This decouples two things teams routinely conflate. Correctness asks “is the claim true?” Faithfulness/attribution asks “does the cited source actually support this claim?” A system can score well on the first and badly on the second, and in a regulated context the second is what matters, because the whole legal defense of a RAG answer is “we didn’t make it up, here’s the source.” If the source doesn’t say it, that defense evaporates even when the answer happens to be right.

Why multi-hop makes it worse

Single-hop citation is hard; multi-hop is brutal. When an agentic pipeline chains several retrieval steps, retrieve, reason, retrieve again, attribution errors compound across hops. Each step can introduce a topically-plausible but unsupported source, and the final answer stitches together claims whose citations were never individually verified. Attribution requires citations to map to specific retrieved passages with verified correct mapping, and multi-hop evaluation has to track factual coverage across every hop, not just the last one (Label Your Data, RAG Evaluation). The more reasoning steps you add to get a smarter answer, the more places a citation can silently detach from its claim.

Evaluate citations at the claim level

You cannot evaluate citation accuracy by reading the answer. You have to decompose the answer into atomic claims and, for each one, verify that the span it cites actually entails it. This is an NLI (natural language inference) problem at heart: does the cited passage entail the claim, or merely sit near it topically?

# Citation accuracy = per-claim entailment against the CITED span (not all context)
def citation_accuracy(answer, retrieved):
    claims = decompose_into_claims(answer)          # atomic, individually checkable
    results = []
    for claim in claims:
        cited = claim.cited_source_ids              # what the answer points to
        if not cited:
            results.append(('uncited', claim))      # a claim with no support at all
            continue
        cited_text = concat(retrieved[i].text for i in cited)
        verdict = nli_entailment(premise=cited_text, hypothesis=claim.text)
        # The trap: claim may be TRUE but NOT entailed by the cited span
        results.append((verdict, claim))            # 'entailed' | 'neutral' | 'contradicted'

    supported = sum(1 for v, _ in results if v == 'entailed')
    return {
        'citation_accuracy': supported / max(len(claims), 1),
        'misattributed': [c for v, c in results if v in ('neutral', 'contradicted')],
        'uncited':       [c for v, c in results if v == 'uncited'],
    }

The distinction baked into that function is the whole point: a claim that’s neutral against its cited span is a misattribution, the source neither supports nor refutes it, so the citation is decorative. The CiteCheck line of work formalizes exactly this task of detecting citation faithfulness, and CiteFix shows it can even be corrected in post-processing, re-mapping each claim to the passage that actually supports it.

Gate on attribution, then repair

For a fact-bearing system, citation accuracy belongs in the release gate alongside groundedness, and ideally in a post-processing repair step before the answer ever reaches the user.

# Pre-response repair: fix attributions, refuse if a claim has no real support
def enforce_citations(answer, retrieved, min_accuracy=0.95):
    report = citation_accuracy(answer, retrieved)

    # Try to re-map misattributed claims to a span that actually entails them
    for claim in report['misattributed']:
        better = best_entailing_span(claim, retrieved)
        if better: claim.recite(better)             # CiteFix-style correction
        else:      claim.mark_unsupported()         # no source says this

    report = citation_accuracy(answer, retrieved)   # recompute after repair
    # In regulated domains, an unsupportable claim should be removed or refused
    if report['citation_accuracy'] < min_accuracy or report['uncited']:
        return refuse_or_strip_unsupported(answer, report)
    return answer

The bottom line

Citations are the trust mechanism of RAG, and roughly a quarter of them are broken promises. The error that hurts most isn’t the false claim, it’s the true claim with a fabricated attribution, because correctness is not faithfulness and the cited source is the entire legal defense of the answer. Multi-hop pipelines compound the problem across every retrieval step. Evaluate citation accuracy at the claim level with entailment against the specific cited span, not the whole context; repair misattributions in post-processing; and gate releases so an answer that can’t cite a real source either refuses or removes the claim. Anything fact-bearing, and certainly anything regulated, cannot ship without it.