The Summary That Added a Fact: Faithfulness Eval for Summarization

The fabrication you didn’t ask for

Summarization feels like the safe LLM task. You’re not asking the model to know anything, you’re handing it the source and asking it to compress. What could it invent? Quite a lot, it turns out. Abstractive summarizers don’t copy; they paraphrase and synthesize, and in that synthesis they introduce claims the source never made. The model writes a fluent, plausible summary that reads as a faithful condensation and quietly contains a detail you can’t find anywhere in the original document.

The historical rates are sobering. Even strong abstractive models have been found to produce unfaithful content in a meaningful share of outputs, figures around 25-30% of summaries containing at least one factual inconsistency for older abstractive systems, with modern LLMs reducing but not eliminating the problem (A hallucination detection and mitigation framework for faithful text summarization, Scientific Reports 2025). In a low-stakes context (summarizing a blog post) that’s tolerable. In a high-stakes one, a clinician reading an AI summary of a patient record, a lawyer relying on a summarized deposition, an analyst acting on a summarized filing, an added fact is exactly the kind of error that produces harm and liability.

Two kinds of unfaithfulness: intrinsic and extrinsic

The faithfulness literature splits hallucination in summarization into two types, and the distinction is operationally important because they fail differently:

• Intrinsic hallucination, the summary misrepresents or contradicts information that is in the source. Swapping subject and object (“the plaintiff owes the defendant” when the source says the reverse), getting a number wrong, dropping a negation so “not liable” becomes “liable.” The fact is grounded in the source but mangled (Hallucination to truth, AI Review 2025).
• Extrinsic hallucination, the summary adds information not present in the source at all. This is the “added fact.” It is not always false, the model may pull a true fact from its training, but it is unverifiable against the source, which in a legal, medical, or financial summary is exactly the problem: you cannot trace it, so you cannot trust it.

Both are faithfulness failures, but extrinsic hallucination is the one that surprises teams, because it’s the model adding value-looking content that no one requested and no source supports.

Why ROUGE can’t see any of this

The uncomfortable truth is that the metric most summarization systems still report, ROUGE, is structurally incapable of detecting faithfulness failures. ROUGE measures n-gram overlap between the generated summary and a reference summary. It rewards using the same words. A summary can have high ROUGE and contain a fabricated number; a perfectly faithful summary phrased differently from the reference can have low ROUGE. Overlap is not truth, and optimizing for ROUGE optimizes for the wrong thing entirely.

The faithfulness-specific metrics that replaced it correlate far better with human judgments of factual consistency. They fall into three families: entailment-based (SummaC, FactCC) that check whether the source entails each summary sentence; QA-based (QAGS, QuestEval) that generate questions from the summary and verify the answers against the source; and LLM-as-judge faithfulness scoring that decomposes the summary into atomic facts and checks each against the source. Benchmarks like HalluLens (ACL 2025) and Vectara’s grounded-summarization leaderboard, built on 7,700+ articles scored by a dedicated hallucination-evaluation model (HHEM), operationalize exactly this. All three share the insight ROUGE lacks: faithfulness is about whether claims are supported, not whether words match.

Evaluate faithfulness at the atomic-fact level

# Faithfulness = every atomic fact in the summary is supported by the source
def summary_faithfulness(summary, source):
    facts = extract_atomic_facts(summary)        # one checkable claim each
    results = []
    for fact in facts:
        # Entailment of each fact against the SOURCE (not a reference summary)
        verdict = nli_entailment(premise=source, hypothesis=fact)
        results.append((fact, verdict))          # 'entailed'|'neutral'|'contradicted'

    n = max(len(facts), 1)
    return {
        'faithfulness':  sum(v == 'entailed'     for _, v in results) / n,
        # Intrinsic: contradicts the source (mangled fact)
        'intrinsic':     [f for f, v in results if v == 'contradicted'],
        # Extrinsic: not in the source at all (added fact)
        'extrinsic':     [f for f, v in results if v == 'neutral'],
    }

# Gate releases, added facts and contradictions are hard fails in high-stakes use
def faithfulness_gate(model, eval_set, min_faithfulness=0.98):
    scores, intrinsic, extrinsic = [], 0,0
    for case in eval_set:
        r = summary_faithfulness(model(case.source), case.source)
        scores.append(r['faithfulness'])
        intrinsic += len(r['intrinsic'])
        extrinsic += len(r['extrinsic'])

    avg = sum(scores) / len(scores)
    assert avg >= min_faithfulness, f"faithfulness regressed to {avg:.1%}"
    # In legal/medical/financial summaries, zero tolerance for contradictions
    assert intrinsic == 0, f"{intrinsic} contradicted (intrinsic) facts"
    return {'faithfulness': avg, 'intrinsic': intrinsic, 'extrinsic': extrinsic}

Tune the bar to the stakes

Not every summary needs the same faithfulness threshold, and pretending otherwise either over-constrains low-stakes features or under-protects high-stakes ones. For a casual content summary, an occasional benign extrinsic fact may be acceptable. For a clinical, legal, or financial summary, the policy should be strict: zero contradictions (intrinsic), and extrinsic facts flagged or stripped because an unverifiable claim in a regulated document is a defect regardless of whether it happens to be true. The same atomic-fact eval supports both, you just set the threshold and the extrinsic-fact policy per use case, and gate accordingly.

The bottom line

The summarization failure that hurts isn’t the dropped detail, it’s the added one. Abstractive summarizers have historically introduced a factual inconsistency in a quarter to a third of outputs, and an invented fact in a legal, medical, or financial summary is a liability, not a rounding error. ROUGE measures word overlap and is blind to all of it; faithfulness metrics, entailment, QA-based, and atomic-fact LLM judges, measure whether claims are actually supported. Decompose every summary into atomic facts, check each against the source, separate intrinsic (contradicted) from extrinsic (added) hallucinations, and gate with thresholds tuned to the stakes. The summary that added a fact is the one that turns a helpful feature into a false statement your company made.