Financial-Services AI Eval: Model Risk Management for LLMs

The gate that stops AI in finance is validation, not engineering

Outside regulated finance, an AI feature ships when the team thinks it is good enough. Inside a bank, there is a second, harder gate: an independent model risk management (MRM) function has to validate it. Surveys in 2025 show the majority of large financial firms using or piloting AI, a Bank of England and FCA survey found roughly 75% already using it, but foundation models account for a small share of use cases and mostly low-materiality, back-office roles like document processing and research summarization. Customer-facing and decision-making uses (credit, fraud, trading) stay cautious because they trip model-risk requirements. The thing blocking the high-value deployments is not the model’s capability. It is the inability to prove the model is sound to a validator paid to be skeptical.

This is the reframe a financial-services AI team needs: your evaluation suite is not just quality assurance, it is your validation evidence. A robust, documented eval is what gets the model out of MRM review and into production. A weak one is what gets the project shelved after six months of work.

SR 11-7 in one paragraph, and why LLMs break it

SR 11-7 is the Federal Reserve and OCC supervisory guidance on model risk management. It defines a “model” broadly, any quantitative method that turns input data into estimates, and rests on three pillars: robust development with documentation; ongoing validation including conceptual soundness, monitoring, and outcomes analysis; and governance with independent “effective challenge.”

The friction is structural. SR 11-7 was written for deterministic, interpretable models, a logistic regression where you can inspect every coefficient. LLMs are non-deterministic, opaque, and have an effectively unbounded output space. As Deloitte’s analysis of MRM for generative AI notes, “conceptual soundness” and “outcomes analysis” mean something different when you cannot read the model’s logic and the same input can yield different outputs. The old validation playbook does not translate, but the obligations still apply. Evaluation is how you discharge them for systems you cannot inspect. (Even as the agencies modernized this guidance into a more principles-based framework, the underlying expectations, inventory, validation, monitoring, challenge, carry straight over to AI.)

Mapping the three pillars to evals

Conceptual soundness → behavioral and capability evals

You cannot read an LLM’s weights to argue it is fit for purpose, so you demonstrate soundness behaviorally: a golden dataset that proves the system does what it claims across the range of inputs it will see, including the adversarial and edge cases. For an LLM you replace “inspect the math” with “measure the behavior comprehensively.” The validator’s question, “how do you know this works?”, is answered by your eval coverage and results.

Ongoing monitoring + outcomes analysis → production evaluation

SR 11-7 demands that model performance be monitored over time and outcomes compared against expectations. For LLMs that is online evaluation: sample live outputs, score them for accuracy, groundedness, and policy adherence, and track the metrics as a time series. Because hosted models drift silently, this is not optional in finance, it is the only way to satisfy the ongoing-monitoring pillar for a system whose internals can change underneath you.

Effective challenge → independent, reproducible eval

“Effective challenge” requires that someone independent of the developers can critically review the model. That demands your evaluation be reproducible and documented: versioned datasets, versioned prompts, recorded metrics, dated runs. A validator must be able to re-run your eval and get your numbers. An eval that lives in a notebook on one engineer’s laptop cannot survive effective challenge.

# Validation-grade eval: reproducible, documented, independently re-runnable
def validation_eval(system, dataset, run_id):
    record = {
        'run_id':       run_id,
        'date':         now(),
        'model':        system.model_version,      # pinned, for reproducibility
        'prompt_hash':  sha256(system.prompt),     # exact prompt under test
        'dataset_hash': dataset.version_hash,      # exact golden set
        'seed':         system.seed,
    }
    scores = {'accuracy': [], 'grounded': [], 'policy': [], 'refusal_ok': []}
    for case in dataset:
        out = system(case.input)
        scores['accuracy'].append(grade(out, case.gold))
        scores['grounded'].append(is_grounded(out, case.sources))
        scores['policy'].append(matches_regulatory_policy(out))   # fair-lending, disclosures
        scores['refusal_ok'].append(refuses_when_required(out, case))
    record['metrics'] = {k: round(mean(v), 4) for k, v in scores.items()}
    archive_immutable(record)        # the validator re-runs this and must reproduce it
    return record

The finance-specific failure modes your eval must target

A generic accuracy eval misses the failures that get a financial model pulled. Build the golden set around them:

• Fair-lending / disparate impact. If the model touches credit decisions, ECOA and fair-lending rules apply. Compute disparate-impact metrics across protected classes as a release gate, a biased credit model is a regulatory and reputational catastrophe.
• Numerical and calculation errors. Finance is unforgiving about numbers. An LLM that confidently miscomputes a payment, a rate, or a balance is worse than useless. Test arithmetic and figure-citation explicitly.
• Hallucinated terms, products, or disclosures. A bot inventing an account fee, a rate, or a regulatory disclosure is making a statement the institution is bound by, the same liability logic that lost the Air Canada chatbot case.
• Required disclosures and suitability. Investment and lending contexts carry mandatory disclosures and suitability standards. The model omitting them is a compliance violation, not a stylistic miss.
• Appropriate refusal. The model must decline to give regulated advice (tax, legal, individualized investment recommendations) when it is not authorized to. Reward correct refusal.

# Fair-lending gate for any AI touching credit decisions (ECOA / four-fifths)
def fair_lending_gate(decisions_by_group):
    # decisions_by_group: {protected_group: [approved: bool, ...]}
    rates = {g: sum(d)/len(d) for g, d in decisions_by_group.items()}
    best  = max(rates.values())
    flags = {g: rates[g]/best for g in rates if best > 0 and rates[g]/best < 0.80}
    assert not flags, f"FAIR-LENDING RISK: groups below 0.80 selection ratio: {flags}"
    return rates

The single most common exam finding: the model nobody inventoried

Before validation even enters the picture, there is a more basic failure that examiners keep finding: models in production that the bank’s own model inventory does not list. The pattern is familiar, a team stands up an LLM to draft credit memos or triage AML alerts or answer customer questions, treats it as “just a tool” rather than a model, and never registers it with MRM. Then an exam finds an unmanaged, unvalidated model making or shaping regulated decisions, and the finding is severe precisely because it reveals a governance blind spot, not just a quality issue.

This is the financial-services version of shadow AI, and it is an evaluation problem at root. The discipline that prevents it is the same one that satisfies SR 11-7: if a system’s output influences a decision, it is a model, it goes in the inventory, and it gets an eval-backed validation file. The cheapest way to avoid the worst exam finding is to treat every LLM that touches a regulated workflow as in scope for evaluation from day one, not to discover at exam time that it was never governed at all.

The bottom line

In financial services the binding constraint on AI is validation, not capability, and SR 11-7’s expectations, written for transparent models, still govern opaque LLMs by analogy. The way to satisfy its three pillars is a disciplined evaluation practice: behavioral golden-set evals for conceptual soundness, production monitoring for ongoing outcomes analysis, and reproducible, documented runs for effective challenge. Target the finance-specific failure modes, fair lending, calculation errors, hallucinated terms, missing disclosures, and appropriate refusal, and archive every run. Treat your eval suite as your model-validation evidence, and your AI feature ships instead of stalling in MRM review.