The EU AI Act Wants Evidence. Your Evals Are That Evidence.

The deadline that turns “we should test this” into law

The EU AI Act entered into force on 1 August 2024 and phases in over years. The dates that matter for anyone shipping a serious AI feature: prohibitions and AI-literacy duties applied from 2 February 2025, general-purpose AI model obligations from 2 August 2025, and the big one, most high-risk system requirements from 2 August 2026, with a longer runway to 2027 for certain embedded systems. Article 113 lays out the staggered application dates.

The penalty structure under Article 99 is tiered and deliberately large enough to change board behavior: up to €35 million or 7% of total worldwide annual turnover (whichever is higher) for prohibited practices; up to €15 million or 3% for breaching obligations like the high-risk requirements; and up to €7.5 million or 1% for supplying incorrect or misleading information to authorities. As the European Commission’s own framework guidance makes clear, the practical work, classifying systems, building technical documentation, standing up post-market monitoring, needs to be well underway now, not in mid-2026.

What “high-risk” means for you

You are in scope if your AI system falls into the Annex III high-risk categories, which include AI used in employment and worker management, access to essential private and public services (including creditworthiness and credit scoring), education, critical infrastructure, law enforcement, and more. A customer-service chatbot may be low-risk; an AI that scores loan applications, ranks job candidates, or triages benefits eligibility is squarely high-risk. The classification is the first thing you must document, because everything else flows from it.

For high-risk systems, two articles do most of the technical heavy lifting, and both are fundamentally about measurement.

Article 9: a continuous risk-management system

Article 9 requires a risk-management system that runs across the entire lifecycle, identify foreseeable risks, estimate and evaluate them, adopt mitigations, and test to confirm the mitigations work and the residual risk is acceptable. The word the engineering team should hear is “continuous”: this is not a launch gate, it is an ongoing obligation that must be revisited as the system and its environment change.

Article 15: accuracy, robustness, and cybersecurity

Article 15 requires high-risk systems to achieve appropriate levels of accuracy, robustness, and cybersecurity, to perform consistently across their lifecycle, and, critically, to declare their accuracy metrics and remain resilient against errors, faults, and adversarial manipulation. “Declare your accuracy metrics” is a sentence written by a regulator that reads exactly like a sentence written by an ML engineer. You cannot declare a metric you never computed.

Why “we tested it” is not the same as “we can prove it”

Many teams will read the high-risk requirements and conclude they already comply, because they do test their systems. That confidence usually evaporates on contact with an auditor, because the Act does not ask whether you tested, it asks for the evidence, in a form an independent party can examine and trust. Ad-hoc testing leaves no durable artifact: a few engineers tried some prompts, it seemed fine, and the knowledge lives in their heads and a closed Slack thread. That is testing without evidence, and under a documentation regime it is indistinguishable from not testing at all.

The distinction that matters is between a process and an artifact. A regulator cannot inspect your good intentions or your team’s diligence; they can only inspect what you wrote down. So the compliance question reduces to a concrete engineering one: does every release of your high-risk system produce a dated, versioned record of its measured accuracy, robustness, and per-group performance, archived where someone other than the author can find and reproduce it? If yes, you have conformity evidence. If the answer is “we’re pretty sure it’s good, ” you have exposure scaled to your global turnover.

Your eval suite is the conformity evidence

Here is the mapping that should reframe how your team budgets for this. Each compliance obligation has a direct evaluation counterpart you probably already know how to build:

• Article 15 accuracy → a versioned golden dataset with graders, producing a declared accuracy/quality number per release.
• Article 15 robustness → adversarial and perturbation test suites (paraphrases, edge cases, noisy inputs, prompt-injection probes) with pass-rate thresholds.
• Article 9 risk management → a regression suite plus continuous production monitoring that re-evaluates risk as the system and inputs drift.
• Non-discrimination (fundamental-rights duties) → disparate-impact and fairness metrics computed per release.
• Post-market monitoring (Article 72) → online evaluation that samples and scores live traffic, with dated alerts.

The compliance team needs a binder of evidence. The engineering team needs to know its model still works. These are the same artifact viewed from two desks. The conformity-assessment burden becomes dramatically lighter when the technical file is auto-generated from eval runs you were going to do anyway.

# A conformity-evidence eval run: emits the numbers Article 15 asks you to "declare"
def conformity_eval(system, golden_set, adversarial_set, slices):
    report = {'system': system.id, 'version': system.version, 'date': today()}

    # Article 15: accuracy, declared metric per release
    acc = mean(grade(system(c.input), c.reference) for c in golden_set)
    report['accuracy'] = round(acc, 4)

    # Article 15: robustness, performance under perturbation / adversarial input
    rob = mean(grade(system(c.input), c.reference) for c in adversarial_set)
    report['robustness'] = round(rob, 4)
    report['robustness_drop'] = round(acc - rob, 4)   # consistency of performance

    # Fundamental rights: per-group performance (no group left behind)
    report['slice_accuracy'] = {
        name: round(mean(grade(system(c.input), c.reference) for c in cases), 4)
        for name, cases in slices.items()
    }

    # Acceptance gate = your declared, defensible thresholds
    report['conformant'] = (acc >= 0.90 and rob >= 0.85
                            and min(report['slice_accuracy'].values()) >= 0.85)
    return report   # archive every run -> this IS your technical documentation

Archive each run with its model version, dataset version, and date. That immutable history is precisely the “logging” and “record-keeping” the Act expects, and it lets you answer the auditor’s real question: on the day this system made this decision, what evidence did you have that it met your declared accuracy and robustness levels?

General-purpose AI: the obligation flows downhill

If you build on a hosted foundation model, the GPAI obligations that applied from August 2025 matter to you indirectly. Providers of general-purpose models must publish technical documentation, training-data summaries, and, for models with systemic risk, conduct model evaluations and adversarial testing. But that does not discharge your duty for the high-risk system you build on top. You still must evaluate the integrated system in your context, on your data, for your use case. A provider’s model card is upstream evidence; it is not your conformity assessment.

Build the technical file now, not in July 2026

The conformity assessment for high-risk systems requires a technical documentation file: system description, intended purpose, risk-management records, data governance, accuracy/robustness metrics and the methodology behind them, human-oversight measures, and post-market monitoring plans. Most of that is prose your compliance and product teams write. The hard, defensible core, the numbers, is what your eval pipeline produces.

# Post-market monitoring (Article 72): score live traffic, keep dated evidence
def post_market_monitor(sample):
    for case in sample:                       # representative production sample
        out = system(case.input)
        record = {
            'date': now(), 'version': system.version,
            'grounded':  is_grounded(out, case.sources),
            'in_policy': matches_policy(out),
            'group':     case.demographic_group,   # for ongoing fairness tracking
        }
        log_immutable(record)                 # auditable, append-only
        if not record['grounded'] or not record['in_policy']:
            open_incident(case, out)          # closes the Article 9 risk loop

    # Periodically recompute declared metrics from the live sample;
    # if accuracy or any group's performance drifts below the declared floor,
    # that is a reportable change to your risk-management system.

Start now. Inventory your AI systems, classify each against Annex III, and for every high-risk one stand up the eval suite that emits accuracy, robustness, and per-group metrics on every release plus a production-monitoring loop. By August 2026 you want the technical file to be a query against eval history, not a fire drill.

The bottom line

The EU AI Act is a documentation-and-evidence regime backed by fines scaled to global turnover, and the high-risk requirements bite from August 2026. Articles 9 and 15 demand exactly what a mature evaluation practice produces: declared accuracy, tested robustness, continuous risk management, and per-group fairness. Treat compliance not as a separate legal workstream but as the natural output of an eval pipeline you run on every release and against live traffic. Build the evals, archive the runs, and your conformity evidence writes itself.