Test Your AI for Bias Before a Regulator (or Reporter) Does

The defense that AI is “just a tool” is collapsing

For years the legal theory protecting AI vendors and the employers who buy them was simple: the software is a neutral instrument, and only the human decision-maker can discriminate. That theory is failing in court. In Mobley v. Workday, Judge Rita Lin of the Northern District of California held that an AI screening vendor could be directly liable for employment discrimination under an “agent” theory, the screening tool was performing a function the employer would otherwise do itself, so it stepped into the employer’s shoes for anti-discrimination purposes.

In May 2025 the same court went further and granted preliminary certification of a nationwide collective action under the Age Discrimination in Employment Act, covering applicants age 40 and older denied employment recommendations through Workday’s platform since September 2020. The plaintiff class could number in the millions, and the opt-in window ran into 2026. The dollar exposure of an ADEA collective at that scale dwarfs any audit budget you would ever spend to prevent it.

What the regulators already require

New York City’s Local Law 144 has been enforced by the Department of Consumer and Worker Protection since July 2023. It requires any employer using an automated employment decision tool to commission an independent bias audit, publish the results, and notify candidates. The audit is not a vibe check, it must compute and publish selection-rate impact ratios across race, ethnicity, and sex categories.

The math comes straight from the EEOC’s decades-old adverse-impact framework: the four-fifths (80%) rule. If the selection rate for any protected group is less than 80% of the rate for the most-selected group, that gap is a red flag for adverse impact. Local Law 144 takes that established standard and makes the calculation mandatory and public. Deloitte’s analysis frames the audit as a recurring assurance obligation, not a one-time gate.

Enforcement is sharpening. In December 2025 the New York State Comptroller released an audit concluding that DCWP’s enforcement of Local Law 144 had been “ineffective, ” citing misrouted complaints and superficial review of posted audits, and DCWP committed to fixing every gap. Translation: the agency is moving from reactive to proactive, and the employers who have been quietly skipping their audits are about to become targets. Local Law 144 is also the template; Colorado, Illinois, and the EU AI Act all impose overlapping algorithmic-fairness obligations on hiring and lending systems.

Bias is a measurable output property

Here is the engineering reframe that turns a legal problem into a testable one. Bias in an AI system is not a mood; it is a statistical relationship between a protected attribute and the model’s outputs. You can measure it the same way you measure latency or accuracy, with a dataset, a metric, and a threshold.

The three metrics that map directly to the legal standards:

• Disparate impact ratio (four-fifths rule). For each protected group, the selection (positive-outcome) rate divided by the rate of the most-selected group. Below 0.80 is the regulatory tripwire.
• Equal opportunity / true-positive-rate gap. Among genuinely qualified candidates, does the model recommend each group at the same rate? A gap here means the model is worse at recognizing merit in one group.
• Demographic parity difference. The raw difference in positive rates across groups, useful as a coarse early-warning signal before you have ground-truth labels.

None of these requires reading the model’s mind. They require holding out a labeled evaluation set, attaching demographic attributes (or proxy-inferred attributes where direct collection is restricted), and computing rates by group.

# Disparate-impact bias audit you can run in CI, before every model ship
# groups: dict[str, list[bool]]  -> per-group list of positive outcomes (selected = True)
def four_fifths_audit(groups):
    rates = {g: sum(outcomes) / len(outcomes) for g, outcomes in groups.items()}
    best  = max(rates.values())
    report = {}
    for g, rate in rates.items():
        impact_ratio = rate / best if best > 0 else 0.0
        report[g] = {
            'selection_rate': round(rate, 4),
            'impact_ratio':   round(impact_ratio, 4),
            'flag':           impact_ratio < 0.80,   # EEOC four-fifths tripwire
        }
    failed = [g for g, r in report.items() if r['flag']]
    if failed:
        raise AssertionError(f"DISPARATE IMPACT: groups below 0.80 ratio: {failed}\n{report}")
    return report

Wire that into the same pipeline that ships your model. A model that fails the four-fifths check does not deploy, exactly like a failing unit test. The artifact it produces is also the artifact Local Law 144 requires you to publish, so the compliance deliverable and the engineering gate become the same object.

Intersectional and proxy bias: where naive audits miss

The cases that generate headlines are usually not the ones a single-axis audit catches. A model can pass the four-fifths rule on race and on sex independently while badly failing for, say, women over 50, the intersection. The famous 2018 facial-analysis findings that error rates were near-zero for lighter-skinned men but exceeded a third for darker-skinned women were precisely an intersectional failure invisible to one-variable testing.

# Intersectional sweep + proxy leakage check
import itertools

def intersectional_audit(records, attrs=('race', 'sex', 'age_band')):
    # records: list of {'outcome': bool, 'race':..., 'sex':..., 'age_band':...}
    for combo in itertools.combinations(attrs, 2):           # all attribute pairs
        groups = {}
        for r in records:
            key = tuple(r[a] for a in combo)
            groups.setdefault(key, []).append(r['outcome'])
        groups = {k: v for k, v in groups.items() if len(v) >= 30}  # ignore tiny cells
        rates  = {k: sum(v)/len(v) for k, v in groups.items()}
        best   = max(rates.values())
        for k, rate in rates.items():
            if best > 0 and rate / best < 0.80:
                print(f"INTERSECTIONAL FLAG {combo}={k}: ratio {rate/best:.2f}")

# Proxy check: can a simple model recover a protected attribute from the "neutral" features?
# If AUC is high, your feature set leaks the attribute and your audit must account for it.

From one-time audit to continuous monitoring

A bias audit dated last March tells you nothing about the model running today. Hiring and lending models drift as the applicant pool shifts, as upstream data sources change, and, for systems built on hosted LLMs, as the provider silently updates the base model. A model that was fair at launch can develop disparate impact months later with no code change on your side.

That is why the credible posture is continuous: run the disparate-impact and equal-opportunity metrics on a schedule against live scoring data, alert when any impact ratio crosses 0.80, and keep a dated history so you can prove the model was fair on the day each applicant was scored. When discovery comes, “we monitor disparate impact nightly and here is the time-series” is a categorically stronger position than a single PDF audit from a year ago.

It is also cheaper than the alternative. The cost of a continuous bias-monitoring harness is a rounding error against the cost of an ADEA collective, a Local Law 144 penalty, or the reputational hit of a reporter publishing your impact ratios before you measured them yourself.

The bottom line

Courts are holding both employers and AI vendors liable for discriminatory outputs, regulators already require published disparate-impact audits, and enforcement is intensifying. Bias is not a values question you can answer with a policy statement, it is a measurable property of your model’s outputs. Compute the four-fifths impact ratio, equal-opportunity gaps, and intersectional slices; gate deploys on them; monitor them continuously against live traffic; and keep dated evidence. Measure your AI for bias on your own terms, or a regulator or a reporter will do it on theirs.