The go/no-go checklist
Step 11Live-room allocation: 3:00
A check only protects you from the failure it was designed to detect. Illustration: Neil Webb.
Everything in this workshop leads to one operational question:
What has to be true before you are willing to ship?
Not just whether the model performs well.
Whether the measurement feeding it is appropriate. Whether the metric can see the consequence you care about. Whether a subgroup is being averaged away. Whether a search result survives how hard you looked. Whether a fix actually reaches the people it was meant to help. Whether a perfectly reproducible number is still wrong.
The six checks below did not come from a fairness framework.
They came from failures we encountered while building this project.
1 · Does your metric look where the consequence lands?
Section titled “1 · Does your metric look where the consequence lands?”In Act II, AUROC barely moved while 13 patients changed metformin prescribing tier.
That was not a failure of AUROC.
AUROC measures ranking. The clinical consequence happened at a fixed threshold.
If the decision changes at a boundary and your audit only measures discrimination, the metric can remain stable while the downstream action changes.
Ask: what decision does this model ultimately influence, and would the metric you chose move if that decision changed?
2 · Have you looked beneath the global number?
Section titled “2 · Have you looked beneath the global number?”The model’s global AUROC looked strong.
But when the same predictions were separated by insurance status, performance fell to 0.776 for uninsured patients compared with 0.911 for insured patients.
Nothing about the model changed.
The subgroup view changed.
A global metric can be correct and still be incomplete.
Ask: which groups have you evaluated separately, and who decided those were the groups worth checking?
3 · How much searching produced the result?
Section titled “3 · How much searching produced the result?”The Hidden Harm Challenge searches many possible subgroups.
The default lattice considers 8 cells. After accounting for the size of that search, only 3 survive and 5 disappear under multiplicity correction.
Those discarded findings were not fabricated.
They were results that looked notable until we accounted for how many opportunities the search had to find something notable.
A subgroup result without the search history behind it is incomplete evidence.
Ask: how many groups did you examine before reporting this one, and does the statistical test account for all of them?
4 · Did the fix change the thing you claimed to fix?
Section titled “4 · Did the fix change the thing you claimed to fix?”Improving a model metric is not the same as improving the consequence that motivated the intervention.
If the concern is prescribing access, measure prescribing access.
If the concern is a clinical threshold, measure threshold crossings.
If the concern is subgroup performance, measure that subgroup after the change.
Do not substitute a convenient proxy simply because it is easier to optimize.
A model can improve globally while the people the intervention was intended to help experience no meaningful change.
Ask: after the intervention, did you re-measure the original consequence for the original population?
5 · Would you detect a silent change in the pipeline?
Section titled “5 · Would you detect a silent change in the pipeline?”This workshop has two supported installations.
The default Colab environment imputes missing laboratory values with the column median. The full environment uses nearest-neighbour imputation.
Some resulting values differ in the third decimal place.
The conclusions do not.
But we know that because the two environments were deliberately run and compared.
Every generated artifact also records which environment produced it.
Without that comparison, a dependency or implementation change could silently alter results while every notebook still executed successfully.
Reproducibility requires more than rerunning today’s code.
It requires knowing when tomorrow’s environment is no longer equivalent to today’s.
Ask: what would alert you if a dependency, preprocessing step, or environment changed your results without breaking the pipeline?
6 · Is the number reproducible or is it correct?
Section titled “6 · Is the number reproducible or is it correct?”This was the most dangerous failure in the project.
A national projection carried a 95% confidence interval that was too narrow by approximately a factor of √2.
The failure was not random.
The expected deflation factor was 1.4142.
The observed factor was 1.4194.
The error was therefore predictable, repeatable, and extremely stable.
And that was precisely the problem.
A regression test pinned the incorrect interval and passed across multiple development slices.
Every run reproduced the same answer.
The test worked. The estimator was wrong.
Regression testing can tell you whether a result changed.
It cannot tell you whether the original result was valid.
What exposed the problem was computing the same quantity using a genuinely different statistical approach: Taylor linearisation versus a cluster bootstrap.
That distinction matters.
Two implementations of the same flawed estimator can agree perfectly and give you false confidence.
Independent validation requires more than different code.
It requires a different route to the answer.
Ask: which high-stakes result in your pipeline has been independently computed using a genuinely different method?
Before you ship
Section titled “Before you ship”You do not need every audit to produce a reassuring answer.
You need every important uncertainty to become visible before deployment.
Before you ship, ask:
- Does the metric measure where the consequence actually occurs?
- Have we looked beneath the global result?
- Have we accounted for how many places we searched?
- Did the intervention change the consequence we said we were fixing?
- Would we detect a silent change in the pipeline?
- Has the result been validated by a genuinely different method?
A passing model metric is not the same thing as a safe system.
A reproducible number is not the same thing as a correct number.
And an audit is only as good as the failures it is capable of detecting.
Decide what would stop deployment before you see the result.
Then make the pipeline prove that it is ready to ship.