Clinical Decisions at the Threshold

Step 6Live-room allocation: 3:00

At this point in the workshop the question is no longer whether the model ranks patients well. It is whether a changed eGFR pushes a patient across a treatment boundary.

A clinician in a white coat with a stethoscope talks with a smiling patient in glasses and a headscarf during a consultation. — The the difference between receiving medication or not.

For metformin, those boundaries are explicit: standard at 45 and above, caution from 30 to 44, and contraindicated below 30. Crossing one of those lines can change what a patient is eligible to receive.

That is exactly what AUROC cannot tell us. AUROC follows the ranking.
Clinical decisions follow the threshold.

Among the 306 Black diabetic adults in this cohort, 4.25% — 13 patients — were placed in a more permissive prescribing tier by the race equation than by the race-free one. 11 crossed 45, where dosing guidance tightens. 2 crossed 30, into contraindication.

For Non-Black adults that figure is 0.00%.

The 0.00% is only half the audit

For Non-Black adults, the masking rate is 0.00%.

That number is correct. It is also incomplete.

It only counts one direction: patients for whom the race equation produced a more permissive metformin tier than the race-free equation.

But a threshold can be crossed in either direction.

So instead of asking only who became less eligible under the race-free equation, count everyone whose prescribing tier changed at all.

Across all 1,301 diabetic adults in this cohort, 13 moved to a less permissive tier under the race-free equation, while 16 moved to a more permissive tier.

And the split by race is complete:

Direction	n	Black	Non-Black
Masked — race equation more permissive	13	13	0
Reverse — race-free equation more permissive	16	0	16

Every masked patient is Black. Every reverse-direction patient is Non-Black. No patient falls on the other side of either count.

The same diabetic adults, both directions. Neither group is unaffected.

Did the change help?

Not in a way these counts alone can tell us.

The two directions should not be netted against each other. 13 Black patients move to a less permissive metformin tier under the race-free equation, while 16 Non-Black patients move to a more permissive tier.

Those are different patients facing different clinical consequences. One change does not cancel the other.

And moving to a more permissive tier is not automatically a benefit. 6 patients move from contraindication into caution under the race-free equation, making metformin available where it was previously restricted.

Whether that is the more appropriate decision depends on the patient’s true kidney function. This cohort does not contain measured GFR, so we cannot use it to determine which equation placed each patient in the clinically correct tier.

What we can say is narrower, and more important for the audit:

Changing the upstream equation changed who crossed the treatment boundary, and it changed them in different directions.

The goal is not to make those counts cancel. It is to identify each consequence and evaluate it separately.

Where we are in the data supply chain

Blood draw
Serum creatinine
eGFR equation
CKD stage
Mortality model
Metformin tier
Clinical decision

The number changes at the eGFR equation.
The decision changes at the metformin tier.
The metric is looking at the mortality model.

A different question needs a different gate

At the beginning of Act II, our gate asked whether the model still performed well.

Here we are auditing something different:

Did changing the upstream equation change anyone’s prescribing tier?

For this workshop, we pre-commit to a simple review rule:

If even one patient’s metformin tier changes, stop and investigate.

REVIEW — Black diabetic masking 4.25% > 0.0%

The review threshold is therefore zero affected patients.

Zero is not a clinical standard or a statistical cutoff. It is an audit choice. We are saying that a treatment-tier change is consequential enough that we do not want it silently absorbed into an aggregate metric.

And we do not subtract one direction from the other. A patient moving to a more permissive tier does not cancel a different patient moving to a less permissive one.

Crossing the threshold does not prove harm. It tells us where review is required.