Every vendor pitch for a high-stakes model now has the same slide. "Post-hoc fairness layer." Procurement reads that as the bias problem settled.
Nobody I know tests what the layer actually does. Four researchers just did, on six of the leading implementations.
The vendor demo in Zurich
A head of risk at a mid-size European consumer-credit lender, in a finalist pitch last month. The model was a default-risk scorer. The vendor pack was polished. Slide 14 made the standard claim: a post-hoc layer ensures demographic parity across protected classes. The procurement room read it as the bias problem settled.
I asked the vendor's technical lead whether the correction mechanism uses group size to define disadvantage.
He paused. He said it depended on the configuration. I asked which configuration shipped by default. He said he would come back to us.
The head of risk looked at me. I told her the question was the whole audit. If the layer defines disadvantage by group size, the correction flows to the big group and the smallest subgroups move the wrong way. She asked if there was a paper. There is now.
What the paper found
Kelly McConvey, Angelina Zhai, Rebecca Li and Shion Guha reproduced a vendor-deployed Early Warning System on real data. The cohort: 168,550 student records from a public Ontario college across 15 years. They bolted on six of the leading post-hoc fairness interventions, one at a time, under the procurement constraints an executive faces. Output only. No model inspection. No retraining.
None of the six consistently reduced error disparity across subgroups. Two of them made the smallest subgroups worse. MBS hit the ≤0.05 gender equal-opportunity threshold a vendor would show on a dashboard. It got there by raising Female FPR from 0.561 to 0.837 and Male FPR from 0.406 to 0.734. Convergence by concurrent harm.
The Unknown subgroups, 154 students for gender and 3,100 for age, were where the group-size interventions concentrated the damage. The dashboard reads "parity achieved." The actual error burden on the smallest groups rose.
"Dashboard metrics converge while groups' error burdens persist or worsen."
This sits in the same shape as yesterday's piece on Stack Overflow. Different mechanism, different domain, different decision point. A surface metric converges while the underlying distribution of real reward or real harm stays stuck. Two independent findings pointing the same way is a pattern worth naming, not a theory yet.
If you cannot inspect the model, the fairness-layer slide is a claim about redistribution of error, not a reduction of it. Ask the group-size question in the next vendor meeting. If the answer is yes, the smallest group in your population is paying for the dashboard.