Most execs rolling out AI copilots are running an unmonitored experiment. One model, one answer, one professional reading it. When the model is right, the professional looks sharp. When the model is wrong, so is the professional, about four times out of five.

That number used to be my guess. Now I don't have to guess.

The second model that fixed nothing

A VP of Operations at a mid-market consulting firm, in a coaching call three weeks ago. She had read a trade press piece on AI ensembles over the summer and quietly rolled out a second model inside her analyst pool. Two providers, side by side, visible when they disagreed. She wanted to tell me what had happened.

The senior analysts had become sharper. The partners said the quality of first-draft work was finally where they wanted it. The junior analysts had not budged. Same error rate as before, same client complaints about half-checked numbers. Same meetings.

She asked me what she had done wrong. I told her she had done the second half of the right thing without the first half. The first half is knowing which people the second half will reach.

What the research says

Wu and colleagues at Nanchang University and HKUST just published the first registered multicenter RCT on exactly this. Three hospitals in China, July to September this year. 132 residents, 123 analyzed, 60 radiographs each, read once without AI and once with. One arm saw a single AI suggestion. Two arms saw two suggestions from different models.

The error-catch number is the one every leader will want. When the single AI was wrong, radiology residents still reached the correct answer 20.0% of the time. With two models visible, 40.1% and 40.4%. A doubling.

The second finding is the one most leaders will miss.

The dual-advisor arms lifted radiology residents by 6.69 to 7.87 percentage points (Holm-adjusted P=.030). Non-radiology residents gained nothing significant (P=.20). The specialty-by-dual-suggestion interaction was 10.44 percentage points, P<.001.

The ensemble is a disagreement signal. It only pays when there is someone next to it who can tell which model is right.

If the people you handed the second AI to could not catch the first one, you have bought the UX of a safer pipeline. None of the safety. You have given your juniors a tie-breaking vote in an argument they do not understand.

Pick the professionals first. Then pick the architecture.