Every consumer brand I coach right now is running the same math on the back of the same slide. Cut the concept-test panel. Buy digital twins. Save half the research budget by pretesting campaigns against a synthetic customer who never sleeps.
The interview economics are real. The prediction economics are a mirage. Executives are signing contracts on the wrong half.
The moment the pitch broke
A VP of consumer insights at a mid-market beverage brand in Chicago showed me her vendor deck last Tuesday. Twenty AI-moderated interviews per SKU. A digital twin built from each transcript. A pretest bench that would score every new can design before it ever went to a concept-test panel. She wanted my read on the vendor.
What she wanted, actually, was permission.
I asked her one question. If the twin's prediction lines up with what your real customer does when they see the real can, how would you know? She paused. That is what the vendor is for.
That is the exact place the pitch breaks. The vendor is the party being tested. The twin cannot audit itself.
What the paper actually measured
Columbia Business School's Digital Twins Lab just published the head-to-head I have been waiting for. Deng, Liu, Toubia and Jain ran a preregistered between-subjects study. N=317. Three industry partners. A real North American building-products client. Three arms: AI-moderated interviews, human-moderated interviews, static interviews. Then a held-out prediction task where each participant's twin had to guess how the real participant would react to real mailers, real ads, real claim rankings.
The input side reads like the vendor's slide. AI moderation covered more themes than a human interview in three of four blocks. Cost per unique need was $19 for AI, $27 for static, $36 for human. Under a fixed budget, AI recovered 238 unique customer needs, static recovered 188, human recovered 109. If needs discovery is the goal, the pitch holds.
The output side is where it fails.
Differences between AI-moderation and Static were small and non-significant for Accuracy, τ-b, and τ in both Demo and Full personas.
Prediction accuracy on the mailer and commercial block sat at .743 for the Full persona built from AI-moderated interview data. A twin built from a cheap static survey did no worse. The extra qualitative richness the AI generates does not carry through to prediction. One more finding worth carrying: participants sound more positive, activated, and assertive when speaking to a human than to an AI moderator. The qualitative texture is still not synthetic.
Buy AI moderation for the front of the funnel where cost per unique need decides the game. Do not retire the concept-test panel on a promise no one has verified. Ask any vendor for the calibration experiment against a static-survey baseline rather than a demographics baseline. If they cannot show you that comparison, they are selling you the input and hoping you never check the output.