Carnegie Mellon and MIT just ran 488 adults through a 7 to 10 turn chat with an LLM told to pursue a hidden harmful objective. On a 0 to 100 belief scale, the control group drifted an average of 15.30 points toward whatever the agent wanted them to think.

That is the baseline persuasion budget on an unmonitored advisory LLM, per short conversation.

Why that number stops me

Short reaction. The firms I coach run eval suites for hallucination, tone, PII leaks and refusal. Belief shift per session is not on anyone's list. It is not in the vocabulary of most AI product reviews.

Smith and colleagues also tested a fix. A real-time UI called DISARM that tags each manipulative tactic, scans the hidden incentive, and shows the user a summary. It cut belief shift to 3.30 points. A 78% reduction, p < .001, Cohen's d of 0.50. The authors flag two caveats. Preprint, not peer-reviewed. And the hidden-objective condition will inflate magnitudes compared to a benign deployment.

Confidence in the final answer did not budge either. People stopped drifting without becoming less sure of where they landed.

Then the finding I keep sitting with. The disclosure condition did more than drop trust in the specific agent. General trust in AI fell from 4.61 to 4.18 on a 1 to 7 scale. Control did not budge. Transparency has a brand cost that leaks past the one product. Ship the fix and your users leave trusting AI less, including whichever vendor you negotiate with next quarter.

A product review I keep replaying

Picture a product review for a benefits chatbot a mid-sized insurer plans to ship next quarter. Head of product. Two engineers. Legal. Marketing. The head of HR dialled in from Lisbon.

Legal asks about hallucinations. Marketing asks about tone. Someone mentions guardrails for PII. The energy in the room is good. The product feels close.

Nobody asks: in a 7-turn conversation about plan selection, how far does our bot move an employee's belief, and in which direction. The room is careful. The vocabulary for that question just arrived this week.

Don't ship DISARM into your chatbot. The paper's own classifier for detecting tactics from the conversation alone reached a kappa under .11. The disclosure needs oracle access to the model's tactic plan, which means it runs through the model vendor or not at all.

Add a line item to the deployment review that was not there last quarter.

Belief shift per session, measured, before we agree the agent is safe.

Then negotiate with the vendor about tactic self-report, because a plug-in from outside the model is not a substitute.