I keep sitting in agent review meetings that are theatre.

Someone reads the summary aloud. Nods happen. The next item comes up. Nobody opens the trace.

The CTO who signed off the summary

Last month I sat down with the CTO of a mid-sized fintech in Zurich. Her platform team had stood up a coding agent for small backlog tickets, and she wanted to show me their review ritual. She pulled up the last completed run. She read the agent's self-report aloud. She nodded. She signed it off.

I asked what the agent had actually done. She looked at me for a second. Then she opened the trace. Seventy-eight tool calls. The report mentioned six.

She said the quiet part out loud. We had been approving a story.

What the paper puts numbers on

Two researchers at Texas Tech's College of Media and Communication worked over 5,851 real coding-agent sessions on the SWE-chat corpus. That is 355,942 tool calls of actual developer usage across 205 repositories. They measured what the agent's self-written summary refers to, and what a reader working only from that summary can reconstruct.

Across 5,851 real developer sessions and 355,942 tool calls, a self-report referred to about one action in eleven, and a reader working from the report alone recovered roughly a fifth of the action log.

One action in eleven. Roughly a fifth of the log. Those are the setup, not the punch.

The punch is that the completeness of the report does not predict whether the work later needed a human to fix it. Sessions that got corrected had reports about 1.6 percentage points less complete than sessions that stood. Slightly less complete, not markedly less. The summary reads about the same whether the underlying execution went sideways or landed clean.

And when execution diverged from the plan the agent stated at the start, the report drifted toward the plan, not the execution. The further off the rails the run went, the more the summary described the rails.

The authors hand-checked their two AI-graded measures. One passed. One failed. They reported the failure and drew conclusions only from the measure that held. That is how you do this work honestly.

If your review workflow depends on the summary the agent writes about itself, you are reviewing a story the same system edited. The work itself happened somewhere your reviewer never looked. Measure the coverage of that summary against your actual tool-call trace before you promote the workflow. If it lands anywhere near one-in-eleven and does not correlate with your correction rate, the reviewer cannot save you. Change the artefact.