Why a second pair of eyes stops working

The failure that gets past careful people

Give a competent professional a machine that is right most of the time, and their error rate on the cases where it is wrong goes up — not down, and often above what it would have been with no machine at all.

This is automation bias, and it is one of the best-established findings in the human factors literature, measured for forty years in aviation, air traffic control, process plants and clinical decision support long before anybody typed into a chat window. It is not a character flaw and it is not solved by telling people to be careful. It has a mechanism, and once you know the mechanism you can build around it.

Two errors, not one

The literature separates them, and the distinction is worth keeping.

Errors of commission: you do what the machine said, and the machine was wrong. The flagged transaction that was legitimate. The suggested diagnosis that was not the right one.

Errors of omission: the machine said nothing, and so you missed a problem you would have spotted unaided. This is the more dangerous half, because there is no artefact to review afterwards. Nothing happened. The error leaves no trace except the thing that was missed.

A checker who trusts the tool becomes a checker of the tool's output rather than a checker of the underlying reality, and those two jobs have different coverage.

The evidence, including the part that stings

The most instructive case is not from AI at all. Computer-aided detection for screening mammography was approved in the United States in 1998 and adopted extremely widely — by 2012 it was used on the large majority of screening mammograms, and Medicare paid a premium for it. A 2015 analysis of around 320,000 women screened across 66 facilities found that radiologists using it performed no better on sensitivity or specificity than radiologists not using it. A tool that flagged plausible regions, in the hands of trained specialists, produced no measurable gain across the population, at very large cost.

Nothing about that finding transfers automatically to language models — different task, different failure mode, different decade. What transfers is the shape of the lesson: an aid that is usually right, used by experts, can fail to improve the joint outcome, and everybody involved will nonetheless report that it helps.

Which brings the second, more uncomfortable finding: the better the aid, the worse the complacency. Trust is calibrated to the base rate. A tool that is wrong one time in three keeps you alert. A tool that is wrong one time in fifty trains you, over a few hundred uses, to stop looking — and the fiftieth case is the one where your attention was the entire point of your job.

Why a second reviewer does not fix it

The instinct is to add a colleague. It does much less than you think, for a reason worth stating plainly.

Two people reviewing the same generated draft are not two independent checks. They are two people anchored by the same text, in the same order, with the same omissions invisible to both. Independence is what makes a second check valuable, and reading somebody else's finished prose destroys it. This is the same effect that makes a second opinion worth much less when the second doctor has already read the first doctor's notes.

Give the second reviewer the source — the transcript, the file, the original figures — and ask for their answer, not their assessment of this answer. Then compare. That is a real second check and it costs more, which is why it is reserved for work where being wrong is expensive.

Four things that actually help

1. Commit before you look. For any decision that matters, write down your own answer, or at least your expected shape of the answer, before you read the model's. Thirty seconds. It converts a passive review into a comparison, and the disagreements become visible instead of being smoothed over.
2. Watch your edit rate. Keep a rough count of how often you change something before sending. If you have accepted twenty drafts in a row unchanged, you are no longer reviewing them, whatever you believe about your own diligence. That number is the earliest available warning and it costs nothing to keep.
3. Sample against ground truth on a schedule. Once a fortnight, take three completed items and check them properly against the source — not for a feeling, for a count. Errors found per ten items is a number. "It seems reliable" is not.
4. Vary the direction of the check. Instead of reading the output and asking whether it is right, take the source and ask what should be in the output, then look for it. This finds omissions, and reading the output never does.

The honest part

You will not eliminate this. Nobody has. Aviation reduced it with checklists, mandatory cross-checks and a culture in which challenging the automation is expected rather than awkward, and the effect is still measurable in trained crews.

The realistic aim at work is not immunity. It is to keep the tool out of the places where an unnoticed omission is unrecoverable, and to build one or two cheap mechanical habits — the committed answer, the edit-rate count — that do not depend on you feeling alert on any particular Tuesday.