Note · July 2026
The verifier problem —
the piece most people miss.
01
It sounds like a control. It's the sentence that ends the risk conversation and lets the project go ahead. And most of the time, it's a claim nobody has tested.
02
But the moment an AI system acts — sends the email, moves the money, changes the record — verification stops being a safety net you can use later. It becomes the only thing standing between a mistake and its consequence. And you cannot catch a bad action after it has already happened.
03
“Is there a human in the loop?” is the wrong question. The real one is narrower and much harder: can this human catch this error, reliably, under time pressure? It fails three different ways — and each one is invisible until it's expensive.
Three ways the loop fails
Each one looks like oversight. None of them is.
Can they?
The expertise gap.
The person signing off doesn't have the domain depth to judge what they're looking at. The output is plausible, fluent, internally consistent — and they have no way to tell whether it's right. They approve it because nothing about it looks wrong. That isn't verification. It's a rubber stamp wearing verification's clothes.
Will they?
Vigilance decay.
Give someone ninety-nine correct outputs in a row and their attention quietly leaves the room. A checker who never finds an error stops looking for one — not through laziness, but because that's how human attention works. The hundredth output is the wrong one, and it sails through, because the job of “check this” slowly became the habit of “approve this.”
Could anyone?
The comprehension ceiling.
Sometimes the output has simply outrun any available checker. It's not that this person can't verify it — it's that no one in the building can, not in the time they have. I've hit this myself: a system generating work I'd have needed to be a full-time specialist to check. At that point the output isn't unchecked. It's uncheckable — a different, worse problem.
The line that matters
A Chief Risk Officer put it to me in a room like this: “It matters that the human can tell right from wrong. But maybe they can — will they? I worry we've put false confidence in the human-in-the-loop thing.” The one person paid to be cautious had already found the hole.
What to do instead
Stop asserting the human. Design the check.
The fix isn't a better prompt or a smarter model — it's design. Route only the work a person can actually judge to that person. Surface confidence, so low-certainty items get flagged rather than buried in a hundred confident ones. Keep the human doing enough of the real work to stay sharp enough to check it. And before an AI system is allowed to act rather than suggest, ask the three questions out loud — can they, will they, could anyone — and don't ship until the answers are yes.
David Kolb advises expertise-heavy organisations on adopting AI without letting the safety net turn into theatre. This note is written to be forwarded — if it's useful to a colleague, send it on.
One thing you can do this week
Pick one place AI already touches your work and name the person who would catch a mistake. Then decide which of the three it is: they can, they probably wouldn't under pressure, or nobody reasonably could.
What it won't tell you
If it's the third, more training won't solve it. It still won't tell you what controls belong there instead.
If this is live for you.
Send me the thing you can't get a straight answer on. I'll tell you whether I'm useful.
Email me →