Skip to content
Bonrock
Back to articles
AI Strategy

What Can the Person Clicking Approve Actually Check?

A high approval rate tells us little about effective oversight. The useful test is whether a reviewer can recognize a bad action and prevent it from taking effect.

Claude Code users approved roughly 93% of permission requests, according to Anthropic's telemetry. The company describes approval fatigue and reports that sandboxing reduced prompts by 84%.

The tempting conclusion is that people were clicking without thinking. The approval rate alone cannot tell us that. It would also be high if nearly all proposed actions were appropriate. To evaluate oversight, we need to know what happened when an action deserved a refusal.

That missing denominator changes what a company should ask of its AI supplier. A record of approvals shows that someone had an opportunity to intervene. Whether that opportunity was useful still needs testing.

A reviewer needs something to disagree with

Consider the design of a hypothetical refund assistant: the agent proposes a refund and asks a support worker to approve it.

If the screen shows only the amount and the agent's explanation, the worker can judge whether the explanation sounds plausible. To check the decision, they also need the order, the applicable policy, previous refunds and any exception the agent relied on. A confident summary cannot substitute for the records it summarizes; an error could appear in both the proposal and its explanation.

There is a cost to showing those records. Reading them may take as long as processing the refund manually. At that point, the organization has a choice: narrow what the agent may propose, improve the evidence shown to the reviewer, or accept that this workflow offers little saving. Adding an approve button does not resolve that choice.

The design should also let the worker reject or amend the proposal without having to fight the system. If refusing creates more work than approving, management has built a reason to agree and then called agreement a safeguard.

Ben Green's review of 41 government oversight policies challenges the assumption that a human can perform whatever checking a policy assigns them. Ben Shneiderman's human-centered AI framework holds that high automation and high human control can coexist. These positions leave a concrete burden on the designer: demonstrate what the person can control in this particular arrangement. Neither the job title “reviewer” nor a diagram with a human in it answers that question.

Test the refusal before trusting the approval

For the refund example, we would test the review step before allowing it to authorize payments. Use a controlled environment with no live money, and include proposals with known mistakes: a duplicate refund, an ineligible order, a correct amount sent to the wrong recipient. The reviewers should have the information and time they would get in ordinary work, without being told which proposals contain mistakes.

Count which errors they catch and which they miss. Also count correct proposals they reject, the time spent investigating and the cases they cannot resolve. A gate that catches errors by stopping almost everything may protect the account while making the service unusable.

We would repeat that check after launch through sampled reviews and incident analysis. An exercise can make people unusually attentive; real queues contain distractions and changing cases.

Some actions should remain outside the agent's permissions even when a reviewer is available. In the refund example, access to unrelated customer accounts would add risk without helping the task. A reviewer should not have to detect every attempt to use access that the agent never needed. Anthropic’s sandboxing decision addresses that part of the problem by restricting what the agent can reach.

Checking depends on skills that the workflow can stop teaching

A reviewer can have every relevant record and still lack the knowledge to use it. Anthropic's randomized study of 52 developers learning an unfamiliar Python library found average quiz scores of 50% with AI assistance and 67% without it. This short exercise raises a question about learning new skills; it does not establish long-term deskilling among experienced developers.

For a team asking junior staff to check generated work, the study raises a practical question: where will they learn to recognize an error? An employee who sees only polished proposals may rarely encounter the failed attempts that explain why a rule exists. The organization needs to give them opportunities to investigate cases, receive corrections and work through decisions themselves. Training time belongs in the cost of this form of oversight.

Responsibility needs similar care. Madeleine Clare Elish's “moral crumple zone” describes how blame can fall on an operator who had limited control over an automated system. A signed approval can make that misattribution easier. The deployment owner should be able to explain why the worker had enough information, competence and authority to make the decision they were assigned.

In the refund workflow, we would want to see a reviewer catch the duplicate, identify the wrong recipient and explain an unresolved exception before relying on the gate. If they cannot, the next task is to change the information, permissions or training around that decision.

Sources

Found this helpful? Pass it on.

Working on something like this?

We help organizations build AI systems that are secure, private and ready for real load.

Talk to us →