TL;DR
Did you know? Human-in-the-loop is specifically built into almost every AI security program. A human sitting in the workflow is not oversight. Oversight involves five things being true at the moment someone clicks approve: they can see what the agent saw, they know where it breaks, the volume fits their attention, saying no costs them nothing, and the decision leaves a record. Miss one and you have a signature instead of a decision. This post shows you which one you have, and what to do about it.
Introduction
Close to 3 out of 4 companies plan to run AI agents within the next two years. Only 21% say they have a mature way to govern them, according to Deloitte’s State of AI in the Enterprise survey of 3,235 business and IT leaders.
That gap is not about missing humans. Most teams added the human. The gap is about what happens to that person once the agent starts running.
Three signs your loop is already a rubber stamp
You do not need an audit to find out. You need three numbers you probably already have.
1. Approvals get faster
Pull your median approval time for week one and compare it to week twelve. A healthy loop stays flat. A rubber stamp drops. When reviewers started at 90 seconds and now sit at 4 seconds, the gate is still there. The reading stopped.
2. Almost nothing gets rejected
An override rate near zero does not prove the agent is right. It proves nobody is checking. Any system making thousands of calls will get some wrong. If your reviewers never disagree, the disagreement is not reaching them.
3. Nobody can say who approved what
Pick a containment action from last Tuesday. Ask who approved it and why. If the answer takes ten minutes to find, or comes back as a team name instead of a person, that approval was a workflow step, not a decision.
What oversight has to mean, according to the people writing the rules
The clearest published definition of human monitoring sits in Article 14 of the EU AI Act. It applies to high-risk systems placed on the EU market, so it may not directly bind your program. It is still the most specific answer anyone has written down, and it is worth borrowing.
Article 14 says the person assigned to oversee a system must be able to do five things:
- Understand what the system can and cannot do, and notice when it behaves strangely.
- Stay aware of their tendency to trust the output, which the law calls automation bias.
- Read the output correctly, using whatever tools exist to interpret it.
- Decide, case by case, whether to ignore, reverse, or override what it produced.
- Step in or stop it, through a stop function or a similar way to reach a safe state.
Read that list again. Four of the five are about the human, not the machine. Most security programs buy the stop button and skip the rest.
The five conditions that turn a signature into a decision
Five conditions that turn a signature into a decision.
A person sitting in the workflow is not oversight. Oversight is these five things being true at the moment someone clicks approve. Miss one and the gate still exists, but the judgment is gone.
Four of the five are about the human, not the machine. Most programs buy the stop button and skip the rest.
Here is that list rebuilt for a security team. You can check each one this week.
1. The reviewer sees what the agent saw
An approval screen that reads “isolate host WS4471, confidence 94%” is not reviewable. It gives nothing to agree or disagree with.
A reviewable approval shows the signals that fired, the assets and identities involved, what the agent ruled out, and what breaks if the action is wrong.
The test is simple. Could a second analyst reach a different conclusion from the same screen? If not, you are not asking for a review. You are asking for a confirmation.
2. The reviewer knows where the agent is weak
Most teams skip this condition, and the research on it is uncomfortable.
A randomized trial of 44 physicians published in NEJM AI in 2026 gave every participant 20 hours of AI literacy training first. Then it fed half of them deliberately wrong AI suggestions on three of six clinical cases. Accuracy on the top diagnosis fell from 90.5% to 76.1%, an adjusted drop of 18.3 percentage points.
Trained professionals, warned, still got worse when the machine was confidently wrong. The sample is small, so treat it as directional. The direction is the point.
Your analysts are not built differently. The fix is not more general training about AI. It is a short written list of where this specific agent misreads things: the data sources it handles badly, the asset types it misclassifies, the attack patterns it has never seen. Reviewers need the failure modes, not the feature list.
Same action. Two screens. Only one is reviewable.
An approval screen with nothing to disagree with is not asking for a review. It is asking for a confirmation. Here is the difference, on the same containment action.
CONFIDENCE: 94%
REVERSIBLE: Yes, 1 click, 30 sec
The test: could a second analyst reach a different conclusion from the same screen? If the answer is no, the gate is decoration.
3. The volume fits a human attention budget
A reviewer facing 40 approvals an hour is not reviewing. They are clearing a queue. This is where most loops break first, and it deserves more room than one section.
The short version: how many approvals a person sees per shift is a design choice. If you have not made that choice, the agent made it for you.
4. Saying no costs the reviewer nothing
Ask an L1 analyst what happens when they reject the agent and the agent turns out to have been right.
If the honest answer involves an awkward conversation with their manager, you have built a system where approving is safe and rejecting is risky. People respond exactly as you would expect.
Real oversight treats a wrong rejection as the system working. The reviewer paused something that deserved a second look. That is the job, and it should read that way in their review.
5. The decision leaves evidence behind
An approval record that holds only a timestamp and a user ID proves someone clicked. It does not prove that anyone decided.
Useful evidence captures what the reviewer saw, what the agent proposed, what the reviewer chose, and why they went the other way.
That record is what makes the loop auditable later. It is also what makes the next section possible at all.
Four numbers that tell you if oversight proves real
Four numbers that tell you if anyone is actually reading.
You cannot fix oversight you are not measuring. None of these need new tooling. They need an approval record that captures more than a timestamp.
Watch the direction over 90 days, not the number on any given week. Oversight decays quietly, and the trend is where it shows up first.
- Override rate. The share of agent recommendations a human changes, rejects, or sends back. Under 2% usually means nobody is reading. Over 30% usually means the agent is not ready to be trusted with that action type.
- Time to decision. Median seconds between an approval appearing and a human acting on it. Flat or rising is healthy. A steady fall over weeks is the clearest warning sign on this list.
- Reversal rate. How often an approved action gets undone within 24 hours. Climbing reversals mean approvals are moving faster than understanding.
- Sampled review. Pick and review 20 approvals a month at random and have a senior analyst decide them cold, without reviewing the original call. Disagreement above 20% means the loop needs work, whichever direction the disagreement runs.
Where a human in the loop makes things worse
Oversight is not free, and spending it in the wrong place is how you lose it.
Putting a gate on a reversible, low-impact action that happens 200 times a day adds clicks, not safety. Each click costs you attention you will need later. Worse, it teaches the reviewer to approve without looking. By the time a decision that actually matters shows up, the habit is set.
The goal was never more human assessment everywhere. It is human decision where it changes the outcome.
How Secure.com approaches governed action
Secure.com builds governed AI Teammates that do security work above your existing stack. Your team sets the rules, and the Teammates work within them.
- Scope. Every Teammate runs inside defined boundaries: which accounts, which assets, which repositories, which actions it may touch.
- Permissions. You classify actions before anything runs, so you decide what’s advisory, what needs approval, and what may run within policy, rather than discovering it in production.
- Approval. Consequential actions stop for a human, and the approval carries the signals, the reasoning, and what the action affects, not just a score.
- Escalation. Anything falling outside policy routes to a named owner instead of aging in a queue.
- Evidence. Record every action, approval, and override with its context, so you can prove what happened and measure how the loop is holding up.
FAQs
Does law actually require human oversight of AI?
What is a healthy override rate for an AI security agent?
How do I stop analysts from approving on autopilot?
Does more oversight mean slower response?
Can oversight scale as we add more agents?
The takeaway
Pull one number this week: your median approval time, week one against this week. If it has fallen by more than half, your loop is already a rubber stamp, and nobody has noticed yet. Fix it in the order the conditions are listed. Put real context on the approval screen first, because none of the other four conditions can do anything without it.
