Designing autonomy levels for AI agents: assist, approve, autonomous

Every agent that takes action in a business system raises the same question: who decides before it acts? Teams tend to answer it once, for the whole agent, and usually at one of two extremes. Either every output goes to a person, which keeps risk low until the approval queue becomes the bottleneck and people start approving without reading. Or the agent runs unattended from day one, which works until the first expensive mistake and then gets switched off entirely.

We design autonomy as a property of each action, set by its risk, and changed only on evidence. An agent can tag returns on its own, prepare refunds for approval and only suggest account closures, all in the same workflow. Each action starts at a level that matches what it could break, and moves up when the record shows it has earned the move. It moves down again when something goes wrong.

This article sets out the three levels we use, how we assign risk tiers, what a good approval screen looks like, and how an action earns its way from one level to the next.

Three levels, defined by who acts

Human-factors research has described degrees of automation for decades. Parasuraman, Sheridan and Wickens (2000) proposed a ten-point scale and, more usefully, pointed out that automation can be set at different levels for different functions: gathering information, analysing it, choosing an action and carrying it out. Driving automation uses a six-level scale in SAE J3016. For business agents we find three levels are enough, and each one is defined by a plain question: who carries out the action, and who decides that it should happen?

At Assist, the agent drafts, suggests or summarises, and a person decides and acts. The agent holds no write permission for the action. At Approve, the agent prepares the complete action (the exact API call, with its arguments) and a person approves, edits or rejects it before it runs. At Autonomous, the agent acts within limits and people audit a sample afterwards. Figure 1 sets them side by side.

The levels are deliberately coarse. Finer scales tend to produce arguments about whether something is a 6 or a 7, while three levels map directly onto implementation: no write permission, a write that waits in a queue, or a write that executes. That mapping matters, because a level that exists only in a prompt ("ask the user before refunding") is a suggestion to the model. A level enforced by the tool gateway is a control.

Autonomy belongs to actions, not agents

Consider an illustrative returns agent for an online retailer. In one conversation it might classify the return reason, look up the order, generate a return label, issue a refund, flag the customer for suspected return fraud and, rarely, recommend closing the account. Treating these as one agent with one autonomy level forces the whole agent down to the level of its riskiest action. Treating them separately lets the low-risk work flow while the high-risk work waits for a person.

So we write autonomy into policy per action, and the gateway enforces it:

agent: returns-agent
actions:
  classify_return_reason:  { level: autonomous }
  create_return_label:     { level: autonomous, cap_per_day: 500 }
  issue_refund:
    - when: "amount <= 40 and order.age_days <= 30"
      level: autonomous
    - when: "amount <= 250"
      level: approve
      approver_role: support-agent
    - otherwise:
      level: approve
      approver_role: support-lead
  flag_suspected_fraud:    { level: approve, approver_role: risk-analyst }
  recommend_account_close: { level: assist }   # no write permission at all

The thresholds above are illustrative. What matters is the shape: conditions on the arguments, a level per condition, a named approver role, and a hard cap on volume even for autonomous actions.

Assigning risk tiers

The starting level for an action comes from its risk tier. We assess two things first. Blast radius is how much one wrong action can affect: a single record, a single customer, or money and many customers at once. Reversibility is how cheaply a wrong action can be undone: a re-tag is free to reverse, a refund can be clawed back only with effort and goodwill, and an email to a customer cannot be unsent. Figure 2 shows how those two combine into a starting level.

Three further factors override the grid. If the action is a decision about a person in a regulated area such as credit, employment or access to essential services, it starts no higher than Approve whatever the grid says, and the applicable rules may require meaningful human involvement. If the action reaches an external party (a customer, a supplier, a regulator), it starts one level lower than the grid suggests, because the reputational cost of a wrong message is hard to reverse. And if the agent reads untrusted content on the way to the action, the risk of prompt injection pushes the action down a level until the controls described in our piece on securing AI agents are in place.

We record the tier, its reasoning and the owner in the same repository as the policy. When an auditor or a new team member asks why an agent can issue refunds on its own, the answer is a reviewed file in the repository.

Designing approval that people actually read

Approve is the level where most actions spend most of their early life, and it fails in a predictable way. People who approve the same kind of request many times an hour stop reading it. The EU AI Act names this tendency directly: Article 14 requires that people overseeing high-risk systems are enabled to remain aware of "automation bias", the habit of over-relying on a system's output. OWASP's Top 10 for Agentic Applications lists human-agent trust exploitation as a risk in its own right, because an attacker can rely on a tired approver as much as on a weak control.

The approval screen has to make reading cheap and blind approval visible. We design it around five rules.

1. Show the exact action, with the real arguments, in plain language: "Refund £182.40 to the card ending 4417 for order 10-55821", never "Approve refund?".
2. Show the evidence the agent used and its stated reason, with links to the source records, so the approver can check one fact in one click.
3. Show what is unusual. If the amount is three times the customer's average order, or the reason code is rare, say so at the top.
4. Make edit as easy as approve, and ask for a short reason on reject or edit. Those reasons are the most valuable training and evaluation data you will collect.
5. Avoid bulk approval for anything above the lowest tier. "Approve all 40" turns forty decisions into one.

We also measure the approvers. Median time to approve, the share approved in under a few seconds and the rate of edits per approver tell you whether review is real. If one approver accepts everything in two seconds, the action is effectively autonomous already, without the caps and sampling an autonomous action would have. Either fix the workload or promote the action properly.

Earning autonomy with evidence

The Approve level produces exactly the data needed to decide whether an action can move up. Every approval, edit and rejection is a labelled example of whether the agent's proposed action was right. Figure 3 shows the loop.

Three measures do most of the work. Agreement rate is the share of proposals approved without change. Severity-weighted error counts rejections and edits by how much harm the original proposal would have caused, because a wrong reason code and a refund to the wrong customer are not the same failure. Coverage checks that the evidence spans the conditions the action will meet: busy periods, unusual products, each region or store.

Sample size matters more than teams expect. A simple statistical rule helps: if an action has run n times with zero serious errors, the 95% upper confidence bound on its true serious-error rate is roughly 3/n. Fifty clean approvals only tell you the serious-error rate is probably below 6%. Three hundred tell you it is probably below 1%. Decide in advance what rate you can tolerate for the action, and the number of clean approvals you need follows from it.

Promotion is a decision made by the action's named owner, recorded with the evidence, and it moves the action one tier at a time. A promotion can also be partial, which is often the right answer: refunds below a threshold become autonomous while larger ones stay at Approve.

Autonomous does not mean unwatched

An autonomous action keeps four controls. A cap on volume and value per hour and per day limits the damage from a bad deployment or an attack. A sampling audit sends a small random share of executed actions, plus every action that trips an anomaly rule, to a person after the fact. A kill switch returns the action to Approve immediately, owned by whoever is on call. Drift monitoring watches the inputs and outcomes: the mix of reasons, average amounts, reversal rates and complaint rates.

Demotion is automatic on clear triggers, such as a serious error found in sampling, a breach of the cap, a model or prompt change, or a significant shift in the input mix. Demotion carries no blame; it lets the system collect fresh evidence under a changed situation. A model upgrade in particular resets the evidence, because the record was earned by a different model.

Where this breaks

Some actions never produce enough volume to earn autonomy. If an action runs ten times a month, it will take years to accumulate a meaningful record, and it should stay at Approve. That is fine; the approval cost of ten decisions a month is small.

Agreement with reviewers is only as good as the reviewers. If approvers share a blind spot, the agent learns to match it and the metrics look excellent. Periodic expert review of a sample, by someone outside the usual approval rota, is the check on this.

Errors with delayed consequences are hard to measure. A refund is visibly right or wrong within days. A supplier-terms change or a credit-limit adjustment may only show its cost months later, long after the action was promoted. For these we hold actions at Approve longer and track outcomes, not just approvals.

Finally, per-action policy adds design work. Someone has to enumerate actions, assign tiers and maintain thresholds as the business changes. We think that work is the job, but teams should budget for it and assign an owner, or the policy file will drift from reality as quickly as any other documentation.

A decision rule for your next agent action

Before an action ships, answer five questions in the policy file. What is its blast radius and how reversible is it? Which level does that give, after the overrides for regulated decisions, external parties and untrusted input? Who owns it and who approves it? What evidence, in numbers, would promote it one tier? And what triggers would send it back down?

If you cannot answer the last two, the action is not ready to leave Approve. If you cannot answer the first three, it is not ready to leave Assist.