Human Gates, Agent Throughput: An autonomy ladder that scales without losing control

If you want agents to move fast and stay safe, you need two things: a clear autonomy ladder and explicit human gates. This article proposes a maintainable ladder (L0–L5), the controls required at each step, and the KPIs that prove you can scale without losing governance. It borrows the idea of staged autonomy from mature safety-critical domains, then adapts it to B2B software and services. (SAE International, UNECE Wiki)

Why a ladder (not a leap)

Risk frameworks emphasise governance, traceability, and human oversight as autonomy increases; a ladder lets you earn autonomy with evidence rather than grant it by belief. Use NIST's AI RMF to drive governance, measurement, and manage functions, and ISO/IEC 42001 to institutionalise the management system behind them. (NIST, NIST Publications, ISO, KPMG)

Human-in/on-the-loop (working definitions)

We'll use "human-in-the-loop" when a person must approve or co-produce an outcome, and "human-on-the-loop" when a person supervises, with authority to intervene or stop. These are common regulatory and research notions; they vary by context but capture the oversight we need. (wp0, NIST Publications)

The autonomy ladder (L0 → L5)

Each level defines what agents may do, the required gates/controls, and upgrade criteria. Keep your evidence in a tamper-evident log (traces, evals, costs, decisions).

L0 - Suggest

Scope: Drafts, analyses, candidate actions; no external effects.
Gates/controls: Prompt and output schema validation; logging; basic evals.
Upgrade criteria: ≥90% eval pass-rate on curated cases for 2 weeks; no safety policy violations.

L1 - Execute with pre-approval

Scope: Executes actions after a human taps "approve."
Gates/controls: Mandatory review UI, per-action diff, spend estimate; rollback button.
Upgrade criteria: Reviewer acceptance ≥95%; average approval latency < 5 min; zero unapproved sends.

L2 - Templated autonomy

Scope: Executes within tight templates (e.g., support replies with citations; price updates with bounds).
Gates/controls: Guardrails (schema, policy allow-lists), rate limits, per-route cost caps, audit trail.
Upgrade criteria: KPI lift with no increase in incident rate; cost/unit stable; MTTR < 15 min on rollbacks.

L3 - Budget-bounded autonomy

Scope: Executes and spends inside pre-set budgets (tokens/media) with human gates at milestones.
Gates/controls: Daily and per-request spend ceilings with alerts; milestone approvals; canary traffic only.
Upgrade criteria: Meet targets on a 10–20% cohort for 2–4 weeks; zero critical incidents.

L4 - Canary end-to-end

Scope: Full mission slice (e.g., leadgen for one segment) end-to-end in a canary.
Gates/controls: Release gates tied to evals and SLOs; change-management; incident playbooks.
Upgrade criteria: KPI deltas hold at canary → 50% without regressions; budget variance < 10%.

L5 - Broad autonomy (policy-bounded)

Scope: Wide execution with policy and budget guardrails; humans handle exceptions and audits.
Gates/controls: Continuous monitoring, drift detection, periodic post-hoc audits, kill-switch.
Stay conditions: No systemic issues; compliance duties (oversight, logging) continue under EU AI Act where applicable. (Trilateral Research)

Control stack (what changes as autonomy rises)

Governance: Map decision rights; who can raise budgets, switch models, or expand scope. Align to NIST AI RMF's GOVERN/MAP/MESURE/MANAGE functions; make them visible in docs and dashboards. (NIST)
Oversight: Keep humans in or on the loop depending on level; enforce gates at external-facing milestones (brand, legal, financial). The EU AI Act requires human oversight and logging for high-risk uses - design for this from L1 upward. (Trilateral Research)
Management system: Treat the ladder as a management process (policy → plan → operate → review) per ISO/IEC 42001. (ISO)
Traceability: Capture inputs/outputs, tokens, cost, latency, model+prompt versions, tool I/O, and decisions for every action; this underpins audits and post-mortems recommended by NIST. (NIST Publications)

Reference workflow

flowchart LR
  A[Mission Spec] --> B[Risk & Scope Check]
  B --> C[Level Assignment (L0–L5)]
  C --> D[Controls & Budgets Loaded]
  D --> E[Canary Execution]
  E --> F[Evals & KPIs Gate]
  F -->|Pass| G[Scale Up Level or Traffic]
  F -->|Fail| H[Rollback & Post-mortem]
  H --> B

KPIs that prove you can move up a level

Reliability: p95 latency, error rate, MTTR, incident count/severity (weekly).
Quality: Eval pass-rate (faithfulness/accuracy), QA acceptance rate, policy-violation count.
Economics: Cost per outcome (tokens+infra+minutes), budget utilisation, unit margin.
Control health: % actions with approver, % actions with complete traces, audit findings closed.

30-day pilot to introduce the ladder

Week 1 - Frame & assign. Pick one mission and run a structured risk/scope check; start at L0/L1. Publish decision rights, budgets, and gates. Ground oversight in NIST AI RMF concepts (govern, measure, manage). (NIST)
Week 2 - Templated autonomy (L2). Add schemas, policy filters, rate limits, and per-route cost ceilings. Start a small canary; implement rollback.
Week 3 - Budget-bounded autonomy (L3). Introduce daily/per-request caps and milestone approvals. Track KPIs and evals; aim for MTTR < 15 min.
Week 4 - Canary E2E (L4). Expand to a full slice with release gates and post-mortems tied to traces. If you operate in the EU, verify human-oversight and logging align with the AI Act's high-risk controls before scaling. (Trilateral Research)

Compliance touchpoints (don't bolt them on later)

Human oversight & logging: Designate oversight roles; ensure logs capture what a regulator or auditor would need (who, what, when, why). This tracks with EU AI Act obligations for high-risk systems. (Trilateral Research)
Management system: Document policies, roles, competence, supplier controls, and continual improvement loops; that's the spine of ISO/IEC 42001. (ISO)

Notes on analogy

I draw the ladder idea from established autonomy taxonomies (e.g., SAE J3016's Levels 0–5 in driving) where capability rises while human responsibility changes form. The analogy is conceptual - our levels are for B2B agents, not vehicles - but the staged approach helps avoid "all-or-nothing" thinking. (SAE International)

Bottom line: Treat autonomy as something you earn with evidence. Define levels, enforce human gates, and tie every step to traces, evals, and budgets. With that discipline, you can increase agent throughput without losing control.