Human-in-the-Loop
Automation
Autonomous AI agents call tools, send emails, update records, make purchases, spawn other agents. They run unattended for long periods of time. Inevitably, teams are going to want to add some human-in-the-loop checkpoints into agent-driven workflows. But most companies are not deploying agents in a similar fashion to how they onboard and manage a new employee. Agents can easily get away with more than a human could, with no sign-off, no monitoring, no accountability.
AI prepares the work
Risk is checked
A person decides
People keep control of important decisions.
The risk is in the doing
If an agent just plans or drafts things, there’s little risk. But when they’re allowed to do the scary bits, like sending, deleting, publishing, and paying, there’s a lot of room for things to go badly.
Human-in-the-loop can be a safety net to keep agents on track, but only when implemented well. Here are some considerations:
Who reviews, and where
Who’s going to review agent output? Ideally, this person understands the task and is empowered to reject the agent’s actions. Otherwise, the human-in-the-loop becomes a rubber stamp, which is worse than having no human-in-the-loop because it looks safer than it actually is.
Where will they do it? Some sort of lightweight UI might make sense, but it’s possible agents will be doing work in domains that already have UIs, for example, reviewing sales leads. It’s unlikely anyone wants to context-switch to a new app to review agent actions. It’ll get forgotten. Approvals are best sent to Slack or email.
How much detail do they need to approve/reject actions? The reviewer must see the exact action and its arguments rather than a summary. Other times, it’ll be useful to dive into the details. The reviewer should have visibility into the raw inputs/outputs of tools called by the agent. The goal is to make approvals fast, meaning a couple of button-clicks at most. If changes are common, allow one-click updates of arguments/inputs.
Where a gate belongs
Where might you need human-in-the-loop for your workflow? We suggest considering:
How reversible are agent actions? Drafting an email is reversible. Payments can be refunded, but items can’t be unsold.
What’s the blast-radius of bad actions? Is the agent managing one account, or many? What’s the worst that could happen? How much of a headache is recovery?
Do board members/insurers/regulators ask for logs? If so, you’ll need detailed audit logs. We’ve seen this come up for payments, healthcare, and other highly-regulated industries. Log all actions by agents, such as the name of tools called, arguments passed to those tools, model and prompt used, human approvers, and timestamps. This is good practice anyway, but becomes critical in some domains.
Would bad actions make the news? If so, you’ll want a human-in-the-loop for those actions.
How the gates work
Once you’ve identified the scary actions:
Add logic around these actions, which we call gates, that pauses agent workflows. Gating logic should live outside the LLM, so that agents cannot talk them out of it.
None of this gets built from scratch. Paused workflows need to be resumable later, which means they must be durable. Also, it’s safer to check permissions just before execution rather than rely on permissions assigned when the task was initially enqueued.
Gates should be deterministic, meaning each tool-call gets allowed, blocked with a helpful error message to the agent, or queued for review.
Each stage should have an owner, with clear SLAs for response times. Otherwise, items will languish in queues.
Measure the behaviour of approvers. If nobody rejects anything, reviewers might not be taking it seriously. If they’re approving/rejecting dozens of items per day, gates might be in the wrong places.
What we do
If you’re curious how this could apply to your agent workflows, get in touch. We can:
- Review your agent workflow, step-by-step, and consider reversibility, blast-radius, logging, and news-worthiness
- Identify opportunities for human-in-the-loop and wire these into your workflow
- Add human-in-the-loop to existing agent workflows via Slack/email or build a simple approval UI if needed
- Add logging for agent workflows
Get in touch
This work is charged at a fixed price per phase.