Human-in-the-loop AI: where people should stay in the loop

Human-in-the-loop AI means a person reviews, approves or corrects what a model produces before it takes effect. The hard part is deciding where. Put a person on every output and the AI saves nobody any time. Put a person nowhere and the first confident mistake lands on a customer. Good human-in-the-loop design puts people exactly where a mistake would be costly, and makes their review fast enough that they actually do it.

Two systems I've worked on recently shape how I think about this. On a compliance and risk platform I work on, an AI analyst reads KRI (key risk indicator) breaches and proposes reassessment conclusions for a human to accept. In the AI assistant for the Recond Flutter app, when the agent proposes creating a record, the user gets a create card to confirm or cancel. Both keep the model useful and leave the decision with a person.

Decide by the cost of a mistake

Before designing any review flow, sort each AI action by two questions: how bad is a wrong result, and can it be undone?

• Low stakes, reversible. Search results, suggested replies, draft text the user is already editing. Let it run. The user is the review.
• Low stakes, hard to reverse. Notifications, messages sent on the user's behalf. A quick confirm step is enough.
• High stakes, reversible. Records created or updated in a business system. Propose, then confirm.
• High stakes, hard to reverse. Compliance conclusions, payments, deletions, anything a regulator or customer will rely on. A person decides, and the model only proposes.

A risk reassessment falls in the last bucket. The analyst can do the slow reading and draft a conclusion, but the conclusion is a judgement someone in the organisation has to own. So the model proposes and a human accepts. That's the right split whenever accountability sits with a person.

Treat model output as a proposal

The pattern underneath both systems is the same: the model produces a proposal, and the proposal is a first-class object in your data model with its own state.

A proposal should carry:

• What would happen. The exact record, conclusion or change, rendered from validated data.
• Why. The evidence the model used and its reasoning, in a form a reviewer can check quickly.
• A status. Pending, accepted, edited, rejected or cancelled, with who decided and when.

The status matters more than it looks. In the Recond assistant, a cancelled create card stays cancelled. It can't be confirmed later by accident, and the agent can't keep re-offering the same action until the user gives in. Treat a human "no" as a final state for that proposal.

Showing the why is what makes review fast. On the same compliance platform, uploaded policy documents are evaluated against their template, and the system says why an analysis passed or failed. A reviewer who sees the reason can agree or disagree in seconds; one who sees only a verdict has to redo the analysis. I wrote more about that review in checking uploaded policies against a template.

Route with signals you can trust

Not every output needs the same level of review, so route work by risk. The temptation is to ask the model how confident it is. Self-reported confidence is a weak signal. Better signals come from code:

• Did the output pass schema and business-rule validation?
• Does it touch money, external parties or regulated data?
• Did retrieval find strong supporting sources, or nothing relevant?
• Do two independent checks agree?

type Route = "auto" | "review" | "reject";

type Proposal = {
  validationErrors: string[];
  affectsExternalParty: boolean;
  regulated: boolean;
  sourcesFound: number;
};

export function routeProposal(p: Proposal): Route {
  if (p.validationErrors.length > 0) return "reject";
  if (p.regulated || p.affectsExternalParty) return "review";
  if (p.sourcesFound === 0) return "review";
  return "auto";
}

The thresholds are product decisions, so write them down in the spec and expect to tune them once you see real traffic.

Design the accept, edit and reject flow

A review queue lives or dies on how quick it is to use. A few things I'd build in from the start:

• Accept in one action. If accepting a good proposal takes four clicks, people will batch-approve without reading.
• Edit in place. Most proposals are nearly right. Let the reviewer fix the one wrong field without starting over.
• Reject with a reason. A short reason code (wrong data, wrong conclusion, not needed) is cheap to collect and very useful later.
• Keep context on screen. Show the source material next to the proposal so the reviewer never has to hunt for it.

The edits and rejections are your best training and evaluation data. Every time a person changes a proposal, you've learned something about where the model is wrong. Feed those cases into your golden dataset, as described in how to evaluate an LLM feature before you ship it.

Watch for rubber-stamping

The quiet failure of any review step is that people stop reviewing. When the model is right most of the time, accepting without reading becomes a habit. Some ways to counter it:

• Track how long reviews take and how often proposals are edited. An approval rate near 100% with very short review times is a warning sign.
• Keep the queue small by auto-handling the genuinely low-risk work, so attention goes to the items that need it.
• Make the stakes visible on the item itself, so a regulated decision looks different from a routine one.

The short version

Sort each AI action by how costly and how reversible a mistake is, and put people where both are high. Model output becomes a proposal with evidence and a status, and a human "no" is final. Route with code-based signals, make accept and edit fast, and use every correction as a test case. The technical side of enforcing this is in guardrails for AI agents in production.