AI agent guardrails in production: permissions and approval

An AI agent in production is a program that picks its own next action based on text you don't fully control. The guardrails around it decide whether that's useful or dangerous, and the ones that count are code: permission checks, budgets, confirmation steps and logs that hold no matter what the model decides.

These are the AI agent guardrails I put in place by default, including two from systems I've built recently: a permission gate on a compliance and risk platform I work on, and the confirm-or-cancel cards in the AI assistant for the Recond Flutter app.

Least privilege, enforced in code

Start from what the agent is for and register only the tools that job needs. An assistant that answers questions about records needs read tools. If it never gets a delete tool, no prompt injection can make it delete anything.

Then scope each tool:

• Tenant and user scope. In a multi-tenant system, the tool takes the tenant from the authenticated session, never from the model's arguments. The model shouldn't be able to type a different tenant id.
• The agent never exceeds the user. An agent acting for a user gets at most that user's permissions, and often fewer.
• Narrow tools beat general ones. update_job_status is safer than run_sql. You can validate, log and reason about a narrow tool.

Check permissions before the transaction opens

On the compliance and risk platform, the agent's permission gate is checked before the database transaction opens. That ordering is deliberate. A denied call should cost nothing: no connection checked out from the pool, no transaction started, no locks taken, nothing half-done to roll back. It also keeps the authorization decision in one place, ahead of all tool code.

Here's the shape of a tool executor built that way:

import { Pool, type PoolClient } from "pg";
import { z } from "zod";

type RunContext = { runId: string; userId: string; tenantId: string; permissions: Set<string> };
type ToolCall = { name: string; args: unknown };
type Tool = {
  permission: string;
  needsConfirmation: boolean;
  schema: z.ZodType;
  run: (db: PoolClient, args: any, ctx: RunContext) => Promise<string>;
};

declare const registry: Record<string, Tool>;
declare function audit(ctx: RunContext, call: ToolCall, outcome: string): Promise<void>;
declare function proposeAction(ctx: RunContext, call: ToolCall): Promise<void>;

const pool = new Pool();

export async function executeTool(ctx: RunContext, call: ToolCall): Promise<string> {
  const tool = registry[call.name];
  if (!tool) return `Error: unknown tool "${call.name}"`;

  if (!ctx.permissions.has(tool.permission)) {
    await audit(ctx, call, "denied");
    return `Error: not permitted (${tool.permission})`;
  }

  const parsed = tool.schema.safeParse(call.args);
  if (!parsed.success) return `Error: invalid arguments: ${parsed.error.message}`;

  if (tool.needsConfirmation) {
    await proposeAction(ctx, { name: call.name, args: parsed.data });
    return "Proposed to the user for confirmation. Do not retry.";
  }

  const client = await pool.connect();
  try {
    await client.query("BEGIN");
    const result = await tool.run(client, parsed.data, ctx);
    await client.query("COMMIT");
    await audit(ctx, call, "ok");
    return result;
  } catch (err) {
    await client.query("ROLLBACK");
    await audit(ctx, call, "error");
    return `Error: ${(err as Error).message}`;
  } finally {
    client.release();
  }
}

Permission, then validation, then confirmation, then the transaction. Errors go back to the model as text so it can explain the refusal to the user.

Budgets: steps, tokens and time

Every agent run needs hard limits, because a confused model will happily loop:

• A step budget. A maximum number of model turns per run. Ten is a sensible starting point for most assistants.
• A token budget. Track usage from your provider's response and stop the run when it passes a ceiling.
• A wall-clock timeout. Per model call and per run. A hung request should fail cleanly.
• A per-user rate limit. One user, or one runaway script, shouldn't be able to spend your whole monthly allowance.

When a budget runs out, stop and tell the user what was done so far. A partial answer with an honest explanation beats an endless spinner.

Human approval for anything that writes

The model can propose a write. A person should confirm it. In the Recond assistant, when the agent proposes creating a record, the app shows a create card with the details, and the user confirms or cancels it. Nothing is written until they confirm.

Two details matter here:

• A cancelled card stays cancelled. The agent can't quietly re-propose the same action until the user gives in, and a stale card can't be confirmed later by accident.
• Render the card from the validated arguments. The user should approve the exact record that will be created. A summary written by the model could drift from what actually gets saved.

How much to gate depends on what a mistake costs. I go through that decision in human-in-the-loop AI design, but my rule of thumb: reads run freely, writes get confirmed, and irreversible actions like deleting, sending or paying always need a person.

Audit logs you can replay

Log every run under an id: the user, the tenant, each model call, each tool call with arguments, the permission decision, the outcome and the duration. Two reasons:

1. Debugging. Agent bugs are trajectory bugs. You need the transcript to see where it went sideways.
2. Accountability. When someone asks "why did this record change?", you can answer with the run, the proposal and the confirmation.

Be careful with what you store. Tool arguments and results can hold personal data, so apply the same retention and redaction rules you use for the rest of your logs.

Treat everything in the context as untrusted

Documents, emails, web pages and tool results all land in the model's context, and any of them can contain instructions aimed at the model. Your guardrails have to hold even when the model has been talked into something. That's why the checks above live in code, outside the model's reach. The rest of that threat model is in prompt injection: securing LLM features in web apps, and the basics of the loop itself are in agentic AI explained for engineers.

The short version

Register only the tools the job needs and scope them to the user and tenant. Check permissions before any database work starts. Cap steps, tokens and time. Make writes go through a confirm card the user can cancel for good, and log every run so you can replay it.