AI agent or deterministic workflow? How to choose

When a team decides to add an LLM to a product, the first architecture question is usually framed as "should we build an AI agent?" The more useful version: should this be an AI agent or a deterministic workflow? Both use a model. The difference is who decides what happens next, and that one choice shapes your reliability, cost, testing and debugging for the life of the feature.

My default answer is a deterministic workflow, with an agent only at the specific step that needs one. Here's how I reach that decision.

The one question: who owns the control flow?

In a deterministic workflow, your code defines the steps and their order. The model is called at fixed points to do a bounded job: classify this, extract that, draft this section. Given the same input, the same steps run every time. The outputs of the model calls can vary, but the path doesn't.

In an agent, the model owns the control flow. It gets a goal and a set of tools, and on each turn it decides which tool to call next or whether it's finished. The path can differ on every run. I cover the mechanics in agentic AI explained for engineers.

Everything else follows from that split.

	Deterministic workflow	Agent
Path through the steps	Fixed in code	Chosen by the model each turn
Cost per request	Predictable	Varies with the number of turns
Latency	Predictable	Varies, often higher
Testing	Test each step on its own	Test whole trajectories
Debugging	Find the failing step	Read the transcript
Handles the unexpected	Poorly	Well

When a deterministic workflow wins

If you can draw the flowchart, write the flowchart. A workflow is the better choice when:

• The steps are known. Intake, extract, validate, store. The order never changes, so there's nothing for a model to decide.
• Mistakes are expensive. Fixed paths are easier to audit, and you can put a validation step or a human review exactly where it belongs.
• Cost and latency need to be predictable. A workflow makes the same number of model calls every time, so you can price and capacity-plan it.
• You need to explain the result. "Step 2 extracted these fields and step 3 rejected one" is an answer a support team can give a customer.

REQ Agent, a CLI I built that turns raw client discovery material into client, designer and dev-handoff documents, works this way despite the name. It makes two sequential LLM calls in a fixed order. The first turns raw input into an internal spec and a designer brief. The second turns those into the final client, design and dev outputs. Then code validates the generated documents for structure, feature coverage and design-to-dev token consistency before writing anything. The model does the hard language work at each step, and the code decides the sequence. I wrote up the details in turning client discovery notes into specs with a two-stage LLM pipeline.

When an agent earns its place

An agent is worth its unpredictability when the path genuinely depends on what's discovered along the way:

• Investigation tasks. Debugging, root-cause analysis, searching a codebase. What you look at next depends on what you just found.
• Open-ended user requests. An in-app assistant where users can ask about any record, in any phrasing, and the answer needs one lookup or five.
• Feedback loops. Run the tests, read the failure, change the code, run them again. The number of iterations isn't known in advance.

If you find yourself writing a workflow with dozens of branches to cover every case, that's a sign the model should own that part of the decision.

The hybrid I default to

Most production systems I'd build end up in between: a deterministic workflow on the outside, with model decisions confined to the steps that need them.

1. Router step. A single model call classifies the request into one of a few known paths. Each path is a fixed workflow. The model picks the lane, and code drives within it.
2. Bounded agent step. One step in the workflow is an agent with a narrow tool set and a small step limit, such as "find the relevant records". The workflow takes its output, validates it and carries on.
3. Code-owned checkpoints. Validation, permission checks and human approval live in the workflow around the agent, outside the model's reach.

This keeps most of the system testable and predictable, and spends the agent's cost only where it buys you something.

A checklist for deciding

Before choosing, I answer these:

• Can I list the steps in order today? If yes, start with a workflow.
• Does the number of steps depend on intermediate results? If yes, that step may need an agent.
• What does a wrong action cost, and can I undo it? The higher the cost, the more control belongs in code.
• Can I write test cases for it? Workflows are tested step by step; agents need recorded scenarios and trajectory checks, which I describe in how to evaluate an LLM feature before you ship it.
• What's the latency budget? Agents with several turns rarely fit inside a snappy request-response interaction without streaming.

Start fixed, then loosen

The cheapest path is to ship the deterministic version first and log where it fails. Requests that fall outside your known paths show you exactly where model-owned decisions would help. Moving a step from code to an agent later is easy. Pulling an agent back into a predictable workflow after users depend on its behaviour is much harder.

The short version

Decide who owns control flow before you write any prompts. Use a workflow when the steps are known and an agent when the path depends on what's found along the way. In practice, wrap a small, bounded agent inside a workflow that keeps validation, permissions and approvals in code.