Flow Engineering: How to Make Your AI Agents 10x More Reliable

The difference between a reliability score of 0.53 and a score of 0.67 on the same task, achieved by restructuring the workflow around the model rather than changing the model itself, is the entire argument for flow engineering stated as a number. The model did not get smarter. The procedure around it got clearer. That single insight saves more production failures than switching to a better model does, and it is available to any team willing to do the upfront design work.

Map the full procedure as a flowchart before writing any prompt

The first and most important step in flow engineering happens before you open a code editor or write a single prompt. It happens on paper, or in a diagramming tool, as a complete flowchart of every step a human would take to complete the task correctly.

This step is where most AI workflow failures originate. A developer decides to automate a process, describes it loosely in a system prompt, and sends it to a model. The model produces something that looks approximately right in a test. Then it goes into production and starts failing in ways that are hard to predict and harder to debug, because the procedure was never made explicit. The model was asked to infer a process that was never clearly defined, and it filled the gaps with its own assumptions.

Mapping the procedure as a flowchart forces a precision that a sentence cannot. You cannot draw a box labeled "handle the exceptions." You have to specify which exceptions occur, what happens when each one does, where the flow goes after the exception is handled, and how the rest of the process knows the exception occurred. That precision, which feels like overhead before the build, is what separates a flow that works from one that needs to be rebuilt every time it encounters an unfamiliar input.

The flowchart should capture every decision point where the outcome could branch in more than one direction, every step where the model's output needs to be verified before the flow can proceed, every step where a failure should trigger a retry rather than an error, and the end state for every path through the flow including the paths that end without a clean result.

An accounting firm building an agent to answer routine data questions from its financial records would map the flow before writing any code. That map would show: a step that identifies which tables are relevant to the question, a step that retrieves the actual schema for those tables, a step that drafts a query using verified column names, a step that checks the query for logical errors before execution, an execution step, a step that checks whether the result is plausible for the kind of question asked, and a routing step that sends implausible results back to the drafting step for another attempt. Six distinct steps, each with one job, before a single line of code is written. The map is the design. The code is just the implementation of what the map already determined.

Break the flowchart into nodes, each with one clear job and one clear output

Once the flowchart exists, the next step is converting each box into a node. A node is a single step in the flow with one clearly defined job and one clearly defined output. Not two jobs. Not an output that might take one of several forms depending on what happened inside the step. One job, one output, every time the node runs.

The single-job discipline is what makes a flow debuggable in production. When a combined step that handles multiple things produces a wrong result, the source of the error is somewhere inside the combined processing and finding it requires reasoning about all of it simultaneously. When each node has one job, a wrong result localizes to the node that produced the wrong output. Debugging means inspecting that one step in isolation, with its exact inputs and its exact output, which is a tractable problem.

The single-output discipline is what makes the connections between nodes reliable. If a node sometimes returns a list and sometimes returns a single item depending on internal conditions, the node downstream has to be written to handle both cases, and the cases that were not tested against will produce unexpected behavior in production. A node that always returns the same structure lets the downstream node be written once, tested once, and trusted.

For the accounting firm's query agent, the table identification node has one job, identifying which tables the question requires, and one output, a list of table names. The schema retrieval node has one job, pulling the actual column names and types for those tables, and one output, a structured schema object. The query drafting node has one job, writing a SQL query using the verified schema, and one output, a SQL string. Each node's output is the exact input the next node expects. There is no ambiguity at any handoff.

Small nodes also make the flow easier to extend over time. When a new requirement appears, such as handling a new category of data source or a new type of exception, you add a node or modify one existing node rather than rewriting a large combined step. Flows built from small, single-purpose nodes grow cleanly. Flows built from large, combined steps become harder to change with every addition, which is how automations that started simple accumulate technical debt that eventually requires a full rebuild.

Define shared state as the minimum data each node actually needs to pass forward

State management is where flow engineering most clearly diverges from the way most people think about AI workflows. The instinct is to pass as much context as possible to every step, so the model always has the full picture. That instinct produces slow, expensive, and hard-to-debug flows.

The correct principle is minimal shared state. Each node receives only the data it actually needs to do its one job, nothing more. The node that drafts a SQL query needs the question in natural language and the verified schema of the relevant tables. It does not need the full conversation history that led to the question, the account information for the person who asked, the organizational context of the department that submitted the request, or the results of previous queries this session. Giving it those things does not improve the query. It increases the token cost of every run and introduces information that might cause the model to make assumptions that were not intended and are not detectable in the output.

Minimal state also makes individual nodes testable in isolation. If the query drafting node receives only a question and a schema, you can run it against any question and any schema and know exactly what you are testing. If it receives a full conversation history, testing requires reconstructing the full context for every test case, and the behavior of the node depends on information that is not visible in the node's defined inputs. Isolated testability is what allows you to fix one step without wondering whether the fix broke something in another step.

For the accounting firm's flow, the shared state object carries only what is needed at each step. The table identification step reads the question and adds a list of relevant table names. The schema retrieval step reads those table names and adds the schema for each. The query drafting step reads the question and the schema and adds a candidate SQL string. The verification step reads the query and the schema. The execution step reads only the verified query. The plausibility check reads the result and the original question. No step has access to more than what it requires. The state grows incrementally, one precisely defined addition per step.

This discipline reduces cost at scale. A free-form agent that reprocesses its entire conversation history at every step to decide what to do next consumes tokens proportional to the full length of that history at every step. A structured flow that passes only the minimum necessary state at each step consumes tokens proportional to the current node's actual inputs. For a flow running dozens or hundreds of times per day, that difference compounds into a meaningful cost reduction that compounds further as the volume scales.

Wire conditional edges that route around failures instead of stopping

A conditional edge is a connection between nodes that activates based on the outcome of the node before it rather than unconditionally. Conditional edges are what give a flow the ability to recover from failures rather than simply stopping and surfacing an error.

The most important conditional edge pattern in any AI workflow is the retry loop. When a step produces an output that fails a check, rather than surfacing an error message to the user or stopping the flow entirely, the conditional edge routes the flow back to the failing step with information about what went wrong. The step receives that information alongside its original inputs and produces a new attempt with the benefit of knowing what did not work the previous time. A counter tracks the number of retry attempts and stops the loop if the step has failed a defined number of times, routing to a different outcome rather than looping indefinitely.

For the accounting firm, the retry loop lives on the query drafting step. If the executed query produces a database error, or if the plausibility check determines the result does not match the kind of answer the question called for, a conditional edge routes the flow back to the query drafting node. The drafting node receives the original question, the verified schema, and the specific error or implausibility note from the failed attempt. It drafts a new query with that additional context. If three consecutive attempts all fail, the flow routes to a node that prepares a notification for a human staff member rather than delivering a wrong answer to the person who asked.

That routing is what separates a flow that handles real-world data from one that works only in controlled test conditions. Real queries fail against real databases. Real data has shapes the test cases did not anticipate. Real questions sometimes reference columns that do not exist by the assumed name. A flow without conditional edges stops on the first failure. A flow with conditional edges recovers automatically and involves a human only when automated recovery is exhausted.

Conditional edges also handle the branching logic that makes a flow useful across a range of inputs. A question answerable from a single table takes a different path than one requiring a join across multiple tables. A result that passes the plausibility check takes a different path than one that fails. Those branches are not error conditions. They are the normal variation of real use, and conditional edges route each case to the appropriate handling rather than pretending every input is identical.

Add a verification node before any output leaves the system

The verification node is the single most valuable addition to any AI workflow that produces output a human will act on, make decisions from, or deliver to a client. It sits immediately before the final output step and checks whether the output meets a defined set of criteria before it is allowed to proceed.

What a verification node checks depends entirely on what the flow produces. For a SQL query agent, the verification node checks whether the result is structurally plausible for the question asked. If the question was about total payroll for last quarter and the result is a list of individual transaction records rather than an aggregated total, the verification node catches that before it reaches the staff member who asked. If the question was about accounts past sixty days overdue and the result contains current accounts mixed in, the verification node catches that before a report is generated with incorrect data.

The verification node is not the model checking its own work in the same context it produced that work. It is a dedicated step with its own context, its own explicit check criteria, and its own output that is either a pass or a specific description of what failed and why. That separation is important because a model reviewing its own output in the same session context is less likely to catch systematic errors than a model reviewing with fresh context and explicit criteria defined for that check.

For the accounting firm, the verification node receives the original question, the schema of the tables queried, and the query result. It checks whether the result's structure matches what a correct answer to that question should look like, whether the row count is plausible, whether the values are in expected ranges for that type of data, and whether any aggregations the question implies are present in the result. It produces a structured output: pass, meaning the result moves forward to be formatted and delivered, or fail, with a description of which check failed and what was observed. That fail output is what the conditional edge reads to route the flow back to the query drafting step for another attempt.

The measured improvement from a score of 0.53 for a free-form agent to a score of 0.67 for a structured flow on the same task is attributable to the combination of the defined procedure, the retry loop, and the verification step working as a system. The procedure defines the path. The retry loop handles recoverable failures. The verification node catches errors before they leave the system. No single one of those elements produces the improvement on its own. They work together, and together they add fourteen percentage points of reliability on a task neither the model nor the procedure changed.

Test the main path until it is clean, then add branches one at a time

Testing a flow is not the same as testing a prompt. A prompt test gives you one exchange and one output to evaluate. A flow test requires running the full sequence of nodes on a real input and checking not just the final output but the output of every intermediate step, because an intermediate step producing a wrong output in an unexpected way can propagate errors forward that only surface in unexpected ways at the end.

The testing order that produces the least debugging overhead is the main path first. The main path is the set of nodes that execute when everything goes as expected: no retries, no alternative branches, no exception handling. Run that path on real inputs until every node in the sequence produces the right output consistently and the final result is correct. Only then add the first branch.

The reason for this order is isolation. Every new branch is a new possible source of failure. If you add multiple branches at once and something fails, the failure could be in any of them and diagnosing it requires reasoning about all of them simultaneously. Adding one branch at a time means any new failure is almost certainly in the branch you just added, which makes the diagnosis immediate.

For the accounting firm, the main path test runs a set of twenty real questions drawn from the actual data requests the staff receives each week. The questions cover payroll queries, accounts payable status, balance queries, and period-over-period comparisons. Running twenty real questions across the main path before adding any branches reveals whether the core procedure is solid and which categories of question the flow handles correctly by default.

Once the main path is clean across the full test set, the retry loop is the first branch to add. The retry loop matters most for production reliability, because it handles the cases where the model's first attempt is recoverable rather than terminal. Add it, then rerun the full test set to confirm the clean cases still work correctly and the first-attempt-failure cases now recover as designed.

The result of that disciplined testing sequence, for the accounting firm, is a flow that handles the large majority of routine data requests correctly and automatically, reduces the time staff spend on manual data lookup from hours to minutes per week, and surfaces the genuinely complex requests to a human reviewer rather than generating a plausible wrong answer. That is a practical translation of what the score improvement from 0.53 to 0.67 means: not just a better number, but fewer errors reaching the people who depend on the answers being right.