Agent Loop

A model is fully capable of writing a patch that looks complete in a single response, which is exactly where the question "if it can be written in one pass, why go back and forth in a loop" comes from. But between "looking complete" and "actually usable" lies an entire layer of facts that the model cannot see at generation time. Before generation, the model does not know the repository's actual directory structure, the exact function signatures, or what the tests will output at that moment; after generation, it also does not know whether the patch can actually be applied, or whether the tests actually pass. A single long answer can only put these unknowns into the text as guesses: no matter how detailed the text is, a guess is still a guess.

What the loop changes is the way information is acquired. It no longer asks the model to "think up" all the facts at once before acting; instead, it first takes a small action, has the environment return new facts, and then chooses the next step based on those facts. Thus, "maybe the file is here" becomes the real list returned by a directory command, and "maybe it's fixed" becomes the real exit code of a test process. Assumptions in the text are replaced one by one by observations from the environment—this is exactly what single-shot generation cannot do: a single generation cannot see the real result after an action, while a loop obtains new facts round by round through small actions, gradually reducing uncertainty.

The criterion for judging whether a loop is making progress is information gain. Every round should ask: Did it obtain new observations? Did it narrow the range of candidate causes? Did it change the verifiable state? If none of the three happened, running one more round is just repeated sampling—the output text may differ, but certainty has not increased. That is not progress; it is just an expensive redraw. Conversely, as long as some round brings new observations, reduces candidates, or changes the verifiable state, the loop has done work that single-shot generation cannot replace.

Laid out, the loop's interface is: inputs are the goal, the unknown environment state, the available set of actions, and the acceptance criteria; outputs are the round-by-round observations, actions, and verification states. The goal and acceptance criteria run throughout; the environment state is continuously updated with each round's observations; and actions are re-selected each round based on what is currently known. This structure explains the fundamental difference between a loop and single-shot generation: single-shot generation treats environment state as assumptions in text, while a loop treats environment state as something that can be queried round by round.

The applicable boundaries are equally clear. A loop promises neither faster nor cheaper. When all the key facts of a task are already determined before generation and the results of actions are completely predictable, a loop provides no information gain, and single-shot generation is sufficient; the value of a loop lies precisely in scenarios where the environment state is unknown and action results are unpredictable. Conversely, in such scenarios, relying on repeated calls without information gain to "try a few more times" does not get you closer to the answer than single-shot generation.

In a single loop, who exactly is thinking, who is acting, and who finally says "passed"? Imagining the AI Agent as an all-powerful brain hides the answer: inside the loop are actually four roles each holding their own state and respecting their own boundaries in a relay; each role can do one kind of thing and is explicitly forbidden from another kind.

The model holds the goal summary, historical observations, and the list of available tools; its responsibility is to propose candidate actions and their parameters; its boundary is that it cannot directly write files or grant itself permissions. The controller holds the turn count, budget, protected constraints, and action history; it is responsible for parsing actions, scheduling execution, pausing, and terminating the entire loop; its boundary is that it cannot treat model text as authorization evidence. The tool executor holds the actual file and API state; it executes actions according to the schema and permissions; its boundary is that it cannot accept unvalidated free-text commands. The verifier holds tests, policies, and acceptance rules; it provides reproducible pass/fail evidence; its boundary is that it cannot be arbitrarily weakened by the candidate solution.

The causal chain of this division of labor points to the same conclusion: the model saying "done" is only a prediction. The completed state is not announced by any single role; it is only truly entered after the controller observes that protected tests pass, the working tree diff matches the expected range, and there are no pending side effects. Why must this be the case? Because the model holds only the goal summary, observations, and tool list, not the actual file and API state, and it has no write permission; its judgment of its own output and the conclusion drawn by the verifier based on protected standards are two different things; prediction needs evidence to adjudicate. The controller holds the budget and constraints, and is the only role that simultaneously has the action history and validation results; the executor provides the real state; the verifier provides evidence using standards that cannot be weakened by the candidate. The respective "cannot" of the four roles is precisely to separate the four things: proposal, authorization, execution, and adjudication.

From the interface perspective, the inputs of this role architecture are the protected goal, candidate actions, actual resource state, and acceptance rules; the outputs are the model proposal, control decision, tool result, and validation conclusion. The model does not directly write resources, the controller manages budget and authorization, the executor operates according to the schema, and the verifier adjudicates using standards that cannot be weakened by the candidate. The four types of outputs are connected end to end: the model proposal is only executed by the executor after being adjudicated by the controller, the execution result becomes the input of the verifier, and the validation conclusion returns to the controller, which decides the next action or terminates the loop.

The pitfall of natural-language Agent is that if the model is left to improvise in each round, constraints such as goals, budgets, and authorizations can drift or even be lost as the conversation proceeds. Turning it into a state machine gives the loop an explicit state definition and a deterministic transition rule.

The state consists of five components: the protected goal and acceptance criteria G, the latest observation O, the compressed action/result history H, the remaining step count and cost budget B, and the current authorization A. The state at round t is written as

Sₜ = (G, Oₜ, Hₜ, Bₜ, Aₜ)

Each component has its own owner: once G and the acceptance criteria are set, they are no longer modified by any action within the loop; Oₜ is the fact most recently returned by the environment; Hₜ is the compressed record of actions and results; Bₜ is the steps and cost not yet consumed; and Aₜ is the authorization currently in effect.

The transition has three steps. The first is proposal: the policy proposes an action only from the current state, that is, aₜ = policy(Sₜ). Here the model only produces a candidate action and cannot directly send aₜ to execution; before execute, the candidate must also undergo a permission check to confirm that it falls within the authorization scope of Aₜ. The second is execution and observation: the tool executor executes the action according to the schema, and the environment returns the result rₜ. The third is update: update uses aₜ and rₜ to refresh the observation and history and obtains the next state

S₍ₜ₊₁₎ = update(Sₜ, aₜ, rₜ)

The content returned by tools can never overwrite G, B, or A. This is the key to the state machine not losing constraints: no matter what the model says or what a tool returns, the protected goal, budget, and authorization can only be modified by the controller according to its own rules.

Figure 1 shows this path of state, decision, execution, and verification: the model is inside the control loop but is not the controller itself. External observations can update facts and plans but cannot rewrite goals, budgets, or authorizations. That is, environment information can change "what to do next" but cannot change "why do it" or "how far it can go".

The final engineering requirement: states must be versioned. Only if each Sₜ carries a traceable version can we return to a particular round in history and reproduce the basis for that decision at the time. A state machine without versioning can only run forward; when something goes wrong, it cannot locate the round from which the deviation began.

Faced with the same test failure, blindly changing code only creates new noise; the difference with four-round repair is that each round uses new evidence to narrow uncertainty. The following set of run logs shows how the state updates round by round.

In round 0, the model has only the goal G, one failure signature 100 ≠ 90, and a 6-round budget. It does not modify code directly; instead it first searches for the location of the discount calculation and related tests, obtaining two new facts: the suspicious formula is price * (1-discount), and the test input is discount=10.

Round 1 pieces these two facts together: discount=10 and 90 in the failure signature appear together, so 10 probably means "ten percent", but the code treats it as a decimal between 0 and 1 — substituting price=100 into price*(1-discount) yields 100*(1-10)=-900. This illegal negative result is caught by the code's guard and the price is returned unchanged; the actual output 100 that the test sees is exactly this fallback path, while the expected value 90 corresponds to a ten percent discount on price=100. Based on this, the model proposes changing the formula to 1-discount/100, making 100*(1-10/100)=90 match the expectation. The patch is applied successfully; at the same time, authorization only allows modifying source files, which sets the scope for later verification — test files must not be touched.

Round 2 sees that the working tree contains only one source-code difference, so it runs the target test. The exit code is 0, and the target test passes 1/1. But this only proves that the fixed assertion now holds; it does not prove there are no regressions elsewhere, so the uncertainty is merely reduced, not eliminated.

Under the conditions that the target test passes and 3 rounds of budget remain, round 3 runs the full test suite and checks the diff: 48/48 all pass, the test files have not been changed, then it generates a patch summary, and the loop stops at "waiting for release confirmation".

To quantify progress, count the number of unsatisfied conditions Uₜ: initially 3 — target test fails, full test suite fails, and diff scope is unconfirmed; after round 2 it drops from 3 to 2; after round 3 it drops to 0. Only a decrease in Uₜ, or obtaining new evidence that can change the diagnosis, counts as measurable progress; release authorization is not among these three technical conditions, and cannot be automatically derived from "all tests green". This is clear in the example: after 48/48 pass, the loop still stops at "waiting for release confirmation" instead of declaring it released.

The complete interface for this case is: inputs are the failure signature 100 ≠ 90, a six-round budget, prohibition on modifying tests, and release requires separate confirmation; outputs are the localization conclusion, minimal patch, target test passing, full regression passing, and pending-release status. Each round relies on new facts to lower Uₜ, from 3 all the way down to 0; all technical acceptance checks turn green, yet release authorization still cannot be derived. Conversely, if a round neither reduces Uₜ nor produces new evidence that can change the diagnosis, then that round has made no progress.

Three questions distinguish the three mechanisms: Plan answers "What are the dependencies, and roughly what order should we do them in?", Loop answers "Does a new observation require changing the next step?", and Verifier answers "Does the result satisfy externally checkable conditions?". Each fills a gap, and each has a typical symptom when missing.

First, why do we still need a Loop when we have a Plan? A Plan provides direction and a rough order; it is born at a moment when environmental information is incomplete. As soon as the real environment returns an observation that doesn't match expectations, rigidly carrying out the original plan only leads further and further astray. The Loop's job is to absorb this feedback and make subsequent actions respond to the environment: when a new observation appears, change the next step; only when the observation matches expectations do we continue in the original direction. Therefore, Plan and Loop are not substitutes; rather, "Plan provides direction, Loop absorbs feedback".

Next, why do we still need independent verification when we have a Loop? A Loop can make actions increasingly fit the environment, but it cannot answer "Does the result really satisfy the contract?" — the model may perfectly well give a fluent explanation instead of actually passing. The Verifier's role is to make an externally checkable judgment on the result, rather than trusting the model's self-report.

When any one of the three is missing, the symptoms are distinct: without a Plan, the Agent only makes local reactions, easily taking detours and missing steps; without a Loop, the plan is still rigidly executed when the real environment changes; without a Verifier, the model substitutes a fluent explanation for actually passing. The robust pattern is to chain the three together: Plan provides direction, Loop absorbs feedback, Verifier adjudicates.

The Verifier must also be protected. If an Agent can delete failing tests or change the passing threshold from 100% to 50%, it can pass by modifying the referee rather than completing the task — the evaluation is contaminated and the verdict is meaningless. Therefore, the verification criteria must be placed outside the loop; the Agent can only accept the verdict, not rewrite the criteria.

To consolidate the interfaces of this division of labor: inputs are task dependencies, new observations, and external acceptance; outputs are Plan direction, Loop adjustment, and Verifier verdict. Plan describes the rough order, Loop makes subsequent actions respond to the environment, and Verifier judges whether the result satisfies the contract; missing any component respectively leads to detours, rigidity, or false completion.

The loop cannot consist of only two states, "success" and "keep running". There are five categories of reasons for stopping, each triggered by explicit conditions and decided by the controller, not by the model itself.

Success termination: all machine-checkable acceptance conditions are satisfied. This is the only exit caused by goal achievement; the decision is based on reproducible evidence supplied by the verifier.

Budget termination: steps, time, tokens, or cost reach a hard limit. Budget is non-negotiable by design—it is not a recommended value but a hard boundary enforced by the controller; the model cannot ignore it on its own.

No-progress termination: state summaries and actions are equivalent for k consecutive rounds, or progress indicators no longer improve. This corresponds to the information gain criterion mentioned earlier: if there are no new observations, no candidate reduction, and no verifiable state changes, the loop should not continue consuming resources.

Dependency failure: tools are unavailable, the test environment is damaged, and retries cannot produce new information. At this point the problem is not in the Agent's strategy but in the environment itself; continuing the loop yields no incremental progress, so it can only exit and report the dependency status.

Authorization suspension: publishing, deleting, paying, or expanding privileges requires a new decision by the user. Such actions are beyond the scope of the current round of authorization; the loop must stop and wait for the user, rather than trying to replace the authorization decision with a technical conclusion such as "all tests green".

Putting the five exits side by side reveals a common principle: "try again" is not by itself a termination strategy. Retrying is worthwhile only when it carries new parameters, backoff, alternative tools, or new evidence; otherwise it merely replicates the same failure into the next round. Accordingly, hard budgets and authorization gates must be counted and enforced by the controller—the controller's action history records rounds and budget, and it is also responsible for detecting equivalent action loops, triggering no-progress termination when there is no substantial change for k consecutive rounds.

The interface of the termination strategy is: inputs are acceptance conditions, steps/time/tokens/cost, progress history, dependency status, and authorization requirements; output is one of five exits—success, budget exhausted, no progress, dependency failure, or authorization suspension. The model can suggest "should stop" in the status summary, but it cannot ignore these exits on its own; whether to leave the loop and in what way to leave is always the controller's decision.

When the loop gets stuck, the fault may hide in any link of decision, tools, state, or verification. The way to distinguish them is to look at observable signals in the trajectory, then apply targeted repairs to each failure.

The signal of action oscillation is the most intuitive: A→B→A oscillation in the trajectory, files repeatedly changed back to their original state. This is because the decision layer continues sampling even when there is no new information; the repair is state deduplication and prohibiting repeated unproductive actions—the controller identifies equivalent actions accordingly and blocks replay.

Error observation pollution is more hidden: a tool timeout is summarized as "no result", and the next round treats "no result" as a fact to continue reasoning. The key to repair is to preserve structured raw information before it enters the model: status code, source, time, and the original error text—rather than allowing a failure to be compressed into a harmless description.

Goal drift manifests as later summaries quietly omitting key constraints, such as "do not modify tests" disappearing from the summary. The repair is to put goals and constraints into non-compressible state—they do not participate in history compression, so no matter how the summary is simplified, constraints will not be lost.

The signal of pseudo-completion is natural language claiming success, but with no test record. The repair is to bind the termination condition to verifier evidence and artifact hash: without reproducible passing evidence, it is not considered complete; self-report cannot replace adjudication.

Privilege escalation manifests as a sudden change in the operation sequence: after a series of read operations, it suddenly requests to publish credentials. The repair is per-action authorization: any authorization escalation triggers a pause, waiting for user decision.

From these five groups of correspondences, we can derive the records needed for debugging the loop: state version—candidate action—validation decision—tool result—verification conclusion. This chain is sufficient to reproduce each round of decisions and locate which link has the fault; there is no need to save or display the complete private chain of thought. Observability serves reproduction and attribution, not collecting as much natural language as possible.

The diagnostic interface is: input state version, candidate action, validation decision, tool status code, result source, and verification evidence; output attributes the fault to one of action oscillation, observation pollution, goal drift, pseudo-completion, or privilege escalation. After attribution, the repairs correspond one-to-one: state deduplication, preserve original errors, non-compressible goals, evidence-bound termination, and per-action authorization. The entire process does not require the complete private chain of thought.

Starting from "a single generation cannot see the real result" we can step by step derive why a reliable Agent Loop must be a constrained, observable, and terminable state machine.

The starting point is the blind spot of single-shot generation: the model can only guess the environment state based on the context before the action; no matter how long the text is, the guess cannot become fact. To break this blind spot, someone must actually execute and observe, so four roles emerge: the model, the controller, the tool executor, and the verifier. Separating them enforces responsibility boundaries—who proposes, who authorizes, who executes, who adjudicates, each in its place.

After the roles are separated, the loop's operating rules are established: actions produce observations, observations update facts and plans, but cannot overwrite protected goals and authorizations. Environmental feedback can change what to do next, but cannot change why it is done or how far it can go. This invariant is the foundation that keeps the loop from drifting across iterations.

With rules in place, the loop's value truly exceeds repeated generation: round-by-round new evidence reduces unmet conditions—goal tests, full test suite, and diff scope turn green one by one, and uncertainty is narrowed round by round; repeated generation is just an expensive resampling of the same set of unknowns.

To keep this process running stably, three things must each perform their own function: the plan maintains direction, the loop adapts to change, and the verifier adjudicates completion. Direction, feedback, and adjudication are all indispensable; otherwise they lead respectively to detours, rigidity, and false completion.

The loop must also have exits, and exits are not limited to "success": success, budget exhausted, no progress, dependency failure, and awaiting authorization are all legitimate exits. Exits are enforced by the controller based on budget count, progress history, and authorization status; the model cannot ignore them on its own, nor can it use "try again" to delay indefinitely.

Stringing this chain together, the conclusion naturally emerges: a reliable Agent Loop is essentially a constrained, observable, and terminable state machine. Constrained means that goals, budget, and authorization are not overwritten by actions within the loop; observable means that state versions, candidate actions, validation decisions, tool results, and verification conclusions are all traceable; terminable means that the five types of exits are forcibly enforced by the controller. Only when all three hold can the Agent be considered reliable, rather than just a model that continuously outputs.