Agent Atlas / Chapter 4

Loops, Workflows, and Recovery

Repeated model-tool interaction is only one control structure. Production runtimes combine probabilistic loops with host-defined workflows, queues, interrupts, terminal states, and recovery operations.

Learning objective

Use public evidence to answer: Who decides the next step, and what happens when execution must stop or change direction?

Working question

Who decides the next step, and what happens when execution must stop or change direction?

A loop lets observations influence the next proposal. A workflow constrains which transitions are possible.
A recovery workflow shows load, model proposal, policy gate, bound execution, and an environment effect branching into either an observed outcome or an unknown outcome that must be reconciled before retry.
Workflow edges around a model loop The workflow owns admissible transitions and blocks retry when an external effect may have committed without a known outcome. Open full-size diagram
Case file 04 / Rejection becomes input

A denied tool call must not become an automatic retry

LangChain commit 17d1c274 changes the default human-in-the-loop rejection message. The model is told that the tool did not execute and should not retry the same call unless the user asks. An integration test resumes after rejection and asserts that the agent does not interrupt for the tool again.

The host workflow owns the interrupt, rejection decision, and resume transition. The model loop chooses what to propose after receiving the rejection. Policy prevents execution; the message guides later behavior. Those are different controls.

Latest newsroom receipts

What changed since the first Atlas draft

Mechanism trace

What actually happens

  1. Load run state

    identity, queue, budgets

  2. Propose action

    probabilistic next step

  3. Gate transition

    authorize, block, pause

  4. Bound executor

    invoke selected principal

  5. Environment effect

    may commit before reply

  6. Observed outcome

    known success or failure

  7. Unknown outcome

    effect may exist

  8. Reconcile before retry

    lookup, dedupe, compensate

A loop is not a workflow

In an agent loop, observations return to a model that may propose another tool call, produce an answer, or stop. The next action is partly probabilistic. A workflow instead defines host-owned transitions through a sequence, state machine, queue, event handler, or graph.

A deterministic workflow may contain probabilistic model calls without surrendering control of its edges. That distinction determines where budgets, stop conditions, approvals, and recovery rules can be enforced.

Repeated execution requires explicit state ownership

Codex replaces an override side channel with queued ThreadSettings and moves pending input, next-turn items, and mailbox coordination into InputQueue. Ordered state prevents later iterations from observing an accidental mixture of settings and input.

Concurrent work also needs narrower identity than a session ID. Crush carries a caller-supplied RunID through error events and ignores failures from unrelated prompts sharing the event channel. Without that boundary, one run can capture another run's failure.

Reject, cancel, rewind, and compensate are different

A rejection means an action was not authorized. Cancellation means active work must receive a termination path. Codex keeps interrupted shell dispatch alive long enough to send SIGTERM, wait briefly, and then hard-kill remaining descendants.

Hermes undo marks transcript rows inactive, retains them for audit, evicts cached state, and rebuilds from active history. It rewinds conversational state, not files, API calls, or processes already changed. External effects need separate compensation.

A timeout creates another state: the effect may have committed even though the runtime never observed success. Safe recovery needs an explicit unknown-outcome state, idempotency keys or deduplication, and an effect journal before any retry.

Reliability belongs in transition tests

Tests should assert transition invariants, not merely inspect a final answer: a rejected tool is not re-invoked, a foreign RunID error does not abort the current run, cancellation emits a terminal state, and rewind excludes inactive rows from the next context.

A serious test rig records workflow, run, attempt, source revision, provider, and environment metadata. These controls separate model variance from orchestration races and test-environment failures.

Failure modes

Where the contract breaks

Unbounded retry

A denial or deterministic failure is interpreted as a transient observation.

Failure signalThe same call recurs after blocked or rejected outcomes.

Cross-run event capture

Session-wide events are treated as belonging to every concurrent run.

Failure signalA run stops after receiving a missing or foreign RunID.

Transcript rollback mistaken for world rollback

Removing messages is assumed to reverse performed actions.

Failure signalFiles or remote systems retain changes absent from active history.

Unknown outcome after timeout

The environment commits an effect but the runtime loses the success response.

Failure signalA retry begins without an idempotency key, effect lookup, or explicit unknown-outcome state.
Release tests

What the product must prove

  1. Label every transition as model-owned, workflow-owned, policy-owned, human-owned, or executor-owned.
  2. Enumerate budgets, stop predicates, terminal outcomes, cancellation paths, checkpoints, and irreversible effects.
  3. Replay denial, cancellation, foreign-event, and rewind traces; assert state and side-effect invariants.
  4. Force a timeout after an external commit; block retry unless the runtime can prove whether the effect occurred.

Projects In This Lesson

24 projects

Recent Reporting For This Lesson

6 articles