AgentCanvas / Pages / Developer Guide / Design Docs / Graph / Graph Executor
2026-10-17

This page is in two parts. Part I teaches the concepts with diagrams. Part II prints the engine as one continuous annotated listing β€” colored region bands segment run() so you can see at a glance which block does what β€” keyed to agentcanvas/backend/app/agent_loop/graph_executor.py.


Part I Β· Concepts β€” for people Two viewpoints on one example: first the graph you draw (Β§1), then the engine that runs it (Β§2–§5).
View 1 Β· The graph β€” what you draw The contract a drawn graph makes: when nodes fire, and which loop structures the engine recognises. In Β§1's figures, every box is a node of one concrete graph.

1. The drawn graph and its contract

A canvas graph is nodes joined by typed wires. Its execution semantics rest on exactly two things: a firing rule that decides when any node runs (Β§1.1), and one recognised structure β€” the loop pivot pair β€” that expresses iteration, halting, and the episode verdict (Β§1.2). Everything else on the canvas is ordinary dataflow. This section states the contract; Β§2–§5 describe the algorithm that honours it.

1.1 The firing rule

The firing rule. A node fires when every required input port holds a value. The fire consumes those inputs and produces an output dict; the engine writes each output onto its outgoing wires, filling downstream ports β€” which may make further nodes ready. Fire β†’ propagate β†’ repeat, until nothing is ready.
rgb βœ“ state β€” predict waits β€” a required input is empty state arrives rgb βœ“ state βœ“ predict env_step viewer fires β€” outputs fan out in one pass, consumers become ready

Three consequences follow, and they shape everything below:

1.2 Anatomy of a loop

Plain dataflow ends when data stops flowing β€” enough for a pipeline, not for an agent that must observe, act, and observe again. Iteration is expressed by the one structure the engine recognises: a paired IterIn / IterOut, the loop pivots. The smallest loop:

loop-carry β€” handed back by pairing, not a drawn wire entry nodeinit_* Β· run-invariants IterInopens each iteration predict env_step IterOutcloses each iteration done β†’ stop final_* Β· once evaluate graphOut wire you draw engine-timed

Reading the figure in run order:

  1. Run start. The entry node fires once β€” it needs no inputs β€” and its init_* edges hand the run-invariants (instruction, episode config) to the IterIn's init side.
  2. Each iteration. The IterIn emits the loop-carry bundle; the body β€” here predict and env_step, ordinary nodes with no special treatment β€” computes; the IterOut collects the next carry. The engine hands the carry back to the IterIn by pairing: the top edge is not a wire you draw, and the drawn graph stays acyclic.
  3. Halting. Any BOOL wired into the IterOut's stop input β€” here the env's done β€” ends the loop after the current iteration. Left unwired, the loop runs to its step budget.
  4. Termination. The IterOut's final_* handles mirror its carry ports (plus a constant final_stop). They are real wires, but deliver exactly once β€” the terminal iteration's values.
  5. Verdict. Everything downstream of final_* β€” here evaluate and graphOut β€” is the after-loop band: it runs once, after the loop, and may not take inputs from the loop body.

The drawing therefore has three places, and a run three times β€” the map that View 2 walks in order:

Place in the drawingWhen it runsEngine view
init_* edges into the IterInonce, at run startpre-loop β€” Β§3
the body between the pivotsevery iterationin-loop β€” Β§4
downstream of final_* β€” the after-loop bandonce, at terminationafter-loop β€” Β§5
View 2 Β· The engine β€” how run() executes it Β§2–§5: where the engine sits, then the run's three phases β€” pre-loop Β· in-loop Β· after-loop. From here on, boxes in figures are engine actions and decisions; no node of a concrete graph appears.

2. Where the engine sits

One class owns everything this page describes β€” GraphExecutor, entered through a single method, run(graph, session, …). The layers above it (HTTP endpoint, LoopRunner, GraphRun) own lifecycle β€” pause, stop, resume β€” and never look inside the graph; the layer below (the node handlers) computes and never schedules. The executor is the only place that decides what fires when.

POST /api/navigate/run api/execution/run.py LoopRunner β†’ GraphRunpause/stop/resume Β· one executor per run Β· execution_id GraphExecutorthis page β€” firing Β· iteration Β· scopes Β· state graph_executor.py CATALOG (node catalog)per-type BaseCanvasNode.forward() Β· ports_of(type, config) builtin_nodes.py

A fresh executor is built per run, so no state leaks between episodes. For parallel batch eval it exposes two per-worker overrides β€” get_env_panel(name) and get_server_url(nodeset) β€” so each eval worker talks to its own env subprocess (ADR-eval-002); outside batch eval they pass through.

Inside run(), every run has the same three-phase shape β€” the outline of the rest of View 2, and the engine-side mirror of Β§1.2's three places: pre-loop builds live state and primes the queue (Β§3) Β· in-loop fires until a halt (Β§4) Β· after-loop delivers the verdict and finalises (Β§5).

iterate Pre-loop Β· Β§3 build & discover BUILD Β· ENTRIES β†’ queue In-loop Β· Β§4 fire until a halt guards Β· IterOut 4-phase After-loop Β· Β§5 verdict & finalise final_* drain Β· run_end

3. Pre-loop β€” build & discover entries

Before a single node fires, run() does a deterministic, once-per-run setup in two movements. BUILD turns the static graph JSON into live, mutable runtime state; ENTRY DISCOVERY finds the entry nodes β€” the nodes that can fire with no inputs β€” and primes the queue with them. Nothing here loops β€” it is pure preparation, identical on every run.

graph JSONstatic def BUILD β‘  hooks Β· β‘‘ flatten Β· β‘’ containers β‘£ step_budget Β· β‘€ instances β‘₯ scopes Β· ⑦ adjacency static def β†’ live, mutable state ENTRIES queue no-input nodes never IterIn Β§4

3.1 BUILD β€” compile the graph into live state

Seven steps, strictly in order β€” plain top-to-bottom code, no concurrency. Each step turns one part of the authored GraphDefinition into something the loop can read or mutate. (Run-start args need no build step of their own: they arrive over ordinary canvas edges into the iterIn's init_* handles β€” ADR-dataflow-008.)

#StepWhat it doesWhy this order
1Merge hooksmerge the shell-hook lists from all three levels β€” global, graph, per-node β€” into one runnerhook precedence must be settled before anything can fire
2Flattenexpand every composite node's inner subgraph in place, leaving one flat node list (ADR-dataflow-001)early, so every later step sees the flat graph β€” no hidden subgraphs
3State containerscreate the shared state containers and record which node is allowed to read/write which container (the access grants)shared state must exist before any node touches it
4Resolve step_budgetfix the per-episode iteration cap to one number (see chain below)the loop's halt check needs a fixed cap
5Node instancescreate one live NodeInstance per authored node β€” its own input buffer (pending_inputs) and persistent statethis is where the static definition becomes mutable per-run state
6Scope statefind every (iterIn, iterOut) loop and how loops nest; give each loop its own step counter and budget. Nodes outside any loop land in the implicit graph scopenested loops count steps independently; a root-loop stop halts the run, an inner one only ends its own loop
7Adjacencyindex every node's outgoing edges by source idrouting a fired node's outputs becomes a single lookup in the loop
Step-budget resolution chain (step 4). The budget is fixed once, in strict precedence: step_budget_override (passed in by the eval-batch runner β€” e.g. HM-EQA's scene-adaptive int(√scene Β· 3)) β†’ the graph's authored step_budget β†’ the framework default. The override is passed in as an argument, never written back onto the shared graph object, so parallel eval workers can each carry a different per-episode budget. (This is the anchor batch-eval links to for the resolver chain.)

3.2 ENTRY DISCOVERY β€” find the entry nodes, prime the ready-queue

An entry node is discovered automatically β€” no author marking β€” as any node satisfying all three structural conditions; every entry node is queued at run-start:

validate_graph_connectivity (in graph_def.py) rejects required-but-unwired ports at load time, so the third condition only ever excludes well-formed leaf consumers β€” it never silently drops a real entry node. With the queue primed, three things arm the loop:

Sharp edge. IterIn is never an entry node (excluded by type). A loop is entered only via the init-side edges (upstream producers wired into the IterIn's init_* handles) or via an entry node that fills IterIn's iterout_* slots on iteration 0. A loop whose IterIn has no incoming init edge will never start β€” the body simply starves.

4. In-loop β€” the firing loop

With the queue primed, the engine starts turning. This chapter walks one turn from the inside out β€” the loop skeleton (Β§4.1), the guards before each fire (Β§4.2), the IterOut boundary that closes every iteration (Β§4.3), the propagation path every other node takes (Β§4.4) β€” then reassembles the pieces into the whole-run flowchart (Β§4.5) and a concrete trace (Β§4.6).

4.1 One iteration & the loop skeleton

Keep Β§1.2's graph in mind β€” entry node, two pivots, a two-node body; this chapter is what the engine does with it. The main loop is deliberately tiny. Each turn:

  1. Gate. Run while the queue holds a node and total firings stay under the runaway cap (while ready_queue and total_firings < max_firings); honour pause; a user Stop breaks out.
  2. Pop. Take the front of the FIFO ready-queue.
  3. Guard, then fire. Two structural guards (Β§4.2), then _fire β€” the one error policy, wrapping the per-node fire path _fire_node (graph_executor.py).
  4. Fork. An IterOut runs the once-per-iteration boundary (_iter_boundary, Β§4.3); every other node takes the ordinary delivery path (_after_fire β†’ _deliver, Β§4.4).

Nothing else is special: an agent's body is plain dataflow firing, and IterOut is the only node type the engine schedules differently. The loop itself only schedules; the work sits in three seams it calls at fixed points β€” _fire, _iter_boundary, _after_fire β†’ _deliver β€” and the pivots' slot / latch / snapshot rules (LoopCarry, final_payload, graphout_*) live in agent_loop/pivots.py, not in the loop.

4.2 The guards before a fire

A popped node is not fired blindly β€” two cheap structural guards run first, in order; each either skips the node or readies state for the fire:

  1. Scope barrier. A node belonging to an inner scope that has already terminated is frozen (pending_inputs cleared, fire skipped) until that scope re-enters β€” otherwise in-flight inner-body nodes fire one extra time after termination and leak per-iteration values through cross-scope wires. Exempt: iterIn / graphIn, the cascade that reopens the scope.
  2. IterIn step_start. An iterIn broadcasts step_start (per scope) so lifetime="step" state resets to a clean slate for the iteration about to run; a re-fired iterIn whose inner scope was terminated resets that scope's terminated flag + counter β€” the multi-scope re-entry path.

Then the node fires β€” through _fire, the one error policy shared by every fire site (the main loop, the settle drain, the after-loop pass). A StopExecution raised by the fire halts the run immediately. Any other exception is published to the event sink (NODE_EXEC_FAIL), recorded in the per-run node_errors ledger for the end-of-run conviction (Β§5.2), and the node's result becomes {"error": …} β€” which routes to no declared port, so downstream starves rather than consuming garbage; under strict errors (config.strict_errors_enabled(), env AGENTCANVAS_STRICT_ERRORS) it re-raises after the bus emit and fails fast instead. A returned {"error": …} result is convicted the same way (NODE_RESULT_ERROR) unless the node declares an error output port, in which case the key is data.

(Until 2026-06-11 there were four guards. The termination-defer and post-loop-skip guards were deleted with the termination node and config.post_loop β€” the stop signal and the after-loop stage now live on the iterOut pivot itself, so there is nothing left to defer or skip.)

4.3 The IterOut boundary β€” four phases

IterOut fires exactly once per iteration, so the engine hangs every per-iteration chore on it (_iter_boundary). The order of the four phases is load-bearing β€” in particular Settle runs before Decide:

IterOut β€” once per iteration β‘  Recordstep++ Β· checkpoint Β· log Β· broadcast nav_step β‘‘ Settlefire this iteration's leftover sinks (e.g. evaluate) β‘’ Decidestop? budget hit? β†’ emit final_* + halt, else loop β‘£ Hand offtransfer state β†’ IterIn, next iteration settle before decide terminal-step sinks fire before the loop can halt

What each phase does, and the subtle rule inside it:

What the final side bought. Before 2026-06-11 the episode verdict came from an in-loop evaluate whose terminal-step fire depended on the settle drain catching it β€” the SIMPLER / smartway metric-drop bug class (roadmap #64: 5/20 successes recorded as 0). Now the verdict consumes final_* values handed over by the engine at termination, so it is correct by construction; in-loop sinks are telemetry only.

4.4 Ordinary nodes & propagation

Every non-IterOut fire ends the same way (_after_fire), in four moves:

  1. Consume. The node's pending_inputs are cleared β€” inputs are used up by the fire.
  2. Pivot cases. An iterIn additionally drops its persist=false slots (LoopCarry.after_fire), so one-shot values emit once and are gone. An inner-scope graphOut buffers its value into a latch (pivots.graphout_latch, state["latched_value"]) instead of delivering outward β€” the latch is flushed to outer-scope consumers only when its scope terminates (_propagate_graphout_latches).
  3. Deliver. Everything else goes through _deliver, the one delivery rule: for every outgoing edge the value is payload[sourceHandle] β€” the whole payload for the default handle, nothing when the handle is absent (so an error-shaped result routes nowhere) β€” and is written onto the target's input port by _route_value_to_port. This is also where LIST[T] coercion happens (ADR-027): a scalar producer feeding a list consumer is auto-wrapped, and fan-in is concatenated in edge order; a write landing on an iterIn goes into its LoopCarry slot.
  4. Enqueue. Any target whose required inputs are now all present joins the ready-queue (_enqueue dedupes and stamps the ready-time).

The same _deliver serves the final side and the after-loop pass with terminal=True: a graphOut reached there is routed-to and recorded β€” graph scope: the harvest snapshot (pivots.graphout_snapshot); inner scope: the latch β€” but never fired.

(Run-start args need no special branch: the iterIn's init_* handles are fed by ordinary canvas edges β€” ADR-dataflow-008 two-sided iterIn.)

4.5 The whole run, end to end

Zoom out from one iteration to the whole of run(). The flowchart below stitches Β§3 and Β§4 into one picture, with the ready-queue at its centre β€” ENTRY DISCOVERY primes it, every turn pops it, and both return paths feed it again. Read it top-down:

  1. Set up once. BUILD compiles the graph JSON into live state (Β§3.1); ENTRY DISCOVERY finds the no-input entry nodes and primes the ready-queue (Β§3.2).
  2. Turn the loop. While the queue has a ready node β€” and neither the global firing cap nor a user Stop has hit β€” pop the front one and fire it: guards first (Β§4.2), then _fire.
  3. Fork on what fired. An ordinary node takes the fast path: route its outputs, enqueue whatever became ready (Β§4.4), turn again β€” this is almost every fire. An IterOut instead runs the once-per-iteration boundary (Β§4.3): Record, Settle, Decide, Hand off. Unless Decide halts, Hand off arms the paired IterIn and the loop turns again β€” that is one full iteration.
  4. Exit. Two doors out. Decide halts β€” the stop signal or the step budget β€” and the terminal iteration's values leave exactly once on the final_* handles. Or the main loop runs dry β€” queue drained, firing cap, user Stop β€” and the fallback reconstructs the final-side values best-effort. Every door lands in the same after-loop + finalise stage (Β§5); Β§7 tables every exit, including the StopExecution escape hatch.
BUILD β€” Β§3.1 graph JSON β†’ live state ENTRY DISCOVERY β€” Β§3.2 finds the no-input entry nodes ready-queue Β· FIFO … pop & fire one node guards Β§4.2 β†’ _fire fired an IterOut? route outputs enqueue ready β€” Β§4.4 IterOut boundary β€” Β§4.3 β‘  Record step++ Β· checkpoint β‘‘ Settle leftover sinks β‘’ Decide stop? budget? β‘£ Hand off carry β†’ IterIn AFTER-LOOP + FINALISE final_* β†’ verdict Β· run_end β€” Β§5 prime has a node β€” pop front yes no empty Β· firing cap Β· user Stop halt: stop or budget emit final_* once enqueue ready re-enqueue IterIn

That is the entire control flow β€” and the engine-view counterpart of Β§1.2: there the boxes were the nodes of one drawn graph; here they are the engine's actions on any graph. Part II Β§6 prints this same shape as code β€” the colored region bands there are exactly these boxes.

4.6 A concrete trace

One episode of the Β§1.2 graph, turn by turn. Watch the ready-queue column β€” it is the hub of the Β§4.5 flowchart; entry discovery put entry in it (and only entry: IterIn is never an entry node, Β§3.2). Each turn pops the front, fires it, and the effect column says what that made ready next.

Turnready-queueFiresEffect
1entryentryinit_* edges deliver the run-invariants into iter_in's init slots β†’ iter_in ready
2iter_initer_inemits the loop-carry bundle β†’ predict ready
3predictpredict{action} β†’ env_step ready
4env_stepenv_step{obs, done: false} β€” obs fills iter_out's carry port, done lands on its stop input β†’ iter_out ready
5iter_outiter_out β€” the Β§4.3 boundaryβ‘  Record step 1 Β· β‘‘ Settle leftover sinks Β· β‘’ Decide: stop false, budget fine β†’ continue Β· β‘£ Hand off: carry into iter_in's slots β†’ iter_in re-enqueued
⟳iter_inturns 2–5 repeat β€” iterations 2 … Nβˆ’1
6env_stepenv_stepiteration N: {obs, done: true} β€” the stop input is now truthy
7iter_outiter_outβ‘’ Decide: stop true β†’ the final side emits once (terminal obs, final_stop=True) β†’ evaluate ready; the loop ends
8evaluateevaluate β€” after-loopreads the terminal values off final_* β†’ verdict; graphOut re-snapshotted, metrics harvested (Β§5)

Note what never happens: evaluate appears in the queue exactly once, at turn 8 β€” during the loop its inputs simply never arrive (Β§5.1), so the verdict cannot read anything but the terminal iteration.

5. After-loop β€” the verdict stage & finalise

However the loop exits β€” the stop signal fired, the step budget hit, the queue drained, or a node crashed β€” control lands in one finalise stage. It has four movements: the after-loop pass (Β§5.1), the node-error conviction gate and clean finalisation (Β§5.2), and the error path (Β§5.3).

5.1 The after-loop pass

The after-loop band is every node wired downstream of the root iterOut's final_* handles β€” the verdict chain, typically evaluate β†’ graphOut, possibly with adjudicator steps in between. Two facts keep it dormant while the loop runs: the final side never emits mid-loop, and the validator forbids edges from the loop body into the band. Band nodes therefore cannot become ready before termination β€” there is nothing to skip, defer, or flag.

When the final-side values arrive, _after_loop_pass drains whatever they made ready: fire a ready node (through _fire, so a failure here is convicted like any other), deliver its outputs (_deliver(terminal=True)), enqueue the consumers that became ready in turn. A node whose upstream band node hasn't fired yet is re-queued behind it, so a chain resolves in dependency order without a topological sort; the queue is bounded (max(nodesΓ—4, 64)), so a never-satisfiable node cannot spin forever.

Two rules keep the pass honest:

Membership is topological, not declared: there is no flag to set (the former config.post_loop was removed 2026-06-11) β€” a node is after-loop because it hangs off the final side. The input contract is engine-guaranteed: the band receives exactly the terminal iteration's values ("verdict inputs ride the pivot" β€” anything the verdict needs must be an iterOut port, consumed via final_<name>). If the loop ends without a boundary emission (drained queue, max_firings, user stop, or a crash), _emit_final_fallback reconstructs the last completed iteration's values from the paired iterIn's iterout_* slots (LoopCarry.last_carried) and emits best-effort before the drain.

5.2 Clean finalise

The conviction gate comes first (2026-07-04). At fire time _fire (Β§4.2) records every node failure β€” a raised exception, or an {"error": ...}-shaped result such as a server-mode proxy surfacing an HTTP 500 β€” into a per-run node_errors ledger (nodes that declare a legitimate error output port are exempt; returned errors also emit a NODE_RESULT_ERROR bus event). If the ledger is non-empty here, a NodeErrorAggregate naming the culprits is raised and control transfers to the error path (Β§5.3). The placement is deliberate: the verdict stage (Β§5.1) has already run, so final-side metrics are collected either way β€” but the run can no longer masquerade as completed while its downstream silently starved (the old status="completed" / step_count=0 eval pathology).

On a clean exit (empty ledger), in order:

  1. Harvest metrics β€” from any node's state["metrics"]; a value stored on session._metrics overrides node values.
  2. Broadcast run_end β€” lifetime="run" state containers clear.
  3. Broadcast nav_complete β€” final step and metrics, to the frontend.
  4. Fire the GraphComplete hook.
  5. Flush the execution log. Session status becomes done.

5.3 The error path

If the loop body raises β€” or the conviction gate fired (Β§5.2) β€” the finalise stage still runs, and runs the verdict first:

  1. Fallback emission + after-loop pass, best-effort. The ordering is deliberate: even after a crash the verdict evaluate can often still emit fresh metrics from the env's real final step, so the harvest reports the true result rather than a stale pre-crash value. The pass is tolerant β€” a band node that raises because the env is already torn down is logged and skipped (and both stages are idempotent, so re-entry from the conviction gate is a no-op).
  2. Publish the exception to the ErrorBus β€” GRAPH_CRASH for a real crash, NODE_ERRORS for a conviction; session status becomes error, broadcast to the frontend as a nav_status error.
  3. run_end still fires, carrying the error β€” run-lifetime state resets even on a crash.
  4. Fire the GraphError hook, then the final log flush.
Part II Β· Implementation reference β€” for Claude Code & maintainers Two listings, synced from the source file with its comments: run() β€” the scheduling loop β€” and the seams it calls (_fire, _deliver, _after_fire, _iter_boundary, _settle_drain). The per-node fire path (_fire_node) and the pivots module (pivots.py) are read in source.

6. The run() listing β€” synced from source

The full method with its real comments: the annotations you read here are the source file's comments, so the listing and the engine cannot disagree about what a block does. The blue band headers are the file's own # ─── banner comments β€” the same phase vocabulary as Part I β€” with the matching section linked on each band, and the line numbers are real. After editing run() or one of the seams, refresh with python3 docs/_lib/_sync_run_listing.py; the sync stamp below names the commit it was last generated against.

Synced from graph_executor.py:308–775 at 1e54b3e by docs/_lib/_sync_run_listing.py β€” the band headers are the file's own banner comments; re-run the script after editing the source.

run() β€” signature & docstring ── graph_executor.py:308–337
308      async def run(
309          self,
310          graph: GraphDefinition,
311          session: Any,
312          execution_id: str | None = None,
313          step_delay_ms: int = 200,
314          stop_event: asyncio.Event | None = None,
315          pause_event: asyncio.Event | None = None,
316          global_hooks: list[Any] | None = None,
317          step_budget_override: int | None = None,
318      ) -> None:
319          """Main graph execution loop.
320
321          ``step_budget_override`` (when set) wins over the graph's authored
322          ``step_budget``. Used by the eval batch resolver chain to push an
323          env-supplied per-episode value (e.g. HM-EQA's
324          ``int(sqrt(scene_size) * 3)``) without mutating the shared graph
325          object across worker coroutines.
326
327          Structure (the ``# ───`` banners below mark the phases):
328          pre-loop (BUILD steps 1-7, then ENTRY DISCOVERY) β†’ in-loop (the
329          scheduling loop; the iterOut boundary β€” ``_iter_boundary`` β€” closes
330          each iteration in four phases: record β†’ settle β†’ decide β†’ hand off)
331          β†’ after-loop (the verdict stage) β†’ finalise. The loop fires through
332          ``_fire`` (the one error policy) and delivers through
333          ``_after_fire`` / ``_deliver`` (the one delivery rule). Narrated
334          walkthrough with diagrams:
335          docs/pages/developer-guide/design-docs/graph-executor.html (Part I).
336          """
337
pre-loop Β· BUILD β€” compile the static graph into live state ── graph_executor.py:338–458 Β· ↔ Β§3.1
339
340          # BUILD 1/7 β€” merge hooks from 3 sources: global β†’ graph β†’ node (R3)
341          node_hooks = extract_node_hooks(graph.nodes)
342          merged = merge_hooks(global_hooks or [], graph.hooks, node_hooks)
343          self._hook_runner = HookRunner(merged)
344
345          # BUILD 2/7 β€” flatten composite nodes before execution (ADR-dataflow-001)
346          from .flatten import flatten_graph
347
348          graph, self._flatten_map = flatten_graph(graph)
349
350          # BUILD 3/7 β€” state containers + access grants from the graph definition
351          if graph.containers:
352              from .state_containers import build_containers
353
354              self.containers = build_containers(graph.containers)
355              for ag in graph.access_grants:
356                  self._access_grant_index.setdefault(ag.node_id, set()).add(ag.container_id)
357              log.info(
358                  "State containers: %d containers, %d access grants",
359                  len(self.containers),
360                  len(graph.access_grants),
361              )
362              # If a container with the well-known id "graph_state" exists,
363              # set up the convenience binding.  Access is still gated by
364              # explicit access grants β€” no auto-inject.
365              if "graph_state" in self.containers:
366                  self._graph_state_id = "graph_state"
367
368          # BUILD 4/7 β€” resolve the step budget. Order: explicit override β†’
369          # graph authored value β†’ framework default. The eval batch runner
370          # computes the override via its resolver chain (env-dynamic β†’ API
371          # override).
372          from ..config import DEFAULT_STEP_BUDGET as _DEFAULT_STEP_BUDGET
373
374          if step_budget_override is not None:
375              self.step_budget = step_budget_override
376          elif graph.step_budget is not None:
377              self.step_budget = graph.step_budget
378          else:
379              self.step_budget = _DEFAULT_STEP_BUDGET
380          self.edges = [e.to_dict() for e in graph.edges]
381          node_defs = graph.nodes
382
383          if not node_defs:
384              log.error("Empty graph β€” nothing to execute")
385              return
386
387          # BUILD 5/7 β€” node instances: the static definition becomes
388          # per-run mutable state (pending_inputs buffer + persistent state)
389          for nd in node_defs:
390              self.nodes[nd.id] = NodeInstance(
391                  id=nd.id,
392                  type=nd.type,
393                  config=nd.config,
394                  label=nd.label,
395              )
396
397          # BUILD 6/7 β€” per-scope execution state. Always includes a
398          # synthetic GRAPH_SCOPE_ID entry; for single-scope graphs there is
399          # also one entry keyed by the outermost author scope's iter_in id.
400          # Multi-scope graphs add one entry per author scope. The root
401          # scope's stop signal halts the entire run; a non-root scope's stop
402          # only ends that inner loop and lets the outer one continue.
403          self.scope_forest, scope_errors = analyze_scopes(graph)
404          if scope_errors:
405              for err in scope_errors:
406                  log.warning("Scope analysis: %s", err)
407          # Always create the synthetic graph-scope entry β€” holds graph-scope
408          # nodes (run-start entry nodes, after-loop sinks).
409          self.scope_state[GRAPH_SCOPE_ID] = _ScopeState(
410              scope_id=GRAPH_SCOPE_ID,
411              parent_scope_id=None,
412              step_budget=self.step_budget,  # graph-level cap also applies here
413              member_node_ids=set(self.scope_forest.graph_scope_node_ids),
414          )
415          # Per-author-scope state; resolve step_budget with fallback chain
416          # (scope's own iter_in.step_budget β†’ parent scope's step_budget β†’
417          # graph step_budget).
418          for scope_id, info in self.scope_forest.scopes.items():
419              resolved_budget = info.step_budget
420              if resolved_budget is None:
421                  # Walk up parent chain looking for a budget; fall back to graph
422                  p = info.parent_scope_id
423                  while p is not None and resolved_budget is None:
424                      p_info = self.scope_forest.scopes.get(p)
425                      if p_info is None:
426                          break
427                      resolved_budget = p_info.step_budget
428                      p = p_info.parent_scope_id
429              if resolved_budget is None:
430                  resolved_budget = self.step_budget
431              self.scope_state[scope_id] = _ScopeState(
432                  scope_id=scope_id,
433                  parent_scope_id=info.parent_scope_id,
434                  step_budget=resolved_budget,
435                  member_node_ids=set(info.member_node_ids),
436                  graphout_node_ids=list(info.graphout_node_ids),
437                  iter_out_id=info.iter_out_id,
438              )
439          # The "outermost" scope id β€” used for backward compat: self.step_counter
440          # and self.terminated mirror this scope's state. For 0-scope graphs
441          # this stays GRAPH_SCOPE_ID; for single-scope graphs it's the sole
442          # author scope's id; for multi-scope graphs it's the (single) root.
443          if len(self.scope_forest.root_scope_ids) == 1:
444              self._outermost_scope_id = self.scope_forest.root_scope_ids[0]
445          elif len(self.scope_forest.root_scope_ids) > 1:
446              # Multiple peer roots β€” pick the first deterministically; legacy
447              # self.step_counter mirrors only its counter (multi-root graphs
448              # are an advanced shape; consumers should read scope_state directly).
449              self._outermost_scope_id = self.scope_forest.root_scope_ids[0]
450          else:
451              self._outermost_scope_id = GRAPH_SCOPE_ID
452
453          # BUILD 7/7 β€” adjacency: outgoing edges indexed by source id, so
454          # routing a fired node's outputs is a single lookup in the loop
455          for edge in self.edges:
456              src = edge.get("source", "")
457              self.adjacency.setdefault(src, []).append(edge)
458
pre-loop Β· ENTRY DISCOVERY β€” prime the ready-queue, arm the loop ── graph_executor.py:459–530 Β· ↔ Β§3.2
460
461          # A node is an entry node if: explicitly marked OR (has no required
462          # inputs AND has no incoming edges). Nodes with incoming edges to
463          # optional ports should wait for data even if no required ports exist.
464          incoming_targets = set()
465          for edge in self.edges:
466              tgt = edge.get("target", "")
467              if tgt:
468                  incoming_targets.add(tgt)
469
470          # Entry discovery: queue at run-start iff all three structural
471          # conditions hold:
472          #   1. type != "iterIn" (iterIn fires from init edges / iterOut transfer)
473          #   2. no incoming edges
474          #   3. no required input ports
475          # validate_graph_connectivity (graph_def.py) rejects required-but-
476          # unwired ports at load time, so condition 3 normally only excludes
477          # well-formed leaf consumers β€” never silently drops an entry node.
478          # System Log: ready-time stamps for queue_wait_ms (set on enqueue,
479          # popped on fire in _fire_node). Run-scoped β€” reset on each run().
480          self._t_ready: dict[str, float] = {}
481          ready_queue: list[str] = []
482          for node in self.nodes.values():
483              if node.type == "iterIn":
484                  continue
485              if node.id in incoming_targets:
486                  continue
487              if self._get_required_ports_for_node(node):
488                  continue
489              self._enqueue(ready_queue, node.id)
490
491          log.info(
492              "GraphExecutor: %d nodes, %d edges, entry_nodes=%s, step_budget=%d",
493              len(self.nodes),
494              len(self.edges),
495              [self.nodes[nid].label or nid for nid in ready_queue],
496              self.step_budget,
497          )
498
499          stop = stop_event or asyncio.Event()
500          pause = pause_event or asyncio.Event()
501          if not pause.is_set():
502              pause.set()
503
504          session._status = "running"
505          # Node failures recorded during this run β€” convicts the run at the
506          # finalise stage (see NodeErrorAggregate).
507          self.node_errors: list[dict] = []
508          _suppress = getattr(getattr(session, "principles", None), "suppress_nav_events", False)
509          if not _suppress:
510              await broadcast(session._ws("nav_status", {"status": "running", "step": 0}))
511
512          # run_start signal fires before any node fires, so state
513          # containers with lifetime="run" reset to a clean baseline.
514          self.broadcast_signal("run_start", {"graph_name": graph.name})
515
516          # GraphStart hook
517          if self._hook_runner.has_hooks():
518              await self._hook_runner.run_hooks(
519                  "GraphStart",
520                  payload={
521                      "graph_name": graph.name,
522                      "node_count": len(self.nodes),
523                  },
524              )
525          await self._emit("graph_start", graph_name=graph.name, node_count=len(self.nodes))
526
527          # Safety: max total firings to prevent infinite loops
528          max_firings = self.step_budget * len(self.nodes) * 3
529          total_firings = 0
530
in-loop Β· the scheduling loop β€” pop, guard, fire, pivot or deliver ── graph_executor.py:531–649 Β· ↔ Β§4.1–§4.2
532          # The loop only schedules. The work happens in three seams called
533          # at fixed points: ``_fire`` (the one error policy),
534          # ``_iter_boundary`` (the iterOut pivot: record β†’ settle β†’ decide β†’
535          # hand off) and ``_after_fire`` β†’ ``_deliver`` (the one delivery
536          # rule).
537          try:
538              while ready_queue and total_firings < max_firings:
539                  # Pause/stop check
540                  await pause.wait()
541                  if stop.is_set():
542                      break
543
544                  node_id = ready_queue.pop(0)
545                  node = self.nodes.get(node_id)
546                  if node is None:
547                      continue
548
549                  # Guard 1 β€” scope barrier. Once an inner scope is marked
550                  # terminated (by the iterOut boundary's decide check at the
551                  # end of the final inner iter), forbid any further fire of nodes belonging
552                  # to that scope until the scope is re-entered. Without this
553                  # guard, in-flight inner-body nodes that were enqueued
554                  # before the scope stopped (e.g. ``move_to_pose`` already
555                  # running, or ``episode_info`` triggered by it) fire one
556                  # extra time AFTER the stop and leak per-iter outputs through any
557                  # cross-scope wires (post-flatten direct edges from inner
558                  # producers to outer consumers, OR direct graphOut→outer
559                  # wires when graphOut is preserved), satisfying outer
560                  # iterOut's required gates and causing outer to cycle
561                  # without inner actually re-entering.
562                  #
563                  # Exemption set β€” scope-entry cascade nodes that MUST be
564                  # allowed to fire so the next outer iter can reopen this
565                  # scope: ``iterIn`` (existing re-entry path below clears
566                  # the terminated flag; its init slots re-capture
567                  # outer-supplied scope-entry args), ``graphIn`` (ferries
568                  # outer→inner values across the boundary into iterIn's
569                  # init side). Body nodes and iterOut stay barred β€” they
570                  # have nothing useful to do between scope terminate and
571                  # scope re-entry.
572                  _n_scope_id = self._scope_of(node_id)
573                  _n_scope = (
574                      self.scope_state.get(_n_scope_id)
575                      if _n_scope_id and _n_scope_id != GRAPH_SCOPE_ID
576                      else None
577                  )
578                  if (
579                      _n_scope is not None
580                      and _n_scope.terminated
581                      and _n_scope.parent_scope_id is not None
582                      and node.type not in ("iterIn", "graphIn")
583                  ):
584                      node.pending_inputs = {}
585                      continue
586
587                  # Guard 2 β€” iterIn step_start. iterIn marks the start of each
588                  # iteration β€” emit step_start before the node fires so
589                  # lifetime="step" states see a fresh slate for the upcoming
590                  # iteration.
591                  # Multi-scope re-entry: an inner scope's iter_in firing
592                  # while its scope is marked terminated indicates outer just
593                  # entered a new outer iter and is re-invoking inner. Reset
594                  # inner scope's terminated flag and step_counter β€” inner
595                  # gets a fresh loop. (Root-scope iter_in re-firing after
596                  # termination would only happen if main loop didn't break;
597                  # we still allow the reset for symmetry, though in practice
598                  # root termination breaks the loop above.)
599                  if node.type == "iterIn":
600                      _ii_scope = self.scope_state.get(node.id)
601                      if _ii_scope is not None and _ii_scope.terminated:
602                          if _ii_scope.parent_scope_id is not None:
603                              # Inner scope re-entry from outer iter β€” reset
604                              log.info(
605                                  "Scope re-entry: %s (parent=%s) β€” resetting terminated/counter",
606                                  node.id,
607                                  _ii_scope.parent_scope_id,
608                              )
609                              _ii_scope.terminated = False
610                              _ii_scope.step_counter = 0
611                              # Re-arm the final side: an inner scope emits
612                              # final_* once per termination, i.e. once per
613                              # outer iteration.
614                              self._final_emitted.discard(node.id)
615                              # Reset per-fire state on body nodes that hold
616                              # transient counters (only those tracked via
617                              # state['_scoped_reset']=True)? β€” out of scope
618                              # for v1; user tests must use cumulative counters.
619                          else:
620                              # Root scope: should not happen (main loop breaks
621                              # on root termination). Defensive: skip the fire.
622                              node.pending_inputs = {}
623                              continue
624                      # step_start signal β€” add scope_id (additive, single-scope
625                      # readers using just `step` keep working).
626                      _next_step = (
627                          _ii_scope.step_counter + 1
628                          if _ii_scope is not None
629                          else self.step_counter + 1
630                      )
631                      self.broadcast_signal(
632                          "step_start",
633                          {"step": _next_step, "scope_id": node.id},
634                      )
635
636                  # Fire the node. A failed fire yields an ``{"error": ...}``
637                  # result (or raises under strict errors); StopExecution is
638                  # the node-side halt and ends the run here.
639                  try:
640                      result = await self._fire(node, session)
641                  except StopExecution as e:
642                      log.info(
643                          "StopExecution from %s: %s at step %d", node.id, e.reason, self.step_counter
644                      )
645                      self.terminated = True
646                      break
647
648                  total_firings += 1
649
iterOut boundary β€” the pivot closes this iteration ── graph_executor.py:650–655 Β· ↔ Β§4.3
651                  if node.type == "iterOut":
652                      if await self._iter_boundary(node, result, ready_queue, session, step_delay_ms):
653                          break
654                      continue
655
ordinary path β€” consume Β· pivot cases Β· deliver ── graph_executor.py:656–665 Β· ↔ Β§4.4
657                  self._after_fire(node, result, ready_queue)
658
659                  if total_firings <= 5:
660                      log.info(
661                          "After %s: queue=%s",
662                          node.label or node.id,
663                          [self.nodes[nid].label or nid for nid in ready_queue],
664                      )
665
after-loop Β· verdict stage β€” fallback emission + band drain ── graph_executor.py:666–675 Β· ↔ Β§5.1
667              # If the loop ended without a boundary final-side emission
668              # (drained queue, max_firings, user stop), reconstruct the last
669              # completed iteration's values from the paired iterIn's slots
670              # and emit best-effort; then drain the after-loop band β€” the
671              # nodes fed by the root iterOut's final_* handles (evaluate,
672              # graphOut chains). This is the run-end verdict stage.
673              self._emit_final_fallback()
674              await self._after_loop_pass(session)
675
conviction β€” surface node failures at end of run ── graph_executor.py:676–689 Β· ↔ Β§5.2
677              # Runs AFTER the verdict stage so final-side metrics are already
678              # collected; the raise routes through the error path below and
679              # the run finishes with status="error" instead of masquerading
680              # as completed (both stages are idempotent on the re-entry).
681              if self.node_errors:
682                  _n_err = len(self.node_errors)
683                  _head = "; ".join(
684                      f"{e['node_id']}@step{e['step']}: {e['error']}" for e in self.node_errors[:3]
685                  )
686                  raise NodeErrorAggregate(
687                      f"{_n_err} node error(s) during run: {_head}" + ("; ..." if _n_err > 3 else "")
688                  )
689
finalise β€” clean exit ── graph_executor.py:690–732 Β· ↔ Β§5.2
691              session._status = "done"
692              metrics = None
693              # Find metrics from any node that stored them (generic β€” any step node)
694              for node in self.nodes.values():
695                  if node.state.get("metrics"):
696                      metrics = node.state["metrics"]
697              if session._metrics:
698                  metrics = session._metrics
699
700              # run_end signal β€” fires after the loop finishes cleanly.
701              # lifetime="run" states clear here.
702              self.broadcast_signal(
703                  "run_end",
704                  {"step": self.step_counter, "terminated": self.terminated},
705              )
706
707              if not _suppress:
708                  await broadcast(
709                      session._ws(
710                          "nav_complete",
711                          {"step": self.step_counter, "metrics": metrics},
712                      )
713                  )
714
715              # GraphComplete hook
716              if self._hook_runner.has_hooks():
717                  await self._hook_runner.run_hooks(
718                      "GraphComplete",
719                      payload={
720                          "step": self.step_counter,
721                          "terminated": self.terminated,
722                      },
723                  )
724              await self._emit(
725                  "graph_complete", terminated=self.terminated, metrics=metrics
726              )
727
728              # Final flush of any remaining log entries
729              if self._logger:
730                  self._logger.flush()
731
732          except Exception as e:
finalise β€” error path (verdict first, then error reporting) ── graph_executor.py:733–775 Β· ↔ Β§5.3
734              # Best-effort after-loop stage first: a final-side evaluate can
735              # still emit fresh metrics even when the main loop crashed, so
736              # eval harvest reports the actual final env step count instead
737              # of a stale pre-crash value. Tolerant of cascading failures
738              # (the env may be torn down already).
739              try:
740                  self._emit_final_fallback()
741                  await self._after_loop_pass(session)
742              except Exception as _fpe:
743                  log.warning("after_loop_pass on error path raised: %s", _fpe)
744              get_bus().from_exception(
745                  e,
746                  source="graph",
747                  code="NODE_ERRORS" if isinstance(e, NodeErrorAggregate) else "GRAPH_CRASH",
748                  scope={
749                      "step": self.step_counter,
750                      "execution_id": getattr(session, "_execution_id", None),
751                  },
752                  title=(
753                      f"Run finished with node errors at step {self.step_counter}"
754                      if isinstance(e, NodeErrorAggregate)
755                      else f"Graph execution crashed at step {self.step_counter}"
756                  ),
757              )
758              session._status = "error"
759              # run_end also fires on the error path so lifetime="run"
760              # states reset even when the loop crashes.
761              self.broadcast_signal(
762                  "run_end",
763                  {"step": self.step_counter, "error": str(e)},
764              )
765              if not _suppress:
766                  await broadcast(session._ws("nav_status", {"status": "error", "error": str(e)}))
767
768              # GraphError hook β€” fires on unhandled exceptions
769              if self._hook_runner.has_hooks():
770                  await self._hook_runner.run_hooks("GraphError", payload={"error": str(e)})
771              await self._emit("graph_error", error=str(e))
772
773              # Flush log entries even on error
774              if self._logger:
775                  self._logger.flush()

6.1 The seams β€” _fire Β· _deliver Β· _after_fire Β· _iter_boundary Β· _settle_drain

The methods the scheduling loop calls at fixed points, in source order, with the same band / Β§ convention. The iterOut boundary's four phases are the # ── banners inside _iter_boundary.

Synced from graph_executor.py:777–1102 at 1e54b3e by the same script.

async def _fire(…) ── graph_executor.py:779–818 Β· ↔ Β§4.2
779      async def _fire(self, node: NodeInstance, session: Any) -> dict:
780          """Fire one node under the one error policy.
781
782          Every fire site β€” the main loop, the boundary's settle drain, the
783          after-loop pass β€” goes through here, so a node failure means the
784          same thing wherever it happens: it is put on the error bus (the
785          Report tab), recorded in ``node_errors`` for the end-of-run
786          conviction (``NodeErrorAggregate``), and the fire yields an
787          ``{"error": ...}`` result β€” which routes to no declared port, so
788          downstream starves rather than consuming garbage. Under
789          ``AGENTCANVAS_STRICT_ERRORS`` an exception is re-raised after the
790          bus emit instead, so smoke / eval runs fail fast. A node that
791          legitimately declares an ``error`` output port (env_libero tools,
792          navgpt navigate) is exempt: its ``error`` key is data.
793          ``StopExecution`` is not an error and passes through to the caller.
794          """
795          from_exception = False
796          try:
797              result = await self._fire_node(node, session)
798          except StopExecution:
799              raise
800          except Exception as e:
801              self._report_node_error(node, session, exc=e)
802              if strict_errors_enabled():
803                  raise
804              result = {"error": str(e)}
805              from_exception = True
806          if isinstance(result, dict) and "error" in result and not self._declares_error_output(node):
807              if not from_exception:
808                  self._report_node_error(node, session, message=str(result["error"]))
809              self.node_errors.append(
810                  {
811                      "node_id": node.id,
812                      "node_type": node.type,
813                      "step": self.step_counter,
814                      "error": str(result["error"]),
815                  }
816              )
817          return result
818
def _report_node_error(…) ── graph_executor.py:819–849 Β· ↔ Β§4.2
819      def _report_node_error(
820          self, node: NodeInstance, session: Any, *, exc: Exception | None = None, message: str = ""
821      ) -> None:
822          """Put one node failure on the error bus: the exception itself
823          (``NODE_EXEC_FAIL``) or the message of a returned error result
824          (``NODE_RESULT_ERROR``)."""
825          scope = {
826              "node_id": node.id,
827              "node_type": node.type,
828              "origin": self._flatten_map.trace(node.id) if self._flatten_map else node.id,
829              "step": self.step_counter,
830              "execution_id": getattr(session, "_execution_id", None),
831          }
832          if exc is not None:
833              get_bus().from_exception(
834                  exc,
835                  source="node",
836                  code="NODE_EXEC_FAIL",
837                  scope=scope,
838                  title=f"Node {node.id} ({node.type}) failed",
839              )
840              return
841          get_bus().emit(
842              severity="error",
843              source="node",
844              code="NODE_RESULT_ERROR",
845              title=f"Node {node.id} ({node.type}) returned an error result",
846              message=message,
847              scope=scope,
848          )
849
def _deliver(…) ── graph_executor.py:850–894 Β· ↔ Β§4.4
850      def _deliver(
851          self, src_id: str, payload: dict, queue: list[str], *, terminal: bool = False
852      ) -> None:
853          """Route one fired node's outputs along its outgoing edges and
854          enqueue the consumers that are ready β€” the one delivery rule.
855
856          For every edge out of ``src_id`` the value is
857          ``payload[sourceHandle]`` β€” the whole payload when the handle is
858          ``"default"``, nothing when the handle is absent (so an
859          error-shaped result routes nowhere). The write goes through
860          ``_route_value_to_port`` (LIST[T] coercion, the iterIn slot
861          write). A target that is ready afterwards is enqueued;
862          ``_enqueue`` dedupes and stamps the ready-time.
863
864          ``terminal`` marks the after-loop band (the final side, the
865          after-loop pass): a ``graphOut`` reached there is routed-to and
866          recorded β€” graph scope: the harvest snapshot; inner scope: the
867          latch β€” but never fired. In-loop a graphOut fires like any node.
868          """
869          for edge in self.adjacency.get(src_id, []):
870              target = self.nodes.get(edge.get("target", ""))
871              if target is None:
872                  continue
873              src_handle = edge.get("sourceHandle", "default")
874              tgt_handle = edge.get("targetHandle", src_handle)
875              if src_handle in payload:
876                  value = payload[src_handle]
877              elif src_handle == "default":
878                  value = payload
879              else:
880                  value = _NOT_ROUTED
881              if value is not _NOT_ROUTED:
882                  self._route_value_to_port(target, tgt_handle, value)
883              if terminal and target.type == "graphOut":
884                  if value is _NOT_ROUTED:
885                      continue
886                  if self._scope_of(target.id) == GRAPH_SCOPE_ID:
887                      if target.pending_inputs:
888                          graphout_snapshot(target)
889                  else:
890                      graphout_latch(target, value)
891                  continue
892              if self._is_ready(target):
893                  self._enqueue(queue, target.id)
894
def _after_fire(…) ── graph_executor.py:895–909 Β· ↔ Β§4.4
895      def _after_fire(self, node: NodeInstance, result: Any, queue: list[str]) -> None:
896          """A body node has fired in-loop: consume its inputs, apply the
897          pivot cases, deliver. An iterIn clears its ``persist=False`` slots;
898          an inner-scope graphOut latches its value instead of delivering
899          (``_propagate_graphout_latches`` flushes the latch when the scope
900          terminates); everything else is delivered along its edges."""
901          node.pending_inputs = {}
902          if node.carry is not None:
903              node.carry.after_fire()
904          if node.type == "graphOut" and self._scope_of(node.id) != GRAPH_SCOPE_ID:
905              if isinstance(result, dict):
906                  graphout_latch(node, result.get("value"))
907              return
908          self._deliver(node.id, result if isinstance(result, dict) else {}, queue)
909
async def _iter_boundary(…) ── graph_executor.py:910–933 Β· ↔ Β§4.3
910      async def _iter_boundary(
911          self,
912          node: NodeInstance,
913          result: Any,
914          ready_queue: list[str],
915          session: Any,
916          step_delay_ms: int,
917      ) -> bool:
918          """The iterOut boundary β€” closes one iteration of its scope in four
919          phases: record β†’ settle β†’ decide β†’ hand off. Returns ``True`` when
920          the root scope terminated (stop input, budget exhaust, or a
921          StopExecution during the settle drain) and the scheduling loop must
922          halt. An inner scope's termination flushes its graphOut latches and
923          lets the outer loop continue.
924
925          Edges FROM the iterOut are final-side only (``final_*`` handles):
926          they emit once at scope termination via ``_emit_final_side``, never
927          per iteration, so the iterOut's own result is never delivered here.
928          """
929          # The scope is keyed by the paired iterIn's id (== scope_id).
930          io_scope_id = node.config.get("pairedWith", "") or self._outermost_scope_id
931          io_scope = self.scope_state.get(io_scope_id)
932          is_root = io_scope_id == self._outermost_scope_id
933
phase 1/4 Β· record β€” counters, checkpoint, log, nav_step ── graph_executor.py:934–974 Β· ↔ Β§4.3 β‘ 
935          # Advance the per-scope counter. For backward compat,
936          # self.step_counter mirrors the OUTERMOST scope's counter
937          # (single-scope graphs unchanged).
938          if io_scope is not None:
939              io_scope.step_counter += 1
940              scope_step = io_scope.step_counter
941          else:
942              scope_step = self.step_counter + 1
943          if is_root:
944              self.step_counter += 1
945              session._current_step = self.step_counter
946
947          log.info(
948              "Iter: scope=%s step=%d (parent=%s, root=%s)",
949              io_scope_id,
950              scope_step,
951              io_scope.parent_scope_id if io_scope is not None else None,
952              is_root,
953          )
954          self.broadcast_signal("step_end", {"step": scope_step, "scope_id": io_scope_id})
955
956          # Checkpoint all containers AFTER the iter boundary β€” the snapshot
957          # is the state step N+1 begins from. Root-scope iterations only
958          # (nested inner scopes don't checkpoint).
959          if self.containers and is_root:
960              self._checkpoints[self.step_counter] = {
961                  cid: c.checkpoint() for cid, c in self.containers.items()
962              }
963              if len(self._checkpoints) > self.max_checkpoints:
964                  del self._checkpoints[min(self._checkpoints)]
965
966          # Flush execution log entries to JSONL at the iteration boundary.
967          if self._logger:
968              self._logger.flush()
969
970          # One consolidated nav_step per root iteration, from every
971          # graphOut's harvest snapshot.
972          if is_root:
973              await self._broadcast_step(session)
974
phase 2/4 Β· settle β€” drain this iter's leftover sinks ── graph_executor.py:975–978 Β· ↔ Β§4.3 β‘‘
976          if await self._settle_drain(io_scope_id, io_scope, is_root, ready_queue, session):
977              return True
978
phase 3/4 Β· decide β€” stop input, then budget check ── graph_executor.py:979–1030 Β· ↔ Β§4.3 β‘’
980          # The stop signal is structurally bound to its scope (it sits on
981          # the scope's own iterOut) and is read exactly once per iteration,
982          # after the settle drain and before the handoff. Unwired stop =
983          # budget-only loop. Termination (stop or budget) emits the terminal
984          # iteration's values exactly once on the final_* handles.
985          if stop_requested(result):
986              log.info(
987                  "Stop: iterOut %s stop=True at scope=%s step=%d",
988                  node.id,
989                  io_scope_id or "(graph)",
990                  scope_step,
991              )
992              if io_scope is not None:
993                  io_scope.terminated = True
994              self._emit_final_side(
995                  node, result, io_scope_id, ready_queue=None if is_root else ready_queue
996              )
997              if is_root:
998                  self.terminated = True
999                  return True
1000
1001         scope_budget = io_scope.step_budget if io_scope is not None else self.step_budget
1002         if scope_step >= scope_budget:
1003             log.info(
1004                 "Step budget (%d) exhausted for scope=%s", scope_budget, io_scope_id or "(graph)"
1005             )
1006             if io_scope is not None:
1007                 io_scope.terminated = True
1008             self._emit_final_side(
1009                 node,
1010                 result if isinstance(result, dict) else {},
1011                 io_scope_id,
1012                 ready_queue=None if is_root else ready_queue,
1013             )
1014             if is_root:
1015                 self.terminated = True
1016                 return True
1017             # Inner-scope budget exhaust: flush the inner graphOut latches
1018             # and let the outer loop continue; no handoff β€” the scope is done.
1019             self._propagate_graphout_latches(io_scope, ready_queue)
1020             return False
1021
1022         if step_delay_ms > 0 and is_root:
1023             await asyncio.sleep(step_delay_ms / 1000)
1024
1025         # An inner scope stopped above without halting the root loop: flush
1026         # its graphOut latches and skip the handoff.
1027         if io_scope is not None and io_scope.terminated:
1028             self._propagate_graphout_latches(io_scope, ready_queue)
1029             return False
1030
phase 4/4 Β· hand off β€” carry β†’ paired iterIn, next iteration ── graph_executor.py:1031–1042 Β· ↔ Β§4.3 β‘£
1032         # The fired bundle lands in the paired iterIn's ``iterout_<x>``
1033         # slots (``stop`` is not loop-carried; final_* handles never fire
1034         # here) and the iterIn is queued for the next iteration.
1035         paired_id = node.config.get("pairedWith", "")
1036         paired = self.nodes.get(paired_id) if paired_id else None
1037         if paired is not None and paired.carry is not None:
1038             paired.carry.handoff(result if isinstance(result, dict) else {})
1039             self._enqueue(ready_queue, paired.id)
1040         node.pending_inputs = {}
1041         return False
1042
async def _settle_drain(…) ── graph_executor.py:1043–1102 Β· ↔ Β§4.3 β‘‘
1043     async def _settle_drain(
1044         self,
1045         io_scope_id: str,
1046         io_scope: _ScopeState | None,
1047         is_root: bool,
1048         ready_queue: list[str],
1049         session: Any,
1050     ) -> bool:
1051         """Boundary phase 2/4: fire the just-closed iteration's leftover
1052         sinks (viewer / telemetry nodes queued behind the iterOut) BEFORE
1053         the stop check, so they still emit on the terminal step.
1054
1055         Restricted to nodes of the iterOut's own scope β€” outer / peer-scope
1056         nodes are not drained by an inner boundary β€” with one exception: a
1057         ROOT boundary also drains graph-scope nodes. Dead-end sinks are
1058         never on a path to any iterOut, so scope analysis leaves them in
1059         the graph scope; without this they would miss the terminal step
1060         entirely (mid-loop it merely fires them earlier than the main loop
1061         would). iterIn is never drained β€” it advances to the next iteration
1062         and is fired only via the handoff after the stop check.
1063
1064         Bounded by ``max_settle`` fires. Returns ``True`` when a drained
1065         node raised ``StopExecution`` β€” the run halts exactly as it does
1066         for a StopExecution in the main loop.
1067         """
1068         max_settle = 64  # safety cap against pathological queues
1069         settled = 0
1070         while ready_queue and settled < max_settle:
1071             next_idx = None
1072             for i, nid in enumerate(ready_queue):
1073                 cand = self.nodes.get(nid)
1074                 if cand is None or cand.type == "iterIn":
1075                     continue
1076                 cand_scope = self._scope_of(nid)
1077                 if (
1078                     io_scope is not None
1079                     and cand_scope != io_scope_id
1080                     and not (is_root and cand_scope == GRAPH_SCOPE_ID)
1081                 ):
1082                     continue
1083                 next_idx = i
1084                 break
1085             if next_idx is None:
1086                 return False  # nothing to settle in this scope
1087             nid = ready_queue.pop(next_idx)
1088             cand = self.nodes[nid]
1089             try:
1090                 r = await self._fire(cand, session)
1091             except StopExecution as e:
1092                 log.info(
1093                     "StopExecution from %s during settle: %s at step %d",
1094                     nid,
1095                     e.reason,
1096                     self.step_counter,
1097                 )
1098                 self.terminated = True
1099                 return True
1100             self._after_fire(cand, r, ready_queue)
1101             settled += 1
1102         return False

7. Ways a run ends

ConditionWhereEffect
iterOut.stop truthyDecide phase, iterOut boundaryscope ends + final side emits; root β†’ break
scope_step β‰₯ budgetDecide phase, after the stop checkscope ends + final side emits; root break, inner continue
StopExecutionraised by node code (escape hatch)terminated=True, break now; fallback emission covers the final side
stop_eventmain-loop gateuser Stop β†’ break; fallback emission
queue emptymain-loop conditiongraph exhausted (DAGs end here); fallback emission for loop graphs
total_firings β‰₯ maxmain-loop conditionbackstop budget Γ— nodes Γ— 3; fallback emission

8. Execution properties

Concurrencysingle-threaded async β€” one node at a time; forward() may await
Determinismedge-declaration order + FIFO queue fix firing order
CyclesIterIn/IterOut pairedWith transfer, not edges
Step countingonce per IterOut fire, per scope; root mirrors self.step_counter
Stateper-node hidden + shared grant-gated StateContainers; a grant to a container homed in another process resolves to a RemoteContainerProxy (see State Containers Β§9)
Errorsone policy in _fire: capture as {"error":…} + bus event + node_errors ledger (conviction at run end); AGENTCANVAS_STRICT_ERRORS=1 (config.strict_errors_enabled) re-raises

9. File reference

app/agent_loop/graph_executor.pyGraphExecutor β€” the scheduling loop (run) and its seams _fire / _report_node_error / _deliver / _after_fire / _iter_boundary / _settle_drain; NodeInstance (carry: LoopCarry on an iterIn), _ScopeState, _NodeStateProxy
app/agent_loop/pivots.pyThe pivots' runtime rules: LoopCarry (iterIn slots β€” write / handoff / ready / fire_inputs / after_fire / last_carried), iterout_port_names / stop_requested / final_payload, graphout_snapshot / graphout_last / graphout_latch / graphout_latched
app/agent_loop/loop_runner.pyLoopRunner β€” lifecycle, policy loading
app/agent_loop/builtin_nodes.pyCATALOG (the process-wide NodeCatalog) + built-in handlers
app/components/catalog.pyNodeCatalog β€” registration, ports_of(type, config) (the one required-port / wire-type rule), schema
app/agent_loop/graph_run.pyGraphRun / RunSession / RunOutcome β€” one executor per run, outcome harvest
app/agent_loop/scope_analysis.pyanalyze_scopes β†’ ScopeForest
app/agent_loop/flatten.pyflatten_graph + FlattenMap
app/components/bases.pyFireList / FireSpec
app/api/execution/run.pyPOST /api/navigate/run (ADR-platform-002)
AgentCanvas docs