Graph Executor
How a canvas graph runs β diagrams first, one annotated source listing second
This page is in two parts. Part I teaches the concepts with diagrams. Part II prints the engine as one continuous annotated listing β colored region bands segment run() so you can see at a glance which block does what β keyed to agentcanvas/backend/app/agent_loop/graph_executor.py.
1. The drawn graph and its contract
A canvas graph is nodes joined by typed wires. Its execution semantics rest on exactly two things: a firing rule that decides when any node runs (Β§1.1), and one recognised structure β the loop pivot pair β that expresses iteration, halting, and the episode verdict (Β§1.2). Everything else on the canvas is ordinary dataflow. This section states the contract; Β§2βΒ§5 describe the algorithm that honours it.
1.1 The firing rule
Three consequences follow, and they shape everything below:
- Order is derived, not authored. There is no step list; whichever node is ready runs. A branch that never receives data never runs, so conditional paths need no special construct.
- Inputs are consumed. A fire empties the ports it read; the node fires again only when fresh values arrive. One delivery is one firing opportunity, not a standing subscription.
- No drawn cycles. The graph is acyclic as drawn β a wired cycle would give the data no valid order. Iteration is expressed instead by the pivot structure of Β§1.2 (ADR-executor-001).
1.2 Anatomy of a loop
Plain dataflow ends when data stops flowing β enough for a pipeline, not for an agent that must observe, act, and observe again. Iteration is expressed by the one structure the engine recognises: a paired IterIn / IterOut, the loop pivots. The smallest loop:
Reading the figure in run order:
- Run start. The entry node fires once β it needs no inputs β and its
init_*edges hand the run-invariants (instruction, episode config) to the IterIn's init side. - Each iteration. The IterIn emits the loop-carry bundle; the body β here
predictandenv_step, ordinary nodes with no special treatment β computes; the IterOut collects the next carry. The engine hands the carry back to the IterIn by pairing: the top edge is not a wire you draw, and the drawn graph stays acyclic. - Halting. Any BOOL wired into the IterOut's
stopinput β here the env'sdoneβ ends the loop after the current iteration. Left unwired, the loop runs to its step budget. - Termination. The IterOut's
final_*handles mirror its carry ports (plus a constantfinal_stop). They are real wires, but deliver exactly once β the terminal iteration's values. - Verdict. Everything downstream of
final_*β hereevaluateandgraphOutβ is the after-loop band: it runs once, after the loop, and may not take inputs from the loop body.
The drawing therefore has three places, and a run three times β the map that View 2 walks in order:
| Place in the drawing | When it runs | Engine view |
|---|---|---|
init_* edges into the IterIn | once, at run start | pre-loop β Β§3 |
| the body between the pivots | every iteration | in-loop β Β§4 |
downstream of final_* β the after-loop band | once, at termination | after-loop β Β§5 |
2. Where the engine sits
One class owns everything this page describes β GraphExecutor, entered through a single method, run(graph, session, β¦). The layers above it (HTTP endpoint, LoopRunner, GraphRun) own lifecycle β pause, stop, resume β and never look inside the graph; the layer below (the node handlers) computes and never schedules. The executor is the only place that decides what fires when.
A fresh executor is built per run, so no state leaks between episodes. For parallel batch eval it exposes two per-worker overrides β get_env_panel(name) and get_server_url(nodeset) β so each eval worker talks to its own env subprocess (ADR-eval-002); outside batch eval they pass through.
Inside run(), every run has the same three-phase shape β the outline of the rest of View 2, and the engine-side mirror of Β§1.2's three places: pre-loop builds live state and primes the queue (Β§3) Β· in-loop fires until a halt (Β§4) Β· after-loop delivers the verdict and finalises (Β§5).
3. Pre-loop β build & discover entries
Before a single node fires, run() does a deterministic, once-per-run setup in two movements. BUILD turns the static graph JSON into live, mutable runtime state; ENTRY DISCOVERY finds the entry nodes β the nodes that can fire with no inputs β and primes the queue with them. Nothing here loops β it is pure preparation, identical on every run.
3.1 BUILD β compile the graph into live state
Seven steps, strictly in order β plain top-to-bottom code, no concurrency. Each step turns one part of the authored GraphDefinition into something the loop can read or mutate. (Run-start args need no build step of their own: they arrive over ordinary canvas edges into the iterIn's init_* handles β ADR-dataflow-008.)
| # | Step | What it does | Why this order |
|---|---|---|---|
| 1 | Merge hooks | merge the shell-hook lists from all three levels β global, graph, per-node β into one runner | hook precedence must be settled before anything can fire |
| 2 | Flatten | expand every composite node's inner subgraph in place, leaving one flat node list (ADR-dataflow-001) | early, so every later step sees the flat graph β no hidden subgraphs |
| 3 | State containers | create the shared state containers and record which node is allowed to read/write which container (the access grants) | shared state must exist before any node touches it |
| 4 | Resolve step_budget | fix the per-episode iteration cap to one number (see chain below) | the loop's halt check needs a fixed cap |
| 5 | Node instances | create one live NodeInstance per authored node β its own input buffer (pending_inputs) and persistent state | this is where the static definition becomes mutable per-run state |
| 6 | Scope state | find every (iterIn, iterOut) loop and how loops nest; give each loop its own step counter and budget. Nodes outside any loop land in the implicit graph scope | nested loops count steps independently; a root-loop stop halts the run, an inner one only ends its own loop |
| 7 | Adjacency | index every node's outgoing edges by source id | routing a fired node's outputs becomes a single lookup in the loop |
step_budget_override (passed in by the eval-batch runner β e.g. HM-EQA's scene-adaptive int(βscene Β· 3)) β the graph's authored step_budget β the framework default. The override is passed in as an argument, never written back onto the shared graph object, so parallel eval workers can each carry a different per-episode budget. (This is the anchor batch-eval links to for the resolver chain.)
3.2 ENTRY DISCOVERY β find the entry nodes, prime the ready-queue
An entry node is discovered automatically β no author marking β as any node satisfying all three structural conditions; every entry node is queued at run-start:
- not an
iterInβ IterIn is the loop's entry; it fires only when itsinit_*edges deliver (iter 0) or via the IterOut transfer (iter 1+), never via entry discovery; - no incoming edges β nothing upstream will ever fill it;
- no required input ports β it can fire on nothing.
validate_graph_connectivity (in graph_def.py) rejects required-but-unwired ports at load time, so the third condition only ever excludes well-formed leaf consumers β it never silently drops a real entry node. With the queue primed, three things arm the loop:
run_startsignal β state containers withlifetime="run"reset to a clean baseline;- the
GraphStarthook fires; max_firings = step_budget Γ nodes Γ 3is computed β a runaway-loop backstop set far above any real run.
init_* handles) or via an entry node that fills IterIn's iterout_* slots on iteration 0. A loop whose IterIn has no incoming init edge will never start β the body simply starves.
4. In-loop β the firing loop
With the queue primed, the engine starts turning. This chapter walks one turn from the inside out β the loop skeleton (Β§4.1), the guards before each fire (Β§4.2), the IterOut boundary that closes every iteration (Β§4.3), the propagation path every other node takes (Β§4.4) β then reassembles the pieces into the whole-run flowchart (Β§4.5) and a concrete trace (Β§4.6).
4.1 One iteration & the loop skeleton
Keep Β§1.2's graph in mind β entry node, two pivots, a two-node body; this chapter is what the engine does with it. The main loop is deliberately tiny. Each turn:
- Gate. Run while the queue holds a node and total firings stay under the runaway cap (
while ready_queue and total_firings < max_firings); honour pause; a user Stop breaks out. - Pop. Take the front of the FIFO ready-queue.
- Guard, then fire. Two structural guards (Β§4.2), then
_fireβ the one error policy, wrapping the per-node fire path_fire_node(graph_executor.py). - Fork. An IterOut runs the once-per-iteration boundary (
_iter_boundary, Β§4.3); every other node takes the ordinary delivery path (_after_fireβ_deliver, Β§4.4).
Nothing else is special: an agent's body is plain dataflow firing, and IterOut is the only node type the engine schedules differently. The loop itself only schedules; the work sits in three seams it calls at fixed points β _fire, _iter_boundary, _after_fire β _deliver β and the pivots' slot / latch / snapshot rules (LoopCarry, final_payload, graphout_*) live in agent_loop/pivots.py, not in the loop.
4.2 The guards before a fire
A popped node is not fired blindly β two cheap structural guards run first, in order; each either skips the node or readies state for the fire:
- Scope barrier. A node belonging to an inner scope that has already terminated is frozen (
pending_inputscleared, fire skipped) until that scope re-enters β otherwise in-flight inner-body nodes fire one extra time after termination and leak per-iteration values through cross-scope wires. Exempt:iterIn/graphIn, the cascade that reopens the scope. - IterIn step_start. An
iterInbroadcastsstep_start(per scope) solifetime="step"state resets to a clean slate for the iteration about to run; a re-fired iterIn whose inner scope was terminated resets that scope'sterminatedflag + counter β the multi-scope re-entry path.
Then the node fires β through _fire, the one error policy shared by every fire site (the main loop, the settle drain, the after-loop pass). A StopExecution raised by the fire halts the run immediately. Any other exception is published to the event sink (NODE_EXEC_FAIL), recorded in the per-run node_errors ledger for the end-of-run conviction (Β§5.2), and the node's result becomes {"error": β¦} β which routes to no declared port, so downstream starves rather than consuming garbage; under strict errors (config.strict_errors_enabled(), env AGENTCANVAS_STRICT_ERRORS) it re-raises after the bus emit and fails fast instead. A returned {"error": β¦} result is convicted the same way (NODE_RESULT_ERROR) unless the node declares an error output port, in which case the key is data.
(Until 2026-06-11 there were four guards. The termination-defer and post-loop-skip guards were deleted with the termination node and config.post_loop β the stop signal and the after-loop stage now live on the iterOut pivot itself, so there is nothing left to defer or skip.)
4.3 The IterOut boundary β four phases
IterOut fires exactly once per iteration, so the engine hangs every per-iteration chore on it (_iter_boundary). The order of the four phases is load-bearing β in particular Settle runs before Decide:
What each phase does, and the subtle rule inside it:
- β Record. Advance the per-scope step counter (the root scope's counter also mirrors into
self.step_counter+session._current_step); broadcaststep_end; checkpoint all state containers (root scope only); flush the execution log; broadcast the consolidatednav_step(root only). IterOut's own outgoing edges are not routed per iteration β they are final-side handles, emitted once at termination (β’). - β‘ Settle. Drain this iteration's leftover sinks (bounded at 64, restricted to the iterOut's own scope, skipping only
iterIn). This is where in-loop viewer / telemetry sinks scheduled in this iteration's wave still emit β including on the terminal step. One exception to the scope filter (2026-07-04): a root boundary also drains graph-scope nodes β dead-end sinks are never on a path to any iterOut, so scope analysis leaves them in the graph scope, and without the exception they would miss the terminal step entirely (the run exits with them still queued). Mid-loop this only fires them earlier than the main loop would have; inner boundaries keep the strict filter. Drained nodes fire through_firelike any other; aStopExecutionraised here halts the run exactly as it would from the main loop (_settle_drain). - β’ Decide. Read the just-fired iterOut's own
stopinput: truthy means this scope is done. A root-scope stop halts the run; an inner one ends only that loop. Then the per-scopestep_budgetexhaust check. Either way, termination triggers the final-side emission: the terminal iteration's collected values go out once on thefinal_*handles (root β seeds the after-loop stage, Β§5; inner β feeds outer-body consumers and the graphOut latch flush). The stop signal sits on the scope's own iterOut, so it is structurally bound to the right loop and read exactly once per iteration. - β£ Hand off. Sleep
step_delay_ms(root only), then write each IterOut outputXinto the paired IterIn'siterout_Xslot (LoopCarry.handoff) and re-enqueue the IterIn β arming the next iteration. (stopis not loop-carried, and the final-side handles never fire here β only at termination.)
evaluate whose terminal-step fire depended on the settle drain catching it β the SIMPLER / smartway metric-drop bug class (roadmap #64: 5/20 successes recorded as 0). Now the verdict consumes final_* values handed over by the engine at termination, so it is correct by construction; in-loop sinks are telemetry only.
4.4 Ordinary nodes & propagation
Every non-IterOut fire ends the same way (_after_fire), in four moves:
- Consume. The node's
pending_inputsare cleared β inputs are used up by the fire. - Pivot cases. An
iterInadditionally drops itspersist=falseslots (LoopCarry.after_fire), so one-shot values emit once and are gone. An inner-scopegraphOutbuffers its value into a latch (pivots.graphout_latch,state["latched_value"]) instead of delivering outward β the latch is flushed to outer-scope consumers only when its scope terminates (_propagate_graphout_latches). - Deliver. Everything else goes through
_deliver, the one delivery rule: for every outgoing edge the value ispayload[sourceHandle]β the whole payload for thedefaulthandle, nothing when the handle is absent (so an error-shaped result routes nowhere) β and is written onto the target's input port by_route_value_to_port. This is also whereLIST[T]coercion happens (ADR-027): a scalar producer feeding a list consumer is auto-wrapped, and fan-in is concatenated in edge order; a write landing on an iterIn goes into itsLoopCarryslot. - Enqueue. Any target whose required inputs are now all present joins the ready-queue (
_enqueuededupes and stamps the ready-time).
The same _deliver serves the final side and the after-loop pass with terminal=True: a graphOut reached there is routed-to and recorded β graph scope: the harvest snapshot (pivots.graphout_snapshot); inner scope: the latch β but never fired.
(Run-start args need no special branch: the iterIn's init_* handles are fed by ordinary canvas edges β ADR-dataflow-008 two-sided iterIn.)
4.5 The whole run, end to end
Zoom out from one iteration to the whole of run(). The flowchart below stitches Β§3 and Β§4 into one picture, with the ready-queue at its centre β ENTRY DISCOVERY primes it, every turn pops it, and both return paths feed it again. Read it top-down:
- Set up once. BUILD compiles the graph JSON into live state (Β§3.1); ENTRY DISCOVERY finds the no-input entry nodes and primes the ready-queue (Β§3.2).
- Turn the loop. While the queue has a ready node β and neither the global firing cap nor a user Stop has hit β pop the front one and fire it: guards first (Β§4.2), then
_fire. - Fork on what fired. An ordinary node takes the fast path: route its outputs, enqueue whatever became ready (Β§4.4), turn again β this is almost every fire. An IterOut instead runs the once-per-iteration boundary (Β§4.3): Record, Settle, Decide, Hand off. Unless Decide halts, Hand off arms the paired IterIn and the loop turns again β that is one full iteration.
- Exit. Two doors out. Decide halts β the stop signal or the step budget β and the terminal iteration's values leave exactly once on the
final_*handles. Or the main loop runs dry β queue drained, firing cap, user Stop β and the fallback reconstructs the final-side values best-effort. Every door lands in the same after-loop + finalise stage (Β§5); Β§7 tables every exit, including theStopExecutionescape hatch.
That is the entire control flow β and the engine-view counterpart of Β§1.2: there the boxes were the nodes of one drawn graph; here they are the engine's actions on any graph. Part II Β§6 prints this same shape as code β the colored region bands there are exactly these boxes.
4.6 A concrete trace
One episode of the Β§1.2 graph, turn by turn. Watch the ready-queue column β it is the hub of the Β§4.5 flowchart; entry discovery put entry in it (and only entry: IterIn is never an entry node, Β§3.2). Each turn pops the front, fires it, and the effect column says what that made ready next.
| Turn | ready-queue | Fires | Effect |
|---|---|---|---|
| 1 | entry | entry | init_* edges deliver the run-invariants into iter_in's init slots β iter_in ready |
| 2 | iter_in | iter_in | emits the loop-carry bundle β predict ready |
| 3 | predict | predict | {action} β env_step ready |
| 4 | env_step | env_step | {obs, done: false} β obs fills iter_out's carry port, done lands on its stop input β iter_out ready |
| 5 | iter_out | iter_out β the Β§4.3 boundary | β Record step 1 Β· β‘ Settle leftover sinks Β· β’ Decide: stop false, budget fine β continue Β· β£ Hand off: carry into iter_in's slots β iter_in re-enqueued |
| β³ | iter_in | turns 2β5 repeat β iterations 2 β¦ Nβ1 | |
| 6 | env_step | env_step | iteration N: {obs, done: true} β the stop input is now truthy |
| 7 | iter_out | iter_out | β’ Decide: stop true β the final side emits once (terminal obs, final_stop=True) β evaluate ready; the loop ends |
| 8 | evaluate | evaluate β after-loop | reads the terminal values off final_* β verdict; graphOut re-snapshotted, metrics harvested (Β§5) |
Note what never happens: evaluate appears in the queue exactly once, at turn 8 β during the loop its inputs simply never arrive (Β§5.1), so the verdict cannot read anything but the terminal iteration.
5. After-loop β the verdict stage & finalise
However the loop exits β the stop signal fired, the step budget hit, the queue drained, or a node crashed β control lands in one finalise stage. It has four movements: the after-loop pass (Β§5.1), the node-error conviction gate and clean finalisation (Β§5.2), and the error path (Β§5.3).
5.1 The after-loop pass
The after-loop band is every node wired downstream of the root iterOut's final_* handles β the verdict chain, typically evaluate β graphOut, possibly with adjudicator steps in between. Two facts keep it dormant while the loop runs: the final side never emits mid-loop, and the validator forbids edges from the loop body into the band. Band nodes therefore cannot become ready before termination β there is nothing to skip, defer, or flag.
When the final-side values arrive, _after_loop_pass drains whatever they made ready: fire a ready node (through _fire, so a failure here is convicted like any other), deliver its outputs (_deliver(terminal=True)), enqueue the consumers that became ready in turn. A node whose upstream band node hasn't fired yet is re-queued behind it, so a chain resolves in dependency order without a topological sort; the queue is bounded (max(nodesΓ4, 64)), so a never-satisfiable node cannot spin forever.
Two rules keep the pass honest:
- Graph-scope only. The loop is over, so re-firing a loop-body node β or any
iterIn/iterOut, or any node inside an author (loop) scope β is meaningless and barred. Only graph-scope nodes (the band and its sinks) fire. - graphOut is harvested, not fired. A graph-scope
graphOutthe chain reaches is routed-to and re-snapshotted (_deliver(terminal=True)βpivots.graphout_snapshot, intostate["_last_inputs"]β the channel the metric harvest reads) but never itself fired; an inner-scope graphOut is latched instead (its flush is scope-bound).
Membership is topological, not declared: there is no flag to set (the former config.post_loop was removed 2026-06-11) β a node is after-loop because it hangs off the final side. The input contract is engine-guaranteed: the band receives exactly the terminal iteration's values ("verdict inputs ride the pivot" β anything the verdict needs must be an iterOut port, consumed via final_<name>). If the loop ends without a boundary emission (drained queue, max_firings, user stop, or a crash), _emit_final_fallback reconstructs the last completed iteration's values from the paired iterIn's iterout_* slots (LoopCarry.last_carried) and emits best-effort before the drain.
5.2 Clean finalise
The conviction gate comes first (2026-07-04). At fire time _fire (Β§4.2) records every node failure β a raised exception, or an {"error": ...}-shaped result such as a server-mode proxy surfacing an HTTP 500 β into a per-run node_errors ledger (nodes that declare a legitimate error output port are exempt; returned errors also emit a NODE_RESULT_ERROR bus event). If the ledger is non-empty here, a NodeErrorAggregate naming the culprits is raised and control transfers to the error path (Β§5.3). The placement is deliberate: the verdict stage (Β§5.1) has already run, so final-side metrics are collected either way β but the run can no longer masquerade as completed while its downstream silently starved (the old status="completed" / step_count=0 eval pathology).
On a clean exit (empty ledger), in order:
- Harvest metrics β from any node's
state["metrics"]; a value stored onsession._metricsoverrides node values. - Broadcast
run_endβlifetime="run"state containers clear. - Broadcast
nav_completeβ final step and metrics, to the frontend. - Fire the
GraphCompletehook. - Flush the execution log. Session status becomes
done.
5.3 The error path
If the loop body raises β or the conviction gate fired (Β§5.2) β the finalise stage still runs, and runs the verdict first:
- Fallback emission + after-loop pass, best-effort. The ordering is deliberate: even after a crash the verdict
evaluatecan often still emit fresh metrics from the env's real final step, so the harvest reports the true result rather than a stale pre-crash value. The pass is tolerant β a band node that raises because the env is already torn down is logged and skipped (and both stages are idempotent, so re-entry from the conviction gate is a no-op). - Publish the exception to the ErrorBus β
GRAPH_CRASHfor a real crash,NODE_ERRORSfor a conviction; session status becomeserror, broadcast to the frontend as anav_statuserror. run_endstill fires, carrying the error β run-lifetime state resets even on a crash.- Fire the
GraphErrorhook, then the final log flush.
6. The run() listing β synced from source
The full method with its real comments: the annotations you read here are the source file's comments, so the listing and the engine cannot disagree about what a block does. The blue band headers are the file's own # βββ banner comments β the same phase vocabulary as Part I β with the matching section linked on each band, and the line numbers are real. After editing run() or one of the seams, refresh with python3 docs/_lib/_sync_run_listing.py; the sync stamp below names the commit it was last generated against.
Synced from graph_executor.py:308β775 at 1e54b3e by docs/_lib/_sync_run_listing.py β the band headers are the file's own banner comments; re-run the script after editing the source.
run() β signature & docstring ββ graph_executor.py:308β337 308 async def run( 309 self, 310 graph: GraphDefinition, 311 session: Any, 312 execution_id: str | None = None, 313 step_delay_ms: int = 200, 314 stop_event: asyncio.Event | None = None, 315 pause_event: asyncio.Event | None = None, 316 global_hooks: list[Any] | None = None, 317 step_budget_override: int | None = None, 318 ) -> None: 319 """Main graph execution loop. 320 321 ``step_budget_override`` (when set) wins over the graph's authored 322 ``step_budget``. Used by the eval batch resolver chain to push an 323 env-supplied per-episode value (e.g. HM-EQA's 324 ``int(sqrt(scene_size) * 3)``) without mutating the shared graph 325 object across worker coroutines. 326 327 Structure (the ``# βββ`` banners below mark the phases): 328 pre-loop (BUILD steps 1-7, then ENTRY DISCOVERY) β in-loop (the 329 scheduling loop; the iterOut boundary β ``_iter_boundary`` β closes 330 each iteration in four phases: record β settle β decide β hand off) 331 β after-loop (the verdict stage) β finalise. The loop fires through 332 ``_fire`` (the one error policy) and delivers through 333 ``_after_fire`` / ``_deliver`` (the one delivery rule). Narrated 334 walkthrough with diagrams: 335 docs/pages/developer-guide/design-docs/graph-executor.html (Part I). 336 """ 337 pre-loop Β· BUILD β compile the static graph into live state ββ graph_executor.py:338β458 Β· β Β§3.1 339 340 # BUILD 1/7 β merge hooks from 3 sources: global β graph β node (R3) 341 node_hooks = extract_node_hooks(graph.nodes) 342 merged = merge_hooks(global_hooks or [], graph.hooks, node_hooks) 343 self._hook_runner = HookRunner(merged) 344 345 # BUILD 2/7 β flatten composite nodes before execution (ADR-dataflow-001) 346 from .flatten import flatten_graph 347 348 graph, self._flatten_map = flatten_graph(graph) 349 350 # BUILD 3/7 β state containers + access grants from the graph definition 351 if graph.containers: 352 from .state_containers import build_containers 353 354 self.containers = build_containers(graph.containers) 355 for ag in graph.access_grants: 356 self._access_grant_index.setdefault(ag.node_id, set()).add(ag.container_id) 357 log.info( 358 "State containers: %d containers, %d access grants", 359 len(self.containers), 360 len(graph.access_grants), 361 ) 362 # If a container with the well-known id "graph_state" exists, 363 # set up the convenience binding. Access is still gated by 364 # explicit access grants β no auto-inject. 365 if "graph_state" in self.containers: 366 self._graph_state_id = "graph_state" 367 368 # BUILD 4/7 β resolve the step budget. Order: explicit override β 369 # graph authored value β framework default. The eval batch runner 370 # computes the override via its resolver chain (env-dynamic β API 371 # override). 372 from ..config import DEFAULT_STEP_BUDGET as _DEFAULT_STEP_BUDGET 373 374 if step_budget_override is not None: 375 self.step_budget = step_budget_override 376 elif graph.step_budget is not None: 377 self.step_budget = graph.step_budget 378 else: 379 self.step_budget = _DEFAULT_STEP_BUDGET 380 self.edges = [e.to_dict() for e in graph.edges] 381 node_defs = graph.nodes 382 383 if not node_defs: 384 log.error("Empty graph β nothing to execute") 385 return 386 387 # BUILD 5/7 β node instances: the static definition becomes 388 # per-run mutable state (pending_inputs buffer + persistent state) 389 for nd in node_defs: 390 self.nodes[nd.id] = NodeInstance( 391 id=nd.id, 392 type=nd.type, 393 config=nd.config, 394 label=nd.label, 395 ) 396 397 # BUILD 6/7 β per-scope execution state. Always includes a 398 # synthetic GRAPH_SCOPE_ID entry; for single-scope graphs there is 399 # also one entry keyed by the outermost author scope's iter_in id. 400 # Multi-scope graphs add one entry per author scope. The root 401 # scope's stop signal halts the entire run; a non-root scope's stop 402 # only ends that inner loop and lets the outer one continue. 403 self.scope_forest, scope_errors = analyze_scopes(graph) 404 if scope_errors: 405 for err in scope_errors: 406 log.warning("Scope analysis: %s", err) 407 # Always create the synthetic graph-scope entry β holds graph-scope 408 # nodes (run-start entry nodes, after-loop sinks). 409 self.scope_state[GRAPH_SCOPE_ID] = _ScopeState( 410 scope_id=GRAPH_SCOPE_ID, 411 parent_scope_id=None, 412 step_budget=self.step_budget, # graph-level cap also applies here 413 member_node_ids=set(self.scope_forest.graph_scope_node_ids), 414 ) 415 # Per-author-scope state; resolve step_budget with fallback chain 416 # (scope's own iter_in.step_budget β parent scope's step_budget β 417 # graph step_budget). 418 for scope_id, info in self.scope_forest.scopes.items(): 419 resolved_budget = info.step_budget 420 if resolved_budget is None: 421 # Walk up parent chain looking for a budget; fall back to graph 422 p = info.parent_scope_id 423 while p is not None and resolved_budget is None: 424 p_info = self.scope_forest.scopes.get(p) 425 if p_info is None: 426 break 427 resolved_budget = p_info.step_budget 428 p = p_info.parent_scope_id 429 if resolved_budget is None: 430 resolved_budget = self.step_budget 431 self.scope_state[scope_id] = _ScopeState( 432 scope_id=scope_id, 433 parent_scope_id=info.parent_scope_id, 434 step_budget=resolved_budget, 435 member_node_ids=set(info.member_node_ids), 436 graphout_node_ids=list(info.graphout_node_ids), 437 iter_out_id=info.iter_out_id, 438 ) 439 # The "outermost" scope id β used for backward compat: self.step_counter 440 # and self.terminated mirror this scope's state. For 0-scope graphs 441 # this stays GRAPH_SCOPE_ID; for single-scope graphs it's the sole 442 # author scope's id; for multi-scope graphs it's the (single) root. 443 if len(self.scope_forest.root_scope_ids) == 1: 444 self._outermost_scope_id = self.scope_forest.root_scope_ids[0] 445 elif len(self.scope_forest.root_scope_ids) > 1: 446 # Multiple peer roots β pick the first deterministically; legacy 447 # self.step_counter mirrors only its counter (multi-root graphs 448 # are an advanced shape; consumers should read scope_state directly). 449 self._outermost_scope_id = self.scope_forest.root_scope_ids[0] 450 else: 451 self._outermost_scope_id = GRAPH_SCOPE_ID 452 453 # BUILD 7/7 β adjacency: outgoing edges indexed by source id, so 454 # routing a fired node's outputs is a single lookup in the loop 455 for edge in self.edges: 456 src = edge.get("source", "") 457 self.adjacency.setdefault(src, []).append(edge) 458 pre-loop Β· ENTRY DISCOVERY β prime the ready-queue, arm the loop ββ graph_executor.py:459β530 Β· β Β§3.2 460 461 # A node is an entry node if: explicitly marked OR (has no required 462 # inputs AND has no incoming edges). Nodes with incoming edges to 463 # optional ports should wait for data even if no required ports exist. 464 incoming_targets = set() 465 for edge in self.edges: 466 tgt = edge.get("target", "") 467 if tgt: 468 incoming_targets.add(tgt) 469 470 # Entry discovery: queue at run-start iff all three structural 471 # conditions hold: 472 # 1. type != "iterIn" (iterIn fires from init edges / iterOut transfer) 473 # 2. no incoming edges 474 # 3. no required input ports 475 # validate_graph_connectivity (graph_def.py) rejects required-but- 476 # unwired ports at load time, so condition 3 normally only excludes 477 # well-formed leaf consumers β never silently drops an entry node. 478 # System Log: ready-time stamps for queue_wait_ms (set on enqueue, 479 # popped on fire in _fire_node). Run-scoped β reset on each run(). 480 self._t_ready: dict[str, float] = {} 481 ready_queue: list[str] = [] 482 for node in self.nodes.values(): 483 if node.type == "iterIn": 484 continue 485 if node.id in incoming_targets: 486 continue 487 if self._get_required_ports_for_node(node): 488 continue 489 self._enqueue(ready_queue, node.id) 490 491 log.info( 492 "GraphExecutor: %d nodes, %d edges, entry_nodes=%s, step_budget=%d", 493 len(self.nodes), 494 len(self.edges), 495 [self.nodes[nid].label or nid for nid in ready_queue], 496 self.step_budget, 497 ) 498 499 stop = stop_event or asyncio.Event() 500 pause = pause_event or asyncio.Event() 501 if not pause.is_set(): 502 pause.set() 503 504 session._status = "running" 505 # Node failures recorded during this run β convicts the run at the 506 # finalise stage (see NodeErrorAggregate). 507 self.node_errors: list[dict] = [] 508 _suppress = getattr(getattr(session, "principles", None), "suppress_nav_events", False) 509 if not _suppress: 510 await broadcast(session._ws("nav_status", {"status": "running", "step": 0})) 511 512 # run_start signal fires before any node fires, so state 513 # containers with lifetime="run" reset to a clean baseline. 514 self.broadcast_signal("run_start", {"graph_name": graph.name}) 515 516 # GraphStart hook 517 if self._hook_runner.has_hooks(): 518 await self._hook_runner.run_hooks( 519 "GraphStart", 520 payload={ 521 "graph_name": graph.name, 522 "node_count": len(self.nodes), 523 }, 524 ) 525 await self._emit("graph_start", graph_name=graph.name, node_count=len(self.nodes)) 526 527 # Safety: max total firings to prevent infinite loops 528 max_firings = self.step_budget * len(self.nodes) * 3 529 total_firings = 0 530 in-loop Β· the scheduling loop β pop, guard, fire, pivot or deliver ββ graph_executor.py:531β649 Β· β Β§4.1βΒ§4.2 532 # The loop only schedules. The work happens in three seams called 533 # at fixed points: ``_fire`` (the one error policy), 534 # ``_iter_boundary`` (the iterOut pivot: record β settle β decide β 535 # hand off) and ``_after_fire`` β ``_deliver`` (the one delivery 536 # rule). 537 try: 538 while ready_queue and total_firings < max_firings: 539 # Pause/stop check 540 await pause.wait() 541 if stop.is_set(): 542 break 543 544 node_id = ready_queue.pop(0) 545 node = self.nodes.get(node_id) 546 if node is None: 547 continue 548 549 # Guard 1 β scope barrier. Once an inner scope is marked 550 # terminated (by the iterOut boundary's decide check at the 551 # end of the final inner iter), forbid any further fire of nodes belonging 552 # to that scope until the scope is re-entered. Without this 553 # guard, in-flight inner-body nodes that were enqueued 554 # before the scope stopped (e.g. ``move_to_pose`` already 555 # running, or ``episode_info`` triggered by it) fire one 556 # extra time AFTER the stop and leak per-iter outputs through any 557 # cross-scope wires (post-flatten direct edges from inner 558 # producers to outer consumers, OR direct graphOutβouter 559 # wires when graphOut is preserved), satisfying outer 560 # iterOut's required gates and causing outer to cycle 561 # without inner actually re-entering. 562 # 563 # Exemption set β scope-entry cascade nodes that MUST be 564 # allowed to fire so the next outer iter can reopen this 565 # scope: ``iterIn`` (existing re-entry path below clears 566 # the terminated flag; its init slots re-capture 567 # outer-supplied scope-entry args), ``graphIn`` (ferries 568 # outerβinner values across the boundary into iterIn's 569 # init side). Body nodes and iterOut stay barred β they 570 # have nothing useful to do between scope terminate and 571 # scope re-entry. 572 _n_scope_id = self._scope_of(node_id) 573 _n_scope = ( 574 self.scope_state.get(_n_scope_id) 575 if _n_scope_id and _n_scope_id != GRAPH_SCOPE_ID 576 else None 577 ) 578 if ( 579 _n_scope is not None 580 and _n_scope.terminated 581 and _n_scope.parent_scope_id is not None 582 and node.type not in ("iterIn", "graphIn") 583 ): 584 node.pending_inputs = {} 585 continue 586 587 # Guard 2 β iterIn step_start. iterIn marks the start of each 588 # iteration β emit step_start before the node fires so 589 # lifetime="step" states see a fresh slate for the upcoming 590 # iteration. 591 # Multi-scope re-entry: an inner scope's iter_in firing 592 # while its scope is marked terminated indicates outer just 593 # entered a new outer iter and is re-invoking inner. Reset 594 # inner scope's terminated flag and step_counter β inner 595 # gets a fresh loop. (Root-scope iter_in re-firing after 596 # termination would only happen if main loop didn't break; 597 # we still allow the reset for symmetry, though in practice 598 # root termination breaks the loop above.) 599 if node.type == "iterIn": 600 _ii_scope = self.scope_state.get(node.id) 601 if _ii_scope is not None and _ii_scope.terminated: 602 if _ii_scope.parent_scope_id is not None: 603 # Inner scope re-entry from outer iter β reset 604 log.info( 605 "Scope re-entry: %s (parent=%s) β resetting terminated/counter", 606 node.id, 607 _ii_scope.parent_scope_id, 608 ) 609 _ii_scope.terminated = False 610 _ii_scope.step_counter = 0 611 # Re-arm the final side: an inner scope emits 612 # final_* once per termination, i.e. once per 613 # outer iteration. 614 self._final_emitted.discard(node.id) 615 # Reset per-fire state on body nodes that hold 616 # transient counters (only those tracked via 617 # state['_scoped_reset']=True)? β out of scope 618 # for v1; user tests must use cumulative counters. 619 else: 620 # Root scope: should not happen (main loop breaks 621 # on root termination). Defensive: skip the fire. 622 node.pending_inputs = {} 623 continue 624 # step_start signal β add scope_id (additive, single-scope 625 # readers using just `step` keep working). 626 _next_step = ( 627 _ii_scope.step_counter + 1 628 if _ii_scope is not None 629 else self.step_counter + 1 630 ) 631 self.broadcast_signal( 632 "step_start", 633 {"step": _next_step, "scope_id": node.id}, 634 ) 635 636 # Fire the node. A failed fire yields an ``{"error": ...}`` 637 # result (or raises under strict errors); StopExecution is 638 # the node-side halt and ends the run here. 639 try: 640 result = await self._fire(node, session) 641 except StopExecution as e: 642 log.info( 643 "StopExecution from %s: %s at step %d", node.id, e.reason, self.step_counter 644 ) 645 self.terminated = True 646 break 647 648 total_firings += 1 649 iterOut boundary β the pivot closes this iteration ββ graph_executor.py:650β655 Β· β Β§4.3 651 if node.type == "iterOut": 652 if await self._iter_boundary(node, result, ready_queue, session, step_delay_ms): 653 break 654 continue 655 ordinary path β consume Β· pivot cases Β· deliver ββ graph_executor.py:656β665 Β· β Β§4.4 657 self._after_fire(node, result, ready_queue) 658 659 if total_firings <= 5: 660 log.info( 661 "After %s: queue=%s", 662 node.label or node.id, 663 [self.nodes[nid].label or nid for nid in ready_queue], 664 ) 665 after-loop Β· verdict stage β fallback emission + band drain ββ graph_executor.py:666β675 Β· β Β§5.1 667 # If the loop ended without a boundary final-side emission 668 # (drained queue, max_firings, user stop), reconstruct the last 669 # completed iteration's values from the paired iterIn's slots 670 # and emit best-effort; then drain the after-loop band β the 671 # nodes fed by the root iterOut's final_* handles (evaluate, 672 # graphOut chains). This is the run-end verdict stage. 673 self._emit_final_fallback() 674 await self._after_loop_pass(session) 675 conviction β surface node failures at end of run ββ graph_executor.py:676β689 Β· β Β§5.2 677 # Runs AFTER the verdict stage so final-side metrics are already 678 # collected; the raise routes through the error path below and 679 # the run finishes with status="error" instead of masquerading 680 # as completed (both stages are idempotent on the re-entry). 681 if self.node_errors: 682 _n_err = len(self.node_errors) 683 _head = "; ".join( 684 f"{e['node_id']}@step{e['step']}: {e['error']}" for e in self.node_errors[:3] 685 ) 686 raise NodeErrorAggregate( 687 f"{_n_err} node error(s) during run: {_head}" + ("; ..." if _n_err > 3 else "") 688 ) 689 finalise β clean exit ββ graph_executor.py:690β732 Β· β Β§5.2 691 session._status = "done" 692 metrics = None 693 # Find metrics from any node that stored them (generic β any step node) 694 for node in self.nodes.values(): 695 if node.state.get("metrics"): 696 metrics = node.state["metrics"] 697 if session._metrics: 698 metrics = session._metrics 699 700 # run_end signal β fires after the loop finishes cleanly. 701 # lifetime="run" states clear here. 702 self.broadcast_signal( 703 "run_end", 704 {"step": self.step_counter, "terminated": self.terminated}, 705 ) 706 707 if not _suppress: 708 await broadcast( 709 session._ws( 710 "nav_complete", 711 {"step": self.step_counter, "metrics": metrics}, 712 ) 713 ) 714 715 # GraphComplete hook 716 if self._hook_runner.has_hooks(): 717 await self._hook_runner.run_hooks( 718 "GraphComplete", 719 payload={ 720 "step": self.step_counter, 721 "terminated": self.terminated, 722 }, 723 ) 724 await self._emit( 725 "graph_complete", terminated=self.terminated, metrics=metrics 726 ) 727 728 # Final flush of any remaining log entries 729 if self._logger: 730 self._logger.flush() 731 732 except Exception as e: finalise β error path (verdict first, then error reporting) ββ graph_executor.py:733β775 Β· β Β§5.3 734 # Best-effort after-loop stage first: a final-side evaluate can 735 # still emit fresh metrics even when the main loop crashed, so 736 # eval harvest reports the actual final env step count instead 737 # of a stale pre-crash value. Tolerant of cascading failures 738 # (the env may be torn down already). 739 try: 740 self._emit_final_fallback() 741 await self._after_loop_pass(session) 742 except Exception as _fpe: 743 log.warning("after_loop_pass on error path raised: %s", _fpe) 744 get_bus().from_exception( 745 e, 746 source="graph", 747 code="NODE_ERRORS" if isinstance(e, NodeErrorAggregate) else "GRAPH_CRASH", 748 scope={ 749 "step": self.step_counter, 750 "execution_id": getattr(session, "_execution_id", None), 751 }, 752 title=( 753 f"Run finished with node errors at step {self.step_counter}" 754 if isinstance(e, NodeErrorAggregate) 755 else f"Graph execution crashed at step {self.step_counter}" 756 ), 757 ) 758 session._status = "error" 759 # run_end also fires on the error path so lifetime="run" 760 # states reset even when the loop crashes. 761 self.broadcast_signal( 762 "run_end", 763 {"step": self.step_counter, "error": str(e)}, 764 ) 765 if not _suppress: 766 await broadcast(session._ws("nav_status", {"status": "error", "error": str(e)})) 767 768 # GraphError hook β fires on unhandled exceptions 769 if self._hook_runner.has_hooks(): 770 await self._hook_runner.run_hooks("GraphError", payload={"error": str(e)}) 771 await self._emit("graph_error", error=str(e)) 772 773 # Flush log entries even on error 774 if self._logger: 775 self._logger.flush()
6.1 The seams β _fire Β· _deliver Β· _after_fire Β· _iter_boundary Β· _settle_drain
The methods the scheduling loop calls at fixed points, in source order, with the same band / Β§ convention. The iterOut boundary's four phases are the # ββ banners inside _iter_boundary.
Synced from graph_executor.py:777β1102 at 1e54b3e by the same script.
async def _fire(β¦) ββ graph_executor.py:779β818 Β· β Β§4.2 779 async def _fire(self, node: NodeInstance, session: Any) -> dict: 780 """Fire one node under the one error policy. 781 782 Every fire site β the main loop, the boundary's settle drain, the 783 after-loop pass β goes through here, so a node failure means the 784 same thing wherever it happens: it is put on the error bus (the 785 Report tab), recorded in ``node_errors`` for the end-of-run 786 conviction (``NodeErrorAggregate``), and the fire yields an 787 ``{"error": ...}`` result β which routes to no declared port, so 788 downstream starves rather than consuming garbage. Under 789 ``AGENTCANVAS_STRICT_ERRORS`` an exception is re-raised after the 790 bus emit instead, so smoke / eval runs fail fast. A node that 791 legitimately declares an ``error`` output port (env_libero tools, 792 navgpt navigate) is exempt: its ``error`` key is data. 793 ``StopExecution`` is not an error and passes through to the caller. 794 """ 795 from_exception = False 796 try: 797 result = await self._fire_node(node, session) 798 except StopExecution: 799 raise 800 except Exception as e: 801 self._report_node_error(node, session, exc=e) 802 if strict_errors_enabled(): 803 raise 804 result = {"error": str(e)} 805 from_exception = True 806 if isinstance(result, dict) and "error" in result and not self._declares_error_output(node): 807 if not from_exception: 808 self._report_node_error(node, session, message=str(result["error"])) 809 self.node_errors.append( 810 { 811 "node_id": node.id, 812 "node_type": node.type, 813 "step": self.step_counter, 814 "error": str(result["error"]), 815 } 816 ) 817 return result 818 def _report_node_error(β¦) ββ graph_executor.py:819β849 Β· β Β§4.2 819 def _report_node_error( 820 self, node: NodeInstance, session: Any, *, exc: Exception | None = None, message: str = "" 821 ) -> None: 822 """Put one node failure on the error bus: the exception itself 823 (``NODE_EXEC_FAIL``) or the message of a returned error result 824 (``NODE_RESULT_ERROR``).""" 825 scope = { 826 "node_id": node.id, 827 "node_type": node.type, 828 "origin": self._flatten_map.trace(node.id) if self._flatten_map else node.id, 829 "step": self.step_counter, 830 "execution_id": getattr(session, "_execution_id", None), 831 } 832 if exc is not None: 833 get_bus().from_exception( 834 exc, 835 source="node", 836 code="NODE_EXEC_FAIL", 837 scope=scope, 838 title=f"Node {node.id} ({node.type}) failed", 839 ) 840 return 841 get_bus().emit( 842 severity="error", 843 source="node", 844 code="NODE_RESULT_ERROR", 845 title=f"Node {node.id} ({node.type}) returned an error result", 846 message=message, 847 scope=scope, 848 ) 849 def _deliver(β¦) ββ graph_executor.py:850β894 Β· β Β§4.4 850 def _deliver( 851 self, src_id: str, payload: dict, queue: list[str], *, terminal: bool = False 852 ) -> None: 853 """Route one fired node's outputs along its outgoing edges and 854 enqueue the consumers that are ready β the one delivery rule. 855 856 For every edge out of ``src_id`` the value is 857 ``payload[sourceHandle]`` β the whole payload when the handle is 858 ``"default"``, nothing when the handle is absent (so an 859 error-shaped result routes nowhere). The write goes through 860 ``_route_value_to_port`` (LIST[T] coercion, the iterIn slot 861 write). A target that is ready afterwards is enqueued; 862 ``_enqueue`` dedupes and stamps the ready-time. 863 864 ``terminal`` marks the after-loop band (the final side, the 865 after-loop pass): a ``graphOut`` reached there is routed-to and 866 recorded β graph scope: the harvest snapshot; inner scope: the 867 latch β but never fired. In-loop a graphOut fires like any node. 868 """ 869 for edge in self.adjacency.get(src_id, []): 870 target = self.nodes.get(edge.get("target", "")) 871 if target is None: 872 continue 873 src_handle = edge.get("sourceHandle", "default") 874 tgt_handle = edge.get("targetHandle", src_handle) 875 if src_handle in payload: 876 value = payload[src_handle] 877 elif src_handle == "default": 878 value = payload 879 else: 880 value = _NOT_ROUTED 881 if value is not _NOT_ROUTED: 882 self._route_value_to_port(target, tgt_handle, value) 883 if terminal and target.type == "graphOut": 884 if value is _NOT_ROUTED: 885 continue 886 if self._scope_of(target.id) == GRAPH_SCOPE_ID: 887 if target.pending_inputs: 888 graphout_snapshot(target) 889 else: 890 graphout_latch(target, value) 891 continue 892 if self._is_ready(target): 893 self._enqueue(queue, target.id) 894 def _after_fire(β¦) ββ graph_executor.py:895β909 Β· β Β§4.4 895 def _after_fire(self, node: NodeInstance, result: Any, queue: list[str]) -> None: 896 """A body node has fired in-loop: consume its inputs, apply the 897 pivot cases, deliver. An iterIn clears its ``persist=False`` slots; 898 an inner-scope graphOut latches its value instead of delivering 899 (``_propagate_graphout_latches`` flushes the latch when the scope 900 terminates); everything else is delivered along its edges.""" 901 node.pending_inputs = {} 902 if node.carry is not None: 903 node.carry.after_fire() 904 if node.type == "graphOut" and self._scope_of(node.id) != GRAPH_SCOPE_ID: 905 if isinstance(result, dict): 906 graphout_latch(node, result.get("value")) 907 return 908 self._deliver(node.id, result if isinstance(result, dict) else {}, queue) 909 async def _iter_boundary(β¦) ββ graph_executor.py:910β933 Β· β Β§4.3 910 async def _iter_boundary( 911 self, 912 node: NodeInstance, 913 result: Any, 914 ready_queue: list[str], 915 session: Any, 916 step_delay_ms: int, 917 ) -> bool: 918 """The iterOut boundary β closes one iteration of its scope in four 919 phases: record β settle β decide β hand off. Returns ``True`` when 920 the root scope terminated (stop input, budget exhaust, or a 921 StopExecution during the settle drain) and the scheduling loop must 922 halt. An inner scope's termination flushes its graphOut latches and 923 lets the outer loop continue. 924 925 Edges FROM the iterOut are final-side only (``final_*`` handles): 926 they emit once at scope termination via ``_emit_final_side``, never 927 per iteration, so the iterOut's own result is never delivered here. 928 """ 929 # The scope is keyed by the paired iterIn's id (== scope_id). 930 io_scope_id = node.config.get("pairedWith", "") or self._outermost_scope_id 931 io_scope = self.scope_state.get(io_scope_id) 932 is_root = io_scope_id == self._outermost_scope_id 933 phase 1/4 Β· record β counters, checkpoint, log, nav_step ββ graph_executor.py:934β974 Β· β Β§4.3 β 935 # Advance the per-scope counter. For backward compat, 936 # self.step_counter mirrors the OUTERMOST scope's counter 937 # (single-scope graphs unchanged). 938 if io_scope is not None: 939 io_scope.step_counter += 1 940 scope_step = io_scope.step_counter 941 else: 942 scope_step = self.step_counter + 1 943 if is_root: 944 self.step_counter += 1 945 session._current_step = self.step_counter 946 947 log.info( 948 "Iter: scope=%s step=%d (parent=%s, root=%s)", 949 io_scope_id, 950 scope_step, 951 io_scope.parent_scope_id if io_scope is not None else None, 952 is_root, 953 ) 954 self.broadcast_signal("step_end", {"step": scope_step, "scope_id": io_scope_id}) 955 956 # Checkpoint all containers AFTER the iter boundary β the snapshot 957 # is the state step N+1 begins from. Root-scope iterations only 958 # (nested inner scopes don't checkpoint). 959 if self.containers and is_root: 960 self._checkpoints[self.step_counter] = { 961 cid: c.checkpoint() for cid, c in self.containers.items() 962 } 963 if len(self._checkpoints) > self.max_checkpoints: 964 del self._checkpoints[min(self._checkpoints)] 965 966 # Flush execution log entries to JSONL at the iteration boundary. 967 if self._logger: 968 self._logger.flush() 969 970 # One consolidated nav_step per root iteration, from every 971 # graphOut's harvest snapshot. 972 if is_root: 973 await self._broadcast_step(session) 974 phase 2/4 Β· settle β drain this iter's leftover sinks ββ graph_executor.py:975β978 Β· β Β§4.3 β‘ 976 if await self._settle_drain(io_scope_id, io_scope, is_root, ready_queue, session): 977 return True 978 phase 3/4 Β· decide β stop input, then budget check ββ graph_executor.py:979β1030 Β· β Β§4.3 β’ 980 # The stop signal is structurally bound to its scope (it sits on 981 # the scope's own iterOut) and is read exactly once per iteration, 982 # after the settle drain and before the handoff. Unwired stop = 983 # budget-only loop. Termination (stop or budget) emits the terminal 984 # iteration's values exactly once on the final_* handles. 985 if stop_requested(result): 986 log.info( 987 "Stop: iterOut %s stop=True at scope=%s step=%d", 988 node.id, 989 io_scope_id or "(graph)", 990 scope_step, 991 ) 992 if io_scope is not None: 993 io_scope.terminated = True 994 self._emit_final_side( 995 node, result, io_scope_id, ready_queue=None if is_root else ready_queue 996 ) 997 if is_root: 998 self.terminated = True 999 return True 1000 1001 scope_budget = io_scope.step_budget if io_scope is not None else self.step_budget 1002 if scope_step >= scope_budget: 1003 log.info( 1004 "Step budget (%d) exhausted for scope=%s", scope_budget, io_scope_id or "(graph)" 1005 ) 1006 if io_scope is not None: 1007 io_scope.terminated = True 1008 self._emit_final_side( 1009 node, 1010 result if isinstance(result, dict) else {}, 1011 io_scope_id, 1012 ready_queue=None if is_root else ready_queue, 1013 ) 1014 if is_root: 1015 self.terminated = True 1016 return True 1017 # Inner-scope budget exhaust: flush the inner graphOut latches 1018 # and let the outer loop continue; no handoff β the scope is done. 1019 self._propagate_graphout_latches(io_scope, ready_queue) 1020 return False 1021 1022 if step_delay_ms > 0 and is_root: 1023 await asyncio.sleep(step_delay_ms / 1000) 1024 1025 # An inner scope stopped above without halting the root loop: flush 1026 # its graphOut latches and skip the handoff. 1027 if io_scope is not None and io_scope.terminated: 1028 self._propagate_graphout_latches(io_scope, ready_queue) 1029 return False 1030 phase 4/4 Β· hand off β carry β paired iterIn, next iteration ββ graph_executor.py:1031β1042 Β· β Β§4.3 β£ 1032 # The fired bundle lands in the paired iterIn's ``iterout_<x>`` 1033 # slots (``stop`` is not loop-carried; final_* handles never fire 1034 # here) and the iterIn is queued for the next iteration. 1035 paired_id = node.config.get("pairedWith", "") 1036 paired = self.nodes.get(paired_id) if paired_id else None 1037 if paired is not None and paired.carry is not None: 1038 paired.carry.handoff(result if isinstance(result, dict) else {}) 1039 self._enqueue(ready_queue, paired.id) 1040 node.pending_inputs = {} 1041 return False 1042 async def _settle_drain(β¦) ββ graph_executor.py:1043β1102 Β· β Β§4.3 β‘ 1043 async def _settle_drain( 1044 self, 1045 io_scope_id: str, 1046 io_scope: _ScopeState | None, 1047 is_root: bool, 1048 ready_queue: list[str], 1049 session: Any, 1050 ) -> bool: 1051 """Boundary phase 2/4: fire the just-closed iteration's leftover 1052 sinks (viewer / telemetry nodes queued behind the iterOut) BEFORE 1053 the stop check, so they still emit on the terminal step. 1054 1055 Restricted to nodes of the iterOut's own scope β outer / peer-scope 1056 nodes are not drained by an inner boundary β with one exception: a 1057 ROOT boundary also drains graph-scope nodes. Dead-end sinks are 1058 never on a path to any iterOut, so scope analysis leaves them in 1059 the graph scope; without this they would miss the terminal step 1060 entirely (mid-loop it merely fires them earlier than the main loop 1061 would). iterIn is never drained β it advances to the next iteration 1062 and is fired only via the handoff after the stop check. 1063 1064 Bounded by ``max_settle`` fires. Returns ``True`` when a drained 1065 node raised ``StopExecution`` β the run halts exactly as it does 1066 for a StopExecution in the main loop. 1067 """ 1068 max_settle = 64 # safety cap against pathological queues 1069 settled = 0 1070 while ready_queue and settled < max_settle: 1071 next_idx = None 1072 for i, nid in enumerate(ready_queue): 1073 cand = self.nodes.get(nid) 1074 if cand is None or cand.type == "iterIn": 1075 continue 1076 cand_scope = self._scope_of(nid) 1077 if ( 1078 io_scope is not None 1079 and cand_scope != io_scope_id 1080 and not (is_root and cand_scope == GRAPH_SCOPE_ID) 1081 ): 1082 continue 1083 next_idx = i 1084 break 1085 if next_idx is None: 1086 return False # nothing to settle in this scope 1087 nid = ready_queue.pop(next_idx) 1088 cand = self.nodes[nid] 1089 try: 1090 r = await self._fire(cand, session) 1091 except StopExecution as e: 1092 log.info( 1093 "StopExecution from %s during settle: %s at step %d", 1094 nid, 1095 e.reason, 1096 self.step_counter, 1097 ) 1098 self.terminated = True 1099 return True 1100 self._after_fire(cand, r, ready_queue) 1101 settled += 1 1102 return False
7. Ways a run ends
| Condition | Where | Effect |
|---|---|---|
iterOut.stop truthy | Decide phase, iterOut boundary | scope ends + final side emits; root β break |
scope_step β₯ budget | Decide phase, after the stop check | scope ends + final side emits; root break, inner continue |
StopExecution | raised by node code (escape hatch) | terminated=True, break now; fallback emission covers the final side |
stop_event | main-loop gate | user Stop β break; fallback emission |
| queue empty | main-loop condition | graph exhausted (DAGs end here); fallback emission for loop graphs |
total_firings β₯ max | main-loop condition | backstop budget Γ nodes Γ 3; fallback emission |
8. Execution properties
| Concurrency | single-threaded async β one node at a time; forward() may await |
| Determinism | edge-declaration order + FIFO queue fix firing order |
| Cycles | IterIn/IterOut pairedWith transfer, not edges |
| Step counting | once per IterOut fire, per scope; root mirrors self.step_counter |
| State | per-node hidden + shared grant-gated StateContainers; a grant to a container homed in another process resolves to a RemoteContainerProxy (see State Containers Β§9) |
| Errors | one policy in _fire: capture as {"error":β¦} + bus event + node_errors ledger (conviction at run end); AGENTCANVAS_STRICT_ERRORS=1 (config.strict_errors_enabled) re-raises |
9. File reference
app/agent_loop/graph_executor.py | GraphExecutor β the scheduling loop (run) and its seams _fire / _report_node_error / _deliver / _after_fire / _iter_boundary / _settle_drain; NodeInstance (carry: LoopCarry on an iterIn), _ScopeState, _NodeStateProxy |
app/agent_loop/pivots.py | The pivots' runtime rules: LoopCarry (iterIn slots β write / handoff / ready / fire_inputs / after_fire / last_carried), iterout_port_names / stop_requested / final_payload, graphout_snapshot / graphout_last / graphout_latch / graphout_latched |
app/agent_loop/loop_runner.py | LoopRunner β lifecycle, policy loading |
app/agent_loop/builtin_nodes.py | CATALOG (the process-wide NodeCatalog) + built-in handlers |
app/components/catalog.py | NodeCatalog β registration, ports_of(type, config) (the one required-port / wire-type rule), schema |
app/agent_loop/graph_run.py | GraphRun / RunSession / RunOutcome β one executor per run, outcome harvest |
app/agent_loop/scope_analysis.py | analyze_scopes β ScopeForest |
app/agent_loop/flatten.py | flatten_graph + FlattenMap |
app/components/bases.py | FireList / FireSpec |
app/api/execution/run.py | POST /api/navigate/run (ADR-platform-002) |