ADR-executor-004
Dynamic Fire-List dispatch — runtime-emitted child firings inside static topology
Context
VoxPoser-class agents have a fundamentally dynamic call structure: a composer LLM emits a Python body that decides at run-time which sub-LMPs to call (parse_query_obj, 5 voxel-map LMPs, execute) and in what order. The static canvas topology — flatten_graph() expands all composites once before the run, then GraphExecutor precomputes adjacency / scope_forest / _termination_ids and treats them as immutable — cannot represent this: edges in the graph JSON are the only thing that can dispatch a node firing. Today the project copes by hosting the entire VoxPoser LMP runtime inside one black-box canvas node (v3's env_libero__voxposer_run_subtask); SR=1.0 is achievable but the LMP layer is invisible, unswappable, and unprofilable from canvas. Capture-then-replay in static topology (record the call sequence in a dry run, then replay through static gates) requires double execution and first-class conditional-fire semantics the engine doesn't have. Architecture-search (AAS) and the myloop performance loop both want per-LMP introspection so they can ablate / swap / mock individual LMP calls across iters — a need that a black-box node cannot serve. The framework needs a primitive that lets a single node's forward() spawn a runtime sequence of node firings while keeping the editor's static-topology assumption intact.
Decision
(1) Three new component types in app/components/bases.py: FireSpec (dataclass — node_type, inputs, config, label, capture_outputs), FireList (dataclass sentinel — specs: list[FireSpec], spawner_outputs: dict, aggregator: dict), and DynamicFireListNode (BaseCanvasNode subclass with abstract forward() → FireList and concrete-with-clear-error aggregate(child_results) → dict). (2) Engine intercept in _fire_node: when a node's forward() returns a FireList, the executor calls _fire_dynamic_children(spawner, fire_list, session), then applies the FireList's declarative aggregator OR the spawner's aggregate() method, merges with spawner_outputs, and replaces result for the spawner's normal log + edge-propagation path. (3) Ephemeral children: each FireSpec becomes a NodeInstance with id {spawner.id}::dyn{i} that is NOT in self.nodes / adjacency / scope_forest / ready_queue. Children are fired sequentially via the existing _fire_node path (so they get the full hook / log / ctx machinery); their outputs are collected explicitly, never propagated through the static graph. (4) Declarative aggregators: FireList.aggregator recipes (passthrough_last, passthrough_index, merge_all, rename) collapse children's outputs framework-side without needing the spawner's Python aggregate() method — required for server-mode spawners whose framework-side proxy is a plain BaseCanvasNode subclass that doesn't inherit the method. (5) Hierarchical logging: ephemeral children carry _dynamic_parent_id and _dynamic_index attributes which _fire_node reads and plumbs into ExecutionLogger.log_node(); NodeLogEntry grows two optional fields (parent_node_id, dynamic_index, default None, backward-compatible). (6) Access-grant inheritance: _fire_node's access-grant lookup uses _dynamic_parent_id or node.id as the lookup key so children inherit the spawner's container access without per-child grants (essential when the spawner shares a runtime container with its children, as VoxPoser v4 does). (7) HTTP boundary: server-side /call/{fn} route detects isinstance(result, FireList) and serializes to a {"__firelist__": {specs, spawner_outputs, aggregator}} envelope; the framework-side proxy detects __firelist__ in the response and reconstructs FireList/FireSpec objects. (8) Hard constraints (engine rejects on attempt): forbidden child types — iterIn, iterOut, initialize, termination, graphIn, graphOut — would either no-op outside static topology or corrupt scope / termination state. Nested FireList (a child itself returning FireList) is rejected as NotImplementedError — that is full subgraph territory (deferred). Sequential semantics: child raise stops the sequence; parallel_group extension deferred.
Alternatives
(a) Full subgraph spawn (option "C") — spawner returns a SubgraphSpawn(GraphSpec) with nodes + edges + nested scope pivots; engine instantiates a sub-executor with proper scope nesting, hierarchical termination, cross-process coordination, and an editor trace viewer that lets the author expand a spawner to see the actually-instantiated subgraph. Rejected for current scope: estimated 8–11 weeks of engine + editor work for capabilities (recursive HTN, MCTS-style tree search, ReAct retry loops, multi-agent debate) the current paper roadmap doesn't need. The C.5 fire-list shape is a strict subset; if a future agent needs subgraph wires the upgrade path is clean (FireList becomes a special case of GraphSpec). (b) Capture-then-replay in static topology (option "B") — first pass records the dynamic call sequence with all sub-LMPs stubbed, second pass dispatches the captured calls through static topology with conditional-fire gates. Rejected: requires double execution and a new "fire only if input matches" semantics that doesn't exist as a first-class engine feature; the conditional-gate fan-out also bloats the editor graph with 7+ dispatcher nodes per dynamic node, defeating the introspection win. (c) Black-box hosting (option "A", v3) — keep the entire dynamic dispatch inside one canvas node, log internals via _self_log(). Already in production as VoxPoser v3 (kept on disk as design reference); the new primitive coexists with it. Black-box is the right answer when there's no need for per-step introspection; once AAS / myloop need to ablate single LMPs, the black box becomes the bottleneck. (d) Two-stage LLM → graph compile (option "D") — LLM emits a complete graph JSON; canvas treats it as a new run. Rejected: hard to retrofit VoxPoser's existing composer prompt format (which writes Python with sub-LMP calls, not declarative JSON); also loses the recursive LLM reasoning structure where each layer's prompt depends on the previous layer's output.
Rationale
The framework gains one new branch in _fire_node + one new private helper (_fire_dynamic_children, ~90 LOC) + four declarative aggregator recipes; no new firing modes, no new scope, no new executor instance, no editor changes. ~600 LOC engine + 15 unit tests vs ~4000 LOC for full subgraphs. Editor's static-topology assumption is preserved — children are runtime-only firings that never appear on the canvas, just as log entries with parent_node_id provenance. Server-mode boundary works cleanly via the JSON envelope; declarative aggregator decouples the spawner's Python class from the framework-side proxy so dispatch crosses subprocess boundaries without a second HTTP round-trip. VoxPoser v4 dogfood validates the abstraction is faithful: SR=1.0 / 1177 steps / 43.1s on LIBERO task 0 — exact parity with v3's black-box baseline (42.9s), 11 lmp_call children correctly captured across 4 subtasks with proxy refs resolving through a per-spawn runtime scratch dict. Headless deployment paths (AAS, myloop) consume log.jsonl directly via node_type=env_libero__voxposer_lmp_call filter + parent_node_id grouping — no canvas UI needed. Hard rejection of recursion + control-type children keeps the engine's existing scope / termination / settle machinery untouched (no nested executor lifetime, no cross-scope error propagation, no editor trace viewer required).
Affected docs
core/glossary.html (new terms: FireSpec, FireList, DynamicFireListNode, dynamic fire-list dispatch, declarative aggregator, ephemeral child, parent_node_id, dynamic_index); core/codebase-map.html (new files: app/agent_loop/test_dynamic_firelist.py; new methods: _fire_dynamic_children, _apply_declarative_aggregator; new classes: FireSpec, FireList, DynamicFireListNode; new constants: _DYNAMIC_FIRELIST_FORBIDDEN_CHILD_TYPES); design-docs/graph/graph-executor.html (new "Dynamic Fire-List dispatch" section); design-docs/components/base-canvas-node.html (new DynamicFireListNode subclass section); core/decisions/executor/index.html + core/decisions/index.html (this ADR added)