Graph System

Every agent pipeline in AgentCanvas is one JSON document. This page follows that document through its whole life — what its schema is (§2), which node types it may reference (§3), what makes it legal (§4), how it nests (§5), where it lives on disk and in the UI (§6) — then reads one real artifact end to end (§7) and walks it to the API boundary where it is handed to the engine (§8).

1. The artifact

Because it is only data, the same document moves freely between every surface of the system: saved to disk under workspace/graphs/ or workspace/graph_nodes/, sent over HTTP (POST /api/navigate/run, POST /api/graphs), embedded inside another graph via NodeDef.subgraph (§5), authored on the canvas, loaded from a file, or generated by code — including by the AAS architect loop, which edits graphs purely through the API.

The division of labour with the engine is the blueprint/factory split. The graph names what exists; only at fire time does the executor turn a type string into a live object:

# GraphExecutor._fire_node() — the only place type strings become objects
handler_cls = NODE_HANDLERS.get(node.type)   # registry lookup — None + warn if unknown
instance = handler_cls()                 # fresh live instance
instance.config = node.config            # per-node knobs from the JSON
result = await instance.forward(inputs, ctx)

This separation is what makes graphs portable: the same JSON runs on any backend that registers the same node types, and nothing about a saved graph can hold a stale object alive.

2. The schema

All model types live in agentcanvas/backend/app/graph_def.py as plain recursive @dataclasses, mirrored one-to-one by the frontend's types.ts. Two fragments carry almost all of the structure — a node reference and a wire:

{ "id": "predict", "type": "policy_adapter_vlnce__predict",
  "label": "Predict", "position": { "x": 420, "y": 96 },
  "config": { "model_adapter": "cma" } }

{ "id": "e7", "source": "predict", "sourceHandle": "model_output",
  "target": "adapt_model_to_canonical", "targetHandle": "model_output" }

2.1 GraphDefinition, NodeDef, EdgeDef

GraphDefinition is the universal container — the same dataclass serves the root canvas graph, an agent-loop inner graph, a composite's subgraph, and a saved template:

NodeDef and EdgeDef are deliberately thin:

(label and position on NodeDef are canvas presentation only.)

2.2 State containers and access grants

Shared state is part of the artifact, not a runtime side channel. A ContainerDef is a visible canvas element holding named StateDef entries; an AccessGrantDef authorizes one node to read/write one container. Grants render as dashed violet lines but are not wires: they carry no data and never make a node ready.

A container with the well-known id "graph_state" plays the role of the graph-level blackboard — but since ADR-dataflow-004 there is no auto-inject: every node that wants it needs its own grant.

2.3 Hooks

A HookDef attaches a shell command to a lifecycle event: event (PreNodeExecute / PostNodeExecute / GraphStart / GraphComplete / GraphError), command (stdin receives a JSON payload), match_node_type (*, exact, or prefix-glob), optional match_node_id, timeout_ms (default 1000), enabled. One caveat lives in §5: hooks on inner composites do not survive flattening.

3. The node-type vocabulary

NodeDef.type draws from two namespaces, distinguished by a double-underscore separator (ADR-components-003).

NodeSet nodes are ${nodeset}__${node} — the prefix is what lets the frontend auto-load the owning nodeset when a graph is opened (§6.4):

Built-in nodes have no separator. The 14 registered in NODE_HANDLERS at import time (agent_loop/builtin_nodes.py) — roles only; the pivots' execution semantics live in Graph Executor §1.2:

compositeNode is deliberately absent: it is a frontend-only shell carrying a subgraph, expanded by flatten_graph() before the executor ever sees it. The former initialize, termination, and outputPort types are removed — graphs that still carry the first two are rejected with migration hints (§4.2).

4. What makes a graph legal

This section is the authority on legality. §4.1 explains the one structure with non-trivial data shape — the pivot pair; §4.2 lists every rule the validator enforces; §4.3 covers the conventions that matter but are not machine-checked.

4.1 The pivot pair, data side

Loops are expressed by a paired iterIn/iterOut (ADR-dataflow-008 + ADR-dataflow-009). The authoring rule: run-start values (initial observation, instruction, episode config) go into iterIn's init side; values that change per iteration ride the loop-carry through iterOut. In the JSON, each pivot is two-sided:

• iterIn — its input side is declared under config.initPorts; run-start values arrive as ordinary edges targeting the prefixed init_<name> handles. Its output side exposes init_<name> + iterout_<name> handles to the loop body.
• iterOut — its input side takes the loop-carry values under config.ports, plus an optional stop BOOL halt input. Its output side mirrors the carry ports as final_<name> handles plus a constant final_stop — these feed the after-loop verdict stage. The pivots are linked by config.pairedWith; there is no canvas wire between them.

iterIn's effective config.ports is synthesised at graph load by _synthesize_iterin_ports from two writers — its own initPorts (handles prefixed init_, default persist=false) and the paired iterOut's config.ports (handles prefixed iterout_, default persist=true) — and is never hand-written; the legacy init_ports/loop_ports keys are rejected at load. Direct canvas edges into iterIn targeting an undeclared handle are also accepted as init-writer surrogates (handle name used as-is).

What these declarations mean at runtime — which value a consumer sees on which iteration, what persist does to a slot, when the final side delivers — is the execution contract, stated in Graph Executor §1.2 with the boundary mechanics in §4.3. The one trap worth flagging at authoring time: a name may appear on both sides (init_X for iter 0, iterout_X for iter ≥1) only if the init entry stays one-shot — §7.2 shows the canonical dual-wire example.

4.2 Load-time validation — the rule set

All rules are enforced by validate_graph_connectivity() (graph_def.py) at the API boundary (POST /api/navigate/run, POST /api/eval/v2/start); violations are batched into one HTTP 400 listing every error. Port requirements are resolved per instance — for each builtin the validator calls cls._resolve_ports(node.config) when present (so config.ports overrides are honoured), falling back to the class-level input_ports; nodeset types are skipped, since their handlers aren't registered at validation time.

The after-loop rules are structural guarantees, not style: they make the “verdict lags the terminal step” bug unwritable rather than merely discouraged — the why is in ADR-dataflow-009, the runtime counterpart in Graph Executor §5.1.

4.3 Conventions the validator does not enforce

LLM profiles are user-level config, never graph-level. Profile-bearing nodes (currently llmCall) must save with config.profile = "", which resolves to the user's active profile at run time. A profile binds a name to (provider, model, api_key, base_url) and lives in /api/profiles/ — per-deployment state, not portable topology. Hardcoding a name breaks portability (the graph fails for anyone without that exact profile; get_llm_config("missing") short-circuits to an empty response) and invites silent miscredit — a named profile can point at an exhausted or wrong-org key whose quota error surfaces only in the backend log. If a graph genuinely needs different LLMs in different roles, document it in a note node beside the llmCall and let the user pick per node in the UI.

5. Composition

A NodeDef may carry subgraph: GraphDefinition — itself a full graph, recursively. The executor never sees nesting: at execution time flatten_graph() (app/agent_loop/flatten.py) expands every composite into the parent namespace.

Concretely, flatten_graph() performs five renames and rewires:

1. Inner node ids are prefixed ${composite_id}__${inner_id}.
2. Inner container ids are prefixed the same way.
3. Inner AccessGrantDefs get node_id remapped through the same id map; container_id is prefixed.
4. graphIn/graphOut boundary nodes are rewired — parent edges connect to the prefixed inner source/target (composites without an iterIn erase the boundary nodes entirely; loop-bearing composites keep graphOut as a latch).
5. The composite shell node is removed.

A FlattenMap records flat-id → origin-composite-path so execution errors can be traced back to the drawing the author actually made.

6. Storage and the two kinds

6.1 Graph vs Graph Node

One schema, two UX surfaces, routed by the kind field (ADR-canvas-003):

Both kinds appear in the Explorer, expand into their internal node tree (recursively for nested composites), and are served by the same API (§9.1) — kind alone decides the root directory and the UX affordances.

6.2 On disk

workspace/
├── graphs/                          ← editable templates (kind="graph")
│   ├── demo/                        ← subfolders are free-form; the API scans
│   │   └── tutorial_demo.json          recursively. Current convention:
│   ├── examples/                       <domain>/<verified|unverified>/
│   ├── vln/
│   │   ├── verified/
│   │   │   └── straightforward.json
│   │   └── unverified/
│   │       ├── navgpt_ce.json
│   │       └── mapgpt_mp3d.json  …
│   ├── eqa/
│   │   ├── verified/                ← openeqa_em_*.json
│   │   └── unverified/              ← explore_eqa_hmeqa.json …
│   └── vla/
│       └── unverified/              ← voxposer_libero_decomposed.json …
└── graph_nodes/                     ← archived composites (kind="node")
    └── voxposer/  …

A graph's id is its root-relative path. Folders are plain directories — list/create them via GET/POST /api/graphs/folders, and POST /api/graphs accepts an optional folder to save into one.

6.3 Instantiation and snapshot semantics

Dragging a graph node onto a canvas runs through four stages: the drag payload carries the entire GraphDefinition; the drop creates a React Flow node of type compositeNode with data.subgraph set to a full deep copy; CompositeCanvasNode renders its handles from the graphIn/graphOut nodes inside; and at run time getGraphForExecution() serializes the composite with .subgraph intact for the backend to flatten (§5).

This is copy, not reference — deliberately. A graph node is an archive: editing the source file later does not reach into canvases or parent graphs that already embedded it.

6.4 Opening a graph: nodeset auto-load

Opening a saved graph must produce a working canvas even on a fresh backend. nodesetLoader.ts parses the node types, extracts nodeset names from the __ prefixes (env_habitat, policy_adapter_vlnce, …), diffs them against GET /api/components/nodesets, auto-loads the missing ones, then fetches fresh schemas (GET /api/components/node-schemas) and injects _schema into each node's data — which is what lets the renderer derive ports and ui_config without hardcoding any node type.

7. A complete artifact: Straightforward (VLN-CE)

The simplest complete agent graph in the repo — a neural-policy loop driven by the 5-stage General Policy Adapter pipeline (env_adapter + policy_adapter_vlnce), with the policy's hidden state riding the loop-carry (ADR-eval-002). Full JSON at workspace/graphs/vln/verified/straightforward.json. It exercises every structure this page defined: init edges, the pivot pair, the stop halt input, and an after-loop band feeding the eval latch.

7.1 Node breakdown

7.2 The iterIn config, and the dual-wire pattern

{
  "id": "iterIn_e5f6a7b8",
  "type": "iterIn",
  "config": {
    "version": 3,
    "pairedWith": "iterOut_c9d0e1f2",
    "initPorts": [
      { "name": "rgb",     "wire_type": "IMAGE", "persist": false },
      { "name": "depth",   "wire_type": "DEPTH", "persist": false },
      { "name": "raw_obs", "wire_type": "TEXT",  "persist": false },
      { "name": "hidden",  "wire_type": "ANY",   "persist": false }
    ]
  }
}

The init edges target the prefixed handles (env_habitat__observe.rgb → iterIn.init_rgb, …). The iterout_* half of the handle set comes from the paired iterOut's config.ports (rgb, depth, pose, action, raw_obs, hidden) — synthesised at load, never hand-written (§4.1). All four initPorts are one-shot (persist: false) because the same names are loop-carried from iter 1 onward.

The graph also carries the canonical dual-wire pattern: both init_raw_obs and iterout_raw_obs are wired into adapt_env_to_canonical.raw_obs — legal exactly because the init entry is one-shot, so the two writers never compete after iter 0.

7.3 How it runs

This page stops at the artifact. The run itself — how observe is discovered as an entry, how an iteration turns, how terminated → stop ends the loop, and how the final_stop edge delivers exactly once into evaluate — is the executor's story: Graph Executor §4.6 walks a loop of exactly this shape turn by turn, ready-queue and all.

8. The handoff

8.1 Run request

POST /api/navigate/run
{
  "loop_definition": { "...": "GraphDefinition JSON" },
  "execution_id": "exec_abc123",
  "step_delay_ms": 200
}

This endpoint is where the artifact leaves this page's jurisdiction. app/api/execution/run.py parses it (GraphDefinition.from_dict()), judges it (validate_graph_connectivity() — the full §4.2 rule set, HTTP 400 on violation), and hands the validated graph to LoopRunner → GraphExecutor.run(), which flattens it (flatten_graph(), §5) during build, before any node fires. Everything from that call onward — flattening, scheduling, guards, phases, how the run ends — is Graph Executor §2's territory.

Sibling lifecycle endpoints on the same prefix: POST /api/navigate/run/pause, /run/stop, GET /api/navigate/run/status, GET /api/navigate/run/checkpoints, POST /api/navigate/run/restore/{step}.

8.2 WebSocket events

What comes back during a run is also a data contract this page owns. The executor emits per-node events — there is no single consolidated nav_step dump; each viewer sink serializes its own wired inputs and broadcasts on its own node id (ADR-observability-003 / ADR-components-007 / ADR-components-008):

Every event carries execution_id for multi-run routing on the frontend.

9. API and serialization

9.1 Storage endpoints

All graph storage lives behind /api/graphs (app/api/canvas/graphs.py); every payload crosses the boundary through GraphDefinition.from_dict() / to_dict(), so malformed JSON fails fast:

9.2 Serialization in code

import json
from app.graph_def import GraphDefinition

with open("workspace/graphs/vln/verified/straightforward.json") as f:
    gd = GraphDefinition.from_dict(json.load(f))

gd.name                # "Straightforward (VLN-CE)"
len(gd.nodes)          # 15
gd.nodes[0].type       # "env_habitat__observe_egocentric"
gd.kind                # "graph"

with open("workspace/graphs/copy.json", "w") as f:
    json.dump(gd.to_dict(), f, indent=2)

to_dict() drops empty lists and defaults for readability — kind="graph" and empty containers/access_grants/hooks simply don't appear in saved files. The models are plain dataclasses rather than pydantic by choice: the recursive self-reference (NodeDef.subgraph contains NodeDef) works cleanly with from __future__ import annotations, the explicit from_dict/to_dict pair gives full control over backward-compat aliases (the old graph_state field, the state_edges → access_grants rename), and the dependency count stays at zero.

10. File reference