AgentCanvas / Pages / Developer Guide / Capabilities / Visual Canvas Editor
2026-07-14

Every other capability page in this section describes a backend mechanism. This one is about the surface researchers actually touch: the in-browser canvas where graphs are authored, configured, run, and drilled into β€” one React app, one multiplexed WebSocket, no separate "design" and "monitor" tools. Load a graph from the explorer tree, drop nodes from the live catalog, wire them with type-checked connections, press Play, and watch every port light up β€” then open a node's Python source in the bottom drawer and hot-patch it without leaving the browser.

one canvas β€” author Β· configure Β· run & watch Β· drill into composites Browser β€” React app tabs Β· run toolbar Β· env panel Explorer graphs tree node catalog typed wires Β· status dots Properties auto-rendered config form drawer: State Β· Logs Β· Source (edit node code) Β· Report FastAPI backend component registry GraphExecutor ErrorBus graph store + layout HTTP /api/* load Β· save run Β· layout WS /ws nav_status Β· nav_step viewer_data Β· error_event React Flow owns geometry Β· Zustand owns graph semantics Β· WSManager fans one socket out to every panel drilling into a composite swaps the canvas for its inner graph β€” a breadcrumb bar (canvasStack) leads back out

Design docs: Canvas System Β· Graph System Β· Execution Logs


1. Problem

An agent-graph authoring tool needs four things that don't combine easily:

If any of these is split across a separate page, the round-trip cost dominates. The unified editor folds all four into one ReactFlow canvas backed by a single WebSocket stream.

2. Three-Layer Architecture

The canvas editor: explorer tree on the left, Text Analysis Demo graph mid-canvas with live viewer output, auto-rendered properties panel on the right
The whole surface in one window: explorer (graphs folder tree + node catalog) on the left, tabs and the run toolbar on top, the demo graph just after a run β€” viewers holding their last frames β€” and the selected node's auto-rendered config form on the right. Top-right of the canvas: Auto Layout and the Curved / Orthogonal edge-routing toggle.

State-changing actions (load nodeset, run graph, cancel run) are HTTP because they need a synchronous result. Live execution is WS because it's a high-volume one-way fan-out.

3. Graph Editor

UnifiedGraphEditor is the same component at the root canvas and inside a composite's inner graph:

ModeState sourceSave semantics
rootReads/writes the active tab's useFlowStore slice directlyIDE-style Ctrl+S: if the tab is bound to a saved graph, PUT in place with no prompts; otherwise (new tab from a template) open the Save-As dialog
subgraphLocal React state seeded from the composite's subgraphExplicit Save & Back / Reset; onSave writes back to the parent composite node's data.subgraph

Both modes share the drag-drop pipeline, typed-wire compatibility check (isCompatibleWireConnection), and the same proxiedNodeTypes registry, so a composite editor looks and behaves identically to the root β€” only the persistence model differs.

3.1 Typed Wire Compatibility

Connections go through useFlowStore.onConnect β†’ isCompatibleWireConnection. The check pulls each port's wire_type from portResolution (which walks _schema + live config.ports overrides for instance-derived ports such as iterIn, stateContainer, imageViewer) and rejects mismatches before the edge enters the store. This means the canvas can never reach the backend in a state that would only be caught by validate_graph_connectivity.

3.2 Explorer: Graphs Tree and Node Catalog

The left rail is ExplorerPanel, a VS Code-style sidebar with two sections:

Because a dropped node's initial state is the schema's default_config plus the schema itself (_schema), a freshly dropped node validates and renders its ports immediately, before any backend round-trip.

3.3 Auto-Layout and Edge Routing

The Auto Layout button sends the current graph to POST /api/graphs/layout together with each node's real rendered size (React Flow's measured width/height) β€” so wide nodes don't overlap the next fixed-pitch column. The backend returns new positions plus per-edge orthogonal waypoints; a Curved / Orthogonal toggle switches between default bezier wires and RoutedEdge, which draws the orthogonal path through the reserved channels.

4. Configuration Panel

Selecting a node opens PropertiesPanel, which renders fields straight from the node's ui_config.config_fields (or, for non-customised nodes, infers them from configSchema). One renderer (ConfigFieldRenderer) covers the widgets:

LLM-backed nodes get an extra block from LlmModelControls. ModelRefPicker is one dropdown with three reference modes: Default (follow the active profile), a named profile, or Browse… (a provider β†’ model cascade pinned inline on the node as {provider, model}). The panel then fetches GET /api/providers/{id}/capabilities for the resolved pair and imposes the parameter rulebook on the controls at edit time β€” locked parameters show the value and the reason, ranges clamp the slider, unsupported parameters gray out. The same rulebook is applied again at call time; the panel is just the "before" render of it.

There is no per-node React form. Adding a new node type means adding its ui_config and config_schema on the Python side; the UI is automatic.

5. Live Execution View

While the executor runs, the editor consumes four WS frame types from state.broadcast:

FrameEmitterFrontend handler
nav_statusGraphExecutor + LoopRunnerUpdates the run-state banner (idle / running / paused / error) and toolbar
nav_stepGraphExecutor._broadcast_stepConsolidated per-step payload from graphOut sinks β€” feeds the agent node's nodeOutputs entry (RGB / depth / action / position / containers / metrics)
viewer_datatextViewer / textScroll / imageViewerPer-node payload ({node_id, step, fields}) β€” routed by node_id straight into that node's nodeOutputs.fields slice
error_eventErrorBus (ADR-observability-004)Pushed to errorStore + Report panel + per-node error badge on the canvas

Each node's shell (NodeShell + HandleDot) renders a per-handle status dot driven by nodeOutputs[node_id]; receiving a value flips the handle to "filled", emitting flips the output handle to "fresh".

6. Composite Drill-Down

A composite node renders as a tile showing its name and inner node count, with two affordances: Preview expands an inline list of the inner node types, and Edit drills in. Drilling is not a modal β€” the canvas area swaps to the inner graph (UnifiedGraphEditor mode="subgraph", seeded from the composite's subgraph), and a breadcrumb bar appears above it: ← Root β€Ί {composite name}. Multi-level dives push frames onto the store's canvasStack; every crumb is clickable, so you can jump straight back to any depth. Edits live in local React state until Save & Back writes them into the parent composite node's data (Reset discards); the parent tab's nodes, positions, and selection are untouched in the store throughout.

Drill-down view: breadcrumb bar 'Root β€Ί Text Analysis Composite', the 7-node inner graph on the canvas, Reset and Save & Back buttons
Inside a composite: the demo graph saved as an archived node, dragged onto a canvas, drilled into via its Edit button. The breadcrumb bar (← Root β€Ί Text Analysis Composite β€” 7 nodes, 7 edges) leads back out; Save & Back / Reset control whether the inner edits are written back to the parent node.

7. Panels

The canvas is wrapped by a tab-aware shell: a tab bar for multiple open graphs, one resizable left rail, one resizable right rail, and one resizable bottom drawer. Every panel is a thin Zustand consumer:

PanelWhereRole
TabBartopOne tab per open graph, each an independent store slice; dirty dot until saved
ExecutionToolbartopRun / pause / step / stop / restore-checkpoint Β· step-delay slider Β· step budget (Max) Β· State / Viewer / Annotation toggles
EnvPaneltopEnv-panel methods exposed by loaded env nodesets (split / episode / reset)
ExplorerPanelleftGraphs folder tree + node catalog (Β§3.2)
GraphStateBannerabove canvasGraph-level state containers with live value previews; add/remove named states
PropertiesPanelrightSelected-node config form (auto-rendered from ui_config, Β§4) + per-node hook editor
OutputDrawerbottomFour fixed tabs β€” State (live state-container values), Logs (per-node inner_log timeline, ADR-observability-003), Source (Β§8), Report (ErrorBus envelopes with an unread badge; clicking one jumps to the offending node)
TemplatePickermodalStarter graphs from TEMPLATES (defaultGraph.ts)
SaveGraphDialogmodalSave the canvas as a named graph β€” or, via Save as Node, as an archived composite under workspace/graph_nodes/ that later drags onto any canvas (Β§6)

Keyboard shortcuts follow IDE conventions β€” Ctrl+S saves in place (Β§3), ? opens the shortcut cheatsheet overlay.

8. Editing Node Source In Place

The bottom drawer's Source tab turns the canvas into a light IDE for nodeset code. Select a nodeset node and SourcePanel shows only that node's slice of its nodeset file β€” the module-level globals, the functions its class transitively references, and the class itself β€” one stacked CodeMirror editor per segment, each tagged with its kind and line range (CLASS SentimentTag Β· L105–151). Save splices the edited segments back into the file by line range; the backend syntax-checks the whole file before writing anything, and the nodeset watcher then hot-reloads the component β€” the save button walks "Saved" β†’ "Reloaded βœ“" as the components_changed broadcast comes back. Edit the forward() of a node, press Play again, and the new behaviour runs β€” no terminal round-trip.

Source tab: the Sentiment Tagger class shown as a scoped CodeMirror segment (CLASS SentimentTag L105-151) inside the bottom drawer, canvas still visible above
The Source tab scoped to the selected Sentiment Tagger node: its nodeset file (example.py, LOCAL badge) is reduced to the one class that defines the node, ports and forward() included. Saving splices this segment back by line range, syntax-checks the full file, and hot-reloads the nodeset.

9. Run Flow

Pressing Run in the toolbar fires runPipeline(graph):

  1. Convert the live ReactFlow nodes/edges back to a GraphDefinition via fromFlowNodes + fromFlowEdges.
  2. Client-side pre-flight: for eval_graph: true (default), require β‰₯1 graphOut node. Surfaces a clearer error than waiting for the backend's validate_graph_connectivity.
  3. Build the LoopDefinition wire payload (nodes, edges, containers, access_grants, step_budget, terminationCondition, presetId, hooks).
  4. POST to /api/navigate/run (ADR-platform-002 β€” single endpoint) with an optional execution_id and step_delay_ms.
  5. The backend acknowledges with { execution_id } and starts emitting nav_status / nav_step / viewer_data / error_event frames over the already-open WS β€” no new connection.

Cancel goes through POST /api/navigate/run/stop; the toolbar disables itself based on nav_status.status.

The visual editor is one of several React surfaces backed by the same store and WS:

11. Key Files

FileRole
agentcanvas/frontend/src/canvas/UnifiedGraphEditor.tsxRoot + subgraph editor; ReactFlow host, drag-drop, save/reset
agentcanvas/frontend/src/canvas/useFlowStore.tsZustand store: tab-aware nodes/edges/nodeOutputs/canvasStack; integrates ReactFlow change handlers
agentcanvas/frontend/src/canvas/unifiedCatalog.tsCatalog merge: built-ins + loaded nodesets + composite graph_nodes
agentcanvas/frontend/src/canvas/unifiedNodeTypes.tsReactFlow nodeTypes registry β€” wraps every node type with the shared shell
agentcanvas/frontend/src/canvas/nodes/shared/NodeShell.tsxUniversal node frame: handle rows, status dots, error badge, composite-drill button
agentcanvas/frontend/src/canvas/nodes/shared/HandleDot.tsxPer-handle status dot driven by nodeOutputs
agentcanvas/frontend/src/canvas/nodes/composite/CompositeNodeView.tsxComposite tile + drill button β†’ subgraph modal
agentcanvas/frontend/src/canvas/panels/PropertiesPanel.tsxAuto-rendered config form from ui_config.config_fields
agentcanvas/frontend/src/canvas/panels/ExplorerPanel.tsxLeft rail: graphs folder tree (open-in-tab / drag composite) + node catalog
agentcanvas/frontend/src/canvas/panels/OutputDrawer.tsxBottom drawer host β€” State / Logs / Source / Report tabs
agentcanvas/frontend/src/canvas/panels/SourcePanel.tsxScoped per-node source editing with splice-back save + hot reload
agentcanvas/frontend/src/canvas/panels/TabBar.tsxMulti-graph tabs over per-tab store slices
agentcanvas/frontend/src/canvas/panels/GraphStateBanner.tsxGraph-level state bar above the canvas with live previews
agentcanvas/frontend/src/canvas/panels/LlmModelControls.tsxModelRefPicker + capability-rulebook rendering for LLM nodes
agentcanvas/frontend/src/canvas/panels/ExecutionToolbar.tsxRun/pause/cancel/step-delay controls; observes nav_status
agentcanvas/frontend/src/canvas/runPipeline.tsEditor β†’ LoopDefinition β†’ POST /api/navigate/run
agentcanvas/frontend/src/canvas/graphConversion.tsGraphDefinition ↔ ReactFlow Node[]/Edge[] (incl. containers, access grants)
agentcanvas/frontend/src/canvas/portResolution.tsResolve instance ports (iterIn / stateContainer / imageViewer) from live config
agentcanvas/frontend/src/ws.tsWSManager singleton β€” auto-reconnect, typed handler map, wildcard channel
agentcanvas/frontend/src/store.tsWS subscriptions: routes nav_step / viewer_data / nav_status / error_event into the appropriate store slices

Status

ItemStatusNotes
Unified root/subgraph editorDoneUnifiedGraphEditor handles both modes with one component
Typed-wire client-side validationDoneisCompatibleWireConnection in useFlowStore.onConnect
Auto-rendered properties panelDoneConfigFieldRenderer covers text / textarea / select / slider / checkbox / port_list / llmRef
Catalog = backend registryDoneunifiedCatalog built from GET /api/components/node-schemas; re-fetched on nodeset load/unload
Explorer graphs folder treeDoneBoth kind roots; open-in-tab, drag-composite, folder CRUD
In-place node source editingDoneSource tab: scoped segments, splice-back save, syntax check, hot reload
Auto-layout + orthogonal routingDonePOST /api/graphs/layout with measured node sizes; Curved/Orthogonal toggle
Single multiplexed WebSocketDoneOne wsManager with typed channels + wildcard
Per-handle live statusDoneNodeShell + HandleDot driven by nodeOutputs
Consolidated nav_step from graphOut sinksDoneRecognised port names (rgb/depth/action/state/done/metrics/response) fed into agent node slice
Per-node viewer_data routingDoneSink nodes emit their own viewer_data frames during forward()
Tab-aware canvas slicesDoneEach tab owns nodes/edges/nodeOutputs/canvasStack
Composite drill-down with save/resetDoneCanvas swaps to the inner graph with a clickable breadcrumb trail (canvasStack); parent tab state preserved in the store
Pre-flight: eval_graph requires graphOutDoneCaught client-side before /api/navigate/run round-trip
Step-delay slider for human-readable runsDonestep_delay_ms passed on every /api/navigate/run call
Restore-checkpoint UIDoneToolbar reads get_checkpoints(); per-scope (ADR-executor-003)
Error-bus badges on offending nodeDoneReport panel click jumps to node on canvas
Mobile / touch UXPlannedReactFlow supports it but no UI tuning done; right panels do not collapse on narrow viewports
Multi-selection composite extractionPlanned"Select N nodes β†’ wrap into composite" β€” group-selection scaffolding exists in groupSelection.ts; UI command not wired
Legacy panels in the treeStale codeNodeLibrary, ObservationPanel, StepLogPanel are no longer mounted anywhere β€” superseded by ExplorerPanel and the drawer tabs; candidates for deletion
AgentCanvas docs