2Customizable Node System
Subclass BaseCanvasNode, drop the file in workspace/ β the registry discovers it and the canvas renders it. No frontend code, no manual registration.
A canvas node is a Python class. Subclass BaseCanvasNode, declare what it is (node_type, category, icon), what it plugs into (PortDef inputs and outputs with wire types), what users can tweak (ui_config), and what it does (an async forward()) β then drop the file into workspace/. The WorkspaceComponentRegistry discovers it at startup or hot-reload, and the frontend's GenericBlockRenderer draws it from metadata alone: sidebar entry, ports, inline widgets β no frontend code, no manual registration. Nodesets (BaseNodeSet) bundle related nodes under one shared lifecycle, so one initialize() loads the model they all use. This page is the reference for that surface.
1. BaseCanvasNode
Design docs: Canvas System Β· Loop Control (for kind="control" nodes)
Every canvas element β blocks, composites, controls β inherits from BaseCanvasNode. The class uses ClassVars for declarative metadata and an async forward() method for runtime logic.
from __future__ import annotations
from app.components.bases import BaseCanvasNode, PortDef
class MyNode(BaseCanvasNode):
# Identity
node_type = "myCustomNode"
display_name = "My Node"
description = "Does custom processing"
category = "custom" # sidebar group
icon = "Sparkles" # Lucide icon name
# Kind: "block" | "composite" | "control"
kind = "block"
# Ports
input_ports = [
PortDef(name="text", wire_type="TEXT", description="Input text"),
]
output_ports = [
PortDef(name="result", wire_type="TEXT", description="Processed output"),
]
# Config (JSON Schema for the properties panel)
config_schema = {"temperature": {"type": "number", "default": 0.7}}
default_config = {"temperature": 0.7}
async def forward(self, inputs: dict, ctx: Any) -> dict:
text = inputs["text"]
result = process(text, self.config.get("temperature", 0.7))
return {"result": result}
1.1 Lifecycle Hooks
async def initialize(self) -> None:
"""Called once before first execution. Load models, allocate GPU memory."""
async def shutdown(self) -> None:
"""Called on teardown. Release resources."""
1.2 Instance Attributes
Set by the executor at graph-build time:
| Attribute | Type | Description |
|---|---|---|
self.config |
dict |
Per-node config from graph JSON |
self.node_id |
str |
Node ID from graph JSON |
self._log_buffer |
list[dict] |
Voluntary inner log entries (per-firing). Written by _self_log(), read by log(). |
1.3 Voluntary Logging β _self_log() / log()
The executor automatically captures port I/O and timing (the exterior log layer). Nodes add interior detail β assembled LLM prompts, API responses, token counts, intermediate scores β by calling _self_log() inside forward(). The buffer is drained via log() after forward() returns (ADR-observability-003).
async def forward(self, inputs: dict, ctx: Any) -> dict:
prompt = self._build_prompt(inputs)
self._self_log("prompt", prompt) # record what we asked
response = await llm_complete(prompt)
self._self_log("tokens_used", response.usage.total_tokens)
return {"text": response.text}
Override log() to filter or add computed summaries; the default returns the raw _self_log() buffer.
1.4 _resolve_ports(config) β Dynamic Ports
Some nodes derive their ports from per-instance config rather than class-level declarations β IterIn / IterOut (ADR-dataflow-003) and ImageViewerSink (ADR-components-007) are the canonical examples. Override the _resolve_ports classmethod to return (input_ports, output_ports) computed from config:
@classmethod
def _resolve_ports(cls, config: dict) -> tuple[list, list]:
ports_cfg = config.get("ports", [])
resolved = [PortDef(p["name"], p["wire_type"], optional=True) for p in ports_cfg]
# Sink nodes: inputs only
return (resolved, [])
Default behaviour (on BaseCanvasNode) returns class-level ports unchanged. The graph executor, load-time validator (validate_graph_connectivity), and the frontend's portResolution.ts all consult this method β single source of truth. Expose the config to users via a ConfigField(name="ports", field_type="port_list", β¦) in ui_config.
1.5 Batched Inference β batched / batch_dim
Opt-in ClassVars (ADR-eval-002 PC-1) that route a node's forward() through the per-AutoServerApp BatchedInferenceServer rendezvous. K parallel worker callers submit per-sample inputs; the underlying forward() is invoked once with inputs stacked along batch_dim; K result slices scatter back to K awaiting futures.
from typing import ClassVar
class PolicyForwardNode(BaseCanvasNode):
node_type = "my_policy__forward"
batched: ClassVar[bool] = True
batch_dim: ClassVar[str] = "rgb" # input port carrying the per-sample slot
input_ports = [
PortDef(name="rgb", wire_type="IMAGE"),
PortDef(name="hidden_in", wire_type="ANY"),
]
output_ports = [
PortDef(name="action", wire_type="ACTION"),
PortDef(name="hidden_out", wire_type="ANY"),
]
Contract: the server is pure-functional β any per-call state (RNN hidden state, etc.) must travel on the wire as explicit input/output ports. register_node() validates at scan time that batch_dim names an actual input port.
2. PortDef and Wiring
Design doc: Wire Types
PortDef declares an input or output port. The name becomes the React Flow Handle ID on the frontend and the key in inputs / return dict on the backend.
@dataclass
class PortDef:
name: str # port name (= Handle id)
wire_type: str # "IMAGE", "ACTION", "TEXT", "DEPTH", etc.
description: str = ""
optional: bool = False # if False, node won't fire until this port has data
Wire types are checked at connection time β only matching types can be wired together. WIRE_TYPES contains 12 inner types: IMAGE, DEPTH, ACTION, DISCRETE_ACTION, CONTROL, POSE, TEXT, BOOL, METRICS, OBSERVATION, STEP_RESULT, ANY (POSE was renamed from STATE in ADR-dataflow-004; ANY is a first-class catch-all, not a boundary-only escape hatch).
Any inner type T may be wrapped as LIST[T] β a consumer-side modifier (ADR-dataflow-005). Producer ports stay scalar T; a consumer port declared LIST[T] always sees a list[T], because the executor wraps scalar inputs to length-1 lists and concatenates fan-in in edge declaration order at the port-binding seam. Used for multi-image LLM prompts, multi-LLM debate, and search-population fan-in. Nested LIST[LIST[T]] is not supported in v1.
3. Node UI Config
Design doc: Canvas System
NodeUIConfig controls how GenericBlockRenderer draws the node β no custom .tsx needed.
from app.components.bases import NodeUIConfig, ConfigField, DisplayField
class MyNode(BaseCanvasNode):
ui_config = NodeUIConfig(
color="amber", # Tailwind color key ("" = auto from category)
layout="block", # "block" | "strip" | "viewer" | "imageGrid" | "note"
width="", # explicit width (e.g. "44px" for strip gates)
min_width="",
max_width="",
min_height="",
rounding="", # CSS rounding class (e.g. "rounded-r-lg")
config_fields=[
ConfigField(
name="temperature",
field_type="slider",
label="Temperature",
default=0.7,
min=0.0, max=2.0, step=0.1,
),
ConfigField(
name="model",
field_type="select",
label="Model",
options=[
{"value": "gpt-4o", "label": "GPT-4o"},
{"value": "claude-sonnet", "label": "Claude Sonnet"},
],
),
],
display_fields=[
DisplayField(
name="response",
display_type="log_list",
data_key="text",
max_visible=20,
accumulate=True,
),
],
)
3.1 ConfigField Types
ConfigField declares user-editable inline widgets.
field_type |
Widget | Extra fields |
|---|---|---|
"label" |
Read-only text | β |
"slider" |
Range input | min, max, step |
"text" |
Single-line input | placeholder |
"select" |
Dropdown | options: [{value, label}] |
"toggle" |
Checkbox / switch | β |
"textarea" |
Multi-line input | placeholder |
"port_list" |
Editable port list | Used by IterIn, IterOut, LLMCallNode, ImageViewerSink for configurable ports (ADR-dataflow-003, ADR-components-007) |
3.2 DisplayField Types
DisplayField declares read-only runtime data widgets that render WebSocket viewer_data payloads keyed by data_key (ADR-components-006, ADR-observability-003). Set accumulate=True to append entries over time (bounded by max_visible); False to replace.
display_type |
Widget |
|---|---|
"image_viewer" |
Base64-encoded image rendered as <img> |
"log_list" |
Scrollable list of entries with step numbers (supports accumulate) |
"metric_table" |
Keyβvalue metrics table (SPL, SR, nDTW, β¦) |
"text_viewer" |
Plain string display β scalar shows one block, array shows a scrollable stack with hairline dividers (ADR-components-008) |
3.3 Layout Modes
layout |
When to use |
|---|---|
"block" |
Default β standard rectangle with title, ports, and config/display fields |
"strip" |
Narrow vertical gate (IterIn, IterOut, GraphIn, GraphOut) β ports only, no body |
"viewer" |
Display-sink nodes (TextViewer, TextScroll, ActionLog, Metrics) β hides config_fields, renders display_fields full-width |
"imageGrid" |
ImageViewerSink β grid of image panels derived from config.rows / config.cols / config.ports (ADR-components-007) |
"note" |
Free-text annotation node (NoteLayout) β a canvas sticky-note with no ports or runtime behaviour |
4. BaseNodeSet
Design docs: Server Mode Β· Env Panels
A nodeset bundles multiple BaseCanvasNode tools under shared initialization and lifecycle. Tools in a nodeset load/unload atomically.
from __future__ import annotations
from typing import ClassVar
from app.components.bases import BaseNodeSet
class SamNodeSet(BaseNodeSet):
name = "sam"
description = "Segment Anything Model tools"
# Optional: run this nodeset in its own interpreter via AutoServerApp (ADR-server-001).
# None = use sys.executable.
server_python: ClassVar[str | None] = "/path/to/sam/python"
# Optional: control-plane panel (ADR-server-002) β BaseEnvPanel subclass.
# WorkspaceComponentRegistry instantiates and registers it on load. Used for
# episode control, checkpoint selection, etc.
env_panel: ClassVar[type | None] = None
# Optional: per-episode timeout for batch eval (ADR-eval-002). The batch
# runner clamps each episode at max_steps * default_per_step_budget_sec.
# Habitat ~2.0, LLM-heavy MapGPT ~30.0, framework default 30.0.
default_per_step_budget_sec: ClassVar[float] = 30.0
def get_tools(self) -> list:
return [SegmentTool(), EmbedTool()]
async def initialize(self, **kwargs) -> None:
# Load SAM model, allocate GPU memory
...
async def shutdown(self) -> None:
# Release GPU memory
...
async def get_eval_metadata(self) -> dict:
# Env nodesets override this to expose env_name, splits,
# episode_counts, metrics, supports_set_episode, step_budget.
# Non-env nodesets return {} (default).
return {}
Each tool returned by get_tools() is itself a BaseCanvasNode subclass instance with node_type, input_ports, output_ports, and forward().
5. Auto-Discovery
The WorkspaceComponentRegistry scans workspace/ subdirectories at startup and on hot-reload.
workspace/
nodesets/{env,method,policy,model,common,other}/ *.py
β BaseNodeSet subclasses, grouped by role (ADR-platform-008;
server-mode is per-nodeset via server_python, not a directory)
policies/ *.py β BaseCanvasNode subclasses with policy metadata
nodes/ *.py β BaseCanvasNode subclasses (registered immediately)
5.1 Scan Flow
1. WorkspaceComponentRegistry.scan_all()
2. Pass 1 β frozen workspace (Settings.workspace_dir):
For each subdirectory (nodesets/ with its role buckets, policies/, nodes/):
a. Import every .py file (or package with __init__.py)
b. Find concrete subclasses of the target base class
c. Instantiate and register
3. Pass 2 β active workspace overlay (Settings.active_workspace_dir, optional):
Same scan against the active dir. register_node()'s last-write-wins on
NODE_HANDLERS gives Python-resource override; same for _discovered_nodesets.
None / unset = no Pass 2 = bit-identical behaviour. (ADR-components-009)
4. NodeSets: discovered but NOT eagerly loaded at scan time. They load via
(a) explicit load_nodeset(name) from the NodeSet Manager page, or
(b) ensure_nodesets_for_graph(graph_def) β auto-loads every nodeset
referenced by a graph when the graph is opened or a run is
dispatched (ADR-components-003).
5. Nodes: registered immediately in NODE_HANDLERS
6. Policies: registered in NODE_HANDLERS + policy registry
Non-Python resources (graph JSON, .exp.yaml, hooks.json) are not scanned
into a dict; they're looked up per-call via resolve_graph_path(name) /
resolve_exp_yaml_path(name) on the registry (active-first then frozen
fallthrough). hooks.json uses full-file override (no merge).
5.2 Hot Reload
POST /api/components/reload
β WorkspaceComponentRegistry.unregister_all()
β WorkspaceComponentRegistry.scan_all()
β invalidate_cache() # frontend re-fetches node schemas
6. Frontend Rendering
Design doc: Canvas System
The frontend's GenericBlockRenderer renders any registered node automatically from its metadata. When scan_all() completes (or on reload), the backend serves node schemas via GET /api/components/node-schemas. The frontend uses this to:
- Build sidebar catalog β nodes grouped by
category - Render node blocks β ports, colors, inline config controls, live display widgets from
ui_config - Validate wiring β type-check connections using
wire_type(includingLIST[T]compatibility)
No custom .tsx component is needed unless the node requires specialized visualization beyond what GenericBlockRenderer provides.
7. Key Files
| File | Role |
|---|---|
agentcanvas/backend/app/components/bases.py |
BaseCanvasNode, BaseNodeSet, PortDef, ConfigField, DisplayField, NodeUIConfig |
agentcanvas/backend/app/components/registry.py |
WorkspaceComponentRegistry β scanning, registration, ensure_nodesets_for_graph, load/unload |
agentcanvas/backend/app/agent_loop/builtin_nodes.py |
NODE_HANDLERS dict, register_node() (scan-time batched/batch_dim validation) β moved out of the retired graph_executor.py module |
agentcanvas/backend/app/standard/wire_types.py |
Wire type constants, LIST[T] helpers, serialization |
agentcanvas/backend/app/server/batched_inference.py |
BatchedInferenceServer β batched-inference rendezvous (ADR-eval-002) |
agentcanvas/frontend/src/canvas/nodes/agentloop/inner/GenericBlockRenderer.tsx |
Universal node renderer |
agentcanvas/frontend/src/canvas/portResolution.ts |
Frontend mirror of _resolve_ports β applies config.ports overrides |
Status
| Item | Status | Notes |
|---|---|---|
| BaseCanvasNode (all 14 ClassVars) | Done | node_type, display_name, description, category, icon, kind, input_ports, output_ports, children, config_schema, default_config, ui_config, batched, batch_dim |
| PortDef | Done | name, wire_type, description, optional |
_resolve_ports(config) classmethod |
Done | Instance-level port override (ADR-dataflow-003, ADR-components-007) |
_self_log() / log() |
Done | Voluntary interior log layer (ADR-observability-003) |
Batched inference (batched + batch_dim) |
Done | Scan-time validated; routed via BatchedInferenceServer (ADR-eval-002) |
| NodeUIConfig attrs | Done | color, layout, width, min_width, max_width, min_height, rounding, config_fields, display_fields |
| ConfigField (7 widget types) | Done | label, slider, text, select, toggle, textarea, port_list |
| DisplayField (4 display types) | Done | image_viewer, log_list, metric_table, text_viewer |
| Layout modes (5) | Done | block, strip, viewer, imageGrid, note |
| Wire types (12 inner + LIST[T]) | Done | POSE replaced STATE (ADR-dataflow-004); adds DISCRETE_ACTION + CONTROL; LIST[T] consumer-side modifier (ADR-dataflow-005) |
| BaseNodeSet (name, server_python, env_panel, default_per_step_budget_sec) | Done | plus get_tools, initialize, shutdown, get_eval_metadata |
| WorkspaceComponentRegistry auto-discovery | Done | Scans nodesets/ (role buckets: env/ method/ policy/ model/ common/ other/), policies/, nodes/; ensure_nodesets_for_graph auto-loads on graph open |
Hot reload (POST /api/components/reload) |
Done | Shuts down, re-scans, re-initializes |
| GenericBlockRenderer | Done | Renders any node from metadata; block + strip + viewer + imageGrid + note layouts |
| Node schema endpoint | Done | GET /api/components/node-schemas |
| Legacy base classes removed | Done | BaseTool, BaseSkill, BaseAgent, BasePolicy removed from bases.py |
All items fully implemented.