2Customizable Node System
Subclass BaseCanvasNode, register in WorkspaceComponentRegistry β new node types appear in the workspace automatically.
New canvas nodes are Python classes. Drop a file into workspace/, the platform auto-discovers it at runtime. This page covers the class hierarchy, auto-discovery mechanism, and frontend rendering.
1. BaseCanvasNode
Design docs: Canvas System Β· Loop Control (for kind="control" nodes)
Every canvas element β blocks, composites, controls β inherits from BaseCanvasNode. The class uses ClassVars for declarative metadata and an async forward() method for runtime logic.
from __future__ import annotations
from app.components.bases import BaseCanvasNode, PortDef
class MyNode(BaseCanvasNode):
# Identity
node_type = "myCustomNode"
display_name = "My Node"
description = "Does custom processing"
category = "custom" # sidebar group
icon = "Sparkles" # Lucide icon name
# Kind: "block" | "composite" | "control"
kind = "block"
# Ports
input_ports = [
PortDef(name="text", wire_type="TEXT", description="Input text"),
]
output_ports = [
PortDef(name="result", wire_type="TEXT", description="Processed output"),
]
# Config (JSON Schema for the properties panel)
config_schema = {"temperature": {"type": "number", "default": 0.7}}
default_config = {"temperature": 0.7}
async def forward(self, inputs: dict, ctx: Any) -> dict:
text = inputs["text"]
result = process(text, self.config.get("temperature", 0.7))
return {"result": result}
1.1 Lifecycle Hooks
async def initialize(self) -> None:
"""Called once before first execution. Load models, allocate GPU memory."""
async def shutdown(self) -> None:
"""Called on teardown. Release resources."""
1.2 Instance Attributes
Set by the executor at graph-build time:
| Attribute | Type | Description |
|---|---|---|
self.config |
dict |
Per-node config from graph JSON |
self.node_id |
str |
Node ID from graph JSON |
self._log_buffer |
list[dict] |
Voluntary inner log entries (per-firing). Written by _self_log(), read by log(). |
1.3 Voluntary Logging β _self_log() / log()
The executor automatically captures port I/O and timing (the exterior log layer). Nodes add interior detail β assembled LLM prompts, API responses, token counts, intermediate scores β by calling _self_log() inside forward(). The buffer is drained via log() after forward() returns (ADR-observability-003).
async def forward(self, inputs: dict, ctx: Any) -> dict:
prompt = self._build_prompt(inputs)
self._self_log("prompt", prompt) # record what we asked
response = await llm_complete(prompt)
self._self_log("tokens_used", response.usage.total_tokens)
return {"text": response.text}
Override log() to filter or add computed summaries; the default returns the raw _self_log() buffer.
1.4 _resolve_ports(config) β Dynamic Ports
Some nodes derive their ports from per-instance config rather than class-level declarations β IterIn / IterOut (ADR-dataflow-003) and ImageViewerSink (ADR-components-007) are the canonical examples. Override the _resolve_ports classmethod to return (input_ports, output_ports) computed from config:
@classmethod
def _resolve_ports(cls, config: dict) -> tuple[list, list]:
ports_cfg = config.get("ports", [])
resolved = [PortDef(p["name"], p["wire_type"], optional=True) for p in ports_cfg]
# Sink nodes: inputs only
return (resolved, [])
Default behaviour (on BaseCanvasNode) returns class-level ports unchanged. The graph executor, load-time validator (validate_graph_connectivity), and the frontend's portResolution.ts all consult this method β single source of truth. Expose the config to users via a ConfigField(name="ports", field_type="port_list", β¦) in ui_config.
1.5 Batched Inference β batched / batch_dim
Opt-in ClassVars (ADR-eval-002 PC-1) that route a node's forward() through the per-AutoServerApp BatchedInferenceServer rendezvous. K parallel worker callers submit per-sample inputs; the underlying forward() is invoked once with inputs stacked along batch_dim; K result slices scatter back to K awaiting futures.
from typing import ClassVar
class PolicyForwardNode(BaseCanvasNode):
node_type = "my_policy__forward"
batched: ClassVar[bool] = True
batch_dim: ClassVar[str] = "rgb" # input port carrying the per-sample slot
input_ports = [
PortDef(name="rgb", wire_type="IMAGE"),
PortDef(name="hidden_in", wire_type="ANY"),
]
output_ports = [
PortDef(name="action", wire_type="ACTION"),
PortDef(name="hidden_out", wire_type="ANY"),
]
Contract: the server is pure-functional β any per-call state (RNN hidden state, etc.) must travel on the wire as explicit input/output ports. register_node() validates at scan time that batch_dim names an actual input port.
2. PortDef and Wiring
Design doc: Wire Types
PortDef declares an input or output port. The name becomes the React Flow Handle ID on the frontend and the key in inputs / return dict on the backend.
@dataclass
class PortDef:
name: str # port name (= Handle id)
wire_type: str # "IMAGE", "ACTION", "TEXT", "DEPTH", etc.
description: str = ""
optional: bool = False # if False, node won't fire until this port has data
Wire types are checked at connection time β only matching types can be wired together. WIRE_TYPES contains 12 inner types: IMAGE, DEPTH, ACTION, DISCRETE_ACTION, CONTROL, POSE, TEXT, BOOL, METRICS, OBSERVATION, STEP_RESULT, ANY (POSE was renamed from STATE in ADR-dataflow-004; ANY is a first-class catch-all, not a boundary-only escape hatch).
Any inner type T may be wrapped as LIST[T] β a consumer-side modifier (ADR-dataflow-005). Producer ports stay scalar T; a consumer port declared LIST[T] always sees a list[T], because the executor wraps scalar inputs to length-1 lists and concatenates fan-in in edge declaration order at the port-binding seam. Used for multi-image LLM prompts, multi-LLM debate, and search-population fan-in. Nested LIST[LIST[T]] is not supported in v1.
3. Node UI Config
Design doc: Canvas System
NodeUIConfig controls how GenericBlockRenderer draws the node β no custom .tsx needed.
from app.components.bases import NodeUIConfig, ConfigField, DisplayField
class MyNode(BaseCanvasNode):
ui_config = NodeUIConfig(
color="amber", # Tailwind color key ("" = auto from category)
layout="block", # "block" | "strip" | "viewer" | "imageGrid" | "note"
width="", # explicit width (e.g. "44px" for strip gates)
min_width="",
max_width="",
min_height="",
rounding="", # CSS rounding class (e.g. "rounded-r-lg")
config_fields=[
ConfigField(
name="temperature",
field_type="slider",
label="Temperature",
default=0.7,
min=0.0, max=2.0, step=0.1,
),
ConfigField(
name="model",
field_type="select",
label="Model",
options=[
{"value": "gpt-4o", "label": "GPT-4o"},
{"value": "claude-sonnet", "label": "Claude Sonnet"},
],
),
],
display_fields=[
DisplayField(
name="response",
display_type="log_list",
data_key="text",
max_visible=20,
accumulate=True,
),
],
)
3.1 ConfigField Types
ConfigField declares user-editable inline widgets.
field_type |
Widget | Extra fields |
|---|---|---|
"label" |
Read-only text | β |
"slider" |
Range input | min, max, step |
"text" |
Single-line input | placeholder |
"select" |
Dropdown | options: [{value, label}] |
"toggle" |
Checkbox / switch | β |
"textarea" |
Multi-line input | placeholder |
"port_list" |
Editable port list | Used by IterIn, IterOut, LLMCallNode, ImageViewerSink for configurable ports (ADR-dataflow-003, ADR-components-007) |
3.2 DisplayField Types
DisplayField declares read-only runtime data widgets that render WebSocket viewer_data payloads keyed by data_key (ADR-components-006, ADR-observability-003). Set accumulate=True to append entries over time (bounded by max_visible); False to replace.
display_type |
Widget |
|---|---|
"image_viewer" |
Base64-encoded image rendered as <img> |
"log_list" |
Scrollable list of entries with step numbers (supports accumulate) |
"metric_table" |
Keyβvalue metrics table (SPL, SR, nDTW, β¦) |
"text_viewer" |
Plain string display β scalar shows one block, array shows a scrollable stack with hairline dividers (ADR-components-008) |
3.3 Layout Modes
layout |
When to use |
|---|---|
"block" |
Default β standard rectangle with title, ports, and config/display fields |
"strip" |
Narrow vertical gate (IterIn, IterOut, GraphIn, GraphOut) β ports only, no body |
"viewer" |
Display-sink nodes (TextViewer, TextScroll, ActionLog, Metrics) β hides config_fields, renders display_fields full-width |
"imageGrid" |
ImageViewerSink β grid of image panels derived from config.rows / config.cols / config.ports (ADR-components-007) |
"note" |
Free-text annotation node (NoteLayout) β a canvas sticky-note with no ports or runtime behaviour |
4. BaseNodeSet
Design docs: Server Mode Β· Env Panels
A nodeset bundles multiple BaseCanvasNode tools under shared initialization and lifecycle. Tools in a nodeset load/unload atomically.
from __future__ import annotations
from typing import ClassVar
from app.components.bases import BaseNodeSet
class SamNodeSet(BaseNodeSet):
name = "sam"
description = "Segment Anything Model tools"
# Optional: run this nodeset in its own interpreter via AutoServerApp (ADR-server-001).
# None = use sys.executable.
server_python: ClassVar[str | None] = "/path/to/sam/python"
# Optional: control-plane panel (ADR-server-002) β BaseEnvPanel subclass.
# WorkspaceComponentRegistry instantiates and registers it on load. Used for
# episode control, checkpoint selection, etc.
env_panel: ClassVar[type | None] = None
# Optional: per-episode timeout for batch eval (ADR-eval-002). The batch
# runner clamps each episode at max_steps * default_per_step_budget_sec.
# Habitat ~2.0, LLM-heavy MapGPT ~30.0, framework default 30.0.
default_per_step_budget_sec: ClassVar[float] = 30.0
def get_tools(self) -> list:
return [SegmentTool(), EmbedTool()]
async def initialize(self, **kwargs) -> None:
# Load SAM model, allocate GPU memory
...
async def shutdown(self) -> None:
# Release GPU memory
...
async def get_eval_metadata(self) -> dict:
# Env nodesets override this to expose env_name, splits,
# episode_counts, metrics, supports_set_episode, step_budget.
# Non-env nodesets return {} (default).
return {}
Each tool returned by get_tools() is itself a BaseCanvasNode subclass instance with node_type, input_ports, output_ports, and forward().
5. Auto-Discovery
The WorkspaceComponentRegistry scans workspace/ subdirectories at startup and on hot-reload.
workspace/
nodesets/{env,method,policy,model,common,other}/ *.py
β BaseNodeSet subclasses, grouped by role (ADR-platform-008;
server-mode is per-nodeset via server_python, not a directory)
policies/ *.py β BaseCanvasNode subclasses with policy metadata
nodes/ *.py β BaseCanvasNode subclasses (registered immediately)
5.1 Scan Flow
1. WorkspaceComponentRegistry.scan_all()
2. Pass 1 β frozen workspace (Settings.workspace_dir):
For each subdirectory (nodesets/ with its role buckets, policies/, nodes/):
a. Import every .py file (or package with __init__.py)
b. Find concrete subclasses of the target base class
c. Instantiate and register
3. Pass 2 β active workspace overlay (Settings.active_workspace_dir, optional):
Same scan against the active dir. register_node()'s last-write-wins on
NODE_HANDLERS gives Python-resource override; same for _discovered_nodesets.
None / unset = no Pass 2 = bit-identical behaviour. (ADR-components-009)
4. NodeSets: discovered but NOT eagerly loaded at scan time. They load via
(a) explicit load_nodeset(name) from the NodeSet Manager page, or
(b) ensure_nodesets_for_graph(graph_def) β auto-loads every nodeset
referenced by a graph when the graph is opened or a run is
dispatched (ADR-components-003).
5. Nodes: registered immediately in NODE_HANDLERS
6. Policies: registered in NODE_HANDLERS + policy registry
Non-Python resources (graph JSON, .exp.yaml, hooks.json) are not scanned
into a dict; they're looked up per-call via resolve_graph_path(name) /
resolve_exp_yaml_path(name) on the registry (active-first then frozen
fallthrough). hooks.json uses full-file override (no merge).
5.2 Hot Reload
POST /api/components/reload
β WorkspaceComponentRegistry.unregister_all()
β WorkspaceComponentRegistry.scan_all()
β invalidate_cache() # frontend re-fetches node schemas
6. Frontend Rendering
Design doc: Canvas System
The frontend's GenericBlockRenderer renders any registered node automatically from its metadata. When scan_all() completes (or on reload), the backend serves node schemas via GET /api/components/node-schemas. The frontend uses this to:
- Build sidebar catalog β nodes grouped by
category - Render node blocks β ports, colors, inline config controls, live display widgets from
ui_config - Validate wiring β type-check connections using
wire_type(includingLIST[T]compatibility)
No custom .tsx component is needed unless the node requires specialized visualization beyond what GenericBlockRenderer provides.
7. Key Files
| File | Role |
|---|---|
agentcanvas/backend/app/components/bases.py |
BaseCanvasNode, BaseNodeSet, PortDef, ConfigField, DisplayField, NodeUIConfig |
agentcanvas/backend/app/components/registry.py |
WorkspaceComponentRegistry β scanning, registration, ensure_nodesets_for_graph, load/unload |
agentcanvas/backend/app/agent_loop/builtin_nodes.py |
NODE_HANDLERS dict, register_node() (scan-time batched/batch_dim validation) β moved out of the retired graph_executor.py module |
agentcanvas/backend/app/standard/wire_types.py |
Wire type constants, LIST[T] helpers, serialization |
agentcanvas/backend/app/server/batched_inference.py |
BatchedInferenceServer β batched-inference rendezvous (ADR-eval-002) |
agentcanvas/frontend/src/canvas/nodes/agentloop/inner/GenericBlockRenderer.tsx |
Universal node renderer |
agentcanvas/frontend/src/canvas/portResolution.ts |
Frontend mirror of _resolve_ports β applies config.ports overrides |
Status
| Item | Status | Notes |
|---|---|---|
| BaseCanvasNode (all 14 ClassVars) | Done | node_type, display_name, description, category, icon, kind, input_ports, output_ports, children, config_schema, default_config, ui_config, batched, batch_dim |
| PortDef | Done | name, wire_type, description, optional |
_resolve_ports(config) classmethod |
Done | Instance-level port override (ADR-dataflow-003, ADR-components-007) |
_self_log() / log() |
Done | Voluntary interior log layer (ADR-observability-003) |
Batched inference (batched + batch_dim) |
Done | Scan-time validated; routed via BatchedInferenceServer (ADR-eval-002) |
| NodeUIConfig attrs | Done | color, layout, width, min_width, max_width, min_height, rounding, config_fields, display_fields |
| ConfigField (7 widget types) | Done | label, slider, text, select, toggle, textarea, port_list |
| DisplayField (4 display types) | Done | image_viewer, log_list, metric_table, text_viewer |
| Layout modes (5) | Done | block, strip, viewer, imageGrid, note |
| Wire types (12 inner + LIST[T]) | Done | POSE replaced STATE (ADR-dataflow-004); adds DISCRETE_ACTION + CONTROL; LIST[T] consumer-side modifier (ADR-dataflow-005) |
| BaseNodeSet (name, server_python, env_panel, default_per_step_budget_sec) | Done | plus get_tools, initialize, shutdown, get_eval_metadata |
| WorkspaceComponentRegistry auto-discovery | Done | Scans nodesets/ (role buckets: env/ method/ policy/ model/ common/ other/), policies/, nodes/; ensure_nodesets_for_graph auto-loads on graph open |
Hot reload (POST /api/components/reload) |
Done | Shuts down, re-scans, re-initializes |
| GenericBlockRenderer | Done | Renders any node from metadata; block + strip + viewer + imageGrid + note layouts |
| Node schema endpoint | Done | GET /api/components/node-schemas |
| Legacy base classes removed | Done | BaseTool, BaseSkill, BaseAgent, BasePolicy removed from bases.py |
All items fully implemented.