AgentCanvas / Pages / Developer Guide / Capabilities / 2Customizable Node System
2026-06-17

New canvas nodes are Python classes. Drop a file into workspace/, the platform auto-discovers it at runtime. This page covers the class hierarchy, auto-discovery mechanism, and frontend rendering.


1. BaseCanvasNode

Design docs: Canvas System Β· Loop Control (for kind="control" nodes)

Every canvas element β€” blocks, composites, controls β€” inherits from BaseCanvasNode. The class uses ClassVars for declarative metadata and an async forward() method for runtime logic.

from __future__ import annotations
from app.components.bases import BaseCanvasNode, PortDef

class MyNode(BaseCanvasNode):
    # Identity
    node_type = "myCustomNode"
    display_name = "My Node"
    description = "Does custom processing"
    category = "custom"        # sidebar group
    icon = "Sparkles"          # Lucide icon name

    # Kind: "block" | "composite" | "control"
    kind = "block"

    # Ports
    input_ports = [
        PortDef(name="text", wire_type="TEXT", description="Input text"),
    ]
    output_ports = [
        PortDef(name="result", wire_type="TEXT", description="Processed output"),
    ]

    # Config (JSON Schema for the properties panel)
    config_schema = {"temperature": {"type": "number", "default": 0.7}}
    default_config = {"temperature": 0.7}

    async def forward(self, inputs: dict, ctx: Any) -> dict:
        text = inputs["text"]
        result = process(text, self.config.get("temperature", 0.7))
        return {"result": result}

1.1 Lifecycle Hooks

async def initialize(self) -> None:
    """Called once before first execution. Load models, allocate GPU memory."""

async def shutdown(self) -> None:
    """Called on teardown. Release resources."""

1.2 Instance Attributes

Set by the executor at graph-build time:

Attribute Type Description
self.config dict Per-node config from graph JSON
self.node_id str Node ID from graph JSON
self._log_buffer list[dict] Voluntary inner log entries (per-firing). Written by _self_log(), read by log().

1.3 Voluntary Logging β€” _self_log() / log()

The executor automatically captures port I/O and timing (the exterior log layer). Nodes add interior detail β€” assembled LLM prompts, API responses, token counts, intermediate scores β€” by calling _self_log() inside forward(). The buffer is drained via log() after forward() returns (ADR-observability-003).

async def forward(self, inputs: dict, ctx: Any) -> dict:
    prompt = self._build_prompt(inputs)
    self._self_log("prompt", prompt)                       # record what we asked
    response = await llm_complete(prompt)
    self._self_log("tokens_used", response.usage.total_tokens)
    return {"text": response.text}

Override log() to filter or add computed summaries; the default returns the raw _self_log() buffer.

1.4 _resolve_ports(config) β€” Dynamic Ports

Some nodes derive their ports from per-instance config rather than class-level declarations β€” IterIn / IterOut (ADR-dataflow-003) and ImageViewerSink (ADR-components-007) are the canonical examples. Override the _resolve_ports classmethod to return (input_ports, output_ports) computed from config:

@classmethod
def _resolve_ports(cls, config: dict) -> tuple[list, list]:
    ports_cfg = config.get("ports", [])
    resolved = [PortDef(p["name"], p["wire_type"], optional=True) for p in ports_cfg]
    # Sink nodes: inputs only
    return (resolved, [])

Default behaviour (on BaseCanvasNode) returns class-level ports unchanged. The graph executor, load-time validator (validate_graph_connectivity), and the frontend's portResolution.ts all consult this method β€” single source of truth. Expose the config to users via a ConfigField(name="ports", field_type="port_list", …) in ui_config.

1.5 Batched Inference β€” batched / batch_dim

Opt-in ClassVars (ADR-eval-002 PC-1) that route a node's forward() through the per-AutoServerApp BatchedInferenceServer rendezvous. K parallel worker callers submit per-sample inputs; the underlying forward() is invoked once with inputs stacked along batch_dim; K result slices scatter back to K awaiting futures.

from typing import ClassVar

class PolicyForwardNode(BaseCanvasNode):
    node_type = "my_policy__forward"
    batched: ClassVar[bool] = True
    batch_dim: ClassVar[str] = "rgb"   # input port carrying the per-sample slot

    input_ports = [
        PortDef(name="rgb", wire_type="IMAGE"),
        PortDef(name="hidden_in", wire_type="ANY"),
    ]
    output_ports = [
        PortDef(name="action", wire_type="ACTION"),
        PortDef(name="hidden_out", wire_type="ANY"),
    ]

Contract: the server is pure-functional β€” any per-call state (RNN hidden state, etc.) must travel on the wire as explicit input/output ports. register_node() validates at scan time that batch_dim names an actual input port.

2. PortDef and Wiring

Design doc: Wire Types

PortDef declares an input or output port. The name becomes the React Flow Handle ID on the frontend and the key in inputs / return dict on the backend.

@dataclass
class PortDef:
    name: str             # port name (= Handle id)
    wire_type: str        # "IMAGE", "ACTION", "TEXT", "DEPTH", etc.
    description: str = ""
    optional: bool = False  # if False, node won't fire until this port has data

Wire types are checked at connection time β€” only matching types can be wired together. WIRE_TYPES contains 12 inner types: IMAGE, DEPTH, ACTION, DISCRETE_ACTION, CONTROL, POSE, TEXT, BOOL, METRICS, OBSERVATION, STEP_RESULT, ANY (POSE was renamed from STATE in ADR-dataflow-004; ANY is a first-class catch-all, not a boundary-only escape hatch).

Any inner type T may be wrapped as LIST[T] β€” a consumer-side modifier (ADR-dataflow-005). Producer ports stay scalar T; a consumer port declared LIST[T] always sees a list[T], because the executor wraps scalar inputs to length-1 lists and concatenates fan-in in edge declaration order at the port-binding seam. Used for multi-image LLM prompts, multi-LLM debate, and search-population fan-in. Nested LIST[LIST[T]] is not supported in v1.

3. Node UI Config

Design doc: Canvas System

NodeUIConfig controls how GenericBlockRenderer draws the node β€” no custom .tsx needed.

from app.components.bases import NodeUIConfig, ConfigField, DisplayField

class MyNode(BaseCanvasNode):
    ui_config = NodeUIConfig(
        color="amber",           # Tailwind color key ("" = auto from category)
        layout="block",          # "block" | "strip" | "viewer" | "imageGrid" | "note"
        width="",                # explicit width (e.g. "44px" for strip gates)
        min_width="",
        max_width="",
        min_height="",
        rounding="",             # CSS rounding class (e.g. "rounded-r-lg")
        config_fields=[
            ConfigField(
                name="temperature",
                field_type="slider",
                label="Temperature",
                default=0.7,
                min=0.0, max=2.0, step=0.1,
            ),
            ConfigField(
                name="model",
                field_type="select",
                label="Model",
                options=[
                    {"value": "gpt-4o", "label": "GPT-4o"},
                    {"value": "claude-sonnet", "label": "Claude Sonnet"},
                ],
            ),
        ],
        display_fields=[
            DisplayField(
                name="response",
                display_type="log_list",
                data_key="text",
                max_visible=20,
                accumulate=True,
            ),
        ],
    )

3.1 ConfigField Types

ConfigField declares user-editable inline widgets.

field_type Widget Extra fields
"label" Read-only text β€”
"slider" Range input min, max, step
"text" Single-line input placeholder
"select" Dropdown options: [{value, label}]
"toggle" Checkbox / switch β€”
"textarea" Multi-line input placeholder
"port_list" Editable port list Used by IterIn, IterOut, LLMCallNode, ImageViewerSink for configurable ports (ADR-dataflow-003, ADR-components-007)

3.2 DisplayField Types

DisplayField declares read-only runtime data widgets that render WebSocket viewer_data payloads keyed by data_key (ADR-components-006, ADR-observability-003). Set accumulate=True to append entries over time (bounded by max_visible); False to replace.

display_type Widget
"image_viewer" Base64-encoded image rendered as <img>
"log_list" Scrollable list of entries with step numbers (supports accumulate)
"metric_table" Key–value metrics table (SPL, SR, nDTW, …)
"text_viewer" Plain string display β€” scalar shows one block, array shows a scrollable stack with hairline dividers (ADR-components-008)

3.3 Layout Modes

layout When to use
"block" Default β€” standard rectangle with title, ports, and config/display fields
"strip" Narrow vertical gate (IterIn, IterOut, GraphIn, GraphOut) β€” ports only, no body
"viewer" Display-sink nodes (TextViewer, TextScroll, ActionLog, Metrics) β€” hides config_fields, renders display_fields full-width
"imageGrid" ImageViewerSink β€” grid of image panels derived from config.rows / config.cols / config.ports (ADR-components-007)
"note" Free-text annotation node (NoteLayout) β€” a canvas sticky-note with no ports or runtime behaviour

4. BaseNodeSet

Design docs: Server Mode Β· Env Panels

A nodeset bundles multiple BaseCanvasNode tools under shared initialization and lifecycle. Tools in a nodeset load/unload atomically.

from __future__ import annotations
from typing import ClassVar
from app.components.bases import BaseNodeSet

class SamNodeSet(BaseNodeSet):
    name = "sam"
    description = "Segment Anything Model tools"

    # Optional: run this nodeset in its own interpreter via AutoServerApp (ADR-server-001).
    # None = use sys.executable.
    server_python: ClassVar[str | None] = "/path/to/sam/python"

    # Optional: control-plane panel (ADR-server-002) β€” BaseEnvPanel subclass.
    # WorkspaceComponentRegistry instantiates and registers it on load. Used for
    # episode control, checkpoint selection, etc.
    env_panel: ClassVar[type | None] = None

    # Optional: per-episode timeout for batch eval (ADR-eval-002). The batch
    # runner clamps each episode at max_steps * default_per_step_budget_sec.
    # Habitat ~2.0, LLM-heavy MapGPT ~30.0, framework default 30.0.
    default_per_step_budget_sec: ClassVar[float] = 30.0

    def get_tools(self) -> list:
        return [SegmentTool(), EmbedTool()]

    async def initialize(self, **kwargs) -> None:
        # Load SAM model, allocate GPU memory
        ...

    async def shutdown(self) -> None:
        # Release GPU memory
        ...

    async def get_eval_metadata(self) -> dict:
        # Env nodesets override this to expose env_name, splits,
        # episode_counts, metrics, supports_set_episode, step_budget.
        # Non-env nodesets return {} (default).
        return {}

Each tool returned by get_tools() is itself a BaseCanvasNode subclass instance with node_type, input_ports, output_ports, and forward().

5. Auto-Discovery

The WorkspaceComponentRegistry scans workspace/ subdirectories at startup and on hot-reload.

workspace/
  nodesets/{env,method,policy,model,common,other}/ *.py
                         β†’ BaseNodeSet subclasses, grouped by role (ADR-platform-008;
                           server-mode is per-nodeset via server_python, not a directory)
  policies/        *.py  β†’ BaseCanvasNode subclasses with policy metadata
  nodes/           *.py  β†’ BaseCanvasNode subclasses (registered immediately)

5.1 Scan Flow

1. WorkspaceComponentRegistry.scan_all()
2. Pass 1 β€” frozen workspace (Settings.workspace_dir):
   For each subdirectory (nodesets/ with its role buckets, policies/, nodes/):
     a. Import every .py file (or package with __init__.py)
     b. Find concrete subclasses of the target base class
     c. Instantiate and register
3. Pass 2 β€” active workspace overlay (Settings.active_workspace_dir, optional):
   Same scan against the active dir. register_node()'s last-write-wins on
   NODE_HANDLERS gives Python-resource override; same for _discovered_nodesets.
   None / unset = no Pass 2 = bit-identical behaviour. (ADR-components-009)
4. NodeSets: discovered but NOT eagerly loaded at scan time. They load via
   (a) explicit load_nodeset(name) from the NodeSet Manager page, or
   (b) ensure_nodesets_for_graph(graph_def) β€” auto-loads every nodeset
       referenced by a graph when the graph is opened or a run is
       dispatched (ADR-components-003).
5. Nodes: registered immediately in NODE_HANDLERS
6. Policies: registered in NODE_HANDLERS + policy registry

Non-Python resources (graph JSON, .exp.yaml, hooks.json) are not scanned into a dict; they're looked up per-call via resolve_graph_path(name) / resolve_exp_yaml_path(name) on the registry (active-first then frozen fallthrough). hooks.json uses full-file override (no merge).

5.2 Hot Reload

POST /api/components/reload
  β†’ WorkspaceComponentRegistry.unregister_all()
  β†’ WorkspaceComponentRegistry.scan_all()
  β†’ invalidate_cache()  # frontend re-fetches node schemas

6. Frontend Rendering

Design doc: Canvas System

The frontend's GenericBlockRenderer renders any registered node automatically from its metadata. When scan_all() completes (or on reload), the backend serves node schemas via GET /api/components/node-schemas. The frontend uses this to:

  1. Build sidebar catalog β€” nodes grouped by category
  2. Render node blocks β€” ports, colors, inline config controls, live display widgets from ui_config
  3. Validate wiring β€” type-check connections using wire_type (including LIST[T] compatibility)

No custom .tsx component is needed unless the node requires specialized visualization beyond what GenericBlockRenderer provides.

7. Key Files

File Role
agentcanvas/backend/app/components/bases.py BaseCanvasNode, BaseNodeSet, PortDef, ConfigField, DisplayField, NodeUIConfig
agentcanvas/backend/app/components/registry.py WorkspaceComponentRegistry β€” scanning, registration, ensure_nodesets_for_graph, load/unload
agentcanvas/backend/app/agent_loop/builtin_nodes.py NODE_HANDLERS dict, register_node() (scan-time batched/batch_dim validation) β€” moved out of the retired graph_executor.py module
agentcanvas/backend/app/standard/wire_types.py Wire type constants, LIST[T] helpers, serialization
agentcanvas/backend/app/server/batched_inference.py BatchedInferenceServer β€” batched-inference rendezvous (ADR-eval-002)
agentcanvas/frontend/src/canvas/nodes/agentloop/inner/GenericBlockRenderer.tsx Universal node renderer
agentcanvas/frontend/src/canvas/portResolution.ts Frontend mirror of _resolve_ports β€” applies config.ports overrides

Status

Item Status Notes
BaseCanvasNode (all 14 ClassVars) Done node_type, display_name, description, category, icon, kind, input_ports, output_ports, children, config_schema, default_config, ui_config, batched, batch_dim
PortDef Done name, wire_type, description, optional
_resolve_ports(config) classmethod Done Instance-level port override (ADR-dataflow-003, ADR-components-007)
_self_log() / log() Done Voluntary interior log layer (ADR-observability-003)
Batched inference (batched + batch_dim) Done Scan-time validated; routed via BatchedInferenceServer (ADR-eval-002)
NodeUIConfig attrs Done color, layout, width, min_width, max_width, min_height, rounding, config_fields, display_fields
ConfigField (7 widget types) Done label, slider, text, select, toggle, textarea, port_list
DisplayField (4 display types) Done image_viewer, log_list, metric_table, text_viewer
Layout modes (5) Done block, strip, viewer, imageGrid, note
Wire types (12 inner + LIST[T]) Done POSE replaced STATE (ADR-dataflow-004); adds DISCRETE_ACTION + CONTROL; LIST[T] consumer-side modifier (ADR-dataflow-005)
BaseNodeSet (name, server_python, env_panel, default_per_step_budget_sec) Done plus get_tools, initialize, shutdown, get_eval_metadata
WorkspaceComponentRegistry auto-discovery Done Scans nodesets/ (role buckets: env/ method/ policy/ model/ common/ other/), policies/, nodes/; ensure_nodesets_for_graph auto-loads on graph open
Hot reload (POST /api/components/reload) Done Shuts down, re-scans, re-initializes
GenericBlockRenderer Done Renders any node from metadata; block + strip + viewer + imageGrid + note layouts
Node schema endpoint Done GET /api/components/node-schemas
Legacy base classes removed Done BaseTool, BaseSkill, BaseAgent, BasePolicy removed from bases.py

All items fully implemented.

AgentCanvas docs