Server Mode

Server mode lets an external system — one that needs a different Python env, a non-Python runtime, or a separate machine — appear on the canvas as ordinary nodes. The system runs as a standalone HTTP service that exposes typed functions through a /manifest; AgentCanvas launches it, reads the manifest, and auto-generates proxy nodes that forward each firing over the wire. From the executor's side a proxy node is indistinguishable from a local one. This page owns the server classes, their lifecycle, and the parallelism gate; the nodeset that gets auto-hosted is NodeSets, and the control panel it may carry is Env Panels.

The two sides — one class per process.

1. What server mode is, and when you need it

Reach for server mode whenever a capability can't live inside the 3.11 backend process. Four drivers recur:

• Dependency isolation — Habitat-Sim needs Python 3.8 + CUDA + GL; hosting it in-process would drag all of that into the web server.
• Language — a C++ SLAM stack or a Rust planner can't be a Python node at all.
• Deployment — run the simulator on a GPU box and the dashboard on a laptop.
• Resource isolation — keep several GPU-heavy systems from contending in one process.

The common thread: move the system to its own process and talk to it over HTTP with typed wire data. If a capability runs fine in the backend env, it does not need a server — a plain local nodeset is simpler and lower-latency.

2. The smallest server

You rarely write a server by hand. The canonical path is to let a nodeset auto-host itself — set one ClassVar and the registry spawns the subprocess, reads the manifest, and generates the proxy nodes for you:

class HabitatNodeSet(BaseNodeSet):
    name = "env_habitat"
    server_python = "python"  # ← that's the whole trigger
    # get_tools() returns the observe / step / evaluate nodes as usual

Load it and env_habitat__observe, env_habitat__step, … appear in the catalog as proxy nodes, backed by a subprocess running that interpreter. You only write a ServerApp by hand (§8) — or a ServerNodeSet — when you need a non-Python service, a custom protocol, or a remote machine. The auto-host machinery is §10.

3. The two sides at a glance

A server-mode nodeset runs as two cooperating classes, one in each process:

4. Lifecycle

4.1 Server States

A /health OK while connected keeps it connected (recovery is silent); stop() returns any state to stopped.

4.2 How start() works

1. subprocess.Popen(command, shell=isinstance(command, str)) — a list command (preferred) spawns with no shell, so PR_SET_PDEATHSIG binds directly to the python child (§4.4); a str command keeps the legacy /bin/sh -c path. Records the PID and starts a new session (os.setsid).
2. _wait_for_health() — polls GET /health, interval 0.5 s backing off to 3.0 s. Liveness-driven: a dead subprocess fast-fails immediately, and startup_timeout is only the ceiling for a pathological hang.
3. On first healthy probe, server.connected = True and server.status = "connected".

4.3 How stop() works

1. os.killpg(SIGTERM) — signals the whole process group; waits up to 5 s for a graceful exit.
2. os.killpg(SIGKILL) — if anything is still alive after the grace window.
3. Resets state: server.connected = False, server.pid = None, server.status = "stopped".

4.4 Crash-safe cleanup

stop() only runs when the framework is in control — graceful shutdown, REST /stop, normal exit. If the backend dies a way that bypasses cleanup hooks — kill -9, OOM-kill, segfault, kernel panic, force-quit before stop() is called — every auto_host subprocess becomes an init-orphan (PPID=1) and keeps running with its GPU memory and ports still held, blocking the next backend start. This was a recurring failure mode for stateful nodesets (e.g. a 14 GiB Prismatic VLM left squatting on a 24 GiB GPU after a crashed eval).

The framework defends against this with Linux PR_SET_PDEATHSIG: each auto_host child arms SIGTERM as its parent-death signal at startup. The kernel then guarantees that when the immediate parent dies any way, this process receives SIGTERM within milliseconds — no user-space code in the parent has to run.

Two arming points (belt-and-suspenders, since the spawn chain typically layers bash -c and a conda run wrapper between uvicorn and the python child, and tail-exec optimisations vary by shell):

Either arming point catches the typical case; together they catch all spawn-chain shapes. End-to-end test (SIGKILL the backend with active auto_host children): all children die within ~300 ms, GPU memory released, ports freed.

Scope and limits

• Linux only. AgentCanvas only targets Linux anyway (habitat-sim is Linux-first; macOS/Windows aren't supported simulator targets), so the platform restriction has no practical cost.
• Does not protect against the auto_host process itself hanging in an uninterruptible D-state syscall, or against a nodeset import installing a custom SIGTERM handler that ignores the signal. Both are pathological and out of scope.
• Does not retroactively protect orphans that existed before this mechanism was deployed — they need a one-time manual cleanup.

The framework-side stop() (§4.3) remains the primary cleanup path; PDEATHSIG is the kernel-level backstop for the fraction of crash modes where stop() does not get to run.

5. YAML Config

Servers are registered via YAML files in workspace/servers/:

# workspace/servers/habitat.yaml
url: http://localhost:9100
enabled: true          # false to skip during scan
managed: true          # true = framework starts/stops the process
command: "python -m app.server.examples.habitat_server --port 9100"
working_dir: /path/to/vlnworkspace/agentcanvas/backend
startup_timeout: 60    # seconds to wait for /health
auto_restart: false    # restart on crash

5.1 Three deployment modes

All three share the same manifest protocol and proxy node generation.

6. REST API (on AgentCanvas)

Example response from GET /api/components/servers:

[
  {
    "name": "habitat",
    "description": "Habitat-Sim VLN-CE environment",
    "url": "http://localhost:9100",
    "status": "connected",
    "pid": 12345,
    "connected": true,
    "error": null,
    "auto_restart": false,
    "nodes": ["env_habitat__observe", "env_habitat__step", ...]
  }
]

7. Manifest Protocol (on the server)

Every server exposes these endpoints:

If the server hosts an env panel, it also exposes the env panel bridge routes — see Section 8.1.

7.1 Wire type serialization

Same wire types as the canvas (see Wire Types). Transport is msgpack (serialization.py · pack_body / unpack_body): the whole {"inputs", "config"} body is packed in one shot, and the binary types JSON can't hold — ndarray, torch.Tensor, PIL.Image — ride a single blob ExtType as raw bytes (no base64), while bytes use msgpack's native bin type. Decoding is type-driven and degrades when the receiving env lacks a type (a torch/PIL blob decoded in an env without torch/PIL comes back as ndarray rather than crashing — cross-boundary = cross-env, so rebuilding a wrapper type is not free). /call negotiates by Content-Type and still accepts the legacy JSON encoding (__ndarray__ base64 markers) during the migration window. (Before 2026-06-15 this path was JSON+base64 only, so opaque non-JSON-safe values 500'd — see roadmap #67.)

7.2 Dynamic Fire-List over the wire

A server-side DynamicFireListNode emits a variable number of child firings, not a fixed output dict. To carry that across HTTP, the server serializes the result as a single envelope {"__firelist__": {...}} (server_app.py · call_function); the proxy's forward detects the marker and unwraps it back into a fire-list (proxy.py · forward). This keeps the dynamic-fan-out node type working identically in local and server modes — the executor sees the same fire-list either way. No node-author action is required; the envelope is internal to the serialization boundary.

7.3 Reverse channel: log/error push

A subprocess is otherwise write-only over /call, so its logs and handler errors never reached the canvas — a handler exception was demoted to a swallowed {"error": ...} value by the proxy. The subprocess now POSTs structured events back to the executor's POST /api/internal/events (server/event_push.py buffers + flushes at call boundaries; api/execution/internal_events.py receives), which republishes each on the ErrorBus → error_event WebSocket frame, so server-node logs/errors surface on the canvas like local-node ones. Handler exceptions are first-classed (pushed with node/execution scope + traceback) instead of being swallowed; a WARNING+ logging bridge forwards general subprocess logs. The executor base URL is handed to each subprocess via AGENTCANVAS_EXECUTOR_URL (resolved from Settings); the channel is a no-op when unset. See roadmap #54.

8. ServerApp (building the service)

Server authors subclass ServerApp to build the HTTP service:

from app.server import ServerApp, ServerFunction, PortSchema

class MySLAMApp(ServerApp):
    name = "slam"
    port = 9001

    def get_functions(self):
        return [
            ServerFunction(
                name="localize",
                input_ports=[PortSchema("rgb", "IMAGE"), PortSchema("depth", "DEPTH")],
                output_ports=[PortSchema("position", "STATE")],
                handler=self.localize,
            ),
        ]

    async def localize(self, inputs: dict, config: dict) -> dict:
        rgb = inputs["rgb"]   # np.ndarray, auto-deserialized
        return {"position": {"position": [1, 0, 2], "orientation": [0, 0, 0, 1]}}

if __name__ == "__main__":
    MySLAMApp().serve()

ServerApp._build_app() registers get_functions() as POST /call/{fn} routes and unconditionally mounts GET /manifest and GET /health. It then calls get_env_panel_instance() — if the result is non-None the env panel bridge routes are also mounted (see below).

8.1 Env panel bridge routes (ADR-server-002)

When a server hosts a BaseEnvPanel, _build_app conditionally mounts five additional routes:

These routes are only mounted when get_env_panel_instance() returns non-None. For AutoServerApp, the env panel is instantiated in __init__ from nodeset_cls.env_panel (so _build_app can see it at app-construction time, before on_startup runs). On the AgentCanvas side, _register_remote_env_panel fetches /env-panel/info and registers a RemoteEnvPanelProxy that forwards every BaseEnvPanel call over HTTP — making the canvas control panel work identically in local and server modes.

9. Habitat Example

Two classes in app/server/examples/habitat_server.py:

9.1 HabitatApp (ServerApp) — the actual service

Wraps HabitatEnvManager methods as server functions:

Run directly: python -m app.server.examples.habitat_server --port 9100

9.2 HabitatServer (BaseServer) — the process manager

Defines Habitat-specific launch properties:

class HabitatServer(BaseServer):
    name = "habitat"
    port = 9100
    startup_timeout = 60   # Habitat init loads 3D scenes — slow
    command = "python -m app.server.examples.habitat_server --port 9100"
    working_dir = ".../agentcanvas/backend"

Used by WorkspaceComponentRegistry when YAML has managed: true.

10. Auto-Generated Servers (ADR-server-001)

Instead of manually writing a ServerApp subclass, any BaseNodeSet can be auto-hosted in server mode. AutoServerApp is the canonical path — it introspects the nodeset's nodes and auto-generates ServerFunction entries from their input_ports/output_ports. It is also responsible for instantiating and hosting the nodeset's BaseEnvPanel (if declared) via the env panel bridge.

10.1 How It Works

1. BaseNodeSet.get_tools() yields the BaseCanvasNode instances with their input_ports/output_ports.
2. AutoServerApp converts each PortDef → PortSchema automatically and serves the standard /manifest, /call/{fn}, /health protocol.
3. If nodeset_cls.env_panel is set, it is instantiated in __init__ and the /env-panel/* routes are mounted via ServerApp._build_app.

AutoServerApp.on_startup initialises the nodeset (await self._nodeset.initialize()) and starts the BatchedInferenceServer. on_shutdown drains the batched server then shuts down the nodeset.

10.2 CLI Usage

# By dotted module path (preferred for package nodesets with __init__.py):
python -m app.server.auto_host \
    --module workspace.nodesets.model.model_sam \
    --class SamNodeSet \
    --port 9200 [--host 0.0.0.0]

# By file path (single-file nodesets):
python -m app.server.auto_host \
    --file /path/to/workspace/nodesets/model/model_sam.py \
    --class SamNodeSet \
    --port 9200

The registry auto-selects --module for package nodesets (those exposed via an __init__.py) and --file for single-file ones.

10.3 Registry Integration

The WorkspaceComponentRegistry launches auto-hosted servers transparently via _load_nodeset_as_server (e.g. POST /api/components/nodesets/sam/load?mode=server):

1. Finds the nodeset class and source file via inspect.getfile().
2. Picks a free port.
3. Spawns python -m app.server.auto_host --file … --class … --port ….
4. Waits for /health, fetches /manifest.
5. Generates proxy nodes — identical to YAML-based servers.
6. If the nodeset has an env panel, calls _register_remote_env_panel to fetch /env-panel/info and register a RemoteEnvPanelProxy locally.

When to use manual ServerApp instead: non-Python servers, custom protocols, or servers that need logic beyond what forward() provides.

There is also an ephemeral spawn path — load_nodeset_ephemeral / unload_nodeset_ephemeral (registry.py) — which starts a tagged auto-server without registering proxy node classes in the local NODE_HANDLERS. The subprocess-side eval bootstrap uses it: the worker already has the manifest baked in and only needs the live process, not a fresh round of proxy-class generation.

10.4 Auto-host ClassVar knobs

Two nodeset ClassVars steer how the registry spawns the server subprocess:

11. Worker-Pool Multi-Server (ADR-eval-002 PB)

For parallel evaluation, the registry can spawn N independent subprocesses for the same nodeset, each on its own port. This is controlled by the worker_count parameter on load_nodeset and ensure_nodesets_for_graph.

# Spawn 4 parallel env workers (e.g. for batched eval)
await registry.load_nodeset("env_habitat", worker_count=4)

worker_count > 1 forces mode="server" regardless of the request — fan-out only makes sense for isolated subprocesses.

11.0 Parallelism contract gate (ADR-server-003)

Before reading the fan-out mechanics, check the nodeset's parallelism ClassVar. Whether worker_count > 1 actually spawns N subprocesses depends on it:

class BaseNodeSet(ABC):
    ...
    parallelism: ClassVar[Literal["shared", "replicated"]] = "shared"     # bases.py · BaseNodeSet.parallelism

Picking the wrong mode is a class of bug worth knowing about: marking an env-shaped nodeset shared serialises every worker through one simulator (no parallel speedup, episode state collides); marking a model-shaped nodeset replicated spawns N copies of a multi-GB VRAM model (OOM at any non-trivial worker_count). When in doubt: env → replicated, model/inference → shared. §11.1 onwards describes the replicated path; shared takes the §4 singleton lifecycle.

11.1 Spawn and port allocation

All N ports are allocated up front (via _find_free_port) before any subprocess starts, to minimise bind-race collisions. For each worker k:

• A BaseServer is constructed with label auto_{name}#{k} and stored under key {name}#{k} in _auto_servers.
• server.start() spawns the subprocess; server.fetch_manifest() validates the manifest.
• Proxy node classes are generated once from worker 0's manifest (all N subprocesses serve identical manifests). Worker 0's URL is baked into each proxy closure as the default; per-worker routing is added at call time (see PB-1.5).

11.2 Per-worker URL routing (PB-1.5)

Each proxy node's forward method (generated in proxy.py::_make_forward) consults ctx._executor.get_server_url(nodeset_name) at call time. When the eval runner sets a per-worker URL override (e.g. pointing worker 2 to http://localhost:9102), the proxy uses that URL instead of the baked-in default. With no override (canvas Play / single-worker eval) behaviour is bit-identical to pre-PB-1.5.

11.3 Tagged RemoteEnvPanelProxy

If the nodeset declares an env_panel, _register_remote_env_panel is called for each worker with tag=k. The resulting RemoteEnvPanelProxy gets name = f"{nodeset_name}#{k}" so each worker's canvas control panel can address its own subprocess via get_env_panel(f"{nodeset_name}#{k}").

Tagged registration raises on failure; the outer try/except block rolls back all already-started servers (see below).

11.4 Rollback on partial failure

If any worker fails to start or its RemoteEnvPanelProxy registration fails, the registry stops all already-started workers, removes their _auto_servers entries, and de-registers their proxy node types before re-raising the exception — leaving no half-initialised state.

12. Batched Inference (ADR-eval-002 PC)

For GPU-heavy nodes like policy_adapter_vla__predict, sending one sample per HTTP request wastes throughput. The batched inference tier rendezvous K concurrent requests inside the server subprocess, calls the underlying handler once with all K samples stacked, then scatters K result slices back to the K awaiting callers.

The rendezvous is server-side (inside the AutoServerApp subprocess) because that is where K concurrent proxy calls from K LoopRunner instances naturally converge. This avoids a separate inference-server subprocess — one AutoServerApp per nodeset, one subprocess, one /health.

12.1 BatchedInferenceServer

BatchedInferenceServer lives inside each AutoServerApp instance. It lazily creates one _BatchQueue per batch key. Handlers are registered eagerly at get_functions() time via _make_handler when type(tool).batched is True.

Key constants:

12.2 Batch key and flush rule

The batch key is (function_name, config_hash) where config_hash is an 8-char SHA-1 of the sorted JSON config. Two callers with equivalent configs (any key order) share a batch queue. Two policy nodes pointing at different checkpoints get separate queues.

The flush timer is a restart-on-each-submit timer: each new submission cancels and restarts the flush_timeout_ms countdown. A late caller joins the in-flight batch instead of forcing a partial flush early — keeping the rendezvous correct even when workers arrive at uneven times.

12.3 Node opt-in

Two ClassVar fields on BaseCanvasNode control batching:

class PolicyCMAForward(BaseCanvasNode):
    batched: ClassVar[bool] = True
    batch_dim: ClassVar[str] = ""   # name of the input port carrying the batch axis

batched = False (default) — _make_handler creates a plain async handler that calls tool.forward once per request. batched = True — the handler submits to the shared BatchedInferenceServer and awaits its slice. register_node validates the combination at scan time.

The retired policy_cma__forward was the first node to opt in; today's user is policy_adapter_vla__predict.

12.4 Handler contract

A batched node's forward receives:

inputs = {SAMPLES_KEY: [inputs_dict_0, inputs_dict_1, ..., inputs_dict_K-1]}

and must return:

{OUTPUTS_KEY: [outputs_dict_0, outputs_dict_1, ..., outputs_dict_K-1]}

The list lengths must match. The node owns per-port stacking semantics (e.g. torch tensors stacked along batch_dim, lists of dicts gathered as lists). RNN hidden states must travel as explicit hidden_in/hidden_out ports so they ride the wire and remain with the caller across steps — the server holds no per-worker state.

13. File Reference