AgentCanvas / Pages / Developer Guide / Design Docs / Operations / Server Mode
2026-10-17

Server mode lets an external system โ€” one that needs a different Python env, a non-Python runtime, or a separate machine โ€” appear on the canvas as ordinary nodes. The system runs as a standalone HTTP service that exposes typed functions through a /manifest; AgentCanvas launches it, reads the manifest, and auto-generates proxy nodes that forward each firing over the wire. From the executor's side a proxy node is indistinguishable from a local one. This page owns the server classes, their lifecycle, and the parallelism gate; the nodeset that gets auto-hosted is NodeSets, and the control panel it may carry is Env Panels.

The two sides โ€” one class per process.

Server process (any language) ServerApp builds the FastAPI service /manifest ยท /call ยท /health ยท /env-panel AgentCanvas process BaseServer launches & monitors the subprocess Popen ยท /health poll ยท proxy-node gen start() HTTP

1. What server mode is, and when you need it

Reach for server mode whenever a capability can't live inside the 3.11 backend process. Four drivers recur:

The common thread: move the system to its own process and talk to it over HTTP with typed wire data. If a capability runs fine in the backend env, it does not need a server โ€” a plain local nodeset is simpler and lower-latency.

2. The smallest server

You rarely write a server by hand. The canonical path is to let a nodeset auto-host itself โ€” set one ClassVar and the registry spawns the subprocess, reads the manifest, and generates the proxy nodes for you:

class HabitatNodeSet(BaseNodeSet):
    name = "env_habitat"
    server_python = "python"  # โ† that's the whole trigger
    # get_tools() returns the observe / step / evaluate nodes as usual

Server mode has two launch vehicles: Native Launch โ€” the default โ€” spawns a host subprocess on the interpreter named by server_python (usually a conda env; sys.executable when unset), while Container Launch runs the same auto_host inside docker run for systems that cannot be a conda env at all (C++ stacks, GPL isolation) โ€” same protocol, same downstream; see Container Launch.

Load it and env_habitat__observe, env_habitat__step, โ€ฆ appear in the catalog as proxy nodes, backed by a subprocess running that interpreter. Every server the framework runs is built this way, by the Launcher from the NodeSet's ClassVars (ยง5); a server launched by another process is attached by URL (ยง9). The auto-host machinery is ยง10; ServerApp, the base the auto-host builds on, is ยง8.

3. The two sides at a glance

A server-mode nodeset runs as two cooperating classes, one in each process:

Class Side Responsibility
ServerApp Server process Builds a FastAPI service from get_functions(). Handles serialization. Optionally mounts /env-panel/* routes when get_env_panel_instance() returns a non-None env panel. Server authors subclass this.
AutoServerApp Server process Subclass of ServerApp that auto-generates ServerFunction entries by introspecting a BaseNodeSet's nodes. Canonical path for all auto-hosted nodesets. Also instantiates the nodeset's BaseEnvPanel (if declared) and hosts it via the env panel bridge.
BaseServer AgentCanvas process Launches the subprocess, tracks the PID, polls /health, fetches /manifest. The framework uses this.

4. Lifecycle

4.1 Server States

stopped starting connected unreachable start() /health OK /health fail timeout / process exits auto_restart โ†’ start()

A /health OK while connected keeps it connected (recovery is silent); stop() returns any state to stopped.

4.2 How start() works

  1. subprocess.Popen(command, shell=isinstance(command, str)) โ€” a list command (preferred) spawns with no shell, so PR_SET_PDEATHSIG binds directly to the python child (ยง4.4); a str command keeps the legacy /bin/sh -c path. Records the PID and starts a new session (os.setsid).
  2. _wait_for_health() โ€” polls GET /health, interval 0.5 s backing off to 3.0 s. Liveness-driven: a dead subprocess fast-fails immediately, and startup_timeout is only the ceiling for a pathological hang.
  3. On first healthy probe, server.connected = True and server.status = "connected".

4.3 How stop() works

  1. os.killpg(SIGTERM) โ€” signals the whole process group; waits up to 5 s for a graceful exit.
  2. os.killpg(SIGKILL) โ€” if anything is still alive after the grace window.
  3. Resets state: server.connected = False, server.pid = None, server.status = "stopped".

4.4 Crash-safe cleanup

stop() only runs when the framework is in control โ€” graceful shutdown, REST /stop, normal exit. If the backend dies a way that bypasses cleanup hooks โ€” kill -9, OOM-kill, segfault, kernel panic, force-quit before stop() is called โ€” every auto_host subprocess becomes an init-orphan (PPID=1) and keeps running with its GPU memory and ports still held, blocking the next backend start. This was a recurring failure mode for stateful nodesets (e.g. a 14 GiB Prismatic VLM left squatting on a 24 GiB GPU after a crashed eval).

The framework defends against this with Linux PR_SET_PDEATHSIG: each auto_host child arms SIGTERM as its parent-death signal at startup. The kernel then guarantees that when the immediate parent dies any way, this process receives SIGTERM within milliseconds โ€” no user-space code in the parent has to run.

Two arming points (belt-and-suspenders, since the spawn chain typically layers bash -c and a conda run wrapper between uvicorn and the python child, and tail-exec optimisations vary by shell):

Where What Covers
_preexec_setsid_pdeathsig() in base_server.py runs in the spawned child after fork, before exec โ€” combines os.setsid() with prctl(PR_SET_PDEATHSIG, SIGTERM). Survives exec() (cleared only for setuid binaries). the shell-wrapper layer; the prctl persists into the python child via the standard exec inheritance rule
_arm_pdeathsig() in auto_host.py main() re-arms inside the python interpreter at module entry reparenting after shell tail-exec; any spawn-chain shape that falls outside the preexec_fn's reach

Either arming point catches the typical case; together they catch all spawn-chain shapes. End-to-end test (SIGKILL the backend with active auto_host children): all children die within ~300 ms, GPU memory released, ports freed.

Scope and limits

The framework-side stop() (ยง4.3) remains the primary cleanup path; PDEATHSIG is the kernel-level backstop for the fraction of crash modes where stop() does not get to run.

5. Where a server comes from โ€” the Launcher

There is no file-based server registration. Every server-mode process the registry runs is built by Launcher.build (server/launcher.py) from the NodeSet's own ClassVars, and the framework attaches to it through the same BaseServer + AutoHostClient pair whichever vehicle launched it:

VehicleSelected byWhat runs
Native Launch (default)server_python โ€” an interpreter path (sys.executable when unset)python -m app.server.auto_host --module|--file โ€ฆ --class โ€ฆ --port โ€ฆ as a direct child, PDEATHSIG-armed (ยง4.4)
Container Launchserver_image โ€” an image name (with server_image_gpu, server_mounts, server_container_python, server_container_user)The same auto_host inside docker run โ€” see Container Launch (ADR-server-005)

The remaining knobs โ€” server_env, the class-level startup_timeout (default 1800 s, sized for HF weight downloads) โ€” are ClassVars too (ยง10.4). The YAML-registered manual-server path (workspace/servers/*.yaml, managed / command, the /api/components/servers* endpoints, app/server/examples/) was deleted on 2026-10-17: no in-tree NodeSet used it, auto-hosting covers every Python case, and a foreign process is attached by URL instead (ยง9).

6. REST API (on AgentCanvas)

Server lifecycle is a NodeSet operation, mounted under /api/components (app/api/platform/components.py):

EndpointMethodPurpose
/api/components/nodesetsGETDiscovered NodeSets with their loaded status
/api/components/nodesets/{name}/load?mode=serverPOSTLaunch the NodeSet as a server-mode subprocess (default mode=local); a loaded NodeSet is reloaded โ€” this is also the Manager page's restart
/api/components/nodesets/{name}/unloadPOSTStop the subprocess, deregister its proxy nodes and env panel
/api/components/nodesets/ensurePOSTLoad every NodeSet a list of node types needs (server mode auto-selected per NodeSet)

7. Manifest Protocol (on the server)

Every server exposes these endpoints:

Endpoint Method Purpose
/manifest GET Returns function schemas with typed ports
/call/{function_name} POST Invoke function: {inputs, config} โ†’ {outputs}
/health GET Liveness check: {status: "ok", name, version}

If the server hosts an env panel, it also exposes the env panel bridge routes โ€” see Section 8.1.

7.1 Wire type serialization

Same wire types as the canvas (see Wire Types). Transport is msgpack (serialization.py ยท pack_body / unpack_body): the whole {"inputs", "config"} body is packed in one shot, and the binary types JSON can't hold โ€” ndarray, torch.Tensor, PIL.Image โ€” ride a single blob ExtType as raw bytes (no base64), while bytes use msgpack's native bin type. Decoding is type-driven and degrades when the receiving env lacks a type (a torch/PIL blob decoded in an env without torch/PIL comes back as ndarray rather than crashing โ€” cross-boundary = cross-env, so rebuilding a wrapper type is not free). /call negotiates by Content-Type and still accepts the legacy JSON encoding (__ndarray__ base64 markers) during the migration window. (Before 2026-06-15 this path was JSON+base64 only, so opaque non-JSON-safe values 500'd โ€” see roadmap #67.)

7.2 Dynamic Fire-List over the wire

A server-side DynamicFireListNode emits a variable number of child firings, not a fixed output dict. To carry that across HTTP, the server serializes the result as a single envelope {"__firelist__": {...}} (server_app.py ยท call_function); the proxy's forward detects the marker and unwraps it back into a fire-list (proxy.py ยท forward). This keeps the dynamic-fan-out node type working identically in local and server modes โ€” the executor sees the same fire-list either way. No node-author action is required; the envelope is internal to the serialization boundary.

7.3 Reverse channel: log/error push

A subprocess is otherwise write-only over /call, so its logs and handler errors would never reach the canvas. Every process publishes to one EventSink interface (errors.py ยท get_bus()); a subprocess's adapter is the PushSink (server/event_push.py): it buffers ErrorEnvelopes and POSTs them to the parent's POST /api/internal/events (api/execution/internal_events.py), which republishes each on the parent's ErrorBus โ†’ error_event WebSocket frame โ€” the same envelope a local-node failure uses, so the Report tab shows both alike. Producers are the ordinary ones: invoke.py publishes a handler exception with get_bus().from_exception(...) (node / execution scope + traceback) and flushes at the call boundary; the one LogBridgeHandler forwards WARNING+ records. An error envelope is flushed at once, the rest at the next /call or episode boundary or when the buffer fills; a failed push is dropped โ€” observability never breaks execution. install_push_sink(url, nodeset=โ€ฆ) is called by auto_host (URL from AGENTCANVAS_EXECUTOR_URL, set by the registry) and by the eval subprocess (its --backend-url, flushing after every episode); without a URL the process keeps its in-process bus. See roadmap #54.

8. ServerApp (building the service)

Server authors subclass ServerApp to build the HTTP service:

from app.server import ServerApp, ServerFunction, PortSchema

class MySLAMApp(ServerApp):
    name = "slam"
    port = 9001

    def get_functions(self):
        return [
            ServerFunction(
                name="localize",
                input_ports=[PortSchema("rgb", "IMAGE"), PortSchema("depth", "DEPTH")],
                output_ports=[PortSchema("position", "STATE")],
                handler=self.localize,
            ),
        ]

    async def localize(self, inputs: dict, config: dict) -> dict:
        rgb = inputs["rgb"]   # np.ndarray, auto-deserialized
        return {"position": {"position": [1, 0, 2], "orientation": [0, 0, 0, 1]}}

if __name__ == "__main__":
    MySLAMApp().serve()

ServerApp._build_app() registers get_functions() as POST /call/{fn} routes and unconditionally mounts GET /manifest and GET /health. It then calls get_env_panel_instance() โ€” if the result is non-None the env panel bridge routes are also mounted (see below).

8.1 Env panel bridge routes (ADR-server-002)

When a server hosts a BaseEnvPanel, _build_app conditionally mounts five additional routes:

Endpoint Method Purpose
/env-panel/info GET Name, display_name, fields, actions
/env-panel/state GET Current state via panel.on_load()
/env-panel/options/{field} GET Dynamic option list via panel.get_options(field)
/env-panel/field/{field} POST Field-change callback: {value: ...}
/env-panel/action/{action} POST Action callback: {params: {...}}

These routes are only mounted when get_env_panel_instance() returns non-None. For AutoServerApp, the env panel is instantiated in __init__ from nodeset_cls.env_panel (so _build_app can see it at app-construction time, before on_startup runs). On the AgentCanvas side, _register_remote_env_panel fetches /env-panel/info and registers a RemoteEnvPanelProxy that forwards every BaseEnvPanel call over HTTP โ€” making the canvas control panel work identically in local and server modes.

9. Attaching a running server โ€” register_remote_nodeset

A server another process already launched is attached by URL: registry.py ยท register_remote_nodeset(name, url) fetches its /manifest, generates the same proxy node classes (ยง10.3) pointed at that URL, and installs a _RemoteAutoServerShim whose stop() is a no-op, so the borrowed process outlives the borrower. The eval subprocess uses it for every shared singleton the parent backend has up (shared_urls.json โ€” Batch Eval ยง4), which is how a 14 GB model loads once and serves every run. A hand-written ServerApp (ยง8) that speaks /manifest + /call is attached the same way, programmatically; there is no REST route or file that registers one.

10. Auto-Generated Servers (ADR-server-001)

Instead of manually writing a ServerApp subclass, any BaseNodeSet can be auto-hosted in server mode. AutoServerApp is the canonical path โ€” it introspects the nodeset's nodes and auto-generates ServerFunction entries from their input_ports/output_ports. It is also responsible for instantiating and hosting the nodeset's BaseEnvPanel (if declared) via the env panel bridge.

10.1 How It Works

  1. BaseNodeSet.get_tools() yields the BaseCanvasNode instances with their input_ports/output_ports.
  2. AutoServerApp converts each PortDef โ†’ PortSchema automatically and serves the standard /manifest, /call/{fn}, /health protocol.
  3. If nodeset_cls.env_panel is set, it is instantiated in __init__ and the /env-panel/* routes are mounted via ServerApp._build_app.

AutoServerApp.on_startup initialises the nodeset (await self._nodeset.initialize(), reusing the instance get_functions() built the handlers over) and builds its owned containers; on_shutdown shuts the nodeset down and clears them.

10.2 CLI Usage

# By dotted module path (preferred for package nodesets with __init__.py):
python -m app.server.auto_host \
    --module workspace.nodesets.model.model_sam \
    --class SamNodeSet \
    --port 9200 [--host 0.0.0.0]

# By file path (single-file nodesets):
python -m app.server.auto_host \
    --file /path/to/workspace/nodesets/model/model_sam.py \
    --class SamNodeSet \
    --port 9200

The registry auto-selects --module for package nodesets (those exposed via an __init__.py) and --file for single-file ones.

10.3 Registry Integration

The WorkspaceComponentRegistry launches auto-hosted servers transparently via _load_nodeset_as_server (e.g. POST /api/components/nodesets/sam/load?mode=server):

  1. Finds the nodeset class and source file via inspect.getfile().
  2. Picks a free port.
  3. Spawns python -m app.server.auto_host --file โ€ฆ --class โ€ฆ --port โ€ฆ.
  4. Waits for /health, fetches /manifest.
  5. Generates proxy nodes under the same node_types as local mode.
  6. If the nodeset has an env panel, calls _register_remote_env_panel to fetch /env-panel/info and register a RemoteEnvPanelProxy locally.

A hand-written ServerApp (ยง8) is only for a service that cannot be a NodeSet at all; it is attached by URL (ยง9), never launched by the registry.

There is also an ephemeral spawn path โ€” load_nodeset_ephemeral / unload_nodeset_ephemeral (registry.py) โ€” which starts a tagged auto-server (through the same Launcher.build, so server_image nodesets launch as containers here too) without registering proxy node classes in the local node catalog. The subprocess-side eval bootstrap uses it: the worker already has the manifest baked in and only needs the live process, not a fresh round of proxy-class generation.

10.4 Auto-host ClassVar knobs

Two nodeset ClassVars steer how the registry spawns the server subprocess:

ClassVar Default Effect
server_python None (= sys.executable) Interpreter for the subprocess. Setting it non-None auto-routes the nodeset to server mode regardless of the requested mode (registry.py ยท load_nodeset) โ€” the parallel of the worker_count > 1 gate. Resolved from a named conda env (e.g. Habitat's Python 3.8 ac-vlnce env).
server_env {} Extra environment variables merged into the subprocess env (e.g. an LD_PRELOAD shim). Read via getattr at spawn time (registry.py ยท _load_nodeset_as_server, registry.py ยท load_nodeset_ephemeral); not a declared ClassVar on BaseNodeSet.

11. Worker-Pool Multi-Server (ADR-eval-002 PB)

For parallel evaluation, the registry can spawn N independent subprocesses for the same nodeset, each on its own port. This is controlled by the worker_count parameter on load_nodeset and ensure_nodesets_for_graph.

# Spawn 4 parallel env workers (e.g. for a worker-pool eval)
await registry.load_nodeset("env_habitat", worker_count=4)

worker_count > 1 forces mode="server" regardless of the request โ€” fan-out only makes sense for isolated subprocesses.

11.0 Parallelism contract gate (ADR-server-003)

Before reading the fan-out mechanics, check the nodeset's parallelism ClassVar. Whether worker_count > 1 actually spawns N subprocesses depends on it:

class BaseNodeSet(ABC):
    ...
    parallelism: ClassVar[Literal["shared", "replicated"]] = "shared"     # bases.py ยท BaseNodeSet.parallelism
ModeSubprocesses at worker_count=NEnv PanelsTypical occupants
shared (default) 1 (singleton โ€” fan-out ignored) 1 untagged env panel ("env_X") Inference-shaped nodesets that don't own per-episode state โ€” policy_adapter_vlnce, policy_adapter_vla, voxposer, navgpt_mp3d_tools (BLIP-2/RCNN wrappers, lazy-loaded under threading.Lock). All workers share one subprocess
replicated N (one per worker) N tagged env panels ("env_X#0" โ€ฆ "env_X#N-1") Env nodesets that own mutable per-episode state โ€” env_habitat, env_matterport3d, env_hmeqa, env_simpler, env_libero. Routed via the Fleet's WorkerHandle (env_panel_overrides + server_url_overrides)

Picking the wrong mode is a class of bug worth knowing about: marking an env-shaped nodeset shared serialises every worker through one simulator (no parallel speedup, episode state collides); marking a model-shaped nodeset replicated spawns N copies of a multi-GB VRAM model (OOM at any non-trivial worker_count). When in doubt: env โ†’ replicated, model/inference โ†’ shared. ยง11.1 onwards describes the replicated path; shared takes the ยง4 singleton lifecycle.

11.1 Spawn and port allocation

All N ports are allocated up front (via _find_free_port) before any subprocess starts, to minimise bind-race collisions. For each worker k:

11.2 Per-worker URL routing (PB-1.5)

Each proxy node's forward method (generated in proxy.py::_make_forward) consults ctx._executor.get_server_url(nodeset_name) at call time and builds its AutoHostClient (server/client.py) against the resolved URL. When the eval runner sets a per-worker URL override (e.g. pointing worker 2 to http://localhost:9102), the proxy uses that URL instead of the baked-in default. With no override (canvas Play / single-worker eval) behaviour is bit-identical to pre-PB-1.5.

11.3 Tagged RemoteEnvPanelProxy

If the nodeset declares an env_panel, _register_remote_env_panel is called for each worker with tag=k. The resulting RemoteEnvPanelProxy gets name = f"{nodeset_name}#{k}" so each worker's canvas control panel can address its own subprocess via get_env_panel(f"{nodeset_name}#{k}").

Tagged registration raises on failure; the outer try/except block rolls back all already-started servers (see below).

11.4 Rollback on partial failure

If any worker fails to start or its RemoteEnvPanelProxy registration fails, the registry stops all already-started workers, removes their _auto_servers entries, and de-registers their proxy node types before re-raising the exception โ€” leaving no half-initialised state.

12. Serialized calls โ€” one inference lock

K eval workers calling a parallelism="shared" nodeset converge on one AutoServerApp subprocess, and a GPU node such as policy_adapter_vla__predict must not run two forwards on that device at once. The app holds one asyncio.Lock for exactly that (auto_server_app.py ยท _get_inference_lock, created lazily inside the server's running loop โ€” the Python 3.8 envs that host some nodesets bind a Lock to the loop current at construction time).

12.1 Node opt-in

class PredictTool(BaseCanvasNode):
    serialized: ClassVar[bool] = True   # one forward in flight on this subprocess's device

serialized = False (default) โ€” _make_handler returns a plain async handler and the node's calls run concurrently. serialized = True โ€” the handler is wrapped in async with inference_lock(); _make_handler raises at get_functions() time if no lock provider was given. In local mode the flag has no effect.

12.2 Handler contract

The server is pure-functional: it holds no per-caller state between calls, so RNN hidden states travel as explicit hidden_in / hidden_out ports and remain with the caller across steps. Today's user is policy_adapter_vla__predict (one model_batch per call).

12.3 What this replaced

Until 2026-10-17 this section was the batched-inference rendezvous of ADR-eval-002 PC (BatchedInferenceServer, one _BatchQueue per (function_name, config_hash), a 50 ms restart-on-submit flush timer, batched / batch_dim on the node, SAMPLES_KEY / OUTPUTS_KEY stacking). Its one consumer unrolled every batch sequentially, so the lock was all it provided; the tier was deleted and the lock kept.

13. File Reference

File Side Role
app/server/server_app.py Server ServerApp base class + FastAPI route builder (manifest, call, health, env panel bridge)
app/server/auto_server_app.py Server AutoServerApp โ€” canonical auto-host for any BaseNodeSet; holds the one inference lock for serialized nodes
app/server/base_server.py Framework BaseServer process launcher + monitor
app/server/manifest.py Both PortSchema, FunctionSchema, ServerManifest
app/server/serialization.py Both Wire type HTTP serialization (standalone)
app/server/proxy.py Framework create_proxy_node(), generate_proxy_nodes(), per-worker URL routing (PB-1.5)
app/server/client.py Framework AutoHostClient โ€” the one HTTP client every framework-side caller uses against an auto_host (call / panel / manifest / health / evict / container ops); CallError / ServerUnreachable
app/server/invoke.py Server invoke โ€” the one server-side function call behind POST /call and the MCP tools/call: decode, run the handler, capture its log + usage, encode
app/server/event_push.py Server PushSink โ€” the subprocess EventSink adapter; install_push_sink(url, nodeset=, log_level=) (ยง7.3)
app/components/registry.py Framework _load_nodeset_as_server(worker_count=N), _register_remote_env_panel, register_remote_nodeset(name, url)
app/api/platform/components.py Framework REST endpoints for NodeSet load (?mode=server) / unload / ensure (mounted at /api/components)
app/server/_loopback_proxy.py Framework loopback_httpx_kwargs() โ€” bypasses any ambient HTTP proxy for localhost server traffic (health, manifest, /call, env-panel)
app/server/auto_host.py Framework CLI entry point for auto-hosted servers (--module/--file, --class, --port, --host)
AgentCanvas docs