Server Mode
BaseServer, ServerApp, manifest, lifecycle
Server mode lets an external system โ one that needs a different Python env, a non-Python runtime, or a separate machine โ appear on the canvas as ordinary nodes. The system runs as a standalone HTTP service that exposes typed functions through a /manifest; AgentCanvas launches it, reads the manifest, and auto-generates proxy nodes that forward each firing over the wire. From the executor's side a proxy node is indistinguishable from a local one. This page owns the server classes, their lifecycle, and the parallelism gate; the nodeset that gets auto-hosted is NodeSets, and the control panel it may carry is Env Panels.
The two sides โ one class per process.
1. What server mode is, and when you need it
Reach for server mode whenever a capability can't live inside the 3.11 backend process. Four drivers recur:
- Dependency isolation โ Habitat-Sim needs Python 3.8 + CUDA + GL; hosting it in-process would drag all of that into the web server.
- Language โ a C++ SLAM stack or a Rust planner can't be a Python node at all.
- Deployment โ run the simulator on a GPU box and the dashboard on a laptop.
- Resource isolation โ keep several GPU-heavy systems from contending in one process.
The common thread: move the system to its own process and talk to it over HTTP with typed wire data. If a capability runs fine in the backend env, it does not need a server โ a plain local nodeset is simpler and lower-latency.
2. The smallest server
You rarely write a server by hand. The canonical path is to let a nodeset auto-host itself โ set one ClassVar and the registry spawns the subprocess, reads the manifest, and generates the proxy nodes for you:
class HabitatNodeSet(BaseNodeSet): name = "env_habitat" server_python = "python" # โ that's the whole trigger # get_tools() returns the observe / step / evaluate nodes as usual
Server mode has two launch vehicles: Native Launch โ the default โ spawns a host subprocess on the interpreter named by server_python (usually a conda env; sys.executable when unset), while Container Launch runs the same auto_host inside docker run for systems that cannot be a conda env at all (C++ stacks, GPL isolation) โ same protocol, same downstream; see Container Launch.
Load it and env_habitat__observe, env_habitat__step, โฆ appear in the catalog as proxy nodes, backed by a subprocess running that interpreter. Every server the framework runs is built this way, by the Launcher from the NodeSet's ClassVars (ยง5); a server launched by another process is attached by URL (ยง9). The auto-host machinery is ยง10; ServerApp, the base the auto-host builds on, is ยง8.
3. The two sides at a glance
A server-mode nodeset runs as two cooperating classes, one in each process:
| Class | Side | Responsibility |
|---|---|---|
| ServerApp | Server process | Builds a FastAPI service from get_functions(). Handles serialization. Optionally mounts /env-panel/* routes when get_env_panel_instance() returns a non-None env panel. Server authors subclass this. |
| AutoServerApp | Server process | Subclass of ServerApp that auto-generates ServerFunction entries by introspecting a BaseNodeSet's nodes. Canonical path for all auto-hosted nodesets. Also instantiates the nodeset's BaseEnvPanel (if declared) and hosts it via the env panel bridge. |
| BaseServer | AgentCanvas process | Launches the subprocess, tracks the PID, polls /health, fetches /manifest. The framework uses this. |
4. Lifecycle
4.1 Server States
A /health OK while connected keeps it connected (recovery is silent); stop() returns any state to stopped.
4.2 How start() works
subprocess.Popen(command, shell=isinstance(command, str))โ a list command (preferred) spawns with no shell, soPR_SET_PDEATHSIGbinds directly to the python child (ยง4.4); a str command keeps the legacy/bin/sh -cpath. Records the PID and starts a new session (os.setsid)._wait_for_health()โ pollsGET /health, interval 0.5 s backing off to 3.0 s. Liveness-driven: a dead subprocess fast-fails immediately, andstartup_timeoutis only the ceiling for a pathological hang.- On first healthy probe,
server.connected = Trueandserver.status = "connected".
4.3 How stop() works
os.killpg(SIGTERM)โ signals the whole process group; waits up to 5 s for a graceful exit.os.killpg(SIGKILL)โ if anything is still alive after the grace window.- Resets state:
server.connected = False,server.pid = None,server.status = "stopped".
4.4 Crash-safe cleanup
stop() only runs when the framework is in control โ graceful shutdown, REST /stop, normal exit. If the backend dies a way that bypasses cleanup hooks โ kill -9, OOM-kill, segfault, kernel panic, force-quit before stop() is called โ every auto_host subprocess becomes an init-orphan (PPID=1) and keeps running with its GPU memory and ports still held, blocking the next backend start. This was a recurring failure mode for stateful nodesets (e.g. a 14 GiB Prismatic VLM left squatting on a 24 GiB GPU after a crashed eval).
The framework defends against this with Linux PR_SET_PDEATHSIG: each auto_host child arms SIGTERM as its parent-death signal at startup. The kernel then guarantees that when the immediate parent dies any way, this process receives SIGTERM within milliseconds โ no user-space code in the parent has to run.
Two arming points (belt-and-suspenders, since the spawn chain typically layers bash -c and a conda run wrapper between uvicorn and the python child, and tail-exec optimisations vary by shell):
| Where | What | Covers |
|---|---|---|
_preexec_setsid_pdeathsig() in base_server.py |
runs in the spawned child after fork, before exec โ combines os.setsid() with prctl(PR_SET_PDEATHSIG, SIGTERM). Survives exec() (cleared only for setuid binaries). |
the shell-wrapper layer; the prctl persists into the python child via the standard exec inheritance rule |
_arm_pdeathsig() in auto_host.py main() |
re-arms inside the python interpreter at module entry | reparenting after shell tail-exec; any spawn-chain shape that falls outside the preexec_fn's reach |
Either arming point catches the typical case; together they catch all spawn-chain shapes. End-to-end test (SIGKILL the backend with active auto_host children): all children die within ~300 ms, GPU memory released, ports freed.
Scope and limits
- Linux only. AgentCanvas only targets Linux anyway (habitat-sim is Linux-first; macOS/Windows aren't supported simulator targets), so the platform restriction has no practical cost.
- Does not protect against the
auto_hostprocess itself hanging in an uninterruptible D-state syscall, or against a nodeset import installing a custom SIGTERM handler that ignores the signal. Both are pathological and out of scope. - Does not retroactively protect orphans that existed before this mechanism was deployed โ they need a one-time manual cleanup.
The framework-side stop() (ยง4.3) remains the primary cleanup path; PDEATHSIG is the kernel-level backstop for the fraction of crash modes where stop() does not get to run.
5. Where a server comes from โ the Launcher
There is no file-based server registration. Every server-mode process the registry runs is built by Launcher.build (server/launcher.py) from the NodeSet's own ClassVars, and the framework attaches to it through the same BaseServer + AutoHostClient pair whichever vehicle launched it:
| Vehicle | Selected by | What runs |
|---|---|---|
| Native Launch (default) | server_python โ an interpreter path (sys.executable when unset) | python -m app.server.auto_host --module|--file โฆ --class โฆ --port โฆ as a direct child, PDEATHSIG-armed (ยง4.4) |
| Container Launch | server_image โ an image name (with server_image_gpu, server_mounts, server_container_python, server_container_user) | The same auto_host inside docker run โ see Container Launch (ADR-server-005) |
The remaining knobs โ server_env, the class-level startup_timeout (default 1800 s, sized for HF weight downloads) โ are ClassVars too (ยง10.4). The YAML-registered manual-server path (workspace/servers/*.yaml, managed / command, the /api/components/servers* endpoints, app/server/examples/) was deleted on 2026-10-17: no in-tree NodeSet used it, auto-hosting covers every Python case, and a foreign process is attached by URL instead (ยง9).
6. REST API (on AgentCanvas)
Server lifecycle is a NodeSet operation, mounted under /api/components (app/api/platform/components.py):
| Endpoint | Method | Purpose |
|---|---|---|
/api/components/nodesets | GET | Discovered NodeSets with their loaded status |
/api/components/nodesets/{name}/load?mode=server | POST | Launch the NodeSet as a server-mode subprocess (default mode=local); a loaded NodeSet is reloaded โ this is also the Manager page's restart |
/api/components/nodesets/{name}/unload | POST | Stop the subprocess, deregister its proxy nodes and env panel |
/api/components/nodesets/ensure | POST | Load every NodeSet a list of node types needs (server mode auto-selected per NodeSet) |
7. Manifest Protocol (on the server)
Every server exposes these endpoints:
| Endpoint | Method | Purpose |
|---|---|---|
/manifest |
GET | Returns function schemas with typed ports |
/call/{function_name} |
POST | Invoke function: {inputs, config} โ {outputs} |
/health |
GET | Liveness check: {status: "ok", name, version} |
If the server hosts an env panel, it also exposes the env panel bridge routes โ see Section 8.1.
7.1 Wire type serialization
Same wire types as the canvas (see Wire Types). Transport is msgpack (serialization.py ยท pack_body / unpack_body): the whole {"inputs", "config"} body is packed in one shot, and the binary types JSON can't hold โ ndarray, torch.Tensor, PIL.Image โ ride a single blob ExtType as raw bytes (no base64), while bytes use msgpack's native bin type. Decoding is type-driven and degrades when the receiving env lacks a type (a torch/PIL blob decoded in an env without torch/PIL comes back as ndarray rather than crashing โ cross-boundary = cross-env, so rebuilding a wrapper type is not free). /call negotiates by Content-Type and still accepts the legacy JSON encoding (__ndarray__ base64 markers) during the migration window. (Before 2026-06-15 this path was JSON+base64 only, so opaque non-JSON-safe values 500'd โ see roadmap #67.)
7.2 Dynamic Fire-List over the wire
A server-side DynamicFireListNode emits a variable number of child firings, not a fixed output dict. To carry that across HTTP, the server serializes the result as a single envelope {"__firelist__": {...}} (server_app.py ยท call_function); the proxy's forward detects the marker and unwraps it back into a fire-list (proxy.py ยท forward). This keeps the dynamic-fan-out node type working identically in local and server modes โ the executor sees the same fire-list either way. No node-author action is required; the envelope is internal to the serialization boundary.
7.3 Reverse channel: log/error push
A subprocess is otherwise write-only over /call, so its logs and handler errors would never reach the canvas. Every process publishes to one EventSink interface (errors.py ยท get_bus()); a subprocess's adapter is the PushSink (server/event_push.py): it buffers ErrorEnvelopes and POSTs them to the parent's POST /api/internal/events (api/execution/internal_events.py), which republishes each on the parent's ErrorBus โ error_event WebSocket frame โ the same envelope a local-node failure uses, so the Report tab shows both alike. Producers are the ordinary ones: invoke.py publishes a handler exception with get_bus().from_exception(...) (node / execution scope + traceback) and flushes at the call boundary; the one LogBridgeHandler forwards WARNING+ records. An error envelope is flushed at once, the rest at the next /call or episode boundary or when the buffer fills; a failed push is dropped โ observability never breaks execution. install_push_sink(url, nodeset=โฆ) is called by auto_host (URL from AGENTCANVAS_EXECUTOR_URL, set by the registry) and by the eval subprocess (its --backend-url, flushing after every episode); without a URL the process keeps its in-process bus. See roadmap #54.
8. ServerApp (building the service)
Server authors subclass ServerApp to build the HTTP service:
from app.server import ServerApp, ServerFunction, PortSchema class MySLAMApp(ServerApp): name = "slam" port = 9001 def get_functions(self): return [ ServerFunction( name="localize", input_ports=[PortSchema("rgb", "IMAGE"), PortSchema("depth", "DEPTH")], output_ports=[PortSchema("position", "STATE")], handler=self.localize, ), ] async def localize(self, inputs: dict, config: dict) -> dict: rgb = inputs["rgb"] # np.ndarray, auto-deserialized return {"position": {"position": [1, 0, 2], "orientation": [0, 0, 0, 1]}} if __name__ == "__main__": MySLAMApp().serve()
ServerApp._build_app() registers get_functions() as POST /call/{fn} routes and unconditionally mounts GET /manifest and GET /health. It then calls get_env_panel_instance() โ if the result is non-None the env panel bridge routes are also mounted (see below).
8.1 Env panel bridge routes (ADR-server-002)
When a server hosts a BaseEnvPanel, _build_app conditionally mounts five additional routes:
| Endpoint | Method | Purpose |
|---|---|---|
/env-panel/info |
GET | Name, display_name, fields, actions |
/env-panel/state |
GET | Current state via panel.on_load() |
/env-panel/options/{field} |
GET | Dynamic option list via panel.get_options(field) |
/env-panel/field/{field} |
POST | Field-change callback: {value: ...} |
/env-panel/action/{action} |
POST | Action callback: {params: {...}} |
These routes are only mounted when get_env_panel_instance() returns non-None. For AutoServerApp, the env panel is instantiated in __init__ from nodeset_cls.env_panel (so _build_app can see it at app-construction time, before on_startup runs). On the AgentCanvas side, _register_remote_env_panel fetches /env-panel/info and registers a RemoteEnvPanelProxy that forwards every BaseEnvPanel call over HTTP โ making the canvas control panel work identically in local and server modes.
9. Attaching a running server โ register_remote_nodeset
A server another process already launched is attached by URL: registry.py ยท register_remote_nodeset(name, url) fetches its /manifest, generates the same proxy node classes (ยง10.3) pointed at that URL, and installs a _RemoteAutoServerShim whose stop() is a no-op, so the borrowed process outlives the borrower. The eval subprocess uses it for every shared singleton the parent backend has up (shared_urls.json โ Batch Eval ยง4), which is how a 14 GB model loads once and serves every run. A hand-written ServerApp (ยง8) that speaks /manifest + /call is attached the same way, programmatically; there is no REST route or file that registers one.
10. Auto-Generated Servers (ADR-server-001)
Instead of manually writing a ServerApp subclass, any BaseNodeSet can be auto-hosted in server mode. AutoServerApp is the canonical path โ it introspects the nodeset's nodes and auto-generates ServerFunction entries from their input_ports/output_ports. It is also responsible for instantiating and hosting the nodeset's BaseEnvPanel (if declared) via the env panel bridge.
10.1 How It Works
BaseNodeSet.get_tools()yields theBaseCanvasNodeinstances with theirinput_ports/output_ports.AutoServerAppconverts eachPortDefโPortSchemaautomatically and serves the standard/manifest,/call/{fn},/healthprotocol.- If
nodeset_cls.env_panelis set, it is instantiated in__init__and the/env-panel/*routes are mounted viaServerApp._build_app.
AutoServerApp.on_startup initialises the nodeset (await self._nodeset.initialize(), reusing the instance get_functions() built the handlers over) and builds its owned containers; on_shutdown shuts the nodeset down and clears them.
10.2 CLI Usage
# By dotted module path (preferred for package nodesets with __init__.py): python -m app.server.auto_host \ --module workspace.nodesets.model.model_sam \ --class SamNodeSet \ --port 9200 [--host 0.0.0.0] # By file path (single-file nodesets): python -m app.server.auto_host \ --file /path/to/workspace/nodesets/model/model_sam.py \ --class SamNodeSet \ --port 9200
The registry auto-selects --module for package nodesets (those exposed via an __init__.py) and --file for single-file ones.
10.3 Registry Integration
The WorkspaceComponentRegistry launches auto-hosted servers transparently via _load_nodeset_as_server (e.g. POST /api/components/nodesets/sam/load?mode=server):
- Finds the nodeset class and source file via
inspect.getfile(). - Picks a free port.
- Spawns
python -m app.server.auto_host --file โฆ --class โฆ --port โฆ. - Waits for
/health, fetches/manifest. - Generates proxy nodes under the same
node_types as local mode. - If the nodeset has an env panel, calls
_register_remote_env_panelto fetch/env-panel/infoand register aRemoteEnvPanelProxylocally.
A hand-written ServerApp (ยง8) is only for a service that cannot be a NodeSet at all; it is attached by URL (ยง9), never launched by the registry.
There is also an ephemeral spawn path โ load_nodeset_ephemeral / unload_nodeset_ephemeral (registry.py) โ which starts a tagged auto-server (through the same Launcher.build, so server_image nodesets launch as containers here too) without registering proxy node classes in the local node catalog. The subprocess-side eval bootstrap uses it: the worker already has the manifest baked in and only needs the live process, not a fresh round of proxy-class generation.
10.4 Auto-host ClassVar knobs
Two nodeset ClassVars steer how the registry spawns the server subprocess:
| ClassVar | Default | Effect |
|---|---|---|
server_python |
None (= sys.executable) |
Interpreter for the subprocess. Setting it non-None auto-routes the nodeset to server mode regardless of the requested mode (registry.py ยท load_nodeset) โ the parallel of the worker_count > 1 gate. Resolved from a named conda env (e.g. Habitat's Python 3.8 ac-vlnce env). |
server_env |
{} |
Extra environment variables merged into the subprocess env (e.g. an LD_PRELOAD shim). Read via getattr at spawn time (registry.py ยท _load_nodeset_as_server, registry.py ยท load_nodeset_ephemeral); not a declared ClassVar on BaseNodeSet. |
11. Worker-Pool Multi-Server (ADR-eval-002 PB)
For parallel evaluation, the registry can spawn N independent subprocesses for the same nodeset, each on its own port. This is controlled by the worker_count parameter on load_nodeset and ensure_nodesets_for_graph.
# Spawn 4 parallel env workers (e.g. for a worker-pool eval) await registry.load_nodeset("env_habitat", worker_count=4)
worker_count > 1 forces mode="server" regardless of the request โ fan-out only makes sense for isolated subprocesses.
11.0 Parallelism contract gate (ADR-server-003)
Before reading the fan-out mechanics, check the nodeset's parallelism ClassVar. Whether worker_count > 1 actually spawns N subprocesses depends on it:
class BaseNodeSet(ABC): ... parallelism: ClassVar[Literal["shared", "replicated"]] = "shared" # bases.py ยท BaseNodeSet.parallelism
| Mode | Subprocesses at worker_count=N | Env Panels | Typical occupants |
|---|---|---|---|
shared (default) |
1 (singleton โ fan-out ignored) | 1 untagged env panel ("env_X") |
Inference-shaped nodesets that don't own per-episode state โ policy_adapter_vlnce, policy_adapter_vla, voxposer, navgpt_mp3d_tools (BLIP-2/RCNN wrappers, lazy-loaded under threading.Lock). All workers share one subprocess |
replicated |
N (one per worker) | N tagged env panels ("env_X#0" โฆ "env_X#N-1") |
Env nodesets that own mutable per-episode state โ env_habitat, env_matterport3d, env_hmeqa, env_simpler, env_libero. Routed via the Fleet's WorkerHandle (env_panel_overrides + server_url_overrides) |
Picking the wrong mode is a class of bug worth knowing about: marking an env-shaped nodeset shared serialises every worker through one simulator (no parallel speedup, episode state collides); marking a model-shaped nodeset replicated spawns N copies of a multi-GB VRAM model (OOM at any non-trivial worker_count). When in doubt: env โ replicated, model/inference โ shared. ยง11.1 onwards describes the replicated path; shared takes the ยง4 singleton lifecycle.
11.1 Spawn and port allocation
All N ports are allocated up front (via _find_free_port) before any subprocess starts, to minimise bind-race collisions. For each worker k:
- A
BaseServeris constructed with labelauto_{name}#{k}and stored under key{name}#{k}in_auto_servers. server.start()spawns the subprocess;server.fetch_manifest()validates the manifest.- Proxy node classes are generated once from worker 0's manifest (all N subprocesses serve identical manifests). Worker 0's URL is baked into each proxy closure as the default; per-worker routing is added at call time (see PB-1.5).
11.2 Per-worker URL routing (PB-1.5)
Each proxy node's forward method (generated in proxy.py::_make_forward) consults ctx._executor.get_server_url(nodeset_name) at call time and builds its AutoHostClient (server/client.py) against the resolved URL. When the eval runner sets a per-worker URL override (e.g. pointing worker 2 to http://localhost:9102), the proxy uses that URL instead of the baked-in default. With no override (canvas Play / single-worker eval) behaviour is bit-identical to pre-PB-1.5.
11.3 Tagged RemoteEnvPanelProxy
If the nodeset declares an env_panel, _register_remote_env_panel is called for each worker with tag=k. The resulting RemoteEnvPanelProxy gets name = f"{nodeset_name}#{k}" so each worker's canvas control panel can address its own subprocess via get_env_panel(f"{nodeset_name}#{k}").
Tagged registration raises on failure; the outer try/except block rolls back all already-started servers (see below).
11.4 Rollback on partial failure
If any worker fails to start or its RemoteEnvPanelProxy registration fails, the registry stops all already-started workers, removes their _auto_servers entries, and de-registers their proxy node types before re-raising the exception โ leaving no half-initialised state.
12. Serialized calls โ one inference lock
K eval workers calling a parallelism="shared" nodeset converge on one AutoServerApp subprocess, and a GPU node such as policy_adapter_vla__predict must not run two forwards on that device at once. The app holds one asyncio.Lock for exactly that (auto_server_app.py ยท _get_inference_lock, created lazily inside the server's running loop โ the Python 3.8 envs that host some nodesets bind a Lock to the loop current at construction time).
12.1 Node opt-in
class PredictTool(BaseCanvasNode): serialized: ClassVar[bool] = True # one forward in flight on this subprocess's device
serialized = False (default) โ _make_handler returns a plain async handler and the node's calls run concurrently. serialized = True โ the handler is wrapped in async with inference_lock(); _make_handler raises at get_functions() time if no lock provider was given. In local mode the flag has no effect.
12.2 Handler contract
The server is pure-functional: it holds no per-caller state between calls, so RNN hidden states travel as explicit hidden_in / hidden_out ports and remain with the caller across steps. Today's user is policy_adapter_vla__predict (one model_batch per call).
12.3 What this replaced
Until 2026-10-17 this section was the batched-inference rendezvous of ADR-eval-002 PC (BatchedInferenceServer, one _BatchQueue per (function_name, config_hash), a 50 ms restart-on-submit flush timer, batched / batch_dim on the node, SAMPLES_KEY / OUTPUTS_KEY stacking). Its one consumer unrolled every batch sequentially, so the lock was all it provided; the tier was deleted and the lock kept.
13. File Reference
| File | Side | Role |
|---|---|---|
app/server/server_app.py |
Server | ServerApp base class + FastAPI route builder (manifest, call, health, env panel bridge) |
app/server/auto_server_app.py |
Server | AutoServerApp โ canonical auto-host for any BaseNodeSet; holds the one inference lock for serialized nodes |
app/server/base_server.py |
Framework | BaseServer process launcher + monitor |
app/server/manifest.py |
Both | PortSchema, FunctionSchema, ServerManifest |
app/server/serialization.py |
Both | Wire type HTTP serialization (standalone) |
app/server/proxy.py |
Framework | create_proxy_node(), generate_proxy_nodes(), per-worker URL routing (PB-1.5) |
app/server/client.py |
Framework | AutoHostClient โ the one HTTP client every framework-side caller uses against an auto_host (call / panel / manifest / health / evict / container ops); CallError / ServerUnreachable |
app/server/invoke.py |
Server | invoke โ the one server-side function call behind POST /call and the MCP tools/call: decode, run the handler, capture its log + usage, encode |
app/server/event_push.py |
Server | PushSink โ the subprocess EventSink adapter; install_push_sink(url, nodeset=, log_level=) (ยง7.3) |
app/components/registry.py |
Framework | _load_nodeset_as_server(worker_count=N), _register_remote_env_panel, register_remote_nodeset(name, url) |
app/api/platform/components.py |
Framework | REST endpoints for NodeSet load (?mode=server) / unload / ensure (mounted at /api/components) |
app/server/_loopback_proxy.py |
Framework | loopback_httpx_kwargs() โ bypasses any ambient HTTP proxy for localhost server traffic (health, manifest, /call, env-panel) |
app/server/auto_host.py |
Framework | CLI entry point for auto-hosted servers (--module/--file, --class, --port, --host) |