ADR-server-003

Date

2026-04-24

Status

accepted

Field

server

Context

ADR-eval-002 ships one deployment shape for batch eval at worker_count>1: spawn N tagged subprocesses of the env nodeset (env_habitat#0, #1, …), each carrying its own process-local simulator state, and route every worker's set-episode/play + in-graph proxy calls into its own subprocess via controller_overrides + server_url_overrides. This is the only correct shape for Habitat-Sim (live GL context, scene state, episode iterator cannot cross JSON IPC — same class of constraint that blocked state-container integration for HM-EQA's TSDFPlanner, TODO #17). But it is not the only shape. On the policy side we already recognise a second shape: policy_cma__forward declares batched=True, K concurrent callers converge on one subprocess, and BatchedInferenceServer coalesces them into one stacked forward pass. The asymmetry is currently silent — env nodesets have no way to declare "I am pure-functional, deploy me as a singleton and batch K callers". Consequences: (1) any future pure-data or vectorisable env (MatterSim if re-wrapped, simple gym envs, teleport-only pose-lookup envs, data-indexed panoramic envs) forces us to spawn N Python interpreters + N scene loads for parallelism that could have been one process + K coalesced calls; (2) the extension contract lacks a single place where an env author says what their nodeset is. MatterSim itself has native batch support (setBatchSize(K) since v0.2) but our MP3D wrapper pins batch=1 and runs on a single-thread executor for GL affinity — so even though the underlying C++ library is batch-capable, our wrapper is stateful + thread-affine and must stay on the N-subprocess path. The contract has to accept both realities: some envs are stuck subprocess-per-worker regardless of library capability, others could legitimately be singletons if/when authored pure-functionally.

Decision

Add a single ClassVar on BaseNodeSet that every env nodeset declares:

class BaseNodeSet:
    # Deployment topology under eval worker_count>1. Default matches the only
    # shape that exists today — no behavioural change for current nodesets.
    parallelism: Literal["replicated", "shared"] = "replicated"

Two modes, two contracts:

1. replicated (default, today's behaviour). Nodeset holds live per-episode state the subprocess cannot forget between calls — simulator instance, loaded scan, GL context, episode iterator. ensure_nodesets_for_graph(worker_count=N) spawns N tagged copies; EnvWorkerPool.__aenter__ populates tagged controller_overrides + server_url_overrides per worker; each LoopRunner's executor routes to its own subprocess via those overrides. Current envs stay here: env_habitat, env_mp3d, hmeqa. Migration cost: zero.

2. shared. Nodeset is pure-functional — every tool call takes the full per-call state (episode_index, step_index, action, …) as explicit input ports; the subprocess holds no per-caller state between calls. The nodeset's step/observe tools additionally opt in with batched=True, batch_dim="<port name>", exactly like policy_cma__forward. ensure_nodesets_for_graph(worker_count=N) spawns one copy regardless of N. EnvWorkerPool leaves controller_overrides + server_url_overrides empty; the executor falls through to the global registry; K concurrent /call/{tool_name} requests converge inside the one subprocess; the already-hosted BatchedInferenceServer (ADR-eval-002 PC) coalesces them into a single underlying handler call keyed on (function_name, config_hash).

No new infrastructure. shared reuses the entire Phase-C batched-inference tier end-to-end. The registry's only new responsibility is reading parallelism and branching the spawn step.

Controller is orthogonal. Whether an env nodeset is replicated or shared, its BaseController (episode browser, set_episode/play from the UI) stays singleton and stateful — "the user is previewing episode 42" is inherently a single-user UI concept. For shared envs, set_episode is not called per-worker before running the graph; instead episode_index rides the wire as an input port on the step/observe tool, and the graph itself is responsible for encoding which episode each call is for. _run_one_episode's pre-flight on_field_change("episode_index", idx) → on_action("play") cascade becomes an internal no-op for shared envs (controller stays in whatever state the UI last left it, which nobody looks at during batch eval).

Invariant carried forward from ADR-eval-002: at worker_count=1, both modes must stay bit-identical to today's single-tenant path. replicated achieves this by construction (single copy, empty overrides). shared achieves it because _BatchQueue flushes after flush_timeout_ms with batch size 1 when only one caller submits — same forward pass count, just with a 50 ms extra wall-clock for the single-caller case. This is already true for policy_cma__forward today.

Alternatives

(a) Unify everything as shared and pin batch size to 1. Serialising Habitat's K callers through one subprocess makes worker_count>1 effectively =1 — parallelism gone. Rejected.

(b) Keep everything as replicated, forever. Forces N Python-interpreter + N scene-load overhead for envs that genuinely don't need per-process state. Wastes the entire BatchedInferenceServer machinery that already lives in every AutoServerApp. Rejected.

(c) Auto-detect parallelism mode from whether the nodeset declares any batched=True tools. Too implicit; an author writing their first batched=True tool shouldn't silently flip their deployment topology. Rejected — explicit declaration is cheap.

(d) Introduce a third mode threadsafe_singleton (one subprocess, K callers run concurrently in separate asyncio tasks without coalescing). No real consumer today, and shared with batch_size coalescing already dominates it (one stacked forward pass > K un-stacked forward passes on GPU). Rejected as over-design; if a real consumer appears (rare CPU-only lookup env), we re-open this ADR.

Rationale

The contract acknowledges a distinction that already exists in the codebase — stateful subprocess vs. pure-functional server — but was only recognised on the policy side. Making it an explicit ClassVar on BaseNodeSet (a) gives every env author exactly one decision to make, with a safe default, (b) reuses Phase-C infrastructure instead of inventing new routing, (c) preserves the worker_count=1 bit-identical invariant, (d) keeps the controller (singleton-by-nature) orthogonal to the graph-execution path (per-call). The split between "what the library can do" and "what our wrapper does today" is important: even batch-capable libraries like MatterSim land on replicated if our wrapper holds state — the contract follows the wrapper, not the upstream library.

Current consumer audit:

Scope of this ADR: we are not rewriting any existing wrapper to become pure-functional. The user has explicitly chosen to count on the upstream wrappers (MatterSim, habitat-sim, Prismatic) as-is. This ADR defines the contract so that (a) the default path is named and documented, not just implicit, and (b) the next env nodeset — whenever one is authored pure-functionally, most likely an F3-style parallelised agent or a future gym-style lookup env — lands on the already-built batched tier instead of forcing another N-subprocess spawn.

Migration path (no code landing in this ADR — consumer-driven):

1. Land the ClassVar on BaseNodeSet with replicated default. Zero-line change for every existing nodeset.
2. In WorkspaceComponentRegistry.ensure_nodesets_for_graph, branch on type(nodeset).parallelism: replicated → existing N-spawn path; shared → spawn 1 copy, skip the tagged overrides.
3. In EnvWorkerPool.__aenter__, skip the tagged lookup for shared env nodesets (handles carry empty overrides → executor falls through → single copy serves K callers).
4. In _run_one_episode, skip the on_field_change/on_action("play") pre-flight when the resolved env nodeset is shared (the graph carries episode_index as an input port instead).
5. Document in the env-nodeset tutorial: "if your env is stateful, leave the default; if your env is pure-functional, set parallelism = 'shared' and declare batched=True, batch_dim='...' on your step/observe tools".

Steps 1–4 are safe to land behind the default without any consumer migrating. Step 5 is the call-to-action for future env authors.

Affected docs

• docs-site/docs/developer-guide/core/decisions/server/index.md — add ADR-server-003 row
• docs-site/docs/developer-guide/core/glossary.md — new term parallelism under §3 Component System, sibling to batched / batch_dim
• docs-site/docs/developer-guide/core/decisions/eval/adr-eval-002-worker-pool-and-batched-inference.md — cross-reference note that per-worker subprocess fan-out is one of two shapes
• docs-site/docs/developer-guide/core/architecture.md §5 — add a short "Parallelism modes" subsection after §5.1 Auto-Hosted Deployment
• Env-nodeset tutorial (when written, TODO #41) — document the two modes