General Policy Adapter

env_adapter · policy_adapter_vla · policy_adapter_vlnce · CanonicalDict · four adapter stages + one model-forward · three-nodeset composition · filename discovery

AgentCanvas runs a neural policy not as one black-box node but as a short pipeline of adapter stages mediated by a CanonicalDict lingua franca. Since 2026-07-04 the pipeline is cut along that lingua franca into three nodesets: one general env_adapter (env-side stages, pure numpy, in-process) plus two per-domain policy adapters — policy_adapter_vla (manipulation: LIBERO / SIMPLER) and policy_adapter_vlnce (VLN-CE navigation: Habitat R2R-CE / RxR-CE), each hosting its models in its own conda env. Running a policy composes three nodesets: the env itself + the env adapter + one policy adapter (e.g. env_habitat + env_adapter + policy_adapter_vlnce). The hard promise: the pipeline shape never changes. An env-side adapter and a model-side adapter sit on either side of the model, each pipeline stage is exactly one canvas node, and adding a new policy or a new env is always "drop a .py file in a folder, pick it from a dropdown" — never a new node type, never a bypass path. This page is the single home for the paradigm and both instances (it absorbed the former VLA Policy NodeSet and CMA Policy pages); for the env contract see env_habitat.

1. What it does

A trained policy needs the env's raw observation reshaped into its input tensor, and its raw output reshaped back into the env's action. Rather than bury those two translations inside one opaque "run the policy" node, AgentCanvas splits each forward pass into four adapter stages bracketing one model-forward node, and exposes every stage as its own canvas node:

1. env → canonical — the env-side adapter wraps the raw observation into a CanonicalDict[obs].
2. canonical → model — the model-side adapter packs the canonical obs into the model's input batch.
3. predict — the GPU policy's forward pass (the only heavy node).
4. model → canonical — the model-side adapter unpacks the raw output into a CanonicalDict[action].
5. canonical → env — the env-side adapter turns the canonical action into an env action.

The two adapters never know about each other. The env-side adapter knows nothing about which model consumes its canonical; the model-side adapter knows nothing about which env produced it. They communicate only through the CanonicalDict + its info carrier. That decoupling is the whole point: any env × any model is a valid pair, and onboarding either side is a local edit that doesn't touch the other.

1.1 The three-nodeset cut (2026-07-04)

The nodeset boundary follows the adapter boundary. The env-side stages of both domains live in one general env_adapter nodeset (workspace/nodesets/env/env_adapter/); the model-side stages live in per-domain policy nodesets (workspace/nodesets/policy/policy_adapter_vla/, …/policy_adapter_vlnce/). Two facts force this exact cut:

• One nodeset binds one interpreter. server_python is a nodeset-level ClassVar (app/components/bases.py) — Pi0/SmolVLA/RT-1-X need the ac-vla-policy env while CMA/Seq2Seq need the Python-3.8 ac-vlnce env, so a single cross-domain policy nodeset cannot exist. The policy side therefore splits per model family.
• Env-side adapters are pure numpy. No torch, no habitat, no model imports (stage 1 reformats, stage 5 decodes) — so all of them, across both domains, share one nodeset that loads in-process in the hub env (server_python = None), with no GPU manager at all.

Running a policy composes three nodesets: env_habitat + env_adapter + policy_adapter_vlnce for VLN-CE; env_libero/env_simpler + env_adapter + policy_adapter_vla for manipulation. The cut also moves the env-side adapter selection next to the env it serves — the structural half of the old "robot dropdown should be env-locked" wart (§8).

A note on stage numbering. This page numbers the pipeline 1–5 with predict as stage 3, matching the shipped nodesets: env_adapter owns stages 1 & 5, the policy adapters own stages 2/3/4.

1.2 Mental model — standardize, then policy-process

The pipeline has one load-bearing division of labour: the env-side adapter only standardizes; every kind of processing happens after the waist, on the model side. Standardizing means reformatting the env's raw observation into one uniform shape without changing its content; processing means everything a specific policy needs — resize, normalize, augment, tokenize.

Six rules that fall out of this:

1. Decoupling. M policies + N sims → write only M+N adapters; any pair composes (no M×N glue).
2. Standardize ≠ process. env→canonical reformats only (content unchanged); canonical→model does resize / normalize / tokenize / augment.
3. Physical, de-tokenized, neutral. The canonical carries physical quantities; each model's tokenization / normalization / chunking lives in its own model-side adapter, never in the waist.
4. The env stays dumb. The simulator emits its raw output and knows nothing of the canonical; structure is imposed by the adapter.
5. info is a plain dict on the wire (msgpack-safe), never a frozen dataclass — enforced at the nodeset boundary in both domains since the split (§2).
6. Incompatibility is handled elsewhere. A modality a policy needs but a sim can't produce is not the canonical's concern.

Scope. The shipped code targets manipulation VLAs (OpenVLA / Pi / Diffusion Policy) on a single arm + gripper in joint-position or eef-pose space; the per-part embodiment registry for other morphologies (dual-arm, mobile base, WAM torque, VLN navigation) is parked until a real second morphology arrives. The VLN side follows the standardize/process split as of 2026-06-29: the env adapter standardizes (raw rgb / depth / raw instruction text), and tokenization + obs_transforms run in canonical_to_model (§8).

1.3 Smallest instance

The minimal pipeline is the five-node chain wired in series between an env's observe/step pair — the two outer nodes from env_adapter, the three inner from a policy adapter:

observe ─▶ [env_adapter] env→canonical ─▶ [policy] canonical→model ─▶ [policy] predict
                                                                              │
        step ◀─ [env_adapter] canonical→env ◀─ [policy] model→canonical ◀────┘

For VLN-CE the recurrent CMA/Seq2Seq policy adds two wires (predict.hidden_out → iter_out.hidden, iter_in.iterout_hidden → predict.hidden_in); the per-episode instruction text rides in as a raw string through an iter_in init slot (CMA-vocab tokenization happens later, model-side). The verified R2R-CE graph that runs this end-to-end is workspace/graphs/vln/verified/straightforward.json (15 nodes, 27 edges).

2. The CanonicalDict lingua franca

Every env maps to this format; every model maps from it. It is a plain {"data": …, "info": …} dict travelling on an ANY-typed wire — the contract is enforced by dict shape, not by wire_type. The two domains carry different payloads but the same envelope.

Since the split the canonical always crosses a process boundary (hub-resident env_adapter ↔ policy-side server subprocess), so info travels as a plain dict in both domains: the msgpack ANY-wire (app/server/serialization.py) raises on a raw dataclass. VLN's make_canonical_obs stores dataclasses.asdict(info) at construction; VLA's stage-1 node flattens the CanonicalInfo before returning, and the policy-side stage-2 node reconstructs the dataclass on receipt (VLA model adapters read it by attribute). The schema modules themselves (canonical.py) are duplicated per nodeset — only plain dicts cross the wire, so the copies stay runtime-independent; the dict shape is the contract.

2.1 VLA — manipulation payload

policy_adapter_vla/adapters/canonical.py (same schema copied at env_adapter/robots/canonical.py) · make_canonical_obs. Multi-camera images + a structured robot state + a language prompt; the info carrier (CanonicalInfo) records rotation representation and key dims so a model adapter can normalise without knowing the robot:

CanonicalDict[obs] = {
    "data": {
        "images":  {"front": ndarray | None, "wrist": ndarray | None},
        "state":   {"pos": ..., "rot": ..., "gripper": ..., "joint_position": ...},
        "actions": {"pos": ..., "rot": ..., "gripper": ..., "joint_position": ...},  # output side
        "prompt":  str,
    },
    # built as the CanonicalInfo dataclass; flattened to a plain dict at the
    # env_adapter node boundary, reconstructed by the policy-side stage 2
    "info": CanonicalInfo(state_type, state_rot_repr, action_type,
                          action_rot_repr, state_dims, action_dims),
}

2.2 VLN — navigation payload

policy_adapter_vlnce/adapters/canonical.py (same schema copied at env_adapter/envs/canonical.py) · make_canonical_obs. RGB-D frame + the raw instruction (raw text for R2R-CE, precomputed BERT feature for RxR-CE) + step metadata; the info carrier (CanonicalNavInfo) records env family, action cardinality, and instruction kind. The canonical carries standardized raw data only — tokenization is model-side (§1.2):

CanonicalDict[obs] = {
    "data": {
        "rgb":         ndarray uint8  (H, W, 3),
        "depth":       ndarray float  (H, W, 1),       # metres
        "instruction": {"kind": "raw_text", "text": str}                      # R2R-CE
                     | {"kind": "feat",     "embedding": ndarray (T, 768)},   # RxR-CE
        "step_meta":   {"step_index": int, "episode_id": str},
    },
    # asdict() so a frozen dataclass survives the msgpack ANY-wire
    "info": asdict(CanonicalNavInfo(env_family, action_dim, image_size,
                                    depth_max, instruction_kind)),
}

The canonical action mirrors the asymmetry: VLA's is a dict of continuous action components ({"data": {"actions": {pos, rot, gripper, …}}}) that decode to a (K, 7) chunk; VLN's is a single discrete index ({"data": {"action_index": int}} with action_dim riding on info).

3. The stages as canvas nodes

Each stage is one node backed by one abstract method on the env-side or model-side adapter. The node's prefix says which nodeset owns it:

Stage 4 carries no config in either domain — it reads the already-built model-adapter slot off the policy-side manager, so it auto-follows whatever the obs side selected. On the env side, the VLN stage-5 node is config-less (action index and cardinality both ride on the canonical_action itself), but the VLA stage-5 node carries the same robot select as stage 1: decode logic is robot-class-side (SimplerRobot.canonical_to_env ≠ LiberoRobot's) and delta-action config is robot config, not wire data — the two selects must match (§8).

4. The three nodesets and their runtimes

4.1 env_adapter — in-process, no manager

workspace/nodesets/env/env_adapter/. Four nodes (the VLA pair + the VLN pair), server_python = None — everything is numpy-only, so the nodeset loads in local mode inside the hub interpreter. There is no singleton manager: the old ensure_robot / ensure_env_adapter slices became module-level lazy caches (_ensure_robot / _ensure_env in env_adapter/__init__.py) — build the selected adapter from its file's DEFAULT_KWARGS on first use, cache by config key under a threading.Lock (adapters are immutable after construction, so read-sharing across the hub's K loop runners is safe). Dropdowns come from filename discovery over robots/ and envs/ (§5).

4.2 policy_adapter_vla — VLA model side

workspace/nodesets/policy/policy_adapter_vla/, server mode on ac-vla-policy (server_python ← $VLA_POLICY_PYTHON; install: scripts/install/install_ac_vla_policy.sh), parallelism = "shared": one subprocess hosts one VlaPolicyManager singleton and K eval workers fan in. The manager pins all torch/CUDA work to a single-thread executor and owns two idempotent ensure_* slices plus one cache:

• ensure_model_adaptor(model_module) — owned by stage 2's model select; module change drops the loaded policy (a Pi0 checkpoint can't load into a SmolVLA net).
• ensure_policy(policy_module, checkpoint_path, num_inference_steps) — owned by predict's config; builds the GPU policy and loads weights. First call is the expensive one (Pi0 does JAX→PT conversion + torch.compile; default_per_step_budget_sec = 60).
• _cached_info — the CanonicalInfo captured from the first canonical seen by stage 2; stage 4's model_to_canonical(model_output, info) reads it (the robot adapter that used to provide it lives across the process boundary now).

predict keeps batched=True (BatchedInferenceServer rendezvous; the TF SavedModel path pins batch=1, so the K-flush is unrolled — the win is one model on GPU, not throughput). RT-1's recurrent policy_state threads through the optional policy_state_in/out ports; Pi0/SmolVLA/DP/DROID-DP are stateless. The subprocess env sets TF_USE_LEGACY_KERAS=1, TF_FORCE_GPU_ALLOW_GROWTH=true, and prepends the conda env's lib/ to LD_LIBRARY_PATH.

4.3 policy_adapter_vlnce — VLN-CE model side

workspace/nodesets/policy/policy_adapter_vlnce/, server mode on ac-vlnce (Python 3.8, habitat-sim 0.1.7; server_python ← $VLNCE_PYTHON), parallelism = "shared", singleton VlnceManager. Its chain entry is stage 2 (adapt_canonical_to_model), which owns the single variant select and primes everything in one pass, in an order that preserves the pre-split device placement:

1. ensure_model_adaptor(spec.model_adaptor, spec.exp_config) — rebuild on variant flip; drops the policy when the net class or exp-config changes.
2. note_canonical(canonical) — cache the first-seen canonical (ensure_policy derives the CMA obs-space from it; stage 4 rebuilds CanonicalNavInfo(**info) from it).
3. ensure_policy(spec.policy, checkpoint_path) — strict checkpoint load, .to(device).eval().
4. canonical_to_model(canonical, hidden_in) — vlnce_baselines obs_transforms + CMA-vocab tokenization, tensors built on the (already-loaded) policy's device.

predict stays config-less and batched=False (§8). The recurrent hidden dict ({rnn_states, prev_actions, not_done_masks}) rides the hidden_in/out ports through the loop pivots.

Because each manager is a singleton and the adapter nodes are pure readers of its slots, the heavy work (checkpoint load, GPU placement) happens once and is amortised across every step and every worker.

5. Filename discovery — drop-in extension

No nodeset hard-codes its variant list. At import time each scans its folders and turns every .py stem (excluding __init__, base_*, _*, and per-folder excludes like canonical / dp_defaults) into a dropdown option:

Each file carries its own DEFAULT_KWARGS module constant (transcribed from the upstream Hydra / YAML config), so a variant is fully described by its file — no canvas-side JSON config. Drop a file in or out, then POST /api/components/reload. The scan is filename-only — no module imports happen at scan time, so heavy deps (torch, diffusers, transformers) stay out of the hub process until the user actually picks an option. Adding a policy is the same recipe in both domains: one model-side adapter file (both directions) + one policy file, consuming the existing CanonicalDict and emitting a canonical action the env-side adapter can already invert — the VLA recipe with the RT-1-X worked example is §7.4.

6. VLA vs VLN — the deltas

Everything above is shared. Here is exactly where the two instances diverge — the domain forces different payloads, action types, and recurrent-state shapes, and the two ports made different config-placement choices:

The single biggest structural difference is config placement. VLA treats robot, model, and policy as three independent choices (because any robot × any model is meant to be mixable). VLN bundles all four strings (model adapter, exp-config YAML, policy module, checkpoint) into one variant selection on its chain-entry node, because a VLN-CE baseline is a fixed (architecture, training-config, checkpoint) tuple — picking "CMA +PM+DA+Aug" must pick all four together or the checkpoint won't load. The variant dropdown is populated from variants.REGISTRY (12 R2R-CE entries; only CMA_PM_DA_Aug and Seq2Seq_DA have released checkpoints — the other 10 are listed for topological completeness with a "(not released by paper authors)" label). The old implicit env pin (r2rce for every variant) became the explicit env_family select on the env_adapter node when the nodesets split.

7. The VLA instance in depth

7.1 Design principle: env-side vs policy-side adapter coupling

The four adapter stages split into two halves with very different ownership rules — this split dictated the nodeset cut itself:

A graph's env nodes (env_libero__* / env_simpler__*) determine which RobotAdaptor is in play; the env-side nodes consume that robot, they don't independently choose it. Mixing (a LIBERO graph with simpler_robot) is structurally invalid — different CanonicalInfo, different state schema, different action contract. The policy side is the one thing the user genuinely picks, and it moves as a linked triple: the BasePolicy subclass, its canonical_to_model packing, and its model_to_canonical unpacking (one ModelAdaptor owns both directions). Stage 4 carries no selector — it reads the manager's _model_adaptor slot, so it auto-follows stage 2's choice; pairing model with the right policy on predict remains the user's responsibility.

Mental model. A VLA graph is a sandwich: the bread (env-side adapters) is fixed by which kitchen you're in (the env nodeset), and the filling (policy + 2 policy-side adapters) is the one thing the user picks — and they pick it as one item from the menu, not three independent ingredients.

UI state vs principle. The nodeset split moved the robot select onto env-side nodes that live beside the env — the structural half of env-locking. The remaining half is the interlock: the dropdown still lists every file under env_adapter/robots/, so picking an env-incompatible robot is possible and fails as an obs-shape mismatch deep in the model. End-state per the principle: the env nodeset advertises a compatible-robot list (env_libero → [libero_robot, libero_robot_absolute], env_simpler → [simpler_robot]) and the dropdown filters to it. Per-robot kwargs (use_delta_actions, delta_action_mask) are not canvas fields — they are DEFAULT_KWARGS module constants in each robot file; selecting a different file = selecting a different defaults preset.

7.2 Adapter class contracts

RobotAdaptor (env_adapter/robots/base_robot.py) — translates between an env's native obs/action format and CanonicalDict:

get_canonical_info()           → CanonicalInfo
dataset_to_canonical(data)     → CanonicalDict   # for training
env_to_canonical(env_obs)      → CanonicalDict   # for inference (no actions)
canonical_to_env(canonical_action, state=None) → dict   # action chunk for env
get_state_dim() / get_action_dim() / get_norm_stats_keys() / env_obs() / env_action() / datasets()

ModelAdaptor (policy_adapter_vla/adapters/models/base_model.py) — translates between CanonicalDict and model-specific input/output dicts, and owns normalisation (norm_stats JSON loaded at construction; mean/std/q01/q99/min/max per key):

canonical_to_model(canonical) → dict     # the batch fed into BasePolicy
model_to_canonical(model_output, info) → CanonicalDict
canonical_to_norm_stats_format(canonical) / get_norm_stats_mode() / get_norm_stats_keys()
model_input() / model_output() → dict    # shape descriptors

BasePolicy (policy_adapter_vla/policies/base_policy.py) — the actual GPU model: weights + predict_action() + load_checkpoint() + dtype/device, no canonical-format knowledge. Distinct from ModelAdaptor because two policies can share one adapter (DiffusionUnetHybridImagePolicy and DroidDiffusionPolicy both use DPModel) and because a policy backend need not be torch (RT-1-X is a TF SavedModel behind a dummy nn.Parameter).

The composed Adaptor is gone. Upstream vlaworkspace ships an Adaptor(robot, model) bridge object; the canvas path always called the atomic stage methods instead, and with robots and models now living in different nodesets (different processes) the composer had nothing left to compose — adapters/adaptor.py and the model_to_env fused alias were deleted in the 2026-07-04 split.

7.3 Currently supported model × robot matrix

Every cell that has both a robot file and a model file is a valid graph configuration. Cells marked ⚠ are structurally valid (the pipeline runs) but empirically fail because the checkpoint trained on robot A doesn't transfer to robot B's distribution. The shipped graphs are workspace/graphs/vla/unverified/vla_policy_libero.json and vla_policy_simpler.json.

7.4 Extension cookbook

Adding a model (e.g. OpenVLA) — drop-in pattern, no central registry edits:

1. Add a ModelAdaptor subclass at policy_adapter_vla/adapters/models/<name>_model.py — implement canonical_to_model, model_to_canonical, the norm-stats hooks, and append a DEFAULT_KWARGS: dict = {…} constant transcribed from the upstream config (Hydra-style _target_: dicts are materialized by _instantiate()).
2. Add a BasePolicy subclass at policy_adapter_vla/policies/<name>_policy.py — at least predict_action(batch) → dict, load_checkpoint(path), to(device), eval(); its own DEFAULT_KWARGS.
3. Vendored backbone under policy_adapter_vla/models/<name>/ (verbatim copy; AgentCanvas never imports upstream repos at runtime).
4. Hot-reload (POST /api/components/reload) — both files appear in the model / policy dropdowns immediately.

Adding a robot (e.g. Franka) — one RobotAdaptor subclass at env_adapter/robots/<name>_robot.py with its DEFAULT_KWARGS; hot-reload; it appears in the robot selects on both env-side VLA nodes. You never write env-side adapters when adding a policy, and never write model-side ones when adding a robot.

Worked example — RT-1-X (2026-05-04). The first VLA outside vlaworkspace's Pi0/SmolVLA/DP family: a Google TF SavedModel served via tf-agents, with its own image preprocessing, USE language embedding, and per-embodiment action rescaling. It folded in with zero edits to node code, manager logic, or graph topology:

What it proves: the "any robot × any model" product is real (Rt1Model doesn't know its robot; SimplerRobot doesn't know its model); BasePolicy = nn.Module doesn't force every VLA to be torch; additive dependency stacks coexist (TF 2.15 + tf-agents joined torch 2.6 + JAX 0.5.3 in ac-vla-policy). Octo's jax==0.4.20 pin is not additive — that's why it sits in the separate policy_octo nodeset as an explicit escape hatch until ported forward.

7.5 Open questions (maintainer)

1. Robot env-locking interlock. The nodeset split delivered the structural half (§7.1); the dropdown filter / env-swap confirmation modal is still open. New corollary since the split: the VLA stage-5 node carries its own robot select that must match stage 1's — an interlock should keep the two in sync automatically.
2. Norm-stats schema. _load_norm_stats_from_path expects mean/std/q01/q99 per key. Stats that don't fit (OpenVLA's 256-bin discretisation, RT-2's no-norm path) need a subclass override or a different loading path.
3. canonical_to_env contract on action chunks. Today emits {"actions": ndarray (K, 7)}, JSON-encoded. For non-7-DoF embodiments (mobile manipulators, dual-arm) is (K, 7) still the contract or does the wire type evolve?
4. Cross-env state vector. current_state is documented as 8-D LIBERO state at stage 5. Is there a canonical state schema across envs, or does each RobotAdaptor own its own shape with the wire staying ANY?
5. models/ vs policies/ split. Why two layers (raw nn.Module vs Policy wrapper)? Is a new VLA expected to follow the split or can it ship as a single fused class?
6. Octo. Lives in policy_octo on ac-octo purely because of the jax/flax pin conflict; once ported forward it folds back in as octo_model.py + octo_policy.py.

8. The VLN-CE instance in depth

8.1 Upstream inference contract (Krantz et al., VLN-CE)

The Cross-Modal Attention policy is the classic trained VLN-CE baseline: a recurrent network attends across instruction tokens and RGB-D features and emits one of four discrete actions per step. The VLN-CE evaluation loop drives it with four recurrent inputs per step:

1. Inputs: observations (RGB, depth, tokenized instruction), rnn_states (GRU hidden, zeros at episode start), prev_actions (int64, 0 at start), not_done_masks (0 on the first step of an episode, 1 after — this is how the recurrent state resets without re-allocating).
2. Forward: instruction and visual encoders → cross-modal attention → GRU → action distribution; eval uses the distribution mode (deterministic), not a sample.
3. Outputs: a discrete action in {0 STOP, 1 FORWARD, 2 LEFT, 3 RIGHT} + the next rnn_states.
4. Episode protocol: STOP (or budget) ends the episode; success is position-based (≤ 3 m), computed by the env.

8.2 The port — CMA as a pipeline variant

The CMA baseline ships as the cma model adapter + cma_policy inside policy_adapter_vlnce, selected via the variant bundle (CMA_PM_DA_Aug pairs cma + r2r_baselines/cma_pm_da.yaml + cma_policy + the released checkpoint data/habitat/checkpoints/CMA_PM_DA_Aug.pth). Key contract facts, with where they live now:

• Architecture parity by strict load. The CMA/Seq2Seq nets are vendored from vlnce_baselines; the checkpoint loads via load_state_dict with default strict matching (policies/_recurrent.py:43-45) — any layer-shape mismatch refuses to load, so a successful load is the arch-parity proof.
• Eval-mode determinism. distribution.mode() under deterministic=True (policies/cma_policy.py:145, seq2seq_policy.py:110) — the upstream eval protocol.
• Recurrent quadruple on the wire. {rnn_states, prev_actions, not_done_masks} rides the hidden_in/out ports through the loop pivots. A fresh episode starts from hidden_in=None (→ zero-init) instead of the upstream masks=0 flag — the wire's absence carries the episode boundary; equivalent reset semantics, simpler wiring.
• Tokenization is model-side. CMA's vocab tokenization (a faithful replica of the offline VocabDict preprocessing — verified 200/200 vs the dataset's precomputed tokens; self-contained adapters/r2r_instruction_vocab.json) runs in cma.canonical_to_model, memoized per episode; the env adapter passes the raw text through verbatim.
• Instruction port is required, not optional — so the executor waits for it before firing stage 1 (optional → the node races ahead with instruction=None → 0-step episodes).

8.3 Delta vs upstream — the buckets

8.4 Evaluation

9. File layout

workspace/nodesets/env/env_adapter/
├── __init__.py                # EnvAdapterNodeSet + 4 nodes + lazy adapter caches
├── robots/                    # VLA env side (from policy_vla/adapters/robots/)
│   ├── base_robot.py          # RobotAdaptor (abstract); excluded from discovery
│   ├── canonical.py           # VLA schema copy (excluded from discovery)
│   ├── libero_robot.py        # + DEFAULT_KWARGS (LIBERO delta preset)
│   ├── libero_robot_absolute.py  # variant — same class, empty DEFAULT_KWARGS
│   └── simpler_robot.py       # WidowX + Google Robot, no wrist cam
└── envs/                      # VLN env side (from policy_vlnce/adapters/envs/)
    ├── base_env.py            # VlnEnvAdaptor (abstract); excluded
    ├── canonical.py           # VLN schema copy (excluded)
    ├── r2rce.py               # 4-action discrete, raw-text instruction
    └── rxrce.py               # 6-action, BERT-feature instruction

workspace/nodesets/policy/policy_adapter_vla/
├── __init__.py                # 3 nodes + VlaPolicyManager + discovery helpers
├── _assets/norm_stats/        # per-model norm-stats JSONs
├── _vendored/openpi-client/   # vendored (was private upstream)
├── adapters/
│   ├── canonical.py           # CanonicalDict / CanonicalInfo / make_canonical_*
│   └── models/                # ⬅ model-side adapters (auto-discovered)
│       ├── base_model.py · dp_defaults.py        # excluded from discovery
│       └── pi0_model.py · smolvla_model.py · dp_model.py · rt1_model.py
├── models/                    # vendored backbones (openpi, smolvla, dp, droid, …)
└── policies/                  # ⬅ BasePolicy wrappers (auto-discovered)
    ├── base_policy.py         # excluded
    └── pi0_policy.py · smolvla_policy.py · dp_policy.py · droid_dp_policy.py
        · rt1_policy.py · rt1_policy_google_robot.py

workspace/nodesets/policy/policy_adapter_vlnce/
├── __init__.py                # 3 nodes + VlnceManager (stage-2 chain entry)
├── variants.py                # VariantSpec REGISTRY (12 R2R-CE baselines)
├── adapters/
│   ├── canonical.py           # CanonicalDict / CanonicalNavInfo
│   ├── r2r_tokenizer.py + r2r_instruction_vocab.json   # CMA-vocab replica
│   └── models/                # ⬅ cma.py · seq2seq.py (base_model.py excluded)
└── policies/                  # ⬅ cma_policy.py · seq2seq_policy.py
    └── base_policy.py · _recurrent.py   # excluded from discovery

10. Provenance and history

The VLA adapter system (CanonicalDict, CanonicalInfo, the RobotAdaptor / ModelAdaptor base classes) is vendored verbatim from the upstream vlaworkspace training repo per AgentCanvas's no-upstream-import policy — source copied with rewritten imports, bug-for-bug parity with the vendoring snapshot, no live import at runtime. The VLN side is the VLN-flavored sibling: the same envelope and stage shape, re-authored for navigation (the VlnEnvAdaptor / VlnModelAdaptor / VlnPolicy hierarchy is local, the CMA / Seq2Seq nets are vendored from vlnce_baselines). All three nodesets keep workspace/ standalone — no runtime dependency on third_party/.

Lineage of this page and these nodesets:

• 2026-07-04 — the General Policy Adapter split. policy_vla and policy_vlnce were re-cut along the CanonicalDict boundary into env_adapter + policy_adapter_vla + policy_adapter_vlnce; upstream's composed Adaptor dissolved; the VLA info-dataclass wire asymmetry was fixed (both domains now flatten at the boundary); verified bit-identical on the R2R-CE reference slice. This page absorbed the former VLA Policy NodeSet and CMA Policy pages the same day.
• Legacy policy_cma (deleted 2026-07-04). The original CMA port wrapped the whole policy as a single batched forward node (policy_cma__forward, workspace/nodesets/policy/policy_cma.py). Its working code paths were lifted verbatim into the pipeline's cma adapter + cma_policy, the pipeline version was verified end-to-end where the single-node port never was (no graph ever shipped for it), and the file was removed — git history keeps it.
• 2026-06-29 — standardize/process refactor. The VLN env adapter was reduced to pure standardization and tokenization moved model-side (the standalone tokenize_instruction node was retired, graph 16→15 nodes); verified bit-identical.