VLA Support Status

This page does two jobs. Part I — Support answers which VLA methods AgentCanvas runs or wants to run: a single support matrix (verified · in-progress · retired), then the env / policy / method machinery underneath. Part II — Landscape places those methods in the wider VLA field — Track-A trained policies, Track-B zero-shot manipulation, and the 2024–26 frontier (§8) — condensed from the VLA — Methods survey. Companion to VLN Support Status and EQA Support Status.

Part I — Methods we support

1. Support matrix

Every VLA graph AgentCanvas ships or is building (retired graphs kept for the record). This is the single source of truth for "what we support" — the env / policy / method backing is in §2–§4, the wider field is Part II. Verified graphs live under vla/verified/, the rest under vla/unverified/.

This page is fresh — the verified rows have hard ground-truth datapoints from the author. Update the others as runs are confirmed.

VLN graphs are out of scope (see VLN Support Status); EQA graphs are out of scope (see EQA Support Status).

2. Environment Nodesets

Manipulation simulators wrapped as BaseNodeSets under server mode. Both ship in folder form — <name>/__init__.py plus sidecar wrappers — so the binary-incompatible upstream code stays packaged with its owner.

2.1 LIBERO (E12)

• Folder: workspace/nodesets/env/env_libero/ (__init__.py + _wrapper.py)
• Source: LIBERO manipulation benchmark — robosuite + MuJoCo, 7-D action contract
• Coverage: 130 tasks across 5 suites — libero_spatial / object / goal / 10 / 90
• Why it's in: every modern VLA paper reports it; "fact-on-the-ground 2024–25 VLA benchmark"
• Bounds calibration (2026-05-16, superseded 2026-06-28): z ∈ [+0.85, +1.25] is the libero_spatial (table-mounted) frame only — NOT canonical across suites. Floor-mounted suites (libero_object, robot base z≈0) sit ~0.9 m lower; baking that single z box as a constant left every libero_object grasp target unreachable → 0/50 for 29 AAS iterations (misread as an OSC step-budget problem). Fixed 2026-06-28: derive the z box per scene from robot0_base z (_Z_BELOW_BASE=0.10 / _Z_ABOVE_BASE=0.35), x/y fixed. (The earlier DEFAULT_BOUNDS = z∈[-0.1, +0.6] "EE freezes at z=0.94" symptom — 30 cm below the table — was a separate, older red herring.)
• Doc: (no per-nodeset page yet — TBD)

2.2 SIMPLER (E11)

• Folder: workspace/nodesets/env/env_simpler/ (__init__.py + _wrapper.py)
• Source: SimplerEnv real-to-sim VLA eval — SAPIEN + ManiSkill2_real2sim
• Coverage: 25 tasks — 4 WidowX/Bridge + 21 Google Robot
• Why it's in: drop-in eval of any HF-hosted VLA checkpoint (low friction, cross-paper adoption)
• Doc: SIMPLER nodeset

3. Policy Adapter Nodeset

A single registry hosts all VLA policies behind a uniform 4-stage pipeline.

3.1 VLA Policy (E13)

• Folder: workspace/nodesets/policy/policy_adapter_vla/
• adapters/ — env↔canonical and canonical↔model adapters (per env × per model matrix)
• models/ — model-specific I/O shape conversions
• policies/ — checkpoint loaders + predict() per policy family
• presets/ — graph snippets for common (env, policy, ckpt) combos
• Independent conda env: ac-vla-policy
• Decomposition — 4 explicit canvas nodes per VLA call, replacing vlaworkspace's monolithic Adaptor system:

adapt_env_to_canonical → adapt_canonical_to_model → predict → adapt_model_to_env

• Policy registry (4 today): Pi0, SmolVLA, Diffusion Policy (DP), DROID-DP
• Checkpoints: Lightning .ckpt and safetensors both supported by Pi0Policy.load_checkpoint
• Doc: General Policy Adapter (§7, VLA instance)

This is a generic VLA inference framework, not a Pi0/SmolVLA/DP wrapper. Adding a new policy = adding a policies/<name>.py + adapter rows; the 4-node decomposition is reused unchanged.

3.2 VoxPoser-LIBERO (workspace/nodesets/method/voxposer/)

Motion-planning method (LMP + dynamics models). Not a policy in the policy_adapter_vla sense — instead an LMP-style planner that builds a voxel cost map and emits an OSC waypoint trajectory. Env-decoupled as of 2026-06-10: the whole planning stack (7 LMPs, voxel maps, path planner) runs local-mode in the agentcanvas backend; its only env coupling is the privileged GT snapshot wire (env_libero__observe_objects) in and waypoint actions (env_libero__step_ee_pose) out — any env exposing that observe/step pair can host VoxPoser.

• Pairs with: voxposer_libero_decomposed.json (✅ verified on libero_object 2026-06-30, SR 0.88 @50ep; other suites still open). The monolithic graph was deleted 2026-06-10 — its LMP-exec↔sim-stepping interleave cannot cross a wire boundary
• Why the snapshot is sound: plan_subtask never steps the sim, so all env reads during one plan see a single static state — a frozen snapshot is exactly equivalent to the retired in-subprocess live-adapter reads. The follow loop still drives every waypoint closed-loop (the 2026-05 v2 WiredEnv attempt failed (SR=0) because it buffered whole execute() trajectories open-loop; the backend WiredEnv now raises on any write call)
• GT is explicit: VoxPoser's ground-truth perception is now a first-class privileged observation space on the env, visible in graph topology, instead of a hidden in-subprocess adapter
• Bounds: see §2.1 — the calibrated table-aware bounds ride inside the observe_objects snapshot

3.3 Octo for SIMPLER (workspace/nodesets/policy/policy_octo.py)

Native SIMPLER baselines — RT-1-X / Octo — wrapped as a sibling to policy_adapter_vla. SIMPLER evaluation deliberately uses these native policies (not Pi0 finetunes) per upstream convention; checkpoints under data/vla_policy/checkpoints/.

• Pairs with: octo_simpler.json

4. Ready-to-Run Graphs

When forking a new VLA experiment, start from vla_policy_libero.json — it carries the most complete recorded lineage (pre-migration 5/5 smoke; re-verification pending after the 2026-06-10 gym rewire).

5. Action Contract and Metrics

VLA action shapes are env-side, not yet unified (TODO #46 — VLN/EQA face the same gap):

Standard manipulation metric: success per episode (binary), aggregated as success rate over a task / suite. Wall-clock and step counts are also captured by the eval harness (e.g. the LIBERO smoke run reports avg 83.8 steps, 5.4 s wall).

6. Eval Infrastructure

6.1 Smoke runners

The first-party smoke runner scripts/smoke/smoke_vla_libero.py (used to produce the 2026-05-02 verified Pi0 result) was removed 2026-06-30 as dev scaffolding; for one-off "does this checkpoint load" checks, run the vla_policy_libero graph via /experiment:run instead.

6.2 BatchEvalRunner + EnvWorkerPool (ADR-eval-002)

Same pipeline VLN uses (agentcanvas/backend/app/agent_loop/eval_batch.py + env_worker_pool.py):

• One fresh LoopRunner per episode (clean state isolation)
• worker_count > 1 spawns N tagged env subprocesses; each WorkerHandle carries env_panel_overrides + server_url_overrides
• Per-episode timeout: max_steps × per_step_budget_sec — for VLA-heavy graphs the policy node typically dominates, so set default_per_step_budget_sec on the env nodeset accordingly

6.3 BatchedInferenceServer + parallelism (ADR-eval-002 PC, ADR-server-003)

• policy_adapter_vla nodes are good batched-inference candidates (one GPU forward across K callers), but verify whether each policies/<name>.py::predict is pure-functional (per-call state on explicit ports) before opting in batched=True, batch_dim=...
• Env parallelism (BaseNodeSet.parallelism) — robosuite/MuJoCo/SAPIEN sims are stateful and thread-affine, so LIBERO and SIMPLER stay replicated; the policy subprocess is the natural place to put parallelism="shared" once batched is opted in

6.4 SAPIEN / SIMPLER stability gotchas

SIMPLER eval has known stability cliffs in the upstream stack. Working around them is a hard requirement, not optional:

• Never call env.close() between SIMPLER episodes. SAPIEN 2.2.2 has a GC ordering bug that segfaults ld.so on close-then-reopen. Mirror upstream's no-close pattern: reuse the same env handle across episodes for the whole run.
• SIMPLER auto_host crashes after ~15 set_episode() switches. Reload the nodeset between batches if you need to evaluate more than ~15 distinct episodes in one process — the cumulative state-switch state eventually trips an internal crash.
• BatchEvalRunner is single-selectors per run. Multi-task sweeps go sequential (one selector per run); don't try to pack multiple selectors into one job spec.
• Metrics drop on terminal step: GraphExecutor historically settled sibling sinks after the termination pull-check, silently dropping env_*__evaluate outputs on the terminal step. Now fixed at the engine level (settle-before-pull-check in the loop-control system; see Loop Control §2.8).

7. Known Gaps

VLA-relevant items in roadmap.md:

Also worth tracking informally: a per-nodeset doc page for LIBERO is missing (developer-guide/nodesets/ has SIMPLER but not LIBERO). Adding it would put the env-side documentation symmetric with SIMPLER.

Part II — Field landscape

8. Landscape — representative methods & frontier

Where the supported VLA subset sits in the wider field, condensed from the research survey VLA — Methods (cite counts are that page's Semantic Scholar snapshot). The field splits into Track A (trained VLA / policy models) and Track B (zero-shot / LMP-style manipulation — VoxPoser's lineage). Ours maps to §1 / §3: ✅ verified · ✅ baseline · ⏳ unverified · ▫ in policy_adapter_vla registry · — not ported.

8.1 Representative & most-cited

8.2 Frontier (2024–2026)

Our coverage is deliberately one method per archetype: a Track-B LMP planner (VoxPoser, the only ✅ verified VLA graph) plus Track-A trained policies through policy_adapter_vla (Pi-0 / SmolVLA / DP) and the SIMPLER native baselines (RT-1-X / Octo). The bulk of the field — RT-2 / OpenVLA-class web-scale VLAs and the 2025–26 zero-shot frontier — is unported.