SIMPLER NodeSet — AgentCanvas

The SIMPLER NodeSet (EnvSimplerNodeSet) wraps SAPIEN + ManiSkill2_real2sim for SIMPLER — the VLA evaluation benchmark (Li et al. 2024, real-to-sim manipulation). 25 tasks total split across two embodiments: WidowX/Bridge (4 tasks) and Google Robot (21 tasks).

Sibling of LIBERO — the two are deliberately a matched pair for first-release VLA evaluation. LIBERO uses MuJoCo + robosuite + Franka Panda; SIMPLER uses SAPIEN + ManiSkill2 + WidowX or Google Robot. Same observation-bundle shape, same TEXT-JSON 7-vec action contract.

1. Overview

Purpose

SIMPLER is purpose-built to evaluate pre-trained VLA checkpoints (RT-1, OpenVLA, Octo, etc.) without re-training: <1 GB assets, ~30 s/episode, real-to-sim camera matching, deterministic seeds. It pairs with LIBERO as the first-release VLA env coverage in AgentCanvas.

Architecture

Three-layer pattern mirroring LIBERO:

SimplerEnvManager (singleton) — one simpler_env.make(task_id) instance, rebuilt on every set_episode (each task is a different SAPIEN scene). Pinned single-thread executor for SAPIEN GL/physics affinity. Caches the task-by-embodiment registry once on initialize().
Five canvas tool nodes — reset, step, get_observation, episode_info, evaluate.
EnvSimplerNodeSet — server_python reads $SIMPLER_PYTHON, defaults to /home/$(whoami)/miniforge3/envs/ac-simpler/bin/python (created by scripts/install/install_ac_simpler.sh).

Server-Mode Only

The nodeset always runs server-mode: SAPIEN + numpy<2 conflict with the default agentcanvas env's pins. The framework auto-routes the nodeset into the ac-simpler subprocess when loaded via ?mode=server.

parallelism = "replicated" — each batch-eval worker gets its own SAPIEN scene; per-worker isolation is mandatory because the scene is stateful.

Interface migrated (2026-06-09). This nodeset now implements the gym-like env interface — reset (metadata only), step_* (control signals, no observation), observe_* (pull perception), evaluate. The node list below predates the migration and may be stale; the template's migration map (§6) and capability matrix (§3) are authoritative.

2. Canvas Nodes

Node Type	Display Name	Input Ports	Output Ports	Purpose
`env_simpler__reset`	SIMPLER: Reset	`trigger` (ANY, optional)	`instruction`, `episode_id`, `observation` (LIST[IMAGE]), `pose` (POSE, None), `agentview_image`, `wrist_image` (None), `state`, `split`, `task_id`, `max_steps`	Reset to env panel-selected episode; idempotent
`env_simpler__step`	SIMPLER: Step	`action` (TEXT)	reset bundle + `done`, `reward`, `success`, `step_index`, `truncated`	7-vec or K-step chunk; clip + step; early-break
`env_simpler__get_observation`	SIMPLER: Get Observation	`trigger` (ANY, optional)	same as `reset`	Read-only obs snapshot
`env_simpler__episode_info`	SIMPLER: Episode Info	(none)	`split`, `task_id`, `episode_id`, `instruction`, `max_steps`, `step_index`, `cumulative_reward`, `success`, `done`	Metadata only
`env_simpler__evaluate`	SIMPLER: Evaluate	`trigger` (ANY, optional)	`success` (BOOL), `metrics` (METRICS)	Post-hoc success + metrics

The port list intentionally mirrors LIBERO's so a graph written against LIBERO can be ported to SIMPLER by node-type rename only — wrist_image is kept as a port even though SIMPLER has no wrist cam (it always emits None).

3. Action Contract

7-vec TEXT JSON, single step or K-step chunk:

"[ax, ay, az, arx, ary, arz, grip]"        # single step
"[[ax, ay, az, arx, ary, arz, grip], ...]" # K-step chunk

Index	Meaning	Range
0–2	Delta end-effector position (XYZ, metres)	`[-1, 1]`
3–5	Delta orientation (axis-angle, radians)	`[-π/2, π/2]`
6	Gripper	`[-1, 1]` (SIMPLER wrapper: 0=open/1=close per its step contract; not the same as LIBERO, whose robosuite action is +1=close/−1=open — verified 2026-06-28)

NaN/Inf → nan_to_num; per-channel clipping done inside the manager. Both Bridge and Google Robot share this 7D shape; per-embodiment differences (camera name, control frequency) are dispatched on split.

step() accepts one action at a time at the env layer; the manager loops K times for chunk inputs and early-breaks on terminated or truncated.

4. Observation Bundle

Port	Wire Type	Meaning
`agentview_image`	IMAGE	Third-person camera (480×640 for Bridge, 512×640 for Google Robot), uint8 RGB
`wrist_image`	IMAGE	Always `None` (SIMPLER has no wrist cam — port kept for LIBERO portability)
`observation`	LIST[IMAGE]	`[agentview_image]` (single-view; LIBERO emits two-view)
`state`	ANY	Flat float32 proprio: `qpos + qvel + base_pose` (23-D for WidowX, 29-D for Google Robot)
`pose`	POSE	Always `None` (manipulation, not navigation)
`instruction`	TEXT	`env.unwrapped.get_language_instruction()`
`episode_id`	TEXT	`"{split}/{task_id}/{episode_index}"`
`split`, `task_id`, `max_steps`, `step_index`, `reward`, `success`, `done`, `truncated`	(per LIBERO)	Bookkeeping

max_steps is read per-task from env.spec.max_episode_steps (60 for widowx_spoon_on_towel, 80 for Google Robot tasks; back-stop fallback 60).

Camera dispatch — _SPLIT_CAMERA = {"bridge": "3rd_view_camera", "google_robot": "overhead_camera"}. The wrapper pulls the first preference by name and falls back to the first available camera if naming changes upstream.

5. Env panel Integration

Three cascaded select fields:

Field	Default	Options
`split`	`bridge`	`bridge` (4 tasks) or `google_robot` (21 tasks)
`task_id`	first of split	depends on split — filtered from `simpler_env.ENVIRONMENTS` by prefix
`episode_index`	`0`	`range(50)` (seeds 2022–2071), 50 per task → 1250 total episodes

The cascade follows LIBERO's pattern: changing split resets task_id to the first task of that split and episode_index to 0; changing task_id resets episode_index to 0. All field changes emit signal_name="episode_reset" so lifetime="episode" state containers clear automatically.

Episode counts: {"bridge": 200, "google_robot": 1050} — exposed via get_eval_metadata().

Required actions play / pause / stop / reset exactly per the env-nodeset Tier-1 env panel contract (see Env Panels design doc).

6. Environment Setup

Install

bash scripts/install/install_ac_simpler.sh

Idempotent. Creates conda env ac-simpler (Python 3.10), clones simpler-env/ManiSkill2_real2sim and simpler-env/SimplerEnv into third_party/, installs both editable, pins numpy<2 + setuptools<81 (SAPIEN compat), and runs an obs-key/proprio probe.

Pins (verified 2026-05-01)

Package	Pin	Why
`numpy`	`<2`	numpy 2.x triggers segfaults in SAPIEN's step path
`setuptools`	`<81`	SAPIEN's `renderer_config.py` imports `pkg_resources` (removed in setuptools 81+)
`opencv-python-headless`	`<4.10`	keeps `numpy<2` pin satisfied

Hardware

NVIDIA GPU + Vulkan ICD required — SAPIEN cannot run CPU-only.
The install script does a vulkaninfo precheck and prints a warning (does not auto-install) if Vulkan is missing.
Headless: SAPIEN renders without DISPLAY; expect a one-line GLFW: X11 failed to open display warning at first import — harmless.

Activation

export SIMPLER_PYTHON=/home/$(whoami)/miniforge3/envs/ac-simpler/bin/python

EnvSimplerNodeSet.server_python reads this env var; without it the default fallback python is used.

7. Smoke Verification

workspace/graphs/simpler_smoke.json — Tier-1 fixed-action DAG:

env_simpler__reset → example__source("[0,0,0,0,0,0,1]") → env_simpler__step → env_simpler__evaluate → metrics

Pick the split / task / episode in the SIMPLER env panel panel, click Play. Expected on a fresh run:

instruction shows the task language (e.g. "put the spoon on the towel").
agentview_image viewer renders the third-person camera.
step_index = 1, success = False (one fixed action won't solve the task).
metrics = {success: 0.0, num_steps: 1, split: "bridge", task_id: "widowx_spoon_on_towel", cumulative_reward: 0.0}.

A real VLA-policy node consuming (observation, instruction) and emitting an action TEXT is a separate task — see roadmap E11 follow-ups.