Environment NodeSets
Simulators and benchmark environments. Per the decoupled-nodeset principle, env nodesets are pure tools: they expose observe / step / episode-control surfaces and carry no method reasoning. Method nodesets consume them over wires.
All MDP envs implement the gym-like interface contract — four verbs (reset / step_<actionspace> / observe_<obsspace> / close) plus a thin evaluate sink. One deliberate exception: OpenEQA-EM is exempt (a non-MDP frame-reader with no action and hence no step) — classified in the template's interface taxonomy (template §1). DetAny3D, the stateless detection service that sat here by historical placement, moved to model/model_detany3d on 2026-07-04 (see the Tools catalog).
| NodeSet | File | Description |
|---|---|---|
| Habitat | env_habitat.py | Habitat-Sim 0.1.7 VLN-CE simulator + HabitatEnvPanel |
| VLN-CE @ hs0.3.3 | env_vlnce033/ | R2R-CE episodes on habitat-sim 0.3.3 without habitat-lab — faithful VLN-CE action/sensor caliber (0.25 m / 15° / sliding / RGB 224 + normalized depth 256 / 3 m success), NE/SR/SPL/OSR/nDTW metrics computed in-nodeset; modern caliber, not 0.1.7-comparable. Extra step_size_m/turn_angle_deg knobs for continuous-trajectory (SLAM) work. Dedicated page pending |
| Matterport3D | env_mp3d/ | MatterSim discrete panoramic nav across R2R/R4R/RxR/REVERIE/CVDN/NDH |
| HM-EQA | env_hmeqa/ | HM3D semantic-mesh EQA env — free-pose teleport, episode Q/A, TSDF bounds; serves the HM-EQA and MT-HM3D corpora via dataset specs |
| EXPRESS-Bench | env_express/ | Exploration-aware open-vocab EQA on HM3D — free-pose teleport with geodesic path-length accounting, benchmark-native gpt-4o-mini judge band, C / C* / E_path / d_T metrics |
| ObjectNav | env_objnav.py | HM3D / MP3D object-goal navigation on habitat-lab 0.2.4 — category goal, 1 m success radius, paper-standard agent |
| HM3D-OVON | env_ovon/ | Open-vocabulary object-goal navigation — free-text goals, three val splits grading generalisation, port shapes mirror env_objnav verb-for-verb |
| HM3D-OVON | env_ovon/ | Open-vocabulary object-goal navigation — free-form text goals over 3 val splits (seen / synonyms / unseen); vendored OVON-v1 dataset class. Dedicated page pending the embodiment decision |
| OpenEQA | env_openeqa_em.py | OpenEQA Episodic-Memory EQA — free-form answers scored by LLM-as-judge |
| SIMPLER | env_simpler/ | SimplerEnv real-to-sim VLA eval — 25 tasks, SAPIEN + ManiSkill2_real2sim |
| LIBERO | env_libero/ | LIBERO manipulation benchmark — 130 tasks on a Panda arm, robosuite + MuJoCo |
| Env Adapter | env_adapter/ | Env-side stages of the General Policy Adapter pipeline — env→canonical / canonical→env for VLA robots + VLN-CE families; pure numpy, in-process |
Files live under workspace/nodesets/env/.
Capability matrix
Which env implements which space suffixes. The vocabulary (what a suffix means, what ports it carries) is contract and lives in the template; this table is the registry — it changes when env coverage changes.
| Env | step_* (action spaces) | observe_* (obs spaces) | Notes |
|---|---|---|---|
env_habitat | discrete, pose, hightolow | egocentric, panorama | 3 coexisting step nodes (accepted) |
env_vlnce033 | discrete | egocentric | R2R-CE on habitat-sim 0.3.3; node surface mirrors env_habitat (same port names on observe/step/evaluate) so the CMA adapter graph carries over by node-type rename |
env_mp3d | waypoint | navigable, panorama | Discrete panoramic graph nav |
env_hmeqa | pose (teleport) | egocentric | EQA (HM-EQA + MT-HM3D dataset specs) |
env_express | pose (teleport) | egocentric | Exploration-aware open-vocab EQA; sample_waypoint / sample_frontier ship the paper's RE and FBE baseline action sources (vendored upstream TSDF planner), stop_gate/answer_router the faithful early-stop loop, and judge_prompt/evaluate the benchmark's LLM-judge scoring band |
env_objnav | discrete, pose | egocentric, panorama | Object-goal search; observe_egocentric carries the ObjectNav task sensors (gps, compass) alongside the standard RGB-D/pose/intrinsics payload; step_pose walks to a target via shortest-path primitives without dispatching STOP |
env_ovon | discrete, pose | egocentric, camera_pose, panorama | Open-vocabulary object search. Port shapes mirror env_objnav verb-for-verb, so a graph binds to either by swapping the node_type prefix; the goal arrives as free-form text on reset.object_category rather than a category id, and habitat's objectgoal_sensor is removed because an open vocabulary has no category→id table |
env_simpler | continuous | egocentric | 7-DoF VLA |
env_libero | continuous, ee_pose | egocentric, objects | Manipulation; objects = privileged GT snapshot for GT planners (+ reset_to_home/close_gripper EE-control extras). step_pose → step_ee_pose 2026-07-03: the 8-vec + gripper shape never matched the nav pose row |
env_openeqa_em | — (none) | frames | Non-MDP replay benchmark: reset + frame sampling, no step (exempt from the template) |
Installation prerequisites
Heavy simulators run in server mode: each nodeset auto-hosts itself in a dedicated
conda env, resolved by env name (override with the env var below). Run the matching
scripts/install/install_*.sh once per machine; scripts that need
third_party/ submodules initialize and build them on their own.
| NodeSet | Conda env (override var) | Install script | third_party/ needed |
|---|---|---|---|
| Habitat | vlnce ($VLNCE_PYTHON) | install_ac_vlnce.sh | VLN-CE + habitat-lab submodules |
| Matterport3D | mp3d ($MP3D_PYTHON) | install_ac_mp3d.sh | Matterport3DSimulator submodule (C++ build; connectivity graphs + R2R episodes read at runtime) |
| HM-EQA | ac-habitat033 ($HMEQA_PYTHON; renamed from ac-hmeqa 2026-08-16) | install_ac_hmeqa.sh | — |
| EXPRESS-Bench | ac-habitat033 ($EXPRESS_PYTHON) — shared habitat-0.3.3 stack | — (questions JSON + HM3D scene pool staged under data/hm3d/) | — |
| ObjectNav | ac-objnav ($OBJNAV_PYTHON) | — (conda habitat-sim=0.2.4 + pip habitat-lab==0.2.420230405; install script pending) | — |
| HM3D-OVON | ac-objnav ($OVON_PYTHON) — shared with ObjectNav | — (episodes from HuggingFace nyokoyama/hm3d_ovon into data/datasets/ovon/hm3d/; scenes shared with ObjectNav) | — |
| VLN-CE @ hs0.3.3 | ac-habitat033 ($VLNCE033_PYTHON) — shared habitat-0.3.3 stack | — (R2R_VLNCE_v1-3 episode JSONs + MP3D scenes already staged) | — |
| OpenEQA | ac-habitat033, optional — falls back to local mode | fetch_dataset_openeqa.sh | — |
| SIMPLER | ac-simpler ($SIMPLER_PYTHON) | install_ac_simpler.sh | — |
| LIBERO | ac-libero ($LIBERO_PYTHON) | install_ac_libero.sh | vla_workspace submodule (nested libero source) |