ObjectNav NodeSet

The ObjectNav NodeSet (EnvObjnavNodeSet) wraps habitat-lab 0.2.4 for object-goal navigation: an episode names a category and the agent must reach any instance of it and call STOP within 1 m. Two episode corpora are selectable from the env panel — HM3D (6 categories: chair, bed, plant, toilet, tv_monitor, sofa) and MP3D (21 categories, the Habitat-2020-challenge generation). Unlike the other nav envs here, the goal arrives as a bare noun, not a route description, so the whole task is search under semantic priors.

Sibling of Habitat, not an extension. Both wrap habitat, but they cannot share a process or an interpreter: Habitat pins habitat-sim 0.1.7 + Python 3.8 for VLN-CE, while ObjectNav pins habitat-sim 0.2.4 + habitat-lab 0.2.420230405 + Python 3.9 in its own ac-objnav env. Different simulator generation, different config schema (habitat.* omegaconf vs VLN-CE's TASK_CONFIG.* yacs).

1. Overview

Benchmark configuration is the paper standard, not the challenge

ObjectNav has two live configurations, and they are not comparable. The Habitat Challenge 2023 pins challenge-2023 habitat-lab and a Stretch robot embodiment; the published literature — VLFM (23.12), SG-Nav, and the RSS-2026 pair MVP-Nav / OpenFrontier — instead uses habitat 0.2.4 with the classic cylindrical agent inherited from Challenge 2022. This nodeset implements the latter, because that is what every number in every comparison table was produced under:

One nodeset serves both corpora because habitat-lab 0.2.4's objectnav_mp3d.yaml and objectnav_hm3d.yaml benchmark configs are identical except for data_path (verified by a resolved-config field diff): same agent, sensors, action space, success criterion and budget. Selecting a dataset in the env panel is therefore an episode-corpus choice, not a task change — method graphs never see the difference.

Architecture

The three-layer shape shared by every env nodeset:

1. ObjnavEnvManager (singleton) — owns one non-vectorized habitat.Env, a lock, and a single-thread executor pinning all simulator work to one OS thread (GL affinity). Blocking methods only; the nodes hop onto its executor.
2. Canvas nodes — the four gym verbs, thin adapters over the manager.
3. EnvObjnavNodeSet — server_python = conda_env_python("ac-objnav", "OBJNAV_PYTHON"), so the framework auto-hosts the nodeset in its own subprocess under ?mode=server.

Server-mode only in practice: habitat_sim is not installed in the default agentcanvas env. Override the interpreter with $OBJNAV_PYTHON if your env lives elsewhere.

2. Canvas nodes

Seven nodes on the gym contract: reset carries metadata only, step_* returns control signals only, perception is pulled via observe_*, and evaluate is a thin sink for the after-loop band.

Action space

habitat 0.2.4's ObjectNav defaults, in index order. The order matters: a textParse node configured with these six choices emits an index that is the action, so no mapper node is needed between reasoning and env.

terminated vs truncated

habitat exposes a single episode_over flag; the manager splits it into the two gym signals by looking at what caused it. An explicit STOP (action 0) is terminated — the agent committed, and the metrics reflect its judgment. Running out of the step budget without ever calling STOP is truncated. Terminal metrics ride info["metrics"] in both cases, so a graph that ends on budget still harvests a verdict.

3. Env panel

ObjnavEnvPanel owns episode placement — there are no split/episode canvas nodes. Its cascade is three deep:

• dataset — hm3d_v1 (paper standard), hm3d_v2 (Challenge-2023 episodes, OpenFrontier's choice) or mp3d_v1 (2020-challenge generation, 21 categories). Changing it rebuilds the env and re-scopes the split list. Legacy values v1/v2 are accepted everywhere as aliases for the HM3D pair.
• split — val (2000 episodes on HM3D, 2195 on MP3D) or val_mini (30, for smokes). train is deliberately absent: the HM3D train scenes are not staged, MP3D's train episodes would be eagerly loaded just to fill a dropdown, and zero-shot methods do not train.
• episode_index — labelled {index}: {scene} → {category} so the goal is visible when picking.

Field changes emit the episode_reset signal, clearing every lifetime="episode" state container. play re-seats the selected episode before starting the run.

4. Data staging

Scenes come from the official Matterport distribution (login-gated; an API token works headlessly via curl --user token-id:secret). Episodes are public direct downloads from dl.fbaipublicfiles.com — with one trap: the MP3D episode zip's official URL spells the corpus m3d (…/objectnav/m3d/v1/objectnav_mp3d_v1.zip); the expected mp3d path returns 403.

Semantic meshes are not required for scoring

Success is judged against goal viewpoints embedded in the episode JSON, not against a runtime semantic lookup — verified by driving ShortestPathFollower to the nearest viewpoint and getting SR 1.00 with no semantic files present. The .semantic.glb files are staged anyway because habitat's top_down_map measure reads the semantic scene, and reference implementations (VLFM among them) enable it.

5. Probe graph: objnav_hm3d

workspace/graphs/vln/unverified/objnav_hm3d.json — the ObjectNav analog of simple_navigator, and the graph that exercises this nodeset end to end. Every reasoning node ships with the platform; there is no method-specific code anywhere in it.

• llmCall, textParse, historyLog — built-in glue
• basic_agent__image_analyze, basic_agent__obs_to_text — observation encoding
• the four verbs above — env side

The loop is: observe → caption the frame → format an observation string → LLM picks one of the six actions → parse → step. The parse node's choice list is written in action-index order (STOP,FORWARD,LEFT,RIGHT,LOOK_UP,LOOK_DOWN), so its index output wires straight into step_discrete.action. step_discrete.terminated drives iter_out.stop; evaluate hangs off iter_out.final_stop in the after-loop band, which is what keeps the terminal STOP step inside the harvested metrics. The graph is also corpus-agnostic — nothing in it names a dataset, so the same JSON runs MP3D by submitting with dataset=mp3d_v1.

6. Verification evidence

HM3D rows 2026-07-20 (slot-b backend); MP3D rows 2026-07-21 (slot-c backend). Single RTX 3090, ac-objnav env.

The evaluate count is the load-bearing one. It confirms the after-loop band fires and this graph does not reproduce the harvest miss that affects several gym-migrated graphs (roadmap TODO #64), where a loop-body evaluate never sees the terminal STEP and records success=0.

A sizing trap worth inheriting

The first probe run died at step 10 of 30 with "timeout after 150.0s". ADR-eval-002 caps each episode at step_budget × default_per_step_budget_sec, and the nodeset originally declared 5.0 — correct for a bare habitat step, wildly wrong for a graph making two vision-LLM calls per step (~15 s). It is now 30.0, matching Habitat's value and for the same reason. Any env destined for LLM-in-the-loop graphs needs the larger number, and the symptom of getting it wrong looks like an env bug rather than a budget one.

Oracle stop precision on MP3D

MP3D's 2020-era goal viewpoints are sparser than HM3D's, which breaks naive oracle harnesses twice. A follower goal_radius of 0.25 m stops outside the 0.1 m success criterion — the first attempt scored SR 0.567 with every failure sitting at distance_to_goal 0.11–0.25. Tightening the radius to 0.1 makes the follower oscillate around it on 0.25 m steps until the budget burns, standing at dtg ≈ 0.04 with no STOP ever dispatched. The working form keeps the stop decision metric-blind: goal_radius = success_distance plus a no-progress rule (net displacement under 0.05 m across a 50-step nav call means the discretization is at its limit — commit STOP where standing). HM3D's dense viewpoints mask both traps: the same loose harness scores 1.0 there.

7. Not yet verified

Stated plainly so nobody reads the table above as broader than it is:

• Long episodes. Every batch run so far capped episodes at 8 steps for API cost; the full 500-step budget has only been exercised by the manager-level oracle/budget smokes.
• Canvas Play. Verified through the eval-batch path only; the viewer nodes (imageViewer, textScroll, actionLog, metrics) executed but were never watched in the GUI.
• The STOP path through the graph. The probe loop ended on step budget, so the terminated → iter_out.stop wire has never been driven truthy in a real run. The env-side STOP itself is verified at manager level.
• Any success rate. No method has run on this env — only a nano-model toy agent. No SR number from this stack should be quoted yet.
• MP3D multi-worker. The 4-worker VRAM/RAM figures above were measured on HM3D; MP3D's building-scale meshes are larger. Re-measure on the first real MP3D batch (the objnav-mp3d profile note says the same).