Three-Step Nav on AgentCanvas — paper analysis + threestepnav port

Three-Step Nav (A Hierarchical Global-Local Planner for Zero-Shot Vision-and-Language Navigation, arXiv 2604.26946) is a zero-shot LLM agent for continuous VLN (R2R-CE / RxR-CE) that descends directly from Open-Nav — its agent class literally is class Open_Nav. It runs three steps per episode: (1) global — decompose the instruction into sub-instructions + per-sub landmarks; (2) local — a MapGPT-style image navigator picks the next direction and self-reports Completion Estimation: Yes/No; (3) verify — when completion is "Yes", a continue / stay / backtrack / look-around judge advances a sub-instruction pointer. The threestepnav nodeset (single-version monolith) ports the method-side reasoning; it reuses Open-Nav's env_habitat + opennav_waypoint (TRM) substrate wholesale, with the perception FMs consumed as generic wrappers since the TODO #56 extraction (2026-07-04): model_ram__tag_views (RAM Swin-L @ 224, raw | separators) + vlm_spatialbot__caption_views (SpatialBot-3B), fed by the pure threestepnav__select_candidate_views filter.

1. Upstream method analysis (Zheng et al. 2026)

The driver is base_il_trainer_llm.BaseVLNCETrainerLLM._eval_llm. The agent (spatialNavigator.Open_Nav) and judge (decision_agent) are pure-Python over an OpenAI client (api.llmClient) and the perception stack (api.spatialClient — RAM + SpatialBot-3B). Per episode:

• Pre-loop (Step 1, global) — navigator.detect_actions (ACTION_DETECTION) splits the instruction into action_list; navigator.detect_landmarks (LANDMARK_DETECTION, one call per sub-instruction) builds landmark_list. base_il_trainer_llm.py:589-594
• Per step (Step 2, local) — the policy net predicts candidate waypoints (BinaryDistPredictor_TRM); construct_image_dicts bins them into the 12 directions (30° each); navigator.observe_environment fuses RAM tags + SpatialBot captions per candidate; navigator.move_to_next_vp_single (MAPGPT_NAVIGATOR, image prompt over candidate tiles) returns Prediction (next vp) + Completion Estimation. :607-790
• Per step (Step 3, verify) — gated on completion_estimation == "Yes": decision_agent.make_informed_decision_with_capture returns continue / stay / backtrack / look-around. continue advances current_action_idx (resetting per-sub history); at the last sub-instruction continue → stop_flag; should_stop_navigation also stops on continue/stay at the last sub-instruction. backtrack reverses the last move. :760-839
• Env action — {'action': 4, 'action_args': {'angle': radius_dict[next_vp], 'distance': distance_dict[next_vp]}} (HIGHTOLOW). Loop ends at current_step == step_length (step_length = MAX_EPISODE_STEPS). :849-875

1.1 Upstream invariants worth marking

• The judge is gated on completion. make_informed_decision only fires when the navigator self-reports Completion Estimation: Yes; otherwise the agent silently stays on the current sub-instruction (no extra LLM call). base_il_trainer_llm.py:760, 841-843
• STOP is decide-to-stop, not an env action. There is no STOP in the action space — the episode ends when the agent completes the last sub-instruction (continue/stay at idx == len-1 while completion is Yes). decision_agent.py:957-959
• 12-direction binning is fixed. construct_image_dicts maps each predicted angle to one of 12 viewpoint ids (0 = current orientation, 30° apart). The navigator prompt's geometry depends on this exact mapping. base_il_trainer_llm.py:202-259
• Per-sub-instruction history resets on continue. Advancing the pointer clears nav_history. base_il_trainer_llm.py:783-784
• The judge sees a chosen-path image sequence seeded with the current view and appended each step. base_il_trainer_llm.py:487-491, 687
• Default model is gpt-5 (run.sh: --llm gpt-5-2025-08-07), shipped abilities are continue + stay only (run_OpenNav.yaml: ENABLED_META_ABILITIES), with no max_tokens cap.

2. The AgentCanvas port

workspace/nodesets/method/threestepnav/__init__.py mirrors the upstream method bodies 1:1 within the framework floor. Because Three-Step makes ~4 LLM calls per step (navigator + two history-compression summaries + the gated judge) plus the pre-loop decompose, the floor is already 14 method nodes — most are single-responsibility prompt builders / parsers. The one fat node is judge_decide, the sole writer of all four state keys. The env (env_habitat), waypoint predictor (opennav_waypoint) and perception wrappers (model_ram / vlm_spatialbot) are shared with Open-Nav.

2.1 Per-step flow

The loop is a v3 iterIn/iterOut scope: action_list/landmark_list ride the init_* ports (persist) and are re-emitted each step; select_subinstruction reads the pointer to emit the current/next sub-instruction. The judge LLM is internal to judge_decide (it is conditional on completion, so it cannot be a stand-alone llmCall node). step_hightolow.info triggers the next panorama_rgbd; judge_decide.stop → iter_out.stop ends the loop; iter_out.final_stop fires evaluate once.

2.2 Node inventory

Single version. A Phase-2 decomposition (threestep2 — judge_decide split along the single-writer seam, proven byte-equivalent over 13 scenarios) existed until 2026-07-03 and was removed by decision: the monolith is already fine-grained (its 13 sibling nodes are forced apart by the ~4 graph-level llmCalls per step), and the judge's conditional LLM call cannot be a graph-level node in any split — the decomposition only relocated state-writing. The monolith is the sole version and the ground-truth I/O contract.

2.3 State & memory

2.4 Boundary contract

The nodeset is reasoning-only — it imports no env runtime. It consumes env_habitat (reset / observe_panorama / step_hightolow / evaluate), the opennav_waypoint TRM predictor, the generic model_ram + vlm_spatialbot FM wrappers, and built-in llmCall — all over wires. The import-boundary test (app/test_import_boundary.py) holds. The same substrate serves Open-Nav itself; the waypoint/perception fixes shipped with this port also repaired Open-Nav.

2.5 Prompt assets

All navigator / decompose / summary prompts are verbatim module-level constants in _prompts.py, each citing its upstream source. A programmatic byte-diff (2026-06-22) confirmed ACTION_DETECTION, LANDMARK_DETECTION, OBSERVATION_SUMMARY, THOUGHT_SUMMARY, MAPGPT_NAVIGATOR (system 5278 chars) and DIRECTIONS are byte-identical to the upstream prompts.py dicts — including the "Font Left" typo at DIRECTIONS[1]. The judge prompt (JUDGE_SYSTEM + build_judge_user's enhanced_prompt: capability descriptions, degenerate AST logic blocks, in-context examples, format instructions) was byte-diff-verified against upstream make_informed_decision_with_capture (grill 2026-06-22) — see §3A. The last fragment — the prepended "Image sequence descriptions" block — was added in grill iter-2 (§3A), so the judge prompt is now byte-faithful in full.

3. Delta vs upstream — five buckets

Each row carries an equivalence icon (orthogonal to the bucket letter — the letter says why it differs, the mark says whether behaviour changed): 🟢 equivalent · 🟡 near-equivalent · 🟠 divergent. A genuine defect is a 🟠 row in bucket E with a severity, never an ❌.

A. Preserved verbatim

B. Forced by substrate

C. Forced by environment / cost

D. Intentional divergences

E. Unexplained / defects

None open. The success-metric divergence found after the aligned run was re-filed to D (2026-07-03) — an eval-reporting decision with a recorded dual-reporting policy (§4.1), not a port defect. Earlier candidates were likewise re-filed: navigator random.choice fallback → A (dead at eval), retry-once → D, caption-500 → §4 known engine gaps.

Probed — five independent passes, each walking the upstream source cold (per-pass detail in the changelog):

• 2026-06-22 · grill iter 1–3 — full eval loop, both VLM call sites, capability-AST path, decision rules, stuck subsystem. Closed the navigator/judge detail, image-descriptions and stuck-direction gaps; corrected the config premises (abilities [continue,stay], horizon 8).
• 2026-06-25 · cold re-audit — three parallel inventories; all prompts re-confirmed byte-identical; found upstream's random.choice fallback dead at eval (E→A).
• 2026-07-02 · iter 4 — fresh upstream clone; six findings, five fixed same-day (judge post-append History · 1-based steps · str(list) Current Environment · '0'-first candidate order · strict == "Yes" gate) plus dict_keys D→A — the navigator user prompt became byte-identical end-to-end.
• 2026-07-02 · iter 5 — upstream-import byte harness: the real upstream modules as oracle, compared function-by-function on identical inputs (79 checks). Three fixes (summary truncation · Thought: strip · [Code-Informed] prefix); one new B row (backtrack double-step, config-dead).
• 2026-07-03 · perception round — the fork's pixel path pinned end-to-end; ten fixes, heaviest a silently-never-loaded DDPPO depth encoder (zero depth features in all prior Open-Nav-family runs). Harness grown to 82 byte-checks / 0 mismatch; waypoint path 6/6 tensor-equal; 13/13 decomp gate; smoke in-vivo green.

4. Evaluation

Two success rulers appear below (definitions in §4.1): SR = official VLN-CE stop-gated Success; SR* = upstream's own rule (final distance ≤ 3 m, no STOP gate). The paper's numbers are SR*.

Bottom line (implementation frozen 2026-07-03): on the aligned stack the port reproduces the paper under the paper's own ruler — SR* 0.24 vs 0.34 is within sampling noise at n=25, and the trajectory-shape metrics agree (TL 9.25 vs 9.18 · OSR 0.28 vs 0.39 · nDTW 0.535 vs 0.577). The official stop-gated SR stays low (0.08) for a structural reason: Three-Step rarely decides to stop (2/25 judge-stops on the headline run — both scored SR 1), and the paper's number never required stopping. In-log mechanics on the headline run are clean: the judge fired on all 65 Yes gates, the six SR* successes finished 0.5–2.5 m from the goal, and 2/200 steps were lost to the known tag-singleton 500. The 100-episode mini-tier scale check corroborates the headline at 4× the sample: SR* 0.29 · OSR 0.39 (equal to the paper) · nDTW 0.512, with the official stop-gated SR again low (0.10) for the same stopping-behaviour reason.

4.1 The two SR definitions

• Official VLN-CE SR (habitat-lab Success, habitat/tasks/nav/nav.py:500-539): success ⇔ is_stop_called and distance < 3 m — STOP is a hard requirement. This is the leaderboard definition and what env_habitat__evaluate reads; our judge-stop path maps to habitat STOP, so the port scores under it legitimately.
• Upstream's SR: the trainer strips TASK.MEASUREMENTS to [POSITION, STEPS_TAKEN] (base_il_trainer_llm.py:1309-1311) — the official measures are never computed — and hand-computes success = 1 if distance[-1] ≤ 3 (:966): no STOP gate. Forced by their design: the agent only emits HIGHTOLOW moves and the episode is cut at 8 steps, so official SR would be 0 by construction. The paper's 0.34 is this ruler.

The missing gate is fork-specific, not family convention: SmartWay's trainer — same lineage, same stripped-measures structure — keeps it (… and env_actions[i]['action']['action'] == 0, SmartWay base_il_trainer.py:547); the Three-Step fork's line drops that clause.

Comparability: OSR and TL are comparable across both stacks (neither OSR definition has a stop gate), nDTW methodologically so; SR and SPL are not. If the paper's baseline rows (Open-Nav 19 · SmartWay 29 · CA-Nav 25 · …) quote official numbers from the original papers, its table mixes rulers in the loose one's favour. Reporting policy (the §3D row): paper-reproduction claims quote SR* (recomputed post-hoc from distance_to_goal ≤ 3); benchmark claims quote official SR; every number above states its ruler.

Known engine gaps: high-worker caption-500 silent episode completion (SpatialBot singleton) — keep worker_count ≤ 5; the loop-body evaluate under-count (roadmap TODO #64) does not apply here (evaluate fires post-loop).

5. Usage

Load: POST /api/components/nodesets/threestepnav/load. Graph: workspace/graphs/vln/verified/threestepnav_ce.json. Indicative eval (graph-only form, since 2026-05-07):

/experiment:run threestepnav-ce threestepnav_ce \
    episode_count=25 worker_count=5 split=rand100 step_budget=8 per_step_budget_sec=180

Requires opennav_waypoint (env smartway) + opennav_perception (env ac-ram, SpatialBot-3B weights at data/opennav/SpatialBot-3B) shared servers. Keep worker_count ≤ 5 (caption singleton). Faithful paper config = gpt-5-full + abilities [continue,stay] + step_budget=8 (run_OpenNav.yaml); dev smokes may drop to gpt-5-mini per node config. The graph pins the faithful aligned config (gpt-5 · rgb_resolution=1024 on reset · image_mime=image/jpeg on the navigator); 1024-px renders need per_step_budget_sec=180 and write ~0.7 GB of log.jsonl per episode.

6. What this nodeset is NOT

• Not an env or perception nodeset — it owns no habitat / RAM / SpatialBot / TRM code; those are env_habitat / opennav_* reused from Open-Nav.
• Not model-locked — the LLM profile is per-node config; the verified graph pins gpt-5 (paper config), dev smokes may drop to gpt-5-mini.
• Not two versions — the Phase-2 decomposition (threestep2) was removed 2026-07-03; the monolith is the sole version (§2).
• Not benchmark-verified — under the official stop-gated SR the aligned run scores 0.08 (2/25 judge-stops); the paper's 0.34 is reproduced only under its own distance-only ruler (SR* 0.24, compatible at n=25). See §4.1.

7. Source files