DiscussNav on AgentCanvas — paper analysis + discussnav port

DiscussNav (23.09; Long et al., ICRA 2024 — "Discuss Before Moving: Visual Language Navigation via Multi-expert Discussions") is a zero-shot R2R agent that, every step, runs a panel of role-specialised GPT-4 experts — instruction analysis, vision perception, completion estimation, and a decision-testing pair — over a sampled set of candidate moves before committing to a viewpoint. Section 1 dissects the upstream rollout (DiscussNav.py, the single-file reference). Section 2 documents the AgentCanvas port — the 9-node discussnav nodeset on env_mp3d discrete panoramic nav. Section 3 files every difference into the five delta buckets. Bottom line up front (re-audit 2026-06-16): the per-step reasoning, prompts, and — after a control-flow repair pass — the loop control now match upstream: the adaptive 5/7 horizon drives a live horizon_stop, pred_vp samples n=5 with retry-until-5-valid and an early break_flag, and vision is navigable-gated like upstream's observe_view. Verdict: faithful with justified deviations; verified (smoke).

1. Upstream method analysis (Long et al. 2024)

Upstream DiscussNav is a single-file MatterSim rollout (DiscussNav.py) over four "expert" classes plus a DiscussNav_Agent. The "discussion" is not eight independent agents — it is GPT-4 sampled n=5 times from one prediction agent, then those samples grouped, fused, and adjudicated. Per episode (main, :480–528):

1. Instruction analysis (once per episode). detect_actions (:103–110) first translates the instruction to English (:104), then GPT-4 decomposes it into a newline-separated action list; detect_landmarks (:112–119) extracts landmarks from those actions. Results are cached per instruction (actions_cache).
2. Adaptive horizon. step_length = 5 if len(actions.split("\n")) <= 5 else 7 (:506) — the loop runs exactly step_length steps (for current_step in range(step_length), :507). There is no STOP action; the episode ends at the budget, and the final viewpoint is wherever the agent stands.
3. Vision perception (per step). observe_environment (:206–232) turns to each of 12 directions; per navigable direction observe_view (:161–204) runs RAM tagging + InstructBLIP captioning and emits "Direction d … Navigable Viewpoint ID: vp [(Passed Area)] Elevation: Eye Level Scene Description: {caption} Scene Objects: {tags}; " (:170, :185–187). Crucially it captions only the navigable directions, caching by view (view_record). A "stair"-in-instruction branch additionally looks down −30° and appends an Elevation: Look Down block (:189–200).
4. Completion estimation (per step). review_history (:267–271) joins prior steps as "Step N Observation: … Thought: …" with " -> "; estimate_completion (:273–290) calls GPT-4 and returns only the slice after "Executed Actions" (:286–288).
5. Prediction — the discussion (per step). DiscussNav_Agent.pred_vp (:297–347) issues one GPT-4 call with n=num_predictions=5. Each sample must contain Prediction:, parse to a 32-char hex vp, and be a substring of the observation (:327–338). It retries the whole n=5 call up to num_retry=5 times until 5 valid predictions land (:322, :343–345); if it never does, break_flag=True and the episode loop breaks (:516–517).
6. Decision testing (per step). thought_fusion (:364–382) groups predictions by unique vp and fuses each group's thoughts with a GPT-4 call; test_decisions (:384–405) returns the single vp directly when there is only one, else asks a final GPT-4 (temperature 0, ≤3 retries) to pick.
7. Act + memory (per step). make_action (:349–361) teleports to the chosen vp and returns that direction's observation string; save_history (:256–265) runs two sequential GPT-4 summarisations (observation, then thought) and appends {viewpoint, observation, thought} to nav_history.
8. Evaluate. After the loop, final_vp = sim.getState()…viewpointId (:525) → eval_pred (:407–445) computes SR / SPL / OSR / nDTW against the GT path (ERROR_MARGIN = 3).

The row layout mirrors the port diagram in §2.1 box-for-box, so the two figures compare directly.

1.1 Upstream invariants worth marking

• The horizon is the termination. There is no STOP action. The episode runs step_length (5 or 7) steps and stops where it stands. Adaptive horizon and final-position-at-budget are the whole stopping mechanism — a port that changes the step count changes which viewpoint gets scored.
• The discussion is sampling, not a fan-out of distinct agents. Diversity comes from one GPT-4 call with n=5; the "experts" are role-specialised prompts, not separate models. The fusion + decision pair only runs meaningfully when the 5 samples disagree.
• Five-valid-or-retry is load-bearing. pred_vp retries the whole batch until it has 5 valid predictions; failing that, the episode breaks early. Both the retry and the early break shape the trajectory.
• Vision is navigable-gated and cached. observe_view captions only the views toward navigable viewpoints (~2–5/step), not the whole panorama, and caches by view (view_record). Captioning the full panorama is both unfaithful and ~10× the work.
• Estimation feeds the agent only its "Executed Actions" slice. The Thought half of estimate_completion is dropped before it reaches pred_vp (:286–288).
• History summarises one direction. save_history summarises the chosen direction's observation (the string make_action returned), not the whole panorama.

2. The AgentCanvas port

workspace/nodesets/method/discussnav.py splits the rollout into 9 single-responsibility nodes, wired in discussnav_mp3d.json against env_mp3d and two foundation-model nodesets. The per-step reasoning maps cleanly, and after the 2026-06-16 repair pass the loop scaffolding matches upstream too: pred_vp carries the n=5 retry/break, horizon_stop drives the adaptive 5/7 termination, and panorama_to_views is navigable-gated (§2.1).

2.1 Per-step flow

The loop is a standard iterIn/iterOut scope: observations enter through iterIn's dual slots (init_* from seed_nav/seed_pano at step 0, iterout_* thereafter), and step.info triggers the loop_nav/loop_pano re-observations that iterOut collects for the next iteration. evaluate sits in the after-loop band, fed once by iter_out.final_stop (so TODO #64's loop-body-evaluate under-count does not apply here). Termination now matches upstream: horizon_stop is the single writer of iter_out.stop — it fires when ctx.step ≥ step_budget − 1 (the adaptive 5/7 carried from init_decompose through iterIn) or when pred_vp raises break_flag (fewer than 5 valid predictions after 5 retries). The dead step.terminated → iter_out.stop edge from the first cut is gone.

2.2 Node inventory

The instruction-analysis and completion-estimation GPT calls are not in this nodeset — they are built-in llmCall nodes in the graph (decompose_actions_llm, detect_landmarks_llm, estimate_completion_llm), carrying their system prompts in llmCall.config. The prediction call moved into pred_vp (it needs the n=5 retry/break loop); the nodeset's own GPT calls (prediction / fusion / decision / summarise) use the _SYS_* constants in discussnav.py.

2.3 State & memory

Three access grants are declared (one per reader/writer above). Upstream's view_record caption cache is now ported as a content-hash cache inside model_ram / model_instructblip (§3A); the actions_cache / eval_cache on-disk caches are not (§3C).

2.4 Boundary contract

Reasoning-only nodeset — no simulator calls, no model weights (verified: discussnav.py imports only app.components.bases, app.llm, and stdlib/PIL). Env side: env_mp3d via reset / observe_navigable / observe_panorama / step_waypoint / evaluate. FM side: built-in llmCall (profile-driven), model_ram (RAM tagger, server-mode), and model_instructblip (InstructBLIP flan-t5-xl, dedicated server-mode FM nodeset extracted from navgpt_mp3d_tools per TODO #56's method/foundation-model split). The InstructBLIP server runs in the agentcanvas conda env (transformers 4.45.2) — not ac-ram, whose tokenizers 0.15.2 cannot load the flan-t5 processor (§4 — this 500-ed an entire run).

2.5 Prompt assets

All six system prompts + the 12 direction labels live as verbatim constants in discussnav.py (with DiscussNav.py:line citations, including the preserved "Font Right" typo at :152). The decompose / landmark / estimate system prompts are duplicated into the graph's llmCall.config.system_prompt fields — two copies of the same string, a drift risk (§7). User templates are byte-identical to upstream (see §3A).

3. Delta vs upstream — five buckets

Each row carries an equivalence icon (orthogonal to the bucket letter — the letter says why it differs, the mark says whether behaviour changed): 🟢 equivalent (identical or semantically identical behaviour) · 🟡 near-equivalent (same intent; mechanism, library, or an edge case differs — output usually matches) · 🟠 divergent (behaviour can differ — by design or, rarely, a defect).

A. Preserved verbatim / faithful

B. Forced by substrate (script → typed graph)

C. Forced by environment / cost

D. Intentional divergences

E. Unexplained / defects

Probed and cleared: the RAM + InstructBLIP "Scene Description: {caption} Scene Objects: {tags};" observation format is faithful (upstream :170 uses the same two models — InstructBLIP is not a port-only addition); the 6 system prompts and 12 direction labels are byte-identical; evaluate is correctly on the after-loop band (TODO #64 does not apply). No upstream equivalence test exists to run; the smoke run (§4) is the behavioural check.

4. Evaluation

5. Usage

Load the method nodeset plus its two foundation-model dependencies:

POST /api/components/nodesets/discussnav/load
POST /api/components/nodesets/model_ram/load?mode=server
POST /api/components/nodesets/model_instructblip/load?mode=server

Graph: workspace/graphs/vln/unverified/discussnav_mp3d.json. Smoke eval (graph-only /experiment:run, since 2026-05-07) — the actual command behind run 20260616_172215:

/experiment:run discussnav-mp3d discussnav_mp3d split=val_unseen episode_count=5 worker_count=5 step_budget=7 per_step_budget_sec=300

(discussnav sets default_per_step_budget_sec = 90.0; pass per_step_budget_sec explicitly since the episode timeout is step_budget × per_step_budget_sec, resolved from the env nodeset. The shared model_instructblip server loads InstructBLIP once and content-hash-caches captions, so step 0 is slow (~75 s incl. model load + beam-5 captions) and later steps are fast on revisited views. Beam-5 decoding (LAVIS-faithful) is several× slower than greedy, hence per_step_budget_sec=300.)

6. What this nodeset is NOT

• Not an 8-agent fan-out. Despite the paper's "eight experts" framing, the discussion is one GPT call sampled n=5 + role-specialised prompts, not eight independent models. The port mirrors the code, not the marketing.
• Not a continuous-control method. It is MP3D discrete R2R (waypoint teleport), not VLN-CE.
• Not stairs-aware. The upstream look-down branch is deliberately omitted (§3D).
• Not env-coupled. No simulator or model imports in the nodeset; vision + LLM are external nodes.
• Not yet paper-validated. The §4 number is a 5-ep gpt-5-mini smoke slice; the graph stays in unverified/ until a full split on a GPT-4-class model runs.

7. Source files

Side findings (doc/code drift, candidate fixes — not delta buckets):

• Decompose / landmark / estimate system prompts exist twice — as _SYS_* constants in discussnav.py and as copies in llmCall.config.system_prompt; edit both or they drift.
• panorama_to_views docstring history still references the old full-panorama / model_ram-only wiring in places; the navigable-gating is the current behaviour.
• Upstream is not vendored under workspace/nodesets/_upstream/ (per ADR-platform-007) — only the zz_just_for_refer raw copy exists, with no pinned commit.