Roadmap — Done

§1Done — Shipped Features

• Two-layer canvas architecture (outer graph + inner graph editor)
• Dataflow executor with IterIn/IterOut iteration gates
• DAG executor for acyclic graphs
• Component system with auto-discovery from workspace/
• WebSocket streaming with execution_id routing (8 event types)
• Single /run endpoint with pause/resume/stop lifecycle
• Graph presets: Straightforward (CMA), NavGPT-CE
• Wire type system: IMAGE, DEPTH, ACTION, STATE, TEXT, BOOL, METRICS, OBSERVATION
• Hydra eval harness with native + external policy paths
• MkDocs Material documentation site
• Typed graph model (GraphDefinition, NodeDef, EdgeDef in graph_def.py) — backend validation, JSON file management, CRUD API
• AutoServerApp: write-once deploy-anywhere for nodesets (ADR-server-001)
• Unified canvas paradigm (ADR-canvas-002) — one editor, one registry, one catalog at every graph level
• NodeSet Manager page (ADR-canvas-001) — dedicated UI for nodeset load/unload, server start/stop/restart
• BaseEnv removed — environments are NodeSets (ADR-components-002)
• Standardized node types nodeset__node convention, auto-load nodesets on graph open (ADR-components-003)
• Legacy two-layer files deleted (nodeTypes.ts, innerNodeTypes.ts, FlowCanvas.tsx, InnerGraphEditor.tsx, AgentLoopCanvasNode.tsx, …)
• Python-driven node UI (NodeUIConfig + ConfigField on BaseCanvasNode) — all node rendering driven from Python, GenericBlockRenderer handles block + strip layouts
• Mock/legacy node cleanup — deleted 23 mock .tsx files, all mock nodes labelled [Mock] in sidebar + canvas
• Custom .tsx reduced from 23 → 10 (6 custom UI + 1 renderer + 2 shared + 1 composite helper)
• State container system (dual-wire architecture): StateContainer as dict of named BaseState entries, 4 reducer types (Accumulator, LastWrite, Counter, Ephemeral), extensible value type registry, state edges as separate wire system, canvas toggle, executor integration, flatten support
• Graph node system (ADR-canvas-003): kind field on GraphDefinition, dual storage (graphs/ + graph_nodes/), Explorer split into Graphs + Graph Nodes sections, VS Code-style tree expansion, "Save as Node" button, snapshot semantics
• Auto-layout system: swim-lane layout algorithm (app/layout.py), POST /api/graphs/layout endpoint, Auto Layout button. Updated 2026-04-19 for ADR-dataflow-006 three-pivot: nodes split into init band + loop band stacked vertically with BAND_GAP
• Per-node LLM/VLM model selection via profile config field in LLMCallNode
• LLM config system redesign (ADR-platform-003): layered field-level env var fallback (profile > AGENTCANVAS_* > VLM_* > registry), argparse CLI (8 commands, works without server), mtime-based ProfileStore cache invalidation, key validation, interactive setup script
• VS Code-like UI layout: ExplorerPanel (tree sidebar), TabBar (tabbed multi-graph editing), OutputDrawer
• BaseSkill removed — skills refactored to BaseCanvasNode subclasses in vln_skills nodeset
• agentcanvas general-purpose env (ADR-platform-004): agentcanvas (Python 3.10+, modern torch/transformers + FastAPI) as THE environment for all development, research, and runtime; vlnce (Python 3.8) retained only for the habitat-sim 0.1.7 binary constraint. Framework import guard test enforces the boundary.
• ADR-components-004 recorded for Python-driven node UI
• NodeUIConfig documentation with examples in capabilities/customizable-node-system.md
• Eval page redesign (ADR-eval-001): graph-driven batch evaluation via BatchEvalOrchestrator wrapping LoopRunner, ExecutionPrinciples for batch mode, ExecutionGuard for exclusive canvas/eval lock, env metadata interface on BaseNodeSet, registry-backed graph introspection, shared nodeset auto-loading, v2 API at /api/eval/v2, new frontend (9 components)
• Node output log system (ADR-observability-003): two-layer execution logging — automatic exterior (executor captures port I/O + timing per node firing) + voluntary interior (_self_log() on BaseCanvasNode). ExecutionLogger with JSONL persistence. log_serialize() truncates large values. exec_log WS events (suppressed in eval mode). REST API at /api/logs with 4 GET endpoints. Frontend LogPanel in OutputDrawer Logs tab.
• Canvas environment & episode control (ADR-legacy-002): standalone EnvController Protocol. HabitatEnvController delegates to HabitatEnvManager.get(). REST API at /api/env/{name}/ (5 endpoints). POST endpoints guarded by ExecutionGuard. Pre-flight pattern: frontend calls envSetEpisode() before runPipeline() — keeps RunRequest/LoopRunner env-agnostic. EnvContextBar component, auto-advance on episode completion.
• Configurable IterIn/IterOut ports (ADR-dataflow-003): config.ports on NodeDef overrides class-level ports per instance. BaseCanvasNode._resolve_ports(config) classmethod. Domain ports removed from framework class defaults. Frontend resolveInstancePorts() utility, PortListEditor component. All 3 graph presets migrated with explicit config.ports.
• LIST[T] wire-type modifier (ADR-dataflow-005): opt-in list modifier with consumer-side coercion. LLMCallNode.rgb migrated to LIST[IMAGE], unblocking MapGPT/MC-GPT/DiscussNav/VLN-Zero multi-image prompts. graph_def.validate_edge_wire_type compatibility helper; frontend PortListEditor checkbox modifier + isCompatibleWireConnection + doubled-ring handle visuals.
• Viewer system refactor: per-instance viewer data via viewer_data WS events keyed by node_id. _SinkBase.forward() serializes wired inputs via serialize_for_display(). Display type registry (viewerFieldRegistry.ts) with lazy-loaded field renderers. DisplayField.accumulate for log-type fields. 5 viewer sinks: ImageViewer, ThinkingLog, ActionLog, Metrics, TextViewer.
• Configurable imageViewer replaces fixed observationViewer (ADR-components-007): new ImageViewerSink with _resolve_ports deriving ports from config.ports (IMAGE/DEPTH only) + rows/cols grid; new imageGrid frontend layout + ImageGridViewerLayout.tsx; 4 workspace graphs migrated.
• Hybrid data-layout unification (ADR-platform-005): data/ restructured into habitat/ (VLN-CE framework mirror) + mp3d/v1/scans/ (asset-kind namespace) + opennav/ (tool namespace) + outputs/{eval, eval_mock, eval_native}. New scripts/data/migrate_data_layout.sh (atomic rename(2), --dry-run, idempotent), scripts/data/gen_skybox_rgb_mp3d.sh (inline cv2 merger), scripts/smoke/vlnce_smoke.py. Bugs fixed: matterport3d.py:245 passed parent of v1/scans to setDatasetPath; MATTERPORT_DATA_DIR suffix match was unanchored.
• Worker-pool batch eval + batched inference tier (ADR-eval-002): three-phase rollout — Phase A foundation (controller_overrides, EvalConfig.worker_count/per_step_budget_sec, BaseNodeSet.default_per_step_budget_sec, EnvWorkerPool seam at worker_count=1), Phase B multi-worker env fan-out (worker_count=N spawns N tagged subprocesses + per-worker URL routing + asyncio.gather over shared episode queue with results_lock), Phase C batched inference (BaseCanvasNode.batched/batch_dim ClassVars; BatchedInferenceServer + _BatchQueue + BatchedClient; policy_cma__forward opted in).

§2Completed TODO Items

IDs are permanent, never reused. Full context in Done section and Changelog.

• 1 Doc-site: add glossary entries for NodeUIConfig, GenericBlockRenderer, ConfigField; document mock cleanup in relevant docs
• 2 Doc-site: update capabilities/customizable-node-system.md with NodeUIConfig examples for nodeset authors
• 3 Doc-site: update capabilities/real-time-observability.md — /node-schemas response now includes ui_config
• 4 Doc-site: record ADR for Python-driven node UI (NodeUIConfig replacing custom .tsx)
• 5 Dynamic sidebar catalog — show only tools from activated NodeSets + built-ins
• 7 Delete docs-site/docs/tmp_trash/ — deleted; installation.md rescued
• 8 Layout Component System (ADR-components-006): DisplayField, layout="viewer", refactor GenericBlockRenderer into dispatcher + layout components, eliminate 4 output viewer .tsx files
• 9 Verify saved graphs (Straightforward, NavGPT-CE) render correctly with GenericBlockRenderer
• 10 Add ui_config to nodeset tools — every node in habitat.py, sam.py, policy_cma.py, others.py has a NodeUIConfig ClassVar
• 12 Implement basic_agent_nodeset — general-purpose canvas nodes for LLM agents: note_write/note_read/note_list, web_search/web_fetch, image_analyze, measure_distance/compute_heading, get_instruction/get_step_count/get_history
• 15 Correct skills logic — BaseSkill removed; skills refactored to BaseCanvasNode subclasses
• 16 State container system (dual-wire architecture)
• 18 Better LLM/VLM call — per-node model selection via profile config field
• 20 Choose the actual running env for agentcanvas — ADR-platform-004
• 21 General UI & filesystem refactor — VS Code-like layout
• 24 Call for contribution — CONTRIBUTING.md, label good-first-issues
• 26 File explorer with sub-graph pull-down and graph node
• 31 Running log system — two-layer node output logging (ADR-observability-003)
• 33 Unified proxy node types — auto_server_app.py:132,140 carries tool.node_type into manifest as FunctionSchema.name
• 35 Refactor episode control to use proper AutoServerApp protocol — ADR-server-002: replaced env-specific HabitatEnvController + 5 hidden tools with generic BaseController
• 38 Refactor viewer system — per-node viewer_data WS events, _SinkBase self-serializes
• 42 env nodeset should have a setted pattern, need refactor
• 43 Figure out MP3D data story — resolved by ADR-platform-005: 90 skybox zips downloaded (18 GB), 10,912 panoramas across 90 scans
• 50 Decouple VLM backend from explore-eqa nodeset
• 51 Refactor auto-layout to handle the note built-in node
• 52 Make the Explorer panel resizable like VSCode — drag handle to adjust sidebar width
• 65 VoxPoser-LIBERO gripper convention inverted at the actuation boundary — fixed 2026-06-28 at exactly the env_libero__step_pose boundary (+ two more stacked convention bugs + grasp geometry); voxposer_libero_decomposed re-evaled and promoted to vla/verified/ (libero_object SR 0.88, 2026-06-30)
• 67 Server-mode transport contract — resolved 2026-06-15 with a msgpack codec replacing JSON+base64 (pack_body/unpack_body, blob ExtType for ndarray/torch/PIL with receiver-side degrade); formalized as ADR-server-004
• 68 Guardrail: stateful state-container on a shared server nodeset — resolved 2026-06-15 as a load-time warning (registry._check_container_ownership, escalatable to hard-reject); state_demo migrated to parallelism="replicated"
• E1 Matterport3D Simulator nodeset — shipped 2026-04-18 as EnvMP3DNodeSet with three-field controller cascade (dataset → split → episode_index)
• E5 Benchmark data pipeline — scripts/data/fetch_episodes_vln.sh takes --r2r / --r4r / --rxr / --rxr-full / --reverie / --reverie-features / --cvdn / --all / --dry-run / --force flags and lands 20 GB of episodes
• E6 HM-EQA dataset/benchmark integration — completed 2026-05-09: workspace/nodesets/env/env_hmeqa/ (5 tools, free-pose teleport, ac-hmeqa env) + explore-eqa method nodesets; closed end-to-end on the 100-episode HM-EQA val standard split at SR 0.43 (≥ paper 0.38)
• E9 OpenEQA (EM-EQA) integration — completed 2026-05-04: workspace/nodesets/env/env_openeqa_em.py (5 tools) + three graphs (blind-LLM / single-frame / multi-frame); first benchmark exercising free-text reasoning + LLM-as-judge eval
• E11 SIMPLER nodeset — shipped 2026-05-01 as workspace/nodesets/server/simpler/ with reset/step/observe/metric/dataset_split/controller tools, paired with ac-simpler conda env (SAPIEN/ManiSkill2)
• E12 LIBERO nodeset — shipped 2026-05-01 covering the four LIBERO task suites (Spatial / Object / Goal / 100), built on robosuite + MuJoCo via ac-libero conda env
• E13 VLA Policy nodeset — shipped 2026-05-02 as workspace/nodesets/server/policy_vla/. Hosts Pi0 / SmolVLA / Diffusion Policy / DROID-DP via 4-node decomposition. End-to-end verified with Pi0 LIBERO finetune: 5/5 success, avg 83.8 steps, 5.4 s wall