Credits
Every nodeset and graph in AgentCanvas is credited to its author/maintainer here. When your contribution is merged into workspace/, you're added to this board — with a citation link if your work has an associated paper, so anyone using it can cite the source. Contributing into the shared repo doesn't cost you authorship. This board is the canonical, complete inventory of everything that ships under workspace/; the README Credits carries a condensed summary.
1. Framework
The AgentCanvas framework — canvas editor, graph executor, state containers, server mode, JobScheduler, doc-site — is built and maintained by AC-Team.
2. Methods & benchmarks
The research-facing components — the navigation / EQA / manipulation methods AgentCanvas ships and the simulators they run on. Each method is one row: its own perception, waypoint, map, or scene-graph sub-nodesets fold into the Nodesets cell. The method/benchmark name links its paper. Rows marked call for contribution are wanted and unclaimed (roadmap ID in What it provides) — the first to land each is credited here.
2.1 Benchmarks & environments
| Domain | Benchmark | Nodeset | What it provides | Contributor |
|---|---|---|---|---|
| VLN | Habitat (VLN-CE) | env_habitat |
habitat-sim 0.1.7 continuous navigation — R2R-CE | AC-Team |
| VLN | Matterport3D | env_mp3d |
MatterSim discrete panoramic nav — R2R / R4R / RxR / REVERIE / CVDN | AC-Team |
| ObjectNav | HM3D ObjectNav | env_objnav |
habitat-lab 0.2.4 object-goal navigation on HM3D-Semantics — objectnav_hm3d v1/v2, paper-standard agent | AC-Team |
| VLN | RxR-CE | — | Multilingual VLN-CE (roadmap E2) | call for contribution |
| VLN | REVERIE | — | Remote object grounding (roadmap E3) | call for contribution |
| VLN | AI2-THOR / ALFRED / TEACh | — | Interactive task benchmarks (roadmap E4) | call for contribution |
| EQA | HM-EQA | env_hmeqa |
House-Mesh embodied QA over HM3D — multi-choice, free-pose teleport | AC-Team |
| EQA | OpenEQA EM | env_openeqa_em |
Free-form embodied QA with LLM-as-judge (EM-EQA) | AC-Team |
| EQA | OpenEQA A-EQA | — | Active-EQA mode (roadmap E10, data-blocked) | call for contribution |
| VLA | SIMPLER | env_simpler |
SimplerEnv real-to-sim VLA eval (SAPIEN + ManiSkill2) | AC-Team |
| VLA | LIBERO | env_libero |
Robosuite + MuJoCo manipulation — 5 suites / 130 tasks | AC-Team |
2.2 Methods — VLN
Each method cell stacks its reference graph(s) (top) over its nodesets (below the dashed line).
| Method | Nodesets & graphs | What it provides | Status | Contributor |
|---|---|---|---|---|
| NavGPT | navgpt_mp3d.json, navgpt_ce.jsonnavgpt, navgpt_mp3d_tools |
LLM thought→action reasoning (MP3D + CE) with online BLIP-2 / Faster R-CNN tools | ✅ MP3D (gpt-4) · ⏳ CE | AC-Team |
| MapGPT | mapgpt_mp3d.jsonmapgpt |
Linguistic topological-map LLM agent + adaptive planning | ✅ verified | AC-Team |
| SmartWay | smartway_ce.json, smartway_mono_ce.jsonsmartway, smartway_mono, smartway_perception, smartway_waypoint |
Waypoint-predictor VLN-CE — dual-frame + single-frame, RAM+ tags + TRM waypoint | ✅ CE · ✅ᵖ mono (SR 0.270) | AC-Team |
| Open-Nav | opennav_habitat.jsonopennav, opennav_perception, opennav_waypoint |
Open-vocabulary VLN-CE — RAM + SpatialBot perception, frozen TRM waypoint | ⏳ unverified | AC-Team |
| DiscussNav | discussnav_mp3d.jsondiscussnav |
8-expert multi-LLM discussion → aggregator | ✓ in-progress | AC-Team |
| SpatialNav | spatialnav_mp3d.jsonspatialnav, ssg |
8-compass observation + Spatial Scene Graph fork of NavGPT-MP3D | ⏳ unverified | AC-Team |
| Three-Step Nav | threestepnav_ce.jsonthreestepnav |
Global-local navigation + back-check verify (mono + decomposed) | ✓ in-progress | AC-Team |
| AO-Planner | aoplanner_ce.jsonaoplanner |
Two-VLM Visual Affordances Prompting + pure-FM path planner | ✅ verified (SR 0.35) | AC-Team |
| CMA (Straightforward) | straightforward.jsonpolicy_adapter_vlnce + env_adapter |
Cross-Modal Attention VLN-CE neural baseline (batched node) | ✅ verified (SR 0.38) | AC-Team |
| VLN-CE policy registry | — policy_adapter_vlnce |
R2R-CE policy registry — 12 variants (CMA×7 + Seq2Seq×5; 2 released) | ⏳ unverified | AC-Team |
| HAMT | — | Hierarchical history transformer (roadmap M5) | 🅼 planned | call for contribution |
| DUET | — | Dual-scale graph transformer (roadmap M6) | 🅼 planned | call for contribution |
| InstructNav | — | Dynamic CoN + value maps, CoRL 2024 (roadmap M8) | 🅼 planned | call for contribution |
| VLN-SIG | — | Sub-instruction grounding (roadmap M4) | 🅼 planned | call for contribution |
2.3 Methods — EQA
| Method | Nodesets & graphs | What it provides | Status | Contributor |
|---|---|---|---|---|
| Explore-EQA | explore_eqa_hmeqa.jsonexplore_eqa, explore_eqa_tsdf |
Prismatic frontier exploration + TSDF voxel world-model | ✅ verified (SR 0.42) | AC-Team |
| ToolEQA | tooleqa_hmeqa.jsontooleqa, tooleqa_explore |
ReAct reasoner + fused go_next frontier step (server-mode VQA / detect) | ✓ in-progress | AC-Team |
| OpenEQA EM baselines | openeqa_em_blind_llm.json, openeqa_em_single_frame.json, openeqa_em_multiframe.jsonenv_openeqa_em |
blind-LLM / single-frame / multi-frame QA + LLM-as-judge reasoner | ✅ verified (LLM-Match ≤0.70) | AC-Team |
2.4 Methods — VLA
| Method | Nodesets & graphs | What it provides | Status | Contributor |
|---|---|---|---|---|
| VoxPoser | voxposer_libero_decomposed.jsonvoxposer |
LMP planner — voxel cost-map + OSC waypoint trajectory | ✅ verified (libero_object, SR 0.88) | AC-Team |
| Octo | octo_simpler.jsonpolicy_octo |
Octo / RT-1-X native SIMPLER VLA baselines | ✅ baseline | AC-Team |
| Generic VLA framework | vla_policy_libero.json, vla_policy_simpler.jsonpolicy_adapter_vla + env_adapter |
Pi0 / SmolVLA / DP / DROID-DP behind a 4-stage adapter pipeline | ⏳ unverified | AC-Team |
3. Foundation models
AgentCanvas supports a broad palette of foundation models. We wrap models from many sources behind one thin nodeset shell — lazy load, single-flight GPU, a base64-npy dataflow envelope, graceful degraded fallback — so each becomes a uniform building block available to both human users (wiring graphs on the canvas) and the AAS optimizer (searching architectures). We do not author these models; each is credited to its original authors (linked where available), and our contribution is only the shell. They split by source: §3.1 is transformers-native — the shell is a few lines over AutoModel / pipeline; §3.2 wraps other sources — torch.hub, torchvision, or an upstream repo vendored into a dedicated conda env.
3.1 transformers-native
| Model | Nodeset | Env | Source | Capability |
|---|---|---|---|---|
| CLIP | model_clip | ac-fm | transformers | Language-aligned image/text embeddings + zero-shot classify |
| SigLIP 2 | model_siglip2 | ac-fm | transformers | Image–text embeddings with a sigmoid zero-shot head (per-label, multi-label) |
| AIMv2 | model_aimv2 | ac-fm | transformers | Autoregressive ViT features — mean-pooled per-image embedding |
| OWLv2 | model_owlv2 | ac-fm | transformers | Open-vocabulary detection over a label set (GroundingDINO-compatible schema) |
| SAM | model_sam | ac-fm | transformers (SAM 2.1/3) · segment-anything (SAM 1) | Segment Anything — point / box / auto / text masks + image embedding |
| SAM 2 | model_sam_video | ac-fm | transformers | Video object tracking — propagate a first-frame prompt across a clip |
| Mask2Former | model_segmentation | ac-fm | transformers | Universal segmentation — semantic class-id maps + panoptic maps |
| Florence-2 | model_florence2 | ac-fm | transformers | Unified vision — caption / detect / ground / referring-seg / OCR via a task token |
| Depth Anything V2 | model_depth_anything | ac-fm | transformers | Monocular depth (relative / metric) |
| DepthPro | model_depthpro | ac-fm | transformers | Zero-shot metric depth (metres) + field-of-view recovery |
| Sapiens | model_normal · model_pointmap | ac-fm | transformers | Surface normals + single-image 3D pointmap (per-pixel normals / X,Y,Z) |
| SuperPoint + LightGlue | model_matching | ac-fm | transformers | Sparse keypoint detection + matching — SLAM / relocalization front-end |
| BLIP-2 | model_blip2 | ac-fm | transformers | Per-view captioning VLM (FlanT5-XL) |
| InstructBLIP | model_instructblip | ac-fm | transformers | Instruction-tuned captioning VLM |
| Qwen2.5-VL | vlm_qwen2_5_vl | ac-fm | transformers | Generic VLM — reasoning + VQA |
| Qwen3-VL | vlm_qwen3_vl | ac-fm | transformers | Generic VLM generate over images and video |
| InternVL3 | vlm_internvl3 | ac-fm | transformers | Flagship open VLM generate over images and video (-hf checkpoints) |
| Gemma 3 | vlm_gemma3 | ac-fm | transformers | Google Gemma 3 multimodal generate over images (gated model) |
| SmolVLM2 | vlm_smolvlm2 | ac-fm | transformers | Small/efficient VLM generate over images and video (256M–2.2B) |
3.2 Other sources
| Model | Nodeset | Env | Source | Capability |
|---|---|---|---|---|
| DINOv2 / DINOv3 | model_dinov2 | ac-fm | torch.hub + transformers (hf) | Self-supervised per-image features (DINOv2 hub, byte-exact; DINOv3 / HF DINOv2 via hf) |
| Grounding DINO | model_grounding_dino | ac-detany3d · ac-fm | groundingdino-py + transformers (hf_tiny) | Open-vocabulary text→box detection (+ Grounded-SAM composition) |
| RAFT | model_opticalflow | ac-fm | torchvision | Dense optical flow — per-pixel frame-to-frame motion field |
| VGGT | model_vggt | ac-vggt | upstream repo | Feed-forward 3D reconstruction — poses + depth + world point map from N views |
| VGGT-SLAM 2.0 | model_vggt_slam2 | ac-vggt-slam | upstream repo (BSD-2-Clause; Maggio & Carlone, MIT-SPARK, RSS 2026) | Dense RGB SLAM session — VGGT submaps + SALAD loop closure + GTSAM SL(4); open-set 3D queries (PE-CLIP + SAM 3) |
| pySLAM | model_pyslam | container | upstream repo (GPL-3.0; Luigi Freda) | Streaming visual-SLAM session — ORB features + DBoW3 loop closure (C++ core) behind a container bridge; plus feature / depth / semantic / multiview perception |
| CoTracker | model_cotracker | ac-cotracker | upstream repo | Point tracking through a video / frame sequence with occlusion handling |
| DetAny3D | model_detany3d | ac-detany3d | vendored | Promptable 2D / 3D detection server (also serves Grounded-SAM) |
| RAM / RAM++ | model_ram | ac-ram | recognize-anything | Swin-L image tagging (ram / ram_plus variants) |
| SpatialBot | vlm_spatialbot | ac-ram | Bunny (remote code) | Depth-aware VLM captioning (SpatialBot-3B) |
| Prismatic VLM | vlm_prismatic | ac-hmeqa | upstream repo | Token-likelihood scoring + free-form generation |
4. General components (AgentCanvas-authored)
Reusable building blocks written by AC-Team — not tied to any single method: the general-purpose agent toolkit, geometry utilities, and the learn-to-author demos / mocks.
| Component | Nodeset | What it provides | Contributor |
|---|---|---|---|
| Basic Agent | basic_agent |
Foundational VLN toolkit — note / web / image / spatial / episode-context tools | AC-Team |
| Geometry | geometry |
MeasureDistance — cross-method spatial utility |
AC-Team |
| Example (demo) | example |
Demo nodeset — every ConfigField / DisplayField pattern | AC-Team |
| State Demo (demo) | state_demo |
Minimal demonstrator for nodeset-owned state containers | AC-Team |
| Mock Frontier (mock) | mock_frontier |
[Mock] Score Frontier — random placeholder scores |
AC-Team |
| Memory nodeset | — | Episodic recall + semantic search (roadmap F1) | call for contribution |
5. Open for contribution — framework & infra
Framework / infrastructure slots that aren't nodesets or graphs. Roadmap IDs in parentheses; see the roadmap for scope.
| Item | What it adds | Contributor |
|---|---|---|
| Parallel node execution | Pregel superstep model (roadmap F3) | call for contribution |
| Export graph as standalone Python | Headless batch eval (roadmap F4) | call for contribution |
| Docker server mode | Habitat / MP3D containers (roadmap F7) | call for contribution |
| ROS nodeset | Real-robot deployment via server mode (§3 sim-to-real) | call for contribution |
6. How to get credited
Write a nodeset (a group of canvas nodes) or compose a graph, and open a PR adding it under workspace/. See CONTRIBUTING § PR format for the checklist. When it's merged:
- You're added to this board and the README Credits as the component's author/maintainer.
- If your nodeset reproduces a published method, include the citation in your PR — it's carried onto the Paper column here.