Codebase Map β agentcanvas
Every file in agentcanvas/ in one place. Navigation reference for finding where a behaviour lives.
app/) contains ~22,800 lines of Python (tests excluded) across 10 packages + root files. The frontend (src/) is React 18 + TypeScript + React Flow + Zustand. Line counts in tables below are approximate snapshots and drift between updates β trust the structure (which file holds what role), not the digits.
agent_loop/ (executor, scope analysis, loop runner β the largest package, Β§3) Β·
services/ (job scheduler, watchers, resource sampler β Β§9) Β·
standard/ (wire types, actions, node I/O β Β§8) Β·
replay/ (episode replay backend β Β§9) Β·
logging/ (execution log layer β Β§7) Β·
tools/ (standalone graph validator β Β§9). Every package now has its own section below; tests (test_*.py) are out of scope throughout.
Β§1Backend: Root Files
Entry points, core singletons, and domain modules that live at the package root. Imported by nearly every sub-package β they stay at the top level to avoid circular imports.
1.1 Entry Points
| File | Lines | Role | |
|---|---|---|---|
__init__.py | 0 | Package marker | |
__main__.py | 3 | python -m app entry point β calls llm.cli.main() | |
main.py | 276 | FastAPI application factory β creates the app, registers CORS middleware, mounts 15 API routers across 5 domain sub-packages (canvas, execution, platform, registry, replay), defines lifespan (scans workspace/ on startup via WorkspaceComponentRegistry, starts the JobScheduler tick loop + file watchers + ResourceSampler, shuts down nodesets on exit) |
1.2 Core Singletons
Small files imported everywhere β config, state, models, and the core data model.
| File | Lines | Role | |
|---|---|---|---|
config.py | 42 | Settings singleton β Pydantic BaseSettings loaded from .env: host, port, workspace_dir, Habitat-specific settings. Immutable after init | |
state.py | 110 | ProcessServices + ExecutionGuard + broadcast() β ProcessServices holds WorkspaceComponentRegistry; ExecutionGuard is a thread-safe mutex preventing concurrent canvas + eval (ExecutionMode: idle/canvas/eval); broadcast() sends WSMessage to all connected WebSocket clients | |
models.py | 17 | WSMessage β single Pydantic model for all WebSocket messages (type, data, timestamp, execution_id, source) | |
graph_def.py | 388 | The core data model β pure dataclasses mirroring frontend types.ts: GraphDefinition, NodeDef, EdgeDef, ContainerDef, StateDef, AccessGrantDef, HookDef. JSON-serializable, no live state |
1.3 Domain Modules
Single-file modules with one consumer each β candidates for future sub-packages as their domains grow.
| File | Lines | Role | |
|---|---|---|---|
layout.py | ~750 | Auto-layout algorithm β two-pass: topological layering (leftβright X assignment) + semantic post-processing (viewer stacking, container placement). Also works as CLI: python -m app.layout graph.json | |
errors.py | ~300 | ErrorBus singleton + ErrorEnvelope dataclass (ADR-observability-004). 200-entry ring buffer + 500-deep async dispatch queue, broadcasts error_event WS frames | |
eval_subprocess_main.py | ~310 | Subprocess-per-run entry (ADR-eval-003) β python -m app.eval_subprocess_main --run-dir ...: reads spec.json + shared_urls.json, attaches to parent backend's shared singletons, runs BatchEvalRunner.execute() unmodified, writes _DONE marker on clean exit |
env_controller.py + its HabitatEnvController were superseded by the generic BaseEnvPanel contract in app/components/env_panel.py (ADR-server-002). Episode control is now a nodeset-side concern, not a framework module.
Β§2Backend: API Layer
Grouped into 5 domain sub-packages. Each module exports a router = APIRouter() that main.py mounts.
2.1 Canvas β graph editing + environment control
| File | Lines | Route prefix | Role |
|---|---|---|---|
graphs.py | 165 | /api/graphs | Graph CRUD β list, load, save, delete graph JSON files in workspace/graphs/ and workspace/graph_nodes/. Also POST /layout for auto-layout |
env_panel.py | 175 | /api/env-panels | Episode control β generic env-panel REST API: list panels, state, dynamic options, field change, action invoke (POSTs guarded by ExecutionGuard). Serves any nodeset's registered BaseEnvPanel |
2.2 Execution β run, eval, logs, websocket
| File | Lines | Route prefix | Role |
|---|---|---|---|
run.py | 139 | /api/navigate | Canvas graph execution β POST /run receives GraphDefinition JSON, creates LoopRunner, runs via GraphExecutor. Plus /stop, /pause, /resume, GET /node-schemas |
eval.py | 558 | /api/eval/v2 | Batch eval API β POST /start (enqueue a run on JobScheduler; the admitted run executes BatchEvalRunner in its own subprocess), GET /status, POST /stop, GET /runs, DELETE /runs/{id} |
eval_storage.py | 106 | β | Eval run persistence β saves/loads/deletes eval run results as JSON in outputs/eval_runs/ |
logs.py | 270 | /api/logs | Execution log retrieval β list runs, full run, per-node, summary. Reads JSONL from outputs/runs/ and outputs/eval_runs/ |
websocket.py | 47 | /ws | WebSocket endpoint β accepts connections, registers/unregisters clients with state.py. Broadcast logic lives in state.broadcast() |
internal_containers.py | 136 | /api/internal | Cross-process container broker (not a public API) β lets server-mode subprocess nodes read/write state containers regardless of which process homes them; called by server/remote_container.py::RemoteContainerProxy, msgpack payload path |
internal_events.py | 72 | /api/internal | Subprocess event push β receives structured log/error events POSTed by server-mode subprocesses (server/event_push.py) and republishes them on the ErrorBus β error_event WS frame |
2.3 Platform β config, components, LLM profiles
| File | Lines | Route prefix | Role |
|---|---|---|---|
config.py | 41 | /api/config | Settings β GET / returns current Settings + active LLM profile info |
components.py | 238 | /api/components | Component registry β nodeset list/load/unload, POST /reload (hot-reload workspace/), /eval-metadata |
profiles.py | 248 | /api/profiles | LLM profile management β CRUD, POST /test-key, GET /providers (24 providers), GET /models/{provider} |
errors.py | 32 | /api/errors | Error backfill β returns the ErrorBus ring buffer so the frontend can backfill envelopes published while disconnected |
system.py | 53 | /api/system | System usage β /usage (latest sample) + /history from the always-on ResourceSampler (Β§9) |
2.4 Registry β component snapshot
| File | Lines | Route prefix | Role |
|---|---|---|---|
snapshot.py | 43 | /api/registry | Shared-nodeset snapshot β read-only view of currently-loaded shared nodesets; fallback for run subprocesses that started without a shared_urls.json (the normal path: JobScheduler writes it at submit time), also used by tests |
2.5 Replay β episode replay API
| File | Lines | Route prefix | Role |
|---|---|---|---|
router.py | 207 | /api/replay | Replay API β serves the env-agnostic replay timeline parsed from a run's episode logs (backend package app/replay/, Β§9); consumed by the frontend Replay page (Β§17) |
Β§3Backend: Execution Engine
The core runtime that turns a GraphDefinition JSON into a live agent run. All execution goes through this package. No HTTP/FastAPI imports β pure Python.
3.1 Execution Pipeline
When the user presses Play on the canvas, this is the call chain:
3.2 File by File
loop_runner.py216 lines
The outermost lifecycle shell β first thing called from the API. Owns the lifecycle: pause, stop, resume.
- Receives a
GraphDefinitionJSON from the frontend - Calls
flatten_graph()to expand composites into a flat graph - Builds
StateContainerinstances from the graph's container definitions - Creates a
GraphExecutorand calls.forward() - Manages pause/stop/resume via asyncio events
ExecutionPrinciplesdataclass configures batch eval mode
Does not know how nodes fire β sets up the world and hands control to the executor.
flatten.py300 lines
Pre-processing step. Runs before execution starts. The canvas supports nested graphs (composite nodes containing sub-graphs), but the executor needs a flat list of nodes and edges.
- Recursively walks composite nodes that have a
subgraph - Expands them into the parent graph, rewiring
graphIn/graphOutboundary nodes to parent edges - Prefixes node IDs to avoid collisions (
composite1.inner_node) - Returns a
FlattenMapso error messages can trace back to the original composite context
graph_executor.py1,987 lines Β· largest file
Where nodes actually execute. Implements a dataflow FIFO queue model:
- Build: wraps each
NodeDefinto aNodeInstance(tracks pending inputs per port) - Seed: finds nodes with no required inputs (environment nodes, constants) and enqueues them
- Loop: pull a node from queue β execute β deliver outputs to downstream ports β if downstream has all required inputs, enqueue it
- Fan-out: when a node produces output, all target ports are filled before any downstream node fires
- Fan-in: a node only fires when all required input ports have data
- Cycles:
IterOutNodecopies its inputs to the pairedIterInNodeand re-enqueues it - Termination: IterOut's
stopinput port goes true (Decide phase), step budget reached, stop event set, empty queue, or a node-raisedStopExecution
Each node fires by calling NODE_HANDLERS[node_type].forward(inputs, ctx).
scope_analysis.py416 lines
Computes the scope forest before execution starts.
analyze_scopes()β walks the flat graph and returns every(iterIn, iterOut)scope plus its parent/child topology (ADR-dataflow-007)- Classifies composite boundaries:
graphIn= parameter slot,graphOut= return slot of a scope - Validates scope shape (inner scopes bounded by mandatory graphIn/graphOut) β feeds the executor's per-scope
_ScopeState
builtin_nodes.py1,369 lines
The node handler registry and the home for all built-in framework nodes (see TODO #37).
Registry:
NODE_HANDLERS: dict[str, type[BaseCanvasNode]]β global registry mappingnode_typestring to Python classregister_node(cls)β called byWorkspaceComponentRegistrywhen loading nodesetsExecutionContextdataclass β mutable state passed to every node
Built-in node classes (14):
| Node | What it does |
|---|---|
IterInNode / IterOutNode | Loop gates β IterOut sends data back to IterIn for next iteration; iterIn's left side captures run-start initPorts, iterOut's final side feeds the after-loop band |
LLMCallNode | Calls llm_complete() / vlm_complete() with a prompt, returns text |
TextParseNode | Generic structured parser for LLM response text |
HistoryLogNode | Accumulates one TEXT entry per firing, emits the formatted history |
GraphInNode / GraphOutNode | Composite boundaries β parameter slot / return slot (formerly PortIn/PortOut); graphOut doubles as the sink that buffers _last_inputs for the nav_step broadcast |
NullSourceNode | Emits None on a typed output port β once, at run-start |
NoteNode | Markdown billboard β pure annotation, no inputs/outputs |
ImageViewerSink | Viewer β displays images on canvas via viewer_data WS events (configurable grid, imageGrid layout) |
TextViewerSink / TextScrollSink / ActionLogSink / MetricsViewerSink | Viewers for latest TEXT / accumulating TEXT / action history / SR/SPL/NE metrics |
state_containers.py469 lines
Implements the dual-wire system: nodes communicate via data edges (typed values) and state edges (access grants to shared containers).
BaseStateABC with 3 reducer implementations (ephemeral removed 2026-04-18 β uselifetime="step"):AccumulatorState(append),LastWriteState(overwrite),CounterState(increment)BaseState.on_signal(name, payload)β clears toinitial_valuewhen a name inreset_onfiresStateContainerβ a dict of namedBaseStateentries- Factory function builds containers from
ContainerDefin graph JSON - Executor injects connected containers into
ctx._containersbefore each node fires - Nodeset-owned / keyed containers β
BaseNodeSet.get_containers()lets a nodeset own containers (withallow_opaquevalues and per-key evict for worker parallelism), alongside graph-level containers
eval_batch.py827 lines
Wraps LoopRunner for batch evaluation:
BatchEvalRunnertakes a graph + episode iterator- For each episode: leases a
WorkerHandlefromEnvWorkerPool, creates a freshLoopRunner, sets the episode, runs underasyncio.wait_for(timeout=max_steps Γ per_step_budget_sec), collects metrics - At
worker_count>1: sharedasyncio.Queueof episode indices drained by N coroutines underasyncio.gather, results aggregated underasyncio.Lock, sorted byepisode_indexbefore persistence - Emits WS progress events (
eval_progress,eval_episode_done) tagged withworker_id - Respects
ExecutionGuardβ only one eval or canvas run at a time
env_worker_pool.py244 lines
Sits between BatchEvalRunner and LoopRunner (ADR-eval-002 PA-3 + PB).
EnvWorkerPool(worker_count=N, env_nodeset=name)β async context manager.__aenter__populates NWorkerHandles; atworker_count>1each binds to a taggedRemoteEnvPanelProxy+ tagged URLacquire()β async context manager that leases oneWorkerHandleper episodeWorkerHandlecarriesenv_panel_overridesandserver_url_overridesconsumed by the leasedLoopRunner's executor for per-worker routingresolve_per_step_budget(env_nodeset, override)β resolves the per-step wall-clock budget
hooks.py307 lines
Fires shell commands at graph/node boundaries β like git hooks for graph execution.
- 5 events:
GraphStart,GraphComplete,GraphError,PreNodeExecute,PostNodeExecute - Protocol: event payload sent as JSON on stdin β hook process β JSON response on stdout with
{"action": "continue"|"block"|"modify"} - Fail-open: any error (timeout, crash, malformed JSON) defaults to
action="continue" HookRunnerclass manages execution;load_hooks_file()loads hook definitions from YAML
__init__.py17 lines
PolicyEntry dataclass: metadata for a loadable neural policy checkpoint (id, name, checkpoint path, config path). Used by WorkspaceComponentRegistry when scanning workspace/policies/.
3.3 Layering
GraphExecutor decides when a node fires (input readiness). NODE_HANDLERS decides what happens (domain logic). Adding a new node type is just subclassing BaseCanvasNode in workspace/ β the executor never changes.
Β§4Backend: Component System
The public API for node authors. Files in workspace/ import from this package.
| File | Lines | Role | |
|---|---|---|---|
bases.py | 554 | Base classes for all node authoring β PortDef, ConfigField (7 widget types incl. port_list), DisplayField (4 display types: image_viewer, log_list, metric_table, text_viewer), NodeUIConfig, BaseCanvasNode (abstract: forward(), ports, category, icon, _self_log(), batched/batch_dim), BaseNodeSet (loadable tool group; server_python, parallelism, env_panel, default_per_step_budget_sec, replay_parser ClassVars) | |
registry.py | 1,955 | WorkspaceComponentRegistry β scans workspace/ subdirectories, imports Python files, discovers BaseCanvasNode/BaseNodeSet subclasses via inspect. Registers in NODE_HANDLERS. Handles load/unload lifecycle, server-mode auto-routing, hook file loading, graph auto-loading, register_remote_nodeset (subprocess attach to shared singletons), container-ownership checks | |
env_panel.py | 250 | BaseEnvPanel β control-plane panel base for env-side runtime knobs (suite / split / episode, play / pause / reset); declared via the BaseNodeSet.env_panel ClassVar, served at /api/env-panels/*, bridged into server-mode subprocesses by RemoteEnvPanelProxy. Renamed from "controller" 2026-06-11 (ADR-server-002) | |
content_hash.py | 95 | Content hashing for nodeset source trees β lets /api/eval/v2/start detect whether an active_workspace_dir overlay actually changed a shared nodeset's source vs. the frozen baseline | |
__init__.py | 24 | Re-exports: BaseCanvasNode, BaseNodeSet, PortDef, ConfigField, DisplayField, NodeUIConfig β the public import path |
Β§5Backend: Server SDK
Implements the write-once deploy-anywhere server mode (ADR-server-001). A BaseNodeSet subclass runs identically in-process or as an auto-hosted HTTP server. Three roles: server-side, framework-side, bridge.
| File | Lines | Role | |
|---|---|---|---|
server_app.py | 184 | ServerApp (server-side) β builds a FastAPI HTTP service speaking the manifest protocol: GET /manifest, POST /call/{fn}, GET /health | |
auto_server_app.py | 204 | AutoServerApp (server-side) β auto-generates a ServerApp from any BaseNodeSet by introspecting port shapes. Owns one BatchedInferenceServer per app; _make_handler dispatches batched=True nodes through the rendezvous | |
batched_inference.py | 196 | Batched-inference rendezvous tier. BatchedInferenceServer owns one _BatchQueue per (function_name, config_hash) β collects K parallel callers, calls underlying handler once, scatters K result slices. Pure-functional contract | |
base_server.py | 283 | BaseServer (framework-side) β launches subprocess, polls /health, handles start/stop/restart, fetches manifest. Manages process lifecycle and port allocation | |
nodeset.py | 192 | ServerNodeSet β combines BaseNodeSet + BaseServer: loadable group of canvas nodes backed by an external server process | |
proxy.py | 149 | Proxy node generator (bridge) β reads ServerManifest and auto-generates BaseCanvasNode subclasses whose forward() forwards inputs over HTTP. Canvas can't tell the difference | |
manifest.py | 93 | Manifest protocol types β PortSchema, FunctionSchema, ServerManifest | |
serialization.py | 300 | Wire serialization β msgpack codec (pack_body/unpack_body, ADR-server-004): one blob ExtType carries raw ndarray/torch.Tensor/PIL.Image bytes with receiver-side degrade; JSON+base64 legacy path kept for the migration window | |
event_push.py | 144 | Subprocess β executor event push β forwards structured node logs + first-classed handler exceptions from a server-mode subprocess to the parent's /api/internal/events, so server-node errors surface on the canvas (roadmap #54 Move 3) | |
remote_container.py | 75 | RemoteContainerProxy β subprocess-side handle to an executor-home state container; calls the /api/internal broker so access-granted server nodes can read/write home containers | |
_loopback_proxy.py | 39 | Loopback proxy policy β keeps httpx calls to localhost auto-host children from being routed through a user's HTTP_PROXY/HTTPS_PROXY shell env | |
auto_host.py | 60 | CLI entry point β python -m app.server.auto_host --file nodeset.py --class MyNodeSet --port 9200 | |
examples/habitat_server.py | 230 | Reference example β hand-written ServerApp for Habitat-Sim (pre-dates AutoServerApp). Kept as documentation | |
__init__.py | 30 | Re-exports: AutoServerApp, BaseServer, ServerApp, ServerFunction, ServerNodeSet, manifest types |
Β§6Backend: LLM System
Self-contained package for the LLM/VLM subsystem. __init__.py re-exports the public API so external imports work unchanged.
| File | Lines | Role | |
|---|---|---|---|
__init__.py | 34 | Re-exports: llm_complete, vlm_complete, get_llm_config, LLMConfig, get_profile_store, LLMProfile, PROVIDER_REGISTRY, resolve_provider_config | |
call.py | 251 | LLM/VLM call client β llm_complete() (text-only) and vlm_complete() (multimodal) via litellm. Resolves provider config from active profile using the 3-layer fallback chain (ADR-platform-003). Supports 100+ providers through litellm | |
profiles.py | 157 | ProfileStore β persistent storage at ~/.agentcanvas/profiles.json. Thread-safe with RLock. Mtime-based cache invalidation | |
providers.py | 117 | Provider registry β static dict mapping 24 provider IDs to ProviderDef (label, base_url, api_type, default_model, litellm_prefix). Covers: OpenAI, Anthropic, Google, Ollama, DeepSeek, Together, OpenRouter, Mistral, xAI, NVIDIA NIM, Hugging Face, Moonshot, and more | |
key_validator.py | 94 | API key validation β tests if an API key is valid via a minimal litellm completion request | |
cli.py | 478 | Standalone CLI tool β 8 argparse commands: list, add, remove, set-active, test, show, setup, models |
6.1 Fallback Chain (ADR-platform-003)
Β§7Backend: Logging
Two-layer logging system (ADR-observability-003): the executor captures port I/O automatically (exterior); nodes add domain detail via _self_log() (interior).
| File | Lines | Role | |
|---|---|---|---|
logger.py | 342 | ExecutionLogger β captures two layers per node firing: automatic exterior (executor records all port inputs, outputs, timing) + voluntary interior (nodes call _self_log(key, value)). log_serialize() truncates large values. Persists as JSONL. Broadcasts exec_log WS events (suppressed in eval batch mode) | |
models.py | 39 | Log data models β NodeLogEntry (one node firing) and ExecutionSummary (aggregate) | |
__init__.py | 7 | Re-exports: ExecutionLogger, ExecutionSummary, NodeLogEntry |
Β§8Backend: Standard Library
Shared type definitions and constants used by node implementations and the executor. This is a leaf package with no intra-app dependencies.
| File | Lines | Role | |
|---|---|---|---|
wire_types.py | 443 | Wire type registry β 9 current types (IMAGE, DEPTH, POSE, DISCRETE_ACTION, CONTROL, TEXT, BOOL, METRICS, ANY) + 3 deprecated aliases (ACTION, OBSERVATION, STEP_RESULT) kept so legacy graphs load, + LIST[T] modifier, WIRE_FORMAT_SPEC canonical on-wire formats, display colours. Helpers: image_to_base64(), depth_to_base64(), serialize_for_display() | |
actions.py | 40 | Action constants β ACTION_STOP, ACTION_FORWARD, ACTION_TURN_LEFT, ACTION_TURN_RIGHT (the VLN Discrete(4) space). parse_action_from_text() extracts an action from LLM output | |
node_io.py | 168 | Auto-generated node I/O schema β NODE_IO_SCHEMA + REQUIRED_INPUTS built lazily from the NODE_HANDLERS registry (no handler classes live here). Used by GraphExecutor for fan-in logic | |
__init__.py | 9 | Re-exports everything from all 3 modules |
Β§9Backend: Services, Replay & Tools
Long-running backend-process services owned by the FastAPI lifespan: job scheduling, file watching, resource sampling.
| File | Lines | Role | |
|---|---|---|---|
job_scheduler.py | 941 | JobScheduler β admission + FIFO queue + Popen pool for eval run subprocesses (ADR-eval-003). Singleton owned by ProcessServices; one tick() per second admits queued jobs through the per-resource measured gate (calibrated estimate vs measured free VRAM/RAM minus reservations; declared fallback) while the canvas ExecutionGuard lock is free; spawns python -m app.eval_subprocess_main with PR_SET_PDEATHSIG | |
resource_stats.py | 739 | ResourceStatsTracker β per-owner resource attribution (VRAM per-PID from the sampler, RAM RSS from one /proc walk β shared server | job tree | external), per-run observation windows, calibration store (outputs/system/resource_calibration.json, EWMA + max per resource, source-hash keyed) and the admission estimator behind GET /api/eval/v2/estimate | |
graph_watcher.py | 73 | Graph-directory watcher β pushes graphs_changed when graph JSON files change on disk, so coding-agent edits made outside the web UI show up in the Explorer without a manual refresh | |
nodeset_watcher.py | 156 | Nodeset-source watcher β hot-reloads nodeset .py edits without a backend restart; companion to graph_watcher | |
resource_sampler.py | 346 | ResourceSampler β always-on ~1 Hz machine resource time-series written to outputs/system/system-YYYYMMDD.jsonl; feeds /api/system/usage + /history and the Monitor page (Β§17) | |
run_state_io.py | 137 | Run-dir file protocol β the contract between JobScheduler (parent) and eval_subprocess_main (subprocess) under outputs/eval_runs/{run_id}/: spec.json, shared_urls.json, _DONE, β¦ | |
system_runs.py | 279 | System Log per-run aggregation β joins a run's execution logs with tee'd resource samples into a per-run performance summary for the Monitor page's Run view |
Episode replay backend β turns persisted episode logs into an env-agnostic replay timeline (served by api/replay/, Β§2.5; consumed by the frontend Replay page, Β§17).
| File | Lines | Role | |
|---|---|---|---|
interface.py | 292 | Replay interface β env-agnostic timeline schema + BaseReplayParser ABC; an env nodeset opts into replay by pointing its replay_parser ClassVar at a parser module (pure log-walking, no simulator imports) | |
renderer_client.py | 144 | Renderer client β lazy-spawns and calls a renderer subprocess for parsers' smooth-mode frame requests; held warm for the FastAPI process lifetime | |
renderer_host.py | 61 | Renderer host β launches a renderer class from a file path (mirrors server/auto_host.py but only needs a build_app() method) |
| File | Lines | Role | |
|---|---|---|---|
validate_graph.py | 226 | Standalone graph wire-type validator (dev tool) β loads a graph from disk, runs the same app.graph_def.wire_type_report core the backend uses, prints a report, exits non-zero on any mismatch. No backend needed |
Β§10Backend: Dependency Flow
How the backend packages depend on each other. Arrows point from importer to imported.
- Leaf packages (no intra-app deps):
standard/,logging/,llm/ - Core data model:
graph_def.pyβ imported by everything - Cross-cutting:
state.broadcast()β used by bothagent_loop/andapi/execution/websocket.py. The engine no longer imports from the API layer - Diagram shows the three largest of the five
api/sub-packages;api/registry/andapi/replay/(Β§2.4β2.5) are omitted for legibility, as areservices/,replay/, andtools/(Β§9)
Β§11Frontend: App Shell
| File | Role | ||
|---|---|---|---|
main.tsx | React entry point β mounts <App /> into #root | ||
App.tsx | Top-level router β switches between CanvasPage, EvalPage, LogViewerPage, NodeSetManager based on appMode | ||
store.ts | Zustand global store β connected, appMode, eval run state, nav status, env context bar data | ||
api.ts | REST client β fetchJ, postJ, deleteJ, putJ wrappers around fetch with JSON handling | ||
ws.ts | WebSocket manager β connects to /ws, dispatches typed events to listeners; reconnects on drop | ||
types.ts | Global TypeScript types β NavStatus, EvalRunSummary, AppMode, EnvContextState | ||
errorStore.ts | useErrorStore β Zustand store of ErrorEnvelopes fed by error_event WS frames; backfills from GET /api/errors on (re)connect | ||
errors.ts | ErrorEnvelope types + severity helpers mirroring the backend ErrorBus schema | ||
vite-env.d.ts | Vite environment type declarations |
Shared components (src/components/)
Header.tsx | Top app header β brand, appMode segmented toggle, execution-mode badge (polls /api/navigate/execution-mode), settings-gear entry | ||
SettingsModal.tsx | Settings modal β provider API keys, active provider + default model, profiles/favorites; reads/writes /api/config + /api/profiles | ||
ErrorToast.tsx | Floating toast layer β auto-pops fresh error/warning ErrorEnvelopes (max 3, auto-dismiss); click focuses the Report tab | ||
ViewPanel.tsx | Static placeholder "Views" panel (RGB / Depth tiles) β presentational stub, not wired to data |
Β§12Frontend: Canvas Editor
Core graph editing workspace. React Flow drives the visual layer; a Zustand store owns all graph state.
| File | Role | ||
|---|---|---|---|
CanvasPage.tsx | Canvas workspace page β tab management, error boundary, layout scaffold | ||
UnifiedGraphEditor.tsx | Core editor β root graph mode + subgraph drill-down mode; React Flow instance owner | ||
useFlowStore.ts | Zustand canvas store β tab-aware graph state, node/edge CRUD, selection, execution status per tab | ||
types.ts | GraphDefinition, NodeDef, EdgeDef, ContainerDef, StateDef, AccessGrantDef β mirrors backend schema | ||
graphConversion.ts | Bidirectional conversion between GraphDefinition (backend JSON) and React Flow Node[]/Edge[] | ||
defaultGraph.ts | Template graphs loaded for new tabs (Straightforward CMA, NavGPT-CE) | ||
portResolution.ts | resolveInstancePorts() β applies config.ports overrides to class-level port schemas | ||
unifiedNodeTypes.ts | Node type registry β maps node type strings to React components for React Flow | ||
unifiedCatalog.ts | Component catalog β fetches node schemas from backend; drives the node library sidebar | ||
nodesetLoader.ts | NodeSet loader β fetches loaded nodesets list; triggers nodesets-changed refresh pipeline | ||
groupSelection.ts | Multi-select group logic β bounding box selection, group move | ||
flowStoreRef.ts | Bridge β holds a stable ref to the React Flow instance accessible outside React tree | ||
runPipeline.ts | runPipeline() β builds RunRequest, calls /api/run, manages execution lifecycle from frontend side | ||
edgeTypes.ts | Custom edge type registry β registers AccessGrantEdge alongside default React Flow edge types | ||
edges/AccessGrantEdge.tsx | Renders dashed violet access-grant edges (renamed from StateEdge.tsx with the state-edge β access-grant terminology shift, ADR-dataflow-004) | ||
ShortcutCheatsheet.tsx | Keyboard-shortcut cheatsheet overlay | ||
useKeyboardShortcuts.ts | Global keyboard-shortcut hook β Space = play/pause, F = fit view, ? = cheatsheet; ignores typing targets |
Β§13Frontend: Canvas Nodes
The renderer tree that turns Python-defined NodeUIConfig into visible canvas nodes.
Shared primitives
shared/NodeShell.tsx | Generic node wrapper β selection ring, drag handle, resize handle | ||
shared/HandleDot.tsx | Port handle dot β colour-coded by wire type, tooltip with port name/type |
Special node types
state/StateContainerNode.tsx | Renders a StateContainer on canvas β dashed violet border, database icon, reducer badges | ||
composite/CompositeCanvasNode.tsx | Renders a composite (nested graph) node β port boundaries, drill-down button | ||
composite/CompositeNodeView.tsx | Editing view inside a composite node β full sub-graph editor in a panel |
Generic renderer stack (agentloop/inner/)
All non-special nodes flow through this stack, driven entirely by NodeUIConfig from Python.
GenericBlockRenderer.tsx | Dispatcher β reads ui_config.layout, routes to BlockLayout, StripLayout, ViewerLayout, NoteLayout, or ImageGridViewerLayout | ||
layouts/BlockLayout.tsx | Standard block layout β title bar + config fields + port handles | ||
layouts/StripLayout.tsx | Horizontal strip β compact port-only view for pass-through nodes | ||
layouts/ViewerLayout.tsx | Viewer layout β display-only fields for output sink nodes | ||
layouts/ImageGridViewerLayout.tsx | imageGrid layout β rowsΓcols image grid for the configurable imageViewer (ADR-components-007) | ||
layouts/NoteLayout.tsx | note layout β markdown billboard rendering for the note built-in | ||
layouts/IterInExpandPanel.tsx | iterIn expand panel β edits the two-sided iterIn's initPorts (with per-port persist checkboxes) and shows the synthesised init/iterOut port surface; edits resync handles live | ||
layouts/ConfigFieldRenderer.tsx | Renders a single ConfigField β label, slider, text, select, toggle, textarea, port_list | ||
layouts/PortListEditor.tsx | port_list config field widget β add/remove/rename ports for IterIn/IterOut | ||
layouts/layoutUtils.ts | Shared layout helpers β port position calculation, label truncation | ||
layouts/useNodeOutput.ts | React hook β subscribes to viewer_data WS events for a given node_id | ||
layouts/viewerFieldRegistry.ts | Registry of display field renderers keyed by DisplayField.display_type |
Display field renderers (layouts/fields/)
ImageViewerField.tsx | Renders base64 image frames from viewer_data events | ||
TextViewerField.tsx | Renders plain text output (LLM responses, debug strings) | ||
LogListField.tsx | Scrolling log list with accumulate=true appending behaviour | ||
MetricTableField.tsx | Key-value metric table (SR, SPL, NE, etc.) |
Β§14Frontend: Canvas Panels
Sidebar and toolbar UI components that surround the React Flow canvas.
TabBar.tsx | Tab strip for open graphs β add, close, switch, unsaved-change indicator | ||
ExplorerPanel.tsx | VS Code-style tree β Graphs / Graph Nodes / Nodes sections, drag-to-canvas | ||
NodeLibrary.tsx | Draggable node palette β shows only built-in + currently loaded nodeset nodes | ||
PropertiesPanel.tsx | Selected node property editor β config field forms, port visibility | ||
StatePanel.tsx | State container editor β add/remove named state entries, set reducer type | ||
OutputDrawer.tsx | Bottom drawer β Observation / Logs / Step Log / State / Report tabs | ||
LogPanel.tsx | Execution log tab β lists exec_log events grouped by node | ||
StepLogPanel.tsx | Per-step log tab β full I/O trace for the current step | ||
ExecutionToolbar.tsx | Run / Pause / Stop / Step buttons + execution status badge | ||
EnvPanel.tsx | Env panel strip (renamed from EnvContextBar.tsx; episode browsing folded in) β renders any nodeset's BaseEnvPanel from /api/env-panels: suite/split selectors, episode navigator, panel actions | ||
ObservationPanel.tsx | Current observation display β image + instruction text | ||
ReportPanel.tsx | Report tab β time-ordered ErrorEnvelope list with severity filter chips, click-to-expand (ADR-observability-004) | ||
ResizableBottomPanel.tsx | Drag-resizable wrapper for the bottom drawer | ||
GraphStateBanner.tsx | Banner shown during/after execution β run status, error messages | ||
TemplatePicker.tsx | Graph template picker modal β choose preset to load into new tab | ||
SaveGraphDialog.tsx | Save graph / save-as-node dialog β name, group, kind selection |
Β§15Frontend: Evaluation Page
Batch evaluation launcher and live monitor (ADR-eval-001).
EvalPage.tsx | Top-level eval page β orchestrates all sub-components, subscribes to WS eval events | ||
evalApi.ts | Eval API client β startEval(), stopEval(), getRunHistory() wrappers | ||
types.ts | Eval-specific TypeScript types β EvalRun, EpisodeResult, EvalStatus | ||
GraphSelector.tsx | Graph picker β lists saved graphs, shows nodeset requirements | ||
EnvInfoPanel.tsx | Environment info panel β split counts, episode counts, dataset source | ||
EvalProgressBar.tsx | Progress bar β episodes done / total with estimated completion | ||
MetricCards.tsx | Live metric summary cards β SR, SPL, NE updated after each episode | ||
EpisodesTable.tsx | Per-episode results table β instruction, predicted path, metrics | ||
RunHistory.tsx | Previous runs list β load past run results for comparison |
Β§16Frontend: Log Viewer
Dedicated page for post-hoc inspection of execution logs (ADR-observability-003).
LogViewerPage.tsx | Full-page log viewer β split view: run list + detail pane | ||
logApi.ts | Logs API client β listRuns(), getRunLog(), getNodeLog() | ||
types.ts | Log-specific types β RunRecord, NodeLogEntry, PortSnapshot | ||
LogContext.tsx | React context β selected run, filter state, pagination | ||
LogCanvasView.tsx | Canvas-style log visualisation β replay node firing order on a static graph | ||
ExecutionList.tsx | Left panel β paginated list of past execution runs | ||
LogEntryList.tsx | Right panel β paginated log entries for a selected node | ||
LogSummaryBar.tsx | Summary bar β total nodes fired, duration, error count | ||
LogFilters.tsx | Filter controls β filter by node type, wire type, has-error | ||
nodeColors.ts | Node type to colour mapping for log canvas |
Renderers (logs/renderers/)
Each renderer handles one value type in a log entry's port snapshot.
registry.ts | Renderer registry β maps wire_type strings to renderer components | ||
PrimitiveRenderer.tsx | Numbers, booleans, short strings | ||
TextRenderer.tsx | Long text β LLM responses, prompts | ||
ImageRenderer.tsx | Base64 image frames | ||
JsonTreeRenderer.tsx | Collapsible JSON tree for dicts/lists | ||
ActionRenderer.tsx | ACTION wire β discrete action label + direction arrow | ||
ErrorRenderer.tsx | Error traces β red-highlighted stack output | ||
TensorRenderer.tsx | NumPy array β shape, dtype, value preview | ||
MetricsRenderer.tsx | METRICS wire β key-value metric table | ||
StepResultRenderer.tsx | STEP_RESULT wire (deprecated type) β done flag, info dict, reward | ||
ImageListRenderer.tsx | LIST[IMAGE] β thumbnail strip for multi-image port values |
Β§17Frontend: Replay & Monitor
Two post-hoc inspection pages beyond the log viewer: episode replay and system/run performance monitoring.
replay/ReplayPage.tsx | Episode replay page β steps through the env-agnostic replay timeline served by /api/replay (Β§2.5), with frame scrubbing | ||
replay/replayApi.ts | Replay API client β timeline + frame fetch wrappers | ||
pages/monitor/MonitorPage.tsx | Monitor page β machine resource sparklines (polls /api/system/usage, Β§2.3) + the JobScheduler queue view (polls /api/eval/v2/queue) + per-run performance view (System Log) | ||
pages/monitor/NodeTimingTable.tsx | Per-node timing table β where a run's wall-clock went, node by node | ||
pages/monitor/Sparkline.tsx | Inline sparkline chart for resource/timing series |
Β§18Frontend: NodeSet Manager
NodeSetManager.tsx | Full NodeSet manager page β two sections: Local NodeSets (load/unload), Server Mode (start/stop/restart) | ||
manager/NodeSetCard.tsx | Card for one local NodeSet β name, status badge, load/unload button | ||
manager/ServerCard.tsx | Card for one server-mode nodeset β URL, health status, start/stop/restart controls |
Β§19Config and Build
frontend/package.json | NPM manifest β React 18, @xyflow/react, Zustand, Tailwind, Vite, Three.js | ||
frontend/vite.config.ts | Vite config β API proxy (/api to localhost:8000), React plugin | ||
frontend/tsconfig.json | TypeScript config β strict mode, path aliases | ||
backend/requirements.txt | Python deps β FastAPI, Uvicorn, Pydantic v2, LiteLLM, httpx, python-multipart | ||
backend/.env.example | Environment variable template β AGENTCANVAS_* keys, port config | ||
run_dev.sh | Dev launcher β starts backend (uvicorn) on :8000 and frontend (vite) on :5173 in parallel |