Codebase Map β agentcanvas
Every file in agentcanvas/ in one place. Navigation reference for finding where a behaviour lives.
app/) contains ~28,400 lines of Python (tests excluded) across 10 packages + root files. The frontend (src/) is React 18 + TypeScript + React Flow + Zustand. Line counts in tables below are approximate snapshots and drift between updates β trust the structure (which file holds what role), not the digits.
agent_loop/ (executor, scope analysis, loop runner β the largest package, Β§3) Β·
services/ (job scheduler, watchers, resource sampler β Β§9) Β·
standard/ (wire types, actions, node I/O β Β§8) Β·
replay/ (episode replay backend β Β§9) Β·
logging/ (execution log layer β Β§7) Β·
tools/ (standalone graph validator β Β§9). Root-file additions since then: the Graph SDK cluster (graph_sdk*.py, Β§1.4) plus the agentcanvas public import package next to app/. Every package now has its own section below; tests (test_*.py) are out of scope throughout.
Β§1Backend: Root Files
Entry points, core singletons, and domain modules that live at the package root. Imported by nearly every sub-package β they stay at the top level to avoid circular imports.
1.1 Entry Points
| File | Lines | Role | |
|---|---|---|---|
__init__.py | 0 | Package marker | |
__main__.py | 3 | python -m app entry point β calls llm.cli.main() | |
main.py | 276 | FastAPI application factory β creates the app, registers CORS middleware, mounts 15 API routers across 5 domain sub-packages (canvas, execution, platform, registry, replay), defines lifespan (scans workspace/ on startup via WorkspaceComponentRegistry, starts the JobScheduler tick loop + file watchers + ResourceSampler, shuts down nodesets on exit) |
1.2 Core Singletons
Small files imported everywhere β config, state, models, and the core data model.
| File | Lines | Role | |
|---|---|---|---|
config.py | 42 | Settings singleton β Pydantic BaseSettings loaded from .env: host, port, workspace_dir, Habitat-specific settings. Immutable after init | |
state.py | 110 | ProcessServices + ExecutionGuard + broadcast() β ProcessServices holds WorkspaceComponentRegistry; ExecutionGuard is a thread-safe mutex preventing concurrent canvas + eval (ExecutionMode: idle/canvas/eval); broadcast() sends WSMessage to all connected WebSocket clients | |
models.py | 17 | WSMessage β single Pydantic model for all WebSocket messages (type, data, timestamp, execution_id, source) | |
graph_def.py | 1,065 | The core data model β pure dataclasses mirroring frontend types.ts: GraphDefinition, NodeDef, EdgeDef, ContainerDef, StateDef, AccessGrantDef, HookDef. JSON-serializable, no live state. Plus graph_signature(gd) / graph_diff(built, orig) β the structural fingerprint and diff the examples and tests compare graphs with |
1.3 Domain Modules
Single-file modules with one consumer each β candidates for future sub-packages as their domains grow.
| File | Lines | Role | |
|---|---|---|---|
layout.py | ~750 | Auto-layout algorithm β two-pass: topological layering (leftβright X assignment) + semantic post-processing (viewer stacking, container placement). Also works as CLI: python -m app.layout graph.json | |
errors.py | 359 | EventSink (ABC β publish abstract, from_exception / emit / flush) + ErrorEnvelope (ADR-observability-004). ErrorBus is the parent's adapter (200-entry ring buffer + 500-deep async dispatch queue, broadcasts error_event WS frames); get_bus() / set_sink() select the process's sink; one LogBridgeHandler for every process | |
eval_subprocess_main.py | ~310 | Subprocess-per-run entry (ADR-eval-003) β python -m app.eval_subprocess_main --run-dir ...: reads spec.json + shared_urls.json, attaches to parent backend's shared singletons, runs BatchEvalRunner.execute() unmodified, writes _DONE marker on clean exit |
1.4 Graph SDK
The programmatic authoring surface (capability page): build and run the same GraphDefinition the canvas emits, as a Python library β no canvas, no JSON authoring. Public import path is the agentcanvas package (sits next to app/); pip install agentcanvas is wired by the repo-root pyproject.toml (Β§19).
| File | Lines | Role | |
|---|---|---|---|
graph_sdk.py | 1,283 | The Graph SDK core β Graph builder (add / connect / graph_out / loop & composite sugar), catalog introspection (catalog(), describe(), nodesets()), typed-authoring stub generation (generate_node_stubs), load-time validation (GraphValidationError), and the in-process runner β run() β RunResult, on_event β RunEvent stream, batch eval(episodes, dataset=None, split=None) β EvalResult (selectors pushed only when given β no framework-side dataset / split defaults). Drives the real GraphExecutor; adds no execution machinery of its own | |
graph_sdk_replay.py | 226 | Cassette record/replay β records server-node HTTP round-trips during a live run, replays them hermetically so GPU-nodeset graphs (Habitat, SAM) become deterministic tests with no simulator or GPU present | |
graph_sdk_codegen.py | 188 | Reverse direction β graph_to_code() / Graph.to_code() compiles any GraphDefinition (canvas- or JSON-born) into a self-contained builder script; round-trip verified on MapGPT-MP3D (roadmap F4) | |
graph_sdk_demo.py | 104 | Runnable proof β python -m app.graph_sdk_demo: three pure-Python demo nodes, built + run in-process, round-tripped through canvas JSON. No env, no GPU, no LLM | |
../agentcanvas/__init__.py | 61 | Public namespace package β from agentcanvas import Graph; thin re-export of the app.graph_sdk + graph_to_code surface, so SDK users never import app.* directly |
env_controller.py + its HabitatEnvController were superseded by the generic BaseEnvPanel contract in app/components/env_panel.py (ADR-server-002). Episode control is now a nodeset-side concern, not a framework module.
Β§2Backend: API Layer
Grouped into 5 domain sub-packages. Each module exports a router = APIRouter() that main.py mounts.
2.1 Canvas β graph editing + environment control
| File | Lines | Route prefix | Role |
|---|---|---|---|
graphs.py | 165 | /api/graphs | Graph CRUD β list, load, save, delete graph JSON files in workspace/graphs/ and workspace/graph_nodes/. Also POST /layout for auto-layout |
env_panel.py | 175 | /api/env-panels | Episode control β generic env-panel REST API: list panels, state, dynamic options, field change, action invoke (POSTs guarded by ExecutionGuard). Serves any nodeset's registered BaseEnvPanel |
2.2 Execution β run, eval, logs, websocket
| File | Lines | Route prefix | Role |
|---|---|---|---|
run.py | 139 | /api/navigate | Canvas graph execution β POST /run receives GraphDefinition JSON, creates LoopRunner, runs via GraphExecutor. Plus /stop, /pause, /resume, GET /node-schemas |
eval.py | 558 | /api/eval/v2 | Batch eval API β POST /start (enqueue a run on JobScheduler; the admitted run executes BatchEvalRunner in its own subprocess), GET /status, POST /stop, GET /runs, DELETE /runs/{id} |
eval_storage.py | 79 | β | Eval run persistence β thin wrappers (save_run / load_run / list_runs / delete_run) over RunSummary in services/run_state_io.py, under outputs/eval_runs/ |
logs.py | 172 | /api/logs | Execution log retrieval β list runs, full run, per-node, summary, graph, assets. A live execution from its ExecutionLogger; everything else through logging/store.py Β· LogStore |
websocket.py | 47 | /ws | WebSocket endpoint β accepts connections, registers/unregisters clients with state.py. Broadcast logic lives in state.broadcast() |
internal_containers.py | 136 | /api/internal | Cross-process container broker (not a public API) β lets server-mode subprocess nodes read/write state containers regardless of which process homes them; called by server/remote_container.py::RemoteContainerProxy, msgpack payload path |
internal_events.py | 39 | /api/internal | Subprocess event push β receives ErrorEnvelope batches POSTed by subprocesses (server/event_push.py Β· PushSink β auto_host nodeset servers and the eval subprocess) and republishes each on the parent's ErrorBus β error_event WS frame |
coding_agent.py | 106 | /api/coding-agent | Coding-agent run API (experimental) β POST /start//stop, GET /status, plus per-episode textlog (offset-based JSONL slices) and frames/frame/{name} endpoints; thin HTTP shell over services/coding_agent_runner.py (Β§9) |
2.3 Platform β config, components, LLM profiles
| File | Lines | Route prefix | Role |
|---|---|---|---|
config.py | 41 | /api/config | Settings β GET / returns current Settings + active LLM profile info |
components.py | 383 | /api/components | Component registry β nodeset list/load/unload, POST /reload (hot-reload workspace/), /eval-metadata, /source (+ /scoped), /node-schemas. Server lifecycle is /nodesets/{name}/load?mode=server + /unload β the YAML-registered /servers* endpoints were deleted 2026-10-17 |
profiles.py | 248 | /api/profiles | LLM profile management β CRUD, POST /test-key, GET /providers (24 providers), GET /models/{provider} |
errors.py | 32 | /api/errors | Error backfill β returns the ErrorBus ring buffer so the frontend can backfill envelopes published while disconnected |
system.py | 53 | /api/system | System usage β /usage (latest sample) + /history from the always-on ResourceSampler (Β§9) |
2.4 Registry β component snapshot
| File | Lines | Route prefix | Role |
|---|---|---|---|
snapshot.py | 43 | /api/registry | Shared-nodeset snapshot β read-only view of currently-loaded shared nodesets; fallback for run subprocesses that started without a shared_urls.json (the normal path: JobScheduler writes it at submit time), also used by tests |
2.5 Replay β episode replay API
| File | Lines | Route prefix | Role |
|---|---|---|---|
router.py | 207 | /api/replay | Replay API β serves the env-agnostic replay timeline parsed from a run's episode logs (backend package app/replay/, Β§9); consumed by the frontend Replay page (Β§17) |
Β§3Backend: Execution Engine
The core runtime that turns a GraphDefinition JSON into a live agent run. All execution goes through this package. No HTTP/FastAPI imports β pure Python.
3.1 Execution Pipeline
When the user presses Play on the canvas, this is the call chain:
3.2 File by File
graph_run.py439 lines
The one prepare + run pipeline every entry path (canvas Play, batch eval, the Graph SDK, the eval subprocess) goes through:
prepare_graph(registry, graph, worker_count=, shared_only=, on_failed=, on_unknown=, validate=)β validate connectivity, load the graph's nodesets through the registry, warn-check wire types; returns aPreparedGraph(loaded / failed / unknown nodesets, env nodeset, global hooks).NodesetLoadErrorunder the"raise"policiesGraphRun(graph, session=, logger=, env_panel_overrides=, server_url_overrides=)β oneGraphExecutorbuilt for one run;run(...)drives it,outcome()harvests aRunOutcome(status, step count, outputs, metrics, node errors) at any time, including after a timeoutRunSessionβ thesessionobject the executor writes status / step / metrics into;ExecutionPrinciples(eval-mode switches) lives hereharvest_outputs/harvest_metricsβ read graph-scopegraphOutsinks off the executor;detect_env_nodeset
loop_runner.py193 lines
The canvas Play runner β a RunSession that adds the lifecycle: pause, stop, resume. Process singleton via get_loop_runner().
- Receives a
GraphDefinitionfrom the API - Self-creates an
ExecutionLoggeratoutputs/runs/<execution_id>/(and writesgraph.json) when none is injected - Builds a
GraphRun(graph_run.py) with itself as the session and awaitsgraph_run.run(), passing its stop / pause events - Manages pause/stop/resume via asyncio events; re-exports
ExecutionPrinciples
Does not know how nodes fire β sets up the world and hands control to the executor.
flatten.py300 lines
Pre-processing step. Runs before execution starts. The canvas supports nested graphs (composite nodes containing sub-graphs), but the executor needs a flat list of nodes and edges.
- Recursively walks composite nodes that have a
subgraph - Expands them into the parent graph, rewiring
graphIn/graphOutboundary nodes to parent edges - Prefixes node IDs to avoid collisions (
composite1.inner_node) - Returns a
FlattenMapso error messages can trace back to the original composite context
graph_executor.py2,006 lines Β· largest file
Where nodes actually execute. Implements a dataflow FIFO queue model:
- Build: wraps each
NodeDefinto aNodeInstance(tracks pending inputs per port) - Seed: finds nodes with no required inputs (environment nodes, constants) and enqueues them
- Loop: pull a node from queue β execute β deliver outputs to downstream ports β if downstream has all required inputs, enqueue it
- Fan-out: when a node produces output, all target ports are filled before any downstream node fires
- Fan-in: a node only fires when all required input ports have data
- Cycles:
IterOutNodecopies its inputs to the pairedIterInNodeand re-enqueues it - Termination: IterOut's
stopinput port goes true (Decide phase), step budget reached, stop event set, empty queue, or a node-raisedStopExecution - Seams: the loop only schedules β
_fire(the one error policy),_after_fireβ_deliver(the one delivery rule) and_iter_boundary(record β settle β decide β hand off) are called at fixed points; the pivots' slot / latch / snapshot rules live inpivots.py
Each node fires by resolving its class through the node catalog (CATALOG.get(node_type)), instantiating it and calling forward(inputs, ctx); required ports and consumer wire types come from CATALOG.required_inputs / CATALOG.wire_type.
pivots.py170 lines
The three pivot node types' runtime rules in one module, called by the executor at fixed points: LoopCarry β one iterIn's port slots (write / handoff / ready / fire_inputs / after_fire / last_carried); iterout_port_names, stop_requested, final_payload for iterOut; graphout_snapshot / graphout_last / graphout_latch / graphout_latched β the only writers of a graphOut's _last_inputs and latched_value. Nothing here schedules, fires or routes.
scope_analysis.py416 lines
Computes the scope forest before execution starts.
analyze_scopes()β walks the flat graph and returns every(iterIn, iterOut)scope plus its parent/child topology (ADR-dataflow-007)- Classifies composite boundaries:
graphIn= parameter slot,graphOut= return slot of a scope - Validates scope shape (inner scopes bounded by mandatory graphIn/graphOut) β feeds the executor's per-scope
_ScopeState
builtin_nodes.py1,660 lines
The home for all built-in framework nodes (see TODO #37) and the process-wide node catalog instance.
Catalog:
CATALOG = NodeCatalog(_BUILTIN_NODES)β the process-wide catalog (node_typeβ class, plus each instance's ports); the class lives incomponents/catalog.pyregister_node(cls)β thin wrapper overCATALOG.register, used by tests and SDK demos; the registry callsCATALOG.register/unregisterdirectlyExecutionContextdataclass β mutable state passed to every node
Built-in node classes (14):
| Node | What it does |
|---|---|
IterInNode / IterOutNode | Loop gates β IterOut sends data back to IterIn for next iteration; iterIn's left side captures run-start initPorts, iterOut's final side feeds the after-loop band |
LLMCallNode | Builds one app.llm.Request (prompt / images / history / sampling knobs) and calls complete(), returns text + candidates |
TextParseNode | Generic structured parser for LLM response text |
HistoryLogNode | Accumulates one TEXT entry per firing, emits the formatted history |
GraphInNode / GraphOutNode | Composite boundaries β parameter slot / return slot (formerly PortIn/PortOut); graphOut doubles as the sink that buffers _last_inputs for the nav_step broadcast |
NullSourceNode | Emits None on a typed output port β once, at run-start |
NoteNode | Markdown billboard β pure annotation, no inputs/outputs |
ImageViewerSink | Viewer β displays images on canvas via viewer_data WS events (configurable grid, imageGrid layout) |
TextViewerSink / TextScrollSink / ActionLogSink / MetricsViewerSink | Viewers for latest TEXT / accumulating TEXT / action history / SR/SPL/NE metrics |
state_containers.py469 lines
Implements the dual-wire system: nodes communicate via data edges (typed values) and state edges (access grants to shared containers).
BaseStateABC with 3 reducer implementations (ephemeral removed 2026-04-18 β uselifetime="step"):AccumulatorState(append),LastWriteState(overwrite),CounterState(increment)BaseState.on_signal(name, payload)β clears toinitial_valuewhen a name inreset_onfiresStateContainerβ a dict of namedBaseStateentries- Factory function builds containers from
ContainerDefin graph JSON - Executor injects connected containers into
ctx._containersbefore each node fires - Nodeset-owned / keyed containers β
BaseNodeSet.get_containers()lets a nodeset own containers (withallow_opaquevalues and per-key evict for worker parallelism), alongside graph-level containers
eval_batch.py738 lines
Drives a GraphRun per episode for batch evaluation:
BatchEvalRunner(run, graph, run_dir=, on_episode=)takes anEvalRun+ graph;on_episodefires after each episode's aggregates refresh (the subprocess uses it to rewritesummary.json)- For each episode: on its worker's
Fleethandle, sets the episode through the (tagged) env panel, builds a freshGraphRun, runs underasyncio.wait_for(timeout=step_budget Γ per_step_budget_sec), harvests theRunOutcome's metrics (partial ones after a timeout) - At
worker_count>1: N contiguous chunks (_partition_contiguous), oneasyncio.Queueper worker drained by N coroutines underasyncio.gather, results aggregated underasyncio.Lock, sorted byepisode_indexbefore persistence resolve_per_step_budget(env_nodeset, override)β resolves the per-step wall-clock budget- Emits WS progress events (
eval_progress,eval_episode_done) tagged withworker_id - Respects
ExecutionGuardβ only one eval or canvas run at a time
eval_config.py105 lines
EvalConfig β what one batch-eval run evaluates and the one place its defaults live; built per request in api/execution/eval.py, carried to the subprocess inside JobSpec, recorded in summary.json (RunSummary.config), read back everywhere through from_dict. Re-exported by eval_batch.py.
hooks.py307 lines
Fires shell commands at graph/node boundaries β like git hooks for graph execution.
- 5 events:
GraphStart,GraphComplete,GraphError,PreNodeExecute,PostNodeExecute - Protocol: event payload sent as JSON on stdin β hook process β JSON response on stdout with
{"action": "continue"|"block"|"modify"} - Fail-open: any error (timeout, crash, malformed JSON) defaults to
action="continue" HookRunnerclass manages execution;load_hooks_file()loads hook definitions from a JSON file (array of hook defs)
__init__.py17 lines
PolicyEntry dataclass: metadata for a loadable neural policy checkpoint (id, name, checkpoint path, config path). Used by WorkspaceComponentRegistry when scanning workspace/policies/.
3.3 Layering
GraphExecutor decides when a node fires (input readiness). The classes in the node catalog decide what happens (domain logic). Adding a new node type is just subclassing BaseCanvasNode in workspace/ β the executor never changes.
Β§4Backend: Component System
The public API for node authors. Files in workspace/ import from this package.
| File | Lines | Role | |
|---|---|---|---|
bases.py | 554 | Base classes for all node authoring β PortDef, ConfigField (7 widget types incl. port_list), DisplayField (4 display types: image_viewer, log_list, metric_table, text_viewer), NodeUIConfig, BaseCanvasNode (abstract: forward(), ports, category, icon, _self_log(), serialized), BaseNodeSet (loadable tool group; server_python / server_image, parallelism, env_panel, default_per_step_budget_sec, replay_parser ClassVars) | |
registry.py | 1,817 | WorkspaceComponentRegistry β scans workspace/ subdirectories, imports Python files, discovers BaseCanvasNode/BaseNodeSet subclasses via inspect. Registers in the node catalog (CATALOG); builds every server through Launcher; implements RegistryFacts for admission. Handles load/unload lifecycle, server-mode auto-routing, hook file loading, graph auto-loading, register_remote_nodeset (subprocess attach to shared singletons), container-ownership checks. Keeps no table of nodeset names: declared_nodeset_facts(source) reads name / parallelism off a nodeset file's AST, so a nodeset that cannot import on this env still answers the parallelism contract | |
env_panel.py | 250 | BaseEnvPanel β control-plane panel base for env-side runtime knobs (suite / split / episode, play / pause / reset); declared via the BaseNodeSet.env_panel ClassVar, served at /api/env-panels/*, bridged into server-mode subprocesses by RemoteEnvPanelProxy. Renamed from "controller" 2026-06-11 (ADR-server-002) | |
catalog.py | 184 | NodeCatalog β which node types exist in this process and what ports each instance has: register / unregister, get, ports_of(type, config) (the one rule: _resolve_ports(config) when the class has one, else the class lists), required_inputs, wire_type, schema / schemas (the /node-schemas feed). The process-wide instance is builtin_nodes.CATALOG | |
launcher.py | 291 | Launcher β vehicle_for(nodeset_cls, mode, worker_count) routes a nodeset to local / native / container; Launcher.build(...) assembles the BaseServer or ContainerServer (build_container_server: path translation, mounts, executor-URL rewrite); server_key, find_free_port, auto_host_command | |
fleet.py | 152 | Fleet β the N workers of a batch-eval run over the tagged env copies: one WorkerHandle per worker (env_panel_overrides + server_url_overrides for every replicated nodeset), worker(k), teardown on exit (ADR-eval-002) | |
facts.py | 57 | RegistryFacts (abc) + NodesetFacts β the read-only view of the registry that admission needs (parallelism, loaded / hosted, source file + hash, author presets); the registry implements it, ResourceStatsTracker consumes it | |
content_hash.py | 95 | Content hashing for nodeset source trees β lets /api/eval/v2/start detect whether an active_workspace_dir overlay actually changed a shared nodeset's source vs. the frozen baseline | |
__init__.py | 24 | Re-exports: BaseCanvasNode, BaseNodeSet, PortDef, ConfigField, DisplayField, NodeUIConfig β the public import path |
Β§5Backend: Server SDK
Implements the write-once deploy-anywhere server mode (ADR-server-001). A BaseNodeSet subclass runs identically in-process or as an auto-hosted HTTP server. Three roles: server-side, framework-side, bridge.
| File | Lines | Role | |
|---|---|---|---|
server_app.py | 184 | ServerApp (server-side) β builds a FastAPI HTTP service speaking the manifest protocol: GET /manifest, POST /call/{fn}, GET /health | |
auto_server_app.py | 332 | AutoServerApp (server-side) β auto-generates a ServerApp from any BaseNodeSet by introspecting port shapes. Holds the one inference lock (_get_inference_lock, created lazily inside the running loop); _make_handler wraps a serialized = True node's handler in it, every other node's calls are concurrent | |
base_server.py | 353 | BaseServer (framework-side) β launches subprocess, polls /health, handles start/stop/restart, fetches manifest. Manages process lifecycle and port allocation; built only by Launcher from a nodeset's ClassVars β there is no file-based server registration | |
client.py | 347 | AutoHostClient (framework-side) β the one HTTP client against an auto_host: call / call_sync (msgpack or JSON, usage + log in the reply), env-panel calls, manifest_sync, health, evict, container_op; CallError / ServerUnreachable | |
invoke.py | 153 | invoke (server-side) β the one function call behind POST /call/{fn} and the MCP tools/call: decode per port map, set the per-call log + usage buckets, run the handler, encode outputs or a FireList; a handler exception goes to the process sink (get_bus().from_exception) and the sink is flushed at the call boundary | |
proxy.py | 149 | Proxy node generator (bridge) β reads ServerManifest and auto-generates BaseCanvasNode subclasses whose forward() forwards inputs over HTTP through AutoHostClient (per-worker URL override, hosted log + usage folded into the firing; a server error raises CallError). Canvas can't tell the difference | |
manifest.py | 93 | Manifest protocol types β PortSchema, FunctionSchema, ServerManifest | |
serialization.py | 300 | Wire serialization β msgpack codec (pack_body/unpack_body, ADR-server-004): one blob ExtType carries raw ndarray/torch.Tensor/PIL.Image bytes with receiver-side degrade; JSON+base64 legacy path kept for the migration window | |
event_push.py | 108 | PushSink β the subprocess EventSink adapter: buffers envelopes, POSTs them to the parent's /api/internal/events (an error at once, the rest at every /call or episode boundary); install_push_sink(url, nodeset=, log_level=) installs it + the WARNING+ log bridge in auto_host servers and the eval subprocess (roadmap #54 Move 3) | |
remote_container.py | 75 | RemoteContainerProxy β subprocess-side handle to an executor-home state container; calls the /api/internal broker so access-granted server nodes can read/write home containers | |
_loopback_proxy.py | 39 | Loopback proxy policy β keeps httpx calls to localhost auto-host children from being routed through a user's HTTP_PROXY/HTTPS_PROXY shell env | |
auto_host.py | 60 | CLI entry point β python -m app.server.auto_host --file nodeset.py --class MyNodeSet --port 9200 | |
__init__.py | 30 | Re-exports: AutoServerApp, AutoHostClient, BaseServer, ServerApp, ServerFunction, manifest types |
Β§6Backend: LLM System
Self-contained package for the LLM/VLM subsystem. __init__.py re-exports the public API so external imports work unchanged.
| File | Lines | Role | |
|---|---|---|---|
__init__.py | 34 | Re-exports: complete, complete_sync, Request, Completion, llm_complete, vlm_complete (+ _n), get_llm_config, LLMConfig, get_profile_store, LLMProfile, PROVIDER_REGISTRY, resolve_provider_config, finalize_params, get_capabilities | |
complete.py | 421 | The completion module β complete(config, Request) β Completion (+ complete_sync): message builder, the rulebook over every parameter, provider-native n with a single-sample fill, usage accounting, timeout and strict-error policy. Transport (abc) is the wire seam; LiteLLMTransport is the production adapter (100+ providers through litellm) | |
call.py | 215 | Profile β LLMConfig (get_llm_config / get_llm_config_direct; ADR-platform-003) and the four thin wrappers llm_complete / llm_complete_n / vlm_complete / vlm_complete_n over complete | |
rulebook.py | 215 | Parameter rulebook β verified per-(provider, model-family) facts (locked / required / range / unsupported / min_hint); finalize_params at call time, get_capabilities for the Settings UI | |
profiles.py | 157 | ProfileStore β persistent storage at ~/.agentcanvas/profiles.json. Thread-safe with RLock. Mtime-based cache invalidation | |
providers.py | 117 | Provider registry β static dict mapping 24 provider IDs to ProviderDef (label, base_url, api_type, default_model, litellm_prefix); LLMConfig (one resolved call target) and API_TYPE_TO_LITELLM_PREFIX. Covers: OpenAI, Anthropic, Google, Ollama, DeepSeek, Together, OpenRouter, Mistral, xAI, NVIDIA NIM, Hugging Face, Moonshot, and more | |
key_validator.py | 94 | API key validation β tests if an API key is valid via a minimal litellm completion request | |
cli.py | 478 | Standalone CLI tool β 8 argparse commands: list, add, remove, set-active, test, show, setup, models |
6.1 Fallback Chain (ADR-platform-003)
Β§7Backend: Logging
Two-layer logging system (ADR-observability-003): the executor captures port I/O automatically (exterior); nodes add domain detail via _self_log() (interior).
| File | Lines | Role | |
|---|---|---|---|
logger.py | 402 | ExecutionLogger β captures two layers per node firing: automatic exterior (executor records all port inputs, outputs, timing) + voluntary interior (nodes call _self_log(key, value)). log_serialize() truncates large values. Persists as JSONL. Broadcasts exec_log WS events (suppressed in eval batch mode) | |
store.py | 365 | LogStore β the one reader over execution logs: on-disk layout (canvas_runs/{id}/, eval_runs/{run}/episodes/ep{idx:04d}/), composite episode id {run_id}_ep{idx:04d}, tolerant JSONL read_entries, list_executions / summarize / timing; behind the logs API, the replay API and system_runs | |
models.py | 39 | Log data models β NodeLogEntry (one node firing) and ExecutionSummary (aggregate) | |
__init__.py | 7 | Re-exports: ExecutionLogger, ExecutionSummary, NodeLogEntry |
Β§8Backend: Standard Library
Shared type definitions and constants used by node implementations and the executor. This is a leaf package with no intra-app dependencies.
| File | Lines | Role | |
|---|---|---|---|
wire_types.py | 443 | Wire type registry β 9 current types (IMAGE, DEPTH, POSE, DISCRETE_ACTION, CONTROL, TEXT, BOOL, METRICS, ANY) + 3 deprecated aliases (ACTION, OBSERVATION, STEP_RESULT) kept so legacy graphs load, + LIST[T] modifier, WIRE_FORMAT_SPEC canonical on-wire formats, display colours. Helpers: image_to_base64(), depth_to_base64(), serialize_for_display() | |
actions.py | 40 | Action constants β ACTION_STOP, ACTION_FORWARD, ACTION_TURN_LEFT, ACTION_TURN_RIGHT (the VLN Discrete(4) space). parse_action_from_text() extracts an action from LLM output | |
telemetry.py | 25 | Per-node-firing accounting ContextVars β _current_node_usage (written by app.llm.complete), _current_node_transport (written by the server-proxy codec) and _current_node_log (a hosted node's _self_log buffer under invoke); new_usage_bucket / merge_usage. The executor resets them around each firing and reads them back into the node's log entry; they live here so a pure-local run doesn't drag in litellm/numpy | |
__init__.py | 9 | Re-exports everything from the sibling modules |
Β§9Backend: Services, Replay & Tools
Long-running backend-process services owned by the FastAPI lifespan: job scheduling, file watching, resource sampling.
| File | Lines | Role | |
|---|---|---|---|
job_scheduler.py | 856 | JobScheduler β admission + FIFO queue + Popen pool for eval run subprocesses (ADR-eval-003). Singleton owned by ProcessServices; one tick() per second admits queued jobs through the per-resource measured gate β the job's charge from ResourceStatsTracker.charge vs measured free VRAM/RAM minus reservations β while the canvas ExecutionGuard lock is free; spawns python -m app.eval_subprocess_main with PR_SET_PDEATHSIG. submit(spec: JobSpec, *, shared_urls=) writes spec.json + shared_urls.json + a pending RunSummary | |
resource_stats.py | 739 | ResourceStatsTracker β per-owner resource attribution (VRAM per-PID from the sampler, RAM RSS from one /proc walk β shared server | job tree | external), per-run observation windows, calibration store (outputs/system/resource_calibration.json, EWMA + max per resource, source-hash keyed) and the one admission charge ladder β charge(): calibrated / hint / declared / cold-start, or legacy without a sampler β behind JobScheduler._admit and GET /api/eval/v2/estimate; reads nodeset facts through the RegistryFacts seam | |
graph_watcher.py | 73 | Graph-directory watcher β pushes graphs_changed when graph JSON files change on disk, so coding-agent edits made outside the web UI show up in the Explorer without a manual refresh | |
nodeset_watcher.py | 156 | Nodeset-source watcher β hot-reloads nodeset .py edits without a backend restart; companion to graph_watcher | |
resource_sampler.py | 346 | ResourceSampler β always-on ~1 Hz machine resource time-series written to outputs/system/system-YYYYMMDD.jsonl; feeds /api/system/usage + /history and the Monitor page (Β§17) | |
run_state_io.py | 370 | The typed run files β what crosses the parent β subprocess boundary under outputs/eval_runs/{run_id}/: JobSpec (spec.json: eval: EvalConfig Β· graph Β· Scheduling Β· active_workspace_dir), RunSummary (summary.json: initial / from_run / finish / page_summary), write_shared_urls / read_shared_urls, touch_done / is_done, list_run_summaries, mark_aborted; every write atomic, every read tolerant | |
system_runs.py | 128 | System Log per-run aggregation β joins a run's execution logs with tee'd resource samples, through LogStore, into a per-run performance summary for the Monitor page's Run view | |
coding_agent_runner.py | 248 | CodingAgentRunner (experimental) β singleton owned by ProcessServices; one-click host for the beta-coding-agent/ vanilla coding-agent VLN experiment: spawns an env_habitat auto_host via BaseServer on a free port + the episode driver as a process-group child, derives run status from the driver's artifacts (summary.json + per-episode JSONL) under outputs/beta-coding-agent/. v1: single worker, one run at a time |
Episode replay backend β turns persisted episode logs into an env-agnostic replay timeline (served by api/replay/, Β§2.5; consumed by the frontend Replay page, Β§17).
| File | Lines | Role | |
|---|---|---|---|
interface.py | 282 | Replay interface β env-agnostic timeline schema + BaseReplayParser ABC; an env nodeset opts into replay by pointing its replay_parser ClassVar at a parser module (pure log-walking, no simulator imports) | |
renderer_client.py | 144 | Renderer client β lazy-spawns and calls a renderer subprocess for parsers' smooth-mode frame requests; held warm for the FastAPI process lifetime | |
renderer_host.py | 61 | Renderer host β launches a renderer class from a file path (mirrors server/auto_host.py but only needs a build_app() method) |
| File | Lines | Role | |
|---|---|---|---|
validate_graph.py | 226 | Standalone graph wire-type validator (dev tool) β loads a graph from disk, runs the same app.graph_def.wire_type_report core the backend uses, prints a report, exits non-zero on any mismatch. No backend needed |
Β§10Backend: Dependency Flow
How the backend packages depend on each other. Arrows point from importer to imported.
- Leaf packages (no intra-app deps):
standard/,logging/,llm/ - Core data model:
graph_def.pyβ imported by everything - Cross-cutting:
state.broadcast()β used by bothagent_loop/andapi/execution/websocket.py. The engine no longer imports from the API layer - Diagram shows the three largest of the five
api/sub-packages;api/registry/andapi/replay/(Β§2.4β2.5) are omitted for legibility, as areservices/,replay/, andtools/(Β§9)
Β§11Frontend: App Shell
| File | Role | ||
|---|---|---|---|
main.tsx | React entry point β mounts <App /> into #root | ||
App.tsx | Top-level router β switches between CanvasPage, NodeSetManager, EvalPage, LogViewerPage, ReplayPage, MonitorPage, CodingAgentPage, HumanPage based on appMode | ||
store.ts | Zustand global store β connected, appMode + the PAGES registry (each page carries a workflow / agentic category; per-category last-visited page remembered in localStorage), eval run state, nav status, env context bar data | ||
api.ts | REST client β fetchJ, postJ, deleteJ, putJ wrappers around fetch with JSON handling | ||
ws.ts | WebSocket manager β connects to /ws, dispatches typed events to listeners; reconnects on drop | ||
types.ts | Global TypeScript types β NavStatus, EvalRunSummary, AppMode, EnvContextState | ||
errorStore.ts | useErrorStore β Zustand store of ErrorEnvelopes fed by error_event WS frames; backfills from GET /api/errors on (re)connect | ||
errors.ts | ErrorEnvelope types + severity helpers mirroring the backend ErrorBus schema | ||
vite-env.d.ts | Vite environment type declarations |
Shared components (src/components/)
Header.tsx | Top app header β brand, two-level nav (Workflow / Agentic category selector + the active category's page tabs, driven by the PAGES registry in store.ts), execution-mode badge (polls /api/navigate/execution-mode), settings-gear entry | ||
SettingsModal.tsx | Settings modal β provider API keys, active provider + default model, profiles/favorites; reads/writes /api/config + /api/profiles | ||
ErrorToast.tsx | Floating toast layer β auto-pops fresh error/warning ErrorEnvelopes (max 3, auto-dismiss); click focuses the Report tab | ||
ViewPanel.tsx | Static placeholder "Views" panel (RGB / Depth tiles) β presentational stub, not wired to data |
Β§12Frontend: Canvas Editor
Core graph editing workspace. React Flow drives the visual layer; a Zustand store owns all graph state.
| File | Role | ||
|---|---|---|---|
CanvasPage.tsx | Canvas workspace page β tab management, error boundary, layout scaffold | ||
UnifiedGraphEditor.tsx | Core editor β root graph mode + subgraph drill-down mode; React Flow instance owner | ||
useFlowStore.ts | Zustand canvas store β tab-aware graph state, node/edge CRUD, selection, execution status per tab | ||
types.ts | GraphDefinition, NodeDef, EdgeDef, ContainerDef, StateDef, AccessGrantDef β mirrors backend schema | ||
graphConversion.ts | Bidirectional conversion between GraphDefinition (backend JSON) and React Flow Node[]/Edge[] | ||
defaultGraph.ts | Template graphs loaded for new tabs (Straightforward CMA, NavGPT-CE) | ||
portResolution.ts | resolveInstancePorts() β applies config.ports overrides to class-level port schemas | ||
unifiedNodeTypes.ts | Node type registry β maps node type strings to React components for React Flow | ||
unifiedCatalog.ts | Component catalog β fetches node schemas from backend; drives the node library sidebar | ||
nodesetLoader.ts | NodeSet loader β fetches loaded nodesets list; triggers nodesets-changed refresh pipeline | ||
groupSelection.ts | Multi-select group logic β bounding box selection, group move | ||
flowStoreRef.ts | Bridge β holds a stable ref to the React Flow instance accessible outside React tree | ||
runPipeline.ts | runPipeline() β builds RunRequest, calls /api/run, manages execution lifecycle from frontend side | ||
edgeTypes.ts | Custom edge type registry β registers AccessGrantEdge alongside default React Flow edge types | ||
edges/AccessGrantEdge.tsx | Renders dashed violet access-grant edges (renamed from StateEdge.tsx with the state-edge β access-grant terminology shift, ADR-dataflow-004) | ||
ShortcutCheatsheet.tsx | Keyboard-shortcut cheatsheet overlay | ||
useKeyboardShortcuts.ts | Global keyboard-shortcut hook β Space = play/pause, F = fit view, ? = cheatsheet; ignores typing targets |
Β§13Frontend: Canvas Nodes
The renderer tree that turns Python-defined NodeUIConfig into visible canvas nodes.
Shared primitives
shared/NodeShell.tsx | Generic node wrapper β selection ring, drag handle, resize handle | ||
shared/HandleDot.tsx | Port handle dot β colour-coded by wire type, tooltip with port name/type |
Special node types
state/StateContainerNode.tsx | Renders a StateContainer on canvas β dashed violet border, database icon, reducer badges | ||
composite/CompositeCanvasNode.tsx | Renders a composite (nested graph) node β port boundaries, drill-down button | ||
composite/CompositeNodeView.tsx | Editing view inside a composite node β full sub-graph editor in a panel |
Generic renderer stack (agentloop/inner/)
All non-special nodes flow through this stack, driven entirely by NodeUIConfig from Python.
GenericBlockRenderer.tsx | Dispatcher β reads ui_config.layout, routes to BlockLayout, StripLayout, ViewerLayout, NoteLayout, or ImageGridViewerLayout | ||
layouts/BlockLayout.tsx | Standard block layout β title bar + config fields + port handles | ||
layouts/StripLayout.tsx | Horizontal strip β compact port-only view for pass-through nodes | ||
layouts/ViewerLayout.tsx | Viewer layout β display-only fields for output sink nodes | ||
layouts/ImageGridViewerLayout.tsx | imageGrid layout β rowsΓcols image grid for the configurable imageViewer (ADR-components-007) | ||
layouts/NoteLayout.tsx | note layout β markdown billboard rendering for the note built-in | ||
layouts/IterInExpandPanel.tsx | iterIn expand panel β edits the two-sided iterIn's initPorts (with per-port persist checkboxes) and shows the synthesised init/iterOut port surface; edits resync handles live | ||
layouts/ConfigFieldRenderer.tsx | Renders a single ConfigField β label, slider, text, select, toggle, textarea, port_list | ||
layouts/PortListEditor.tsx | port_list config field widget β add/remove/rename ports for IterIn/IterOut | ||
layouts/layoutUtils.ts | Shared layout helpers β port position calculation, label truncation | ||
layouts/useNodeOutput.ts | React hook β subscribes to viewer_data WS events for a given node_id | ||
layouts/viewerFieldRegistry.ts | Registry of display field renderers keyed by DisplayField.display_type |
Display field renderers (layouts/fields/)
ImageViewerField.tsx | Renders base64 image frames from viewer_data events | ||
TextViewerField.tsx | Renders plain text output (LLM responses, debug strings) | ||
LogListField.tsx | Scrolling log list with accumulate=true appending behaviour | ||
MetricTableField.tsx | Key-value metric table (SR, SPL, NE, etc.) |
Β§14Frontend: Canvas Panels
Sidebar and toolbar UI components that surround the React Flow canvas.
TabBar.tsx | Tab strip for open graphs β add, close, switch, unsaved-change indicator | ||
ExplorerPanel.tsx | VS Code-style tree β Graphs / Graph Nodes / Nodes sections, drag-to-canvas | ||
NodeLibrary.tsx | Draggable node palette β shows only built-in + currently loaded nodeset nodes | ||
PropertiesPanel.tsx | Selected node property editor β config field forms, port visibility | ||
StatePanel.tsx | State container editor β add/remove named state entries, set reducer type | ||
OutputDrawer.tsx | Bottom drawer β Observation / Logs / Step Log / State / Report tabs | ||
LogPanel.tsx | Execution log tab β lists exec_log events grouped by node | ||
StepLogPanel.tsx | Per-step log tab β full I/O trace for the current step | ||
ExecutionToolbar.tsx | Run / Pause / Stop / Step buttons + execution status badge | ||
EnvPanel.tsx | Env panel strip (renamed from EnvContextBar.tsx; episode browsing folded in) β renders any nodeset's BaseEnvPanel from /api/env-panels: suite/split selectors, episode navigator, panel actions | ||
ObservationPanel.tsx | Current observation display β image + instruction text | ||
ReportPanel.tsx | Report tab β time-ordered ErrorEnvelope list with severity filter chips, click-to-expand (ADR-observability-004) | ||
ResizableBottomPanel.tsx | Drag-resizable wrapper for the bottom drawer | ||
GraphStateBanner.tsx | Banner shown during/after execution β run status, error messages | ||
TemplatePicker.tsx | Graph template picker modal β choose preset to load into new tab | ||
SaveGraphDialog.tsx | Save graph / save-as-node dialog β name, group, kind selection |
Β§15Frontend: Evaluation Page
Batch evaluation launcher and live monitor (ADR-eval-001).
EvalPage.tsx | Top-level eval page β orchestrates all sub-components, subscribes to WS eval events | ||
evalApi.ts | Eval API client β startEval(), stopEval(), getRunHistory() wrappers | ||
types.ts | Eval-specific TypeScript types β EvalRun, EpisodeResult, EvalStatus | ||
GraphSelector.tsx | Graph picker β lists saved graphs, shows nodeset requirements | ||
EnvInfoPanel.tsx | Environment info panel β split counts, episode counts, dataset source | ||
EvalProgressBar.tsx | Progress bar β episodes done / total with estimated completion | ||
MetricCards.tsx | Live metric summary cards β SR, SPL, NE updated after each episode | ||
EpisodesTable.tsx | Per-episode results table β instruction, predicted path, metrics | ||
RunHistory.tsx | Previous runs list β load past run results for comparison |
Β§16Frontend: Log Viewer
Dedicated page for post-hoc inspection of execution logs (ADR-observability-003).
LogViewerPage.tsx | Full-page log viewer β split view: run list + detail pane | ||
logApi.ts | Logs API client β listRuns(), getRunLog(), getNodeLog() | ||
types.ts | Log-specific types β RunRecord, NodeLogEntry, PortSnapshot | ||
LogContext.tsx | React context β selected run, filter state, pagination | ||
LogCanvasView.tsx | Canvas-style log visualisation β replay node firing order on a static graph | ||
ExecutionList.tsx | Left panel β paginated list of past execution runs | ||
LogEntryList.tsx | Right panel β paginated log entries for a selected node | ||
LogSummaryBar.tsx | Summary bar β total nodes fired, duration, error count | ||
LogFilters.tsx | Filter controls β filter by node type, wire type, has-error | ||
nodeColors.ts | Node type to colour mapping for log canvas |
Renderers (logs/renderers/)
Each renderer handles one value type in a log entry's port snapshot.
registry.ts | Renderer registry β maps wire_type strings to renderer components | ||
PrimitiveRenderer.tsx | Numbers, booleans, short strings | ||
TextRenderer.tsx | Long text β LLM responses, prompts | ||
ImageRenderer.tsx | Base64 image frames | ||
JsonTreeRenderer.tsx | Collapsible JSON tree for dicts/lists | ||
ActionRenderer.tsx | ACTION wire β discrete action label + direction arrow | ||
ErrorRenderer.tsx | Error traces β red-highlighted stack output | ||
TensorRenderer.tsx | NumPy array β shape, dtype, value preview | ||
MetricsRenderer.tsx | METRICS wire β key-value metric table | ||
StepResultRenderer.tsx | STEP_RESULT wire (deprecated type) β done flag, info dict, reward | ||
ImageListRenderer.tsx | LIST[IMAGE] β thumbnail strip for multi-image port values |
Β§17Frontend: Replay & Monitor
Inspection pages beyond the log viewer: episode replay, system/run performance monitoring, and the experimental coding-agent monitor.
replay/ReplayPage.tsx | Episode replay page β steps through the env-agnostic replay timeline served by /api/replay (Β§2.5), with frame scrubbing | ||
replay/replayApi.ts | Replay API client β timeline + frame fetch wrappers | ||
pages/monitor/MonitorPage.tsx | Monitor page β machine resource sparklines (polls /api/system/usage, Β§2.3) + the JobScheduler queue view (polls /api/eval/v2/queue) + per-run performance view (System Log) | ||
pages/monitor/NodeTimingTable.tsx | Per-node timing table β where a run's wall-clock went, node by node | ||
pages/monitor/Sparkline.tsx | Inline sparkline chart for resource/timing series | ||
pages/coding/CodingAgentPage.tsx | Coding-Agent Monitor page (experimental) β one-click run control panel for /api/coding-agent (Β§2.2) + unified live log at 1 Hz polling: trajectory events with the agent's observe() frames embedded inline (click-to-zoom lightbox), per-episode selector with success badges |
Β§18Frontend: NodeSet Manager
NodeSetManager.tsx | Full NodeSet manager page β the discovered NodeSets with load (local or server mode) / unload; the YAML-server section went with the manual-server path (2026-10-17) | ||
manager/NodeSetCard.tsx | Card for one local NodeSet β name, status badge, load/unload button |
Β§19Config and Build
<repo root>/pyproject.toml | The pip distribution β packages app* + agentcanvas* from agentcanvas/backend/ as the agentcanvas distribution. Core install is deliberately light (stdlib + pydantic-settings); heavier layers are tiered extras: [server] (proxy + cassette record/replay), [llm] (the llmCall builtin), [backend] (full FastAPI canvas backend), [all]. Lives at the repo root so a fresh clone installs with plain pip install -e . | ||
backend/requirements.txt | Python deps for the full backend β FastAPI, Uvicorn, Pydantic v2, LiteLLM, httpx, python-multipart. Kept in sync with the agentcanvas[backend] extra | ||
backend/examples/ | Seven numbered Graph SDK example scripts (01_pure_python.py β¦ 07_habitat_sam.py) walking pure-Python builds β authoring sugar β MapGPT from code β single-episode run β batch eval β reverse codegen β a real Habitat+SAM graph; plus mapgpt_mp3d_sdk.py, the MapGPT-on-MP3D builder the examples import (moved out of app/ 2026-10-17 β domain knowledge is not framework) | ||
backend/.env.example | Environment variable template β AGENTCANVAS_* keys, port config | ||
frontend/package.json | NPM manifest β React 18, @xyflow/react, Zustand, Tailwind, Vite, Three.js | ||
frontend/vite.config.ts | Vite config β API proxy (/api to localhost:8000), React plugin | ||
frontend/tsconfig.json | TypeScript config β strict mode, path aliases | ||
run_dev.sh | Dev launcher β starts backend (uvicorn) on :8000 and frontend (vite) on :5173 in parallel |