LLM/VLM Configuration System
Multi-provider profiles, env vars, CLI, per-node selection
~/.agentcanvas/.keys) β env var, provider endpoints moved to /api/providers, sampling params became two-state with a parameter rulebook, and nodes can pin (provider, model) directly. The current behaviour is documented in LLM Call Node; this page's registry/CLI/protocol sections remain accurate.
The LLM/VLM configuration system lets any canvas node call any of 24 providers β OpenAI, Anthropic, Google, Ollama, and 20+ more β through one litellm-backed async API. A profile names a provider + model; API keys never live in the profile, they're read from the provider's standard env var at call time. Each node can pick its own profile, so a single graph can reason with GPT-4o, caption with Claude, and run cheap inference on local Ollama. Designed in ADR-platform-003, hardened in ADR-platform-006.
How a node gets its model β the whole call path. (Β§3 zooms into the resolve config step; Β§12 into the usage hook.)
1. What it is, and the one rule
Canvas nodes β chiefly LLMCallNode β need to reach an LLM/VLM without hard-coding a provider SDK. This system is that indirection: litellm is the single abstraction layer (no per-provider SDKs; httpx appears only in key_validator.py for the Ollama health check). Four design choices shape it:
- The one rule β no secrets on disk.
profiles.jsoncarries onlyprovider/model/base_url/api_typeand is git-safe; keys live in per-provider env vars (OPENAI_API_KEY, β¦) read at call time (ADR-platform-006). - Field-level fallback β each config field (key, model, base_url, api_type) resolves independently: the profile's value, else the provider registry default.
- Hot reload β mtime-based cache invalidation, so a CLI edit is picked up by a running server within ~2 s.
- Graceful degradation β no active profile, or the provider env var unset, drops to mock mode (
get_llm_config()returnsNone, the node renders a placeholder) β unlessAGENTCANVAS_STRICT_ERRORSis set (Β§11.5).
2. The smallest setup
Two steps put a live model behind every llmCall node β set the provider's env var, then create and activate a profile:
# 1. the key lives in the environment, never in a file export OPENAI_API_KEY=sk-... # 2. create a profile (provider + model) and make it the active default python -m agentcanvas.backend.app config set openai --provider openai --model gpt-4o python -m agentcanvas.backend.app config activate openai
Now an llmCall node with a blank profile field resolves to the active openai profile; a node that sets profile = "anthropic" uses that one instead (Β§9). No key ever touches profiles.json.
3. The resolution chain at a glance
Every LLM call resolves its config in two stages: (1) load the named (or active) profile from profiles.json, (2) read the API key from the provider's standard env var. ADR-platform-006 removed the previous AGENTCANVAS_* / VLM_* global env-var fallback chain β keys are now per provider, never on disk.
The merge of profile fields + registry defaults happens inside resolve_provider_config() in providers.py: empty base_url / api_type / model on the profile inherit from PROVIDER_REGISTRY[provider]. litellm_prefix always comes from the registry (not user-overridable). If api_key ends up empty and the provider is not Ollama, get_llm_config() returns None (mock mode).
Special cases:
- Ollama has
env_var=""in the registry and skips the key check (local server) - Custom provider is the only entry whose env var is
AGENTCANVAS_API_KEY(general-purpose self-hosted endpoints) - No active profile + no explicit
profile_namereturnsNoneimmediately (no env-only mode) - Explicit
profile_name(from per-node selection) skips the active-profile lookup
3.1 Resolution Code Path
Inside a node's forward(), one call resolves everything β get_llm_config(profile_name) (call.py Β· get_llm_config):
ProfileStore.get(name)β disk read with an mtime cache (TTL 2 s).resolve_provider_config(profile)β merge profile + registry defaults, callingget_provider_api_key(provider)=os.environ[ProviderDef.env_var].- If
api_key == "" and api_type != "ollama"β returnNone(mock mode). - Otherwise β
LLMConfig(api_key, base_url, model, api_type, litellm_prefix).
Source: call.py Β· get_llm_config, providers.py Β· get_provider_api_key, providers.py Β· resolve_provider_config.
4. Provider Registry
providers.py defines 24 pre-configured providers. Each entry supplies default base_url, api_type, and default_model, so users only need to provide an API key.
4.1 Supported Providers
| Protocol | Providers | API Type |
|---|---|---|
| OpenAI-compatible | OpenAI, OpenRouter, Together AI, Moonshot, Mistral, xAI (Grok), NVIDIA NIM, Hugging Face, DeepSeek, Baidu Qianfan, Alibaba ModelStudio, Volcengine (ByteDance), BytePlus, Venice AI, MiniMax, vLLM, SGLang | openai |
| Anthropic | Anthropic, Kimi Coding, Xiaomi MiMo, Synthetic | anthropic |
| Google Gemini | google |
|
| Ollama | Ollama (local) | ollama |
| Custom | User-defined (any base URL + API type) | configurable |
Total: 24 providers (count confirmed in PROVIDER_REGISTRY in app/llm/providers.py).
4.2 Registry Entry Structure
@dataclass(frozen=True) class ProviderDef: label: str # Human-readable name (e.g. "OpenAI") base_url: str # Default API endpoint api_type: str # Protocol: "openai" | "anthropic" | "google" | "ollama" default_model: str # Suggested model (e.g. "gpt-4o") litellm_prefix: str # litellm model prefix (e.g. "openai", "gemini", "anthropic") env_var: str # Standard env-var name for the API key (e.g. "OPENAI_API_KEY") # Empty ("") for Ollama β local server, no key needed
When creating a profile with provider="openai", the registry fills in https://api.openai.com/v1, openai, and gpt-4o automatically. The profile can override base_url / api_type / model; litellm_prefix and env_var always come from the registry. The runtime reads os.environ[env_var] at call time (ADR-platform-006) β there is no on-disk slot for the key.
Source: app/llm/providers.py
5. Profile Store
Profiles are persisted in agentcanvas/backend/profiles.json (checked in alongside requirements.txt; override location via AGENTCANVAS_PROFILES_FILE=<absolute-path>). The ProfileStore class manages CRUD with thread-safe locking and mtime-based cache invalidation. The file is git-safe by construction (no secrets) β see ADR-platform-006.
5.1 File Format
{ "schema_version": 1, "active": "openai", "profiles": { "openai": { "provider": "openai", "model": "gpt-4o", "base_url": "", "api_type": "" }, "anthropic": { "provider": "anthropic", "model": "claude-sonnet-4-6", "base_url": "", "api_type": "" } } }
- No
api_keyfield β keys live in env vars (OPENAI_API_KEY,ANTHROPIC_API_KEY, β¦). Legacyapi_keyfields in older files are silently dropped on load and never written back (llm/profiles.py Β· ProfileStore.load) active: name of the default profile used when no profile is specified- Empty
base_url/api_typefields fall back to registry defaults; emptymodelfalls back toProviderDef.default_model - File written with default umask permissions β no
chmod 0o600, since the file carries no secrets (safe for shared checkouts and git)
5.2 Cache Invalidation
ProfileStore avoids re-reading the file on every request:
- In-memory cache of
ProfileData - mtime check gated to at most once every 2 seconds (
_mtime_check_interval) - If file mtime changed, cache is invalidated and re-read on next access
This means CLI changes or manual file edits are picked up by the running server within ~2 seconds, with no restart required.
5.3 Profile Data Model
@dataclass class LLMProfile: """Non-secret per-profile configuration. API keys are NEVER stored here.""" provider: str # Key into PROVIDER_REGISTRY (e.g. "openai") model: str # Model name (e.g. "gpt-4o") base_url: str = "" # "" = use registry default api_type: str = "" # "" = use registry default
LLMProfile is the on-disk shape (no api_key field β ADR-platform-006). At call time, resolve_provider_config() merges it with registry defaults, looks up the env-var key via get_provider_api_key(provider), and resolves litellm_prefix, producing an LLMConfig:
@dataclass class LLMConfig: api_key: str base_url: str model: str api_type: str # "openai" | "anthropic" | "google" | "ollama" litellm_prefix: str # litellm model prefix (e.g. "openai", "gemini", "ollama_chat")
Source: app/llm/profiles.py, app/llm/providers.py (LLMConfig), app/llm/call.py (get_llm_config)
6. Environment Variables
API keys are the only piece of config that comes from env vars β and they are per provider, named after each vendor's de-facto SDK convention. Profile metadata (provider / model / base_url / api_type) lives in profiles.json; there is no global AGENTCANVAS_* or VLM_* env-var fallback for those fields any more (removed in ADR-platform-006).
6.1 Per-Provider Env Vars
Each entry in PROVIDER_REGISTRY declares an env_var field β the env var the runtime reads for that provider's key. Common ones:
| Provider | Env var |
|---|---|
| OpenAI | OPENAI_API_KEY |
| Anthropic / Kimi Coding / Xiaomi MiMo / Synthetic | ANTHROPIC_API_KEY / MOONSHOT_API_KEY / XIAOMI_API_KEY / SYNTHETIC_API_KEY |
| Google Gemini | GEMINI_API_KEY |
| DeepSeek | DEEPSEEK_API_KEY |
| OpenRouter | OPENROUTER_API_KEY |
| Together AI | TOGETHERAI_API_KEY |
| Moonshot AI | MOONSHOT_API_KEY |
| Mistral AI | MISTRAL_API_KEY |
| xAI (Grok) | XAI_API_KEY |
| NVIDIA NIM | NVIDIA_NIM_API_KEY |
| Hugging Face | HF_TOKEN |
| Baidu Qianfan / Alibaba ModelStudio / Volcengine / BytePlus / Venice / MiniMax | QIANFAN_API_KEY / DASHSCOPE_API_KEY / ARK_API_KEY / BYTEPLUS_API_KEY / VENICE_API_KEY / MINIMAX_API_KEY |
| vLLM / SGLang (self-hosted) | VLLM_API_KEY / SGLANG_API_KEY |
| Ollama (local server) | β (no key needed; env_var="") |
| Custom | AGENTCANVAS_API_KEY (the only catch-all) |
Full mapping in providers.py Β· PROVIDER_REGISTRY. To see what's set on a given machine: python -m agentcanvas.backend.app config env.
6.2 Configuring a Provider
One-liner for a typical setup:
# 1. Create the profile (no key, just metadata) python -m agentcanvas.backend.app config set openai --provider openai --model gpt-4o # 2. Export the key in your shell (e.g. ~/.bashrc) export OPENAI_API_KEY=sk-... # 3. Activate the profile python -m agentcanvas.backend.app config activate openai
Note: agentcanvas/backend/.env.example in the repo still shows the legacy AGENTCANVAS_API_KEY/MODEL/BASE_URL/API_TYPE and VLM_* envs. Those credential/model names are dead β never read by call.py β so use the per-provider names above instead. (Two unrelated AGENTCANVAS_* vars are still live and are not credentials: AGENTCANVAS_STRICT_ERRORS (Β§11.5) and AGENTCANVAS_PROFILES_FILE (profile-store path override). File is queued for cleanup.)
6.3 Mock Mode
get_llm_config() returns None whenever it can't resolve a usable config: no active profile, profile name unknown, or the provider's env var is unset (Ollama excepted). Calling nodes treat None as a mock β they emit a placeholder string and continue, so a graph won't crash for missing keys; it will just produce mock outputs visibly in the canvas.
7. CLI Tool
A standalone CLI for profile management that works without the FastAPI server running. Changes are picked up by the running server via mtime invalidation.
7.1 Commands
8 commands total. All accept --json for machine-readable output; show and env also accept --show-keys to unmask values pulled from env vars.
# Entry point (two forms, both work) python -m agentcanvas.backend.app.cli config <command> python -m agentcanvas.backend.app config <command> # List all profiles config list [--json] # Show a single profile (resolved against env) config show <name> [--show-keys] [--json] # Create or update a profile (metadata only β no --api-key flag) config set <name> [--provider <id>] [--model <model>] [--base-url <url>] [--api-type <type>] # On create, prints: "Set the API key via env: export OPENAI_API_KEY=<your-key>" # Delete a profile config delete <name> # Set the active (default) profile; omit name to clear config activate [<name>] # Test API key connectivity (reads key from env var, calls litellm validate) config test [<name>] # List all 24 supported providers (with their env_var names) config providers [--json] # Show which provider env vars are set on this machine + active profile config env [--show-keys] [--json]
API-key entry is intentionally not a CLI step (no --api-key flag, no getpass prompt). The key is read from the provider's env var (OPENAI_API_KEY, β¦) at call time β set it in your shell profile or via your secret manager, then run any command above. ADR-platform-006 keeps the CLI write path entirely free of secrets.
7.2 Example Workflow
# 1. Export keys in your shell (~/.bashrc / ~/.zshrc / .envrc) export OPENAI_API_KEY=sk-... export ANTHROPIC_API_KEY=sk-ant-... # 2. Create profile metadata (one per provider/model pair you want to pick from) python -m agentcanvas.backend.app config set openai --provider openai --model gpt-4o python -m agentcanvas.backend.app config set anthropic --provider anthropic --model claude-sonnet-4-6 python -m agentcanvas.backend.app config set ollama --provider ollama --model llama3.2 # 3. Activate OpenAI as default for nodes that don't specify a profile python -m agentcanvas.backend.app config activate openai # 4. Verify the key actually works (round-trips to the provider) python -m agentcanvas.backend.app config test # 5. Sanity check which env vars are picked up python -m agentcanvas.backend.app config env
Source: app/llm/cli.py
8. REST API
Profile management is exposed via FastAPI endpoints under /api/profiles.
8.1 Endpoints
| Method | Path | Description |
|---|---|---|
GET |
/api/profiles/ |
List all profiles + active + provider registry |
POST |
/api/profiles/ |
Create a new profile |
PUT |
/api/profiles/{name} |
Update an existing profile |
DELETE |
/api/profiles/{name} |
Delete a profile |
POST |
/api/profiles/activate |
Set the active profile |
PUT |
/api/profiles/batch |
Batch upsert (used by frontend Settings Modal) |
GET |
/api/profiles/{name}/models |
Fetch the provider's available models (calls the provider API) |
8.2 Batch Upsert
The frontend sends a single PUT /api/profiles/batch call with all changes:
{ "keys": { "openai": "sk-...", "anthropic": "sk-ant-..." }, "active": "openai", "active_model": "gpt-4o", "overrides": { "ollama": { "base_url": "http://localhost:11434", "model": "llama3.2" }, "custom": { "base_url": "https://my-api.com/v1", "model": "my-model", "api_type": "openai" } } }
keys is accepted but ignored (kept in the request shape only for backward compat with older frontends β see BatchUpsertBody docstring in api/profiles.py Β· BatchUpsertBody). Real API keys live in env vars. The endpoint does create empty profile metadata for each provider listed in keys so the UI's per-provider config slot exists, applies overrides to base_url/model/api_type, and sets the active profile β all atomically.
8.3 Security
- API keys are never in
profiles.jsonβ they live in per-provider env vars (ADR-platform-006). GET responses expose onlyapi_key_set: true/false, derived fromos.environ[env_var]presence at request time profiles.jsonis written with default umask permissions and is git-safe- Frontend Settings Modal still renders a key input field (for legacy frontends), but the value is dropped server-side; the right place to set keys is the shell env, not the modal
Source: app/api/platform/profiles.py
9. Per-Node Model Selection
Individual canvas nodes can override the default profile. LLMCallNode is the only node that carries a profile config field (builtin_nodes.py Β· LLMCallNode):
ConfigField("profile", "select", "Model", default="", options=[{"value": "__DYNAMIC_PROFILES__", "label": ""}])
The __DYNAMIC_PROFILES__ sentinel in options is the load-bearing part: the frontend swaps it for the live list of configured profiles when it renders the dropdown. There is no static option list and no description kwarg.
At execution time:
profile_name = self.config.get("profile", "") llm_config = get_llm_config(profile_name) # "" β active profile β env fallback β None (mock) # "anthropic" β that specific profile
This enables mixed-model graphs: one node reasons with GPT-4o, another captions with Claude, a third does cheap inference with local Ollama.
The frontend renders this as a dropdown in the node's config panel. The dropdown is populated from the list of configured profiles (providers with saved keys + Ollama). A profiles-changed window event refreshes the dropdown when settings are saved.
Source: app/agent_loop/builtin_nodes.py (search for profile ConfigField). The legacy app/agent_loop/graph_executor.py referenced in older docs has been removed; node handlers live in builtin_nodes.py and the executor in graph_executor.py.
10. Frontend Settings Modal
SettingsModal.tsx provides the GUI configuration surface.
10.1 Layout
10.2 Behavior
- Top 5 providers always visible: OpenAI, Anthropic, Google, DeepSeek, Ollama
- More providers collapsible section with 19 additional providers
- Custom provider section for arbitrary endpoints
- Star button sets the default model (active profile)
- Per-provider model override inline text field next to each configured model
- On save:
PUT /api/profiles/batchwith all changes, thenwindow.dispatchEvent(new Event('profiles-changed'))to refresh node dropdowns
Source: agentcanvas/frontend/src/components/SettingsModal.tsx
11. API Call Protocols
One module makes every completion: app/llm/complete.py Β· complete(config, Request) β Completion. A Request carries the current user turn (prompt and/or base64 images with optional image_labels), the prior history, a system_prompt, and the sampling knobs (n, temperature, max_tokens, stop, image_detail, image_mime, retries); a Completion is up to n non-empty texts plus the rulebook adjustments the call actually made. Inside, once: the message builder, the rulebook applied to every parameter, provider-native n with a concurrent single-sample fill, usage accounting, the timeout policy and the strict-errors policy. The wire is the seam: Transport (abc) with LiteLLMTransport as the production adapter (litellm.acompletion, no individual provider SDKs; litellm translates the unified OpenAI-format message structure to provider-native wire formats) and a fake in the tests. The four historical helpers β text-only llm_complete / llm_complete_n and multimodal vlm_complete / vlm_complete_n (app/llm/call.py) β are thin wrappers that reshape their arguments onto one Request; vendored synchronous code (VoxPoser's LMP) uses complete_sync.
11.1 Protocol Dispatch
LLMConfig.litellm_prefix comes from the provider's registry entry (providers.py Β· ProviderDef); API_TYPE_TO_LITELLM_PREFIX (providers.py) is the fallback by api_type when no entry applies:
api_type |
litellm prefix | Examples |
|---|---|---|
openai |
openai |
openai/gpt-4o, openai/gpt-4.1 |
anthropic |
anthropic |
anthropic/claude-sonnet-4-6 |
google |
gemini |
gemini/gemini-2.5-flash |
ollama |
ollama_chat |
ollama_chat/llama3.2 |
The litellm model string is built as {prefix}/{model} by LiteLLMTransport and passed to litellm.acompletion(). litellm handles auth headers, endpoint routing, and protocol translation.
11.2 Image Handling (Request.images)
A Request with images (base64-encoded PNG/JPEG strings; vlm_complete is the wrapper) becomes an OpenAI-format multimodal user turn (complete.py Β· build_messages): the prompt text first, then each image as an image_url block, each preceded by its label when one is given β the same order on every path, single- or multi-sample. litellm translates these to provider-native formats (Anthropic source.type:base64, Google inline_data, Ollama images array) automatically.
An optional image_labels list can interleave per-image text captions before each image (used by MapGPT's gpt_infer interleave shape).
An optional detail argument (Request.image_detail: "low" / "high" / "auto", default "low") goes through the rulebook like every other parameter β where the provider ignores it (anthropic / google / ollama) it is dropped and logged as an adjustment β and otherwise lands on each image_url block as OpenAI's resolution hint: "low" caps every image at a single 512px tile (token-cheap), "high" enables multi-tile high-resolution analysis for fine spatial grounding. vlm_complete_n threads the same hint to every candidate. The builtin llmCall node surfaces this as a second config field β image_detail (select: low / high / auto, default low, builtin_nodes.py Β· LLMCallNode) β so a node whose vision task needs fine grounding (e.g. a viewpoint-picking navigator) can be set to high without touching code.
If images is empty the user turn is plain text β the text-only path, same module.
Multi-sampling. Request.n > 1 (the llm_complete_n / vlm_complete_n wrappers) returns up to n candidate texts, used for self-consistency / multi-candidate planning β the same way on every path, text or vision. When the rulebook lets n through, complete first attempts provider-native n (the prompt is then billed once for all samples); a provider that returns fewer choices or rejects the native call leaves a shortfall, which is filled with concurrent single-sample calls. Where the rulebook says the provider ignores n (anthropic / ollama) the native attempt is skipped and all n samples are single-sample calls. The builtin llmCall node passes config.n on its one Request and exposes the candidates on the responses port.
11.3 Timeouts
| Call type | Timeout |
|---|---|
| Text-only (non-Ollama) | 60s |
| Text-only (Ollama) | 120s |
| Multimodal (non-Ollama) | 90s |
| Multimodal (Ollama) | 120s |
11.4 Public API
from app.llm import get_llm_config, llm_complete, vlm_complete config = get_llm_config("openai") # or "" for active profile # Text-only response = await llm_complete(config, messages, system_prompt="...", max_tokens=1024) # Multimodal (text + list of base64 images, optional per-image labels) response = await vlm_complete( config, prompt="Describe these images", images=[base64_png_1, base64_png_2], image_labels=["Front view", "Side view"], # optional )
Sampling parameters are two-state. temperature and max_tokens default to None on every helper and on Request; None is omitted from the wire so the vendor's own default applies. Explicit values β together with stop, n and image_detail β pass the rulebook (rulebook.py Β· finalize_params) before the wire: locked values are forced, required ones injected (Anthropic's max_tokens), ranges clamped, unsupported ones dropped, and every deviation comes back on Completion.adjustments (the llmCall node logs them as param_adjustments).
Source: app/llm/complete.py, app/llm/rulebook.py, app/llm/call.py
11.5 Strict errors & the gpt-5 rule
By default every helper degrades gracefully: a litellm exception, timeout, or unparseable response makes llm_complete return None and the *_n variants return [], so a misconfigured node renders a placeholder rather than crashing the run. Setting the env var AGENTCANVAS_STRICT_ERRORS (to 1/true/yes/on, read fresh on every call via complete.py Β· strict_errors_enabled β deliberately uncached, so env-worker subprocesses inherit the backend's launch-time setting) flips this: every helper re-raises instead. Eval and smoke runs set it so a silent empty response can't masquerade as a completed episode.
The gpt-5 family is a rulebook entry, not a sharp edge. OpenAI reasoning models (gpt-5*, o1* / o3* / o4*) accept only the default temperature and swallow small max_tokens in their reasoning budget. rulebook.py records that as verified facts: temperature is locked to 1.0 (an explicit other value is forced and logged as an adjustment; an unset one is never sent) and max_tokens carries a min_hint of 2000 that the Settings UI surfaces but the call never alters β so a node that pins max_tokens=500 on gpt-5 can still get an empty reply. The same table drives GET /api/providers/{id}/capabilities (rulebook.py Β· get_capabilities).
12. Per-Node Usage Hook (ADR-observability-005)
Token usage and dollar cost are tracked per node firing, not per nodeset and not globally. Nodesets never need to plumb usage manually β the wiring is one ContextVar in standard/telemetry.py, one accumulator in complete.py, and one set/reset pair around forward() in the executor.
# standard/telemetry.py Β· _current_node_usage (complete.py accumulates into it) _current_node_usage: contextvars.ContextVar[dict | None] = contextvars.ContextVar( "_current_node_usage", default=None ) # Every transport round trip made by complete() inside a forward() β the # llm_complete / vlm_complete wrappers included β adds its usage via _accumulate_usage(): # bucket["calls"] += 1 # bucket["prompt_tokens"] += response.usage.prompt_tokens # bucket["completion_tokens"] += response.usage.completion_tokens # bucket["total_tokens"] += response.usage.total_tokens # bucket["cached_tokens"] += response.usage.prompt_tokens_details.cached_tokens # bucket["usd_cost"] += litellm.completion_cost(response) # bucket["model"] = first non-empty response.model
# graph_executor.py Β· _fire_node usage_bucket: dict = {} usage_token = _current_node_usage.set(usage_bucket) try: result = await node.forward(inputs, ctx) finally: _current_node_usage.reset(usage_token) # bucket now holds aggregate usage for this firing; executor emits one # log entry per node with the bucket attached.
Properties of this design:
- Voluntary, not invasive β nodes that never call
complete(or a wrapper) get an empty bucket; nodes that call N times get one aggregated entry, not N events - Per firing, not per node lifetime β multi-scope graphs and iterated nodes accumulate per call site, exactly what you want for cost reports
- Works inside subprocesses β ContextVars propagate into asyncio tasks; the bucket survives any internal
awaitchain. Server-mode nodesets capture usage in the subprocess (server/invoke.pysets the same ContextVar around the hosted handler) and return it in the/callreply'susagefield; the proxy node merges it into the firing's bucket (proxy.py Β· _make_forwardβtelemetry.merge_usage), so the framework attributes it to the proxy node - Cost may be zero for models litellm doesn't recognise (local Ollama, custom vLLM endpoints) β usage tokens still counted,
usd_coststays 0.0
Source: standard/telemetry.py Β· _current_node_usage (the bucket) + complete.py Β· _accumulate_usage (the accumulator), graph_executor.py Β· _fire_node (set/reset around forward()), complete.py Β· _extract_usage (litellm response β usage dict, inside LiteLLMTransport).
13. Key Files
| File | Purpose |
|---|---|
app/llm/complete.py |
Request / Completion, complete() / complete_sync(), build_messages, the rulebook application, native n + fallback, usage accumulation, timeout and strict-error policy; Transport (abc) + LiteLLMTransport |
app/llm/call.py |
get_llm_config() / get_llm_config_direct() (profile β LLMConfig), and the four thin wrappers llm_complete() / llm_complete_n() / vlm_complete() / vlm_complete_n() |
app/llm/rulebook.py |
RULEBOOK (verified per-provider / model-family parameter facts), finalize_params() (call time), get_capabilities() (edit time) |
app/llm/providers.py |
PROVIDER_REGISTRY (24 providers, each with an env_var), ProviderDef, LLMConfig, API_TYPE_TO_LITELLM_PREFIX, get_provider_api_key(), resolve_provider_config() |
app/llm/profiles.py |
ProfileStore, LLMProfile (no api_key field), ProfileData, mtime cache; legacy api_key entries dropped on load |
app/llm/cli.py |
Standalone CLI (8 commands, no server required); no --api-key flag (ADR-platform-006) |
app/llm/key_validator.py |
API key validation for config test (litellm + Ollama health check) |
app/llm/__init__.py |
Re-exports: complete, complete_sync, Request, Completion, get_llm_config, llm_complete, llm_complete_n, vlm_complete, vlm_complete_n, LLMConfig, LLMProfile, PROVIDER_REGISTRY, resolve_provider_config, get_profile_store, finalize_params, get_capabilities |
app/api/platform/profiles.py |
REST endpoints (/api/profiles/); BatchUpsertBody.keys accepted-but-ignored |
app/agent_loop/graph_executor.py |
Per-node usage bucket set/reset around forward() (_fire_node); emits one usage log entry per node firing (ADR-observability-005) |
frontend/src/components/SettingsModal.tsx |
GUI settings modal β calls PUT /api/profiles/batch; keys field in payload is silently ignored server-side |
agentcanvas/backend/profiles.json |
Default profile store location (git-safe; override via AGENTCANVAS_PROFILES_FILE) |
agentcanvas/backend/.env.example |
Env template β the AGENTCANVAS_* credential / VLM_* envs it lists are dead; use per-provider env vars instead. (AGENTCANVAS_STRICT_ERRORS and AGENTCANVAS_PROFILES_FILE remain live, non-credential.) |