LLM Call Node

Each time an llmCall node fires, it sends one chat request: the template turns wire data into a user message, the node resolves which model to talk to, and the request goes out through litellm — in vision mode whenever images arrive. The hard promise: what leaves the machine is locally valid. Parameters you never set are never sent; values a model is known to reject are corrected and logged before the wire.

One firing. Five steps, and the cost of the call comes back to the card.

1. What it does

llmCall is the built-in node that turns wire data into LLM calls. Three ideas explain how it behaves:

• Requests are correct by construction. A sampling parameter is either set or unset. Unset means it is not in the request at all — the vendor's default applies. Set values are checked against known per-model rules before sending: give a gpt-5 model temperature=0.2 and the node sends 1.0, and logs that it did.
• The card is a gauge, not a form. The canvas card only shows: which model, which parameters, how the template will resolve, what the last call cost. Editing happens in the properties panel, where the same rules lock or clamp the controls — a constraint stops you while editing, not at run time.
• Fixed setup, changing content. The system prompt never changes and is sent verbatim every step. The template re-renders every firing from whatever the wires deliver.

2. The smallest call

One TEXT port and a template — a working LLM node. It follows the active profile, sends no sampling parameters, no system message.

{
  "id": "ask",
  "type": "llmCall",
  "config": {
    "ports": [{ "name": "context", "wire_type": "TEXT" }],
    "template": "Summarize in one line: {context}"
  }
}

Wire any TEXT producer into context; the node fires when data arrives. Want vision? Add an rgb port of type LIST[IMAGE] — that's the whole switch.

3. The prompt

The prompt has two parts with different lifetimes. The system prompt is fixed: sent verbatim as the system message, never templated. The template is live: it renders the user message fresh on every firing.

Each {var} fills from the input port with the same name, or — if the node has an access grant — from graph_state. A variable that finds no value goes one of two ways:

• It names a real port or state key that just has no data this step → it renders as (no var) and the call proceeds. Deliberate grace for optional inputs, like {history} on step 0.
• It names nothing — a typo, a missing port, a key this node can't read → the firing fails with an error naming the variable. A broken prompt never silently reaches the model.

The rendered prompt is written to the run log every firing, so you can always see exactly what the model saw. Images arriving on the rgb port ride along with the user message, optionally captioned one-to-one by image_labels.

4. Choosing the model

A node stores a reference, not a model. It resolves fresh at every firing, in one of three modes:

The pieces live where they belong: providers (endpoints, protocols) are code; API keys live in ~/.agentcanvas/.keys, written from Settings → Credentials and never inside the repository; profiles live in profiles.json, safe to commit because keys never enter it.

If nothing resolves — profile missing, or the provider has no key — the node sends nothing and returns a mock reply, (no LLM profile active). The card warns you before you run: the model chip turns red when the reference is broken, amber when the provider needs a key you haven't set.

5. Sampling parameters

temperature and max_tokens start unset — the panel shows a dashed “provider default” button until you deliberately pick a value. Unset means absent from the request. Not zero, not a framework default: absent.

Values you do set pass a rulebook of verified per-model facts just before sending. A model with no entry passes through untouched. Current entries:

Every correction is logged in the node's run log (param_adjustments). Usually you never get that far: the properties panel shows the same rules while you edit — locked fields go read-only with a 🔒, ranges clamp the slider, unsupported fields dim.

6. On the canvas

The card shows, the panel edits. The card never folds and carries four read-only elements:

• Model chip — what this node will talk to. ↪ = following the active profile; red ⚠ mock = broken reference; amber ⚠ no key = provider needs a key.
• Model & Sampling — the parameter values; unset ones read provider default in italics.
• Prompt preview — the template, each {var} tinted by how it will resolve: sky from a port, violet from graph_state, amber matches nothing and will fail.
• Usage line — what the last firing cost: tokens in→out · latency · $.

Click the node and the right-side properties panel opens, grouped into Model & Sampling, Prompt, and Wiring. The model row picks between the three modes of §4 — “Browse models…” lists a provider's live models and pins one onto the node.

7. Under the hood

Details you rarely need, collected in one place. Code lives in builtin_nodes.py · LLMCallNode.forward, call.py, rulebook.py, and graph_executor.py · _fire_node.

7.1 Conversation mode

mode="conversation" keeps chat history in a graph_state entry named messages: the node appends the user message before the call and the assistant reply after. It needs an access grant on the node — without one the firing fails. It works on both the text and vision paths; with images, earlier turns are replayed as text and images belong to the current turn only. Conversation mode ignores n.

7.2 Outputs, failure, extras

response carries the reply; responses carries all samples when n>1 (providers without native multi-sample are backfilled with concurrent single calls). A failed provider call is logged and yields an empty response; set AGENTCANVAS_STRICT_ERRORS=1 to make it raise instead. write_to_state copies the reply into a graph_state key. A config-only stop key (no UI field) forwards stop sequences. A legacy prompt port bypasses the template with pre-rendered text — don't use it in new graphs.

7.3 Usage metering

The executor meters every firing: it sets up a usage bucket, every LLM call adds its tokens and litellm-computed cost, and afterwards the bucket goes into the run log (usage) and out as an llm_usage WebSocket event that updates the card. The hook lives in the executor, so nodeset nodes calling llm_complete get the same accounting for free.

8. Where it deviates from the mental model