Gemma 3 NodeSet

The Gemma 3 NodeSet (VLMGemma3NodeSet, workspace/nodesets/model/vlm_gemma3.py) wraps Gemma 3 via transformers AutoModelForImageTextToText — Google's open multimodal model, a compact but strong image+text VLM in the same family as Gemma 3's text models. It exposes the same generic generate primitive (a chat message list plus image paths → text) as the other VLM nodesets. Gemma 3 is an image VLM: it reasons over one or more still images and text, and — unlike Qwen3-VL, InternVL3, or SmolVLM2 — has no video port.

env: ac-fm (shared FM env) · backend: transformers AutoModelForImageTextToText + AutoProcessor · default: google/gemma-3-4b-it (gated — accept the licence first, §2)

1. One primitive: multimodal generate

The tool takes either a full chat messages list (preferred — the caller builds the turns) or a plain prompt string, plus a list of image file paths (or URLs) on shared disk. Images are attached to the most recent user turn as chat content blocks; the processor loads them at template time. Stop sequences are truncated from the generated text. The contract mirrors the other VLM generate nodes, minus the video_paths port — Gemma 3 accepts images only.

2. Gated model — accept the licence first

google/gemma-3-* is a gated model. Before the weights will download you must, one time, accept the Gemma license on the model's Hugging Face page and be logged in with an HF_TOKEN that has access. Without that access the engine load latches degraded (§9): the text port returns "" and the load is not retried. To fix it, open https://huggingface.co/google/gemma-3-4b-it, accept the licence, ensure HF_TOKEN is set in the server env, and reload the nodeset — the download then proceeds normally.

3. Messages, image paths, stops

image_paths accepts a JSON list (or a single string). Images are attached to the last user turn — a turn already in block form is left untouched, so a fully-built messages list passes through verbatim. All generation knobs (max tokens, temperature, top-p, repetition penalty) are node config; temperature = 0 selects greedy decoding.

4. Model & generation

The engine builds the input via the modern apply_chat_template(tokenize=True, return_dict=True) path — the processor loads images from path/URL directly. Generation is greedy or sampled per config, then the prompt tokens are trimmed off before decoding:

msgs = [{"role": "user", "content": [{"type": "image", "image": image_path},
                                     {"type": "text", "text": prompt}]}]
inputs = processor.apply_chat_template(msgs, tokenize=True, add_generation_prompt=True,
                                       return_dict=True, return_tensors="pt").to(model.device)
gen = model.generate(**inputs, max_new_tokens=2048, do_sample=False)
text = processor.batch_decode(trimmed, skip_special_tokens=True)[0]

Each model_id is a lazy singleton in a registry. On CUDA it loads in bfloat16 with flash-attn when the wheel is present (sdpa fallback otherwise); on CPU it loads in float32. Generation is single-flight per engine (concurrent KV-cache growth can CUDA-OOM under K eval workers). Verified 2026-07-08: the default google/gemma-3-4b-it resolves to Gemma3ForConditionalGeneration / Gemma3Processor, and an in-process forward() on CPU correctly answered a one-word query about an image.

5. Output

The text port is the decoded generation with the input prompt trimmed and any configured stop sequences cut. Empty on load failure or error (§9).

6. Canvas Node

vlm_gemma3__generate

7. Server Mode

parallelism = "shared" (K callers coalesce through one hosted copy; no per-call state); server_python = conda_env_python("ac-fm", "GEMMA3_PYTHON"). Gemma 3 runs in the shared ac-fm env — transformers loads it through AutoModelForImageTextToText + AutoProcessor there. Weights are HF-downloaded (gated, §2) and load lazily on the first generate. The default gemma-3-4b-it is the smallest multimodal Gemma 3 (the 1B is text-only); point model_id at a 12B / 27B-it checkpoint on a bigger GPU. The file stays Python-3.8-parseable. Load: POST /api/components/nodesets/vlm_gemma3/load?mode=server.

8. Environment Variables

9. Degraded Mode

A weight-load failure latches (_load_failed — no retry storm), and any generation exception is caught; both yield "" on the text port with a degraded/error self-log. The most common cause is gated access (§2): an un-accepted licence or a missing HF_TOKEN latches degraded on first load. An empty string is a clean "no answer this step" signal; consumers keep their own fallback and never receive a fabricated reply.