Coding-Agent Runner

The coding-agent runner is the machinery behind the Coding-Agent Harness: one shared core (coding-agent/driver.py) that runs any harness × model × condition cell of the standard board. Its hard promise is that the harness is the only moved variable — episode placement, prompts, the tool surface, evaluation, retry policy, and the artifact layout are one implementation shared by all three harnesses and both entry points, so two cells differ in nothing but what their names say. For what the experiment measures, read the capability page; for the frontend windows onto the artifacts, the Monitors group; for run recipes, coding-agent/README.md.

One core, two entries, three adapters. Both entries resolve to run_cell; every adapter runs one clean session per episode against the same tool surface; only EventSink writes the log vocabulary. The frozen legacy drivers gate the live surface from below.

1. What it does

It runs one cell — a (harness × model × condition × effort-tier) point of the standard board — end to end: place episodes on an env_habitat auto_host, render the frozen briefing, hand each episode to the harness adapter for one clean agent session, score it with habitat's own env_habitat__evaluate, and write the uniform artifact set the Monitor renders. Before unification (2026-07-20) each harness carried its own full driver; the shared 90% now lives here once, and each harness is a ~100-line adapter. The smallest use is one command — a cell name pins every protocol knob:

# a cell name pins everything; the only free choices are which cell, which servers
python coding-agent/stdrun.py run std_sdk_fable-5_bare_default
python coding-agent/stdrun.py run E17            # paper E-numbers resolve to cells
python coding-agent/stdrun.py board             # protocol truth lives on disk, not in names

2. One episode through the core

run_cell owns everything an episode needs except the agent loop itself (driver.py · run_cell / run_episode):

1. Placement — dataset + split pushed once per server, then per episode: episode_index field + play + env_habitat__reset (carrying the frozen rgb_resolution). All env HTTP rides asyncio.to_thread so parallel workers never stall the loop.
2. Briefing — prompts.py · build_briefing renders the frozen 07-09 drafts (BARE / FULL / WP) with the instruction and budget; a nav cell first passes assert_std_skill_freeze, which refuses to run if the skill body's md5 drifted from f7c74272.
3. Session — a fresh EpisodeContext (instruction, briefing, budgets, bridge env vars, live/raw dirs) goes to adapter.run(ctx, sink); the adapter may emit only through the EventSink, which enforces the log vocabulary by construction and derives tool-call counts + the last step() result uniformly.
4. Scoring — the driver (never the agent) calls env_habitat__evaluate while the trajectory file is still open, so metrics land inside the log; summary.json is rewritten after every episode with aggregate applying the board 口径 (an evaluated episode scores as-is; an unevaluated timeout scores 0; rate_limited and non-engaged infra errors are excluded, driver.py · aggregate).
5. Batch management — N workers pull from one index queue (one env server each, staggered cold loads); --episodes reruns/resumes specific indices into the same run dir; a DRAIN sentinel / SIGUSR1 finishes in-flight episodes and prints the ready-to-paste resume spec; subscription throttling backs off outside the episode's timed scope (the "paused countdown") for up to 6 attempts; a VramSampler records the GPU peak.

3. Who calls what — one door, two clients, one translator

Everything reaches the environment through a single HTTP door — the auto_host surface — but its two clients hold different vocabularies. The driver talks to the door directly and owns the full vocabulary: episode selection and play on the env panel, then env_habitat__reset before the session and env_habitat__evaluate after it (driver.py · run_episode, signing its calls with a literal {"trigger": "driver"}). The agent never sees that door. It speaks MCP to the bridge, and the bridge — itself just another HTTP client of the same door (mcp_bridge.py · _call) — translates exactly two node types: env_habitat__observe_egocentric and env_habitat__step_discrete (look_around is composed from them: spin-in-place turns, one frame per stop). reset and evaluate are not blocked by permission — they simply do not exist in the translated vocabulary, so no harness can reset mid-episode or peek at its score. mini takes the same subset through an in-process shortcut instead of MCP (mini/toolset.py, byte-equal gated in §6). Inside the server, state lives in exactly one place: the per-process HabitatEnvManager singleton (env_habitat.py · HabitatEnvManager.get), whose single-thread executor pins habitat's GL context. The worker structure is what makes this safe: run_cell spawns one worker per env server, workers pull episode indices from one shared queue, and each worker runs its episodes serially against the one auto_host it owns exclusively for the whole run (driver.py · run_cell) — so a manager never faces two live sessions, without any locking. Parallelism is process replication (N workers ⇒ N auto_host processes, one manager each); the agent session and its bridge/toolset are born fresh per episode under their worker and die with it.

Entries → workers → the door. Both entries resolve to run_cell, which spawns one worker per env server off a shared episode queue; each dashed box below is one worker — serial episodes, exclusive owner of its auto_host. Inside a worker, the driver and the tool surface are both HTTP clients of that one auto_host door; the agent itself never touches HTTP — it speaks MCP to the bridge (or direct python to mini's toolset), and the translated vocabulary simply lacks reset and evaluate.

4. The adapter contract

An adapter is the harness-specific residue after the core took its 90% — the HarnessAdapter protocol is four methods (driver.py · HarnessAdapter): prepare (once per run: auth guards, version pins into the recorded inherent dict, any serving stack the harness owns — raising aborts the cell, because a silently degraded server is worse than no run), describe (the session-config block logged into session_inputs), run (one clean session, events through the sink only), and optional finalize (the audit only that harness can produce).

5. The registry — cells, not flags

Comparability is enforced structurally: protocol knobs live in cells.py, not on the command line. STD_FROZEN pins std-v2 (R2R-CE rand100 eps 0–99 · 200 turns · 512 px · 500 actions · 2400 s); CONDITIONS defines bare / nav / persona / wp / wp-nav; EFFORT_TIERS runs each main cell at default and max, with _tier_extra resolving what each label concretely means per harness; MODELS + MODEL_ID_OVERRIDE map board columns to harness-facing slugs; BATCHES names sequential cell lists (treatment-before-control ordering for cuttable local batches); EXPERIMENTS maps the paper's E-numbers onto cells, and resolve_cell accepts either form. Any override requires --nonstd, which renames the run nonstd_* so it can never sit on the board (stdrun.py · _run); the Monitor's uirun.py gets the same treatment structurally — its free knobs make every UI run ui_*, off-board by name. compare runs an exact paired McNemar over same-episode successes, refusing pairs whose episode_ids diverge (stdrun.py · _compare).

6. Freeze & equivalence gates

"Same experiment across harnesses" is asserted, not assumed. Three gates: (a) the skill md5 freeze (§2); (b) mini/check_equivalence.py proves the mini in-process toolset byte-equal to the bridge — tool names, descriptions, schemas, clearance math, waypoint geometry, down to identical annotated-PNG bytes; (c) the same script asserts the live prompts.py texts byte-equal to the frozen legacy SDK driver they were moved from. The fixtures are the point of coding-agent/legacy/: the pre-unification drivers are kept unedited so the live surface can always be diffed against what produced the archived runs.

7. Where it deviates from the mental model

8. Key files