Human Performance Tab

The Human tab puts a person in the exact seat the coding agents sit in: same env_habitat episodes, same discrete action space (0 = STOP, 1 = forward 0.25 m, 2/3 = turn 15°), same 512 px default render, and — the hard promise — the same ruler: every number on the tab comes from habitat's own measures via env_habitat__evaluate, computed by the same auto_host surface the agent drivers call, never browser-side. Human SR/SPL therefore sits directly against the standard board. The agent counterpart of this tab is the Coding-Agent Monitor; the experiment itself is described in the capability page.

The browser owns the control loop. One keypress becomes one POST /api/human/step; the runner translates it into the same auto_host HTTP calls the agent drivers make and hands the new egocentric frame back inline (base64 PNG) in the same response.

1. What it does

It measures human performance on the benchmark the agents run. You pick an episode from the 100-cell grid (rand100 split), read the same instruction the agents get, walk the scene one discrete action at a time from the egocentric view, and press STOP when you believe you are at the goal. The env then scores you exactly as it scores an agent — SR, oracle SR, distance-to-goal, nDTW, SPL — and the tab keeps per-episode records plus a running aggregate. The whole session surface is four HTTP calls:

# 1. GPU up (blocks-then-polls; ~seconds for a warm scene cache)
curl -X POST :8000/api/human/start-server -d '{"split": "rand100"}'
# 2. place episode 7, get instruction + first frame (base64 PNG inline)
curl -X POST :8000/api/human/episode/7/load -d '{"rgb_resolution": 512}'
# 3. one keypress = one discrete action (1 fwd / 2 left / 3 right)
curl -X POST :8000/api/human/step -d '{"action": 1}'
# 4. STOP + evaluate + persist — habitat computes SR/OSR/NE/nDTW/SPL
curl -X POST :8000/api/human/stop

2. Session protocol — one lock, one session, driver-side everything

HumanRunner is a lifespan singleton owning at most one auto_host and one live episode; a threading.Lock serializes every env HTTP call with every session mutation (human_runner.py · HumanRunner).

• Server lifecycle — start_server reuses a live, health-checked auto_host or respawns fresh; the interpreter and source file come from the registry's discovered env_habitat nodeset (import boundary, mirroring CodingAgentRunner). Dataset + split are pushed once at spawn — a split push re-initializes the env against its YAML, the expensive part — and per-episode work only ever pushes episode_index afterwards (HumanRunner._spawn). The call is deliberately blocking on a pooled to_thread worker: PR_SET_PDEATHSIG fires when the spawning thread dies, so spawning from a short-lived thread would SIGTERM the env the instant that thread returned (HumanRunner.start_server docstring).
• Episode session — load_episode(i) (index gated to 0–99) arms the placed episode and returns instruction + first frame; re-loading a tested episode starts a fresh trajectory (the UI's "re-test", record overwritten on the next stop). step(a) accepts only 1/2/3, executes env_habitat__step_discrete, observes the new frame + pose, and folds env-side terminated/truncated into done (a budget truncation ends the episode without a human STOP, end_reason="budget").
• STOP + evaluation — stop() issues action 0 if the episode is still live, then calls env_habitat__evaluate and persists the record. STOP is not accepted through step() so the confirm gate and evaluation stay in one place (HumanRunner.step raises on action 0).
• Liveness — server_status() health-probes outside the lock so a routine poll never blocks an in-flight step; a vanished auto_host folds into a recoverable error state whose message names the usual culprit (backend auto-reload) and the recovery (Start Session again).

3. Persistence & aggregate

Every action and trajectory coordinate is persisted; the aggregate is a plain mean over tested episodes. Under outputs/human/{split}/: episode_{i}.jsonl carries episode_meta (instruction, scene, start position, render resolution) plus one step/stop line per action (action name, position, orientation, terminated/truncated) and a final metrics line; summary.json holds one record per tested episode (metrics, step counts, called_stop, end_reason, wall time) with re-tests replacing their index in place, plus an aggregate that averages every numeric metric and num_steps over the tested set (human_runner.py · _persist_record / _aggregate).

4. The UI surface

The page is a thin, keyboard-first shell over the four endpoints (HumanPage.tsx): a split selector, a 100-cell episode grid with tested/success badges fed by GET /status, the egocentric frame with instruction overlay, a D-pad, and a metrics panel after STOP.

Key events are ignored while a form field has focus, and movement keys are dead while a request is in flight (busy) or the episode is done — the browser enforces the one-action-at-a-time loop the runner's lock assumes.

5. Where it deviates from the agent protocol

6. Key files