(knowledge-distillation orchestrator)
/architect:myloop:loop — a hard-paired
THINK → EXPERIMENT → DISTILL cycle over one frozen graph,
driven by a user-authored goal.md. Maximizing agent
performance is the goal; distilling understanding against
that goal is the implementation mechanism the loop uses to get there,
not an end in itself.
shape three-phase hard-paired loop memory 8 typed files, no archive termination goal-driven, not iter-capped spawns/iter 2 sub-agents
log.jsonl); a dedicated DISTILL phase converts that into
typed, human-readable working memory before the next iter's
THINK plans from it. THINK reasons mostly from that distilled memory —
it may dip into raw logs to target an episode subset, but it never has
to digest a whole run. The file count, the phase split, the two-spawn
cost, the atomic-commit boundary all follow from making distillation a
first-class phase.
ADAS-family variants fold "what just happened" into the next iter's THINK — one sub-agent spawn per iter, but THINK must digest the raw eval artifacts itself. myloop pays for a second spawn so each phase has a single cognitive task: DISTILL converts raw → typed memory while the iter's context is still warm; THINK plans the next experiment from that memory. THINK may still read raw logs — but narrowly, to pick the episodes for a targeted subset, not to digest a run. The split is by cognitive task and timing, not by log access:
The two spawns are not redundant. Each has a single cognitive task on focused input:
record.json
+ the live workspace/. It MAY read raw eval logs —
selectively — to construct a targeted (e.g. failure-mode) episode
subset, but it does not digest a whole run.One tool-augmented sub-agent spawn. Reads goal, knowledge, experience, hypotheses, last 3 iters' records, experiment design, tools, current graph — and may sample raw eval logs to compose a targeted episode subset.
May eager-write hypotheses.jsonl,
knowledge.md, experiment_design.yaml,
tools/*.py during its turn — those writes are not
staged; they go straight to the persistent vN/ files.
Mandatory output
.staging/iter_M/spec.json — a complete
ExperimentSpec. SKIP_THINK_EMPTY only if nothing is
left to test.
Iter-0 rule
iter_0 is a no-patch baseline run
(spec.patch=null) on the search tier (or
perf) — it establishes both a comparison point and the
per-episode log corpus later iters mine for failure-mode subsets.
Owned by loop.md § 3c itself — not a reasoning
sub-agent phase like THINK / DISTILL. myloop has no implementer
skill: the apply step is inline (the loop being a coding agent),
and it does spawn one isolated editing sub-agent per attempt.
If spec.patch non-null, the inline apply step
realizes it into .staging/iter_M/active_workspace/ via
an editing sub-agent and smoke-retries up to 3 attempts. Failures
still count — see Failure semantics.
Then /experiment:run with
spec.eval_profile.name + the staging workspace as
overlay. Probe of 1 ep is the floor; the rhythm cannot be broken.
Output
.staging/iter_M/eval_metadata.json
(run_id, aggregate metrics, per-ep summary,
outcome_class).
Second sub-agent spawn, fired immediately after EXPERIMENT while
context is warm. Reads this iter's spec / think-trace / eval-metadata
plus the raw per-episode log.jsonl files. Sample 2–3 eps
deeply rather than skim all eps shallowly.
For every open hypothesis: decide
confirmed | refuted | inconclusive. Resolved ones get
line-deleted from hypotheses.jsonl and appended to
experience.jsonl atomically (same DISTILL turn).
Output
.staging/iter_M/distill.json
— merged into record.json at commit; omitted on
SKIP_DISTILL_EMPTY (valid; the phase still ran).
myloop deliberately fans out what ADAS keeps in one
archive.jsonl into eight files, each one capturing a
different kind of knowledge. Confusing the kinds is the
fastest way to break the loop — e.g. logging "we tried gpt-4o and
failed" into knowledge.md instead of
experience.jsonl would treat a lesson as a fact.
Set by the user before the run; loop never writes these.
| File | Kind of knowledge |
|---|---|
goal.md |
Direction. Three sections: Ultimate (where to go), Escalation (when to upgrade probe → search → perf), Termination (predicates polled every iter — first hit exits). |
constraints.md (optional) |
Invariants. MUST / MUST NOT bullets
that every phase prompt injects as a non-negotiable preamble.
Bootstrap-merged from up to 4 layers (see § Goal and constraints). |
The "what we know now" state, sharpened iteration by iteration.
| File | Kind of knowledge | Writer |
|---|---|---|
knowledge.md |
Pure facts about system / graph / env / dataset. Append-only in v1; corrections go in as superseding bullets. | THINK + DISTILL (eager) |
experience.jsonl |
Lessons learned — closed-case empirical results, both confirmed and refuted. One entry per resolved hypothesis at minimum. | DISTILL (eager) |
hypotheses.jsonl |
Open conjectures awaiting test. Mutable: append on creation, line-delete on resolve. This is a queue consumed by experiments, not a log. | THINK + DISTILL (eager) |
Per-iter snapshots; appended once per committed iter.
| File | Kind of knowledge | Writer |
|---|---|---|
iteration/iter_M/record.json |
Dense IterRecord — think rationale + experiment
params + metrics + distill block + cost. Written once at commit.
The audit trail. |
loop (commit) |
Note: each iter dir also holds the un-promoted files from
.staging/iter_M/ — spec.json,
think_trace.md, eval_metadata.json,
distill.json, distill_trace.md,
active_workspace/ (if patched), debug_log.md
(if the apply step ran). These are the forensic primaries;
record.json indexes them.
The interplay between hypotheses.jsonl and
experience.jsonl is the loop's central learning mechanic.
A hypothesis is a liability — it owes the loop a test.
Experience is an asset — a closed case that future THINKs can
reason from without re-running the experiment.
Two consequences:
experience.jsonl entry. The loop doesn't
distinguish "we proved it" from "we disproved it" — both are
knowledge.Within an iter, writes go to two different places. The atomic commit at iter-end only promotes one of them — the other is already on disk.
THINK and DISTILL write to these during their sub-agent
turn, directly to the vN/ root:
knowledge.md (both phases)hypotheses.jsonl (both phases — append + delete)experience.jsonl (DISTILL only)experiment_design.yaml (THINK only)tools/*.py (THINK only)If THINK proposes a probe against a brand-new hypothesis, that hypothesis must be visible to EXPERIMENT and DISTILL in the same iter — eager-writes are how that works. The trade-off: these writes survive crashes mid-iter, even if the iter ultimately fails to commit. Recovery is by reasoning (next THINK reads the partial state), not rollback.
Files written to .staging/iter_M/ during the iter,
then bulk-promoted at commit:
spec.json (THINK output)think_trace.md (THINK forensic trace)eval_metadata.json (loop, after EXPERIMENT)distill.json (DISTILL output)distill_trace.md (DISTILL forensic trace)active_workspace/ (if patch applied)debug_log.md (if the apply step retried)Atomic commit is therefore minimal: one mv + one
record.json write. If the commit fails, the iter dir
is rolled back; eager writes from Channel A remain — the next iter
deals with them.
outputs/design_runs/myloop/{graph}/v{N}/
├── goal.md USER-AUTHORED, read-only to orchestrator
├── constraints.md OPTIONAL — merged from 4 layers at bootstrap
├── knowledge.md append-only facts
├── experience.jsonl lessons learned
├── hypotheses.jsonl open conjectures (mutable: line-delete)
├── experiment_design.yaml eval-profile registry
├── tools/ orchestrator-authored *.py utilities
│ └── *.py
├── iteration/ one subdir per committed iter
│ ├── iter_0/
│ │ ├── record.json dense IterRecord
│ │ ├── spec.json ExperimentSpec produced by THINK
│ │ ├── think_trace.md THINK sub-agent forensic trace
│ │ ├── eval_metadata.json run_id + aggregate metrics + per-ep summary
│ │ ├── distill.json DISTILL output (omitted on SKIP_*)
│ │ ├── distill_trace.md DISTILL sub-agent forensic trace
│ │ ├── active_workspace/ patched workspace overlay (if patch applied)
│ │ └── debug_log.md EXPERIMENT apply-step smoke-retry history (if any)
│ └── iter_1/, iter_2/, ...
├── .staging/iter_M/ transient; mv'd to iteration/iter_M/ on commit
├── .loop_state/ bookkeeping for resume / termination
│ ├── last_committed_iter
│ └── consecutive_skips
└── SUMMARY.md written at termination
myloop has no transactions, no revert chains, no rollback semantics.
When things go wrong, the design is to archive the failure as
evidence and let the next iter's THINK reason about it. The
invariant: every committed iter has a think
block and an experiment block, even if the
experiment was "the patch failed smoke retry 3 times" or "the eval
crashed at episode 2".
| Failure mode | Loop response | What gets recorded |
|---|---|---|
| Apply-step SKIP (patch fails smoke retry 3×) | Skip /experiment:run; commit the iter anyway with
outcome_class="implementer_skip". DISTILL still runs and
typically writes a refuted-patch experience. |
The patch and its failure mode become an
experience.jsonl entry: "patch X failed because Y". |
Eval crash (/experiment:run non-zero, missing _DONE) |
outcome_class="crash"; capture stderr tail; commit
iter; DISTILL still runs. |
Crash context goes into experience.jsonl. Often
surfaces hidden bugs in the patched code. |
| THINK SKIP_THINK_EMPTY (nothing left to test) | Terminate run with status SATURATED. This is a
clean exit, not a failure. |
No iter committed; SUMMARY.md records the saturation. |
| THINK / DISTILL malformed output | SKIP_INVALID_SPEC / SKIP_INVALID_DISTILL;
consecutive_skips++; continue. At ≥ max_consecutive_skips
terminate with STUCK. |
Iter not committed (for SPEC) or committed without distill block (for DISTILL). Eager writes already made by the sub-agent stay on disk. |
| Commit step fails (mv or record.json write) | Rollback iter dir if mv succeeded; do not write record. consecutive_skips++. |
No iter committed; Channel A eager writes remain. Next THINK sees the inconsistent state and reasons about it. |
One # Goal header plus three ## sections.
Both forms below are valid; the loop refuses to start with neither.
goal.md with
all three sections (Ultimate / Escalation / Termination) non-empty.
Partial files abort the run.--goal "<text>" on
the command line, with no goal.md yet present. The loop
bootstraps goal.md using <text> as
Ultimate plus built-in defaults for Escalation + Termination (3-tier
escalation ladder probe/custom → search → perf; saturation + stuck +
budget + cap termination predicates).# Goal
## Ultimate
<one paragraph stating what this run is for.>
## Escalation
<plain-English rules for when to upgrade probe → search → perf.>
## Termination
<predicates; loop terminates on first match.
e.g. "perf success ≥ 0.50" (goal-achieved)
"5 consecutive iters with no knowledge / hypothesis movement"
(saturated)
"cumulative GPU-min ≥ 600 OR LLM-tokens ≥ 5M" (budget)>
Hard MUST / MUST NOT rules — distinct from
goal.md (direction) and knowledge.md (facts).
At bootstrap, up to four layers are concatenated (in order, with
## --- from <source> --- audit headers) into
vN/constraints.md. A pre-authored vN/constraints.md
(user edited before invocation) is left untouched.
| Layer | Source | Purpose |
|---|---|---|
| 1 | data/constraints/common.md |
Pipeline-wide myloop discipline (edit scope, cost gates, eval-tier discipline). Ships with the skill. |
| 2 | data/constraints/{graph}.md |
Per-graph hard rules (e.g. mapgpt_mp3d.md: model fixed at gpt-5-mini, temperature must be 1.0). |
| 3 | --cons-file <path> at invoke |
Per-invocation extra file (e.g. a topic-specific overlay the user wrote for this run). |
| 4 | --constraints "<text>" at invoke |
Inline ad-hoc rule, e.g. "MUST NOT change build_options.stop_after this run; only test prompt edits". |
The merged file is injected into every THINK and
DISTILL prompt as a non-negotiable preamble. THINK also does a coarse
string-match guard on spec.patch.intent against
constraint phrases (catches the obvious "intent describes changing
planner_llm.config.profile while a constraint pins it"
cases). Structured validators are deferred to v2.
Patches proposed by THINK and applied by the EXPERIMENT phase MAY
target any path under workspace/: graph JSON, top-level
nodesets, server-mode nodesets, env wrappers, policies, hooks.
All edits land in iteration/iter_M/active_workspace/
(isolated overlay) and never pollute the frozen workspace/
tree. The AgentCanvas backend honors the overlay via the
ACTIVE_WORKSPACE_DIR mechanism (ADR-components-009).
Outside workspace/ is never editable:
agentcanvas/backend/app/** (framework code) and
third_party/** (vendored upstream). The EXPERIMENT
apply-step enforces this at the filesystem level (via
_common/lib/overlay.py).
workspace/nodesets/server/**
edits used to require a manual backend restart — the parent backend's
auto_host singletons (loaded once at startup for
parallelism="shared" nodesets) were unaffected by the
overlay. The JobScheduler now detects content-hash divergence between
frozen and overlay source trees at eval-admit time and spawns an
ephemeral auto_host child loaded from the overlay; it is
torn down when the eval finalizes.
First iter touching a
shared nodeset pays a one-time spawn cost (≈ 30–60 s for
VLM-class models); the eval briefly doubles VRAM usage on that
nodeset — budget marginal_vram_mb accordingly on the
experiment profile.
myloop sits in the /architect:* skill family alongside
ADAS-shape variants (adas-subagent, aflow).
The shared piece is _common/
infrastructure — native overlay editing, smoke retry, edit whitelist,
resolve protocol. The divergence is what they choose to track.
ADAS-shape variants optimize the archive.
They generate candidate agents, score them on a fitness metric, keep
the winners in a flat archive.jsonl, and run until cap or
saturation. Failed generations are thrown out.
myloop optimizes the file system. Each iter's
output is "what changed in the 8 working-memory files"; agent
performance is one signal among many. Failed patches are archived as
experience. The run ends when the user-authored
goal.md § Termination says so — which may be a
metric-threshold predicate or a saturation predicate
("nothing new learned in 5 iters") or a budget cap.
| Dimension | ADAS-family | myloop v1 |
|---|---|---|
| What gets optimized | fitness on archive | distilled understanding against goal |
| Working memory | one flat archive.jsonl |
eight typed files |
| THINK structure | 3-call Reflexion (R0 / R1 / R2) | one sub-agent, free-form, must-emit-spec contract |
| Spawns per iter | 1 (think folds in last iter's results) | 2 (THINK clean-context + DISTILL warm-context) |
| Failed generation | n -= 1; not archived |
committed as evidence; outcome_class records it |
| Termination | iter cap or consecutive_skips | user-authored goal.md § Termination predicates |
| Self-extension | none — agent space is fixed by the prompt | new probes + new analysis tools authored mid-run |
knowledge.md append-only. v1 corrects
by superseding bullets; v2 may add explicit supersede / retract
semantics.tools/ runtime metadata. v1 has no
usage count / last-used / deprecation marker.constraints.yaml with explicit validators.| File | Purpose |
|---|---|
.claude/commands/architect/myloop/README.md | v1 mental model — normative |
.claude/commands/architect/myloop/schemas.md | single source of truth for every file shape |
.claude/commands/architect/myloop/config.yaml | paths, caps, files manifest, bootstrap hint |
.claude/commands/architect/myloop/loop.md | outer orchestrator (THINK → EXPERIMENT → DISTILL → commit) |
.claude/commands/architect/myloop/proposer.md | THINK phase |
.claude/commands/architect/myloop/distill.md | DISTILL phase |
.claude/commands/architect/myloop/reflect.md | REFLECT meta-phase (search-space cartography) |
.claude/commands/architect/myloop/understand.md | standalone (NOT a _common/ stub) — session-start mental-model load |
.claude/commands/architect/myloop/data/seed_knowledge.md | bootstrap content for knowledge.md |
.claude/commands/architect/myloop/data/constraints/common.md | layer-1 constraints (always present) |
.claude/commands/architect/myloop/data/constraints/<graph>.md | layer-2 constraints (per-graph) |
.claude/commands/architect/_common/lib/overlay.py | shared seed-helper — copy-on-touch into the overlay + §7 edit-whitelist check (no patch DSL) |
.claude/commands/architect/myloop/lib/helpers.py | LLM-profile cheat-sheet renderer for proposer |
Auto-rendered execution view of each .md file is at
mirror/myloop/. The page you are
reading is the hand-written, stable algorithm reference; the mirror
is regenerated from the skill markdown by
docs/pages/aas/build_mirror.py.