Search Operator API (RFC)

Agent Architecture Search (AAS) — searching over the space of agent architectures to find high-performing configurations — is the killer use case of AgentCanvas. Every architectural decision in the framework has been made with AAS in mind: GraphDefinition JSON is a serializable search-space element, the graph executor can run any valid graph without code generation, WorkspaceComponentRegistry provides an auto-enumerable set of primitives, and the two-layer execution logging collects fitness signals for free. This document describes the planned plugin API that will turn those building blocks into a platform-native Python-callable AAS operation.

1. Why a platform-native AAS API?

Every AAS paper today reimplements the same infrastructure:

1. A search space representation (usually a custom Python dataclass)
2. A search loop (usually 200–500 lines of boilerplate)
3. An evaluation harness (simulator integration, metric collection, seed management)
4. A mutation / crossover / proposal mechanism
5. Logging, checkpointing, and reproducibility plumbing

Recent papers — AFlow (24.10), ADAS (24.08), MaAS (25.02), AgentSquare, EvoAgentX — each rebuild items 1–5 from scratch because their internal representations differ. This is a structural inefficiency: the search method is the research contribution, but the research code is dominated by the infrastructure around it.

AgentCanvas's bet is that if you accept GraphDefinition as the search-space representation, items 1–5 can all be provided by the platform, and the research contribution becomes a ~200-line plugin — a BaseSearchOperator subclass. This is analogous to how PyTorch's nn.Module made neural network research be about the math, not about autograd plumbing.

2. Design overview

The planned AAS system has four parts, each addressing one of the five reimplementation costs above:

The thing you write as an AAS researcher is just the bottom row: a single Python file in workspace/search_operators/ implementing your search algorithm. Everything else is platform.

3. The BaseSearchOperator interface

# PLANNED interface — does not exist yet (roadmap F9).

from __future__ import annotations
from abc import ABC, abstractmethod
from typing import ClassVar

from app.graph_def import GraphDefinition


class FitnessResult:
    """Result of evaluating one candidate graph."""
    graph: GraphDefinition
    fitness: float              # scalar to maximize (or minimize, via .direction)
    metrics: dict[str, float]   # per-metric breakdown from state containers
    trace_id: str               # pointer to execution log for debugging
    wall_time_s: float
    crashed: bool
    error: str | None


class BaseSearchOperator(ABC):
    """Plugin base class for Agent Architecture Search algorithms.

    Subclass this to implement your search method. Place the file in
    workspace/search_operators/ — it will be auto-discovered by WorkspaceComponentRegistry
    analogous to how nodesets are discovered.
    """

    # Plugin metadata (required, like BaseNodeSet)
    name: ClassVar[str]                  # unique identifier, e.g. "evolutionary"
    display_name: ClassVar[str]          # human-readable, e.g. "Evolutionary Search"
    description: ClassVar[str]

    # Configuration (optional — drives a config panel in the UI)
    config_fields: ClassVar[list] = []   # list of ConfigField, like BaseCanvasNode

    # Direction of fitness: "maximize" or "minimize"
    direction: ClassVar[str] = "maximize"

    # Selection mode — see § 8.4.5 for the Voyager-driven amendment
    selection_mode: ClassVar[str] = "best_fitness"   # "best_fitness" | "latest" | "pareto"

    # Lifecycle
    def setup(self, seed_graph: GraphDefinition, config: dict) -> None:
        """Called once at the start of a search run. Store seed and config."""
        self.seed_graph = seed_graph
        self.config = config

    @abstractmethod
    def propose(self) -> GraphDefinition:
        """Generate the next candidate graph to evaluate.

        Called repeatedly by SearchLoop. Must return a valid GraphDefinition
        that can be flattened and executed by the graph executor. Typically
        implemented using the Graph Mutation API (F8) to derive candidates
        from the seed graph, previously-evaluated candidates, or a population.
        """
        ...

    @abstractmethod
    def update(self, result: FitnessResult) -> None:
        """Record the result of evaluating a proposed candidate.

        Called once per candidate after evaluation. The operator uses this
        feedback to guide future proposals (e.g. population updates, MCTS
        backpropagation, gradient steps for learned operators).
        """
        ...

    @abstractmethod
    def is_done(self) -> bool:
        """Termination check. Called before each propose().

        Return True to stop the search. Typical termination criteria:
        budget exhausted, fitness plateau, explicit max_iterations reached,
        or user pressed Stop.
        """
        ...

    def best(self) -> FitnessResult | None:
        """Return the best candidate found so far, or None if empty.

        Default implementation tracks max/min over all update() calls.
        Override if your operator has a richer notion of "best" (e.g.
        Pareto frontier for multi-objective search, or "latest" for
        accumulative self-evolve operators — see § 8.4).
        """
        return getattr(self, "_best", None)

3.1 Why only 3 abstract methods?

The interface is deliberately minimal. All AAS algorithms we surveyed reduce to a propose → evaluate → update → terminate loop, differing only in how each step is implemented:

• Random search proposes uniform mutations, updates by doing nothing, terminates on budget
• Evolutionary search proposes via tournament selection + crossover + mutation, updates by replacing population weakest, terminates on generation count
• MCTS proposes by tree-policy UCB, updates by backpropagating reward, terminates on simulation budget
• Policy-gradient / learned operators propose via a learned sampler, update by gradient step, terminates on gradient budget

Richer operators can store additional state on the instance — the framework does not impose a specific representation for populations, trees, or learned parameters.

3.2 What the framework guarantees

When SearchLoop calls your operator:

1. setup() is called exactly once with the seed graph and user config
2. propose() is called repeatedly — the returned graph is validated, flattened, and evaluated
3. update() is called exactly once per propose() with the corresponding FitnessResult
4. is_done() is called before each propose() — if True, the loop terminates gracefully
5. The evaluation harness, logging, checkpointing, and simulator integration are all provided by SearchLoop — your operator does not touch them

If propose() returns an invalid graph (dangling edge, port type mismatch), SearchLoop will skip evaluation and call update() with a FitnessResult(crashed=True, error="validation failed"). Your operator is expected to handle crash results in update() — typical strategy is to assign worst-case fitness to discourage similar proposals.

4. The SearchLoop orchestrator

# Planned API — tracked as roadmap F9.
# Users typically don't call this directly; they use the UI "Search" button
# or the CLI. Shown here for transparency.

class SearchLoop:
    def __init__(
        self,
        operator: BaseSearchOperator,
        seed_graph: GraphDefinition,
        fitness_fn: FitnessFunction,
        eval_config: EvalConfig,
        operator_config: dict | None = None,
        checkpoint_dir: Path | None = None,
    ): ...

    async def run(self) -> SearchRunResult:
        """Execute the search loop to completion.

        Returns:
            SearchRunResult with the best candidate, full history, and
            per-candidate execution traces.
        """
        ...

The orchestrator handles:

• Calling operator.setup() once
• The propose → evaluate → update loop
• Integration with JobScheduler for multi-episode evaluation
• Checkpoint persistence (resumable searches after crashes)
• WebSocket events for the frontend search visualization (search_data events)
• Graceful shutdown on user Stop signal

It does not dictate: population sizes, mutation strategies, selection pressure, tree policies, or any search-specific logic. Those live in your operator.

5. Graph Mutation API

Mutation operators are pure functions that take a GraphDefinition and return a new one — never mutating the input:

# Planned API — tracked as roadmap F8.
# Module: agentcanvas/backend/app/search/mutations.py

from app.graph_def import GraphDefinition, NodeDef, EdgeDef
from app.search import mutations

g2 = mutations.add_node(graph, NodeDef(id="new1", type="basic_agent__note_write", config={}),
                        incoming_edges=[...], outgoing_edges=[...])

g3 = mutations.remove_node(graph, node_id="llm_planner")

g4 = mutations.swap_node(graph, node_id="llm_planner",
                         new_type="llmCall", new_config={"model": "gpt-4o"})

g5 = mutations.rewire_edge(graph, edge_id="e1", new_source="observe", new_target="planner")

g6 = mutations.change_config(graph, node_id="planner",
                             config_patch={"temperature": 0.2})

g7 = mutations.mutate_port_config(graph, node_id="iter1", ports={"input_ports": [...]})

5.1 Validity preservation

Every mutation either returns a valid graph or raises InvalidMutationError. Validity means:

• No dangling edges (source and target both exist)
• No port type mismatches (wire types line up)
• No cycles except through IterIn / IterOut pairs
• All required ports are wired

This means your propose() implementation doesn't need to re-validate — if a mutation returns normally, the graph is runnable. If it raises, you can handle the error (typically by trying a different mutation).

5.2 Composition

Mutations compose naturally because they are pure functions:

g = seed_graph
for _ in range(mutation_budget):
    op = random.choice([mutations.swap_node, mutations.change_config, mutations.add_node])
    try:
        g = op(g, **random_args_for(op, g))
    except InvalidMutationError:
        continue  # try a different random op
return g

This 8-line snippet is essentially a complete propose() for a random search operator.

6. Fitness functions

A fitness function reads scalar metrics from state containers at the end of evaluation and aggregates them into a single number (for single-objective search) or vector (for multi-objective).

# Planned API — tracked as roadmap F9.
# Module: agentcanvas/backend/app/search/fitness.py

from app.search.fitness import FitnessFunction, metric, weighted_sum

# Simple: read a single state container metric
success_rate = FitnessFunction(lambda state: state["eval_summary"]["success_rate"])

# Weighted combination
fn = weighted_sum({
    "success_rate": 0.7,
    "spl": 0.2,
    "neg_path_length": 0.1,   # prefer shorter paths (negated to maximize)
})

For most VLN / embodied benchmarks, fitness is a built-in metric from the JobScheduler output, so you rarely need to write a custom fitness function.

7. Reference operators (planned in F10)

7.1 RandomSearch

The simplest possible search operator — uniform random mutations, no memory, fixed budget. Once F10 lands at workspace/search_operators/random_search.py:

from app.components import ConfigField
from app.search import BaseSearchOperator, FitnessResult
from app.search.mutations import swap_node, change_config, InvalidMutationError
from app.graph_def import GraphDefinition


class RandomSearch(BaseSearchOperator):
    name = "random_search"
    display_name = "Random Search"
    description = "Uniform random mutations from the seed graph"
    direction = "maximize"

    config_fields: ClassVar[list] = [
        ConfigField("budget", "slider", label="Candidate budget", default=100),
        ConfigField("mutations_per_candidate", "slider", label="Mutations per candidate", default=3),
    ]

    def setup(self, seed_graph, config):
        super().setup(seed_graph, config)
        self.budget = config.get("budget", 100)
        self.n_mutations = config.get("mutations_per_candidate", 3)
        self.n_evaluated = 0
        self._best: FitnessResult | None = None

    def propose(self) -> GraphDefinition:
        g = self.seed_graph
        for _ in range(self.n_mutations):
            op = random.choice([swap_node, change_config])
            try:
                g = op(g, **self._random_args(op, g))
            except InvalidMutationError:
                continue
        return g

    def update(self, result: FitnessResult) -> None:
        self.n_evaluated += 1
        if not result.crashed:
            if self._best is None or result.fitness > self._best.fitness:
                self._best = result

    def is_done(self) -> bool:
        return self.n_evaluated >= self.budget

~50 lines for a complete operator. The author writes the search method; the platform handles everything else.

7.2 EvolutionarySearch

Slightly richer — maintains a population, does tournament selection, crossover, mutation:

class EvolutionarySearch(BaseSearchOperator):
    name = "evolutionary"
    display_name = "Evolutionary Search"
    description = "Population-based search with tournament selection, crossover, and mutation"
    direction = "maximize"

    config_fields: ClassVar[list] = [
        ConfigField("population_size", "slider", label="Population size", default=20),
        ConfigField("generations", "slider", label="Generations", default=10),
        ConfigField("tournament_size", "slider", label="Tournament size", default=3),
        ConfigField("mutation_rate", "slider", label="Mutation rate", default=0.3),
    ]

    def setup(self, seed_graph, config):
        super().setup(seed_graph, config)
        self.pop_size = config["population_size"]
        self.generations = config["generations"]
        self.tournament_size = config["tournament_size"]
        self.mutation_rate = config["mutation_rate"]
        self.population: list[FitnessResult] = []
        self.current_gen = 0
        self._pending: list[GraphDefinition] = [seed_graph] * self.pop_size
        self._best: FitnessResult | None = None

    def propose(self) -> GraphDefinition:
        if self._pending:
            return self._pending.pop()
        self._next_generation()
        return self._pending.pop()

    def update(self, result: FitnessResult) -> None:
        self.population.append(result)
        # … best-tracking + generation-rollover bookkeeping …

    def is_done(self) -> bool:
        return self.current_gen >= self.generations

    def _next_generation(self) -> None:
        new_pop = []
        for _ in range(self.pop_size):
            parent_a = self._tournament_select()
            parent_b = self._tournament_select()
            child = self._crossover(parent_a.graph, parent_b.graph)
            if random.random() < self.mutation_rate:
                child = self._mutate(child)
            new_pop.append(child)
        self.population = []
        self._pending = new_pop

    def _crossover(self, a, b) -> GraphDefinition:
        # The interesting research part — graph crossover via subgraph splice
        ...

~80 lines. The "interesting part" — the crossover operator — is where the research lives. Everything else is boilerplate that looks the same across every operator, and that's the point.

8. Algorithm compatibility: hosting published methods

The interface in § 3 is small (3 abstract methods) by design — but small interfaces only matter if they actually fit real algorithms. This section walks through 4 published methods, mapping each onto BaseSearchOperator. Any algorithm that doesn't fit cleanly identifies an interface gap that needs to be addressed before F9 freezes.

8.1 Hosting ADAS (Meta Agent Search)

ADAS's loop is roughly:

seed_agents = [hand_designed_baselines]
archive = seed_agents.copy()
while budget > 0:
    parents = sample_top_k(archive, k)         # contextual sampling
    new_code = LLM(propose_new_agent_prompt, parents)
    new_agent = compile(new_code)               # may fail
    score = eval_on_benchmark(new_agent)
    archive.append((new_agent, score))
    budget -= 1
return best(archive)

Maps onto BaseSearchOperator with the archive held on the operator instance: propose() samples top-K parents and asks a meta-LLM to emit a new GraphDefinition JSON; update() appends results; is_done() checks budget. ~200 lines fully fleshed out.

Verdict: ✅ clean fit — ADAS is the canonical case the interface was designed for. The one representation port: ADAS's original outputs Python code; the port outputs GraphDefinition JSON. Faithfulness defence: algorithm preserved (archive + top-K sampling + LLM proposal); only the proposal output format changes; original mutation prompt template is reproducible verbatim from the paper appendix.

8.2 Hosting AFlow (MCTS over workflows)

AFlow's loop is MCTS with UCB tree policy + backpropagation. The MCTS tree lives entirely on the operator instance — the framework treats it as opaque state. UCB selection happens inside propose(), backpropagation inside update(). propose() and update() share state via self._current_node (the framework calls them in pairs); this is well-defined per § 3.2 guarantees. ~250 lines.

Verdict: ✅ clean fit. Faithfulness defence: MCTS algorithm preserved verbatim (tree policy, backprop, UCB). Operator set must be ported from AFlow's repo (Ensemble / Review / Revise / etc. → equivalent GraphDefinition mutations) — operator definitions are the only handcrafted part; F8's mutation API provides the primitives.

8.3 Hosting EvoAgentX optimisers (TextGrad / SEW)

EvoAgentX bundles 5 optimisers. The two most distinct are TextGrad (textual gradient on prompts) and SEW (LLM mutation on workflow representation). Both have the same outer-loop shape — only the proposal step differs. TextGrad has population = 1 (single graph, repeatedly updated) — the same shape as today's /architect:<variant>:loop trajectories. SEW is structurally identical with a different rewriting prompt library. ~200 lines per optimiser.

Verdict: ✅ clean fit, population=1 is a sub-case. Implication for the EvoAgentX comparison: porting one EvoAgentX optimiser proves the F9 interface absorbs EvoAgentX's algorithm shape, defending against the "you're just EvoAgentX renamed" attack without requiring a full benchmark port.

8.4 Hosting a Voyager-stub (mechanism-self-evolve)

Voyager (23.05) is not AAS — it doesn't search over architectures. It accumulates a skill library (Python functions) within a single Minecraft agent's lifetime. The agent's architecture is fixed; what changes is the skill library — a StateContainer in AgentCanvas terms.

This is the "Voyager-as-baseline trap" we defuse on the embodied side: by hosting at least a Voyager-stub on F9, the platform claim "supports both AAS and self-evolve operators" becomes provable rather than aspirational.

8.4.1 Voyager's loop shape

agent = seed_agent_with_empty_skill_library
for episode in episodes:
    task = sample_task()
    relevant_skills = retrieve_skills_for(task, agent.library)
    if any(relevant_skills):
        agent.forward(task, using=relevant_skills)
    else:
        new_skill_code = LLM(write_skill_prompt, task, env_state)
        success = agent.try_skill(new_skill_code, task)
        if success:
            agent.library.add(new_skill_code)

8.4.2 Mapping to BaseSearchOperator

The operator wires a persistent StateContainer (skill library) into the seed graph once during setup(). Then propose() returns the same graph each call, modulo injecting the accumulated library as initial state; update() reads the post-episode library from the execution log and absorbs any new skills the agent added. ~250 lines.

8.4.3 What's different from AAS operators

Three semantic shifts vs § 8.1–8.3:

1. propose() returns essentially the same graph each call — modulo state injection. There is no architecture search.
2. update() doesn't influence future propose() via algorithmic decision-making — it mechanically absorbs results into the agent's persistent state. The "search" is the agent's own lifetime trajectory.
3. best() semantics shift — for AAS, "best" = highest-fitness candidate. For Voyager-style, "best" = the most-evolved agent (typically the latest) since evolution is monotone-additive.

8.4.4 Why the interface still fits

The 3 abstract methods (propose / update / is_done) are algorithmically agnostic — they describe the control flow of any search-or-evolution loop, not a specific algorithm class. AAS uses propose/update/is_done as a search-iteration cycle; Voyager-stub uses it as an episode cycle. Both compile to the same outer loop in SearchLoop.

8.4.5 Interface verdict — fits with the selection_mode amendment

The Voyager-stub case surfaces one interface gap: the default best() impl assumes scalar-fitness max selection, which is wrong for accumulative operators. The amendment (already woven into § 3's BaseSearchOperator stub):

selection_mode: ClassVar[Literal["best_fitness", "latest", "pareto"]] = "best_fitness"

• "best_fitness" (default): standard AAS semantics — best() returns max-fitness FitnessResult
• "latest": accumulative / self-evolve semantics — best() returns the latest non-crashed result
• "pareto": multi-objective — best() returns the Pareto frontier (tuple of FitnessResult)

2-line addition, doesn't break any existing operator, explicitly admits self-evolve operators as first-class.

8.5 Summary verdict

Conclusion: the interface as specified in § 3 absorbs all 4 published algorithms with at most a 2-line amendment (selection_mode), already adopted in this RFC. All four operator implementations come in under 300 lines each — substantially smaller than the original repos (ADAS: ~1500, AFlow: ~2000, Voyager: ~3000) because the platform absorbs the harness / logging / persistence / visualisation that those papers' codebases re-implement.

9. Reproducibility and artifact model

AgentCanvas's bet is that every output of an AAS run is a first-class, shareable artifact:

• The search configuration is a JSON file (operator, seed graph, eval config, seeds)
• The search history is a JSONL file (one line per candidate, in evaluation order)
• The best candidate is a GraphDefinition JSON — the same format as any hand-authored graph
• The execution traces are JSONL files from the two-layer executor logging

This means:

• You can cite a specific evolved agent by pointing at its GraphDefinition JSON commit
• You can replay any candidate by loading its JSON onto the canvas and pressing Play
• You can diff two evolved agents using any JSON diff tool
• You can share a search run as a zip of the output directory; anyone with AgentCanvas can reproduce it

This is a direct advantage over code-represented workflows (AFlow) and hidden-state learned operators (MaAS), which require the author's full codebase to reproduce anything. The variant family (.claude/commands/architect/) already inherits most of this contract via the _common/files-contract.md spec; the F9 Python API will make it load-bearing rather than convention-driven.

10. Current status

The Python plugin API in this RFC is not yet built (re-confirmed 2026-06-13: no BaseSearchOperator, SearchLoop, app/search/, or workspace/search_operators/ in the tree). What AgentCanvas ships today is a parallel surface — the slash-skill variant family — that covers the same use cases via Claude-conversation drivers:

Coexistence model. Once F9 ships, both planes are intended to remain: variants for AAS that's iterating fast on prompts / orchestration / lineage shape (the agile path); Python operators for AAS that's ready to be packaged + cited + shared as a tight artifact (the long-term path). Migration from a stabilised variant to a Python operator is a one-time port — same iter-tree contract, same eval harness, same fitness signals.