Multi-Agent Architectures: Planner, Generator, Evaluator

Why rely on a single, overburdened agent when you can orchestrate an entire team of specialists? In earlier chapters, we discussed the core anatomy of an AI harness and how to manage the context window for a single agent. However, as your applications grow in complexity, a single AI acting as a ‘jack-of-all-trades’ will inevitably start dropping the ball. This chapter introduces the powerful concept of multi-agent architectures, specifically focusing on the Planner, Generator, and Evaluator triad.

The Limits of the Solo Developer

Imagine hiring a human to act simultaneously as your high-level product manager, your deep-in-the-weeds software engineer, and your ruthlessly strict QA tester. Not only is context-switching exhausting, but it also creates blind spots. AI agents suffer from similar limitations. When an LLM is prompted to write a feature and immediately verify it, it often falls victim to confirmation bias—essentially giving its own buggy code a passing grade.

By splitting these responsibilities across specialized agents, we can give each AI a narrow, focused system prompt. This drastically reduces hallucinations and improves the quality of the final output.

Inspiration from GANs: The Adversarial Advantage

To understand why separating these roles is so effective, we can look at Generative Adversarial Networks (GANs), a cornerstone of modern machine learning. In a GAN, two neural networks play a game of cat and mouse: a ‘Generator’ tries to create fake data (like hyper-realistic images), while a ‘Discriminator’ tries to catch the fakes. Over time, the Generator becomes so good that the Discriminator can no longer tell the difference.

We apply this exact adversarial philosophy to AI Harness Engineering. We pit our code-writing agent (the Generator) against a strict, fault-finding agent (the Evaluator). This adversarial tension forces the Generator to write higher-quality code to pass the Evaluator’s relentless checks.

Meet the Trio

Our multi-agent harness is divided into three distinct personas, each with its own specific tools and context boundaries:

Agent Role	Primary Objective	Expected Output
Planner	Defines the ‘What’ and ‘Why’. Translates raw ideas into structured specifications.	Markdown documents, Architecture specs, Step-by-step plans
Generator	Defines the ‘How’. Writes the actual implementation based strictly on the plan.	Source code, Configuration files, Database migrations
Evaluator	Verifies the ‘Did it work?’. Acts as a strict QA, looking for edge cases and bugs.	Pass/Fail verdicts, Remediation feedback, Bug reports

The Planner: Drafting the Blueprint

The Planner agent is the brain of the operation. Before a single line of code is written, the Planner analyzes the user’s request and writes a high-level product specification. The key to a successful Planner is restricting its access to code-execution tools. If the Planner can write code, it will be tempted to start hacking. Instead, its job is to think deeply, break down the problem, and output a detailed execution plan.

Context Needs: High-level project goals, existing architecture rules, and user requirements.
System Prompt Focus: You are a senior software architect. Do not write implementation code. Break the problem into small, testable chunks.

The Generator: Bringing Ideas to Life

Once the blueprint is ready, the Generator takes over. This agent is your workhorse. It reads the Planner’s specifications and begins modifying the codebase. The Generator should be equipped with tools to read files, write files, and run basic compilation checks.

Context Needs: The Planner’s specification document, current file contents, and API documentation.
System Prompt Focus: You are an expert software engineer. Implement the provided plan exactly. Do not invent new features outside the scope of the document.

The Evaluator: The Unforgiving QA

The Evaluator is where the magic of the adversarial system shines. It acts as the gatekeeper. Once the Generator claims a task is ‘done’, the Evaluator steps in to verify. It runs unit tests, invokes linters, and reviews the code diff against the Planner’s original specification.

If the Evaluator finds an issue, it does not fix the code itself. Instead, it generates a harsh but constructive feedback report and sends the Generator back to work. This prevents the Evaluator from muddying its own objective perspective.

Managing File-Based Communication

A common mistake when building multi-agent systems is piping the output of one agent directly into the chat history of the next. This causes the context window to inflate massively, leading to ‘context anxiety’ and forgetfulness.

Instead, modern AI harnesses use file-based communication. The agents communicate by writing to and reading from the file system. The file system acts as the single source of truth.

[User Request]
      |
      v
(Planner Agent) ---> Writes to `spec.md`
                             |
                             v
                     (Generator Agent) <--- Reads `spec.md`
                             |              Writes to `app.js`
                             v
                     (Evaluator Agent) <--- Reads `spec.md` & `app.js`
                             |              Writes to `feedback.md`
                             v
[Loop back to Generator if feedback is negative]

Smooth Handoffs and the Feedback Loop

To orchestrate this workflow, you need a control loop—often referred to as a State Machine. The orchestrator script sits above the agents, waking them up and putting them to sleep based on the current state of the project. Here is a simplified Python example demonstrating how this orchestration loop is structured:

# Pseudo-code for Multi-Agent Orchestration

def run_harness(user_prompt):
    # Step 1: Planner drafts the spec
    write_file("spec.md", planner_agent.generate_spec(user_prompt))

    max_iterations = 5
    iteration = 0
    is_approved = False

    # Step 2: Adversarial Loop
    while iteration < max_iterations and not is_approved:
        print(f"--- Sprint Iteration {iteration + 1} ---")

        # Generator reads the spec (and any previous feedback) and writes code
        generator_agent.implement_feature(read_file("spec.md"), read_file("feedback.md"))

        # Evaluator reviews the work
        evaluation = evaluator_agent.review(read_file("spec.md"), read_dir("./src"))

        if evaluation.passed:
            print("Evaluator approved the changes!")
            is_approved = True
        else:
            print("Evaluator found bugs. Generating feedback...")
            write_file("feedback.md", evaluation.feedback)
            iteration += 1

    if not is_approved:
        print("Harness halted: Max iterations reached without Evaluator approval.")

In this loop, the orchestration script is entirely deterministic. It relies on the LLMs to do the ‘thinking’ but uses standard programming logic to enforce the handoffs. By utilizing this Planner-Generator-Evaluator trio, you transform a fragile, easily distracted AI into a rigorous, self-correcting software development pipeline.