Taming the Context Window: Memory and Resets

Imagine managing a massive software project staffed by brilliant engineers, but there is a major catch: they suffer from complete amnesia at the end of every single shift. Every time they clock in, you have to frantically explain what the project is, what they built yesterday, and what they need to do today. If your explanation gets too long, they get overwhelmed and forget the instructions in the middle; if it is too short, they break existing features because they lack context. This is exactly what it feels like to build long-running AI agents without proper memory management.

As agents run for hours or days, reading files, writing code, and debugging errors, their context windows fill up. This leads to a phenomenon known as context anxiety or forgetfulness. In this chapter, we will explore how AI Harness Engineering solves this through Context Compaction, Context Resets, and Structured Handoff Artifacts.

The Threat of Context Anxiety

Large Language Models (LLMs) are inherently stateless. To maintain a “conversation” or an “agentic session,” the harness must continuously append new actions and responses to a running transcript and send the entire transcript back to the model for every single step. This causes three critical problems:

The ‘Lost in the Middle’ Effect: Models are great at remembering the very beginning of a prompt (system instructions) and the very end (the most recent error message), but they notoriously ignore or forget details buried in the middle of a massive context window.
Cost and Latency: Sending 100,000 tokens back and forth for every minor action is incredibly slow and financially expensive.
Hallucinations and Drift: As the transcript fills with failed attempts, dead-end debugging trails, and temporary logs, the agent gets confused by its own messy history and loses sight of the original goal.

Context Compaction vs. Context Resets

To prevent an agent from drowning in its own history, a harness must actively manage the context window. There are two primary techniques for doing this: Context Compaction and Context Resets.

Technique	How It Works	Best Used For
Context Compaction	Periodically using an LLM to read the long transcript and write a concise summary. The long transcript is then replaced by the summary.	Ongoing sub-tasks where the agent needs to remember the gist of a conversation, but not the exact keystrokes.
Context Resets	Completely clearing the conversational history and starting a brand new agent instance with only the essential state artifacts.	Transitioning between major phases (e.g., from Planning to Implementation) or recovering from a stuck state.

Context Compaction: Squeezing the Sponge

Context compaction is like squeezing out a wet sponge so it can absorb more water. When the context reaches a certain threshold (e.g., 80% of the token limit), the harness intercepts the execution and asks a secondary, cheaper LLM to summarize the journey so far. This condenses thousands of tokens into a few paragraphs.

[Before Compaction: 25,000 tokens]
System: You are a coding agent...
Agent: I will check the file.
Tool: Cat src/main.py -> (10,000 lines of code)
Agent: I found the bug. I will write a test.
Tool: Run test -> (5,000 lines of stack trace)
...

[After Compaction: 500 tokens]
System: You are a coding agent...
Summary: The agent inspected src/main.py, identified a NullPointerException in the auth logic, and ran a test suite which failed on line 42.
Agent: Next, I need to patch line 42.

Context Resets and Structured Handoff Artifacts

While compaction is useful, it is inherently lossy. Eventually, the agent’s “mind” gets too cluttered. The most robust way to manage long-running agents is through hard Context Resets. You kill the current agent process and spin up a fresh, hyper-focused instance.

However, to prevent the “amnesia” problem mentioned earlier, the outgoing agent must prepare a structured briefing for the incoming agent. Think of this as the clinical handoff between nurses during a shift change. We call these Structured Handoff Artifacts.

Progress Logs (The ‘What and Why’): A markdown file maintained by the agent detailing the overarching goal, what has been accomplished, what failed, and what the immediate next step is.
Git Commit History (The ‘Proof’): The ultimate system of record. By enforcing that an agent commits its working state before a reset, the new agent can simply run git log -n 3 and git diff to instantly understand the state of the codebase.
Architecture State Files: JSON or YAML files that track the current state of variables, database schemas, or API endpoints that the new agent will need to reference without re-discovering them.

Implementing the Handoff Mechanism

Let us look at how you might orchestrate this in a harness using Python pseudo-code. The harness monitors the token count or step count. When it is time for a reset, it forces the agent to generate a handoff artifact, saves the workspace, and restarts.

# Harness pseudo-code for managing a context reset

def run_agent_session(task_description, previous_handoff=None):
    # Initialize a clean slate context
    context = [{"role": "system", "content": "You are a software engineer."}]

    # Inject previous shift's memory if it exists
    if previous_handoff:
        context.append({
            "role": "user",
            "content": f"Previous Shift Handoff:\n{previous_handoff}"
        })
    else:
        context.append({"role": "user", "content": f"New Task: {task_description}"})

    # Agent execution loop
    for step in range(MAX_STEPS_PER_SHIFT):
        action = llm.generate(context)
        result = execute_tool(action)
        context.extend([action, result])

        # Check if the task is done
        if is_task_complete(result):
            return "COMPLETED"

        # If getting close to context limit, trigger a reset
        if estimate_tokens(context) > TOKEN_WARNING_LIMIT:
            print("Context limit approaching. Initiating shift change...")
            break

    # The shift is over. Force the agent to write a handoff artifact.
    handoff_prompt = {
        "role": "user",
        "content": "Your shift is ending. Write a concise PROGRESS LOG for the next agent. Include what you did, what failed, and the exact next steps. Commit your code to git."
    }
    context.append(handoff_prompt)

    # Generate and return the artifact to the harness
    handoff_artifact = llm.generate(context)
    return handoff_artifact

# Main Harness Loop
current_task = "Migrate database from SQLite to PostgreSQL"
handoff = None

while True:
    status_or_handoff = run_agent_session(current_task, handoff)
    if status_or_handoff == "COMPLETED":
        print("Task finished successfully!")
        break
    else:
        # Save the artifact and loop again with a fresh instance
        handoff = status_or_handoff
        print("Resetting context... passing baton to new agent instance.")

Conclusion

Taming the context window is a foundational skill in AI Harness Engineering. By shifting your mindset from “one long conversation” to a series of discrete, well-documented “shifts” connected by structured handoff artifacts like git history and progress logs, you ensure your agents remain sharp, cost-effective, and highly capable, no matter how long the project takes.