When an agent scaffold tries to rewrite itself, it tends to learn the benchmark instead of the job.

The setup is straightforward: freeze the base foundation model, then let an outer loop propose changes to system prompts, context management routines, tool definitions, and retry logic. If the score on the benchmark improves, you keep the diff. If it drops, you roll back. Within a dozen iterations, the agent looks great on the eval split. Put that evolved harness on an adjacent repository or a different API suite, and the edge disappears.

That failure mode is what Peng Xia and researchers at Google Cloud AI Research, UNC-Chapel Hill, Stanford, and WashU address in their preprint on Regularized Recursive Self-Improvement (RRSI, arXiv:2609.24972). They treated the outer scaffolding optimization loop like an empirical machine learning problem that needs regularization, rather than an unconstrained search.

Overfitting the prompt harness

The standard unregularized baseline bundles whatever prompt edits and tool wrappers happen to nudge an evaluation harness over a pass mark. Given enough generations, the prompt accumulates benchmark-specific hardcoding: idiosyncratic instructions tuned to edge cases in the test harness, fragile tool call sequences, and bloated reasoning scratchpads.

RRSI puts constraints on both the candidate proposer and the selector.

On the proposal side, the system uses an annealed mutation budget. Early in the search, the model can propose multi-component refactors across prompt instructions, control flows, and tool schemas. As iterations proceed, the budget tightens to smaller, single-component tweaks, preventing late-stage compound mutations from masking regressive edits. It also penalizes trajectories that duplicate historical proposals, forcing the proposer into unexplored architectural changes.

On the selection side, RRSI splits verification between a critic and an explicit pruner:

  • The critic checks proposed diffs before execution, flagging benchmark-specific heuristics like brittle string matches or prompt phrasing tailored to a single test suite’s conventions.
  • The pruner runs after execution. It discards components whose incremental contribution is below a marginal threshold, cuts changes that balloon token cost, and strips out scaffolding elements that earlier iterations added but later control flows made redundant.

What happens when you penalize bloat

The paper tested RRSI across eight benchmarks covering software engineering, agentic workspaces, and engineering design tasks:

  • The regularized harness gained up to 14.1 points on the target development split.
  • On five out-of-distribution test suites, RRSI retained up to 4.7 points of improvement where unconstrained recursive loops degraded back to baseline or negative transfer.
  • The resulting evolved harness consumed 30% fewer policy tokens than unconstrained search variants.

That last number is the practical takeaway. Most agent engineering teams run into prompt bloat within two months of shipping. Every bug report turns into a new warning block in the system prompt. Every failed tool call adds a defensive instruction. After a few quarters, the harness burns 4,000 tokens of input overhead on every turn, and subsequent model updates break the delicate prompt balance.

Automating harness evolution is useful, but only if the selector has a built-in pruner with a bias toward deleting instructions. Scaffolding that survives regularization is usually just cleaner tool contracts and tighter control flow.