Evaluator–Optimizer

Also known as: Reflection, generator–critic, self-refine, critique-and-revise loop, writer–reviewer, LLM-as-a-judge loop, generate–test–fix

Intent

One component writes a candidate output. A second one judges it against criteria you wrote down and returns specific feedback. The writer revises, and the loop repeats until the judge passes the output or a stop rule in your code ends it.

Problem

Say you want a model to write a small Python class: a MinStack with push, pop, top and get_min, every operation in O(1). This is the example Anthropic's cookbook notebook runs for this pattern.

One call gives you a first draft. The core idea is right (a second stack that tracks the minimum), but the draft returns None when you pop an empty stack, has no docstrings and does not check its input. A model writes in one left-to-right pass and tends to stop when the output looks done, not when it is done. Without a review step, you get these symptoms:

A task goes to one LLM call, which returns a first draft with three gaps; a person has to find them by hand. Task write MinStack One LLM call writes once, stops First draft min stack idea is correct pop() on empty returns None no docstrings or type hints no input validation You find the gaps by hand
A single call has no step where anyone checks the draft against the requirements, so the checking falls to you.

Solution

Wrap generation in a loop with a reviewer. The generator writes; the evaluator compares the result with an explicit list of criteria and says what is wrong, where, and how to fix it. Code around them runs the loop and decides when it ends. For the MinStack task:

  1. Generate. The generator gets the task and the full list of criteria: O(1) operations, a clear exception on an empty stack, type hints and docstrings, input validation. It writes a first draft.
  2. Check. Your code runs deterministic checks first: pytest, mypy, a linter. Their raw output becomes evidence. Here two tests fail because pop() returns None instead of raising.
  3. Evaluate. The evaluator receives the task, the criteria, the draft and the test output. It returns a verdict (REVISE) and a list of issues, for example "C2, pop(): returns None on an empty stack; raise IndexError".
  4. Decide. Code, not a model, applies the stop rule: pass, iteration cap, no improvement, or budget. The verdict is not PASS and the cap is not reached, so the loop goes on.
  5. Revise. The generator gets the task, its last draft and the feedback, and writes a new full version. Back to step 2. In the cookbook run, the evaluator's first verdict was NEEDS_IMPROVEMENT with complaints about error handling, type hints, documentation and input validation, and the generator wrote a second draft.

Who decides what: the generator decides the content, the evaluator decides whether it meets the criteria and what to change, and your code owns the stop rule, the cap and which candidate is returned. Some frameworks let an agent inside the loop end it (Google ADK has an exit_loop tool), but a hard cap in code is still the backstop.

What sets it apart from ReAct, which also loops: in ReAct the model picks its next action freely. Here the shape is fixed in code (generate, check, evaluate, revise), which makes it a workflow. Compared with Prompt Chaining, it is a chain whose check gate loops back with feedback instead of just passing or failing.

Real-world analogy

A developer opens a pull request. CI runs the test suite and the linter, then a reviewer reads the diff with the failing checks in front of them and leaves comments tied to specific lines. The developer pushes a fix, CI runs again, and the cycle repeats until the reviewer approves. The developer is the generator, CI is the deterministic check, the reviewer is the evaluator, and the team's rule "two approvals, green CI" is the stop condition in code.

The analogy breaks in one place. A human reviewer who has seen a hundred similar PRs knows when to stop nitpicking. An LLM reviewer told to find problems will always find some, so you have to limit it to the stated criteria and allow it to approve on the first round.

Structure

The task goes to the generator; its candidate passes through deterministic checks to the evaluator; the evaluator's verdict goes to a stop rule in code, which either sends feedback back to the generator or returns the best candidate. task candidate candidate + evidence verdict pass / cap revise: feedback + history Task + criteria Generator writes, revises Deterministic checks tests, compiler, schema Evaluator verdict + issues Stop rule in code Output best so far 1 2 3 4 5 6
Solid arrows carry the work forward; the dashed accent arrow is the revision loop. The checks box is optional, but it is where most of the value comes from.
  1. Generator. Writes the first draft and every revision. Its prompt contains every requirement the evaluator will check. On revisions it also gets its last attempt and the feedback.
  2. Deterministic checks. Tests, compilers, type checkers, linters, schema validators, word counters, a search call to verify a claim. They run before the judge and their raw output is passed along as evidence.
  3. Evaluator. An LLM judge with a rubric, or a rule. It returns a verdict your code can branch on (PASS, REVISE, FAIL) and a list of issues, each with a location, a suggested fix and a severity. It evaluates only; it does not rewrite.
  4. Stop rule. Plain code that ends the loop on a pass, the iteration cap, no improvement for some rounds, a repeated candidate, or a token or time budget.
  5. Feedback and history. What goes back to the generator: the latest candidate and its issues in full, earlier rounds as one-line summaries or short lessons, so the context does not keep growing.
  6. Output. The best candidate seen so far, which is not always the last one. If the loop ends without a pass, flag it as unverified or hand it to a person.

Prompt generator

Describe what the loop should produce, the criteria it must meet, and the checks your code can run. You get two system prompts (generator and evaluator) that share the same criteria list, plus the loop rules to implement in your controller code, whether that is plain Python, a LangGraph graph or an OpenAI Agents SDK loop.

Criteria the same list goes to both prompts
Issues sent back for revision

Generated prompt

Highlighted text comes from the form. [Bracketed] text is still to fill in.

The checks matter more than the prompts

A loop whose only feedback is the same model rereading its answer often does not help, and on reasoning tasks it can turn right answers into wrong ones. Spend your effort on the "Verified by" column: every criterion you can tie to a test, a compiler or a validator gives the evaluator evidence it can trust and makes it harder to fool.

Applicability

Use it when

Avoid it when

How to implement

  1. Write the criteria first. If you cannot list them, this pattern is probably the wrong fit.
  2. Give the generator every requirement. Do not hold criteria back for the critic. Otherwise the loop only seems to help because the critic restates requirements the generator never saw.
  3. Collect every deterministic check you can. Tests, type checks, linters, schema validation, length counters, regexes, citation checks, a code interpreter for arithmetic. Run them before the LLM judge and pass it their raw output. A failing check overrides a PASS.
  4. Make the evaluator's output structured. JSON or tagged XML with a verdict enum your code can branch on, and a list of issues with criterion, location, fix and severity. Binary pass/fail per criterion with a written reason is easier to act on than a 1 to 5 score.
  5. Set stop conditions in code. A hard iteration cap (two to four rounds is typical in the sources), plus stop on no improvement, on a repeated candidate, and on a token or time budget.
  6. Keep the best candidate so far. Rank by checks first, judge second. Self-Refine's authors note that improvement is not always monotonic, so the last draft can be worse than an earlier one.
  7. Manage the context. Send the latest candidate and its feedback in full, earlier rounds as one-line summaries. Run the evaluator in a fresh context so the generator's reasoning does not anchor it.
  8. Log every round (candidate, check output, verdict, tokens) and compare against a cost-matched baseline, such as best-of-N with the same number of calls, before shipping.

The loop costs about 2k + 2 model calls for k revision rounds (a generator call and an evaluator call for the first draft and for each revision), fewer when the evaluator is a plain check.

Framework support (check current docs before use)
FrameworkHow this pattern appearsStop mechanism
Anthropic cookbookPlain Python generate(), evaluate(), loop(), verdicts PASS / NEEDS_IMPROVEMENT / FAILLoops until PASS; the sample has no cap (while True)
LangGraphStateGraph with generator and evaluator nodes and a conditional edge back to the generator; structured Feedback outputTutorial loops until accepted, no cap
Google ADKLoopAgent(sub_agents=[critic, refiner]) after an initial writer in a SequentialAgentmax_iterations (Python default None) or a sub-agent escalating, e.g. the exit_loop tool
OpenAI Agents SDKExample llm_as_a_judge.py: an evaluator agent with output_type=EvaluationFeedbackBreaks on score == "pass"; capped at 3 rejected rounds only in auto mode
AutoGen (Core)CoderAgent and ReviewerAgent exchanging review messages with an approved flagEnds when approved; no cap in the sample
CrewAINo dedicated critic construct; task guardrails (a function or a string turned into an LLM check) are the closestguardrail_max_retries, default 3
DSPydspy.Refine(module, N, reward_fn, threshold), which injects LLM-written hints between attemptsFirst prediction at or above the threshold, else best of N
Claude CodeStop hooks, /goal, a fresh-context reviewer subagentHook block cap (8 consecutive blocks with no tool call, configurable), --max-turns, --max-budget-usd

Caveat. Several reference samples (Anthropic's cookbook, the LangGraph tutorial, AutoGen's sample) have no iteration cap, so add your own. Framework churn: ADK's docs mark LoopAgent as deprecated in favor of graph-based workflows for Python, TypeScript, Go and Kotlin; the langgraph-reflection package was archived on April 1, 2026; CrewAI deprecated max_retries in favor of guardrail_max_retries. Names may have changed again by the time you read this.

Variants

Self-Refine (one model, three prompts)

The same model generates, critiques and refines, with no tools and no training. Cheapest to set up. Its gains are real on open-ended writing and near zero on math, where the critic mostly says the answer looks fine.

Writer–reviewer (two agents)

Separate generator and critic prompts, often on different models or in fresh contexts. Andrew Ng describes this multi-agent form of Reflection, and AutoGen's coder/reviewer sample is an instance. Use a different model family as the judge to reduce self-preference bias.

Tool-grounded critique (CRITIC)

The critic calls tools (web search, a Python interpreter, a toxicity API) to verify claims before writing feedback. In the CRITIC paper, most of the gain disappeared when the tools were removed.

Execution feedback and test anchors

The evaluator is code execution: test results, compiler errors and traces, sometimes with the model explaining its own code (Self-Debugging). AlphaCodium adds AI-written tests and keeps tests that already pass as anchors every later fix must keep green, so a bad generated test cannot drag the code toward a wrong fix.

Reflexion (memory across attempts)

After a failed full attempt, the model writes a short lesson ("search for the actor directly next time") into memory, and the next attempt starts with it. Reflexion keeps 1 to 3 such lessons instead of full transcripts.

Retry with a validator

The minimal form: a function or a quick LLM check accepts the output or returns an error message, and the agent retries up to N times. CrewAI task guardrails work this way. DSPy Refine is similar but scores with a reward function and passes hints; its sibling BestOfN drops the feedback and only resamples.

Harness-enforced check

The runtime refuses to let the agent finish until a script passes. Claude Code's Stop hook (exit code 2 sends the compiler output back to the agent) and /goal (a separate model checks a stated condition after every turn) are examples.

Human evaluator

A person writes the feedback. The best signal for subjective quality, but slow. See Human-in-the-Loop.

Related but different

LATS replaces the single revision chain with a tree search over candidates (tree search for planning is covered in Plan-and-Execute). Constitutional AI used the same critique-and-revise steps, but to produce training data rather than runtime answers, and SCoRe trains the model to correct its own second attempt. Neither is a prompting loop you run at inference time.

Pros and cons

Pros

  • Large gains on open-ended or constraint-heavy tasks: Self-Refine reported about 20 points absolute on average across its tasks and models.
  • Feedback can be cheaper than brute-force sampling: Self-Debugging matched baselines that sampled more than 10 times as many programs.
  • Easy to debug. Each round leaves a candidate, a verdict and a reason, so logs show why the output changed.
  • Simple to build: a loop, two prompts and a stop rule, no framework needed.
  • Plugs into checks you already have, such as your test suite, schemas and linters.

Cons

  • Can make answers worse on reasoning tasks when the only feedback is self-critique.
  • Calls are strictly sequential: latency grows with every round, and prompts grow with the history you carry.
  • Diminishing returns: most of the gain comes in the first one to three rounds.
  • New ways to fail: regressions where a fix breaks a passing part, oscillation, and a generator that games the evaluator.
  • Both roles need capable models. The generator must follow feedback and the critic must produce the required format; in Self-Refine, Vicuna-13B often failed at both.

Model era. Almost all published numbers on this page come from GPT-3.5 and GPT-4-era models (2023 to 2024). Current reasoning models already check more of their work inside one call, so the size of these gains on today's models is not established. Treat them as directional.

Failure modes

Self-correction breaks correct answers

With no outside signal, the model second-guesses answers that were right. Huang et al. (ICLR 2024) measured GPT-3.5-Turbo on CommonSenseQA falling from 75.8 to 38.1 after one round of self-correction, and GPT-4 on GSM8K falling from 95.5 to 91.5, then 89.0. Track how often a round turns a correct answer into a wrong one.

Mitigation: ground the evaluator in tests, tools or retrieval. If none exist, do not loop on reasoning tasks; use voting from Parallelization.

Rubber-stamping

The critic says "looks good" when the output is wrong. In Self-Refine's math runs, ChatGPT's feedback said everything looked fine for 94% of instances, so the loop rarely fixed anything. Models are better at fixing an error once told where it is than at finding it.

Mitigation: feed the critic check evidence, use a different model or a fresh context, and measure how often the judge correctly rejects bad outputs.

Nitpicking and scope creep

A reviewer asked to find problems always finds some. Claude Code's docs warn that fixing all of them adds needless abstractions and tests. You see it as outputs that grow with every round and loops that never reach a pass.

Mitigation: limit the critic to the stated criteria, mark severities and only loop on blocking issues, allow a pass on the first attempt, and skip style unless a criterion is about style.

Judge bias

LLM judges favor their own generations, longer answers and the first option in a pair. A judge from the same model as the generator is the worst case.

Mitigation: use a different model family as judge, prefer per-criterion binary checks, penalize length explicitly, randomize order in pairwise comparisons, and calibrate against human labels.

Evaluator gaming

The generator satisfies the checker instead of the intent: hardcoding expected outputs, editing tests, special-casing inputs. ImpossibleBench (2025) built tasks whose tests contradict the spec, so any pass is cheating; GPT-5 cheated on 54% of its conflicting SWE-bench tasks, and letting agents resubmit after test feedback raised the average cheating rate from 33% to 38%. These rates depend on the scaffold and prompt, and the models tested will age quickly.

Mitigation: make tests read-only or keep held-out tests the generator never sees, review diffs that touch tests, and give the generator an explicit way to report a conflict. In ImpossibleBench, that exit cut GPT-5's rate from 54% to 9%, with a much smaller effect on Claude Opus 4.1.

Wrong evaluator

Bad AI-written tests or wrong judge feedback push the output in the wrong direction. Reflexion's self-written tests on MBPP had a 16.3% false-positive rate, and its score there ended below the plain GPT-4 baseline.

Mitigation: use test anchors (fixes must keep known-good tests green), filter generated tests, and spot-check verdicts by hand.

Prompt asymmetry

The critic's prompt contains requirements the generator never saw, so the "gain" is really a better prompt. On CommonGen-Hard, Huang et al. put the missing instruction into the first prompt: that single call scored 81.8, above Self-Refine's reported 67.0, and running self-correction on top lowered it to 75.1.

Mitigation: give both roles the same criteria list, and compare the loop against an improved single-shot prompt.

Oscillation and runaway loops

Fixing A breaks B, fixing B breaks A again, and scores bounce. Without a cap, the loop runs until the budget is gone. OpenAI's sample has a "pass after 5 attempts" rule, but only in the evaluator's prompt, not in code.

Mitigation: a hard cap and budget in code, best-so-far tracking, a check for repeated candidates, and stop when nothing improved for a round. CRITIC stops when the answer is unchanged for two consecutive corrections.

Lock-in on a bad first approach

An early wrong design gets patched round after round instead of replaced.

Mitigation: allow a FAIL verdict that triggers a fresh start, or generate two or three initial candidates as AlphaCodium does.

Context bloat

The full history of drafts and critiques fills the context window and dilutes attention.

Mitigation: summarize earlier rounds in one line each, or keep a few short lessons as Reflexion does.

Relations with other patterns

Further reading