Self-Refine: letting a model critique and rewrite its own output
A simple inference-time loop — generate, self-critique, revise — measurably improves output quality across seven tasks and multiple model families, without any additional training.
AI-assisted / research-based
This field note was drafted with AI assistance and synthesizes publicly available research papers and disclosed industry practice on an emerging AI technique. It is not based on confidential deployment data, is not investment, legal, medical, or security advice, and every primary claim links directly to its source so you can verify it yourself.
The most common way to improve a language model's output is assumed to require a better prompt or a fine-tuned model. Self-Refine tests a third option that needs neither: keep the same frozen weights, and let the model critique and revise its own answer before anyone sees it.
Generator, critic, and editor can be the same model wearing three different prompts.
A loop with three roles, one set of weights
Self-Refine: Iterative Refinement with Self-Feedback ↗, by Madaan and colleagues, structures inference as a repeating three-step loop, with the same underlying model playing every role:
1. Generate an initial output for the task. 2. Feedback — the same model critiques that output against the task's requirements. 3. Refine — the same model produces a revised output conditioned on both the original attempt and its own feedback.
One model plays three roles
This repeats for a small, bounded number of iterations, or until the model's own feedback indicates no further improvement is needed. Critically, the paper reports this requires no supervised training data, no additional fine-tuning, and no reinforcement learning — it is entirely a prompting and control-flow pattern applied at inference time, using a single frozen model as generator, critic, and refiner.
What the evaluation actually shows
The paper evaluates Self-Refine across 7 diverse tasks — spanning dialog response generation to mathematical reasoning — using GPT-3.5, ChatGPT, and GPT-4. Across every evaluated task, outputs produced with Self-Refine were preferred over conventional single-pass generation from the same model, by both human judges and automatic metrics, with the paper reporting roughly a 20% absolute average improvement in task performance. Notably, this held even for GPT-4 — evidence that self-critique can extract additional quality from a model at the frontier of its own training, not only from weaker models with more obvious room for improvement.
The first revision does most of the work
Where the loop stops helping
Self-critique is bounded by the same model's own judgment, which creates a specific limitation: a model's blind spots in generation are frequently the same blind spots it has in feedback, since both draw on the same underlying capabilities. A model that consistently makes a particular kind of reasoning error is not guaranteed to reliably catch that same error when asked to critique its own output.
In practice, quality gains from iterative refinement tend to be front-loaded — most of the improvement appears in the first revision pass, with diminishing and occasionally negative returns from additional iterations. Each additional round also adds latency and inference cost proportional to the number of loop iterations, which is a real production cost for a technique that requires no training investment but is not free at inference time.
What this changes in production
Writing and code-review assistants
Products that show a visibly revised draft — rather than a single unreviewed generation — can add a self-critique pass on top of an existing deployed model, without retraining or switching to a different, more expensive model.
Structured-output quality gates
A first-pass generation can be checked by the same model against format or completeness requirements and corrected before being returned, catching a class of errors that a single generation pass would otherwise surface only to the end user or a downstream parser.
Incremental quality improvement without a model change
Teams already running a model in production can add an inference-time refinement loop as a lower-risk lever than switching models or fine-tuning, when the goal is a measured quality lift rather than a fundamentally different capability.
A production checklist
Before adding a Self-Refine-style loop to a production path, the team should be able to answer:
- How many iterations actually produce measurable improvement on this specific task, versus added latency with no quality gain?
- Is there a stopping condition beyond a fixed iteration count — does the system detect when feedback stops indicating meaningful problems?
- What is the added cost and latency per request, and does the product's SLO tolerate it?
- Has this been tested on the task's known failure modes specifically, or only on aggregate quality metrics that could mask a persistent blind spot?
- Could iterative refinement introduce a new failure mode — for example, a correct initial answer being "corrected" into a wrong one?
- Is refinement applied selectively to outputs that need it, or uniformly to every request regardless of whether the first pass was already adequate?
Research referenced
- Madaan et al. (2023), Self-Refine: Iterative Refinement with Self-Feedback ↗.