← All field notes

Research Watch · Safety · Evaluation

Automated red-teaming: using one model to find another model's failures

DeepMind's automated red-teaming research uses a language model to generate adversarial test cases at a scale manual security review cannot match, surfacing a larger and more diverse set of failures in the same amount of time.

AI-assisted / research-based

This field note was drafted with AI assistance and synthesizes publicly available research papers and disclosed industry practice on an emerging AI technique. It is not based on confidential deployment data, is not investment, legal, medical, or security advice, and every primary claim links directly to its source so you can verify it yourself.

Manual red-teaming — people deliberately trying to make a model produce harmful or unintended output — is thorough but slow, expensive, and bounded by however many hours of expert time an organization can allocate before a release. A DeepMind research line asks whether a language model can generate the adversarial test cases instead, and what that trades away.

A human red-teamer brings judgment a model cannot yet match. A model brings volume no team of humans can match either.

Red model, target model, classifier

Red Teaming Language Models with Language Models ↗, by Perez, Huang, Song, Cai, Ring, Aslanides, Glaese, McAleese, and Irving — published at EMNLP 2022 — structures automated red-teaming as three roles. A "red" language model automatically generates large numbers of test inputs specifically designed to elicit harmful, offensive, or otherwise undesirable output from a "target" model. A separate classifier scores the target's responses to identify which generated test cases actually succeeded at eliciting problematic behavior — distinguishing a test case that merely looks adversarial from one that actually worked.

Process / adversarial testing

One model probes another, continuously

FIG 01 - MOTION
Automated red-teaming generation and classification loop A red language model generates large numbers of adversarial test cases, a classifier scores the target model's responses to identify which cases actually elicited harmful behavior, and the successful failure cases feed back into mitigation and the next round of test generation. GENERATERed LM test casesat scale, automated TARGETModel under testresponds to each case CLASSIFYScore for failureswhich cases actually worked SCALEsurfaces a larger, more diverse set of failure cases than manual red-teaming in the same time
Automated generation trades the depth of an expert human red-teamer for breadth and repeatability — a complement to manual testing, run continuously rather than once before release.

The paper's central finding is a scale argument, not just a feasibility argument: automated generation surfaces a substantially larger and more diverse set of failure cases than manual red-teaming can practically produce in comparable time. The resulting failure cases then feed directly into studying and reducing those specific behaviors in the target model — the loop is designed to be run repeatedly, not once.

What "automated" trades away

An automated red-team model can only generate the kinds of failures it was designed and prompted to look for — its coverage is bounded by its own training and prompting, in the same way any generative system's outputs are bounded by what it learned to produce. This creates a specific and important risk: the red model can share systematic blind spots with the very class of model it is testing, since both are drawn from similar training approaches and data. A failure mode that is invisible to language-model-style reasoning in general is not a failure mode this technique is well positioned to find.

This is precisely why the paper's contribution is best understood as a complement to manual expert red-teaming, not a replacement for it. Novel, creative, or highly domain-specific adversarial scenarios — the kind a skilled human red-teamer invents by drawing on context the automated system was never exposed to — remain a distinct capability automated generation does not substitute for.

Comparison / testing coverage

Volume and depth are different kinds of coverage

FIG 02
Coverage and depth trade-off between manual and automated red-teaming Manual red-teaming produces fewer test cases with deep, creative human judgment, while automated red-teaming produces a much larger and more diverse volume of cases bounded by what the red model itself was trained to generate, and the two approaches are complementary rather than substitutes. MANUAL RED-TEAMINGFewer cases, deeper judgmentnovel, creative, domain-specific AUTOMATED RED-TEAMINGMore cases, bounded by the red modelcan share the target's blind spots COMPLEMENTthe paper's own framing is scale versus depth, not a replacement for human review
Automated red-teaming trades a human's creative judgment for volume and repeatability — which is exactly why the paper frames it as a complement, not a substitute.

What this changes in production

Pre-release safety testing at scale

Running automated adversarial test generation against a candidate model before it reaches production traffic surfaces a volume and diversity of failure cases that a fixed-size human red-team, working within a release timeline, cannot practically replicate.

Regression testing for safety fixes

After a specific mitigation ships, automated red-teaming can confirm — at a scale manual retesting cannot match — that the fix actually closed the failure classes it targeted, without silently missing regressions in adjacent, similar prompts.

Continuous adversarial testing tied to the release cycle

Because the loop is automated and repeatable, it can run on every candidate release rather than as a one-time audit performed before a single launch, turning red-teaming into an ongoing process rather than a point-in-time gate.

A production checklist

Before relying on automated red-teaming as a safety gate, the team should be able to answer:

  • Is automated red-teaming running alongside human expert red-teaming, or has it replaced human review entirely?
  • What is the classifier's own accuracy at identifying genuine failures, and has it been validated against human judgment on a sample?
  • Could the red model and the target model share systematic blind spots from similar training data or approaches — has this been considered, not just assumed away?
  • Is automated red-teaming re-run on every release candidate, or only at infrequent, scheduled intervals?
  • Are discovered failure cases tracked through to an actual mitigation, or catalogued without a closed remediation loop?
  • Does the red-teaming process specifically probe for novel, creative adversarial scenarios, or only variations on previously known failure patterns?

Research referenced

Continue readingReturn to field notes →