Statistical watermarking: marking generated text without changing how it reads
Kirchenbauer and colleagues' watermarking scheme biases a model's token choices in a way that is statistically detectable without a model-owner API call, at a cost in robustness under paraphrasing that any deployment has to plan around.
AI-assisted / research-based
This field note was drafted with AI assistance and synthesizes publicly available research papers and disclosed industry practice on an emerging AI technique. It is not based on confidential deployment data, is not investment, legal, medical, or security advice, and every primary claim links directly to its source so you can verify it yourself.
Distinguishing AI-generated text from human-written text after the fact, purely through style analysis, is difficult and grows less reliable as models improve. Statistical watermarking takes a different approach entirely: bias the generation process itself, in a way invisible to a human reader but detectable to anyone holding the right key.
The watermark doesn't change what the text says. It changes which of several equally good words got picked, over and over, in a pattern only the detector recognizes.
A green list, chosen fresh at every token
A Watermark for Large Language Models ↗, by Kirchenbauer, Geiping, Wen, Katz, Miers, and Goldstein, operates directly at the token-sampling step. Before each token is generated, a pseudorandom function seeded by the preceding tokens partitions the entire vocabulary into a "green list" and a "red list" — a different, context-dependent split at every position in the text. The sampling procedure then softly increases the probability of choosing from the green list, without hard-restricting the model to it.
A bias too small to read, large enough to detect
Over many generated tokens, this produces a statistically detectable bias: a piece of watermarked text will contain an unusually high proportion of green-list tokens, a pattern the paper shows is very unlikely to occur by chance in ordinary human writing. The paper proposes a formal statistical test for detecting this bias with an interpretable p-value, and — importantly for deployment — the detection algorithm is efficient and open-source, requiring no access to the underlying model's API or parameters, only the same seeding key used during generation. The scheme was tested on a multi-billion-parameter model from the Open Pretrained Transformer (OPT) family.
The trade the "soft" bias is making
A small, controlled per-token bias is precisely what keeps the watermark close to invisible in ordinary reading — a heavier bias would be more reliably detectable but would also more noticeably distort the text's quality and naturalness. This is a genuine trade the scheme's designers made deliberately, not an oversight: detectability and text quality pull in opposite directions, and the scheme's parameters set where on that curve a specific deployment sits.
What breaks the signal
Watermark detection reliability is not uniform across every use case, and two specific weaknesses matter for any real deployment. Robustness degrades under paraphrasing, translation, or adversarial editing — an adversary motivated to strip the watermark, or even an innocent downstream editing pass, can reduce or eliminate the detectable bias. Detection reliability also drops on short passages, because the statistical test needs enough tokens to establish that the observed green-list proportion is unlikely to have occurred by chance; a short snippet simply doesn't carry enough signal.
A watermark also only ever detects text generated by a model that was actually running with this scheme enabled. It says nothing about text from any other, unwatermarked source — a fact worth stating plainly, because "watermarking" can be mistaken for a general AI-text detector rather than the narrower, source-specific signal it actually is.
Detection confidence is not one number
What this changes in production
Content provenance as one signal among several
Watermarking is a credible input to a broader content-provenance strategy — combined with metadata standards and platform-level disclosure — for distinguishing generated text at scale, rather than a standalone, definitive proof of origin on its own.
Academic and content-integrity tooling
As an imperfect but useful signal, watermark detection can flag content for further review in educational or editorial integrity workflows, provided the tool's operators understand and communicate its specific failure modes rather than presenting a detection result as certain.
Internal compliance and provenance tracking
An organization can watermark its own model's outputs specifically to later confirm, for its own purposes, whether a given piece of text originated from that system — a narrower and more tractable use case than general-purpose AI-text detection across arbitrary sources.
A production checklist
Before relying on statistical watermarking for a provenance claim, the team should be able to answer:
- Is the watermark being presented as proof of origin for this specific model's output only, or is it being mistaken for general AI-text detection across any source?
- Has detection reliability been tested on realistic passage lengths for this use case, not just long, ideal-case text?
- What happens to the provenance claim if the text is paraphrased, translated, or edited before it reaches a detector?
- Is the bias strength tuned appropriately for this deployment's tolerance for any perceptible quality impact?
- Who holds the seeding key, and what is the process if that key needs to be verified by a third party?
- Is watermarking being deployed alongside other provenance signals, or as the sole mechanism — and does the stated confidence in any provenance claim reflect that choice?
Research referenced
- Kirchenbauer, Geiping, Wen, Katz, Miers, and Goldstein (2023), A Watermark for Large Language Models ↗.