Definition
LLM-as-a-judge is the practice of having one language model grade another model's output against a written rubric, in place of a human rater. The judge is shown the original question, one answer or a pair of competing answers, and instructions describing what counts as good; it returns a verdict — a score, a winner, or a pass/fail label — usually with a sentence of reasoning attached.
It exists for a cost reason. A careful human comparison of two long answers takes minutes of expert attention; a judge call takes seconds and a fraction of a cent, which is the difference between evaluating a two-hundred-case regression suite on every pull request and evaluating it twice a year. The surprising part is that it works at all. In the paper that named the technique, Zheng et al. (2023) found GPT-4's pairwise judgments agreed with human experts on 85% of non-tied MT-Bench comparisons, while the human experts agreed with each other on only 81%. A good judge is about as close to a panel of people as the people are to one another.
That result is also the trap, because 85% agreement is not 85% accuracy, and the missing 15% is not random noise. It is systematic: the judge prefers the answer it read first, prefers the longer answer, mildly prefers answers from its own model family, and — the failure that matters most — makes mistakes that are correlated with the generator's, because it is often the same model, or a close relative, with the same blind spots. A judge cannot see an error it would have made itself. Everything useful about this technique lives in what you do about that.
How It Works
A judge is a prompt, not a model. There is no separate architecture and usually no training: you
take a capable general model and give it a template with four parts — the user's original question,
the response or responses under evaluation, a rubric spelling out the criteria, and a strict output
format so the verdict can be parsed. Zheng et al.'s pairwise template is a good specimen and is
worth reading in full, because most production judges are variations on it: it lists the criteria
(helpfulness, relevance, accuracy, depth, creativity, level of detail), explicitly instructs the
model to ignore position and length, and demands the verdict as [[A]], [[B]] or [[C]] so a
script can read it. Notably, those anti-bias instructions are in the prompt and the biases below
were measured with them present. Telling a judge not to be biased does not make it unbiased.
Three mechanical details separate a judge that produces a usable number from one that produces noise.
Parse the verdict, do not read it. A judgment that arrives as free text has to be reduced to a symbol before it can be counted, and the reduction is where silent failures live. Zheng et al.'s Table 2 reports an "Error" column for judgments in the wrong output format — 1.2% to 3.8% for the weaker judges, 0.0% for GPT-4. A pipeline that treats an unparseable verdict as a tie, or worse as a loss, is putting the judge's formatting reliability into your quality metric. Count parse failures separately and look at them.
Pin everything and log it. A judge is a non-deterministic component: the model identifier, the temperature, the rubric wording and the template are all inputs to the number you are about to plot on a dashboard. Change any of them and the score moves for reasons that have nothing to do with the system under test. This is why monitoring a generative system treats the judge as a monitored dependency in its own right: a quality "regression" that coincides with a judge upgrade is a measurement artefact, and you cannot tell the two apart after the fact unless the judge's version sat in the change log next to the model's.
Run every comparison in both orders. This is the single highest-value protocol change and it is cheap to reason about. Zheng et al. call it the conservative approach: judge the pair as (A, B), then again as (B, A), and declare a winner only if the same answer wins both times. Disagreement becomes a tie. The cost is exact — two calls per comparison instead of one — and the yield depends on how consistent your judge is. At GPT-4's measured 65% swap consistency on near-identical answers, a 200-pair suite costs 400 judge calls and returns about 130 decided verdicts and 70 forced ties. If you need 200 decided verdicts you must run 308 pairs, or 615 calls. That is a 3× budget multiplier over the naive single-call design, and it buys the only thing that makes the output trustworthy.
Types
The literature recognises three grading modes, and they are not interchangeable.
Pairwise comparison shows the judge two answers to the same question and asks which is better, with ties allowed. Single-answer grading (also called direct scoring or pointwise) shows one answer and asks for a score, typically 1–10 against a rubric. Reference-guided grading adds a known-good answer to the prompt and asks the judge to compare against it before deciding.
Pairwise is the more reliable default, for a reason that has a close parallel in reward models: an absolute score has no fixed zero and no fixed unit. It is whatever the judge's internal rubric happens to be that day. Zheng et al. put it directly — single grading "may be unable to discern subtle differences between specific pairs, and its results may become unstable, as absolute scores are likely to fluctuate more than relative pairwise results if the judge model changes." Direct scores also compress: judges cluster their ratings into a narrow band near the top of whatever scale you hand them, so a rubric with ten levels resolves far fewer than ten. Pairwise asks only for an ordering, which is the one judgment a model makes dependably, and orderings are directly convertible into win rates and Elo-style ratings.
The measured gap is real but modest. On Chatbot Arena data, GPT-4 in pairwise mode agreed with humans on 87% of non-tied votes against 85% for the same model doing single-answer grading; on MT-Bench both reached 85%. Pairwise's decisive advantage is stability across judge versions, not raw accuracy on any one day.
What pairwise costs is scale. Comparisons grow quadratically: ranking 10 candidate prompts pairwise is 45 comparisons, and with order-swapping, 90 calls; 20 candidates is 190 comparisons and 380 calls. Direct scoring is linear and is the right tool when you have many candidates and only need a coarse sort, or when there is no natural opponent to compare against — which is the usual situation in production, where a live response has no counterfactual twin.
Reference-guided grading is the mode people reach for last and should reach for first whenever a reference exists. Zheng et al. tested GPT-4 on ten maths questions where one answer was wrong, scoring each pair in both orders: with the default prompt the judge called an incorrect answer correct 14 times out of 20; with chain-of-thought prompting, 6 out of 20; with a reference answer in the prompt, 3 out of 20. Handing the judge the answer key cut its failure rate by more than four-fifths on exactly the task where an unaided judge is weakest.
Real-World Applications
Public leaderboards run on judges. AlpacaEval scores instruction-following models by having a judge compare each model's answers against a fixed baseline's, and its length-controlled variant reports a Spearman correlation of 0.98 with human votes on LMSYS Chatbot Arena, up from 0.94 before the correction (Dubois et al., 2024). That is a leaderboard built entirely from model judgments tracking a leaderboard built entirely from human ones, which is the strongest single piece of evidence that the technique carries real signal.
Internal regression evaluation is the highest-value use and the least discussed. If you ship a product on an LLM, every prompt edit and model upgrade is a silent risk, and a suite of your own cases with known-good outputs is the only way to catch a break. For anything open-ended — a summary, a support reply, a rewritten paragraph — there is no string match that can score it, so the judge is the assertion. This is the mechanism that turns a benchmark from a public leaderboard into a test suite you run on every pull request, and it is where most of the judge calls in the industry are actually spent.
Production quality monitoring. A deployed generative system has no accuracy metric to degrade: no labels arrive, and a fluent, confident, wrong answer returns a perfectly normal 200 response of perfectly normal length. Sampling live traffic and scoring it with a judge nightly is the closest available substitute for the accuracy curve a classifier gives you for free. It only works if you hold the judge fixed and treat its variance as part of the measurement — see monitoring.
Guardrails and routing. Judges run inline as well as offline: a cheap model checks whether a response is grounded in the retrieved documents, whether it leaked a system prompt, or whether a draft is good enough to send or should be escalated. Here the judge's latency and cost are on the critical path, which usually forces a smaller judge than an offline evaluation would use — and smaller judges are markedly more biased, as the numbers below show.
Training signal. Constitutional AI and RLAIF replace some or all human preference labels with model-generated ones, which is a judge wired into the training loop rather than the test harness. This is where the distinction from a reward model matters: a reward model is a scalar head trained on preference pairs, fast enough to call millions of times during RLHF; a judge is a prompted general model that writes its comparison out in text. Judges dominate evaluation because they are flexible and need no training; reward models dominate training because they are cheap per call.
Key Concepts
Agreement is not accuracy. Every headline figure in this area is an agreement rate against human labels, and humans disagree with each other. On non-tied MT-Bench votes, human experts agreed 81% of the time. That is the ceiling: a judge scoring 85% is not 15% wrong, it is 15% different from a particular panel, and some of that difference is the panel's.
Raw agreement flatters, so convert it. Cohen's kappa subtracts the agreement two random judges would reach anyway. On a forced binary choice, chance is 50%, so 85% raw agreement is a kappa of 0.70, and the human panel's 81% is 0.62. Those are respectable numbers, not the near-perfect ones the percentages suggest. Report kappa when you validate a judge on your own data.
Agreement depends on the size of the gap you are measuring. Zheng et al. broke agreement down by how far apart the two models were and found it rose from about 70% for closely matched pairs to nearly 100% for badly mismatched ones. This is the most practically important caveat on the whole technique: a judge is most reliable exactly when you least need it, and least reliable in the regime you actually work in — comparing yesterday's prompt with today's. A validation set built from obvious wins will certify a judge that cannot do your job.
Judge self-consistency is not judge quality. GPT-4 in pairwise mode agreed with GPT-4 in single-answer mode on 95–97% of non-tied MT-Bench votes, while agreeing with humans on 85%. A judge is far more reproducible than it is right, and reproducibility is what a dashboard shows you. Stable numbers are evidence of a stable judge, not a correct one.
The only real validation is human labels on your own data. Public agreement figures were measured on chat assistant answers to open-ended questions; nothing carries them over to your medical summaries or your SQL rewrites. Hand-label a few hundred of your own cases — weighted towards close calls — and measure the judge against them before you trust a single number it produces. Re-measure when the judge version changes.
Challenges
Position bias. The judge favours whichever answer it read first, and the effect is large. With two deliberately similar answers, Zheng et al. found GPT-4 gave the same verdict after swapping only 65.0% of the time, choosing the first answer in 30.0% of cases; GPT-3.5 was consistent 46.2% of the time and Claude-v1 just 23.8%, picking the first answer in 75.0% of cases. Wang et al. (2023) measured the same thing as a "conflict rate" and showed how it can be exploited: with GPT-3.5 as the evaluator, Vicuna-13B won 2.5% of comparisons against it in one position and 82.5% in the other, and the abstract's headline is that Vicuna could be made to beat ChatGPT on 66 of 80 queries purely by choosing the presentation order. The severity tracks how close the answers are — the same paper reports a 46.3% conflict rate for GPT-4 on a close pair and 5.0% on a mismatched one. Swapping is the fix; few-shot examples help too, lifting GPT-4's consistency from 65.0% to 77.5%.
Verbosity bias. Judges prefer longer answers, independent of whether they are better. Zheng et al. built a "repetitive list" attack: take an answer containing a numbered list, ask a model to rephrase the list without adding information, and prepend the rephrasing, doubling the length while adding nothing. Claude-v1 and GPT-3.5 preferred the padded version 91.3% of the time; GPT-4 fell for it 8.7% of the time. The effect is not confined to adversarial constructions. Dubois et al. (2024) took one model, compared it against itself on AlpacaEval, and varied only the verbosity instruction in its prompt: its win rate moved from 22.9% to 64.3%. Same model, same capability, same judge — a 41-point swing bought with an instruction to be wordy. Their length-controlled correction, a regression that predicts what the preference would have been at zero length difference, narrows that range to 41.9%–51.6% and cuts the normalised standard deviation across verbosity prompts from 25% to 10%.
Self-preference. A judge tends to favour its own family's output. Zheng et al. measured GPT-4 giving its own answers a 10% higher win rate than humans did, and Claude-v1 giving its own a 25% higher one — while cautioning that with their sample size they could not establish this as a controlled effect, since you cannot restyle a response into another model's voice without changing its quality. Panickssery et al. (2024) found the mechanism: LLMs can recognise their own outputs at above-chance accuracy out of the box, and fine-tuning a model to be better at self-recognition increases its self-preference in step — a linear relationship that survives their controls for obvious confounders. The practical rule is simple: never let a model be the sole judge of its own generations, and be suspicious of any evaluation where the judge and the top-scoring model come from the same lab.
Correlated errors — the failure the other three are symptoms of. A judge is useful only to the extent that its mistakes are independent of the generator's. They are not. Zheng et al.'s Chatbot Arena figures make this measurable: GPT-4 as judge agreed with humans on 87% of non-tied votes and GPT-3.5 as judge on 83%, but the two judges agreed with each other on 94%. If their errors were independent, two judges that good would agree with each other 87% × 83% + 13% × 17% = 74.4% of the time. The observed 94% is nearly twenty points of shared error — mistakes both models make, in the same direction, on the same items, invisible to any amount of cross-checking between them.
The mechanism is plain once stated. A judge drawn from the same pretraining distribution as the generator has the same gaps in its knowledge, the same misconceptions and the same stylistic tastes. Ask it to check a claim the generator hallucinated and it will often find the claim plausible for the identical reason the generator produced it. A closely related failure is documented directly: Zheng et al. show GPT-4 grading an elementary maths problem it can solve correctly when asked in isolation, and being talked out of the right answer by the wrong answer it was grading. Whether the judge inherits the mistake or is argued into it, the result is the same — the output under evaluation shapes the evaluation. This is the same structure that makes a model a poor checker of its own work in any error-handling pipeline, and it puts a hard floor under what judge-based evaluation can catch. Verifiable checks — a unit test, an executed query, a retrieved citation — are not the same kind of thing as a judge and should be used instead of one wherever the task admits them.
Goodharting the judge. The moment a judge score becomes a target, the system starts optimising for the judge rather than the quality. This is exactly reward-model overoptimisation with a prompt where the scalar head used to be, and it arrives faster than people expect, because the judge's biases are the cheapest gradient available: adding length, adding structure, adding confident phrasing and adding markdown headers all raise scores without touching substance. LMArena's own answer was style-controlled ratings: a regression that carries answer length and the counts of markdown headers, bold spans and lists as covariates, so credit for formatting is attributed to the formatting rather than to the model. If your judge is also your optimisation target, you need a held-out evaluation the optimiser never sees.
Cost and latency stop being negligible at scale. The per-call price is small, but the protocol multiplies it: two calls for order-swapping, several judges if you use a panel, several samples if you average out judge variance. A single-call design that becomes a swapped three-judge panel is 6× the calls and 6× the bill for the same suite, and a judge that reads two long answers pays input tokens for both.
Future Trends
Juries of small models instead of one large judge. Panels of several cheaper models, voting, consistently trade well against a single frontier judge — partly on cost and partly because a panel drawn from different labs has less correlated error than one model consulted repeatedly. The design question is which axis of diversity actually buys independence: different prompts of the same model buy almost none, different model families buy some, and a rule-based or execution-based checker alongside the models buys the most.
Statistical corrections applied on top of the judge. Length-controlled AlpacaEval is the template: rather than trying to prompt the bias away, fit a model of the bias and condition it out afterwards. The approach generalises to any confound you can measure — formatting, hedging, the presence of code blocks — and it is more honest than an instruction the judge demonstrably ignores.
Verifiable rewards displacing judges where a checker exists. The clear trend in training is to replace subjective scoring with programmatic verification wherever the task allows: unit tests for code, exact-match for maths, executable checks for structured output. Judges retreat to the genuinely open-ended qualities — tone, helpfulness, whether a summary is faithful — which is where they were always least replaceable and, awkwardly, also where their agreement with humans is lowest.
Judge versions becoming first-class artefacts. As evaluation results start feeding release decisions, the judge model, prompt hash, temperature and rubric are moving into the same version control and change log as the system under test, with historical scores recomputed when the judge changes. An evaluation number without a judge version attached is becoming as meaningless as a benchmark score without a harness.
Code Example
Three calculations worth doing before you trust a judge-based result: convert raw agreement into kappa, check how much of two judges' agreement is shared error rather than shared correctness, and size the sample the swap protocol leaves you with.
from math import sqrt
# Cohen's kappa: what survives of a raw agreement rate once you subtract the
# agreement two coin-flipping judges would reach anyway. Non-tie votes only,
# so chance is 50%. Figures from Zheng et al. (2023), Tables 5 and 6.
def kappa(agreement, chance=0.5):
return (agreement - chance) / (1 - chance)
rows = [("GPT-4 judge vs human (Arena)", 0.87),
("GPT-3.5 judge vs human (Arena)", 0.83),
("GPT-4 vs GPT-3.5 judge (Arena)", 0.94),
("human vs human (MT-bench)", 0.81)]
for name, a in rows:
print(f"{name} agreement {a:.0%} kappa {kappa(a):.2f}")
# If two judges erred independently, their agreement with each other would be
# pinned by their agreement with humans: both right, or both wrong the same
# way. Everything above that line is error the two judges share.
p4, p35, observed = 0.87, 0.83, 0.94
expected = p4 * p35 + (1 - p4) * (1 - p35)
print(f"\nindependent-error prediction {expected:.1%} vs observed {observed:.0%}"
f" -> {(observed - expected) * 100:.1f} pts of shared error")
# The conservative swap rule: judge every pair twice, in both orders, and keep
# only the verdicts that survive. 65% is GPT-4's measured swap consistency.
consistency, suite = 0.65, 200
decided = suite * consistency
print(f"\n{suite} pairs -> {suite * 2} judge calls -> {decided:.0f} decided,"
f" {suite - decided:.0f} forced to a tie")
print(f"to keep {suite} decided verdicts: {suite / consistency:.0f} pairs,"
f" {2 * suite / consistency:.0f} calls")
# And what a win rate off that many verdicts can actually resolve.
for n in (130, 200, 1000):
margin = 1.96 * sqrt(0.25 / n) * 100
print(f"n={n:5d} 95% CI on a 50% win rate: +/- {margin:.1f} pts")
Running it prints:
GPT-4 judge vs human (Arena) agreement 87% kappa 0.74
GPT-3.5 judge vs human (Arena) agreement 83% kappa 0.66
GPT-4 vs GPT-3.5 judge (Arena) agreement 94% kappa 0.88
human vs human (MT-bench) agreement 81% kappa 0.62
independent-error prediction 74.4% vs observed 94% -> 19.6 pts of shared error
200 pairs -> 400 judge calls -> 130 decided, 70 forced to a tie
to keep 200 decided verdicts: 308 pairs, 615 calls
n= 130 95% CI on a 50% win rate: +/- 8.6 pts
n= 200 95% CI on a 50% win rate: +/- 6.9 pts
n= 1000 95% CI on a 50% win rate: +/- 3.1 pts
Read the last block before designing an evaluation. A 200-case suite run through the swap protocol leaves about 130 decided comparisons, and 130 comparisons cannot resolve a win rate to better than roughly ±8.6 points. If the prompt change you are testing moves quality by three points, this suite will report noise and you will ship on it. The judge is rarely the binding constraint on a small evaluation — the sample size is, and the swap protocol you need for the judge to be trustworthy makes the sample smaller.