Reliability Scales Inversely: Hallucinations Snowball Faster in Bigger Language Models

TL;DR: This paper argues that hallucination in LLMs has two mathematically separable components — a knowledge gap the model can partly feel, and a decoding risk it structurally cannot — and that scaling fixes the first while making the second worse. Across Qwen3, OLMo-3, and Llama-3 ladders, start-of-response hallucination falls up to 7× with size while within-response degradation grows up to 39×. The proposed mechanism is a self-conditioning loop: sampling converts a moment of risk into permanent bias in the context, the model's felt uncertainty relaxes within a single token while the risk persists 3–5× longer, and each fabrication raises the hazard of the next (1.08×→1.71× with scale). An oracle-in-the-loop intervention that shrinks only the variance of model–oracle disagreement, while pinning its mean, removes 35–74% of web-verified hallucinations — the paper's strongest evidence that the variance term is causal and not just correlated.

The problem with the knowledge-gap story

The dominant account of hallucination is simple: the model doesn't know the fact, calibrated models must guess on rare facts, and evaluations reward guessing over abstention. Under that account, scale should help — more parameters, more memorized facts, fewer gaps — and the residual errors should be detectable by reading the model's own uncertainty, which is what semantic entropy, SelfCheckGPT, and their descendants do.

Three known facts sit awkwardly with this. Models fabricate around facts they demonstrably know. Within a single decode, each fabrication makes the next more likely ("snowballing"). And interventions on representations, not knowledge, reduce hallucination at fixed data and parameters. This paper's contribution is a single quantitative frame that ties all three together and — this is the interesting part — predicts they should get worse with scale.

One variable, split into what the model can and cannot see

Everything runs through one per-token quantity: the disagreement between the model and a stronger oracle over candidate tokens,

where p_M is the model's next-token distribution, p_O is a larger same-family model used as the oracle, and x ranges over candidates drawn from p_M. Take moments of \delta under the model's own distribution and you get an exact split:

The mean of \delta is the reverse KL — the average gap between model and oracle, the "how wrong on average" term. The variance is the spread of disagreement across the candidates the model is actively entertaining: the model and oracle might agree on average, yet the model may be one sampling draw away from a token the oracle considers absurd.

The paper's framing move is to read this split as an interpretability statement. Expanding the KL as H(p_M, p_O) - H(p_M), the model's own entropy — the one uncertainty signal it can read off its own logits — appears only inside the bias term. The risk term \mathrm{Var}[\delta] depends on the oracle's opinion of every candidate; no functional of p_M alone can see it. Every self-monitoring detector in the literature is a functional of p_M. If risk drives hallucination, those detectors are structurally blind to the driver.

Why should risk grow relative to bias with scale? Maximum-likelihood pretraining pushes both model and oracle toward the data distribution, which drives the mean gap toward zero. Nothing in the objective comparably narrows the spread. Empirically this checks out in an unusual way: as Qwen3 scales from 0.6B to 32B, the mean gap |\mathbb{E}[\delta]| falls 1.17→0.28 nats while the excess kurtosis of \delta rises 8.9→33.0 (Llama-3.2: 18.4→32.6). Bigger models agree with the oracle more often on average and disagree more catastrophically when they don't. Risk's share of the squared error climbs from 31% to 49% between 1.7B and 14B.

The mechanism: sampling freezes risk into bias

Here's the aha. At each step the model holds a lottery over candidate tokens, some of which the oracle would score as fabrications. Sampling collapses that lottery to one token and appends it to the context. If the drawn token was an unsupported one, the momentary risk is now a fixed premise every subsequent token must condition on — variance has been converted to bias, permanently, by the autoregressive loop itself.

The two channels then decay on very different clocks. The model's entropy H(p_M) measures only local next-token predictability, and fluent continuation restores that almost instantly: measured half-life under one token across Qwen3 rungs. The risk \sqrt{\mathrm{Var}[\delta]} reflects the model–oracle gap over the now-poisoned context, and it persists with half-lives of 3.4–4.6 tokens plus the highest post-decay plateau of any channel — up to 23× longer-lived than felt uncertainty.

That gap between the two clocks defines a confident-but-precarious regime: low entropy, low mean gap, high risk. A three-state fit over (bias, entropy, risk) shows this regime disproportionately occupies the tokens between consecutive fabrications — the excess likelihood of a bridge token being precarious when a fabrication leads to another fabrication (vs. a supported claim) grows from near zero at small scales to +15% at 8B and +69% at 14B. And the across-claim snowball itself strengthens with scale: the relative risk of a fabrication following a fabrication climbs 1.08×→1.71× (topic-controlled; 1.28×→2.85× raw), even as the spontaneous fabrication rate falls 0.74→0.29. Bigger models fabricate less often but recover worse once they do.

The headline scaling result

On LongFact++ free-run generations across Qwen3 0.6B–32B, the start-of-response hallucination rate ("initial gap") falls from 63% to 11%, while the relative rise in hallucination from start to end of a response ("degradation," from a per-claim mixed-effects logistic fit) climbs from 14% to 153%. The pattern replicates on FActScore biographies, on WildHallucinations entities drawn from real chat logs (significant at 10 of 11 rungs across all three families), and — importantly — survives a split by whether the entity has a Wikipedia page, which is where a pure knowledge-gap account would locate the failure.

The causal test: a variance knob at fixed KL

Correlation between risk and fabrication could be epiphenomenal, so the paper builds a surgical intervention. A co-resident oracle scores \delta(x) over the full vocabulary at every step. When the oracle strongly prefers a token the model underweights (a z-scored trigger), the sampling distribution is replaced by

where \mu = \mathrm{KL}(p_M\|p_O) is the current mean gap, the quadratic term suppresses tokens whose disagreement departs from \mu in either direction, and \nu is solved per step so the mean is preserved exactly. This is a pure variance contraction at fixed KL — it doesn't leak the oracle's answer (that would shift the mean), it just narrows the lottery. The manipulation check is convincing: realized variance falls to the target dose while mean drift stays around 10^{-11} nats, eleven orders of magnitude below the bias it must preserve.

The result: rest-of-response web-verified hallucination drops by an absolute 0.13–0.33 (35–74% relative) at all six model×family rungs, every bootstrap CI excluding zero. Applying the contraction only on the inter-claim bridge prevents 21–60% of second fabrications. Contraction doses help; expansion doses don't — the dose-response you'd want if variance is genuinely the lever. Confounds are checked: response length moves ±0.5%, claim count ±10%, self-BLEU ~1%.

The detector-blindness corollary is verified directly: semantic entropy fires 28–34% less on bridge fabrications than on onset fabrications (Mann–Whitney p<10^{-16}), even though the bridge carries nearly 4× as many fabrications (10,900 vs. 2,858). The failure mode that dominates the error budget is exactly the one self-monitoring cannot see. A silver lining: a label-free Markov-switching fit on \delta alone separates supported from unsupported tokens at AUROC 0.68–0.71 across all scales — a white-box risk marker, though it still requires an oracle.

What to be skeptical about

Everything is defined relative to a bigger sibling. "Risk" is \mathrm{Var}[\delta] against Qwen3-14B or 32B — models that themselves hallucinate heavily (32B shows 230% degradation on FActScore). The paper checks robustness across oracle size and even oracle family (DeepSeek-V3 via vocabulary transport), and the qualitative asymmetry holds, but "disagreement with a stronger model" is not the same as "distance from truth." The web verifier grounds the claim labels, which partially compensates.

The causal test is a ceiling, not a method. The authors say this plainly: the intervention needs the oracle in the loop at every token, both to trigger and to define the contraction. It proves the risk channel is causal; it does not give you a deployable decoding rule. The best arm is also selected post hoc per rung over a K \times \rho grid, and sample sizes are small (15–31 verifiable outcomes per cell; OLMo contributes one rung, Llama two low-N rungs). The trigger — "oracle confident about a token the model misses" — also pre-selects the recoverable knows-but-hallucinates regime, so the 35–74% figure is conditioned on trajectories reaching such an onset.

"Reliability scales inversely" oversells slightly. Absolute factuality still improves with scale: Qwen3's FActScore rises 8.7→65.1% across the ladder. The inverse-scaling claim is about within-response relative decay, and "degradation" divides by a start-of-response gap that shrinks with scale. The paper anticipates this — the GLMM slope \beta_{\text{pos}} is significantly positive at 20 of 22 fits, so the positional effect is real, not a ratio artifact — but note the raw slopes in Table B.1 (2.7→22.3) are not monotone, and the 39× headline is a ratio of ratios. The right takeaway is "capability and reliability are distinct scaling axes," not "bigger models are worse."

Scope. English long-form parametric generation only: biographies, open-domain facts, chat-log entities. Reasoning chains, RAG, summarization, and code are explicitly out of scope, and the whole framework requires a setting where \delta has a verifiable referent. Single-author work, three families, ~25 GPU-hours — impressively lean, but the non-Qwen evidence is thin.

Why it matters anyway

If the decomposition holds up, it reframes two active research threads. Detection: the field's black-box detectors are all functionals of p_M, and this paper gives a structural argument — not just an empirical benchmark miss — for why they will underperform on the dominant, scale-growing failure mode. Any serious detector needs an external referent, however cheap. Interpretability: the open question the paper poses is where risk lives in the network — which layers, heads, or features mediate \mathrm{Var}[\delta] — since the gap-only Markov marker shows the signal is recoverable from logits without labels. And for anyone betting that hallucination melts away with scale: the mean gap does, but the mechanism here says the variance doesn't, and autoregression is a machine for converting variance into committed error.

Where to spend your time: Section 2 for the decomposition and four predictions (it's short and self-contained), then Section 4.3 with Appendix F for the intervention — the mean-preserving contraction and its manipulation check are the paper's most original engineering, and the place to probe if you suspect the causal claim is doing less than it says.