Issue 35 · Pick 08 AI / ML ✓ read
Missing the Butterfly and Predicting the Past: Features or Bugs of Accurate AI Weather Models?
TL;DR: AI weather models like Pangu-Weather do three things that shouldn't go together: they forecast better than physics-based models, they lack the butterfly effect (a defining property of the real atmosphere), and — as this paper newly shows — they can be trained to predict the past, which naively violates the second law of thermodynamics. Hassanzadeh and colleagues trace all three to one cause: the training data (ERA5 reanalysis at ~25 km, 1–24 hour steps) is heavily coarse-grained, so the models learn a smoothed "effective" atmosphere in which fast small scales influence the large scales on average but never inject their explosive error growth. Across a hierarchy from the two-scale Lorenz 96 system to the official Pangu-Weather models, they show the trade-off is dialed by a single knob: reduce coarse-graining and the models become more physics-like — backcasting fails, a butterfly-like effect appears — but forecast skill collapses (Pangu's skillful horizon drops from 9.3 to 2.0 days going from 24 h to 1 h time steps).
Three puzzles that shouldn't coexist
AI weather prediction models are all built the same way: a neural network learns the map \mathbf{x}(t+\Delta t) = \mathcal{NN}_\theta(\mathbf{x}(t)), where \mathbf{x} is the global 3D atmospheric state, \Delta t is typically 6–24 hours, and training data is ~40 years of ERA5 reanalysis. Rolled out autoregressively, these models now beat operational physics-based forecasts at a tiny fraction of the compute. This was not supposed to happen. Lorenz himself was skeptical of data-driven forecasting, and one classical estimate put the required training data at 10^{30} years.
Puzzle two: Selz and Craig showed in 2023 that these models miss the butterfly effect. This is a technical term, not a metaphor for "chaos." In a multi-scale chaotic system, if you shrink an initial perturbation by a factor of 1000, the ensemble spread does not stay 1000× smaller — errors at the small, fast scales grow so explosively and cascade upscale so quickly that the spread catches up, saturating at nearly the same time regardless of initial amplitude. This is what sets the ~2-week hard limit on weather predictability. Physics-based models (ICON, PlaSim) reproduce it. Every AIWP model tested — deterministic or generative — instead shows slow, amplitude-independent exponential growth. They're chaotic (like Lorenz 63), but not multi-scale chaotic (like the atmosphere).
The paper's opening contribution is puzzle three. Nothing in Eq. (1) is committed to the arrow of time, so the authors train backcasting models: identical architecture, data, and loss, just with the input–output pairs reversed, \mathbf{x}(t-\Delta t) = \mathcal{NN}_\phi(\mathbf{x}(t)). This should be hopeless. Time-reversing a dissipative system flips every Lyapunov exponent's sign: dissipation becomes anti-dissipation, and errors grow fastest at exactly the small scales where forward dissipation was strongest. The backward heat equation is the textbook ill-posed problem; numerically backcasting Lorenz 96 blows up within a fraction of a time unit even at double precision. Yet the ERA5 backcasting Transformer stays skillful (ACC > 0.6 for Z500) out to 6.5 days into the past, versus 9.1 days for its forecasting twin.
So: forecasts that theory said were unattainable, a missing butterfly that theory demands, and skillful backcasting that theory forbids. The paper's claim is that these are three symptoms of one disease — or one feature, depending on what you're using the model for.
The mechanism: severing the error channel while keeping the mean
Here's the core intuition. In the real atmosphere, fast small scales do two distinct things to the large scales: they exert an average forcing (the thing subgrid parameterizations in climate models try to capture), and they act as a relentless injection channel for error, seeding the upscale cascade that produces the butterfly effect.
Coarse-graining the training data — filtering out small spatial scales, subsampling in time, dropping state variables — removes the fast scales as explicit states but leaves their averaged fingerprint on the large-scale evolution. A neural network trained on these coarse pairs therefore learns an implicit subgrid parameterization: it accounts for what the small scales do to the large scales, without ever representing the small scales themselves. The catastrophic error dynamics of those scales simply aren't in the learned system.
The same surgery explains backcasting. The scales that make time reversal explosive are precisely the most dissipative ones — under reversal, dissipation becomes anti-dissipation, strongest where dissipation was strongest. Remove them from the data and the effective system is far closer to reversible (formally, this mimics regularization of the ill-posed backward heat problem by truncating high wavenumbers). Two further points sharpen this: the model is trained only on states on the attractor, so it never learns the explosive off-attractor directions that doom backward numerical integration; and a learned backcast isn't integrating a reversed vector field at all — it's regressing to the most compatible earlier attractor state. No thermodynamics is violated: the learned system is less irreversible, not reversible. The residual irreversibility shows up as the persistent forecast/backcast asymmetry (>1 everywhere), which is largest in the tropics — exactly where diabatic and dissipative processes, i.e. entropy production, are strongest. That geographic pattern is a nice out-of-sample check on the interpretation.
The evidence: turning the coarse-graining knob
The elegant part of the paper is that coarse-graining is not just invoked — it's manipulated, at both ends of a model hierarchy.
Lorenz 96 (Section 2.3), the clean experiment. The two-scale Lorenz 96 system couples 8 slow variables X_i to 256 fast variables Y_{i,j} that evolve ~10× faster and dissipate 10× more strongly — a minimal caricature of resolved circulation plus subgrid convection. The authors train 12 MLPs with identical architecture and identical MSE loss, varying only the training data along two axes: which variables are included (X only vs. X and Y) and the temporal stride (\Delta t = 10\delta t, 5\delta t, 1\delta t, where \delta t is the solver step).
The "real-world regime" corner (X-only, \Delta t = 10\delta t) reproduces every AIWP signature: accurate forecasts, skillful backcasts, amplitude-independent error growth, no butterfly. The "perfect-data regime" corner (X{+}Y, \Delta t = 1\delta t) behaves like physics: forecast skill ~7× worse, forecast/backcast asymmetry ~4× larger, backcasts that blow up, and rapid amplitude-dependent spread growth mimicking the butterfly effect. Everything in between interpolates monotonically. Since architecture and loss are held fixed, this cleanly falsifies the popular hypothesis that the missing butterfly comes from MSE training pulling toward a conditional mean — MSE-trained networks do produce butterflies when the data contains the fast scales. (Conversely, diffusion models with negligible spectral blurring still miss butterflies when trained on coarse data.)
The Lyapunov analysis quantifies the mechanism directly:
Pangu-Weather (Section 2.4), the real-world confirmation. The official Pangu-Weather release includes four independently trained networks at \Delta t = 24, 6, 3, 1 hours — an accidental coarse-graining ladder that nobody had run continuously before (Selz and Craig used hierarchical aggregation, where the 24 h model takes the long steps). Running each model autoregressively on its own, the authors find the small-perturbation ensemble spread transitions from slow, amplitude-independent growth at \Delta t = 24 h to rapid, amplitude-dependent growth at \Delta t = 1 h that approaches physics-based ICON simulations — with upscale-cascading perturbation spectra to match. Meanwhile the skillful lead time collapses from 9.3 to 2.0 days. Same trade-off, same direction, in a production model the authors didn't train or touch.
What this changes
It explains the skill. AIWP forecast accuracy stops being mysterious: the models aren't beating chaos, they're solving an easier problem — the coarse-grained effective dynamics, whose predictability horizon is genuinely longer because the fast error-injection scales are gone. Physics models can't access this regime cheaply: their explicit parameterizations are the dominant source of structural error, and increasing their resolution reintroduces fast error growth. The AI models get the parameterization implicitly and the error suppression for free. This also says the classical 10^{30}-year data-requirement estimates were answering the wrong question (about the wrong effective system, with the wrong learning algorithm — analog forecasting is nearest-neighbor regression, cursed by dimensionality).
It reframes "physics violations" as a design trade-off. For medium-range large-scale forecasting, coarse-graining is a feature — the source of the skill. For anything that depends on correct error growth and irreversibility — calibrated probabilistic forecasts at subseasonal leads, predictability theory, and especially long-term climate emulation — it's a bug. This lands hardest on the current practice of training climate emulators on km-scale simulation output that's been coarsened to ~100 km and daily averages: the paper argues the discarded physics may be exactly why such models struggle with gray-swan extremes and forced responses. The authors' pointed conclusion: the field has obsessed over architectures and losses, but "the content of the training data is the central design decision," and neither hybrid modeling (NeuralGCM) nor foundation-model pretraining (Aurora) fixes it — both still miss the butterfly. They gesture at Mori–Zwanzig-style closures (representing lost scales as memory + stochasticity) as the principled path.
Backcasting as a new instrument. Beyond the physics point, autoregressive backcasting is a genuinely new probe — a measure of learned irreversibility — and possibly a practical tool: a stable, fully nonlinear substitute for adjoint methods in finding precursors of extreme events.
How strong is the causal claim, and what to watch
Quite strong by the standards of this literature, with honest caveats. Strengths: the Lorenz 96 grid is a controlled experiment (12 models, one architecture, one loss, only data varies); the Pangu ladder confirms the trend in an untouched production model; PlaSim rules out data-assimilation artifacts as the cause (its training data involves no assimilation); and the tropics-vs-extratropics asymmetry pattern is a successful side prediction. Hardware artifacts — a real trap here, since GPU TF32 noise can fake butterfly-like growth — are explicitly controlled (strict FP32, zero-perturbation checks, Fig. S10).
Caveats the authors themselves flag: the Pangu experiment only varies temporal coarse-graining; ERA5 is spatially coarse no matter what (produced by a ~31 km model with parameterized subgrid physics), so even the 1 h model can only show "butterfly-like" behavior, and the Lorenz 96 analog (X-only at \Delta t=1\delta t) predicts real-world models stay in the coarse regime. The 1 h Pangu model also generates late-time variance at small scales beyond what ERA5 contains and develops high-latitude instabilities — some of its rapid growth may be pathology rather than learned physics. The claimed monotonic trade-off is robust; the identification of the emergent growth with the true butterfly effect is deliberately not claimed. And the bidirectional forecast+backcast model — the obvious "learn causality" follow-up — showed no benefit in preliminary tests.
Where to spend your time: Section 2.3 and Figures 2–3 (the Lorenz 96 grid) are the intellectual core — the cleanest data-only ablation. Then Section 2.4/Figure 4 for the Pangu ladder, and Supplementary B.1 if you want the beautiful dynamical-systems argument for why backcasting should be impossible (heat equation → Lorenz 63 → Lorenz 96) and why coarse-graining plus on-attractor sampling rescues it.