The Weather Forecast Was Wrong in a Predictable Way

When a forecasting model keeps getting things wrong, the instinct is usually to replace the model.

Train something larger. Add more features. Change the architecture. Increase compute. Build a better simulator.

A new paper from researchers at Microsoft, ECMWF and several academic institutions starts from a cheaper question:

What if the model’s mistakes are themselves predictable?

That question matters because modern weather forecasting systems are already extraordinarily complicated. Some numerically simulate the atmosphere. Others use deep learning. Some combine both. Improving the underlying model can take enormous amounts of engineering and compute.

The researchers instead put a lightweight machine-learning layer after the forecast.

It learned how the existing system tends to be wrong.

The result was surprisingly large. Applied to ECMWF’s subseasonal AI Forecasting System, their probabilistic bias-correction method doubled its reported average skill across the evaluated variables and lead times in 2025. Applied to already-debiased dynamical forecasts, it improved results across most of the global grid. And a system using the method placed first across the target variables and lead times in ECMWF’s real-time AI Weather Quest competition. (arXiv paper)

The useful lesson is not really about weather.

It is about a mistake we make throughout machine learning:

We spend a lot of time learning the target and surprisingly little time learning the error.

The hard part of weather forecasting begins after two weeks

Tomorrow’s weather and next month’s weather are not the same prediction problem.

Modern forecasting systems can produce useful deterministic forecasts roughly one to two weeks ahead. Beyond that, small uncertainties in the starting state compound as the atmosphere evolves. A tiny difference today can lead to a very different atmospheric state several weeks later.

The period roughly two to six weeks ahead is called the subseasonal range.

It matters enormously.

Farmers can change planting or irrigation plans with several weeks of warning. Grid operators can prepare for unusual demand. Governments can position resources before floods, heat or cold. Water managers can make decisions while there is still time to act.

But this horizon is difficult enough that researchers sometimes refer to it as a predictability desert.

At that point, asking for one exact number becomes less useful.

Instead of:

“The temperature will be 31°C.”

you want something closer to:

“There is a 60% probability temperature will fall in the warmest historical quintile, 25% in the next quintile, and 15% elsewhere.”

That is a probabilistic forecast.

And once a model produces probabilities, being correct is only part of the job. Those probabilities also have to be calibrated.

If your system says “70% chance” one hundred times, events in that group should occur roughly seventy times. A model that routinely says 80% when the true frequency is closer to 55% may rank events reasonably well while still producing bad probabilities.

That is where this paper enters.

Do not relearn the atmosphere. Learn the mistake.

The method is called Probabilistic Bias Correction, or PBC.

It takes the probability distribution produced by an existing forecast system and applies two small corrections in parallel.

The first component, Debias++, looks through historical forecasts and asks:

When this model made forecasts like this, in this place, around this time of year, how was its probability distribution systematically wrong?

Those historical biases are learned by location, date and quantile.

The second component, Persistence++, asks a different question:

What has the weather been doing recently, and what normally happens at this time of year?

It combines the model forecast with lagged observations and climatology.

The two corrected probability distributions are then projected into valid cumulative distributions and averaged.

Conceptually, the pipeline looks like this:

Existing forecast → identify systematic probabilistic error → incorporate recent observed state → produce corrected probability distribution

There is something important hiding in that simplicity.

PBC does not attempt to learn atmospheric physics again.

The expensive model has already done that work.

PBC learns the part that is easier:

the recurring shape of the expensive model’s errors.

The benchmark is harder than a retrospective test

The researchers evaluated temperature, precipitation and mean sea-level pressure on a global 1.5-degree latitude-longitude grid.

Forecasts were evaluated at two genuinely difficult horizons:

• Week 3: days 19–25
• Week 4: days 26–32

Each forecast predicted probabilities over five climatological quintiles. The reference data was ERA5, ECMWF’s global reanalysis dataset.

The main metric was Ranked Probability Skill Score, or RPSS.

You do not need the formula to understand what matters.

An RPSS of zero means the forecast has not improved on the climatological reference. Positive values are better than climatology. Negative values mean you would have been better off relying on climatology.

That last part is easy to overlook.

A sophisticated forecasting model can generate enormous quantities of computation and still perform worse than the boring baseline at long horizons.

That is precisely why a skill score matters.

A model is not useful simply because it predicts.

It has to beat what you could have done without it.

What actually improved?

On the traditional ECMWF dynamical forecasts, PBC was compared with the debiasing procedure already used operationally.

Across the 2016–2024 evaluation, the researchers report that PBC improved skill over the operationally debiased forecast at 98% of evaluated grid cells for precipitation, 91–92% for temperature, and 89–91% for mean sea-level pressure, depending on lead time.

Then they applied it to an AI forecasting system.

ECMWF’s AIFS-SUBS was already a state-of-the-art data-driven subseasonal forecasting system. Across the 2025 evaluation, PBC reportedly doubled its mean RPSS, whose uncorrected average was only about 0.03 across the target variables and lead times.

That last number needs care.

The researchers also tested PBC on PoET, a hybrid system that had already used neural networks to improve ECMWF’s dynamical ensemble.

That is a tougher test because much of the obvious error has already been attacked.

PBC still improved every reported variable and horizon in the 2024 evaluation: precipitation RPSS rose by 68–98% from baselines around 0.03–0.04, pressure by 15–23% from around 0.08–0.10, and temperature by 5–7% from around 0.16–0.19.

The gains shrink as the input gets stronger.

That is exactly what you would expect if the correction is doing something real rather than manufacturing skill from nowhere.

The most convincing experiment happened in real time

Retrospective forecasting studies have a dangerous advantage.

They already live in the future.

All the observations exist. Every dataset has been cleaned. You know which storms occurred. You know which measurements eventually became available. Without extreme care, future information can slip backward into training.

That creates a very impressive model for predicting yesterday.

The authors explicitly attempted to avoid this. For each forecast date, their correction system was retrained using only observations that would have been available at the time the prediction was issued. In the methods, training observations are constrained so they would have been fully observable before the target date at the relevant lead.

Better still, the system was tested outside the paper.

ECMWF’s AI Weather Quest requires teams to submit global probabilistic forecasts in real time for days 19–25 and 26–32. The competition evaluates near-surface temperature, precipitation and mean sea-level pressure, among other evolving targets. (AI Weather Quest)

The Microsoft-led team used a system called MicroDuet, combining PBC with the PoET post-processing model.

For the September–November 2025 period, the team placed first for all three target weather variables at both forecast horizons. It also placed first at both lead times during the December 2025–February 2026 period, and was the only system among 37 teams reported to outperform the six-agency dynamical multi-model ensemble across every variable.

A result from a live forecasting competition is not automatically proof that a method will dominate forever.

But it closes one loophole that matters enormously:

you cannot accidentally train on weather that has not happened yet.

This paper is really about residuals

Suppose you build a demand forecasting model.

Its average error looks acceptable.

Then you plot the residuals and discover something:

• It systematically underpredicts Friday demand.
• It overshoots new stores.
• It is too conservative immediately after holidays.
• It underestimates the upper tail during heat waves.

Those are not random errors anymore.

They are features of a second prediction problem.

The same idea appears everywhere.

A fraud model may be systematically overconfident for new customers. A hospital readmission model may behave differently across facilities. A recommendation system may under-rank new items because they have little interaction history. A churn model may be well calibrated overall but badly calibrated for annual subscribers.

Once the errors have structure, “train a bigger first model” is only one possible response.

Another is:

model the error.

Calibration is often cheaper than capability

This distinction matters commercially.

Replacing the core forecasting architecture may require new infrastructure, retraining, validation, deployment work and monitoring.

A calibration or correction layer can sometimes sit downstream of the model you already trust.

That does not make it trivial.

The correction layer now becomes part of the production system. It needs its own validation, monitoring and retraining policy. And if the underlying model changes, historical bias patterns may change with it.

But the paper makes a useful engineering point: PBC is computationally inexpensive relative to generating the large dynamical or AI ensembles it corrects. The authors explicitly frame it as post-processing rather than a replacement for improvements to the underlying forecast model.

There is an appealing principle here:

Use the expensive model to learn the world. Use the cheap model to learn how the expensive model tends to be wrong.

The leakage detail deserves more attention than the architecture

There is also a subtle reason I trust this result more than I would otherwise.

The correction model uses lagged observations.

That is exactly the kind of feature that becomes dangerous in time-series work.

Imagine producing a 26-day-ahead prediction but training your correction layer with an observation that would only have become known ten days later.

Nothing in your dataframe may look wrong.

Your model will simply seem unusually good.

The paper’s operational setup explicitly restricts observations according to what would have been available at forecast issuance.

That may sound like a mundane implementation detail.

It is not.

A time-series model is only as honest as the timestamp on the information it was allowed to know.

What about extreme weather?

The paper also evaluates extreme events, defining extremes using historical upper or lower percentile thresholds.

For week-three predictions over 2016–2024, PBC improved extreme-event skill across large portions of the evaluated grid. The authors also examine precipitation forecasts around floods catalogued by the Global Disaster Alert and Coordination System and report substantial improvements over the underlying ECMWF forecasts.

That is promising, but it is easy to overstate.

The model is not predicting “a flood will kill 100 people.”

It is improving probabilistic weather forecasts associated with documented flood events.

Those are different tasks.

A flood becomes a disaster through rainfall, soil conditions, drainage, geography, infrastructure, exposure, warnings, evacuation capacity and human decisions.

Better precipitation forecasts are useful.

They are not an end-to-end disaster prediction system.

What would make me wrong?

The strongest interpretation of this paper is that systematic post-processing can recover valuable signal from both traditional and AI weather forecasts.

The weaker interpretation, which I think the evidence supports even more broadly, is simply that forecast errors deserve to be modeled explicitly.

But there are several reasons not to generalize too aggressively.

First, this is an arXiv preprint, currently version 3 dated September 4, 2026. Its methods and results are available for scrutiny, but a preprint is not the same thing as completed peer review.

Second, weather models have unusually rich repeated structure. The same geographic locations, seasonal cycles, variables and lead times recur for decades. That gives bias correction a particularly fertile problem to learn from. Your one-off enterprise prediction problem may not contain errors this stable.

Third, ERA5 is a reanalysis product, not a perfect direct measurement of the atmosphere everywhere. It combines observations with a modeling and assimilation system. Calling it “ground truth” is operationally convenient, but it is still an estimate.

Fourth, relative improvements can sound larger than their absolute effect. Doubling an RPSS that starts around 0.03 is meaningful in a difficult forecasting regime, but the phrase “doubled forecast skill” should never be read as “the forecasts became twice as accurate.”

Finally, success during a real-time competition is strong evidence against retrospective leakage, but no single year of weather contains every regime the system will face.

The paper is impressive.

It is not permission to bolt a correction model onto anything and declare the problem solved.

The practical experiment I would run

If you have a forecasting or probabilistic classification system in production, do not start by changing it.

Freeze it.

Collect a clean set of predictions generated exactly as they would have been in production, with the prediction timestamp preserved.

Then compare those predictions with what eventually happened.

Now ask:

• Does the error depend on the forecast horizon?
• Does it drift by month?
• Does one geography consistently overshoot?
• Are 80% probabilities actually happening 80% of the time?
• Does performance collapse in one customer segment?
• Does the sign of the residual repeat?

If the answer is yes, you have learned something valuable.

Your model may not only be wrong.

It may be wrong in a way you can predict.

And that is a much easier problem.

Key takeaways

1. A model’s residual error can contain learnable structure. PBC improves forecasts by learning systematic probabilistic errors instead of rebuilding the underlying weather model.
2. The reported gains are substantial but need the right denominator. “Doubled skill” refers to RPSS from a modest baseline, not a doubling of ordinary forecast accuracy.
3. The method improved multiple kinds of systems: traditional ECMWF dynamical forecasts, ECMWF’s AI forecasting system and an already-post-processed hybrid model.
4. Real-time evaluation matters. The AI Weather Quest requires forward-looking submissions, making future-data leakage much harder than in a retrospective benchmark.
5. Point-in-time correctness is part of modeling. A correction model using information unavailable at forecast time is not a better model; it is a leaked experiment.
6. Before replacing a model, inspect its mistakes. Calibration, residual modeling or post-processing may be cheaper and more effective than another architecture.

The interesting question is not always:

How do we build a better predictor?

Sometimes it is:

What has our current predictor been trying to tell us through its mistakes?

Primary sources

• Guan et al., “Advancing Subseasonal Forecasting with Machine Learning,” arXiv
  .16238v3, September 4, 2026.
• Microsoft Research: Subseasonal Climate Forecasting
• ECMWF AI Weather Quest