AIFS single vs AIFS-ENS control — a live deterministic bake-off

The hypothesis: ECMWF's deterministic AIFS is trained on an MSE-type objective, which rewards hedging — it progressively blurs with lead time, sliding toward an ensemble-mean-like state. The AIFS-ENS members are trained on an ensemble (CRPS-type) objective, which punishes blurring — each member, including the control, keeps a realistic kinetic-energy spectrum ten days in. If the control's headline scores match the single's when compared fairly, the control — published only minutes later — may simply be the better deterministic product. Few people seem to have benchmarked this publicly. This page does, on every 00Z run.

Running scores — RMSE, anomaly correlation & activity vs lead
Verified against ERA5 at the valid 00Z on a common 1.5° grid (block means of both models' 0.25° fields — so smoothness is not rewarded by resolution mismatch). The activity ratio is the smoothness police: std(forecast anomaly) / std(ERA5 anomaly). A model can buy RMSE by damping variance; the ratio exposes the purchase. ERA5 lags ~6 days, so points appear about a week after each init and curves firm up as runs accumulate.
⏳ Collecting — every 00Z and 12Z run is archived (00Z since 2026-08-09) and verified against ERA5 as it matures (~6-day lag). First scored points land around 2026-08-16; curves firm up as runs accumulate.
Side-by-side forecast loop — MSLP · 1000–500 thickness · 6-h precipitation
Every 6 hours through the full run, refreshed each 00Z/12Z cycle. Watch the single's precipitation smear into washes at range while the control keeps convective texture — same synoptic systems, different sharpness.
ECMWF AIFS & AIFS-ENS open data (CC BY 4.0) · North America · 61 frames per run

Method

Fair comparison: both models are block-averaged to a common 1.5° grid before scoring — headline scores compare what both fully resolve, so the blurred model is not punished at scales the sharp one carries, and the sharp model is not punished for realistic small-scale displacement (the double penalty). The activity ratio then reports the smoothness directly.

Scores: RMSE and anomaly correlation over the NH extratropics (20–80°N, cosine-weighted); anomalies vs the ERA5 1991–2020 ±7-day day-of-year climatology. Truth is ERA5 at the valid 00Z (ARCO), never either model's own analyses. z500, MSLP, u850; leads 24–240 h; every 00Z run archived since 2026-08-09.

The stakes: if the control matches the single on RMSE/ACC while keeping an activity ratio near 1, the deterministic AIFS is dominated — you would simply use the control (a few minutes later on the feed) and get realistic sharpness for free, plus 50 siblings quantifying its uncertainty.