[frazane]

Verifies forecasters run as ensembles (#287) from all their members. The ensemble mean is scored as the run's deterministic forecast, so existing plots, dashboard and scorecards keep working, and ensemble scores are added to the same verif_*.nc files: fair CRPS, spread and spread-skill ratio, mean single-member RMSE, Brier scores per threshold, and rank histogram and reliability counts (with one plot each per parameter). Deterministic participants show their MAE as CRPS. No config change is needed.

Members are reduced before the region masks are applied, so memory does not grow with members × regions. Wind direction uses a circular ensemble mean; PTYPE gets no ensemble-mean contingency scores, only Brier scores.

Tested on 2 inits × 4 members against REA-L-CH1: CRPS is below the ensemble-mean MAE everywhere, and each verification job peaked at about 5 GB.

Storage: for 21 members over 2025 every 54 h (163 inits), 0/120/6, with the usual fields (2t, 2d, 10u, 10v, sp, msl, tp, z), the inference GRIB takes about 1.9 TB (550 MB per member and init, scaled from the test run), or 1.25 TB without the ifs-* output. The verification files add less than 1 GB. Hourly steps would multiply this by about 6.

Wall time for the same case: about 10 h with the default cap of 16 GPUs (about 115 GPU-hours: 2 min per member, scaled from the test run, so 12–15 min per init with 4 GPUs), with verification running alongside as each init finishes. This assumes no waiting in the queues, which is unrealistic. Set inference_resources: {slurm_partition: short-shared, runtime: 30m} for this case: the default runtime (80 min) sends the jobs to normal-shared, where they can wait for hours.

Changes: verify_ensemble() in src/verification/, the verification metrics, valid mask and aggregation scripts and rules, dashboard, scorecards and metric plots, README, unit tests.

Notes

• Existing deterministic and baseline verification reruns once (src/verification/__init__.py changed); outputs are unchanged.
• Memory for a large case (10 members, 0/120/6) is not measured yet; scorecards with CRPS are only unit-tested.