[frazane]

Adds an optional power-spectra diagnostic to the experiment pipeline, so we can compare the effective resolution of model outputs against the truth and spot over-smoothing or spurious small-scale noise that point metrics like RMSE do not reveal.

The diagnostic computes 2D variance spectra (DCT by default, FFT optional) per participant and lead time, averages over initialisation times, and overlays each model against the truth with grid-resolution, effective-resolution and Nyquist reference lines.

What changed

• New src/spectra package: numpy-only spectral core (DCT/FFT with variance-conserving normalisation and radial averaging), ICON native-to-regular regridding with runtime grid detection, plus field extraction and IO helpers.
• Snakemake wiring (rules/spectra.smk): per-participant compute, aggregate over inits, overlay plot, gated behind an experiment target.
• Config: new experiment.spectra section (enable flag, method, lead times, variables, optional init-hour subset) with validation and regenerated JSON schema.
• Unit tests for the spectral core, regridding and IO.

Notes

• Disabled by default; opt in with experiment.spectra.enabled: true.
• ICON native grids are regridded to a regular grid before the transform; the eckit grid file is fetched automatically.

Example output, T_2M at +24h:

[jonasbhend]

@frazane Thanks for pushing this. I have a few comments in particular around organization and generalization that should be easy to address. Good to go after that.

[Copilot]

🟡 Changes recommended

Baseline spectra can use the wrong ensemble member, configuration changes can reuse stale outputs, and frequency binning can discard valid spectral power.

7 open issues

• Include Nyquist modes and the final bin edge · New
• Track spectra settings as compute-rule parameters · New
• Pass the configured baseline member to the spectrum loader · New
• Skip participants missing the current lead time · New
• Prevent duplicate participant labels from overwriting curves · New
• Derive truth lead times from requested spectra leads · New
• Load truth only for requested spectra lead times · New

What changed in this PR

Adds optional power-spectrum diagnostics to EvalML’s experiment pipeline to compare model and truth variability across spatial scales.

Changes:

• Adds spectral computation, ICON regridding, averaging over initialisation times, and overlay plots.
• Integrates the diagnostic into Snakemake with disabled-by-default configuration and schema validation.
• Adds unit tests for spectral calculations, regridding, field extraction, aggregation, and configuration.

File	Description
workflow/​tools/​config.schema.json	Adds the spectra configuration schema.
workflow/​Snakefile	Includes spectra rules and optional experiment targets.
workflow/​scripts/​spectra_plot.py	Adds the plotting command-line entry point.
workflow/​scripts/​spectra_compute.py	Loads source fields and writes per-init spectra.
workflow/​scripts/​spectra_aggregate.py	Adds the aggregation command-line entry point.
workflow/​rules/​verification.smk	Uses the shared experiment configuration for SAL.
workflow/​rules/​spectra.smk	Wires computation, aggregation, and plotting into Snakemake.
tests/​unit/​test_spectra_regrid.py	Tests regridding weights, indexing, and geometry.
tests/​unit/​test_spectra_io.py	Tests field extraction and variable mapping.
tests/​unit/​test_spectra_core.py	Tests spectral behavior and aggregation.
tests/​unit/​test_config.py	Tests spectra defaults and validation.
src/​spectra/​regrid.py	Selects ICON weights and applies sparse regridding.
src/​spectra/​plot.py	Produces overlays with spatial-resolution reference lines.
src/​spectra/​io.py	Maps variables to components and extracts native fields.
src/​spectra/​core.py	Implements DCT/FFT spectra and radial binning.
src/​spectra/​compute.py	Computes source spectra and averages across initialisations.
src/​spectra/​__init__.py	Exposes the spectra package modules.
src/​evalml/​config.py	Defines and validates optional spectra settings.
pyproject.toml	Includes the spectra package in distribution builds.

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

The spectra diagnostic fails for empty reference-time selections and INCA baselines, while missing memory-proportional core reservations risk overcommitting Balfrin nodes.

No open issues 🎉

3 previously missed issues

In code that hasn't changed since last review

Support or reject INCA regular-grid spectra participants

src/​spectra/​compute.py:30

All participants are forced through the native-ICON path here, but load_forecast_data returns INCA fields on a regular (y, x) grid. Those fields fail native_field's 1-D guard, so enabling spectra for an experiment containing an INCA baseline cannot complete. Add a regular-grid path with the appropriate spacing, or reject unsupported participant types during configuration validation.

Reject empty reference-time selections before scheduling

workflow/​rules/​spectra.smk:19

With midnight-only experiment dates and init_hours: [12] (or an empty list), this returns no reference times. The aggregation rules are still scheduled, but spectra_aggregate.py requires at least one positional input, so the diagnostic fails with an argparse error. Reject an empty selection with an actionable configuration error before scheduling the enabled diagnostic.

Reserve memory-proportional cores for Balfrin jobs

workflow/​rules/​spectra.smk:52

These executor jobs bypass the memory-proportional core reservation documented in workflow/rules/common.smk:579–592, allowing concurrent jobs to overcommit Balfrin nodes where per-job memory limits are not enforced. Follow workflow/rules/verification.smk:105: use balfrin_cpus(50_000, actual_cpus=8) here and at lines 86 and 119, reserving 25 cores for each compute job. The plot rule at line 204 likewise needs balfrin_cpus(20_000, actual_cpus=2) to reserve ten cores.

7 resolved since last review

• Pass the configured baseline member to the spectrum loader
• Track spectra settings as compute-rule parameters
• Include Nyquist modes and the final bin edge
• Load truth only for requested spectra lead times
• Derive truth lead times from requested spectra leads
• Prevent duplicate participant labels from overwriting curves
• Skip participants missing the current lead time

_{Review effort: Balanced}

[lclanzi]

I tested the PR locally, and works well. Thanks!

A few minor points to address (suggested by Claude):

1. Wrong reference line in every plot. plot.py:18 labels a line "CERRA (~9 km)" but draws it at 0.10° × 111.2 ≈ 11 km. CERRA's grid is 5.5 km. Every figure shows a misleading reference.
2. Unsupported grids are only caught at runtime. validate_spectra_sources rejects INCA and station truth only. A forecaster whose GRIB isn't on the CH1 or CH2 native grid (a global model, or stretched output holding global and LAM points together) passes validation. It then fails in regrid.weights_path_for_npoints after the jobs are submitted. This could be relevant in case evalml will transition into the "bring your own gribs" concept.

Good to merge once these minor comments are addressed