Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
82 changes: 55 additions & 27 deletions docs/REFACTOR_PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -3477,11 +3477,15 @@ bumping `smoothing.n_bins` from 151 to 158 and running the suite — and **9 of
25 golden tests fail**, loudly. The golden values live in a committed JSON
file, so any behaviour change breaks them whatever moved it.

The residual gap is narrower than the claim suggests but real: the golden file
records no config, so a *regenerated* reference silently adopts whatever
defaults were current. Pinning a study file, as this plan proposes, is what
would close it. Until then the protection is "the numbers are frozen", not
"the settings are frozen".
**Closed in 0.6.** `specmod.config.using()` holds a configuration across the
ambient reads the pipeline makes — a `ContextVar`, so it does not leak between
threads, and explicit arguments still win — and `tests/golden/reference.toml`
carries every setting the committed numbers were captured under, not an
overlay that would inherit future default changes for whatever it omitted.
`tools/make_golden.py` captures inside the pin and the golden suite compares
inside it, so the property the claim asserted now holds: changing a shipped
default leaves every committed number untouched. Measured by doing it — the
0.6 change of default moved 11 golden tests before the pin and none after.

**Claims that were checked and hold:**

Expand Down Expand Up @@ -3717,33 +3721,57 @@ changes anything shipped.

### Still open

1. **Default estimator.** Multitaper (matching current behaviour) or FFT +
Konno–Ohmachi (more conventional in engineering seismology)? **No longer
blocked** — §6.6's dead `[smoothing] method` is wired up, so the second
option is selectable — and now measured, on the 28 PNR windows with each
station fitted freely:

| Against the shipped multitaper + `log_bins` | Median ΔMw | Per-station range | `fc` ratio |
|---|---|---|---|
| `fft` + `log_bins` | −0.039 | −0.85 to +1.90 | 0.003 to 5.0 |
| `fft` + `konno_ohmachi` | +0.027 | −0.52 to +0.51 | 0.038 to 3.2 |

**The smoothing is what makes the switch defensible, not the estimator.**
FFT with the shipped binner moves a station by up to 1.9 magnitude units,
because `log_bins` reduces the axis without reducing the variance — 151 log
bins over a few hundred Fourier samples is roughly one sample per bin.
Pairing it with a real smoother halves the worst excursion and pulls the
`fc` range in by a third. The median station barely moves either way.

What this event cannot settle: the *event* corner is unconstrained under all
three (stage-one spread 895% to 1042% of the event value), so the ensemble
number is not evidence. Deciding on the ensemble needs an event where it is
constrained.
1. ~~**Default estimator.**~~ **Resolved: FFT + Konno-Ohmachi**, from 0.6. The
measurement below is what decided it, on the 28 PNR channels, each fitted
independently (stage one) on velocity, which is the motion the pipeline fits
by default and from which `llpsp` is the displacement plateau:

| Against the shipped multitaper + `log_bins` | Median ΔMw | Per-station ΔMw | `fc` ratio, median (range) | Median band (decades) | Median points fitted |
|---|---|---|---|---|---|
| `multitaper` + `log_bins` | — | — | — | 1.73 | 113 |
| `fft` + `log_bins` | −0.079 | −0.71 to +0.23 | 0.91 (0.18 to 3.9) | 1.55 | 107 |
| `fft` + `konno_ohmachi` | −0.068 | −0.25 to +0.19 | 0.82 (0.17 to 3.5) | 1.68 | 120 |
| `fft` + `konno_ohmachi`, `fit_bins = true` | −0.073 | −0.25 to +0.15 | 0.90 (0.18 to 2.2) | 1.68 | 120 |

**What each setting reaches.** The estimator sets the spectrum that is
fitted. The smoother sets the band: the signal-to-noise ratio and the gate
are computed on smoothed spectra. With `[fitting] fit_bins = false`, the
default in 0.1.1 and since, the fit and the initial guess both read the
*unsmoothed* amplitudes inside that band, so the smoother reaches `Omega`
only through the band. The two default-path FFT rows therefore fit identical
amplitudes, and the whole difference between them — the per-station range
narrowing from −0.71/+0.23 to −0.25/+0.19 — comes from Konno–Ohmachi
selecting a wider band. Fitting the smoothed spectrum instead
(`fit_bins = true`) changes little further on this event.

Measured on displacement instead, the same comparison gives −0.85/+0.70 for
`fft` + `log_bins` and −0.56/+0.52 for `fft` + `konno_ohmachi`: the same
ordering, a wider spread.

What this event cannot settle, and what the decision therefore does not
rest on: the *event* corner is unconstrained under all three (stage-one
spread 895% to 1042% of the event value), so the ensemble number is not
evidence. On the ensemble, PNR's Mw moves −0.1615 under both constant sets
— a pure Omega shift, since the constants are an offset applied afterwards.
Against the catalogue's 2.9 the two sets move in opposite directions, the
default constants from 0.194 to 0.032 away and §4.7's from 0.160 to 0.321,
so they no longer both sit within 0.25. Recorded, not offered as evidence.

Speed, measured rather than assumed and far too small to decide on: the
transform is 0.209 s against 0.323 s for the 28 windows, and the smoothing
adds 0.14 s (`log_bins`) to 0.83 s (`konno_ohmachi`) on top.

**What had to be built first.** Changing a default moved 11 golden tests,
because §6.6's config pin was a claim with no mechanism: the references were
captured under whatever the defaults were. `config.using()` and
`tests/golden/reference.toml` close that, so the references now hold their
own settings and a default change leaves every committed number untouched.

**The fitting choice is documented where it is made.** Whether the fit
reads the unsmoothed or the smoothed spectrum is a modelling decision a user
should make knowingly, so the tutorial's *What the fit reads* states the
default and fits the event both ways.

2. **Python floor.** 3.11 is proposed. Any users stuck on 3.9/3.10?
3. **History rewrite.** Deleting the 9.9 MB catalog and stripping the notebook
(§5.1) shrinks the *working tree* but leaves both in history, so a fresh clone
Expand Down
78 changes: 69 additions & 9 deletions docs/_builders/specmod_tutorial.py
Original file line number Diff line number Diff line change
Expand Up @@ -237,12 +237,14 @@
```

`spectrum_set_from_streams` transforms both windows, puts the noise on the
signal's frequency axis, bins both, raises the noise to account for what sits
*under* the signal, takes the ratio and selects the band where it passes.
signal's frequency axis, smooths both, raises the noise to account for what
sits *under* the signal, takes the ratio and selects the band where it passes.
The smoothing is Konno–Ohmachi by default (`[smoothing] method`). It is used
for the ratio and the band, not for the fit — see *What the fit reads* below.

Which estimator is used is configuration, not code: `fft`, `welch`,
`multitaper`, `quadratic` and `cwt` all satisfy the same Parseval contract, so
they are interchangeable here. The shipped default is `multitaper`.
they are interchangeable here. The shipped default is `fft`.
""")


Expand All @@ -256,7 +258,7 @@


code("""
# One station in detail: signal, noise, the binned spectra the ratio is
# One station in detail: signal, noise, the smoothed spectra the ratio is
# actually computed on, the selected band (red) and the resolution floor (grey).
from specmod.plotting import plot_pair, plot_set

Expand Down Expand Up @@ -453,9 +455,10 @@ def band_report(spectra: SpectrumSet) -> "pd.DataFrame":
print(f"velocity {spectra[station].signal.unit}")
print(f"displacement {displacement[station].signal.unit}")

# The unbinned signal-to-noise ratio is invariant under this — both spectra are
# divided by the same 2*pi*f — but the *binned* ratio is not, because a bin
# holds a geometric mean. So a few bands do move.
# The unsmoothed signal-to-noise ratio is invariant under this — both spectra
# are divided by the same 2*pi*f — but the smoothed ratio is not: Konno-Ohmachi
# averages amplitudes across a window over which 2*pi*f changes, so some
# bands move.
moved = [i for i in spectra.ids() if spectra[i].band != displacement[i].band]
print(f"bands that moved: {len(moved)} of {len(spectra)}")
""")
Expand All @@ -473,9 +476,10 @@ def band_report(spectra: SpectrumSet) -> "pd.DataFrame":
maximum falls.

`FitSpectra(spectra)` then `fit_spectra()` is the whole thing. The minimiser
(Powell), the `t*` lower bound and whether to fit the binned or unbinned
(Powell), the `t*` lower bound and whether to fit the smoothed or unsmoothed
spectrum all come from `[fitting]` in the configuration, so the defaults are
recorded rather than remembered.
recorded rather than remembered. The last of those is a modelling choice with
consequences; *What the fit reads* below compares the two.
""")


Expand Down Expand Up @@ -551,6 +555,62 @@ def band_report(spectra: SpectrumSet) -> "pd.DataFrame":
""")


md("""
### What the fit reads

The smoothed spectra choose the band — the signal-to-noise ratio and the gate
are computed on them — but **the fit reads the unsmoothed spectrum inside that
band**. Smoothing decides *where* the model is fitted, not *what* it is fitted
to. That is `[fitting] fit_bins = false`, the default, and it is a modelling
choice:

- **Unsmoothed** (the default). Every Fourier sample in the band, each with its
full scatter. The samples are evenly spaced in Hz, so most of them sit in the
top decade of the band and dominate an unweighted misfit;
`[fitting] weight_method = "log"` weights residuals by `1/f` to shift that.
- **Smoothed** (`fit_bins = true`). The spectrum the band was chosen on. Under
Konno–Ohmachi that is the same frequencies with the scatter averaged down,
but neighbouring points share samples, so they are not independent and the
misfit counts the same information more than once. Under `log_bins` it is
one value per occupied bin, evenly spaced in log frequency.

The cell below fits the event both ways.
""")


code("""
smoothed_fits = FitSpectra(spectra, fit_bins=True)
smoothed_fits.fit_spectra()

first = spectra.ids()[0]
raw_fit, smoothed_fit = fits.models[first], smoothed_fits.models[first]
low, high = raw_fit.mod_freq[0], raw_fit.mod_freq[-1]
top_decade = np.mean(raw_fit.mod_freq >= high / 10)
print(
f"{first}: {raw_fit.mod_freq.size} points unsmoothed, "
f"{smoothed_fit.mod_freq.size} smoothed, over {low:.2f}-{high:.2f} Hz"
)
print(f" share of the points in the band's top decade: {top_decade:.0%}")
print(
" median |difference| in the log10 amplitudes the two fits see: "
f"{np.median(np.abs(raw_fit.mod_amp - smoothed_fit.mod_amp)):.3f}"
)

raw_table = fits.table.set_index("id")
smoothed_table = smoothed_fits.table.set_index("id")
d_plateau = (smoothed_table["llpsp"] - raw_table["llpsp"]).abs()
fc_ratio = smoothed_table["fc"] / raw_table["fc"]
print(f"\\nacross {len(raw_table)} channels, smoothed against unsmoothed:")
print(
f" |d log10 plateau| median {d_plateau.median():.3f}, max {d_plateau.max():.3f}"
)
print(
f" fc ratio median {fc_ratio.median():.2f}, "
f"range {fc_ratio.min():.2f} to {fc_ratio.max():.2f}"
)
""")


md("""
### The fit is not unique, and that is not a detail

Expand Down
42 changes: 39 additions & 3 deletions docs/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -191,6 +191,42 @@ found by a live fetch rather than by the suite:
as two independent records of one ground motion. `channel_priorities` and
`location_priorities` rank them, first match wins per station.

## In progress — not yet released

Merged on `main`, and it will name its version when a release goes out.

### The default is FFT with Konno-Ohmachi smoothing

The shipped pair was multitaper with log-binning. Measured on the 28 PNR
channels, each fitted independently on velocity, against that pair:

| | Median ΔMw | Per-station ΔMw | `fc` ratio, median (range) |
|---|---|---|---|
| `fft` + `log_bins` | −0.079 | −0.71 to +0.23 | 0.91 (0.18 to 3.9) |
| `fft` + `konno_ohmachi` | −0.068 | −0.25 to +0.19 | 0.82 (0.17 to 3.5) |

By default the fit reads the unsmoothed spectrum inside the band the smoothed
signal-to-noise ratio selects, so the two FFT rows fit identical amplitudes
and differ only in the band. Konno–Ohmachi's is wider, and that is what
narrows the per-station spread. This event's corner is unconstrained under
every pair, so the event-level number is not evidence either way.

**Every result moves** unless a configuration already pinned both settings.
[Upgrading](upgrading.md#from-05-to-06) has what to pin to keep the old
numbers, and what `[fitting] fit_bins = true` changes: it fits the smoothed
spectrum instead of the unsmoothed one.

### A configuration can be held across an ambient read

`config.using()` pins one configuration for everything inside the block,
including the many pipeline functions that read their settings from call sites
taking no configuration argument. That was the prerequisite for the change
above: the golden references were captured under whatever the defaults were,
so moving a default moved 11 committed numbers, and a deliberate change was
indistinguishable from a regression. They now carry their own settings in
`tests/golden/reference.toml`, and a default change leaves every one of them
untouched.

## Planned

### 1.0 — the API stops moving
Expand Down Expand Up @@ -258,9 +294,9 @@ guess written from documentation cannot substitute for:
Each shipped entry names the version that carries it, so a reader can install
that version and check the claim. Work that is merged but unreleased sits under
*In progress* and gets its version when a release goes out — merged is not
shipped, and this page does not blur the two. **There is no such section right
now**, because nothing is merged and unreleased; that is the state the page
should be in between releases, not a section that went missing.
shipped, and this page does not blur the two. Between releases there is no such
section at all, which is a state the page should be in rather than a section
that went missing.

The move at release time is the step this page got wrong once. The v0.3.0 work
sat under *In progress — not yet released* for a week after v0.3.0 was
Expand Down
70 changes: 66 additions & 4 deletions docs/upgrading.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,77 @@
# Upgrading

Four moves are documented here. Most readers porting code want `0.2` to `0.3`,
Five moves are documented here. Most readers porting code want `0.2` to `0.3`,
which renamed ten functions in `specmod.preprocess` and changed what they do
with the stream they are given. The two moves above it rename nothing: they are
settings that began doing what they always said, so the break is in the numbers
rather than in the call.
with the stream they are given. The three moves above it rename nothing: they
change what a setting does, or which one is the default, so the break is in the
numbers rather than in the call.

`0.x` ships breaking changes in minor bumps without a deprecation cycle,
because shimming an API still being worked out costs more than it protects.
See [Releasing the software](releasing.md#what-a-version-number-means-while-this-is-0x).

## From 0.5 to 0.6

**The default estimator is `fft` and the default smoother is `konno_ohmachi`.**
They were `multitaper` and `log_bins`. Nothing is renamed and no signature
moves; **every result changes** unless your configuration already pinned both.

The two do different jobs. The estimator sets the spectrum that is fitted.
The smoother sets the band the fit is confined to — the signal-to-noise ratio
and the gate are computed on the smoothed spectra — but **by default the fit
reads the unsmoothed spectrum inside that band** (`[fitting] fit_bins = false`),
so the smoother reaches the result only through the band.

On the 28 PNR channels, each fitted independently on velocity, against the old
pair:

| | Median ΔMw | Per-station ΔMw | `fc` ratio, median (range) | Median band (decades) |
|---|---|---|---|---|
| `multitaper` + `log_bins` (0.5) | — | — | — | 1.73 |
| `fft` + `log_bins` | −0.079 | −0.71 to +0.23 | 0.91 (0.18 to 3.9) | 1.55 |
| `fft` + `konno_ohmachi` | −0.068 | −0.25 to +0.19 | 0.82 (0.17 to 3.5) | 1.68 |

The two FFT rows fit identical amplitudes; they differ only in the band.
Konno–Ohmachi gives a wider one — a median 120 points fitted per channel
against 107 — and that is what narrows the per-station spread.

**To keep the old numbers**, pin both:

```toml
[transform]
estimator = "multitaper"

[smoothing]
method = "log_bins"
```

Two things worth knowing:

- **`[fitting] fit_bins = true` fits the smoothed spectrum instead.** Under
Konno–Ohmachi that is the same frequencies with the scatter averaged down;
on PNR the per-station range becomes −0.25 to +0.15. Neighbouring smoothed
points share samples, so they are not independent. The tutorial's *What the
fit reads* compares the two.
- **Published results should pin the version anyway**, but this is the release
that shows why: a re-run under 0.6 of an analysis done under 0.5 will not
reproduce it, and nothing in the call changed to warn you.

**New: `specmod.config.using()`**, a context manager that holds one
configuration across the ambient reads the pipeline makes. This is what lets a
regression test fix the settings its numbers were produced under — most of the
pipeline reads settings from functions that take no configuration argument, so
passing one down was not possible before:

```python
from specmod import config

with config.using("studies/my_study.toml"):
result = spectrum_set_from_streams(signal, noise)
```

The local file and the environment are excluded inside the block, and an
explicit `load_config(project_file=...)` still wins.

## From 0.4 to 0.5

**`max_radius_km` and `min_radius_km` now mean kilometres.** They were converted
Expand Down
4 changes: 3 additions & 1 deletion src/specmod/config/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@

from __future__ import annotations

from .layers import LAYER_NAMES, ResolvedConfig, load_config
from .layers import LAYER_NAMES, ResolvedConfig, load_config, pinned, using
from .provenance import Provenance, config_hash
from .sections import (
AcquireConfig,
Expand Down Expand Up @@ -41,4 +41,6 @@
"WindowsConfig",
"config_hash",
"load_config",
"pinned",
"using",
]
Loading
Loading