Support the saturated likelihood test for any selection mapping

[davidwalter2]

Motivation

The saturated likelihood test (--computeSaturatedProjectionTests) was restricted to the Project mapping. The saturated model built its per-bin scale factors by reshaping the parameter vector to the channel shape and broadcasting over the summed axes, which can only express "keep whole axes, in the order of the channel". Mappings with a bin selection, a rebinning or a transposition could therefore not be tested.

The concrete use case is the W simultaneous extended ABCD fit in pt-eta-charge-mt-relIso: pick the signal region (mt bin 2, relIso bin 0), project it to pt, and test the goodness of fit of that spectrum:

-m Select ch0 None 'eta:sum,charge:sum,mt:2,relIso:0' --computeSaturatedProjectionTests

Changes

1. Implement the Project mapping through the Select mapping (behaviour preserving)

A projection is a selection of a channel where all axes that are not kept are summed, which Select already supports through its axis:sum selections. Project now derives from Select and only adds the transposition into the requested axis order, and Normalize is expressed through the normalize argument of Term.select instead of implementing its own projection. Project therefore also supports process selections, bin selections and rebinning when constructed programmatically. The command line interface and the keys under which results are stored are unchanged.

Also drops the unused Term.selection_idxs and a no-op loop in parse_axis_selection that re-used a leaked loop variable.

2. Support the saturated likelihood test for any selection mapping

Mappings that are a selection and a summation of input bins now report, through a new Mapping.output_indices(), the flat index of the output bin each input bin contributes to (-1 for bins that are not used). The saturated model gathers one free parameter per output bin onto the bins contributing to it and leaves all other bins at one, so any Select or Project mapping can be tested. Mappings that are not a plain sum of input bins, such as Normalize or a selection of processes, return None and are skipped, as are mappings without data.

Fixed along the way:

• projecting to a single bin (-m Project <ch>) raised, because np.prod([]) returns a float
• the number of bins per channel was computed as prod(axis.size), which is wrong for channels with flow bins
• for projections with transposed axes the saturated parameters were named in the order of the output but applied in the order of the channel, so the labels did not match the bins. The test statistic was unaffected, being invariant under a permutation of free parameters.

Parameters of the original model that only affect bins made free by the saturated model are degenerate with it, so the covariance of the saturated fit can be singular. This is now a warning instead of an error; the test statistic itself does not need the hessian.

Validation

• The index map reproduces compute_flat exactly in 15 configurations: bin selections, strided slices, partial rebinning where half of the input bins are dropped, fully summed channels and transposed projections.
• On the test tensor all pre-existing saturated tests are unchanged to 10 decimals: Project ch0 x 4.9745507501, Project ch1 b 1.3181413045, Project ch1 b a 47.3605767679. Select ch1 now gets one as well, giving the same 47.3605767679 over the same 50 bins.
• Project and Normalize outputs are identical to the previous implementation in 22 configurations (axes, flow, masked channels, per-process paths, result keys, ndf_reduction). The exception is Normalize at the 1e-17 level, where the division now happens once on the summed result instead of per bin before summing.
• On a 5D pt-eta-charge-mt-relIso toy the signal-region projection gives ndof 6, 2deltaNLL 1.81, p = 93.6%, agreeing with the independent linear chi2 p-value. Injecting a 12% tilt into the signal-region pt spectrum of the data drives it to 2deltaNLL 272 (p = 0%) while an untouched sideband stays at 3.12 (p = 79%).

🤖 Generated with Claude Code

https://claude.ai/code/session_01WTEH6bvho5ziaobKWG153D