Port CVH refit + WMass NanoAOD customizations to CMSSW_15_0_19_patch2 - #46
Draft
davidwalter2 wants to merge 245 commits into
Draft
Port CVH refit + WMass NanoAOD customizations to CMSSW_15_0_19_patch2#46davidwalter2 wants to merge 245 commits into
davidwalter2 wants to merge 245 commits into
Conversation
This reverts commit 2db7789.
… info for wedge modules, restore ability to apply linearized corrections to two track output, and additional disabled debugging
… hits), exclude loopers from two track fit, add muon matching and id info to two track fit
… into WmassNanoProd_15_0_19_patch2_dev2
… closure
Radiative energy-loss export (Geant4ePropagator + G4ePropagationExport)
The CVH fluctuation model had no bremsstrahlung/pair-production term: the
MEAN is carried by the dE/dx table but the FLUCTUATION was absent. To
build that term from Geant4's own physics rather than a parametrization,
the propagator now records per material step a RadiativeStep (effZ, effA,
xg, Etot, p, d/X0, stepCm, dedxRad, dedxBrem, dedxPair, cs) plus the
tabulated brems and pair differential shapes on a 48-point v grid.
ComputeDMicroscopicCrossSection is protected in G4MuBremsstrahlungModel /
G4MuPairProductionModel, so it is reached through a local probe subclass.
Brems and pair are kept SEPARATE because their shapes differ strongly
(brems ~1/v, pair much softer); pair carries ~58% of the radiative mean
here, so normalizing a single combined shape gets the mixture wrong.
useDefaultField (runCvhResClosure.py, runCvhJpsiGenMC.py)
Both drivers forced one of {useOpera3D, useScalarPot3D} and had no way to
consume the plain unlabelled field. For a gen-matched RESOLUTION closure
on standard MC that is wrong: the SIM propagates through the default
field (OAE parametrization inside the tracker), so refitting with the
full 3D grid injects a SIM-vs-refit field-model difference. Measured on
the mu gun: an eta-coherent shift of dp/p ~ 8e-4, 0.182% of unit
variance, entirely unrelated to resolution. It also contradicts the
stated MC practice ("for MC the default field is kept to remain
consistent with the simulation that generated it").
useDefaultField=True sets fieldlabel = "" so every consumer -- geopro,
Geant4ePropagator, cvhMasterESProducer, globalCor and the CPEs -- takes
the default producer, Lorentz drift included. Verified to bite rather
than merely run: identical input events and genParms, fitted q/p changed
by rms 6.0e-4, and the eta-coherent structure fell 0.182% -> 0.006%.
Tested on runCvhResClosure.py (40-task production). The identical option
on runCvhJpsiGenMC.py is UNTESTED -- same three-line pattern, added to
keep the two drivers consistent.
runCleanPropModel.py: explicit species table (mu/pi/K/p, both signs)
replaces the lepton-only charge rule, so the clean-propagation scan can
run hadrons for the Lambda0/kaon studies.
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UC9UW8duX7kaXzV3fwhKqB
globalTag (runCvhResClosure.py). The GT was hardcoded to 106X_mcRun2_asymptotic_v17 -- correct for the B->J/psi+X MC it was written for, but silently wrong for any other input. The private mu-gun simprod used auto:run2_design (131X_mcRun2_design_v3), and because the fit RE-EVALUATES hit positions with its own CPEs, the mismatched SiPixelLorentzAngle / SiPixelTemplate payloads shifted local-x per module and FAKED a pixel hit-quality bias: >3-sigma mean residuals in 97.5% of BPix L1 modules and an 18.9-sigma phi harmonic in the momentum pull. Matching the GT took those to 0.6% and 1.1 sigma respectively. The clusters had been digitised with 2010-2011 pixel templates and re-reconstructed with 2016-UL ones. The help text records this so the next person does not rediscover it. pixelMinSizeY (all three makers + driver). Default 1 = NO cut, a strict no-op. Added purely as an A/B diagnostic after the question of why sizeX==1 is cut but sizeY==1 is not: sizeX==1 in BPix means the EXPECTED Lorentz sharing failed (defective), whereas sizeY==1 is simply what a perpendicular track produces (sizeY ~ 1 + 1.9|cot theta|) -- different in kind. Measured: the cut moves the BPix L1 residual 97.5% -> 96.9% while discarding 12.6% of BPix and 34.6% of FPix hits concentrated at central eta, and even that shift is mostly the mechanical significance loss from the smaller sample. NOT recommended as a selection; the header and help say so, so the knob is not mistaken for one. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UC9UW8duX7kaXzV3fwhKqB
effZ/effA are MASS-FRACTION averages (CalculateEffectiveZandA), but both Moliere parameters are non-linear in Z, and Geant4 evaluates them PER ELEMENT and sums the cross sections (G4WentzelOKandVIxSection::SetupTarget is called with each element's Z). Averaging Z first is therefore an approximation: chi_c^2 ~ Z(Z+1)/A -- effZ(effZ+1)/effA != sum_i w_i Z_i(Z_i+1)/A_i chi_a^2 ~ Z^(2/3)(1.13+3.76(alpha Z/beta)^2)(1+exp(-Z^2/1000)) CalculateMoliereSums computes both properly where the composition is known and exports two doubles per step: zzp1OverA = sum_i massfrac_i Z_i(Z_i+1)/A_i lnScreenW = sum_i w_i ln[screening_i] / sum_i w_i, w_i = massfrac_i Z_i(Z_i+1)/A_i The weighted GEOMETRIC mean is the correct one for chi_a because the exponent depends on it only through ln(chi_a): from S ~ -(chi_c^2 tau^2/2)[ln(2/(tau chi_a)) - gamma + 1/2], summing per element gives ln(chi_a,eff) = sum_i chi_c,i^2 ln(chi_a,i) / sum_i chi_c,i^2. Offline: chi_a^2 = (4.214e-6/p)^2 * exp(lnScreenW). Fields are appended BEFORE stepGroup, so the layout is 0 effZ 1 effA 2 xg 3 pGeV 4 beta 5 thp2 6 dOverX0 7 zzp1OverA 8 lnScreenW 9 stepGroup Offline readers auto-detect stride 8 (legacy) vs 10 and fall back to the effZ approximation for old files, so existing samples remain readable. NTrack does not write msmoliv and is untouched. HONEST SIZE OF THE EFFECT, measured on a real refit (143931 steps): the per-element screening gives 1.9522 against 1.9471 for the effZ evaluation -- 0.3%, not the ~30% I expected. Pure-silicon control: chi_c ratio 1.000000 exactly. So this does NOT explain why the clean-propagation ground truth wants 0.7 of G4's screening factor rather than 1.0; that remains open. Committed because it is exact and removes an approximation, not because it closes anything. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UC9UW8duX7kaXzV3fwhKqB
SIM TRUTH (doSimDecayTruth_). Exports every SimVertex whose parent is the gen-matched SimTrack, with its G4 process subtype (201 decay, 121 hadronic inelastic, 111 hadronic elastic, 2 delta ray) and its flattened daughters (simDauVtx points back into the vertex arrays). Raw sim quantities only -- kink angle, daughter momentum fraction and the decay/interaction classification are done offline, where the parent direction at the vertex can be obtained by helix propagation. Consumes SimVertex through its own token rather than riding on doSim_, because the B->J/psi+X ALCARECO keeps SimTracks and SimVertices but NOT the PSimHit collections, so the full doSim_ path cannot be enabled there. CROSS-SPECIES ARBITRATION (genMatchPdgIds_). Every listed stable charged species competes for the dR match and the winner must be the species this pass fits; otherwise the track is unmatched. Decay studies must open the gen-match pT window (the decay is what changes the pT), and with the window open a soft gen hadron could steal the match to an unrelated J/psi muon track. On the v3 MC that mislabelled ~2/3 of the "decayed kaon" sample and diluted the kink ROC from AUC 0.78 to 0.59. A pT cut is not an alternative: it would preferentially discard the genuine hard decays, i.e. the signal. Also exports genPdgId/genDR (the match-quality handles the offline classification needs) and adds skipBadFiles to the driver -- the production is written by thousands of condor jobs and a few zero-length outputs (job died during copy-out) otherwise abort the whole cmsRun with FileOpenError. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UC9UW8duX7kaXzV3fwhKqB
runCleanPropSim.py ran at the CMSSW defaults DeltaOneStep=1e-3 / DeltaIntersection=1e-4 -- 100x looser than the official UL16 SIM and than resolution/simprod/step1_gensim.py, which this sample is compared against. It matters here in particular because this is the ground truth the whole transport-fluctuation model is validated on, and the chord error of the field integration is a COHERENT trajectory displacement rather than a random one, so it does not average away. Symptom at pT=3, eta=0.30: the gen-matched residual acquires a mean offset <z> growing with layer (0.00 -> 0.086 by layer 12) that flips sign between modules of opposite local-frame orientation -- a global ~100-200 um displacement projected onto local x. At pT=40 the same quantity is 0.000 at every layer, consistent with a chord error scaling with curvature (R = 2.6 m at pT=3 vs 35 m at pT=40). Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UC9UW8duX7kaXzV3fwhKqB
The comment cited a pT=3 residual mean growing with layer and flipping sign between the members of each double-sided pair as evidence of a field-integration chord error. That pattern is produced entirely by the analysis-side acceptance cut: cf_propagation_test kept only rays whose whole (module, entry-face) sequence was modal, which is one-sided in local x because the reference does not cross a module at its centre. Switching to --acceptance perplane on the same events takes the mean from 0.068 to 0.001 sigma and the closure at u=1 from +0.0198 to +0.0014. The tolerances stay at 1e-5/1e-6 -- 1e-3/1e-4 was 100x looser than the official UL16 SIM and than step1_gensim.py, and should never have differed from the sample it is compared against. Only the justification changes. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UC9UW8duX7kaXzV3fwhKqB
…x table
Diagnostic for the mean-vs-mode energy-loss bias. The reference trajectory
subtracts the MEAN loss (ComputeMuonDEDX sums dedxioni + pair + brem, with the
cut argument = e, i.e. unrestricted) while the typical muon loses the MODE, so
the fit adds back energy never lost: charge-even, pure momentum, p_reco > p_gen.
CVH_IONONLY=1 drops the radiative term, separating it from the ionisation one
by their p-dependence. Paired on identical candidates (22515 pairs):
baseline dm/m = +5.524 +- 0.754 e-4
radiative removed dm/m = +5.262 +- 0.706 e-4
paired shift = -0.220 +- 0.003 e-4
p_harm 3.21 -> -0.082 12.97 -> -0.263
7.49 -> -0.142 27.07 -> -0.468
Radiative is real but sub-dominant and grows with energy (1.5% of the bias at
p=3, 14% at p=27), confirming that the LOW-momentum bias is ionisation
mean-vs-mode. Not flat in dp/p, because b itself rises with energy for muons.
Direct proof of the mechanism from pabs per plane: at pT=3 the model predicts
24.993 MeV against a SIM mean of 25.630 (2.5%) and a SIM median of 22.740, so
the typical muon is over-corrected by 2.25 MeV.
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UC9UW8duX7kaXzV3fwhKqB
…e mass bias DIAGNOSTIC ONLY, not a proposed fix. Multiplies the reference dE/dx table so one question can be answered causally: does moving the energy-loss central value actually remove the +1 MeV J/psi mass bias? Everything before this was circumstantial. Set to the measured SIM median/mean over the tracker (0.887), paired on identical candidates: baseline (x1.000) median dm/m = +5.516 +- 0.762 e-4 scaled (x0.887) median dm/m = +1.643 +- 0.842 e-4 paired shift = -3.062 +- 0.019 e-4 70.2% of the bias removed by an 11.3% reduction. The mean-vs-mode mechanism is established causally. Corrects an earlier claim: the lever from dE/dx to fitted momentum is ~1:1, not ~1/3 (that came from comparing the mean-median gap at p=3.14 with the bias in the 3-5 GeV bin). The residual shows why a uniform scale is the WRONG FUNCTIONAL FORM -- it over-corrects at mid-p (+1.36 -> -3.12 e-4 at p_harm 6.1) while still under-correcting at low and high p (+14.47 -> +8.50 at 2.8; +4.47 -> +2.50 at 30.2). It fails within a single sample, with sign changes, so the correct correction is step- and momentum-dependent: a mixture expresses that naturally, a constant cannot. 30% of the bias remains unexplained. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UC9UW8duX7kaXzV3fwhKqB
…t bias Q's MS is Rossi/Xs (15 MeV, no log) where Highland is 13.6*(1+0.038 ln(x/X0)). MS is symmetric so this cannot bias the reference trajectory, but it changes the hit-vs-kink weighting and with material rising toward large r could shift the effective lever arm. Scanned +-40%: MS x0.7 charge-even (p>3 GeV) = +2.198 +- 0.727 e-4 MS x1.0 = +2.452 +- 0.742 e-4 MS x1.4 = +2.581 +- 0.814 e-4 Differences -0.25 +- 1.04 and +0.13 +- 1.10 e-4, consistent with zero. The real MS error (~10-20%) would contribute <=0.15e-4. MS weighting is EXCLUDED as the cause of the flat +1.66e-4 component. Note on method: 97% of rows change under this knob with median |dm| = 11.3 MeV per track, so the PAIRED difference is ~0 by construction while the sample medians shift. Unpaired is the correct comparison here -- the opposite of the sim-hit ablation, where pairing was essential. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UC9UW8duX7kaXzV3fwhKqB
…luded Scales the thick-scatterer displacement term S1 (and correlation S3, by sqrt(f) so rho = sqrt(3)/2 and positive-definiteness are preserved) relative to the angle term S2. CVH_MS_SCALE multiplies DD and leaves S1/S2 fixed, so it cannot probe a wrong displacement/angle RATIO -- the class of defect found and fixed on the CF side (missing within-step lateral displacement, 8.8% of the position variance). S1 x0.5 +2.352 +- 0.783 e-4 S1 x1.0 +2.395 +- 0.717 e-4 S1 x2.0 +2.404 +- 0.712 e-4 A 4x change moves the bias by +0.053e-4 where -1.66 is needed: EXCLUDED. Also retracts two leads built on 1.5-1.7 sigma structure: "bias shared with the raw tracks points upstream" and "bias grows with layer count" (measured 1.5 sigma; |eta| is consistent with flat at chi2/ndf 6.6/5). The flat component itself is real at 4.6 sigma. Convergence excluded too: dp/p is +2.170/+2.185/+2.185 e-4 across niter 2-10 and flat over 15 orders of magnitude in edmvalref. RESOLUTION (see NOTES 2026-08-10): the flat term is the AN's documented magnetic-field-like residual of the BARE refit. Every test today omitted the `corFile` global corrections that the analysis always applies, and which -- with the residual corrections derived data-relative-to-simulation -- absorb it. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UC9UW8duX7kaXzV3fwhKqB
…y; tested
Makes dolocalupdate switchable (default false = baseline, bit-identical).
false is the AN scheme: only the reference state updates between GN iterations
and each track is re-propagated UNSCATTERED, so material and Jacobians are
evaluated on a trajectory the particle is increasingly not on. true is the
standard GBL reference iteration: per-layer states are updated from the fitted
kinks and carried forward, the next propagation starts from the deformed path
and re-samples material along it, Hp is recomputed at the updated state, and
Q/dQMS/dQI move to local coordinates. dx0 then holds only the residual w.r.t.
the propagated state, so the kink prior is not double-counted.
RESULT (12 tasks, paired on identical events):
CONVERGENCE IMPROVES 133x -- edmvalref median 2.54e-8 -> 1.90e-10, and
tracks hitting the iteration cap fall 1.66% -> 0.17%. I predicted
oscillation/divergence; the opposite happened, which in hindsight is
expected: linearising about the trajectory the particle is actually on is
better conditioned.
THE BIAS IS UNCHANGED. 90.15% of tracks change with median |shift| 0.60e-4,
but symmetrically: paired median shift +0.001 +- 0.001 e-4. The low-p spike
stays at +15.0 -> +14.4e-4. Josh's suggested mechanism ("iterate on reference
state only and always propagate unscattered") is REFUTED as the cause of the
low-pT breakdown.
COSTS are minor: +17% propagation calls, failures 0.0015% -> 0.0035% with 5
new offsurface (deformed path misses the target sensor -- a surface-search
tolerance issue).
UNTESTED RISK: the exported gradients dx/da are evaluated along the
trajectory, which now depends on the fitted kinks. No momentum closure would
reveal a problem there; it would surface only in the alignment / B-field
calibration.
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UC9UW8duX7kaXzV3fwhKqB
…t fake study Lets the single-track maker ask which tracks are reconstructed as muons on inputs where the muon's own track references do not resolve. On the TkAlJpsiX ALCARECO every reco::Muon track reference is non-null but DANGLING -- innerTrack 96/96, muonBestTrack 96/96, outerTrack 85/85 and globalTrack 83/83 point into generalTracks, which the ALCARECO drops, so the legacy doMuons path (which reads muon.bestTrack()->algo()) throws ProductNotFound. The shipped ALCARECOTkAlJpsiXTrackToMuon association is keyed on the unselected track collection, so querying it with a ref from the p>3 GeV TrackSelector copy throws InvalidReference instead. doMuonTrackAssoc_ therefore matches on the muon's OWN four-momentum (charge, |dpt| < 5%, dR < 0.01) and reads only values stored directly on the muon: selector bits, type bits, p4. Measured on the v3 B->J/psi+X MC: real gen muons 99.15% matched / 97.6% Medium / 64.2% Tight, gen kaons 0.78% / 0.45% / 0.19%. Also in the driver: fitAs= decouples the fit mass hypothesis from the gen-matched species, so a hadron can be fitted as a muon the way the real analysis would fit it, and skipBadFiles guards against the handful of zero-length files in the v3 production. BUG FIX: muonIsPF was missing from the per-track reset, so it carried over from the previous track in the event -- true for ~47% of hadron tracks that were never matched to a muon at all. Any earlier study reading muonIsPF from this producer is suspect. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BJNXSeCjnP72EfEBiz26Wu
…the tree `git status` in this area was showing a 34 MB `globalcor_0.root`, two 0-byte Geant4e debug logs, the per-job `runs/` scratch area, and two hand-made backups of the toy tracker XML -- i.e. 34 MB of run product against ~150 kB of new source, which is exactly the ratio that makes `git add -A` unsafe. Two of the entries are not merely large but WRONG to commit: `Analysis/HitAnalyzer/data/tracker.xml` and `test/toyPlanes_pt3.py` are regenerated by `test/gen_toy_config.py` on every geometry change and carry a `do not hand edit` banner; the copy currently on disk is whatever the last scan left behind (layer thickness 0.0 cm). Committing it would freeze a stale scratch state as repo content while the generator, which IS committed, produces something else. Scoped to the directories this work pollutes rather than applied tree-wide: upstream CMSSW legitimately tracks .root and .log files in Alignment, CalibTracker, DQM and elsewhere, and a blanket `*.root` would hide a future one. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UC9UW8duX7kaXzV3fwhKqB
…CVH switches TWO THINGS LIVE HERE, and they are together because the second is the reason the first has to be careful. 1. `cvhcgf::inverseFisher` -- the INVERSE FISHER INFORMATION of a leg's q/p process-noise block, by exact FFT inversion of the block characteristic function. The fit currently weights each block by an alpha = 0.999 TRUNCATED Urban variance; that truncation is a convention (the 1/E^2 delta-ray channel has no second moment without one) and it is not additive under step subdivision. 1/I needs no truncation. Exact inversion rather than a saddlepoint is not a preference: the saddlepoint Fisher route was measured at 5-38 % wrong and unpredictably so -- adding a channel carrying 5e-11 of the variance moved it by 33 % -- because I is dominated by the delta-ray hard edge, which exponential tilting places in the wrong location (Documents/Resolution/NOTES_XXII_msrad.md s4). The exact score psi(z) comes out of the same two transforms and costs nothing extra; the saddlepoint form was 0.52-1.44x wrong over r in [-3, +4]. Prototype, off by default (CVH_CGF_QOP). 2. The environment switches for the four energy-loss corrections, each with EXACTLY ONE reader, so that the C++ half and the offline half of a shared switch cannot be set independently. `cvhcgf::envFlag` makes them TRI-STATE -- unset means the default, `=0` means off -- rather than presence-only. That is what lets a correction be default-ON and still be turned off, which is not a nicety: attribution needs the factorial comparison (all four off against all four on), and a switch that cannot be set to off cannot supply it. CVH_IONI_EXACTDELTA, CVH_IONI_KOKOULIN, CVH_REF_CHARGEAWARE and CVH_REF_SPECIESDEDX default ON (Documents/Resolution/NOTES_DEFAULTON.md). CVH_IONONLY and CVH_IONI_URBAN2021 stay off; they are diagnostics. `blockExponent`/`blockKappa2` now REFUSE a regime-2/3 record instead of misreading it. Those functions branch on `regime == 0` and treat everything else as regime 1, i.e. they read the record's `a3` slot as a delta-ray collision count; in regime 2/3 (exact delta) that slot holds `xi`, an energy -- ~0.072 MeV where the count is ~7.6 -- so the delta channel came out wrong by ~1e-5 and, being a WEIGHT, did not fail. That was tolerable while CVH_IONI_EXACTDELTA was default-off and is not tolerable now that it is on: anyone enabling the CGF prototype would inherit the wrong answer without doing anything. Measured after the change: regimes 0 and 1 return as before, regimes 2 and 3 throw from both functions. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UC9UW8duX7kaXzV3fwhKqB
Two independent defects of the CVH reference trajectory's mean energy loss, both measured against Geant4's own unrestricted dE/dx for the particle in question rather than against a model of it. CVH_REF_CHARGEAWARE. `Initialisation` built ONE muon table, from G4MuonPlus, and ONE hadron table, from G4Proton, then handed the muon table to both signs and scaled the hadron table by q*q. `muonMinus` was a member, assigned in the constructor and never used; there was no antiproton at all. So the reference was not charge-blind at the charge AVERAGE -- it was pinned to the POSITIVE particle and every negative track carried the whole error. What it dropped is the charge-odd half of G4EmCorrections::HighOrderCorrections = 2*(Barkas + Bloch) + Mott: 0.309-0.314 % of the restricted dE/dx at the toy's kinematics, i.e. dp/p = 1.4-1.5e-5 between conjugates, at the 1e-5 Z-mass target. The fix calls the EXISTING ComputeMuonDEDX / ComputeProtonDEDX with G4MuonMinus and G4AntiProton and dispatches on the sign of the track's own G4ParticleDefinition charge. No term of Geant4's expansion is transcribed anywhere, which matters because Barkas and Mott run opposite ways in beta and exchange dominance below beta*gamma ~ 1 -- the V0 regime this calibration wants next. Measured, conjugate difference of ComputeDEDX in the toy material: 3.1905e-3 (mu), 3.1918e-3 (pi), 3.2288e-3 (K), 3.3318e-3 (p) against Geant4's own 3.190/3.192/3.229/3.332e-3. CVH_REF_SPECIESDEDX. Every hadron except the proton is served from the proton table at e = ekin * m_p/m. That scaling preserves beta*gamma and hence every term of Bethe-Bloch except Tmax, whose recoil denominator carries the PROJECTILE mass. Since dE = xi[ln(2 m_e bg^2 Tmax/I^2) - 2beta^2 - delta], d(dE)/d ln Tmax = xi exactly, and the whole error is xi * ln(Tmax_species/Tmax_table): +5.24e-3 for the pion, +2.86e-4 for the kaon, identically zero for the proton and for the muon. Nothing here is a model Geant4 could revise -- it is the exact derivative of the leading logarithm and exact two-body recoil kinematics. ALL THREE of ComputeDEDX / ComputeRange / ComputeEnergy switch together in both cases, and a dedicated driver shows that was not optional: `EnergyAfterStep` picks between `step * ComputeDEDX` and `ComputeEnergy(range - step)` on linLossLimit, and pairing a corrected dE/dx branch with an uncorrected range branch leaves the two disagreeing by 4.8e-3 for a pion, 11x the intrinsic O(step) difference between them. The range carries the correction as the EXACT perturbation integral D(E) = INT d/(u(u+d)) dE', not an expansion. The exact nulls are by construction, not by size: the reference mass is the TABLE PARTICLE's own PDG mass (938.272013 MeV) and not CLHEP::proton_mass_c2 (938.27208816), so both recoil denominators are formed from identical bits and log(1.0) is +0.0. Using the constant would have given the proton a 1.4e-11 relative correction. Both were DEFAULT-OFF when they were written and are default-ON as of 2026-08-16; `CVH_REF_CHARGEAWARE=0` / `CVH_REF_SPECIESDEDX=0` restore the previous behaviour bit-identically (27/27 branches, all eight species). A missing or short table throws rather than returning ComputeValue's silent 0.0, which would give a reference that integrates the CORRECTION ALONE -- a small, finite, plausible-looking stopping power the fit would converge on. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UC9UW8duX7kaXzV3fwhKqB
…anLoss
Three changes to the ionization fluctuation model, two of them physics in the
VARIANCE and one a consistency fix in the MEAN.
CVH_IONI_EXACTDELTA (default ON). The Urban `a3` channel samples a pure
1/E^2 knock-on spectrum. The real one is the exact cross section, whose shape
factor is 1 - beta^2 T/Tmax (+ T^2/2E^2 for spin 1/2), taken from
G4ParticleDefinition::GetPDGSpin() rather than a PDG-id table so a pion and a
proton get the right branch by construction. Regimes 2 and 3 are the spin-0
and spin-1/2 forms. The record's `a3` slot then holds `xi` -- an ENERGY,
2 pi re^2 me c^2 n_e z^2 L/beta^2 -- and NOT a collision count; the stride and
field list are unchanged (beta2 and etot are appended AFTER cs), so a consumer
that switches on `regime` needs no new columns and one that does not must
refuse regime 2 rather than read `a3`. That refusal is now enforced in
cvhcgf::blockExponent.
CVH_IONI_KOKOULIN (default ON). G4MuBetheBlochModel multiplies the knock-on
cross section by Kokoulin's radiative correction
f_K(T) = 1 + (alpha/2pi) a1 (a3 - a1), reaching +6 % (pT = 3) to +9.4 %
(pT = 40) at the hard end. The MEAN already carries it -- Geant4's own
ComputeDEDXPerVolume matches the Kokoulin-weighted rate to 5 significant
digits -- so the dE/dx table, and the reference trajectory, were already
right. What did not carry it is the fluctuation, i.e. the variance the fit
consumes as Q(0,0). Only `gsig2` moves: a1, e1, a2, e2, a3, e0r, tmaxr,
scaling, beta2 and etot are bit-identical with it on. Geant4 applies this in
G4MuBetheBlochModel and nowhere else, so the |PDG| == 13 and ekin > 1 GeV
conditions are MIRRORED, not approximated -- a 3 GeV kaon's regime-3 record is
untouched by design. Verified against scipy Gauss-Kronrod at 3e-10.
THE MEAN FIX. `SetParticleAndCharge` selected fDedxMuon / fDedxProton
UNCONDITIONALLY, with no isNegative dispatch, while
G4EnergyLossForExtrapolatorForCVH::ComputeDEDX -- which builds the REFERENCE
trajectory -- did dispatch once CVH_REF_CHARGEAWARE existed. So with that
switch on, the reference's mean and this class's `meanLoss = length * dedx`
disagreed by exactly the charge-odd term on every negative track. That is not
a small error for being second order in the variance: `meanLoss` sets the
Urban channel weights a1/a2/a3, sets the excitation rescale that pins the
block to the mean, and IS the returned loss in the Gaussian regime. A noise
model centred on a different mean than the trajectory it is the noise of is a
different error, not a smaller one.
Measured with a driver that calls both paths at the same kinematics
(calibration_studies/resolution/meanloss_g4driver.cc; the fluctuation's own
meanLoss is obtained by calling SampleFluctuations2 at a step short enough to
take its `meanLoss < minLoss` early return, so nothing is re-implemented).
dedx(positive) - dedx(negative), MeV/mm, toy material, p = 3136 MeV/c:
pair reference fluctuation before this commit
mu+/mu- 0.0031904577263329 0.0031904577263331 0.0 (pinned to mu+)
pi+/pi- 0.0031918370785240 0.0031918370785238 0.0
K+/K- 0.0032288132004181 0.0032288132004181 0.0
p/pbar 0.0033317941256663 0.0033317941256661 0.0
i.e. the two now agree on the charge-odd part to 2.2e-16 (1 ulp) where they
disagreed by the whole 3.19e-3 to 3.33e-3 before. The pre-fix column was
measured by reverting the dispatch and rebuilding, not inferred from the
source. `speciesRefMass` follows the table actually selected (G4AntiProton
for a negative), so the antiproton keeps its exact null under both reference
switches together. The ~1e-3 absolute offset between the two columns is the
pre-existing difference between the two classes' table sets (100 TeV vs
10 TeV grid; the extrapolator's carries the radiative mean, this one never
does) and is unchanged.
Diagnosed and left open in NOTES_SPECIESDEDX s2.1/s8; closed in
Documents/Resolution/NOTES_DEFAULTON.md s1.
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UC9UW8duX7kaXzV3fwhKqB
…lock weight The propagator now logs, per leg and per step, everything the offline model needs to rebuild the block's characteristic function without re-running Geant4: the Urban ionization record, the per-element Moliere sums, the radiative spectrum, the 5x5 transport Jacobian, and the step's dE -> d(q/p) map cs = E/p^3. All of it is behind setIoniStepLogging and cleared per propagate, so a job that does not ask for it pays nothing. The transport weight is folded into the RESIDUAL coordinate (`gs`) and never into the CGF parameters themselves. That is not a matter of taste: it is what makes d eta_b/d a = 0 exactly for the alignment and B-field global parameters, and therefore what keeps the existing 50-mode scalar-potential production valid (NOTES_EXPORTS s4.3). The charge factor in `gs` is likewise not cosmetic. cs maps dE -> d(q/p) for a POSITIVE charge; for q = -1 the map flips and with it the sign of the ionization skew, i.e. the mode-vs-mean displacement. It cancels in 1/I -- the Fisher information of a reflected density is the same -- so omitting it would be invisible in the WEIGHT and would silently reverse the re-centring the next stage adds. CVH_CGF_QOP=2 computes 1/I and prints it with the returned noise matrices untouched (physics output still bit-identical, only stdout differs); =1 substitutes Qcurv(0,0) <- 1/I. Off by default. The CVH_CGF_QOP + CVH_IONI_EXACTDELTA combination now THROWS rather than warning. The block CGF has no regime-2/3 channel and reads the record's `a3` slot as a collision count where it holds `xi`, an energy; the result is wrong by ~1e-5 in the delta channel and, being a weight, does not fail. A one-shot warning was defensible while both switches were default-off. It stopped being defensible when CVH_IONI_EXACTDELTA went default-ON, because from then on the wrong answer is what you get for doing nothing. Measured both ways on 20 events of Run2016G TkAlJpsiMuMu: with CVH_CGF_QOP=2 and EXACTDELTA at its default the job aborts with the diagnostic and the remedy in the message; with CVH_IONI_EXACTDELTA=0 it runs to completion and prints its CGF lines (Qcgf/Qnom = 0.055-0.17 over the leg). The remedy exists only because the switch stayed operable in both directions. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UC9UW8duX7kaXzV3fwhKqB
… configurable step limit WHERE ALONG THE TRAJECTORY does the fit actually take its energy-loss information from? Two probes, both pure MEAN perturbations, because moving the variance would move the fit weights themselves and the measured response would no longer be d(p_fit)/d(applied loss) at fixed weights. CVH_ELOSS_CYL_R / _Z / _EPS adds `eps` to the log-scale mean-loss offset for every step inside a cylinder. A nested family gives the cumulative influence profile Lambda(u) from the p_fit response and M(u) from the dEref response, whose derivative w(u) = dLambda/dM is the influence weight as a function of position. It lives in G4ErrorEnergyLossForCVH::AlongStepDoIt rather than in MaterialGroupModel because that hook is the mean-loss path only, so the probe is mean-only by construction with nothing to switch off. Unlike the material-group probe it is unambiguous to map onto a cleanprop plane, which carries (refglobr, refglobz) but no logical-volume name. CVH_MATGROUP_MEANONLY withholds the material-group injection from the step's MS and ionization VARIANCE factor, leaving it a pure coherent shift of the group's mean loss -- the same requirement, for the group-based probe. Both are also the NULL-PERTURBATION FLOOR the correction commits are read against: `CVH_ELOSS_CYL_EPS=1e-9` multiplies every step's reference mean loss by exp(1e-9), which is physically null and travels the same code path, so "the exported Jacobians moved" can be separated from "the fit reordered a flat direction". Without it the response sizes in NOTES_CHARGEODD s7 and NOTES_SPECIESDEDX s7 would be unreadable. GeantPropagatorESProducer gains a StepLengthLimit parameter; the default 10.0 mm reproduces the previous hard-coded value exactly, so existing configs are unaffected. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UC9UW8duX7kaXzV3fwhKqB
… physics blocks The exporters follow the fluctuation model's record. `ioniurbanv` has a stride of 11 when CVH_IONI_EXACTDELTA is off -- byte-identical to the historical record -- and 13 when it is on, with `beta2` and `etot` appended AFTER `cs` so that every existing column index is unchanged and an old consumer keeps working. The two extra columns are not padding. beta^2 and E cannot be recovered from `tmaxr` without the particle mass, and getting them wrong for a slow hadron is not a small error: beta^2 is 0.976 for a 3 GeV kaon against 0.99999 for a 3 GeV muon, and it multiplies the whole spin-suppression term of the exact knock-on cross section. They are PHYSICAL values and carry no `scaling` factor, unlike every energy in the record. The mass is then recoverable exactly rather than by a new column: E * sqrt(1 - beta^2) = (gamma m)(1/gamma) = m is an identity. That is what lets the offline model apply Geant4's muon-only Kokoulin factor to muons and not to kaons, which the record could not previously distinguish -- a mistake that made K- look 4x BETTER (0.00311 -> 0.00071) because its u-shape mimics the charge-odd mean-loss bias, i.e. wrong in the direction that looks like success. Also here: the resolution-block export path (doRes) and the factored-Hessian branch used by the 50-mode scalar-potential production. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UC9UW8duX7kaXzV3fwhKqB
A closure test on the real tracker cannot separate "the propagation model is wrong" from "the geometry description is wrong", and it cannot vary the step structure at all. This is the minimal geometry that can: concentric shells at the real tracker's 14 barrel radii, one material, and a generator that scans the layer thickness T at FIXED total material by taking rho ~ 1/T. Total dE and X0 depend only on rho*T*N, so that changes the STEP STRUCTURE while leaving the integrated material alone -- which is the thing under test, since the mean/median ratio is known to be step-size dependent (0.747 per step against 0.887 summed). ToyStateNtuplizer replaces PSimHits with a stepping action that records the true 5D state at chosen radii. That is what cuts the test loose from tracker DetIds: PSimHits only exist where the tracker sensitive detector runs, which is what forced the real geometry in the first place. The model side takes EXPLICIT PLANES rather than DetIds for the same reason -- the tangent planes at the reference crossings, whose azimuth was validated against the sim at 0.9043 predicted vs 0.906 measured. `tracker.xml` and `toyPlanes_pt3.py` are NOT committed: they are regenerated by gen_toy_config.py and carry a `do not hand edit` banner. The generator is the source; the output is a scan point. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UC9UW8duX7kaXzV3fwhKqB
…sure campaigns cmsswlock.sh exists because two cmsRun jobs died during the profiling work with what looked like physics crashes -- one SIGSEGV inside RunManagerMTWorker::beginRun, one SIGBUS mid-event-loop. Neither was: biglib/el9_amd64_gcc12/pluginSimulation.so was rewritten at 10:29:22 and again at 10:33:03 by a concurrent `scram b` in the same shared area, and a process whose mmap'd shared object is replaced underneath it takes SIGBUS on the next page fault into it. That presents as a mysterious segfault with a corrupted stack, which is exactly how much time it costs to diagnose. `run` takes it shared and `build` exclusive, so a build waits for the running jobs and a job started once a build holds it waits for the build. Measured: two `run` holders overlapped and a `build` requested while they held it started 4 ms after the last finished. The one gap is flock's -- a `run` requested while a `build` is merely QUEUED is still granted immediately, so a continuous stream of jobs can starve a build. That delays a build, it does not corrupt anything. The drivers gain the options the closure campaigns need: tightened Geant4 field-integration tolerances in the SIM (the CMSSW defaults are 100x loose and move the widths by 12 %), explicit target planes, and the single-track V0 and gen-MC paths used for the hadron species. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UC9UW8duX7kaXzV3fwhKqB
Two SimWatchers that make the simulation's own answer available for comparison with the propagation model, rather than inferring it. PrimaryLossCensusWatcher records, per primary and per step, the energy the PRIMARY actually lost. That is not what a hit-based estimate gives: a delta ray above the production cut leaves as a secondary and its energy is accounted somewhere else entirely. The census is what makes "the model's hard-delta mean is short by a few per cent" a measurable statement instead of an inference from a closure residual. ProcessActivationWatcher reports which processes were actually ACTIVE in a run. The per-species campaigns inactivate nuclear and decay processes to isolate ionization and multiple scattering, and "the process was inactivated" has to be a log line from Geant4 rather than a claim about a config file -- an inactivation that silently failed would look exactly like a physics result. It also caught the opposite case: it is how the 20 000-event control that showed the LD_PRELOAD Barkas shim to be inert inside cmsRun was verified (md5-identical census files across four repeats). Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UC9UW8duX7kaXzV3fwhKqB
The 2026-08-16 flip (NOTES_DEFAULTON) turned CVH_IONI_EXACTDELTA,
CVH_IONI_KOKOULIN, CVH_REF_CHARGEAWARE and CVH_REF_SPECIESDEDX default-ON.
The attribution gate that note's own section 6 names is unchanged: the global
fit that consumes the exported Jacobians has not been run, and
CVH_REF_CHARGEAWARE is the only charge-ODD member -- degenerate in one fit with
the calibration's M, which is read as physical misalignment. The four stay
opt-in until that fit has been run BOTH WAYS, and closure studies enable them
explicitly.
Reverted:
* the four defaults, true -> false.
KEPT, because they are correctness fixes and not consequences of the default:
* the tri-state cvhcgf::envFlag parse. A presence-only reader cannot express
"off" once a default is on, and a correction that cannot be turned off
cannot be attributed -- so the reader that makes a future re-flip a
one-word change stays.
* the refuseExactDelta throw in blockExponent/blockKappa2 and the earlier
cms::Exception at the block-fill site in Geant4ePropagator. regime >= 2
puts xi (an energy) in the slot the block CGF reads as a collision count;
that is a silent wrong answer, not a failure, and a warning is not a guard
against it at ANY default -- the two switches can still be set together by
hand, and the documented way to run a closure now sets all four.
* the charge-aware meanLoss dispatch in
G4UniversalFluctuationForExtrapolator::SetParticleAndCharge, which made the
noise model's mean agree with the reference trajectory's to 1 ulp where the
two had disagreed by the whole charge-odd term.
Verified after the revert, not asserted:
* meanloss_g4driver with all four unset: mu- == mu+ and pi+ reads
1.7050736911794582, the UNCORRECTED value, rather than the SPECIESDEDX
1.6961937706774735 -- the exact inverse of NOTES_DEFAULTON s4.2.
* chargeodd.py meanloss still PASSes at 2.22e-16 with CVH_REF_CHARGEAWARE=1,
reproducing NOTES_DEFAULTON s1.3 digit for digit, so the retained fix is
live where it is asked for and exactly zero where it is not.
* qvalid.py bitid --defaults, pt3 / pt40 / real: an UNPINNED export is now
27/27 BIT-IDENTICAL to `off`, and 20/27 branches differ from all-four-on.
Both rows inverted relative to 2026-08-16, and cmd_bitid now spells the
expectation out in the row label so the polarity cannot silently rot.
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UC9UW8duX7kaXzV3fwhKqB
A hadron's reference subtracted NO radiative mean and its model carried no
radiative fluctuation, while the simulation ran hBrems/hPairProd.
Two independent causes, both now addressed under one switch, CVH_REF_HADRAD
(single reader cvhcgf::referenceHasHadronRadiative, default OFF):
* ComputeProtonDEDX builds only G4BetheBlochModel, so the proton dE/dx table
-- which serves every hadron, looked up at e = ekin*m_p/m -- has no
radiative term. That scaling holds beta*gamma fixed, which is right for
ionization (a function of beta*gamma) and wrong for radiative loss, which
carries explicit mass dependence: adding brems/pair to the shared table
would hand every hadron the PROTON's radiative loss. So the mean is added
instead as a per-species table (GetHadronRadiativeTable), built lazily on
first use from G4hBremsstrahlungModel/G4hPairProductionModel -- the
mass-aware models the SIMULATION itself uses -- at the particle's OWN
kinetic energy, no proton scaling.
* computeRadiativeDEDX refused non-muons outright and built its models on
muonPlus, so radv was identically zero for hadrons. It now takes the
hadron branch with models cached per (thread, particle).
ONE SWITCH FOR BOTH HALVES, deliberately. The missing mean and the missing
fluctuation cancel to ~90 %, so enabling the fluctuation alone measures 0.00049
against 0.00005 for the complete correction -- a 10x DEGRADATION. A single
reader makes the half-enabled state unrepresentable.
Both deltas are routed through a new totalDedxDelta = speciesDedxDelta +
radDedxDelta, so ComputeDEDX, ComputeRange and ComputeEnergy stay consistent:
the range-defect integral is generic in the dE/dx perturbation, and a
ComputeDEDX-only change was previously measured to leave the two branches of
EnergyAfterStep disagreeing by 4.8e-3 for a pion.
Validated (resolution/hadrad_check.py):
* pre-change build vs post-change build, switch OFF: 27/27 branches
bit-identical for mu-, pi+, p. Measured with a genuine rebuild of the
reverted tree, not argued from the code.
* muon with the switch ON: 27/27 bit-identical. The muon has carried brems +
pair in its own table all along, so GetHadronRadiativeTable refuses
|PDG| == 13 and the propagator keeps its original muonPlus path. Note the
null structure differs from CVH_REF_SPECIESDEDX, whose delta is +0.0 for a
proton because the table IS the proton's -- here the PROTON MOVES TOO.
* all six hadrons with the switch ON: 19/27 branches move (F, Q, dEdxlast,
dQI, ...), the signature of a MEAN change; the fluctuation-only
corrections moved 3 and the other reference switch moved 20.
One guard relaxed with its reasoning recorded: the first version threw on
-3e-27 MeV/mm for a pi+ in vacuum, which is log-spline undershoot where the
true value is ~1e-27. Radiative dE/dx is non-negative by definition, so that
is clamped; the throw is kept for non-finite.
NOT YET MEASURED: whether enabling this improves the closure. Only the
mechanical validation above is done. Expected gain is small -- the missing
channel is 2.12 % (pi), 0.41 % (K), 0.15 % (p) of the q/p kappa2, closure
effect 0.00005 (pi) and <1e-5 (K, p) -- so this is a correctness and
harmonisation argument, not a closure argument. The pion already sits at the
1 % threshold at pT = 3 and all three grow with momentum.
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Hb9SC5QHCMQGVj4W5tFd7w
… one `dEdxlast` feeds curv2localJacobianAltelossD as the dE/dx over the differential path a transversely displaced track needs to reach the target plane. That path lies on the side the track ARRIVES from, so the value must be the medium being traversed on arrival. Taking it from the last step unconditionally does not give that. When the target plane sits on a material boundary -- which it does whenever the target is a sensor FACE rather than its mid-plane, i.e. throughout the clean-propagation study -- the propagation terminates with a degenerate step just past the boundary and Geant4 attributes it to the volume on the FAR side. Measured with the new CVH_DEDX_DEBUG dump on the pT = 3 exports: real leg 8 Air 1.00, Air 0.175, Air 0.113, Silicon 1.9e-07 <- sliver real leg 2 Air 0.026, Air 0.031, Silicon 1.88e-02, Si 2.7e-08 real leg 0 Air ..., Pix_Bar_Hybrid_Full 4.7e-03 (rho 4.42) toy leg 13 Vacuum 1.000, Vacuum 0.892, ToyLayerMat 7.8e-05 <- sliver The two cases need OPPOSITE treatment and a blanket "use the outside material" is wrong: leg 8 arrives through air and its silicon reading is the bug, while leg 2 genuinely traverses 188 um of silicon and its silicon reading is correct. Leg 0 shows the arrival medium need not be either. The floor separates them by length, which the dump shows is clean: largest terminal sliver 7.8e-05 cm (they scale with accumulated path), smallest genuine traverse 3.1e-03 cm -- a factor 40 gap. 5e-4 cm sits 6.4x above the first and 6.3x below the second. Verified, real geometry, 19 legs: 0/19 mismatches against the media the dump reports; only legs 8, 13 and 16 change, all to air. refqop, reflocx, refdxdz and Q differ from the previous export by EXACTLY 0. Toy, 14 legs: all go to 0, which the dump confirms is right (every leg arrives through Vacuum); the H-basis qop closure moves -0.0110 -> -0.0011 at the outermost plane, equal to the legacy basis, while the legacy arm is bit-unchanged. Production is unaffected: ResidualGlobalCorrectionMaker*G4e propagate to surfacemapD_[detid], the DetUnit reference surface, which is the sensor mid-plane and therefore not a material boundary. Known limit, recorded in the comment: this is an absolute floor against an artifact that scales with accumulated path. It is 6x clear on these <= 13 cm legs; longer legs should be re-checked with CVH_DEDX_DEBUG first. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BJNXSeCjnP72EfEBiz26Wu
…a model of the simulation Reverts 4b24be8 and extends the same logic to CVH_REF_HADRAD, added later by f4d0e64. The principle is that the DEFAULT configuration should model what Geant4 actually runs. Each of these exists because the model was missing something the simulation does -- the exact knock-on cross section, the Kokoulin term, a charge-aware mean loss, a species-specific dE/dx, and hadron radiative loss (the sim runs hBrems/hPairProd while a hadron's reference subtracted no radiative mean at all). Default-OFF made the out-of-the-box configuration a model of a simulation nobody runs, and every unqualified number a measurement of a known omission. CVH_IONI_EXACTDELTA, CVH_IONI_KOKOULIN, CVH_REF_CHARGEAWARE, CVH_REF_SPECIESDEDX, CVH_REF_HADRAD false -> true NOT flipped, and why: * CVH_IONI_URBAN2021 -- NOTES_DELTASPEC s10.4 falsified the prediction it was built for. It is a diagnostic; enabling it would move the model AWAY from the sim, which is the opposite of the principle above. * CVH_IONONLY, CVH_LOCAL_UPDATE, the CGF_QOP and MATGROUP families -- diagnostics and in-development work, not model-vs-sim switches. * CVH_EM_HARMONISE -- by this criterion the strongest remaining candidate (its own comment: the model job inherits Geant4 defaults where the sim takes CMS values, 12 of 58 differ), but G4ErrorPhysicsListForCVH.cc has uncommitted work in it. Left for whoever lands that. THE ATTRIBUTION GATE 4b24be8 NAMED IS NOT CLOSED BY THIS COMMIT. The global fit that consumes these Jacobians has still not been run both ways, and CVH_REF_CHARGEAWARE remains the one charge-ODD member -- degenerate in a single fit with the calibration's M, which is read as physical misalignment. Default-ON changes which way that risk points (a real charge-odd loss now enters the reference instead of being absorbed into M) but does not remove it. The gate is kept in the block comment rather than deleted; run the fit both ways before quoting M. Verified: closure drivers set all of these explicitly, so their numbers are unchanged -- allcorr dxdz legacy, seven corrections, reproduces the published ladder -0.0074 / outermost -0.0303 exactly, while the four-correction arm gives -0.0067 / -0.0289, i.e. the check discriminates. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BJNXSeCjnP72EfEBiz26Wu
…ent variables
`getenv` is not how a CMSSW module is configured, and these were the only
physics in the package that did not come from a ParameterSet -- the same class
already took ForCVH, StepLengthLimit, IoniTruncationAlpha and
PropagationPtotLimit that way. Ten switches move onto Geant4ePropagator's
PSet; `cvhcgf::envFlag` and every getenv in CGFQoPBlock and
G4UniversalFluctuationForExtrapolator are gone, with no fallback.
IoniExactDelta IoniKokoulin ReferenceChargeAware ReferenceSpeciesDedx
ReferenceHadronRadiative ReferenceIonizationOnly IoniUrban2021
ReferenceSpeciesDedxNbin IoniKokoulinNbin IoniExactDeltaT0
WHAT THIS BUYS, all of which the environment could not:
* PROVENANCE. The PSet is written into the output file, so edmProvDump
recovers exactly which corrections produced it. An environment variable
left no trace in the data: two model files exported with different physics
were indistinguishable, which is precisely the ambiguity that made an
export-arm mismatch cost two wrong comparisons this week.
* VALIDATION. A misspelled parameter is a configuration error; a misspelled
CVH_* was silently the default. The Simpson interval counts are now
checked (even, >= 2) at configure time instead of being silently replaced.
* ONE SOURCE FOR THE DEFAULTS. cvhcgf::Switches' member initializers,
mirrored once in Geant4ePropagator_cfi.py.
Two deliberate failure modes, both improvements on a silent default:
* cvhcgf::switches() THROWS if configure() has not run. An unconfigured job
must fail rather than quietly pick physics.
* configure() called twice with DIFFERENT values throws. These readers
configure process-global Geant4 model classes, so two propagators wanting
different physics cannot both be served; with getenv that conflict was
invisible.
configure() runs in GeantPropagatorESProducer's CONSTRUCTOR, i.e. before any
physics list or fluctuation model is built.
New TrackPropagation/Geant4e/python/cvhSwitches.py maps command-line options
onto the parameters, option name == parameter name so a run log, a config dump
and edmProvDump all show the same string. Unset means "leave the cfi default",
so passing nothing reproduces the default configuration exactly. Both model
drivers register and apply it and echo the effective state.
NOT migrated, because their C++ readers are untouched here: the LD_PRELOAD
shim (not a CMSSW module), and the knobs in G4TablesForExtrapolatorForCVH,
G4ErrorPhysicsListForCVH, ProcessActivationWatcher, MaterialGroupModel,
G4ErrorEnergyLossForCVH and the Analysis/HitAnalyzer plugins. Three of those
files have uncommitted work in them; the HitAnalyzer ones are EDProducers and
should take ordinary module parameters.
NOTE for anyone with an existing work area: the arm directories hold PATCHED
COPIES of runToyModel.py, so `hadron_probe.py setup` (and the equivalents in
allcorr/deltaspec/toy_pt_scan/toy_radoff) must be re-run. A stale copy now
fails loudly with "'IoniExactDelta' not registered"; under getenv it would have
run happily on whatever the shell exported.
Verified: an export driven through the config path is BIT-IDENTICAL to the
environment-driven one -- refqop and reflocx differ by exactly 0 from the
_all4 baseline -- and both closure arms reproduce their published numbers,
dxdz seven-correction -0.0074 / -0.0303 and four-correction -0.0067 / -0.0289.
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BJNXSeCjnP72EfEBiz26Wu
…the hadElastic kernel watcher
Uncommitted work from an earlier session, recorded here as it stood. All of it
is diagnostic: nothing below changes any number unless it is explicitly asked
for.
* ProcessActivationWatcher: dump which hadronic model and cross-section set
actually handles each process, per species, in the physics list that is
really loaded -- FTFP_BERT_EMM assigns different models per species and
energy range, and reading it off the process table is authoritative where
reading the physics-list source is not. Plus the G4EmParameters block,
tagged SIM. std::cout and not G4cout, because SimG4Core installs a
G4UIsession that swallows G4cout while the model job (no such session)
prints -- a difference that cost a run.
* G4TablesForExtrapolatorForCVH: the MODEL-tagged twin of that dump. The
PAIR is the point: G4EmParameters is a global singleton, the sim and the
model are separate jobs with different physics lists, and the model's list
sets no EM parameter at all -- so it silently inherits Geant4 defaults
where the sim takes CMS values (12 of 58 differ).
* G4ErrorPhysicsListForCVH: CVH_EM_HARMONISE, which sets the sim's measured
values in the model job so that "does that difference reach anything we
use?" can be answered by comparing exports. The comment records where this
has to live and why: the singleton LOCKS once physics is initialised and
its setters then do nothing AND SAY NOTHING, so an attempt from
Initialisation() changed none of the 58 values while printing a banner, and
one from Geant4ePropagator's G4State_PreInit branch never ran at all.
ConstructProcess is the only place it works.
* HadElasticKernelWatcher (new): record the deflection the SIMULATION applies
at each hadElastic collision, to check the nuclear-elastic CF kernel
against the sim rather than against inference from its standalone driver.
Steps that did not end in hadElastic are written prescaled as a CONTROL, so
the field+msc contamination is measured under identical conditions instead
of being assumed small (~1.5 mrad of field bend per step against a 25-35
mrad elastic kick). Self-registers via DEFINE_SIMWATCHER as
PrimaryLossCensusWatcher does; fully ParameterSet-driven.
NOT included, and reverted instead: two unused includes (G4EmParameters.hh,
G4MscStepLimitType.hh) left in Geant4ePropagator.cc. No symbol in that file
references either -- they are residue of the abandoned G4State_PreInit attempt
the physics-list comment describes.
The getenv guards on the first three are migrated to configuration in the
follow-up commit, separately, so this record stays unmodified.
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BJNXSeCjnP72EfEBiz26Wu
Follow-up to 5b093d2, which recorded the previous session's diagnostics as they stood. Their getenv guards now go the same way as the energy-loss switches (b372e08), so that nothing in these two packages configures physics or diagnostics out of the environment except the knobs listed as not-yet-migrated. CVH_DUMP_HADMODELS -> ProcessActivationWatcher.dumpHadronicModels CVH_DUMP_EMPARAMS -> ProcessActivationWatcher.dumpEmParameters (SIM half) CVH_DUMP_EMPARAMS -> Geant4ePropagator.DumpEmParameters (MODEL half) CVH_EM_HARMONISE -> Geant4ePropagator.EmHarmonise CVH_DEDX_SCALE -> Geant4ePropagator.DedxScale The watcher already had a ParameterSet, so its two are ordinary untracked parameters there. The physics list and the extrapolator tables are Geant4 classes with no PSet of their own, so they read cvhcgf::switches() -- the same route the energy-loss switches take. CVH_DEDX_SCALE was not part of that session's work but sat in the same file, and it is PHYSICS (a scale on the dE/dx table, probing the J/psi mass bias) rather than a diagnostic, so leaving it on getenv would have left the one knob that most needs to be recoverable from provenance unrecoverable. It needed its own "unset" sentinel in cvhSwitches: its neutral value is 1.0, not 0. Runtime messages naming CVH_IONONLY in G4EnergyLossForExtrapolatorForCVH and Geant4ePropagator now name ReferenceIonizationOnly. real_radoff verifies the switch by grepping the log for that string, so the message and the grep are renamed together (calibration_studies, same series). Verified: an export through the config path is unchanged -- refqop differs from the _all4 baseline by exactly 0, and the closure arms reproduce dxdz -0.0074 / -0.0303 (seven corrections) and -0.0067 / -0.0289 (four). Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BJNXSeCjnP72EfEBiz26Wu
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Port of the CVH (Continuous Variable Helix) track-refit infrastructure and the WMass custom-NanoAOD producers from
WmassNanoProd_10_6_26to CMSSW_15_0_19_patch2 (el9_amd64_gcc12, Geant4 11.x), plus the development layered on top since the initial port: multi-threaded CVH, the ScalarPot3D spherical-harmonic field model with 5×9 transport Jacobians, the sparse-GBL two-track fit, and the new B±→J/ψ(→μμ)K± calibration chain.146 commits — 104 Josh Bendavid (original CVH development, cherry-picked chronologically with authorship preserved), 31 David Walter (port glue + field-model/threading/validation work), 6 Pietro Lugato (B→J/ψK chain), 2 Davide Bruschini (ForCVH Geant4-11.x physics classes), 2 Elisabetta Manca, 1 Kenneth Long.
131 files, +23.8k/−0.1k lines. Full tree builds clean with
scram b -j8.1. Base port
WmassNanoProd_10_6_26: the CVH residual-maker plugins (ResidualGlobalCorrectionMakerBase/G4e/TwoTrackG4einAnalysis/HitAnalyzer), the customizedGeant4ePropagator(double-precisionpropagateGenericWithJacobianAltDwith per-step MS/ionization error decomposition, Urban-model variance with α=0.999 truncation), value-map producers, and per-channel configs.*ForCVHphysics stack (fromWMassRefits_CMSSW_15_1_X): physics list (no MSC, no secondaries), energy-loss tables, Universal Fluctuation model, ErrorEnergyLoss, ErrorMessenger — adapted to G4 11.x thread-localG4ErrorPropagatorManager/G4ErrorPropagatorData(all G4e init deferred to first propagate per thread, mutex + thread-local guarded).FlatTableaddColumn type inference, ESHandle → ESGetToken,SiStripClusterInfoper-producer construction,Eigen::placeholders::all, dd4hep-default geometry kept.2. Post-port development
CvhMasterThread/CvhMaster/CvhWorker, SimG4CoreOscarMTMasterThread-style master/worker): makers are nowedm::stream::EDProducer<edm::GlobalCache<CvhMasterThread>>with idempotent per-TBB-thread worker setup. One G4 master per module instance → one maker per cmsRun job (drives the per-leg job design below). Replaces the-j1GeometryProducer path (kept for legacy tests).MagneticField/ParametrizedEngine): C++ port of themfsspherical-harmonic scalar-potential basis (Φ = Σ c_{l,m}(R/s)^l P_l^m(cosθ){cos,sin}(mφ), B = ∇Φ), loaded from a coefficient dump, with per-mode basis getters for calibration-fit Jacobians. Thread-safe by construction (immutable state, stack-local per-call cache — 7.5× faster than the naive evaluation) — this replaces the thread-unsafe PolyFit3D, which stays hard-disabled. Element-wise closure vsmfsat 1e-12; validity sphere R ≤ 320 cm. Published underIdealMagneticFieldRecordlabelScalarPot3DMf.sim::Fieldgains per-propagation additive field/material offsets so the G4 stepper and the analytic Jacobian see the same B+dB (no double counting; default path behavior-neutral).runFDClosure).MagneticFieldLabelrouting (opt-in, default""= standard behavior): SiPixel/SiStrip CPE ESProducers,TransientTrackBuilder,TrackRefitter,GeometryProducer,GeantPropagatorESProducercan all consume a labelled field, so the CVH chain runs on ScalarPot3D (or Opera3D grid viauseOpera3D) without touching default reco.SimplicialLDLTsolve replaces dense Hessian assembly — ~3.3× less memory, validated bit-identical.srcCandidates(VertexCompositeCandidates) input — one output row per persisted Stage-1 candidate; per-particle Geant4e propagation for V0 daughters (π/K/p mass hypotheses via a single PDG-table source of truth);V0CandidateProducerremoved.fillGradsSIGSEGV fixed (TTree::SetAutoSave(0)), TTree memory bounded, per-stream output files.FlattenedCandValueMapVectorTableProducerflattens per-muonValueMap<vector<int/float>>(CVH global indices + Jacobians) into FlatTables; eight scalar CVH ExtVars wired intomuonTable.3. B±→J/ψ(→μμ)K± chain (new)
JpsiKCandidateSplitter: splits Stage-1 B⁺ candidates into a dimuon collection + bachelor-kaon TrackCollection, with parallel B-candidate-index vectors and per-leg pdgId vectors for multi-channel mass-hypothesis handling;bCandIdxis written to both makers' trees for offline (run, lumi, event, bCandIdx) joining of the two legs.TwoTrackJpsiKMuMuG4e(J/ψ leg, mass constraint on) andJpsiKSingleTrackKaonG4e(bachelor leg, kaon hypothesis).runCvhBplusJpsiK.py: one maker per job (mode=dimuon|kaon), ScalarPot3D as base field (with fallback A/B switch),kaonAsMuonmass-hypothesis knob, and convergence-study instrumentation (configurable GN iterations/EDM threshold,midPropagatedstarting state, per-iteration debug branches).corparms_with absolute scalar-potential coefficients while the same field was already the process base field madegetCorrectionAt()re-add the full base field as dB (doubling B, halving refit q/p). Fixed by starting corrections at zero — corrections are now deltas on the base field. Open FIXME: parmtype-14 entries incorFilesare currently skipped on load, so fitted B-field corrections cannot yet be re-applied in a refit iteration.RecoTracker/TrackProducer/plugins/.Intentionally excluded
Validation status
scram b -j8clean (all packages, plugins refreshed)mfsreference at 1e-12 (element-wise, standalone test)SiStripClusterdeserialization bug, ROOT Replace boost with standard library equivalents cms-sw/cmssw#19773 — use 15_0-native inputs.)runCvhJpsi.py, ScalarPot3D field) on 50-event repacked ALCARECO with 2 threads/2 streams: ~6000 propagations, 0 failures (0ierr, 0 path-length aborts), mean m(μμ) = 3.090 GeV. Exercises the reworked G4 lifecycle, per-stream propagators, and the particle-set registry. The single-track kaon leg and the hadron fluctuation-table rebind are not yet exercised at runtime (needs a B→J/ψK input).Review findings and their resolution
A full-diff review (July 2026) found the issues below. All high/medium items are fixed on the branch (commits
cc97b5107c1,631b221040e) unless noted:Fixed (was: must fix):
Geant4ePropagatorfluctsegfault with nStreams > nThreadsDuplicate— theDEFINE_FWK_MODULEnamesAnalysis/HitAnalyzercopies ofGlobalIdxProducer/TrackProducerFromPatMuons/TrackExtraRekeyerwere removed; the canonical (global,fillDescriptions, extended) versions live inRecoTracker/TrackProducer.muons_cff.pydanglingtrackrefit/trackrefitbsreferences breaking standard NanoAODnanoAOD_addCvhMuonBranches(process)customization that validates the refit modules exist.SiPixelTemplateDBObjectESProducerbreaking bare HLT-menu instantiationsfillDescriptionswithMagneticFieldLabeldefault""added;GeometryProducertolerates an absent parameter.Fixed (was: should fix):
5.
Multi-run G4 teardown— the G4 world is now an explicitly job-scoped resource:CvhMasterThread::ensureG4Started()starts it exactly once at the firstglobalBeginRun, it survives run boundaries, and teardown happens instopThread()at end of job. Since the world captures the run-1 geometry/field ES products, a mid-job IOV change (e.g. a magnet-current change viaAutoMagneticFieldESProducer) is detected via the ES cache identifiers and refused with a clearcms::Exceptioninstead of silently refitting with stale conditions (b5940c1ee84).6.
Hadron ionization variance using the muon dE/dx table—SampleFluctuations/SampleFluctuations2now rebind particle + table viaSetParticleAndCharge(proton-table scaling for hadrons) before the dE/dx lookup, consistent with the mean-loss model.lastMaterialinitialized.7.
Physics-list particle narrowing vs full-list re-instantiation— job-wide particle-set registry inG4ErrorPhysicsListForCVH: the propagator's no-arg instances reproduce the set configured viaCvhMaster.Particles, so no process-less particle definitions can appear on worker threads.8. Max path length 200 cm → 100 m: intentional, kept, now documented in-code — low-pT curling tracks (B→J/ψK bachelor kaons, helix radius ~50 cm) legitimately accumulate >2 m of path before intersecting the target surface. Note this is a deliberate loosening relative to both stock CMSSW and the 10_6 fork (both 200 cm).
9.
Debug— removed.std::coutinsim::Fieldconstructor10.
ScalarPot3DMagneticFieldmissingsetNominalValue()LogWarninginstead of silently returning B=0. The C¹ blend to the volume-based field outside the sphere remains future work.Low-severity items — swept in
2a32331fe10: deadG4WentzelVIModelForCVHremoved;-gCXXFLAGS, unreferenced files/deps, vestigial counters, and stale comments/help text cleaned up; kaon q/p diagnostic gated on the B→J/ψK path; JpsiK cfiuseIdealGeometrydefaults fixed;FlattenedValueMapVectorTableProducersingle-type null-deref guarded;ScalarPot3DEvalheader validation hardened;EmptyESSourcemoved from the ScalarPot3D cfi to the standalone test cfg. Duplications consolidated by checking out the owning package (RecoTracker/TrackProducer): single home for theValueMap<vector<int>>dictionary and theMagneticFieldOffset/OffsetMagneticFieldhelpers, and its BuildFiles reset to stock + the deps the WMass additions actually need (fixes the missingDataFormats/PatCandidatesfor thepat::Muondictionaries and removes 13 orphaned deps of the deleted residual-maker copies).Deliberately kept: dead
computeLandau/SampleFluctuations2+ always-zerolandauDelta/landauWtuple slots (10_6 parity; removal is API churn across both makers — separate commit if wanted); HitAnalyzer'sMatRepStdSMatrix dictionary (unused by current code but possibly needed to read older trees); channel cfis remain driver-injected forscalarPotentialInitFile/CvhMaster(by design).Remaining work
🤖 Generated with Claude Code