[RF] Faster Hesse in RooFit by advertising which params are independent - #16394
Merged
Conversation
guitargeek
force-pushed
the
hessian_optimization
branch
from
September 9, 2024 21:59
eae7b17 to
6e8c228
Compare
Test Results 21 files 21 suites 3d 11h 46m 15s ⏱️ For more details on these failures, see this check. Results for commit 418e525. ♻️ This comment has been updated with latest results. |
Member
|
The error seems an unresolved symbol on Win |
guitargeek
force-pushed
the
hessian_optimization
branch
2 times, most recently
from
September 14, 2024 15:38
5a2d19a to
04afc01
Compare
dpiparo
self-requested a review
September 15, 2024 08:16
dpiparo
approved these changes
Sep 15, 2024
dpiparo
left a comment
dpiparo
left a comment
Member
There was a problem hiding this comment.
LGTM, thanks for this improvement! Let's not forget to give to this feature the right emphasis in the RNs.
guitargeek
force-pushed
the
hessian_optimization
branch
from
October 28, 2024 13:59
04afc01 to
9ebb89a
Compare
guitargeek
force-pushed
the
hessian_optimization
branch
2 times, most recently
from
February 15, 2025 12:31
f3f761d to
5d5d765
Compare
guitargeek
force-pushed
the
hessian_optimization
branch
from
February 28, 2025 16:57
5d5d765 to
82825d7
Compare
guitargeek
force-pushed
the
hessian_optimization
branch
from
March 11, 2025 23:27
82825d7 to
3fa8122
Compare
guitargeek
marked this pull request as draft
July 29, 2025 07:28
Contributor
Author
|
Converter to draft because the main bottleneck in the Hessian computation was identified and eliminated: It's still good to keep this PR around as a possible solution if the Hessian should become the bottleneck again. |
guitargeek
force-pushed
the
hessian_optimization
branch
from
August 2, 2026 17:57
3fa8122 to
1505c19
Compare
guitargeek
force-pushed
the
hessian_optimization
branch
from
August 9, 2026 20:19
1505c19 to
0c5904a
Compare
guitargeek
force-pushed
the
hessian_optimization
branch
from
September 6, 2026 22:12
0c5904a to
4484a7c
Compare
guitargeek
marked this pull request as ready for review
September 6, 2026 22:17
Objective functions can now advertise pairs of parameters whose mixed second derivative is identically zero, via the new virtual function FCNBase::SecondDerivativeAlwaysVanishes(). The numerical Hessian computation in MnHesse skips the finite-difference evaluations for such parameter pairs, which can speed up Hesse significantly for likelihoods with many mutually independent parameters. The parameter indices in this interface refer to the FCN's own full (external) parameter space, and the advertised information must hold for all parameter values; see the FCNBase documentation for the complete contract. Since MnHesse loops over Minuit-internal indices that exclude fixed parameters, it translates them via MnUserTransformation::ExtOfInt() before querying the predicate, so results stay correct when parameters are fixed in the minimizer. Clients that drive Minuit2 through the ROOT::Math interfaces can inject the predicate with the new Minuit2Minimizer::SetSecondDerivativeAlwaysVanishesFunc(), which follows the same pattern as SetHessianFunction(). This keeps the feature out of the general ROOT::Math function interfaces on purpose: it is only consumed by Minuit2.
guitargeek
force-pushed
the
hessian_optimization
branch
from
September 6, 2026 22:19
4484a7c to
df7545e
Compare
This reduces the time to run Hesse in the ATLAS Higgs benchmark from 123 s to 92 seconds. Given that some models take hours for this, this is a significant improvement for the user experience. RooFit analyzes the computation graph of the minimized function to find pairs of parameters that never appear in the same additive term of the likelihood, meaning their mixed second derivative is identically zero. The analysis is implemented privately in RooMinimizerFcn, on purpose without adding any new public interfaces, so that the design can still evolve: only strictly additive nodes (RooAddition, RooConstraintSum, and the RooEvaluatorWrapper forwarding to its wrapped function) are recursed into; everything else conservatively contributes all of its graph leaves as a single term, which advertises no independence but is always correct. The resulting pairwise mask is stored as a packed bitvector and built lazily on the first query, so that migrad-only fits and toy loops do not pay for it. It is handed to Minuit2 in RooMinimizerFcn::initMinimizer() via Minuit2Minimizer::SetSecondDerivativeAlwaysVanishesFunc(), which is why RooFitCore now has a private link dependency on Minuit2. The mask indices follow Minuit's external parameter convention, so the result is also correct when parameters are fixed after the RooMinimizer was constructed, which is covered by a unit test. Further improvement is possible by analyzing the computation graph a bit more to find more independent parameters (e.g., the different gammas for stat uncertainties from different bins).
guitargeek
force-pushed
the
hessian_optimization
branch
from
September 6, 2026 22:48
df7545e to
418e525
Compare
Contributor
Author
|
Revived the PR in the context of the Hessian work with Clad. These optimizations are obviously good for our users and can also serve as inspiration for similar optimizations in Clad. Faster numeric Hessians will also increase the pressure on the AD code path to keep up 🙂 @dpiparo, I have made this a prominent item in the release notes. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This reduces the time to run Hesse in the ATLAS Higgs benchmark from
123 s to 92 seconds.
Given that some models take hours for this, this is a significant
improvement for the user experience.
Further improvement is possible by analyzing the computation graph a bit
more to find more independent parameters (e.g., the different gammas for
stat uncertainties from different bins).
Benchmark results
The Higgs combination benchmark takes 31 s to minimize, plus 79 s for the Hessian with ROOT
master. It was 130 seconds with ROOTmasterjust a few days ago, before some optimization PRs got merged. This PR reduces the time forhesse()to 57 seconds.