Repository navigation
Bound integer gamma factorial work and stabilize large-shape densities - #180
Merged
Merged
Conversation
The direct integer-shape calculation evaluates factorial(shape - 1) before deciding whether to use the logarithmic fallback. Its O(shape) cost makes large shapes impractical even when the density is zero. Limit the direct factorial calculation to shapes at most 32. Larger shapes use the existing log-factorial fallback, retaining the integer shape without performing the linear factorial calculation. Document the limit and cover a shape of one billion with float, double, and real regression cases. AI-Assisted-By: Codex (GPT-6 Astra Medium)
The logarithmic gamma density subtracts large, nearly equal terms near the mean. At shape and argument 1e16, this cancellation can produce a log-density of zero or 64 instead of about -19.34. Extract the existing centered chi-square calculation into a shared internal helper and use it in both gamma PDF and LPDF overloads for finite shapes at least 32 within half the shape of the mean. Retain the existing calculations outside that region. Preserve the displacement from large integer shapes by subtracting their high and low pieces separately before normalization. This avoids losing low shape bits when real has a 53-bit significand. Add independent density references, scale and region-boundary cases, maximum finite shape coverage, and integer shapes above 2^53. Document the centered calculation and register the helper in Meson. AI-Assisted-By: Codex (GPT-6 Astra Medium)
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix two issues in gamma density evaluation: excessive factorial work for large integer shapes and cancellation near the mean for large shapes.
The integer PDF previously evaluated an O(shape) factorial before deciding whether to discard the direct result. Limit that calculation to shapes at most 32 and use the existing log-factorial fallback for larger shapes.
For large shapes near the mean, the logarithmic density subtracts large, nearly equal terms. For example, at argument and shape 1e16, this can produce a log-density of zero or 64 instead of approximately -19.34. Extract the existing centered chi-square calculation into a shared internal helper and use it in both gamma PDF and LPDF overloads when shape >= 32 and abs(x / scale - shape) < shape / 2. Existing calculations remain in use outside that region.
For integer shapes, compute the displacement by subtracting the high and low pieces separately, preserving low shape bits even when real has a 53-bit significand. Add independent reference cases, scale and region-boundary coverage, maximum finite shapes, and integer shapes above 2^53.
Validation:
No dependency requirements or selections are changed. The two commits were rebased onto current master with unchanged patches.
AI-Assisted-By: Codex (GPT-6 Astra Medium)