Fix finite-range checks and rounding carries in HexFloat::castTo - #6902
davidegrohmann wants to merge 1 commit into
Conversation
da45563 to
8d69de7
Compare
kpet
left a comment
kpet
left a comment
There was a problem hiding this comment.
Thanks for making all the changes I requested offline. This is now much easier to read. I can't fault the change after another round of review but other pairs of eyes are most welcome to try.
s-perron
left a comment
s-perron
left a comment
There was a problem hiding this comment.
The code looks good. My only concern is maintaining the tests. Some tests will be testing many things. Because you use loops to generate the different subtest, it might be hard to figure out which subtest failed from the error message generated by GTEST. Please try to separate it out is some reasonable way. You can have parameterized tests. See HexFloatFP32ToE5M2Tests in the same test file.
8d69de7 to
8a8d461
Compare
Thanks for the review. |
The exponent bias is not always the largest finite exponent. E4M3 and the finite-only FP4/FP6 formats use the all-ones exponent for finite values. Comparing against the bias incorrectly treats these values as overflow; for example, parsing E4M3 decimal 256 produces 448. Check the rounded exponent and significand against the destination's largest finite value. Include the significand to prevent rounding into E4M3's reserved NaN encoding, and propagate the rounding carry when constructing the result. Handle source infinity separately from conversion overflow. Use has_infinity to select infinity or the largest finite result. Add exhaustive finite-value conversions for FP4, FP6 and FP8 across both signs and all rounding modes, plus E4M3 rounding-boundary and decimal-parsing regressions. Cover overflow and infinity saturation for FP4, FP6 and E4M3 across both signs and all rounding modes. Signed-off-by: Davide Grohmann <davide.grohmann@arm.com>
Head branch was pushed to by a user without write access
8a8d461 to
5ffe6d1
Compare
4 participants
The exponent bias is not always the largest finite exponent. E4M3 and the finite-only FP4/FP6 formats use the all-ones exponent for finite values. Comparing against the bias incorrectly treats these values as overflow; for example, parsing E4M3 decimal 256 produces 448.
Check the rounded exponent and significand against the destination's largest finite value. Include the significand to prevent rounding into E4M3's reserved NaN encoding, and propagate the rounding carry when constructing the result.
Handle source infinity separately from conversion overflow. Use has_infinity to select infinity or the largest finite result.
Add exhaustive finite-value conversions for FP4, FP6 and FP8 across both signs and all rounding modes, plus E4M3 rounding-boundary and decimal-parsing regressions. Cover overflow and infinity saturation for FP4, FP6 and E4M3 across both signs and all rounding modes.