Skip to content

[TLE] Stabilize FP16 cumsum correctness test#844

Open
sunnycase wants to merge 4 commits into
mainfrom
fix/tle_cumsum
Open

[TLE] Stabilize FP16 cumsum correctness test#844
sunnycase wants to merge 4 commits into
mainfrom
fix/tle_cumsum

Conversation

@sunnycase

Copy link
Copy Markdown
Collaborator

Background

test_tle_cumsum_exclusive_and_total uses random normal FP16 inputs and compares the result of Triton's parallel scan against a PyTorch reference with fixed tolerances. This made the test intermittently fail in local and CI runs even though repeated launches with the same input were deterministic.

Root cause

Both tl.cumsum and tle.cumsum keep FP16 inputs in FP16 during accumulation. Their parallel scan tree can round differently from the PyTorch reference. Depending on the randomly generated input, the difference can slightly exceed the fixed atol=2e-2, rtol=2e-2 threshold.

A sweep over seeds 0 through 5000 reproduced the failure for 64 of 5001 inputs. For every failing seed, tle.cumsum and native tl.cumsum produced bit-identical results, confirming that this was a test stability issue rather than a TLE-specific race or lowering error.

Fix

Generate bounded integer-valued inputs for the FP16 correctness case and cast them to FP16. With values in [-2, 2], every prefix sum in the tested block remains exactly representable in FP16. This preserves coverage of the FP16 lowering path while making the correctness assertion independent of parallel scan accumulation order.

FP32 and BF16 cases continue to use random normal inputs.

Validation

  • python -m pytest -q python/test/tle/unit/test_tle_cumsum.py: 12 passed, 1 skipped
  • FP16 seed sweep over 10,000 inputs: 0 failures, including exact exclusive and total comparisons

@sunnycase
sunnycase marked this pull request as ready for review July 24, 2026 07:43
zhzhcookie
zhzhcookie previously approved these changes Jul 24, 2026

@zhzhcookie zhzhcookie left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Galaxy1458
Galaxy1458 previously approved these changes Jul 24, 2026

@Galaxy1458 Galaxy1458 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

i3wanna2
i3wanna2 previously approved these changes Jul 24, 2026

@i3wanna2 i3wanna2 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@sunnycase
sunnycase dismissed stale reviews from i3wanna2, Galaxy1458, and zhzhcookie via 09bf751 July 24, 2026 08:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants