Skip to content

Fix integer overflow in custom shader Time calculation#20360

Open
Zish19 wants to merge 1 commit into
microsoft:mainfrom
Zish19:fix-custom-shader-time-overflow
Open

Fix integer overflow in custom shader Time calculation#20360
Zish19 wants to merge 1 commit into
microsoft:mainfrom
Zish19:fix-custom-shader-time-overflow

Conversation

@Zish19

@Zish19 Zish19 commented Jun 24, 2026

Copy link
Copy Markdown

Summary of the Pull Request

Fix an integer overflow in custom shader Time uniform calculation in BackendD3D.

This PR removes an unnecessary 32-bit integer truncation in shader time computation and preserves the original modulo-based architecture while improving precision in the tick-to-seconds conversion.


References and Relevant Issues

Related to #20226

This PR addresses a confirmed overflow bug in custom shader Time computation, although it has not been confirmed as the sole cause of the CPU spike / laggy input reported in the issue.


Detailed Description of the Pull Request / Additional comments

The current implementation computes shader time using:

static_cast<int>(now % _customShaderPerfTickMod) * _customShaderSecsPerPerfTick

Here, now is a 64-bit performance counter value.

At a typical QPC frequency of ~10 MHz:

  • _customShaderPerfTickMod = freq * 1000 = 10,000,000,000
  • INT_MAX = 2,147,483,647

As a result, now % _customShaderPerfTickMod can exceed INT_MAX.

Casting that 64-bit modulo result to a signed 32-bit integer causes overflow, producing corrupted (including negative) Time values passed to custom shaders.

This PR fixes the issue by:

  • removing the unnecessary static_cast<int>
  • keeping the modulo computation entirely in 64-bit integer space
  • upgrading _customShaderSecsPerPerfTick from f32 to double to preserve maximum precision before the final bounded float cast for the shader constant buffer

This preserves the original design:

  • exact integer modulo in tick space
  • bounded periodic shader time
  • minimal per-frame overhead

Validation Steps Performed

  • Investigated the rendering pipeline for custom shader execution in BackendD3D

  • Verified _customShaderPerfTickMod can exceed INT_MAX on typical systems

  • Confirmed overflow mathematically (~214 seconds at 10 MHz QPC)

  • Verified only the following files were modified:

    • src/renderer/atlas/BackendD3D.cpp
    • src/renderer/atlas/BackendD3D.h
  • Confirmed no unrelated code paths or dependencies were affected

  • Performed static code validation and diff review to ensure semantic correctness

Note: Full local build validation via tools/bcz.cmd could not be completed in my environment due to missing Visual Studio/MSBuild toolchain setup.


PR Checklist

  • Closes #xxx
  • Tests added/passed
  • Documentation updated
  • Schema updated (if necessary)

@Zish19

Zish19 commented Jun 24, 2026

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

@DHowett DHowett left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix! I am not convinced that we should move to double precision for the math, because now is only ever going to be a whole number, and _customShaderSecsPerPerfTick can truly represent all seconds per tick with a float. I am also not convinced that this is the cause of the reporter's delay; rather, it should only cause stuttering around the overflow points in shaders that use the Time component.

So I guess I'd say... can we just use uint64_t for the math and be done with it?

@microsoft-github-policy-service microsoft-github-policy-service Bot added the Needs-Author-Feedback The original author of the issue/PR needs to come back and respond to something label Jun 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Needs-Author-Feedback The original author of the issue/PR needs to come back and respond to something

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants