Improve mGPU Gaussian tile intersection#664
Open
matthewdcong wants to merge 4 commits into
Open
Conversation
Signed-off-by: Matthew Cong <mcong@nvidia.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>
Signed-off-by: Matthew Cong <mcong@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Previous iterations of mGPU Gaussian tile intersection include:
main)main)This PR introduces a more performant strategy that pre-partitions the Gaussians by computing the intersections of all Gaussians with only the subset of the tiles/the tile range rendered by that GPU. Since the tile keys are monotonically increasing, this means that the subsequent sorting process is decoupled, i.e. we can sort the per-GPU Gaussian tile intersection lists independently and the resulting flattened array is guaranteed to be sorted. This significantly reduces the amount of communication and data transfer required during the sorting process. Moreover, a switch from radix sort to merge sort enables us to remove the temp output buffers, further reducing stalls due to prefetching as well as decreasing peak memory utilization.
This is a performance improvement across the board, but becomes more significant as the number of GPUs increases. On 8x A100s, this improves end-to-end reconstruction performance about 15% with a batch size of 1 (on a relatively small problem).