Skip to content

Reuse leading gaps in memory planner#10801

Open
teerthsharma wants to merge 1 commit into
google:masterfrom
teerthsharma:codex/memory-planner-prefix-gap
Open

Reuse leading gaps in memory planner#10801
teerthsharma wants to merge 1 commit into
google:masterfrom
teerthsharma:codex/memory-planner-prefix-gap

Conversation

@teerthsharma

Copy link
Copy Markdown

Summary

  • consider the free prefix before the first live memory block as a best-fit candidate
  • preserve the existing append fallback and preference for a strictly smaller internal gap
  • cover single-block reuse, smaller internal gaps, and deterministic equal-gap selection

Root cause

find_value_alloc_offset searched gaps between coalesced live blocks and then
appended after the final block, but never considered
[0, first_live_block.start). Its single-live-block fast path always appended.
This could grow the arena even when the leading interval was large enough and
free for the value's complete lifetime.

Impact

Local subgraph benchmarks on an Intel Core i7-14700HX reported:

  • FP32 MobileNet V1: 23.862980 to 22.331730 MiB peak allocation (6.42% lower)
  • FP16 MobileNet V1: 21.255882 to 19.916039 MiB (6.30% lower)
  • the other seven MobileNet cases were unchanged

A public-API resize/reduce witness reduced runtime workspace from 144 MiB + 32 B
to 112 MiB + 32 B with identical output hashes. This change does not alter the
planner's asymptotic complexity, and no invocation-speed change is claimed.

Validation

  • memory-planner-test: 23/23 passed
  • relevant planner/subgraph CTest group: 4/4 passed
  • 30 randomized baseline/candidate MobileNet pairs reproduced the exact memory deltas
  • public runtime create, reshape, setup, and two invokes passed with matching outputs

Fixes #10800

@teerthsharma
teerthsharma marked this pull request as ready for review July 21, 2026 06:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Memory planner misses reusable leading arena gaps

1 participant