Repository navigation
ADFA-6810 | Log model load time; hint the kernel to read ahead on a selection's first load - #160
Open
davidschachterADFA wants to merge 4 commits into
Open
davidschachterADFA wants to merge 4 commits into
davidschachterADFA wants to merge 4 commits into
Conversation
…ooling for repeatable cold-load measurement Logs load duration and MB/s on every model load, giving Phase 1's cold/warm comparison a real signal instead of guessing whether load time is I/O-bound. evict_cache.c/evict-page-cache.sh are standalone measurement aids (not built into the plugin) that force a file's clean pages out of the page cache via posix_fadvise(DONTNEED), so a cold model load can be repeated on a private- storage (f2fs) file without root or a reboot. They have no effect through FUSE (/sdcard); a reboot is still required there. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAYJVrBKjeghKxnig7N
…l load Issues posix_fadvise(WILLNEED) from the JNI wrapper before the model is mapped, only on the first load of a selection in a process (the load most likely to be cold). Reloads of the same selection skip it. Measurement caveat: non-interleaved runs showed ~31% off a cold FUSE load, but an interleaved on/off test showed no reliable difference. A warm load through FUSE seemed to cost ~140 ms more with the hint. Shipped on the judgement that it can save time on a cold load and rarely costs a little. Needs no llama.cpp fork change; AAR rebuilt from the pinned c9058a713. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAYJVrBKjeghKxnig7N
- Test that a load refused before the engine does not consume the hint; checked it fails if the hint is armed above the refusals. - Document that the hint tracks one selection (A, B, A re-hints A), and correct comments that overstated the benefit or said "concurrently". - Timing log: say it covers the whole engine.load, format MB/s with Locale.ROOT, and skip it for a zero-byte size. - Open the hint's descriptor with O_CLOEXEC. - Eviction tooling: report "advised DONTNEED" instead of "evicted", quote device paths with spaces, and give the missing-NDK message. AAR rebuilt for the C++ change; source stamp matches. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAYJVrBKjeghKxnig7N
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
engine.load(Model load: <N> ms, <X> MB/s).posix_fadvise(WILLNEED)from the JNI wrapper before the model is mapped, on the first load after the app starts or the selection changes. A reload of the same selection skips it. The code remembers one selection, so switching A to B and back to A hints A again.scripts/evict_cache.c,scripts/evict-page-cache.sh); not part of the plugin build.c9058a713(wrapper change only, no fork change).Honest measurement caveat
Non-interleaved runs showed ~31% off a cold FUSE load, but an interleaved on/off test (n=5/n=4) found no reliable cold difference (2748 vs 2693 ms, stdev ~400). A warm load through FUSE seemed ~140 ms slower with the hint (cross-session, not re-checked interleaved). Shipped on the judgement that it can save time on a cold load and rarely costs a little. The main finding on the ticket is that storage location dominates: FUSE cold is ~4.2x warm, private storage ~1.5x.
Review
An independent review (Opus) found no blockers. Its one should-fix, a test that a load refused before the engine does not consume the hint, is added (and shown to fail if the hint is armed too early). Wording, timing-log,
O_CLOEXECand eviction-script nits are fixed in the second commit.Test plan
testDebugUnitTestpasses, including the new first-load/reload/different-selection test and the AAR source-stamp testassemblePluginDebugbuilds; installed and loaded on a Pixel 6 Pro🤖 Generated with Claude Code
https://claude.ai/code/session_01Vh2DAYJVrBKjeghKxnig7N