Repository navigation
metal: synchronous pipeline calls always compile and return their result; completion handlers on workers (second version of #396) - #398
Open
claudiusthebot wants to merge 2 commits into
Open
claudiusthebot wants to merge 2 commits into
claudiusthebot wants to merge 2 commits into
Conversation
…ult; completion handlers on workers (second version of #396) #396 timed the synchronous new*PipelineState calls out after 5 s and refused the pipeline while the compile went on. On phone 2 (19 Nov) a cold cache made backboardd's first render pipelines take longer than that; QuartzCore treats a refused render pipeline as fatal, backboardd aborted three times and launchd rebooted, twice. #397 reverted it. - metal2/CompileQueue.h (new, plain C++17): the rules. A synchronous call compiles on the calling thread, every time, and returns the pipeline or the compile's own error: no timeout, no worker slot, no negative cache. A completion-handler call returns at once; its compile runs on a worker (at most 3 at a time, FIFO list, no thread parked on a semaphore), the handler is called exactly once; an asynchronous compile that failed after more than 10 s is not started again for 10 minutes. Nothing is called with the lock held (handlers may ask for the next pipeline). - patches/frontend-async-compile.diff: the frontend on CompileQueue.h. The 3 pipeline makers go through gCompiler.sync, the 6 completion-handler methods through gCompiler.async (the handler gets the state, reflection and NSError the synchronous method made). The worker queue has no QoS of its own: a compile runs at the QoS of the thread that asked for it. STARBITE_M2V_COMPILE_TIMEOUT_MS is gone (a leftover mtl2-env line changes nothing). - mesa26-panvk-compile-budget.patch: the budget is the compiling thread's CPU time (CLOCK_THREAD_CPUTIME_ID), not wall time: a compile starved of a core is not a runaway, and a failed render pipeline is fatal to backboardd. - metal2/tests/compile-queue-test.cpp + tests/run.sh (and a CI step): sync waits for a slow compile even with the old timeout variable set; sync never waits behind busy async slots; 200 async requests on 3 slots each answered once; make() calling sync() and done() calling async() do not deadlock; a throwing handler keeps no slot; the negative cache is async-only. Under ThreadSanitizer too where the compiler has it. run.sh also checks the frontend diff: it fails on #396's version. - docs/ios27/METAL.md: why #396 was reverted, and what the timeout-0 boot's samples show (SpringBoard's main thread in a synchronous XPC to chronod; backboardd idle; no compile running). Per-user cache (hotmetal2-shared-cache.diff), the warm-cache step (81-m2v-cache.sh, harvest-m2v-cache.sh) unchanged from #396. Built, not phone-tested.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Built, not phone-tested. No phone was reachable on 20 Nov; nothing here has run on a phone. Do not merge before the phone test below passes.
Second version of #396 (reverted by #397). It keeps what #396 got right (completion-handler compiles on worker threads, PanVK's compile budget for runaway compiles, one shader cache per user, the warm-cache step) and drops the part that broke the phone: a synchronous pipeline call now always gets its pipeline or the compile's own error, never a timeout refusal.
Why #396 failed on phone 2 (19 Nov)
5 s timeout → backboardd crash loop. On a cold cache, backboardd's first render pipelines took longer than
STARBITE_M2V_COMPILE_TIMEOUT_MS(5000 ms), so the frontend refused them ("still running … ask again"). QuartzCore treats a refused render pipeline as fatal, so backboardd aborted three times and launchd rebooted, on two boots in a row. A synchronous Metal call can't be told "ask again later".Timeout 0 → frames stop after the boot logo. The kernel's samples of that boot (
con-xnu-boot-L6-1119-0039.log, symbolized against the 24B156 cache) show:mach_msg. The only StarbiteMetal frames were the telemetrysleep()loops. No thread was compiling and there were no PANVK lines.-[CHSChronoServicesConnection _subscribeToExtensions]→xpc_connection_send_message_with_reply_sync). It made no new commits, so backboardd had nothing new to show.With the timeout at 0, metal: compiles off the UI thread (timeout + budget), per-user shader cache, warm-cache step #396's synchronous path compiled on the caller as before. Inferred: the stall was not caused by the frontend. It is not proven, because chronod wasn't sampled. The test image below samples it.
What changes
tools/ios27/metal2/CompileQueue.h(new, plain C++17). It holds the rules, and its header comment states them:patches/frontend-async-compile.diffgCompiler.syncand the 6 completion-handler methods throughgCompiler.async.STARBITE_M2V_COMPILE_TIMEOUT_MSis gone. A leftovermtl2-envline changes nothing.mesa26-panvk-compile-budget.patch. The budget now counts the compiling thread's CPU time (CLOCK_THREAD_CPUTIME_ID) instead of wall time. A compile that is only waiting for a core is not a runaway, and since synchronous callers now see this error, a wall-clock budget could fail backboardd's pipelines on a busy phone. The longest real compile seen took 25 s of CPU; the default budget stays at 90 s.Tests:
tools/ios27/metal2/tests/compile-queue-test.cpp+tests/run.sh, plus a CI step. Each test names the phone failure it would have caught:make()→sync()anddone()→async()re-entry doesn't deadlock;The tests also run under ThreadSanitizer where it can start (with
setarch -Ron Linux kernels with large mmap randomisation).run.shalso lints the frontend diff: no timed wait or timeout refusal, 3gCompiler.syncand 6gCompiler.async. It fails on metal: compiles off the UI thread (timeout + budget), per-user shader cache, warm-cache step #396's diff (checked) and passes on this one.docs/ios27/METAL.mdexplains why metal: compiles off the UI thread (timeout + budget), per-user shader cache, warm-cache step #396 was reverted and what the timeout-0 samples show.Unchanged from #396:
hotmetal2-shared-cache.diff,81-m2v-cache.shandharvest-m2v-cache.sh.The first commit re-applies #396 (reverting #397) so the second one reads as a change to it.
Trade-off, stated plainly
A synchronous caller on a UI thread still waits for its compile, as it did before #396. A cold first Settings open can take as long as its 24 522-NIR compute shader, about 25 s of one core. The fix for that is the warm cache (step 81), not a timer. Opens after the first should come from the per-user cache.
Built
Built on the Mac mini with
mk-panvk27.shandmk-bundle27.sh, using the same Xcode (iOS 18.5 SDK), host tools and bridge library (Nov 12) as #396's bundle:c1ba995d0a19f6af…(dSYM UUID 537181AC-2370-3012-861C-691F97A6FA4F)72136f02b948be76…The binaries contain no "still running after" string. These local checks pass:
tests/run.sh,provenance-check,arch-check --report,builds-test,build-lock-test. The talon preflight lane doesn't apply to this repository.Phone test (to do)
On phone 2, with BOOT L6 +
sweetsample=SpringBoard,backboardd,chronodand the root step "bundle + Preferences in mtl2-clients + step 81, cold cache":Cold-cache boot. Pass if:
PANVK:lines appear (the cache really was cold);refused,abandoned after,SWEET EXITorCRASH;If it stalls after swap 6, symbolize the main threads of SpringBoard, chronod and backboardd from the samples before rebooting.
Settings ×3. The first open may take as long as its compiles. Opens 2 and 3 should show Settings within 3 s with no new
PANVK:lines. No crash and no refusal on any open.Display & Brightness ×3. The page draws each time, with no freeze or crash; the 2nd and 3rd opens in under 2 s.
Warm-cache repeat. Reboot without a root step.
PANVK:lines should be 0 or very few, and Settings' first open within 3 s.The exact commands, images and hashes are in the test notes on the mini (prep-1120/TEST-ON-PHONE.md).
🤖 Generated with Claude Code