diff --git a/.gitignore b/.gitignore index 0f78c001c..23bcd235a 100644 --- a/.gitignore +++ b/.gitignore @@ -162,4 +162,3 @@ __pycache__/ # the loader S-Boot starts, as built (tools/ios27/s9/build-loader.sh) /xnuloader/sboot/out/ -tools/ios27/fullroot/warm-m2v/ diff --git a/docs/ios27/METAL.md b/docs/ios27/METAL.md index 8c4675540..23700e3af 100644 --- a/docs/ios27/METAL.md +++ b/docs/ios27/METAL.md @@ -94,31 +94,3 @@ profile of the textured case the per-draw path is spread over the runtime (`draw thirds of `end_pass` for scissors that differ every draw (it marks the bin grid one tile at a time; a draw whose scissor covers the whole pass skips it, as most of CoreAnimation's main-pass draws should). The M1 runs this single-threaded code several times faster than the S9's M3 at 1794 MHz. - -## Shader compiles: off the UI, bounded, shared and warm (19 Nov 2026) - -What an iOS 27 boot on the S9 showed (phone 2, `con-xnu-boot-L4-1118-2103.log`): every Settings open compiled the same 24 522-NIR compute -shader (250 224 bytes) for 24 to 28 s on one core with the GPU idle and the UI stalled; Preferences then began a fragment shader of 3 023 -NIR instructions that never ended (Preferences at 130 %, launchd at 40 %, for more than three minutes). The causes and what changed: - -| Cause | Change | -|---|---| -| PanVK compiled on the thread that asked for the pipeline, and the completion-handler `new...PipelineState` calls ran the compile on the caller too | `patches/frontend-async-compile.diff`: a compile runs on a worker (three at a time, QOS utility); the synchronous calls wait for it at most `STARBITE_M2V_COMPILE_TIMEOUT_MS` (default 5000, 0 = for ever) and then fail like a refused pipeline, the compile going on in the background and its result waiting for the next request of the same pipeline (key: the functions' bit-code hashes and the state; while one is still running a repeat waits 100 ms, and with four such compiles running, 200 ms); the completion-handler calls return at once. A pipeline whose compile failed after more than 10 s is not tried again for 10 minutes | -| A compile with no end: Bifrost's register allocator spills a node (or, past four rounds, up to 32) per round and rebuilds the interference graph each time | `mesa26-panvk-compile-budget.patch`: `STARBITE_PANVK_COMPILE_BUDGET_MS` (default 90 000, 0 = none) is a wall-clock budget of one compile; the allocator's rounds, its LCRA solve and its liveness loop check it and leave the compile by `longjmp`; the pipeline fails (`PANVK: ... abandoned after N ms (compile budget)` on the console) | -| One cache directory per program (`/var/local/m2v-cache/`), so only backboardd's was ever filled, a sandboxed program fell back to its own caches directory, and Preferences or SpringBoard never saw what another had compiled | `patches/hotmetal2-shared-cache.diff`: one directory per user, `/var/local/m2v-cache/u` (the runtime's files are owner-only), for every process; `STARBITE_M2V_CACHE=/dir` (mtl2-env) names another | -| The directory was made by the first process to run, with nobody to ship anything in it | `steps.d/81-m2v-cache.sh` makes `/var/local/m2v-cache` (0777), removes the per-program directories, and installs a warm cache (below) | - -**A warm cache.** The runtime's entries are named by content: translated functions and pipelines under `v1//` (the build is the -bridge library's), PanVK's pipeline data in the content directory beside them (named by the SPIR-V, the pipeline state and PanVK's cache -UUID, `driver.rs`). PanVK itself cannot compile on the Mac for the Mali-G72 (it needs the kernel's GPU), so a warm cache is harvested from a -phone: boot the bundle that will ship, open Settings, Display, Appearance (Liquid Glass), the app switcher and Control Center until the -`PANVK: ... NIR instructions` lines stop, copy `/private/var/local/m2v-cache` off, run `tools/ios27/metal2/harvest-m2v-cache.sh PULLED` -(it keeps the newest build directory and the content directory of each `u/`, into `tools/ios27/fullroot/warm-m2v`, which is not in git), -and build the root: step 81 installs it, owner-only, owned by the uid. It holds for that bundle and that PanVK (about 33 MB); another -build finds none of it and compiles as before. - -**Format 500 is `MTLPixelFormatYCBCR8_420_2P`** (metal2vulkan's `generated.rs`), not an unknown format: the runtime has it (`pixel_format_info.rs`, -`bind/ycbcr_convert`, converted in the shader), and the frontend's texture over an IOSurface passes the surface's YCbCr matrix on. The -`texture memory fmt 500` line is that path (`how` is "memory" for textures over an IOSurface or a buffer: they have no device upload, by -design, so no `texture device` line follows), and nothing was refused. Why the reference image did not draw is not in that log: the pipelines that sample it may simply not have been made yet (the compute and -fragment compiles above were still running), which the two changes above address; a phone run of the new bundle will say. diff --git a/tools/ios27/fullroot/steps.d/81-m2v-cache.sh b/tools/ios27/fullroot/steps.d/81-m2v-cache.sh deleted file mode 100755 index 012540da4..000000000 --- a/tools/ios27/fullroot/steps.d/81-m2v-cache.sh +++ /dev/null @@ -1,44 +0,0 @@ -#!/bin/bash -# 81-m2v-cache.sh ROOT : the Metal shader cache directory, and a warm cache in it. -# The plug-in (StarbiteMetal.bundle, HotMetal2.mm + patches/hotmetal2-shared-cache.diff) keeps one cache per USER, shared by every -# process of that user: /private/var/local/m2v-cache/u/ (v1// = translated shaders, v1// = PanVK's pipeline data; -# all named by content). It makes the directory itself, but only where /private/var/local/m2v-cache exists and is writable: this -# step makes it (0777), and removes the per-program directories (backboardd/, PosterBoard/, ...) older plug-ins left. -# WARM_M2V_CACHE= (default tools/ios27/fullroot/warm-m2v when it exists) holds u/ directories as harvest-m2v-cache.sh -# makes them from a phone's cache after a warm-up (Settings, Display, Liquid Glass, the app switcher...): they are copied in, owned -# by that uid, owner-only (the runtime refuses files others can write), so the first Settings open compiles nothing. -# A warm cache is valid for the bundle and PanVK it was harvested with (the entries live under the bridge's build hash and -# PanVK's cache UUID); another build finds none of it and compiles as before, leaving the stale files to the runtime's 30-day sweep. -# WARM_M2V_CACHE=none: no warm cache (and the existing directories are left as they are). -set -euo pipefail -R=$1 -H=$(cd "$(dirname "$0")" && pwd) -L=$R/private/var/local -C=$L/m2v-cache -install -d -m 0777 "$L" 2>/dev/null || mkdir -p "$L" -install -d -m 0777 "$C" 2>/dev/null || mkdir -p "$C" -chmod 0777 "$C" -# a plug-in before the shared cache: one directory per program -for d in "$C"/*/; do - [[ -d $d ]] || continue - n=$(basename "$d") - case $n in u[0-9]*) ;; *) rm -rf "$d"; echo "81-m2v-cache: removed the per-program cache $n";; esac -done -WARM=${WARM_M2V_CACHE:-$H/../warm-m2v} -if [[ $WARM == none || ! -d $WARM ]]; then - echo "81-m2v-cache: $C ready (0777); no warm cache (WARM_M2V_CACHE unset or $H/../warm-m2v absent)" - exit 0 -fi -files=0 -for u in "$WARM"/u[0-9]*/; do - [[ -d $u ]] || continue - uid=$(basename "$u"); uid=${uid#u} - rm -rf "$C/u$uid" - mkdir -p "$C/u$uid" - cp -R "$u." "$C/u$uid/" - find "$C/u$uid" -type d -exec chmod 0700 {} + - find "$C/u$uid" -type f -exec chmod 0600 {} + - chown -R "$uid:$uid" "$C/u$uid" 2>/dev/null || echo "81-m2v-cache: u$uid: chown to $uid failed (not root?): the files stay with the building user" - files=$((files + $(find "$C/u$uid" -type f | wc -l))) -done -echo "81-m2v-cache: warm cache from $WARM: $files files, $(du -sk "$C" | cut -f1) KiB in $C" diff --git a/tools/ios27/mesa-ios/mesa26-panvk-compile-budget.patch b/tools/ios27/mesa-ios/mesa26-panvk-compile-budget.patch deleted file mode 100644 index 7977ec2fb..000000000 --- a/tools/ios27/mesa-ios/mesa26-panvk-compile-budget.patch +++ /dev/null @@ -1,147 +0,0 @@ -diff -ru a/src/panfrost/compiler/bifrost/bi_ra.c b/src/panfrost/compiler/bifrost/bi_ra.c ---- a/src/panfrost/compiler/bifrost/bi_ra.c 2026-11-18 21:34:48 -+++ b/src/panfrost/compiler/bifrost/bi_ra.c 2026-11-18 21:34:48 -@@ -7,6 +7,7 @@ - #include "bi_debug.h" - #include "bi_builder.h" - #include "compiler.h" -+#include "pan_compiler.h" - #include "nodearray.h" - #include "valhall.h" - -@@ -145,6 +146,8 @@ - lcra_solve(struct lcra_state *l) - { - for (unsigned step = 0; step < l->node_count; ++step) { -+ if ((step & 1023) == 1023) -+ PAN_COMPILE_CHECK(); - if (l->solutions[step] != ~0) - continue; - if (l->affinity[step] == 0) -@@ -262,7 +265,11 @@ - bi_worklist_push_tail(&worklist, block); - } - -+ unsigned pops = 0; - while (!u_worklist_is_empty(&worklist)) { -+ if ((++pops & 255) == 0) -+ PAN_COMPILE_CHECK(); -+ - /* Pop off in reverse order since liveness is backwards */ - bi_block *blk = bi_worklist_pop_tail(&worklist); - -@@ -1295,6 +1302,7 @@ - - /* Otherwise, use the register file and spill until we succeed */ - while (!success && ((iter_count++) < max_iters)) { -+ PAN_COMPILE_CHECK(); - l = bi_allocate_registers(ctx, &success, true); - - if (success) { -diff -ru a/src/panfrost/compiler/pan_compiler.c b/src/panfrost/compiler/pan_compiler.c ---- a/src/panfrost/compiler/pan_compiler.c 2026-11-18 21:34:48 -+++ b/src/panfrost/compiler/pan_compiler.c 2026-11-18 21:34:48 -@@ -4,6 +4,28 @@ - */ - - #include "pan_compiler.h" -+#ifdef __APPLE__ -+#include -+ -+__thread jmp_buf *pan_compile_abort_jmp = NULL; -+__thread uint64_t pan_compile_deadline_ns = 0; -+ -+void -+pan_compile_check_deadline(void) -+{ -+ if (!pan_compile_abort_jmp || !pan_compile_deadline_ns) -+ return; -+ -+ struct timespec t; -+ clock_gettime(CLOCK_MONOTONIC, &t); -+ uint64_t now = (uint64_t)t.tv_sec * 1000000000ull + (uint64_t)t.tv_nsec; -+ if (now > pan_compile_deadline_ns) { -+ jmp_buf *target = pan_compile_abort_jmp; -+ pan_compile_abort_jmp = NULL; -+ longjmp(*target, 1); -+ } -+} -+#endif - #include "pan_nir.h" - - #include "bifrost/bi_debug.h" -diff -ru a/src/panfrost/compiler/pan_compiler.h b/src/panfrost/compiler/pan_compiler.h ---- a/src/panfrost/compiler/pan_compiler.h 2026-11-18 21:34:48 -+++ b/src/panfrost/compiler/pan_compiler.h 2026-11-18 21:34:48 -@@ -81,6 +81,23 @@ - struct util_dynarray *binary, - struct pan_shader_info *info); - -+#ifdef __APPLE__ -+#include -+#include -+/* Starbite: a wall-clock budget for one compile on the calling thread. A caller that -+ * wants one (PanVK's panvk_compile_nir) sets pan_compile_abort_jmp and -+ * pan_compile_deadline_ns (CLOCK_MONOTONIC, ns) around pan_shader_compile(); the register -+ * allocator, whose spill rounds are what a runaway compile is made of, calls -+ * PAN_COMPILE_CHECK() and past the deadline the compile is left by longjmp (what it -+ * allocated is lost: a few MB, once per abandoned shader). */ -+extern __thread jmp_buf *pan_compile_abort_jmp; -+extern __thread uint64_t pan_compile_deadline_ns; -+void pan_compile_check_deadline(void); -+#define PAN_COMPILE_CHECK() pan_compile_check_deadline() -+#else -+#define PAN_COMPILE_CHECK() ((void)0) -+#endif -+ - #define PAN_PRINTF_BUFFER_SIZE 16384 - - /* Any address with the top bit set is treated OOB by the hardware when -diff -ru a/src/panfrost/vulkan/panvk_vX_shader.c b/src/panfrost/vulkan/panvk_vX_shader.c ---- a/src/panfrost/vulkan/panvk_vX_shader.c 2026-11-18 21:34:48 -+++ b/src/panfrost/vulkan/panvk_vX_shader.c 2026-11-18 21:34:48 -@@ -1060,8 +1060,44 @@ - clock_gettime(CLOCK_MONOTONIC, &t0); - #endif - struct util_dynarray binary = UTIL_DYNARRAY_INIT; -+#ifdef __APPLE__ -+ /* Starbite: a compile that runs past STARBITE_PANVK_COMPILE_BUDGET_MS (default 90 s, 0 -+ * for none) is abandoned and its pipeline fails like a refused one (an iOS 27 Preferences -+ * fragment shader of 3023 NIR instructions never ended, the process at 130% for minutes). -+ * The Metal frontend compiles on a worker and times its callers out, so what an abandoned -+ * compile costs is the worker's CPU for the budget, once (the frontend does not retry a -+ * pipeline that failed after 10 s for 10 minutes). */ -+ jmp_buf budget_jmp; -+ { -+ const char *benv = getenv("STARBITE_PANVK_COMPILE_BUDGET_MS"); -+ unsigned long budget_ms = benv ? strtoul(benv, NULL, 0) : 90000; -+ if (budget_ms) { -+ pan_compile_deadline_ns = (uint64_t)t0.tv_sec * 1000000000ull + -+ (uint64_t)t0.tv_nsec + budget_ms * 1000000ull; -+ if (setjmp(budget_jmp)) { -+ pan_compile_abort_jmp = NULL; -+ pan_compile_deadline_ns = 0; -+ util_dynarray_fini(&binary); -+ char msg[160]; -+ int fd = open("/dev/console", O_WRONLY | O_NOCTTY); -+ int n = snprintf(msg, sizeof(msg), -+ "PANVK: %s shader %.60s abandoned after %lu ms (compile budget)\n", -+ _mesa_shader_stage_to_string(nir->info.stage), -+ nir->info.name ? nir->info.name : "?", budget_ms); -+ if (fd >= 0) { -+ write(fd, msg, n > 0 ? (size_t)n : 0); -+ close(fd); -+ } -+ return VK_ERROR_OUT_OF_HOST_MEMORY; -+ } -+ pan_compile_abort_jmp = &budget_jmp; -+ } -+ } -+#endif - pan_shader_compile(nir, &input, &binary, &shader->info); - #ifdef __APPLE__ -+ pan_compile_abort_jmp = NULL; -+ pan_compile_deadline_ns = 0; - clock_gettime(CLOCK_MONOTONIC, &t1); - { - long ms = (t1.tv_sec - t0.tv_sec) * 1000 + (t1.tv_nsec - t0.tv_nsec) / 1000000; diff --git a/tools/ios27/metal/mk-bundle27.sh b/tools/ios27/metal/mk-bundle27.sh index cd837ed96..9ddfd09d4 100755 --- a/tools/ios27/metal/mk-bundle27.sh +++ b/tools/ios27/metal/mk-bundle27.sh @@ -14,7 +14,7 @@ rm -rf "$B" && mkdir -p "$B" $W/sym $W/stage # the frontend and HotMetal2 are built from a copy with patches/frontend-dexr.diff, hotmetal2-diag.diff, frontend-flush.diff and # frontend-turned.diff, frontend-dexr-surfaces.diff, frontend-nocopy-diag.diff, frontend-passlog.diff and frontend-nocopy-readonly.diff applied, in that order (NOPATCH=1: as they are in tools/ios27/metal2) rm -rf $W/src27 && mkdir -p $W/src27/tools/ios27 && cp -R tools/ios27/metal2 $W/src27/tools/ios27/ -[ -n "${NOPATCH:-}" ] || for p in frontend-dexr hotmetal2-diag frontend-flush frontend-turned frontend-dexr-surfaces frontend-nocopy-diag frontend-passlog frontend-nocopy-readonly frontend-async-compile hotmetal2-shared-cache; do patch -s -p1 -d $W/src27 < tools/ios27/metal/patches/$p.diff || exit 1; done +[ -n "${NOPATCH:-}" ] || for p in frontend-dexr hotmetal2-diag frontend-flush frontend-turned frontend-dexr-surfaces frontend-nocopy-diag frontend-passlog frontend-nocopy-readonly; do patch -s -p1 -d $W/src27 < tools/ios27/metal/patches/$p.diff || exit 1; done M2=$W/src27/tools/ios27/metal2 # frontend-turned.diff includes TurnShaders.inc: tools/ios27/metal2/TurnShaders.metal (the pass that writes a turned texture into its # surface) compiled by the same Xcode's Metal compiler and held as bytes diff --git a/tools/ios27/metal/mk-panvk27.sh b/tools/ios27/metal/mk-panvk27.sh index 4c805cfc7..60843e7ed 100755 --- a/tools/ios27/metal/mk-panvk27.sh +++ b/tools/ios27/metal/mk-panvk27.sh @@ -12,8 +12,7 @@ SB=${SB:-$(cd "$(dirname "$0")/../../.." && pwd)} # The libvulkan_panfrost.dylib of phone 1's c2 bundle (cc241e55..., 14 November) was built from exactly these patches, with the # shim of that day, except for the two braces mesa26-panvk-batch-stats.patch restores in panvk_vX_cmd_query.c. PATCHES="mesa26-panvk-ios mesa26-panvk-null-vs mesa26-panvk-shader-size-cap mesa26-bifrost-ra-multispill mesa26-panvk-host-images - mesa26-panvk-tiler-heap-halves mesa26-panvk-batch-stats mesa26-panvk-cpu-desc-copy mesa26-panvk-cpu-minmax mesa26-panvk-cpu-direct - mesa26-panvk-compile-budget" + mesa26-panvk-tiler-heap-halves mesa26-panvk-batch-stats mesa26-panvk-cpu-desc-copy mesa26-panvk-cpu-minmax mesa26-panvk-cpu-direct" # mesa26-panvk-cpu-desc-copy (18 November): the descriptor copies on the CPU at submission with STARBITE_PANVK_CPU_DESC=1 (off # without it), and the draw and copy counters of the shim's statistics line (docs/ios27/PERFORMANCE.md section 7). # mesa26-panvk-cpu-direct (18 November): with STARBITE_PANVK_CPU_DIRECT=1 an indexed draw whose range the CPU found is recorded diff --git a/tools/ios27/metal/patches/frontend-async-compile.diff b/tools/ios27/metal/patches/frontend-async-compile.diff deleted file mode 100644 index 2b6576573..000000000 --- a/tools/ios27/metal/patches/frontend-async-compile.diff +++ /dev/null @@ -1,290 +0,0 @@ ---- a/tools/ios27/metal2/NativeBridgeMetalFrontend.mm 2026-11-18 21:32:59 -+++ b/tools/ios27/metal2/NativeBridgeMetalFrontend.mm 2026-11-18 21:33:27 -@@ -13,6 +13,10 @@ - #import - #include - #include -+#include -+#include -+#include -+#include - #include - #include - #include -@@ -2204,21 +2208,161 @@ - } - @end - -+ -+// ---- pipeline compiles off the caller's thread -------------------------------------------------------------------------- -+// PanVK compiles a pipeline's shaders on the calling thread: one of iOS 27's compute shaders is 24 522 NIR instructions and -+// took 25 s of a Mali-G72's CPU, and a Preferences fragment shader never ended. On backboardd's or an app's main thread that -+// is the UI frozen for all of it. So a compile runs on a worker (at most three at once, QOS utility); the synchronous -+// creation calls wait for it at most STARBITE_M2V_COMPILE_TIMEOUT_MS (default 5000, 0 = wait for ever) and then fail like any -+// refused pipeline (the caller goes without the effect, or asks again); the compile goes on in the background, its result -+// waits for the next request of the same pipeline (key: the functions' bit-code hashes and the state) and the runtime's -+// caches keep it. The completion-handler creation calls were synchronous too (the handler ran after the compile, on the -+// caller's thread): they now return at once and compile on the worker with no limit. -+static dispatch_queue_t compileQueue() { -+ static dispatch_queue_t queue;static dispatch_once_t once; -+ dispatch_once(&once,^{queue=dispatch_queue_create("starbite.m2v.compile",dispatch_queue_attr_make_with_qos_class(DISPATCH_QUEUE_CONCURRENT,QOS_CLASS_UTILITY,0));}); -+ return queue; -+} -+static uint64_t compileTimeoutMs() { -+ static uint64_t value=~0ull; -+ if(value==~0ull){const char *e=getenv("STARBITE_M2V_COMPILE_TIMEOUT_MS");value=e?strtoull(e,nullptr,0):5000;} -+ return value; -+} -+static thread_local bool tOnCompileQueue=false; -+struct CompileJob { -+ std::mutex lock;std::condition_variable done; -+ bool finished=false,timedOut=false,render=false; -+ void *handle=nullptr;std::string error; -+ uint64_t finishedUs=0; -+}; -+static std::mutex gJobsLock; -+static std::unordered_map> gJobs; // the compiles that outlived their caller's wait -+static std::atomic gBackground{0}; -+static std::unordered_map gSlowFailures; // pipelines whose compile failed after more than 10 s: not tried again for 10 minutes -+static void releaseJobHandle(CompileJob *job) { -+ if(!job->handle)return; -+ if(job->render)nmb_render_pipeline_release((NmbRenderPipeline *)job->handle);else nmb_compute_pipeline_release((NmbComputePipeline *)job->handle); -+ job->handle=nullptr; -+} -+static void *runCompile(const std::function &make,std::string &error) { -+ static dispatch_semaphore_t slots;static dispatch_once_t once; -+ dispatch_once(&once,^{slots=dispatch_semaphore_create(3);}); -+ dispatch_semaphore_wait(slots,DISPATCH_TIME_FOREVER); -+ void *handle=make(error); -+ dispatch_semaphore_signal(slots); -+ return handle; -+} -+// `make` the pipeline: the handle, or null with `error`; `late` says the wait ran out (the compile goes on, `error` says so). -+static void *compileWithin(uint64_t key,bool render,std::function make,std::string &error,bool &late) { -+ late=false; -+ uint64_t limit=compileTimeoutMs(); -+ if(tOnCompileQueue || limit==0)return runCompile(make,error); -+ { -+ std::lock_guard guard(gJobsLock); -+ auto failed=gSlowFailures.find(key); -+ if(failed!=gSlowFailures.end()) { -+ if(nowUs()-failed->second<600000000ull){error="an earlier compile of this pipeline failed after a long time; not tried again for 10 minutes";return nullptr;} -+ gSlowFailures.erase(failed); -+ } -+ } -+ std::shared_ptr job; -+ { -+ std::lock_guard guard(gJobsLock); -+ auto found=gJobs.find(key); -+ if(found!=gJobs.end()) { -+ job=found->second; -+ std::lock_guard g(job->lock); -+ if(job->finished) { // the compile this pipeline timed out on is done: its result is for this request -+ gJobs.erase(found); -+ void *handle=job->handle;job->handle=nullptr;error=job->error; -+ return handle; -+ } -+ limit=std::min(limit,100); // still running: not another compile of it, a short wait for it -+ } -+ } -+ bool own=!job; -+ if(own) { -+ job=std::make_shared();job->render=render; -+ dispatch_async(compileQueue(),^{ -+ const uint64_t began=nowUs();std::string err;void *handle=runCompile(make,err); -+ if(!handle && nowUs()-began>10000000ull){std::lock_guard g(gJobsLock);gSlowFailures[key]=nowUs();} -+ bool abandoned; -+ {std::lock_guard g(job->lock);job->handle=handle;job->error=err;job->finished=true;job->finishedUs=nowUs();abandoned=job->timedOut;} -+ if(abandoned)gBackground--; -+ job->done.notify_all(); -+ }); -+ } -+ if(gBackground.load()>=4)limit=std::min(limit,200); // a backlog of slow compiles: ask, do not wait -+ std::unique_lock hold(job->lock); -+ if(job->done.wait_for(hold,std::chrono::milliseconds(limit),[&]{return job->finished;})) { -+ void *handle=job->handle;job->handle=nullptr;error=job->error; -+ return handle; -+ } -+ late=true; -+ if(!job->timedOut){job->timedOut=true;gBackground++;} -+ hold.unlock(); -+ { -+ std::lock_guard guard(gJobsLock); -+ gJobs[key]=job; -+ if(gJobs.size()>64) { // results nobody asked for again: the oldest finished ones go -+ const uint64_t now=nowUs(); -+ for(auto it=gJobs.begin();it!=gJobs.end();) { -+ std::lock_guard g(it->second->lock); -+ if(it->second->finished && now-it->second->finishedUs>600000000ull){releaseJobHandle(it->second.get());it=gJobs.erase(it);}else ++it; -+ } -+ } -+ } -+ error="the compile is still running after "+std::to_string(limit)+" ms (it goes on in the background; ask again)"; -+ return nullptr; -+} -+static uint64_t mixKey(uint64_t h,uint64_t v) {return (h^v)*1099511628211ull;} -+static uint64_t functionKey(id function) { -+ uint64_t h=1469598103934665603ull; -+ if(![function isKindOfClass:[NBFunction class]])return mixKey(h,(uintptr_t)function); -+ uint64_t a,b;memcpy(&a,((NBFunction *)function)->_bitCodeHash,8);memcpy(&b,((NBFunction *)function)->_bitCodeHash+8,8); -+ return mixKey(mixKey(h,a),b); -+} -+static uint64_t renderKey(MTLRenderPipelineDescriptor *d,MTLPipelineOption options) { -+ uint64_t h=mixKey(functionKey(d.vertexFunction),functionKey(d.fragmentFunction)); -+ for(NSUInteger i=0;i<8;++i) { -+ MTLRenderPipelineColorAttachmentDescriptor *c=d.colorAttachments[i]; -+ h=mixKey(h,c.pixelFormat);h=mixKey(h,(c.writeMask<<1)|(c.blendingEnabled?1:0)); -+ h=mixKey(h,(c.sourceRGBBlendFactor<<16)|(c.destinationRGBBlendFactor<<8)|c.rgbBlendOperation); -+ h=mixKey(h,(c.sourceAlphaBlendFactor<<16)|(c.destinationAlphaBlendFactor<<8)|c.alphaBlendOperation); -+ } -+ h=mixKey(h,d.depthAttachmentPixelFormat);h=mixKey(h,d.stencilAttachmentPixelFormat);h=mixKey(h,d.rasterSampleCount); -+ h=mixKey(h,d.alphaToCoverageEnabled);h=mixKey(h,d.alphaToOneEnabled);h=mixKey(h,options); -+ return h; -+} -+static id nbRefused(NSError **error,NSString *what,const std::string &why) { -+ NSString *text=[NSString stringWithFormat:@"%@: %s",what,why.c_str()]; -+ noteProblem(text);if(error)*error=nbError(text); -+ return nil; -+} -+ - static bool pipelineArchives(NSArray *list,std::vector &archives,NSError **error); - // A render pipeline of `d`, its colors in `keep` (bit i, color i) as the descriptor asks and the others through dexr(). - // A pipeline that dexr() changed keeps a copy of the descriptor for sweetExactPipeline. - static NBRenderPipeline *sweetRenderPipeline(MTLRenderPipelineDescriptor *d,MTLPipelineOption options,unsigned keep,NSError **error) { -- auto *b=new BuiltRenderDescriptor();unsigned held=0; -- if(!renderDescriptor(d,b,error,keep,&held)){delete b;return nil;} -+ auto b=std::make_shared();unsigned held=0; -+ if(!renderDescriptor(d,b.get(),error,keep,&held))return nil; - std::vector archives; -- if(!pipelineArchives(d.binaryArchives,archives,error)){delete b;return nil;} -- const NmbPipelineOptions creation={(uint64_t)options,archives.data(),archives.size()}; -+ if(!pipelineArchives(d.binaryArchives,archives,error))return nil; - const uint64_t pipelineBegan=nowUs(); - const bool logJournal=!gJournalPath.empty(); - const NBCost costBegan=logJournal?NBCost::now():NBCost{}; -- NmbRenderPipeline *handle=nmb_render_pipeline_new_with_options(gDevice,&b->raw,&creation);delete b;gTimePipeline.add(pipelineBegan); -- if(logJournal)journalRenderPipeline(d,pipelineBegan,handle?nullptr:nmb_last_error(),NBCost::now().since(costBegan)); -- if(!handle)return failWith(error,@"render pipeline",nil); -+ NSArray *archiveObjects=[d.binaryArchives copy]; // kept by the compile (it may outlive this call) -+ id vertexFunction=d.vertexFunction,fragmentFunction=d.fragmentFunction; // and the functions -+ std::string why;bool late=false; -+ NmbRenderPipeline *handle=(NmbRenderPipeline *)compileWithin(mixKey(renderKey(d,options),keep),true,[b,archives,archiveObjects,vertexFunction,fragmentFunction,options](std::string &err)->void * { -+ (void)archiveObjects;(void)vertexFunction;(void)fragmentFunction; -+ const NmbPipelineOptions creation={(uint64_t)options,archives.data(),archives.size()}; -+ NmbRenderPipeline *made=nmb_render_pipeline_new_with_options(gDevice,&b->raw,&creation); -+ if(!made)err=nmb_last_error(); -+ return made; -+ },why,late);gTimePipeline.add(pipelineBegan); -+ if(logJournal)journalRenderPipeline(d,pipelineBegan,handle?nullptr:why.c_str(),NBCost::now().since(costBegan)); -+ if(!handle){NBLOG(@"native-metal: render pipeline refused: %s",why.c_str());return (NBRenderPipeline *)nbRefused(error,@"render pipeline",why);} - NBRenderPipeline *state=[NBRenderPipeline new];state.handle=handle;gPipelines++;if(error)*error=nil; - state.passName=d.fragmentFunction.name?:(d.label?:@"-"); - if(held){state.dexrColors=held;state.dexrDescriptor=[d copy];state.dexrOptions=options;} -@@ -2688,10 +2832,15 @@ - reflection:(MTLAutoreleasedComputePipelineReflection *)reflection error:(NSError **)error { - if(reflection)*reflection=nil; - if(![(id)function isKindOfClass:[NBFunction class]]){if(error)*error=nbError(@"foreign function");return nil;} -- const NmbPipelineOptions creation={(uint64_t)options,nullptr,0}; - const uint64_t pipelineBegan=nowUs(); -- NmbComputePipeline *handle=nmb_compute_pipeline_new_with_options(gDevice,[(NBFunction *)function handle],&creation);gTimePipeline.add(pipelineBegan); -- if(!handle){NBLOG(@"native-metal: compute pipeline (%@) refused: %@",[(NBFunction *)function name],lastError());return failWith>(error,@"compute pipeline",nil);} -+ NBFunction *fn=(NBFunction *)function;std::string why;bool late=false; -+ NmbComputePipeline *handle=(NmbComputePipeline *)compileWithin(mixKey(functionKey(fn),options),false,[fn,options](std::string &err)->void * { -+ const NmbPipelineOptions creation={(uint64_t)options,nullptr,0}; -+ NmbComputePipeline *made=nmb_compute_pipeline_new_with_options(gDevice,[fn handle],&creation); -+ if(!made)err=nmb_last_error(); -+ return made; -+ },why,late);gTimePipeline.add(pipelineBegan); -+ if(!handle){NBLOG(@"native-metal: compute pipeline (%@) refused: %s",[fn name],why.c_str());return nbRefused(error,@"compute pipeline",why);} - NBComputePipeline *state=[NBComputePipeline new];state.handle=handle;state.passName=[(NBFunction *)function name];gPipelines++;if(error)*error=nil; - return (id)state; - } -@@ -2701,45 +2850,69 @@ - - (id)newComputePipelineStateWithDescriptor:(MTLComputePipelineDescriptor *)d options:(MTLPipelineOption)options - reflection:(MTLAutoreleasedComputePipelineReflection *)reflection error:(NSError **)error { - if(reflection)*reflection=nil; -- BuiltComputeDescriptor b;if(!computeDescriptor(d,&b,error))return nil; -+ auto b=std::make_shared();if(!computeDescriptor(d,b.get(),error))return nil; - std::vector archives; - if(!pipelineArchives(d.binaryArchives,archives,error))return nil; -- const NmbPipelineOptions creation={(uint64_t)options,archives.data(),archives.size()}; - const uint64_t pipelineBegan=nowUs(); -- NmbComputePipeline *handle=nmb_compute_pipeline_new_with_descriptor(gDevice,&b.raw,&creation);gTimePipeline.add(pipelineBegan); -- if(!handle){NBLOG(@"native-metal: compute pipeline (%@) refused: %@",[(NBFunction *)d.computeFunction name],lastError());return failWith>(error,@"compute pipeline",nil);} -+ NSArray *archiveObjects=[d.binaryArchives copy]; // kept by the compile (it may outlive this call) -+ NBFunction *fn=(NBFunction *)d.computeFunction;std::string why;bool late=false; -+ uint64_t key=mixKey(mixKey(functionKey(fn),options),d.maxTotalThreadsPerThreadgroup); -+ NmbComputePipeline *handle=(NmbComputePipeline *)compileWithin(key,false,[b,archives,archiveObjects,fn,options](std::string &err)->void * { -+ (void)archiveObjects;(void)fn; -+ const NmbPipelineOptions creation={(uint64_t)options,archives.data(),archives.size()}; -+ NmbComputePipeline *made=nmb_compute_pipeline_new_with_descriptor(gDevice,&b->raw,&creation); -+ if(!made)err=nmb_last_error(); -+ return made; -+ },why,late);gTimePipeline.add(pipelineBegan); -+ if(!handle){NBLOG(@"native-metal: compute pipeline (%@) refused: %s",[fn name],why.c_str());return nbRefused(error,@"compute pipeline",why);} - NBComputePipeline *state=[NBComputePipeline new];state.handle=handle;state.label=d.label;state.passName=d.computeFunction.name?:d.label;gPipelines++; - if(error)*error=nil;return (id)state; - } - - (void)newComputePipelineStateWithFunction:(id)function completionHandler:(void (^)(id,NSError *))completionHandler { -- NSError *error=nil;id state=[(id)self newComputePipelineStateWithFunction:function error:&error]; -- if(completionHandler)dispatch_async(dispatch_get_global_queue(QOS_CLASS_USER_INITIATED,0),^{completionHandler(state,error);}); -+ dispatch_async(compileQueue(),^{ -+ tOnCompileQueue=true;NSError *error=nil;id state=[(id)self newComputePipelineStateWithFunction:function error:&error];tOnCompileQueue=false; -+ if(completionHandler)completionHandler(state,error); -+ }); - } - - (void)newComputePipelineStateWithFunction:(id)function options:(MTLPipelineOption)options completionHandler:(void (^)(id,MTLComputePipelineReflection *,NSError *))completionHandler { -- NSError *error=nil;MTLComputePipelineReflection *reflection=nil; -- id state=[(id)self newComputePipelineStateWithFunction:function options:options reflection:&reflection error:&error]; -- if(completionHandler)dispatch_async(dispatch_get_global_queue(QOS_CLASS_USER_INITIATED,0),^{completionHandler(state,reflection,error);}); -+ dispatch_async(compileQueue(),^{ -+ tOnCompileQueue=true;NSError *error=nil;MTLComputePipelineReflection *reflection=nil; -+ id state=[(id)self newComputePipelineStateWithFunction:function options:options reflection:&reflection error:&error];tOnCompileQueue=false; -+ if(completionHandler)completionHandler(state,reflection,error); -+ }); - } - - (void)newComputePipelineStateWithDescriptor:(MTLComputePipelineDescriptor *)d completionHandler:(void (^)(id,NSError *))completionHandler { -- NSError *error=nil;id state=[self newComputePipelineStateWithDescriptor:d error:&error]; -- if(completionHandler)dispatch_async(dispatch_get_global_queue(QOS_CLASS_USER_INITIATED,0),^{completionHandler(state,error);}); -+ MTLComputePipelineDescriptor *copy=[d copy]; // the caller may change its descriptor as soon as this returns -+ dispatch_async(compileQueue(),^{ -+ tOnCompileQueue=true;NSError *error=nil;id state=[self newComputePipelineStateWithDescriptor:copy error:&error];tOnCompileQueue=false; -+ if(completionHandler)completionHandler(state,error); -+ }); - } - - (void)newComputePipelineStateWithDescriptor:(MTLComputePipelineDescriptor *)d options:(MTLPipelineOption)options completionHandler:(void (^)(id,MTLComputePipelineReflection *,NSError *))completionHandler { -- NSError *error=nil;MTLComputePipelineReflection *reflection=nil; -- id state=[(id)self newComputePipelineStateWithDescriptor:d options:options reflection:&reflection error:&error]; -- if(completionHandler)dispatch_async(dispatch_get_global_queue(QOS_CLASS_USER_INITIATED,0),^{completionHandler(state,reflection,error);}); -+ MTLComputePipelineDescriptor *copy=[d copy]; -+ dispatch_async(compileQueue(),^{ -+ tOnCompileQueue=true;NSError *error=nil;MTLComputePipelineReflection *reflection=nil; -+ id state=[(id)self newComputePipelineStateWithDescriptor:copy options:options reflection:&reflection error:&error];tOnCompileQueue=false; -+ if(completionHandler)completionHandler(state,reflection,error); -+ }); - } - - (id)newRenderPipelineStateWithDescriptor:(MTLRenderPipelineDescriptor *)d error:(NSError **)error { - return [(id)self newRenderPipelineStateWithDescriptor:d options:MTLPipelineOptionNone reflection:nil error:error]; - } - - (void)newRenderPipelineStateWithDescriptor:(MTLRenderPipelineDescriptor *)d completionHandler:(void (^)(id,NSError *))completionHandler { -- NSError *error=nil;id state=[(id)self newRenderPipelineStateWithDescriptor:d error:&error]; -- if(completionHandler)dispatch_async(dispatch_get_global_queue(QOS_CLASS_USER_INITIATED,0),^{completionHandler(state,error);}); -+ MTLRenderPipelineDescriptor *copy=[d copy]; // the caller may change its descriptor as soon as this returns -+ dispatch_async(compileQueue(),^{ -+ tOnCompileQueue=true;NSError *error=nil;id state=[(id)self newRenderPipelineStateWithDescriptor:copy error:&error];tOnCompileQueue=false; -+ if(completionHandler)completionHandler(state,error); -+ }); - } - - (void)newRenderPipelineStateWithDescriptor:(MTLRenderPipelineDescriptor *)d options:(MTLPipelineOption)options completionHandler:(void (^)(id,MTLRenderPipelineReflection *,NSError *))completionHandler { -- NSError *error=nil;MTLRenderPipelineReflection *reflection=nil; -- id state=[(id)self newRenderPipelineStateWithDescriptor:d options:options reflection:&reflection error:&error]; -- if(completionHandler)dispatch_async(dispatch_get_global_queue(QOS_CLASS_USER_INITIATED,0),^{completionHandler(state,reflection,error);}); -+ MTLRenderPipelineDescriptor *copy=[d copy]; -+ dispatch_async(compileQueue(),^{ -+ tOnCompileQueue=true;NSError *error=nil;MTLRenderPipelineReflection *reflection=nil; -+ id state=[(id)self newRenderPipelineStateWithDescriptor:copy options:options reflection:&reflection error:&error];tOnCompileQueue=false; -+ if(completionHandler)completionHandler(state,reflection,error); -+ }); - } - - (id)newRenderPipelineStateWithDescriptor:(MTLRenderPipelineDescriptor *)d options:(MTLPipelineOption)options - reflection:(MTLAutoreleasedRenderPipelineReflection *)reflection error:(NSError **)error { diff --git a/tools/ios27/metal/patches/hotmetal2-shared-cache.diff b/tools/ios27/metal/patches/hotmetal2-shared-cache.diff deleted file mode 100644 index e7a20d21d..000000000 --- a/tools/ios27/metal/patches/hotmetal2-shared-cache.diff +++ /dev/null @@ -1,31 +0,0 @@ ---- a/tools/ios27/metal2/HotMetal2.mm 2026-11-18 21:32:59 -+++ b/tools/ios27/metal2/HotMetal2.mm 2026-11-18 21:33:27 -@@ -212,11 +212,24 @@ - setenv("PAN_I_WANT_A_BROKEN_VULKAN_DRIVER", "1", 1); - /* Bifrost v6 has most of Vulkan 1.2 as extensions but reports 1.0; metal2vulkan's floor is 1.2 */ - setenv("MESA_VK_VERSION_OVERRIDE", "1.2", 0); -+ /* -+ * One shader cache for every process of a user (not one per program): the translated shaders and PanVK's pipeline data are -+ * named by their content, so Preferences, SpringBoard and backboardd find what the first of them compiled, and the cache -+ * a root step ships warm (fullroot/steps.d/81-m2v-cache.sh) is the one they all read. The runtime's files are owner-only, -+ * so the directory is the user's: /var/local/m2v-cache/u. STARBITE_M2V_CACHE (mtl2-env) names another directory. -+ */ - char cache[256]; -- mkdir("/var/local/m2v-cache", 0777); -- chmod("/var/local/m2v-cache", 0777); -- snprintf(cache, sizeof(cache), "/var/local/m2v-cache/%s", getprogname()); -- mkdir(cache, 0777); -+ { -+ const char *named = getenv("STARBITE_M2V_CACHE"); -+ if (named && named[0] == '/') { -+ strlcpy(cache, named, sizeof(cache)); -+ } else { -+ mkdir("/var/local/m2v-cache", 0777); -+ chmod("/var/local/m2v-cache", 0777); -+ snprintf(cache, sizeof(cache), "/var/local/m2v-cache/u%u", (unsigned)geteuid()); -+ } -+ mkdir(cache, 0700); -+ } - if (access(cache, W_OK) != 0) { - /* a sandboxed process: its own caches directory */ - char dir[256]; diff --git a/tools/ios27/metal2/harvest-m2v-cache.sh b/tools/ios27/metal2/harvest-m2v-cache.sh deleted file mode 100755 index ba3ed3b74..000000000 --- a/tools/ios27/metal2/harvest-m2v-cache.sh +++ /dev/null @@ -1,32 +0,0 @@ -#!/bin/bash -# harvest-m2v-cache.sh PULLED OUT : a warm cache for steps.d/81-m2v-cache.sh from a phone's /private/var/local/m2v-cache. -# PULLED = that directory, copied off the phone after a warm-up with the bundle that will ship (open Settings, Display & Brightness -# / Appearance, the app switcher, Control Center, a few apps; wait until `PANVK: ... compiled` lines stop). Keeps, per user -# directory (u/), the BUILD directories under v1/ that were written last (the running bundle's: its translated shaders) -# and the content directory (PanVK's pipeline data, valid for as long as PanVK is the same); drops older builds' directories. -# OUT (default tools/ios27/fullroot/warm-m2v) is what the step installs; it is not in git (cache entries are build artefacts). -set -euo pipefail -SRC=${1:?pulled m2v-cache directory} -OUT=${2:-$(cd "$(dirname "$0")/../fullroot" && pwd)/warm-m2v} -# the content directory's name is fixed: sha256 of the tag the runtime names it by -CONTENT=$(printf 'metal2vulkan shader cache content\0' | shasum -a 256 | cut -d' ' -f1) -rm -rf "$OUT"; mkdir -p "$OUT" -n=0 -for u in "$SRC"/u[0-9]*/; do - [[ -d $u/v1 ]] || continue - name=$(basename "$u") - mkdir -p "$OUT/$name/v1" - # newest build directory by the newest file in it - newest=""; best=0 - for b in "$u"v1/*/; do - id=$(basename "$b"); [[ $id == "$CONTENT" ]] && continue - t=$(find "$b" -type f -exec stat -f %m {} + 2>/dev/null | sort -n | tail -1); t=${t:-0} - if (( t >= best )); then best=$t; newest=$id; fi - done - [[ -n $newest ]] && cp -R "$u/v1/$newest" "$OUT/$name/v1/$newest" - [[ -d $u/v1/$CONTENT ]] && cp -R "$u/v1/$CONTENT" "$OUT/$name/v1/$CONTENT" - n=$((n + 1)) - echo "harvest: $name: build ${newest:-none}, content $([[ -d $u/v1/$CONTENT ]] && echo yes || echo NO), $(find "$OUT/$name" -type f | wc -l) files" -done -(( n )) || { echo "harvest: no u/ directory with v1/ in $SRC (a phone that ran the older per-program plug-in has backboardd/ instead: boot the new bundle first)"; exit 1; } -du -sk "$OUT" | awk '{print "harvest: " $1 " KiB in '"$OUT"'"}'