Skip to content

build(scripts): add a performance check and fix what it measured - #85

Open
Mastersam07 wants to merge 17 commits into
devfrom
test/performance-check
Open

Mastersam07 wants to merge 17 commits into
devfrom
test/performance-check

Conversation

@Mastersam07

@Mastersam07 Mastersam07 commented Sep 28, 2026 •

Copy link
Copy Markdown
Owner

What: scripts/performance-check.sh measures what a device window costs, in real windows on booted devices, and compares it with a baseline recorded on the same Mac, one for Low Power Mode and one for normal power. It measures open time (split into sessions, orientation, window and first picture), three reopens (their time, the memory each keeps, and whether the closed window was freed), idle CPU, CPU energy, GPU time and pictures drawn, memory, click to frame latency, a foldable's fold step gaps and CPU, and every booted device's window open at once, idle and then kept busy for three minutes (--busy-minutes) to see whether memory keeps growing. The release steps run it before the dry run.

It also fixes what the check found:

  • The foldable draws once for each change through its own SceneKit renderer. An SCNView drawing on demand draws about 16 more times after every change, so two frames a second from the device kept it drawing 35 times a second. The new view copies the old one's pixel format, colour space and 4x multisampling.
  • Fold steps sleep until the next 16 ms tick instead of 16 ms after each step, so drawing on the main thread no longer stretches them.
  • A closed window's screen view is freed. Its click and gesture handlers captured the view, so every window left its view, drawables and input session behind, about 4 MB each.
  • An ordinary device is asked for its orientation once. Only a foldable repeats its previous answer first, so only a foldable is asked twice.

Verification:

  • swift test: all unit tests pass, including 23 for the comparison and new ones for a closed window being freed (fails without the fix), a single orientation ask on an ordinary device, and the fold step tick.

  • Integration, on an iPhone 17 (iOS 27.0) and an iPhone Duo (iOS 27.1): the foldable suites (presets, rotation, recording across panels, taps, swipes and screenshots open and shut, the body's buttons, poses, tap order and placement) pass on the new foldable view. The iPhone answered right at the first orientation ask on seven turns; asked once, the Duo returned the previous orientation every time, so it keeps the second ask.

  • The new foldable view looks the same: its snapshots and its drawing path matched the old SCNView within 1/255 on every pixel at open, half open and shut, with identical alpha.

  • The check, normal power, against the baseline recorded before the fixes, then a run against the new baseline with no regressions:

    Before After
    Duo idle CPU 9.0 to 9.8% 0.7 to 1.1%
    Duo pictures drawn at rest 34 to 36 a second one per device frame, 1.9 to 2.2
    Duo idle CPU energy / GPU busy 18 to 25 mW / 4.0 to 4.7% 1.6 to 4.7 mW / 0.2 to 0.3%
    Duo fold CPU / step gap median 32 to 36% / 18.1 to 19.2 ms 16 to 22% / 16.1 to 16.6 ms
    iPhone reopen to first picture 374 to 403 ms 230 to 275 ms
    Memory kept by each iPhone reopen, median 4.5 to 4.6 MB 0.1 to 1.6 MB
    Both windows at rest 6.0 to 9.6% CPU 0.7 to 1.2% CPU
  • How the numbers were made honest: energy is the process's CPU energy counter (a GPU bound Metal loop barely moves it), GPU time is the GPU driver's per process count (within a third of Metal's own timing), window memory is the median of a reading every two seconds (a single reading once caught a passing 100 MB rise), and limits were widened only where repeated runs showed the noise: the first open (only a doubling fails), memory growth over the busy minutes (-9 to 13 MB a minute with nothing leaking), and the foldable's reopen memory (moves by up to 50 MB).

Follow-ups:

  • The low power baseline predates the new metrics and the fixes, so in Low Power Mode the check compares only the older metrics, all of which now improve. Recording it needs the Mac in Low Power Mode.
  • While frames arrive, an iPhone window's process holds about 90 MB more for about a second. It is the system presenting a Metal layer: a renderer that only clears does the same, and no layer setting changes it.
  • A single run makes the baseline, so one noisy metric can loosen a limit. Recording from the median of several runs would tighten them.
  • The foldable window is still about 320 MB, mostly SceneKit's own resources after its first render.

…draws

A window that draws while its device sends nothing costs CPU and GPU for no change on screen. The counts make that measurable.
The recorded baseline is this Mac's, taken in Low Power Mode, and shows the foldable window drawing 60 pictures a second at rest while its device sends about two.
@Mastersam07
Mastersam07 force-pushed the test/performance-check branch from daa4e45 to 0f47ec7 Compare September 28, 2026 06:50
It redrew sixty times a second whatever the device sent, so a still screen cost a steady share of the CPU and GPU. SceneKit now draws when the scene changes: a new frame, a pose, a hover. A still screen draws nothing, and each change costs about a quarter of a second of drawing, which is SceneKit's own.
@Mastersam07 Mastersam07 changed the title build(scripts): add a performance check against a per machine baseline build(scripts): add a performance check and draw the foldable on demand Sep 28, 2026
Low Power Mode caps drawing at 60 a second, which hides half of what a window that draws at the display's rate costs on a 120 Hz Mac. The check now compares with the baseline for the mode the Mac is in; the existing one is kept as the Low Power Mode baseline and a normal power one is added.
Energy is read from the process's own CPU energy counter and GPU use from the GPU
driver's per process time, since powermetrics needs admin rights. Every booted
device's window is also opened at once and kept busy, and each window is reopened
a few times, so a leak shows as memory that keeps growing.
A single reading eight seconds after a window opened sometimes caught a passing
rise of about 100 MB in the process, which recorded the iPhone window at 142 MB
instead of about 45.
The click and gesture handlers stored on the screen view captured the view
itself, so every closed window left its view, drawables and input session
behind, about 4 MB each time a window opened.
An SCNView drawing on demand draws about 16 more times after every change, so
two frames a second from the device kept the model drawing 35 times a second.
The model now draws itself through SceneKit's renderer into the same pixel
format, colour space and 4x multisampling, once per change, and on a timer
only while a button lift animates.
Each step slept a fixed 16 ms after its work, so the model now drawing on the main
thread stretched the gap between steps to about 23 ms. Sleeping until the next
tick keeps them 16 ms apart.
@Mastersam07 Mastersam07 changed the title build(scripts): add a performance check and draw the foldable on demand build(scripts): add a performance check and fix what it measured Sep 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant