Skip to content

add mobench support to ProveKit main#450

Merged
dcbuild3r merged 21 commits into
mainfrom
dcbuilder/pr429-main-mobench-fixtures
Jul 20, 2026
Merged

add mobench support to ProveKit main#450
dcbuild3r merged 21 commits into
mainfrom
dcbuilder/pr429-main-mobench-fixtures

Conversation

@dcbuild3r

Copy link
Copy Markdown
Collaborator

Reopened from #429 using the new dcbuilder/* branch naming.

Old PR: #429
Replacement branch: dcbuilder/pr429-main-mobench-fixtures
Old head branch: dcbuild3r/main-mobench-fixtures
Head SHA: e14fa60beadc25bf1e7d238c9bdbf8f6237fa9f2

Previous PR CI / benchmark runs from #429:

Summary

  • add main-based mobench fixtures for passport, OPRF, and P-256 bigcurve
  • run BrowserStack triad profile across Vivo Y21, Pixel 7, S24 and iOS fallback triad
  • fix Android prove timing so fixture clone/setup is outside the measured prove span
  • use jemalloc as the Android fallback allocator under the native C FFI allocator
  • harden Android incomplete-fixture reporting with timeout/build/kill-evidence fields

Android mobench fix note

The previous failing run was https://github.com/worldfnd/provekit/actions/runs/26007468825. The missing Vivo Y21 monolithic cell did not recover a BrowserStack session payload or summary.json; available artifacts only show the BrowserStack fetch timeout after 7200s for build c943753d95fd0b34f5775aa0a3bc6ff58cbcc3ca.

Before/failure memory from that run:

  • Vivo Y21 OPRF: 328 MB process peak, 10.061s
  • Vivo Y21 fragmented age check: 1187 MB process peak, 65.342s
  • Vivo Y21 monolithic age check: no row; BrowserStack timeout/no summary
  • S24 OPRF: 426 MB process peak, 2.809s

What changed:

  • Android and iOS now use the same semantic boundary for prove benches: per-iteration fixture setup happens before the measured closure, and profile_phase("prove") wraps only the prover entry point. Previously prepared.clone() ran inside profile_phase("prove"), so clone cost and peak memory were charged to proving.
  • Android native C FFI fallback allocation now routes through jemalloc instead of Bionic malloc when no host callback/mmap allocator is active.
  • If BrowserStack returns no summary, failure.json now records attempts, fetch timeout seconds, build id, and any LMK/OOM/SIGKILL lines recovered from attempt/device logs.

Latest benchmark numbers were reposted as a PR comment from #429.

Validation

  • cargo fmt --all
  • cargo test -p bench-mobile --lib
  • cargo test -p bench-mobile --test examples_smoke
  • cargo test -p bench-mobile --test passport_smoke
  • cargo check -p provekit-ffi --target aarch64-linux-android with NDK 26.1 aarch64-linux-android34-clang
  • ruby -e 'require "yaml"; YAML.load_file(".github/workflows/mobile-bench-reusable.yml")'
  • git diff --check

Copy link
Copy Markdown
Collaborator Author

Latest mobile benchmark numbers

Reposted from #429 comment #429 (comment).

Source: Mobile Bench PR Auto run 26368378239 on dcbuild3r/main-mobench-fixtures at e14fa60b.
Mobench version: 0.1.42. Artifact summaries report 2 measured runs per function/device plus 1 warmup run.

iOS

Benchmark Device Runs Warmup Mean/run Median p95 Min Max Wall total CPU median/run CPU total CPU/wall Peak growth Peak memory Process peak
OPRF iPhone 16 Pro Max 2 1 766.622ms 766.622ms 775.181ms 758.064ms 775.181ms 1.533s 3.364s 6.728s 438.8% 10.95 MB 10.95 MB 362.28 MB
OPRF iPhone 14 2 1 1.096s 1.096s 1.118s 1.075s 1.118s 2.192s 4.497s 8.994s 410.2% 32.55 MB 32.55 MB 206.95 MB
OPRF iPhone SE 2022 2 1 1.085s 1.085s 1.086s 1.085s 1.086s 2.171s 4.486s 8.972s 413.3% 33.48 MB 33.48 MB 209.44 MB
Passport fragmented iPhone 16 Pro Max 2 1 4.751s 4.751s 4.805s 4.696s 4.805s 9.501s 21.524s 43.049s 453.1% 150.95 MB 150.95 MB 1042.44 MB
Passport fragmented iPhone 14 2 1 6.598s 6.598s 6.625s 6.570s 6.625s 13.195s 28.438s 56.877s 431.0% 264.30 MB 264.30 MB 845.34 MB
Passport fragmented iPhone SE 2022 2 1 7.053s 7.053s 7.233s 6.874s 7.233s 14.106s 30.661s 61.323s 434.7% 265.72 MB 265.72 MB 851.73 MB
Passport complete iPhone 16 Pro Max 2 1 4.499s 4.499s 4.514s 4.484s 4.514s 8.997s 19.933s 39.866s 443.1% 376.08 MB 376.08 MB 1594.42 MB
Passport complete iPhone 14 2 1 7.772s 7.772s 8.889s 6.654s 8.889s 15.543s 28.034s 56.069s 360.7% 487.56 MB 487.56 MB 1220.12 MB
Passport complete iPhone SE 2022 2 1 7.443s 7.443s 7.505s 7.381s 7.505s 14.886s 30.253s 60.507s 406.5% 494.86 MB 494.86 MB 1104.30 MB

Android

Benchmark Device Runs Warmup Mean/run Median p95 Min Max Wall total CPU median/run CPU total CPU/wall Peak growth Peak memory Process peak
OPRF Samsung Galaxy S24 2 1 2.725s 2.725s 2.762s 2.688s 2.762s 5.449s 9.657s 19.315s 354.4% 89.95 MB 89.95 MB 488.93 MB
OPRF Google Pixel 7 2 1 3.900s 3.900s 3.943s 3.858s 3.943s 7.800s 11.518s 23.036s 295.3% 18.23 MB 18.23 MB 237.57 MB
OPRF Samsung Galaxy M32 2 1 6.681s 6.681s 6.926s 6.436s 6.926s 13.362s 22.641s 45.282s 338.9% n/a n/a 352.76 MB
Passport fragmented Samsung Galaxy S24 2 1 14.516s 14.516s 14.697s 14.336s 14.697s 29.033s 54.556s 109.112s 375.8% 202.75 MB 202.75 MB 929.28 MB
Passport fragmented Google Pixel 7 2 1 26.237s 26.237s 28.870s 23.605s 28.870s 52.475s 74.610s 149.220s 284.4% 217.21 MB 217.21 MB 789.64 MB
Passport fragmented Samsung Galaxy M32 2 1 40.629s 40.629s 41.183s 40.075s 41.183s 81.257s 137.225s 274.451s 337.8% 284.89 MB 284.89 MB 859.60 MB
Passport complete Samsung Galaxy S24 2 1 11.250s 11.250s 11.671s 10.829s 11.671s 22.500s 46.655s 93.310s 414.7% 376.77 MB 376.77 MB 1157.10 MB
Passport complete Google Pixel 7 2 1 19.856s 19.856s 22.382s 17.330s 22.382s 39.712s 63.704s 127.408s 320.8% 304.69 MB 304.69 MB 980.12 MB
Passport complete Samsung Galaxy M32 2 1 28.921s 28.921s 30.002s 27.840s 30.002s 57.842s 102.707s 205.414s 355.1% 365.05 MB 365.05 MB 1220.69 MB

Memory growth is the benchmark-reported peak growth above baseline; process peak is the reported process RSS peak.

@github-actions

github-actions Bot commented Jun 3, 2026

Copy link
Copy Markdown

CSP benchmarks

Metric Value
Workflow status [PASS] success
Commit e89795aceded
Run #29726813188
Circuits benchmarked 21
Iterations averaged per circuit 3

Prover time, peak RSS, peak heap, and verifier time are arithmetic means across the iterations. Peak heap comes from the largest peak memory entry in provekit-cli prove's tracing output; peak RSS is reported by /usr/bin/time -v (max-resident-set-size).

Each metric cell shows the current value followed by the percentage delta against the latest successful main run #29429257697. (new) marks circuits absent from the baseline.

Results
Circuit Constraints Witnesses Prover time Peak RSS Peak heap Verifier time Proof size PKP size
ecdsa_p256 143,321 (±0.0%) 258,215 (±0.0%) 1.75 s (-0.2%) 226 MB (±0.0%) 188 MB (±0.0%) 90 ms (±0.0%) 306 KB (-0.2%) 811 KB (±0.0%)
keccak_1024 788,814 (±0.0%) 1,490,812 (±0.0%) 5.11 s (-0.5%) 897 MB (±0.0%) 874 MB (±0.0%) 440 ms (-2.2%) 369 KB (-0.5%) 5.88 MB (±0.0%)
keccak_128 157,522 (±0.0%) 304,529 (±0.0%) 1.22 s (-0.5%) 236 MB (-1.2%) 202 MB (±0.0%) 110 ms (±0.0%) 309 KB (+2.0%) 1.26 MB (±0.0%)
keccak_2048 1,507,374 (±0.0%) 2,840,532 (±0.0%) 10.15 s (-0.2%) 1.70 GB (±0.0%) 1.68 GB (±0.0%) 840 ms (-1.6%) 391 KB (±0.0%) 11.97 MB (±0.0%)
keccak_256 247,782 (±0.0%) 474,010 (±0.0%) 1.40 s (-0.9%) 290 MB (±0.0%) 249 MB (±0.0%) 150 ms (-2.2%) 325 KB (-1.5%) 1.88 MB (±0.0%)
keccak_512 428,126 (±0.0%) 812,944 (±0.0%) 2.63 s (-0.8%) 541 MB (±0.0%) 455 MB (±0.0%) 240 ms (-4.0%) 351 KB (+0.2%) 3.32 MB (±0.0%)
poseidon2_12 479 (±0.0%) 563 (±0.0%) 40 ms (±0.0%) 12.95 MB (+0.2%) 3.26 MB (±0.0%) 72.95 KB (-1.0%) 436 KB (±0.0%)
poseidon2_16 556 (±0.0%) 719 (±0.0%) 40 ms (-20.0%) 13.40 MB (+0.5%) 3.46 MB (±0.0%) 10 ms (±0.0%) 74.47 KB (-0.2%) 530 KB (±0.0%)
poseidon2_2 231 (±0.0%) 278 (±0.0%) 30 ms (-30.7%) 12.02 MB (+0.3%) 2.72 MB (±0.0%) 68.26 KB (-0.9%) 108 KB (±0.0%)
poseidon2_4 529 (±0.0%) 535 (±0.0%) 30 ms (±0.0%) 12.22 MB (±0.0%) 2.86 MB (±0.0%) 73.92 KB (+0.8%) 31.66 KB (±0.0%)
poseidon2_8 363 (±0.0%) 423 (±0.0%) 30 ms (-9.9%) 12.98 MB (+0.6%) 3.15 MB (±0.0%) 70.88 KB (-1.0%) 365 KB (±0.0%)
poseidon_12 504 (±0.0%) 524 (±0.0%) 40 ms (±0.0%) 13.42 MB (+0.3%) 3.53 MB (±0.0%) 10 ms (±0.0%) 73.21 KB (-0.1%) 569 KB (±0.0%)
poseidon_16 609 (±0.0%) 633 (±0.0%) 40 ms (±0.0%) 13.85 MB (-0.2%) 3.98 MB (±0.0%) 10 ms (±0.0%) 73.43 KB (-0.7%) 717 KB (±0.0%)
poseidon_2 240 (±0.0%) 249 (±0.0%) 30 ms (±0.0%) 11.90 MB (±0.0%) 2.68 MB (±0.0%) 66.76 KB (+2.5%) 59.18 KB (±0.0%)
poseidon_4 297 (±0.0%) 309 (±0.0%) 30 ms (-9.9%) 12.61 MB (+0.3%) 3.10 MB (±0.0%) 69.12 KB (+0.3%) 332 KB (±0.0%)
poseidon_8 402 (±0.0%) 418 (±0.0%) 33 ms (±0.0%) 12.90 MB (+0.3%) 3.29 MB (±0.0%) 10 ms (±0.0%) 70.72 KB (-1.7%) 448 KB (±0.0%)
sha256_1024 196,940 (±0.0%) 339,764 (±0.0%) 1.31 s (-1.5%) 267 MB (-1.3%) 236 MB (±0.0%) 160 ms (-2.0%) 315 KB (+0.6%) 1.89 MB (-0.2%)
sha256_128 46,398 (±0.0%) 80,974 (±0.0%) 320 ms (-2.1%) 76.44 MB (-0.1%) 55.50 MB (±0.0%) 40 ms (±0.0%) 263 KB (-11.8%) 506 KB (-0.9%)
sha256_2048 345,399 (±0.0%) 612,724 (±0.0%) 2.60 s (-0.5%) 516 MB (±0.0%) 436 MB (±0.0%) 297 ms (-3.3%) 345 KB (+0.7%) 3.10 MB (+2.2%)
sha256_256 67,904 (±0.0%) 117,944 (±0.0%) 563 ms (-1.7%) 123 MB (+1.1%) 98.23 MB (±0.0%) 60 ms (±0.0%) 280 KB (+1.0%) 709 KB (-0.4%)
sha256_512 110,916 (±0.0%) 191,884 (±0.0%) 700 ms (+1.0%) 153 MB (±0.0%) 126 MB (±0.0%) 93 ms (+3.7%) 293 KB (-0.5%) 1.11 MB (+0.9%)

@dcbuild3r dcbuild3r added the bench Run mobile benchmarks on PRs label Jun 11, 2026
@github-actions

github-actions Bot commented Jun 11, 2026

Copy link
Copy Markdown

Mobench Benchmark Results

Ios Benchmark Results

Benchmark Summary

  • Generated: 2026-07-20T09:23:57.522275362Z
  • Target: iOS
  • Function: multiple
  • Iterations/Warmup: 2 / 1
  • Devices: iPhone 14-16, iPhone 16 Pro Max-18, iPhone SE 2022-15
Device Function Samples Warmup Wall mean / iter Wall total CPU median / iter CPU total CPU / wall Peak growth Process peak
iPhone 14-16 bench_mobile::bench_oprf_prove 2 1 325.611ms 651.222ms 1.043s 2.087s 320.5% 0.25 MB 204.39 MB
iPhone 14-16 bench_mobile::bench_passport_complete_age_check_prove 2 1 4.302s 8.604s 18.066s 36.132s 419.9% 390.64 MB 1223.23 MB
iPhone 14-16 bench_mobile::bench_passport_fragmented_age_check_prove 2 1 3.878s 7.756s 15.697s 31.395s 404.8% 200.83 MB 917.78 MB
iPhone 16 Pro Max-18 bench_mobile::bench_oprf_prove 2 1 271.738ms 543.477ms 775ms 1.550s 285.2% 4.80 MB 363.14 MB
iPhone 16 Pro Max-18 bench_mobile::bench_passport_complete_age_check_prove 2 1 3.005s 6.010s 13.057s 26.115s 434.5% 223.55 MB 1650.97 MB
iPhone 16 Pro Max-18 bench_mobile::bench_passport_fragmented_age_check_prove 2 1 2.579s 5.157s 11.088s 22.176s 430.0% 120.06 MB 1075.41 MB
iPhone SE 2022-15 bench_mobile::bench_oprf_prove 2 1 336.263ms 672.526ms 1.037s 2.074s 308.4% 0.52 MB 197.38 MB
iPhone SE 2022-15 bench_mobile::bench_passport_complete_age_check_prove 2 1 4.459s 8.918s 18.854s 37.708s 422.8% 393.23 MB 1224.80 MB
iPhone SE 2022-15 bench_mobile::bench_passport_fragmented_age_check_prove 2 1 4.292s 8.584s 15.527s 31.055s 361.8% 200.34 MB 869.84 MB

Note: memory growth excludes warmup/baseline retained before the measured iteration.

Android Benchmark Results

Benchmark Summary

  • Generated: 2026-07-20T10:24:29.041296968Z
  • Target: Android
  • Function: multiple
  • Iterations/Warmup: 2 / 1
  • Devices: Google Pixel 7-13.0, Samsung Galaxy M32-11.0, Samsung Galaxy S24-14.0
Device Function Samples Warmup Wall mean / iter Wall total CPU median / iter CPU total CPU / wall Peak growth Process peak
Google Pixel 7-13.0 bench_mobile::bench_oprf_prove 2 1 1.482s 2.964s 3.348s 6.696s 225.9% - 246.55 MB
Google Pixel 7-13.0 bench_mobile::bench_passport_complete_age_check_prove 2 1 14.589s 29.178s 44.450s 88.900s 304.7% 169.10 MB 942.25 MB
Google Pixel 7-13.0 bench_mobile::bench_passport_fragmented_age_check_prove 2 1 14.017s 28.033s 39.366s 78.732s 280.9% 159.56 MB 780.68 MB
Samsung Galaxy M32-11.0 bench_mobile::bench_oprf_prove 2 1 2.329s 4.658s 5.844s 11.688s 250.9% - 312.92 MB
Samsung Galaxy M32-11.0 bench_mobile::bench_passport_complete_age_check_prove 2 1 23.805s 47.610s 83.162s 166.324s 349.3% 285.46 MB 1245.50 MB
Samsung Galaxy M32-11.0 bench_mobile::bench_passport_fragmented_age_check_prove 2 1 25.083s 50.166s 77.916s 155.832s 310.6% 223.29 MB 812.47 MB
Samsung Galaxy S24-14.0 bench_mobile::bench_oprf_prove 2 1 1.087s 2.175s 3.013s 6.027s 277.1% 26.99 MB 547.03 MB
Samsung Galaxy S24-14.0 bench_mobile::bench_passport_complete_age_check_prove 2 1 7.101s 14.203s 30.369s 60.739s 427.7% 294.77 MB 1183.10 MB
Samsung Galaxy S24-14.0 bench_mobile::bench_passport_fragmented_age_check_prove 2 1 8.589s 17.179s 30.966s 61.932s 360.5% 273.82 MB 948.67 MB

Note: memory growth excludes warmup/baseline retained before the measured iteration.

@vercel

vercel Bot commented Jun 20, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
docs-provekit-org Ready Ready Preview Jul 20, 2026 8:07am

Request Review

@dcbuild3r

dcbuild3r commented Jun 21, 2026

Copy link
Copy Markdown
Collaborator Author

@Bisht13 ready for review once the fresh rerun settles.

I pushed 7dbdafb9 to add the five-sample mobile trigger and kicked off full CI with [mobench:both:5]: https://github.com/worldfnd/provekit/actions/runs/27893152208

@dcbuild3r
dcbuild3r merged commit 4b61b5d into main Jul 20, 2026
17 checks passed
@dcbuild3r
dcbuild3r deleted the dcbuilder/pr429-main-mobench-fixtures branch July 20, 2026 12:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bench Run mobile benchmarks on PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants