Skip to content

gfx/shaders: ignore BAHS candidates outside validated binary records - #2943

Open
gino8080 wants to merge 1 commit into
seedds:mainfrom
gino8080:fix/bahs-binary-record-entries
Open

gino8080 wants to merge 1 commit into
seedds:mainfrom
gino8080:fix/bahs-binary-record-entries

Conversation

@gino8080

@gino8080 gino8080 commented Nov 17, 2026 •

Copy link
Copy Markdown

What and why

Mario Kart World 1.8.0 contains BAHS payload that looks like a shader descriptor even though it is outside the binary record entry slots. The scan accepts it, pairs unrelated control and code, and sends a nonexistent shader to the translator. The existing exclusion for descriptors inside control sections does not cover this case.

Reuse the validated binary array already read for variation pairing to identify the real entry offsets. Filter the scan only when that array validates; keep the existing candidates for unrecognized or inconsistent arrays. The shared reader continues to supply variation pairs.

On the local 1.8.0 dump, this removes two false shader keys and leaves 11,601 shaders. The new synthetic regression fixture returns three candidates on upstream and the expected two with this fix. A second test checks that malformed array headers and record sizes retain the scan fallback.

Checked

Rebased onto 0bd40f9fe870d8f68a375e8765298c9b7e852e99 to resolve a conflict in the generated shader-chain header after upstream's ordered-NE and half-FTZ updates. The header was regenerated from the combined source tree; neither side's old hash was retained. The BAHS implementation and its regression tests are byte-for-byte unchanged from the initial PR commit.

After this rebase, all 124 split tests and 42 chain/header, documentation, native-tool and SPIRV-Cross pin tests pass. Pinned Python lint reports 305 existing findings, zero new findings and zero stale baseline entries. The full shader build and game run below are from the original validation base; they were not repeated for the rebased chain.

  • Original validation base: 0afeb5b81bb4e8cd49f5f61750e2f1189db04574.
  • python -m unittest discover -s dm/tools/tests -p 'test_split*.py': 124 tests passed.
  • python -m unittest dm.tools.tests.test_docs: 6 tests passed.
  • Pinned lint_python.py --against HEAD: 0 new findings and 0 stale baseline entries.
  • Chain/header and documentation checks: 32 tests passed.
  • Shader chain header regenerated for this branch: d8de32bc61d8.
  • The same parser implementation was used to build all 11,601 shaders for Mac and iOS on the earlier 2925d7af base. Mario Kart World 1.8.0 (0400C3F00006E000) was installed and driven on an iPhone Duo running iOS 27.2.1, built with Xcode 27.2. This is previous device evidence, not a device test of the new base.
  • regress.py check on the original validation base: 11,601/11,601 shaders built on both sides, no changed MSL, no new shader or interface failures. The translated package was already split with the corrected parser; parser discovery itself is covered by the synthetic before/after test and the earlier full dump split.
  • Both Mac runs stop in the same Metal driver call before the race (exit 139). The harness classifies this as a known baseline failure and reports PASS; this is not a successful gameplay smoke test of that base.
  • Mac app packaging tests: 85 passed with the permissions needed by Xcode. The initial sandboxed invocation failed during Swift macro compilation; it is not an upstream code failure.
  • Broader tools: 1,673 tests run separately from the 85 passing Mac app tests, 39 skipped. There is one pre-existing failing test (test_mesh_compiles_only_requested_targets, three target subcases): dmsh_rewrite is undefined in its isolated test namespace. The same test fails on unchanged toolchain sources at the base revision.
  • The full standalone Metal graphics unit suite (make -C dm/gfx/tests ... run) was not run. Original-base gameplay validation is limited by the baseline driver crash above.
  • Only Mario Kart World is available locally. Other BAHS games have not been rebuilt; the impact report marks broad shader coverage and profiles without usage manifests as unknown.

Docs

The BAHS entry validation reference and the Mario Kart World 1.8.0 note describe the cause, fallback and validation limit.

  • Finding documented; docs_symptoms.py run.
  • Shader chain header regenerated.
  • Fixtures are synthetic. No game files, keys or extracted game data are included.
Regression report

regress.py check (full): PASS

Baseline HEAD 0afeb5b, candidate worktree 0afeb5b (+ uncommitted changes); Apple M4 Pro, macOS 27.2.1, Xcode 27.2; 345.2 s

game status shaders (base → cand) newly failing newly building changed MSL interface (new / known) boot fps p50 frame p99 ms peak MB prepare s new RED
Mario Kart World 0400C3F00006E000 PASS 11601/11601 → 11601/11601 0 0 0 0 / 0 FAIL 0.0 → 0.0 None → None 2836 → 3037 None → None 0

Mario Kart World 0400C3F00006E000: PASS

  • known (baseline too): crash ([dm] CRASH sig=# addr=# pc=-[AGXG#XFamilyRenderContext_mtlnext setRenderPipelineState:]+# lr=draw_+# sp=#)
  • known (baseline too): frame 6000 not reached by 190 s (max 93)
    • baseline: exit 139, first frame 2.8 s, max frame 88, reached False, prepare None s, prebuild 115.5 s (9591 built, 0 failed), fps 0.0, p50/p99 None/None ms, peak 2836 MB
    • candidate: exit 139, first frame 2.9 s, max frame 93, reached False, prepare None s, prebuild 113.3 s (9591 built, 0 failed), fps 0.0, p50/p99 None/None ms, peak 3037 MB
    • time: chain 84.2 s, smoke 260.9 s

@github-actions github-actions Bot added nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) packing Building an IPA or a Mac app: the NSP script, extraction, the shader split, the packers tools Tools and scripts (dm/tools, scripts) docs Documentation shader chain The shader translation chain (DMSH_CHAIN): a change rebuilds every game's shaders game: 0400C3F00006E000 A game profile labels Nov 17, 2026
Use validated BAHS binary record offsets to reject descriptor-shaped payload.
Preserve the scan fallback for unknown layouts and share the record reader
with variation pairing. Add synthetic regression and fallback coverage,
regenerate the chain header, and document the game-specific evidence.
@gino8080
gino8080 force-pushed the fix/bahs-binary-record-entries branch from 18860b4 to fa03c62 Compare November 17, 2026 17:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation game: 0400C3F00006E000 A game profile nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) packing Building an IPA or a Mac app: the NSP script, extraction, the shader split, the packers shader chain The shader translation chain (DMSH_CHAIN): a change rebuilds every game's shaders tools Tools and scripts (dm/tools, scripts)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant