feat(testing): mutation testing on DepthBiasCalculator + retargeting samplers (#282) - #284
Draft
arkavo-com wants to merge 17 commits into
Draft
feat(testing): mutation testing on DepthBiasCalculator + retargeting samplers (#282)#284arkavo-com wants to merge 17 commits into
arkavo-com wants to merge 17 commits into
Conversation
First-target proof-of-concept: muter built from a pinned SHA, direct unit suite written before any mutation run, ≥80% mutation score with every survivor classified. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds make muter-bootstrap to build muter from a pinned master SHA into .build/tools/, and make mutation-test as the run target. The 2023 tagged release (16) predates Swift 6.2 toolchain changes; building from source is the only reliable path until muter cuts a new release. Note: build command uses --product muter to skip test targets, which fail under Swift 6.2 due to @testable import muterCore incompatibility in XCTestExtensions.swift. The muter executable itself builds cleanly. Issue #282 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds 22 direct unit tests covering construction, exact-match lookups, priority ordering, case-insensitive matching, default fallback, overlay offset, scale multiplication, and cache observability. This suite is the muter target for issue #282; HipSkirtTests stays as the integration suite and is not part of the mutation oracle. Issue #282 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…282) Mutates Sources/GLTFCore/Utilities/DepthBiasCalculator.swift using muter's default operator set. Tests run via swift test --filter DepthBiasCalculatorTests --disable-sandbox (sandbox-disabled because project tests need FS access for fixtures, and muter needs FS write access for source rewrites). JSON output path is deterministic for future CI artifact consumption. Issue #282 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
muter v16's config file schema does not support filesToMutate or output-path keys — those are CLI flags. Update the mutation-test target to pass --files-to-mutate, --format json, --output, and --skip-coverage so the run is correctly scoped to DepthBiasCalculator and writes the deterministic JSON artifact. Issue #282 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…n run (#282) Survivor-driven additions to the oracle suite. Added 10 new tests: one for each individual keyword in the clothing OR-chain (cloth, clothing, skirt, bottoms, pants), one each for mouth and lip alone, one for brow alone, and two absolute-value overlay assertions that pin the SwapTernary mutant. Also adds .muter/depth-bias.yml (YAML config required by muter v16's --configuration path resolution) and switches Makefile to use it. Adds --build-path /tmp/vrm-muter-build to avoid module-cache path conflicts when muter copies the project to the _mutated directory. Mutation score: 0% (tool bug; see PR for analysis). Manual review confirms all 7 kill-pending survivors are now covered by the 35-test suite; 1 survivor (RemoveSideEffects on MTLRenderCommandEncoder) is accepted-gap. Issue #282 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
muter v16's SchemataMutationMapping keys schemata by SwiftSyntax tree-position identity, but ApplySchemata re-parses files so the identity-based dict lookup always misses — producing zero-killed mutation runs even when the test suite would kill mutants. Add a parallel mappingsByDescription index (populated at insertion while arenas are alive) as a content-based fallback in schemata(_:). The matched entry is removed immediately after use to prevent re-injection into the else branch of the injected schemata switch, which would otherwise recurse until stack overflow. Apply the patch during muter-bootstrap via git apply, preceded by git checkout -- . to keep bootstrap idempotent. Issue #282 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The prior single-keyword 'alone' tests didn't kill the ChangeLogicalConnector mutants on the explicit if-chains: the partial-match scan in computeBias() is a safety net that returns the same value for single-keyword inputs. Replace with multi-keyword tests (cloth_body, skirt_body, bottoms_body, mouth_eye, brow_eye) that exploit the priority chain — original code hits the higher-priority branch via the if-chain, mutated code falls through to a lower-priority branch with a different result. Mutation score climbs from 12% to 87.5%. Remaining survivor is the Metal GPU side-effect (encoder.setDepthBias), classified as accepted-gap. Issue #282 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
First-target proof-of-concept baseline at 87% mutation score (7/8 killed). Documents the one surviving Metal GPU side-effect as accepted-gap, the muter v16 schemata-identity patch applied during bootstrap, and follow-ups for upstream submission + scope expansion. Issue #282 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
arkavo-com
force-pushed
the
issue/282-mutation-testing
branch
from
May 22, 2026 00:00
9d1aae2 to
04cab90
Compare
arkavo-com
changed the base branch from
issue/280-platform-specific-metallibs
to
main
May 22, 2026 00:00
Second mutation-testing target and the agreed go/no-go run: extract the three retargeting samplers from VRMAnimationLoader into a testable RetargetingSamplers file, then mutation-test that. Baseline doc carries a Verdict section feeding the expand/stop decision. Issue #282 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…282) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…der (#282) Move makeRotationSampler / makeTranslationSampler / makeScaleSampler and their safeDivide helper into a focused RetargetingSamplers.swift so the VRMA retargeting math is an isolated, directly-testable unit. KeyTrack, Interpolation, sampleQuaternion and sampleVector3 are promoted private to internal so the new file (and tests) can reach them. No behavior change. Issue #282 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
15 direct unit tests covering all three retargeting samplers and safeDivide: nil-model-rest passthrough, identity-rest collapse, the W_A==W_B delta-formula collapse, the full W-term change-of-basis, and additive/ratio/edge-case behavior. The muter target for issue #282's second mutation run. Issue #282 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Issue #282 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ate (#282) mutation-test becomes an aggregate of mutation-test-depth-bias and mutation-test-retargeting, each writing its own JSON artifact. Mirrors the shaders aggregate pattern. DepthBiasCalculator baseline doc updated for the renamed target and report path. Issue #282 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Second mutation target. 100% score (6/6) but every mutant landed in the safeDivide guard helper — muter v16's four scalar operators generate zero mutants against simd quaternion/vector math. Verdict: validated not adopted, lean stop; keep mutation testing as an opt-in spot-check for scalar-logic-heavy files only. Issue #282 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
arkavo-com
changed the title
arkavo-com
marked this pull request as draft
May 22, 2026 03:02
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #282
Stands up mutation testing for VRMMetalKit and runs it against two targets to answer issue #282's scoping question: is mutation testing worth adopting as a routine?
Target 1 —
DepthBiasCalculator(POC)muterfrom a pinnedmasterSHA into.build/tools/, with a local patch for a v16 schemata-identity bug (.muter/patches/v16-schemata-content-fallback.patch) that otherwise made every run report 0% killed.Target 2 — VRMA retargeting samplers
makeRotationSampler/makeTranslationSampler/makeScaleSamplerfrom the 977-lineVRMAnimationLoader.swiftinto a testableRetargetingSamplers.swift(behavior-preserving refactor).safeDivideguard helper. The quaternion/SIMD retargeting math generated zero mutants — muter v16's four scalar operators (relational / side-effect / logical / ternary) have nothing to grab insimd-based code.Verdict (see
docs/mutation-testing/retargeting-samplers-baseline.md)Validated, not adopted — lean stop. Mutation testing has real value for scalar control-flow code but is nearly blind to the
simdmath that makes up most of VRMMetalKit's core. Recommendation: do not adopt as a routine/CI gate; keep it as an opt-in spot-check for scalar-logic-heavy files. Infrastructure stays in place (make mutation-test-*) for future use.Out of scope
Test plan
make muter-bootstrapbuilds patched muter from the pinned SHAmake mutation-test-depth-bias→ 7/8 killed (87%)make mutation-test-retargeting→ 6/6 killed (100%)swift test --filter DepthBiasCalculatorTests --disable-sandbox— 32 passswift test --filter RetargetingSamplersTests --disable-sandbox— 15 passRetargetingSamplersextraction is behavior-preserving — full suite unchanged🤖 Generated with Claude Code