Skip to content

chore: Add unit tests for arm64 disassembler - #3245

Open
filzrev wants to merge 9 commits into
dotnet:masterfrom
filzrev:chore-add-arm64-unittests
Open

chore: Add unit tests for arm64 disassembler#3245
filzrev wants to merge 9 commits into
dotnet:masterfrom
filzrev:chore-add-arm64-unittests

Conversation

@filzrev

@filzrev filzrev commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

This PR contains following changes.

1. Cleanup arm64 disassembler related code to preparing to add unit tests

See following PR comment for details.

2. Add AsmArm64 package reference

AsmArm64 package to unit test project.
Currently it's used for test purpose.
It's expected existing arm64 disassembler is replaced to AsmArm64 based implementation. (#3246)

3. Add arm64 disassembler related unit tests.

To ensure existing arm64 disassembler behavior.
Unit test codes are added for major code paths. (It can confirm code coverage results with Analyze Code Coverage on VS)

Note:
Almost of unit tests on .NET Framework are excluded by #if NET directive.

  • private methods are tested with UnsafeAccessor (It requires .NET 8 or later)
  • Capstone's native dependencies seems not copied to bin directory when using xUnit v2 (Because it's Library project)

@timcassell

Copy link
Copy Markdown
Collaborator
AI review

Accumulator bugs

Arm64RegisterValueAccumulator.cs:64ExpectingAdd never resets. Unlike ExpectingMovk, this case has no fall-through to LookingForPossibleLdr, so any instruction that isn't the expected ADD
leaves the state machine parked with the stale ADRP value. adrp x0,#0x1000 ; movz x0,#0x100 ; add x0,x0,#0x100 gives HasValue == true, Value == 0x1100 after x0 was clobbered, so a following BR/BLR x0
resolves to a bogus address and the disassembly prints the wrong symbol. The PR's own skipped AdrpThenOther_ThenAdd_ShouldResetValue fails on this today.

Arm64RegisterValueAccumulator.cs:68ADD ignores lsl #12. The match checks only Operands[2].Type == Immediate, then does _value | Immediate. adrp x0,#0x1000000 ; add x0,x0,#0xfff, lsl #12
yields page | 0xFFF instead of page + 0xFFF000, with HasValue still true — a silently wrong address rather than a bail-out. At minimum, reject a shifted immediate. (Skipped
AdrpThenAdd_WithShiftedImm_ShouldCalculateAddress covers it.)

Arm64RegisterValueAccumulator.cs:41MOVZ discards the shift. Same class: _value = details.Operands[1].Immediate drops the lsl amount, so movz x0,#0x1234, lsl #16 seeds 0x1234 instead of
0x12340000.

Arm64RegisterValueAccumulatorTests.Movz.cs:31 — skipped test asserts MOVK semantics. Movz_WithShiftedImmediateValue_ShouldStartNewValue expects 0x3333_2222_1111 from three MOVZs. A real MOVZ
zeroes the rest of the register, so the architectural result is 0x3333_0000_0000. Whoever un-skips this and "fixes" the accumulator to satisfy it will encode a decoding bug.

Tests that can't fail

Arm64DisassemblerTests.TryFollowJumpTrampoline.cs:73 — slot displacement untested. In all four stub tests the getPointer callback returns ExpectedResultAddress for any address, unlike the
TryResolvePrecode tests which assert the requested slot. Changing parseBase + (ulong)(long)off0 to parseBase + 4 + (ulong)(long)off0 in the StubPrecode branch leaves the whole suite green. Assert the
address inside getPointer, as TryResolvePrecode_* does.

Arm64DisassemblerTests.TryFollowJumpTrampoline.cs:178NonStubHead bails on length, not shape. The test supplies one instruction, so the reader returns 4 bytes and TryReadStubHead bails at read < 12; the stub-shape rejection it names is never reached, and the test passes with all stub matching deleted. Pad to >= 4 non-stub instructions.

Arm64InstructionFormatterTests.cs:19 — padding disabled. The comment says "Use DisassemblyDiagnoserConfig default config value" but FirstOperandCharIndex = 10 is commented out, so the theory runs at
Iced's default of 0 and asserts strings BDN never emits ("b #8" vs. production "b #8"DisassemblyDiagnoserConfig.cs:86). The shipped column width is only incidentally covered by the one
FirstOperandCharIndex = 6 case.

Arm64InstructionFormatterTests.cs:61 — empty symbols map. FormatInstruction_B_WithReferencedAddress passes no symbols, so TryGetValue always misses and the one thing gated on ReferencedAddress
— the Operand.Replace($"#0x{addr:x}", name) substitution — is never run. Add an entry mapping 0x10000 to a name.

Arm64DisassemblerTests.TryGetReferencedAddress.cs:11 — helper skips Init(runtime). _runtime stays null; it works only because no test here feeds an LDR. The first one that does will NRE inside
Feed. The sibling helper (Arm64RegisterValueAccumulatorTests.cs:15) does call Init.

Arm64DisassemblerTests.TryGetReferencedAddress.cs:23_With_BL builds BR X0. Duplicate of _With_BLR; BL is never covered.

Minor

Arm64Disassembler.cs:145 — constant is ISHLD, not ISH. 0xD50339BF has CRm = 0b1001 (DMB ISHLD); DMB ISH is 0xD5033BBF. The new tests construct Arm64BarrierOperationLimitKind.ISHLD,
confirming it. As written a stub prefixed with a plain DMB ISH isn't recognised and precode resolution silently fails — fix the comment/name or widen the match.

ClrMdDisassembler.cs:114IClrRuntime switch introduced unchecked downcasts. foreach (ClrModule module in state.Runtime.EnumerateModules()) and the nested foreach (ClrType type in ...) went from
statically-typed iteration to runtime casts, since both interface members are explicit implementations returning IClrModule/IClrType. Nothing breaks against ClrMD 4.0.732401 (the concrete instances
still come back), but FilterAndEnqueue now throws InvalidCastException for any other IClrRuntime — including the MockClrRuntime this PR adds, which makes that path untestable by the harness being
introduced.

Helpers/Arm64TestInstructions.cs:180ValidateMultipleOf8 throws "...must be multiple of 4".

It looks like it found some possible bugs in the accumulator (I did not verify myself). Fine if you want to fix them here, or defer for out-of-scope.

@filzrev
filzrev force-pushed the chore-add-arm64-unittests branch from 7aa8d2b to 52dc08c Compare September 6, 2026 23:23
@filzrev

filzrev commented Sep 6, 2026

Copy link
Copy Markdown
Contributor Author

It looks like it found some possible bugs in the accumulator (I did not verify myself). Fine if you want to fix them here, or defer for out-of-scope.

This PR intended to add tests to verify existing arm64 disassembler behaviors.
So following tasks are handled on another PRs.

  • Refactor Arm64ValueAccumulator/Arm64Disassebler codes. And fix some existing issues. (and reviewed content)
  • Migrate code to use AsmArm64.

@timcassell

Copy link
Copy Markdown
Collaborator

Findings are all in the new test code; the production changes look good (the formatter padding is a real fix — a mnemonic reaching FirstOperandCharIndex previously ran straight into its operand with no separator).

tests/BenchmarkDotNet.Tests/Disassemblers/Arm64/Helpers/Arm64DisassemblerTestBase.cs:71 — the read delegate ignores its address parameter and always serves rawInstructions from index 0, so the address the disassembler reads from is never asserted for TryReadStubHead, TryFollowJumpTrampoline, or the multi-hop loop. Mutating the trampoline read to dataReader.Read(address + 0x1000, head) leaves all 79 tests green. Suggest keying the delegate off address: serve bytes at a registered base, return 0 elsewhere.

tests/BenchmarkDotNet.Tests/Disassemblers/Arm64/Arm64DisassemblerTests.TryTranslateAddressToName.cs:97NonAlignedAddress = 0x10004 is below MinValidAddress on macOS-arm64 (ClrMdDisassembler.GetMinValidAddress returns 0x100000000 there), so the test returns on the first line and asserts three empty collections without reaching the GetMethodByHandle/GetTypeByMethodTable path it documents. macos-latest is arm64 in the CI matrix. Forcing GetMinValidAddress to 0x100000000 locally keeps all 79 tests passing. Address1 + 4 would fix it — same class of issue as 0ca5d3f/019acb64e, just missed for this one address.

tests/BenchmarkDotNet.Tests/Disassemblers/Arm64/Arm64InstructionFormatterTests.cs:97BeEquivalentTo on a List<string> is order-insensitive, so the instruction ordering Decode is responsible for isn't asserted (swapping two expected lines still passes). Should().Equal(...) instead.

tests/BenchmarkDotNet.Tests/Disassemblers/Arm64/Arm64InstructionFormatterTests.cs:142{ MOVZ(X0, 0x100), "movz x0, #0x100" } duplicates line 140 verbatim; xUnit silently drops it (Skipping test case with duplicate ID in the run output). Presumably one row was meant to be a different value.

tests/BenchmarkDotNet.Tests/Disassemblers/Arm64/Helpers/Arm64DisassemblerTestBase.cs:13DummyMethodNotUsed = default! is a null MockClrMethod. Safe only because the current tests return before TryTranslateAddressToName reaches method.NativeCode == currentMethod.NativeCode; a future test that lets GetMethodByInstructionPointer resolve will NRE inside production code instead of failing usefully.

tests/BenchmarkDotNet.Tests/Disassemblers/Arm64/Arm64DisassemblerTests.Decode.cs:28,39,63,74 (and the same copy-paste in the TryFollowJumpTrampoline/TryResolvePrecode files) — trailing comments name x11/x0 where the code uses x10/x1. The register number is exactly what distinguishes StubPrecode (x10/x12) from FixupPrecode (x11/x12), so someone "fixing" code to match a comment here would break stub recognition.

src/BenchmarkDotNet/Disassemblers/Arm64Disassembler.cs:146 — nit: now that the comment correctly reads DMB ISHLD, the constant name DmbIshInstr is the remaining misnomer (DmbIshLdInstr). Separately, only ISHLD is matched — a build emitting dmb ish (0xD5033BBF) would silently fail stub-head detection. Out of scope for this PR, just noting it.

Reviewed with Claude Code.

@filzrev
filzrev force-pushed the chore-add-arm64-unittests branch from 84d7a22 to a01d20d Compare September 8, 2026 12:06
@filzrev
filzrev force-pushed the chore-add-arm64-unittests branch from a01d20d to 2fdfa41 Compare September 8, 2026 12:07
@filzrev
filzrev force-pushed the chore-add-arm64-unittests branch from 2fdfa41 to 025c00a Compare September 8, 2026 17:58

@timcassell timcassell left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up on the updated head. The fixes from the last round all look right — the macOS-arm64 address, WithStrictOrdering, the duplicate MOVZ row, DmbIshLdInstr, and the Decode.cs comments. A few items are still open, plus one new one and one follow-up note.

Reviewed with Claude Code.

protected static IDataReader CreateMockDataReader(uint[] rawInstructions, Func<ulong, ulong> getPointer)
{
return new MockDataReader(
read: (address, buffer) =>

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still open from the last round: the read delegate ignores its address parameter and always serves rawInstructions from index 0, so the address the disassembler reads from is never asserted for TryReadStubHead, TryFollowJumpTrampoline, or the multi-hop loop. Mutating the trampoline read to dataReader.Read(address + 0x1000, head) leaves the whole suite green.

Keying the delegate off address — serve the bytes at a registered base, return 0 elsewhere — would close it.

internal const ulong DummyBaseAddress = 0x0000_F000_0000_0000UL;

internal static readonly Version DummyTargetFrameworkVersion = new Version(10, 0);
internal static readonly MockClrMethod DummyMethodNotUsed = default!;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still open: DummyMethodNotUsed = default! is a null IClrMethod, passed as the non-nullable currentMethod argument in TryTranslateAddressToName_GetJitHelperFunctionName_ReturnsNonEmptyValue and _NoAlignedAddress. Those pass only because production returns before reaching method.NativeCode == currentMethod.NativeCode (ClrMdDisassembler.cs:343). If that early-return ordering ever changes, they fail with a NullReferenceException instead of a readable assertion. A real MockClrMethod instance would be safer.

};
PrintInstructions(rawInstructions);

var clrRuntime = CreateMockClrRuntime(rawInstructions, address =>

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Raised in the earlier review and still present (also :102 and :131): the getPointer callback returns ExpectedResultAddress for any address, so the slot arithmetic these paths exist for is never asserted. Changing dataReader.ReadPointer(countSlot + 8, ...) to countSlot in the CallCountingStub branch, or following off1/offB instead of off0/offA, keeps all three tests green.

The sibling TryResolvePrecode_* tests already assert address.Should().Be(mdSlot) inside the callback — same assertion belongs here.

private readonly TryReadPointerDelegate _tryReadPointer = (_, out _) => throw new InvalidOperationException($"{nameof(_tryReadPointer)} field is not set.");


public MockDataReader(ulong dummyValue = 0)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This overload sets only _tryReadPointer, leaving _read as the throwing default. It backs DummyClrRuntime and Arm64InstructionFormatterTests.GetArm64Asms via CreateMockClrRuntime(ulong), and GetArm64Asms runs the full Decode loop — so the first test case added there containing an indirect branch (e.g. movz/br) will hit FlushCachedDataIfNeeded -> Read and die with InvalidOperationException: _read field is not set. rather than an assertion failure.

Delegating to CreateMockDataReader([], _ => dummyValue) would give it a working Read.

var rawInstructions = new[]
{
Arm64InstructionFactory.DMB(Arm64BarrierOperationLimitKind.ISHLD), // dmb ishld
Arm64InstructionFactory.LDR(X10, 0x10000), // ldr x11, #0x10000

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Register comments still mismatch the code here and on :69 (BR(X10) commented br x11) and :125 (LDR(X9, ...) commented ldr x11), plus Arm64DisassemblerTests.TryResolvePreCode.cs:20,22. Decode.cs was fixed in the last push; these were missed.

Worth getting right because the register number is exactly what distinguishes StubPrecode (x10/x12) from FixupPrecode (x11/x12) — someone "fixing" code to match a comment here would break stub recognition.

target.Should().Be(ExpectedResultAddress);
}

// TryFollowJumTrampoline seems not support FixupPrecode with pre-backpatch form.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up rather than something to fix here, in the same bucket as the other pre-existing issues you deferred: this test locks in a real asymmetry — TryResolvePrecode handles the pre-backpatch FixupPrecodeCode_Fixup shape (Arm64Disassembler.cs:233) but TryFollowJumpTrampoline does not. A direct BL onto a fixup precode that has never been backpatched therefore fails the trampoline chase, GetMethodByHandle(stubAddress) then fails because the address is the stub rather than the MethodDesc, and the call target is left untranslated in the disassembly.

Pre-existing (from #3208) and out of scope, but a TODO or issue link would read better than the bare "seems not support" comment, which makes it look intentional. (Also a typo there: TryFollowJumTrampoline.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants