Skip to content

Fix flaky ModifiedPathsTests by replaying the on-disk log#2017

Merged
tyrielv merged 2 commits into
microsoft:masterfrom
tyrielv:tyrielv/fix-modified-paths-flake
Jun 11, 2026
Merged

Fix flaky ModifiedPathsTests by replaying the on-disk log#2017
tyrielv merged 2 commits into
microsoft:masterfrom
tyrielv:tyrielv/fix-modified-paths-flake

Conversation

@tyrielv

@tyrielv tyrielv commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes the flaky ModifiedPathsTests.DeletedTempFileIsRemovedFromModifiedFiles test (and any other modified-paths assertion that hits the same race).

System.InvalidOperationException : Sequence contains more than one matching element
   at GVFS.Tests.Should.EnumerableShouldExtensions.ShouldNotContain[T](IEnumerable`1, Func`2)
   at GVFS.FunctionalTests.Tools.GVFSHelpers.ModifiedPathsShouldNotContain(...)

Root cause

ModifiedPathsDatabase is an append-only A/D log that gets compacted at the end of each background-op batch via WriteAllEntriesAndFlush in PostBackgroundOperation. BackgroundOperationCount drops to 0 inside DequeueAndFlush for the last task — before the post-callback (and therefore the compaction) runs. WaitForBackgroundOperations polls until count==0 and can return between those two steps, observing a transient state like:

A temp.txt
D temp.txt

The old ModifiedPathsShouldNotContain matched both lines, fed two results into EnumerableShouldExtensions.ShouldNotContain, and SingleOrDefault threw InvalidOperationException — masking the real race with a confusing exception.

Product code is correct

FileBasedCollection.TryLoadFromDisk replays the A/D log on mount and only the final state reaches the in-memory set, so the on-disk file is always recoverable. The append-then-amortized-rewrite pattern is intentional (one O(1) write per FS event vs. O(N) on every op). No product change needed. The new AddFollowedByDeleteIsRecoveredOnLoad unit test pins down this recovery contract explicitly.

Changes (all test-only)

  • GVFSHelpers.ModifiedPathsShouldContain / ModifiedPathsShouldNotContain — Replay the A/D log into a HashSet<string> (same algorithm as TryLoadFromDisk) and check semantic membership. Matches the intent of all ~110 existing callers and is robust to the flush-race window.
  • GVFSHelpers.ModifiedPathsContentsShouldEqualModifiedPathsRawFileContentsShouldEqual — Renamed and documented to make it obvious this asserts on the raw compacted file layout, not the semantic set. Two callers in CheckoutTests.cs updated.
  • EnumerableShouldExtensions.ShouldNotContain<T>(predicate) — Uses Where(predicate) instead of SingleOrDefault(predicate) so multi-match cases yield a useful Assert.Fail message instead of InvalidOperationException.
  • ModifiedPathsDatabaseTests.AddFollowedByDeleteIsRecoveredOnLoad — New unit test asserting that loading "A temp.txt\r\nD temp.txt\r\n" produces an empty set (only the default .gitattributes).

Verification

  • ✅ Full GVFS.UnitTests suite passes (819/819) on Windows.
  • ✅ New AddFollowedByDeleteIsRecoveredOnLoad test passes.
  • GVFS.FunctionalTests.csproj builds clean.

The modified-paths database file is an append-only log of "A path" /
"D path" entries that ModifiedPathsDatabase compacts at the end of each
background-op batch via WriteAllEntriesAndFlush in PostBackgroundOperation.
BackgroundOperationCount, however, drops to 0 inside DequeueAndFlush for
the last task -- before the post-callback runs. WaitForBackgroundOperations
polls until count == 0 and can therefore return between dequeue and
compaction, leaving the file in a state like:

    A temp.txt
    D temp.txt

ModifiedPathsShouldNotContain matched both lines, fed two results into
EnumerableShouldExtensions.ShouldNotContain, and the SingleOrDefault call
threw "Sequence contains more than one matching element" -- masking the
underlying race with a confusing exception.

The product code is correct: FileBasedCollection.TryLoadFromDisk replays
the A/D log on mount, so the on-disk state is always recoverable. This
change fixes the tests:

  * GVFSHelpers.ModifiedPathsShouldContain / ModifiedPathsShouldNotContain
    now build the current set by replaying the A/D log (same algorithm as
    TryLoadFromDisk) and check membership in that set. This matches the
    semantic intent of every existing caller and is robust to the
    flush-race window.

  * ModifiedPathsContentsShouldEqual is renamed to
    ModifiedPathsRawFileContentsShouldEqual and gains an XML doc comment
    pointing callers at the semantic helpers unless they specifically need
    to validate the compacted file layout. The two existing callers in
    CheckoutTests are updated.

  * EnumerableShouldExtensions.ShouldNotContain uses Where(predicate)
    instead of SingleOrDefault(predicate) so multi-match cases produce a
    useful Assert.Fail message instead of InvalidOperationException.

  * A new ModifiedPathsDatabaseTests.AddFollowedByDeleteIsRecoveredOnLoad
    unit test pins down the recovery contract -- loading
    "A temp.txt\r\nD temp.txt\r\n" produces an empty set
    (only the auto-added .gitattributes entry).

Assisted-by: Claude Opus 4.7
Signed-off-by: Tyrie Vella <tyrielv@gmail.com>
@tyrielv tyrielv marked this pull request as ready for review June 10, 2026 22:57
Comment thread GVFS/GVFS.FunctionalTests/Tests/GitCommands/CheckoutTests.cs Outdated
Per Keith's review, the two CheckoutTests callsites that used
ModifiedPathsRawFileContentsShouldEqual were exposed to the same
compaction race the rest of this PR fixes -- WaitForBackgroundOperations
can return before PostBackgroundOperation rewrites the file, so any
transient "A path / D path" cycle (today: none; future: easy to
introduce) would break the exact-equals raw-bytes assertion.

Those callers' real intent is semantic: "after this checkout sequence,
the only modified path is .gitattributes". Replace the raw helper with
a new ModifiedPathsShouldOnlyContain that compares against the replayed
A/D log as a set. The raw helper now has zero callers and is removed.

Assisted-by: Claude Opus 4.7
Signed-off-by: Tyrie Vella <tyrielv@gmail.com>
@tyrielv tyrielv enabled auto-merge June 11, 2026 17:59
@tyrielv tyrielv merged commit b8fb34c into microsoft:master Jun 11, 2026
51 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants