Fix VectorSessionFunctions element grow/shrink corruption (VectorSet Attributes) + tests#1967
Draft
tiagonapoli wants to merge 5 commits into
Draft
Fix VectorSessionFunctions element grow/shrink corruption (VectorSet Attributes) + tests#1967tiagonapoli wants to merge 5 commits into
tiagonapoli wants to merge 5 commits into
Conversation
Add unit tests over VectorSessionFunctions' element sub-store write path that pin the VectorSet Attributes recovery corruption at the unit level (no Docker / CPU-throttle repro needed). The tests drive a dedicated Vector session directly and Upsert a namespaced element value, mirroring VectorManager's DiskANN write callback. Growing a variable-length element in the mutable region hits InPlaceWriter, which copies without honoring the destination capacity: grow overflows (ArgumentException) and shrink leaves stale trailing bytes. The read-only-region cases (fresh InitialWriter record), round-trip, and same-size overwrite pass. Grow(False)/Shrink(False) fail today, demonstrating the bug; the production fix is intentionally not included in this change. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: dad22029-9339-4e1a-85cc-bb7597125bb6
Drop the AttributesNamespace constant and its DiskANN-specific comment; the Writer path has no namespace-specific behaviour, so the tests just use an arbitrary single-byte namespace inline. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: dad22029-9339-4e1a-85cc-bb7597125bb6
Combine the same-size / grow / shrink Upsert tests into a single multi-stage VectorElementUpsertResizePreservesValue that walks one element through initial write -> equal-size overwrite -> grow -> shrink, verifying the round-trip at each stage, keeping the inReadOnlyRegion parameterization. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: dad22029-9339-4e1a-85cc-bb7597125bb6
InPlaceWriter did newValue.CopyTo(value) against the existing record's value span with no capacity/length handling, so an in-place overwrite of a variable-length element (e.g. the per-element Attributes blob) would overflow the record on a grow (ArgumentException) or leave stale trailing bytes and a wrong content length on a shrink. Mirror the main-store pattern: build the RecordSizeInfo for the new value and call TrySetContentLengths, which resizes/shortens the value in place and returns false when the value cannot grow within the allocated record. On false, return false so Tsavorite falls back to allocating a fresh, correctly-sized record via InitialWriter. Only copy once the length is set. VectorElementUpsertResizePreservesValue now passes in both the mutable (InPlaceWriter) and read-only (InitialWriter) regions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: dad22029-9339-4e1a-85cc-bb7597125bb6
CopyUpdater and InPlaceUpdater had the same unbounded-copy shape that was fixed in InPlaceWriter: they assumed the new value was the same size as the old one. CopyUpdater (callback branch) did oldValueAligned.CopyTo(newValueAligned) with no bound, overflowing the freshly-allocated destination on a shrink (old larger than new), asserted the contradictory WriteDesiredSize <= oldValueAligned.Length (which fails on a grow), and returned true without setting the content length. Now it carries over only the overlapping prefix of the old value before the callback repopulates the buffer, and returns TrySetContentLengths so the destination length is correct. InPlaceUpdater (callback branch) handed the existing record's value span to the native callback and let it write WriteDesiredSize bytes with no capacity check, overflowing the record on a grow and leaving a stale content length on a shrink. Now it resizes the record via TrySetContentLengths first, returning false (-> Tsavorite RCU to CopyUpdater with a fresh record) when the value cannot grow in place, then re-aligns before invoking the callback. These paths are only reached today with fixed-size DiskANN terms and the fixed-size ContextMetadata, so the change is behavior-preserving for the same-size case; the full Garnet.test.vectorset suite is unchanged (143 passed, 22 skipped, 0 failed). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: dad22029-9339-4e1a-85cc-bb7597125bb6
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Fixes a VectorSet element corruption bug in
VectorSessionFunctions.InPlaceWriter, and adds a unit test over the element sub-store write path that pins it.The bug
InPlaceWriterdidnewValue.CopyTo(value)straight into the existing record's value span with no capacity or content-length handling. For a variable-length per-element value (the SETATTR Attributes blob), an in-place overwrite in the mutable region would:ArgumentException: Destination is too short.The base contract computes a
RecordSizeInfoand returnsfalsewhen the new value won't fit, letting Tsavorite allocate a fresh record — the Vector override skipped all of it.The fix
Mirror the main-store pattern: build the
RecordSizeInfofor the new value,PopulateRecordSizeInfo, thenTrySetContentLengths, which resizes/shortens the value in place and returnsfalsewhen it cannot grow within the allocated record. Onfalse, returnfalseso Tsavorite falls back to a fresh, correctly-sized record viaInitialWriter. Only copy once the length is set.Test
VectorElementUpsertResizePreservesValue([Values(false, true)] inReadOnlyRegion)drives a dedicated Vector session directly (mirroringVectorManager's DiskANN write callback) and walks a single element through initial write -> equal-size overwrite -> grow -> shrink, verifying the exact round-trip after each stage. It covers both the mutable region (InPlaceWriter) and the read-only region (freshInitialWriterrecord). PlusVectorElementUpsertRoundTripsas a baseline.Before the fix, the mutable-region case failed at the grow stage; with the fix both regions pass.
Validation
VectorElement*tests: 6/6 pass (both TestFixtures).Garnet.test.vectorsetsuite: 143 passed, 22 skipped, 0 failed — no regressions.Scope note
Only the Upsert/
InPlaceWriterpath is fixed here — that is the path attributes actually use (WriteCallbackUnmanaged->Upsert) and the one the repro hit. The RMW/Updater callback path (InitialUpdater/InPlaceUpdater/CopyUpdater) has the same latent shape but is only invoked by DiskANN with fixed-size data and isn't reachable variable-size through any command, so it is intentionally left unchanged.