Skip to content

<xmemory>: Document the iterator debugging invariants - #6391

Open
Prakriti Sharma (prakriti31) wants to merge 2 commits into
microsoft:mainfrom
prakriti31:gh2084-xmemory-idl-invariants
Open

<xmemory>: Document the iterator debugging invariants#6391
Prakriti Sharma (prakriti31) wants to merge 2 commits into
microsoft:mainfrom
prakriti31:gh2084-xmemory-idl-invariants

Conversation

@prakriti31

Copy link
Copy Markdown

Closes #2084.

The iterator debugging (IDL) proxy/iterator machinery in <xmemory>
(_Container_proxy, _Container_base12, _Iterator_base12) wasn't
commented at all, even though its invariants are non-obvious (intrusive
linked list of iterators, mutual container/proxy pointers, locking rules).
Stephan T. Lavavej (@StephanTLavavej) explained these invariants in the issue thread (originally
via a linked Discord screenshot from 2021), so this PR just captures that
explanation as a comment next to the code it describes, so it doesn't only
live in a screenshot linked from a 4-year-old issue comment.

I verified the described invariants (mutual _Myproxy/_Mycont pointers, the
_Myfirstiter/_Mynextiter intrusive list, the _Lockit(_LOCK_DEBUG) locking
rule) still match the current code exactly before writing this up.

No behavior change -- comment-only.

Disclosure: This change was drafted with AI assistance (Claude Code), at
my direction, using only the existing issue thread and the current header
as source material.

Fixes microsoftGH-2084. Captures the IDL proxy/iterator invariants (explained by
@StephanTLavavej in the issue thread) as a comment next to the
_Container_proxy/_Container_base12/_Iterator_base12 machinery, since they
were previously only recorded in a Discord screenshot linked from the issue.
Copilot AI balanced review requested due to automatic review settings August 1, 2026 07:37
@prakriti31
Prakriti Sharma (prakriti31) requested a review from a team as a code owner August 1, 2026 07:37
@github-project-automation github-project-automation Bot moved this to Initial Review in STL Code Reviews Aug 1, 2026
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Documents iterator-debugging proxy relationships, iterator lists, and locking behavior in <xmemory>.

Changes:

  • Adds comments describing iterator-debugging invariants.
  • Introduces no behavioral changes.
Suppressed comments (2)

stl/inc/xmemory:1222

  • This invariant only holds when _ITERATOR_DEBUG_LEVEL == 2. At level 1, _Iterator_base12::_Adopt (lines 1327–1333) copies _Myproxy but never links the iterator into _Myfirstiter, so valid iterators are deliberately absent from this list. Please qualify the invariant so the documentation matches both _Container_base12 modes.
// * The proxy's _Myfirstiter, together with each iterator's _Mynextiter, forms an intrusive singly linked list of
//   iterators rooted at the proxy. Every valid iterator belonging to a container is reachable through this list;
//   there are no valid "free-floating" iterators.

stl/inc/xmemory:1226

  • The lock rule is not unconditional: level-1 proxy swaps use the unlocked helper (lines 1457–1459), and level-2 constant evaluation also invokes unlocked helpers. Runtime list mutations at level 2 are locked, but proxy allocation/reloading/deletion is not. Please state that narrower invariant instead of saying every proxy manipulation holds _LOCK_DEBUG.
// * Whenever the proxies and the intrusive list are manipulated, the debug lock (_Lockit(_LOCK_DEBUG)) is held.
//   The only things we do outside of that lock are things like iterator compatibility checks that compare proxy
//   pointers: those pointers don't change even if the containers are being swapped concurrently (only the
//   proxies' data members change, not their addresses).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread stl/inc/xmemory Outdated
@prakriti31

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

@StephanTLavavej Stephan T. Lavavej (StephanTLavavej) added the documentation Related to documentation or comments label Aug 1, 2026
@prakriti31

Copy link
Copy Markdown
Author

Stephan T. Lavavej (@StephanTLavavej) What else do we need to merge it. Can you help me? I will push the changes asap.

@StephanTLavavej

Copy link
Copy Markdown
Member

Someone who is deeply familiar with the invariants (i.e. me) needs to review this and think about whether the description is correct and complete.

I have a high-priority task to get through (constexpr <cmath>) and then I'm going on vacation, but I'll get to this when I return and start dealing with the PR backlog. Apologies for the delay.

@prakriti31

Copy link
Copy Markdown
Author

Sure!!

Comment thread stl/inc/xmemory Outdated
static constexpr bool _Unwrap_when_unverified = true;
};

// The iterator debugging library (IDL) below lets a container invalidate ("orphan") its iterators without either

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the letter "L" in "IDL" usually stands for "level". Also, perhaps we shouldn't say the iterator debugging mechanisms form a library.

@prakriti31 Prakriti Sharma (prakriti31) Aug 6, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, I have made the changes. Thankyou for your input, matters a lot.
Would appreciate if you can verify the changes once.

Comment thread stl/inc/xmemory Outdated
Comment on lines +1214 to +1215
// * Every container owns a dynamically allocated _Container_proxy at all times, including in its
// default-constructed and moved-from states.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In additional to Copilot, I think it's better to explain that this is the major reason why many containers in MSVC STL don't have noexcept move constructors while they ought to have.

Also, this is somehow "TRANSITION, ABI". Perhaps we should say we will change this strategy in vNext (see #169).

Comment thread stl/inc/xmemory Outdated
Comment on lines +1223 to +1226
// * Whenever the proxies and the intrusive list are manipulated, the debug lock (_Lockit(_LOCK_DEBUG)) is held.
// The only things we do outside of that lock are things like iterator compatibility checks that compare proxy
// pointers: those pointers don't change even if the containers are being swapped concurrently (only the
// proxies' data members change, not their addresses).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess it's meaningful to say that we don't hold the lock during constant evaluation.

- Clarify that "IDL" refers to _ITERATOR_DEBUG_LEVEL ("level", not
  "library"), per frederick-vs-ja's review comment.
- Explain that the separately-allocated _Container_proxy (TRANSITION,
  ABI) is why several containers' allocator-extended move operations
  aren't unconditionally noexcept, and link to microsoft#169 for the vNext plan.
- Note that the debug lock is skipped during constant evaluation.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

Suppressed comments (6)

stl/inc/xmemory:1232

  • The claim about containers being swapped concurrently reads like a thread-safety guarantee, but swap racing with iterator operations is still undefined behavior and can involve data races even if proxy addresses are stable. Consider rewording to avoid implying concurrency safety (e.g., focus on the intended invariant: proxy objects aren’t relocated, so address comparisons remain meaningful without taking the debug lock).
//   isn't usable there. The only other things we do outside of the lock at runtime are things like iterator
//   compatibility checks that compare proxy pointers: those pointers don't change even if the containers are
//   being swapped concurrently (only the proxies' data members change, not their addresses).

stl/inc/xmemory:1215

  • This bullet is phrased as applying to 'Every container', but the machinery here is specifically about containers participating in IDL (i.e., those using _Container_base12 / the proxy mechanism). To prevent readers from overgeneralizing, consider scoping the statement (e.g., 'Every IDL-enabled container' or 'Every container derived from _Container_base12').
// * Every container owns a dynamically allocated _Container_proxy at all times, including in its

stl/inc/xmemory:1216

  • The inline 'This is TRANSITION, ABI:' reads awkwardly and is inconsistent with typical tag-style notes. Consider restructuring this into a standalone // TRANSITION, ABI: line (or rephrasing the sentence) so the rationale is clearer and the tag is easier to scan.
//   default-constructed and moved-from states. This is TRANSITION, ABI: allocating the proxy separately (instead

stl/inc/xmemory:1211

  • Several of these comment lines are quite long. If this repo follows a column limit for headers, wrapping these lines would improve readability and keep formatting consistent with surrounding code/comments.
// The machinery below implements iterator debugging (informally "IDL", after the _ITERATOR_DEBUG_LEVEL macro,

stl/inc/xmemory:1217

  • Several of these comment lines are quite long. If this repo follows a column limit for headers, wrapping these lines would improve readability and keep formatting consistent with surrounding code/comments.
//   of, say, storing it inline) is the major reason many containers' allocator-extended move constructors and

stl/inc/xmemory:1227

  • Several of these comment lines are quite long. If this repo follows a column limit for headers, wrapping these lines would improve readability and keep formatting consistent with surrounding code/comments.
// * The proxy's _Myfirstiter, together with each iterator's _Mynextiter, forms an intrusive singly linked list of
//   iterators rooted at the proxy. Every valid iterator belonging to a container is reachable through this list;
//   there are no valid "free-floating" iterators.
// * Whenever the proxies and the intrusive list are manipulated at runtime, the debug lock (_Lockit(_LOCK_DEBUG))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Related to documentation or comments

Projects

Status: Initial Review

Development

Successfully merging this pull request may close these issues.

<xmemory>: Document the iterator debugging invariants

4 participants