<xmemory>: Document the iterator debugging invariants - #6391
<xmemory>: Document the iterator debugging invariants#6391Prakriti Sharma (prakriti31) wants to merge 2 commits into
<xmemory>: Document the iterator debugging invariants#6391Conversation
Fixes microsoftGH-2084. Captures the IDL proxy/iterator invariants (explained by @StephanTLavavej in the issue thread) as a comment next to the _Container_proxy/_Container_base12/_Iterator_base12 machinery, since they were previously only recorded in a Discord screenshot linked from the issue.
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
There was a problem hiding this comment.
Pull request overview
Documents iterator-debugging proxy relationships, iterator lists, and locking behavior in <xmemory>.
Changes:
- Adds comments describing iterator-debugging invariants.
- Introduces no behavioral changes.
Suppressed comments (2)
stl/inc/xmemory:1222
- This invariant only holds when
_ITERATOR_DEBUG_LEVEL == 2. At level 1,_Iterator_base12::_Adopt(lines 1327–1333) copies_Myproxybut never links the iterator into_Myfirstiter, so valid iterators are deliberately absent from this list. Please qualify the invariant so the documentation matches both_Container_base12modes.
// * The proxy's _Myfirstiter, together with each iterator's _Mynextiter, forms an intrusive singly linked list of
// iterators rooted at the proxy. Every valid iterator belonging to a container is reachable through this list;
// there are no valid "free-floating" iterators.
stl/inc/xmemory:1226
- The lock rule is not unconditional: level-1 proxy swaps use the unlocked helper (lines 1457–1459), and level-2 constant evaluation also invokes unlocked helpers. Runtime list mutations at level 2 are locked, but proxy allocation/reloading/deletion is not. Please state that narrower invariant instead of saying every proxy manipulation holds
_LOCK_DEBUG.
// * Whenever the proxies and the intrusive list are manipulated, the debug lock (_Lockit(_LOCK_DEBUG)) is held.
// The only things we do outside of that lock are things like iterator compatibility checks that compare proxy
// pointers: those pointers don't change even if the containers are being swapped concurrently (only the
// proxies' data members change, not their addresses).
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
@microsoft-github-policy-service agree |
|
Stephan T. Lavavej (@StephanTLavavej) What else do we need to merge it. Can you help me? I will push the changes asap. |
|
Someone who is deeply familiar with the invariants (i.e. me) needs to review this and think about whether the description is correct and complete. I have a high-priority task to get through ( |
|
Sure!! |
| static constexpr bool _Unwrap_when_unverified = true; | ||
| }; | ||
|
|
||
| // The iterator debugging library (IDL) below lets a container invalidate ("orphan") its iterators without either |
There was a problem hiding this comment.
I think the letter "L" in "IDL" usually stands for "level". Also, perhaps we shouldn't say the iterator debugging mechanisms form a library.
There was a problem hiding this comment.
Hi, I have made the changes. Thankyou for your input, matters a lot.
Would appreciate if you can verify the changes once.
| // * Every container owns a dynamically allocated _Container_proxy at all times, including in its | ||
| // default-constructed and moved-from states. |
There was a problem hiding this comment.
In additional to Copilot, I think it's better to explain that this is the major reason why many containers in MSVC STL don't have noexcept move constructors while they ought to have.
Also, this is somehow "TRANSITION, ABI". Perhaps we should say we will change this strategy in vNext (see #169).
| // * Whenever the proxies and the intrusive list are manipulated, the debug lock (_Lockit(_LOCK_DEBUG)) is held. | ||
| // The only things we do outside of that lock are things like iterator compatibility checks that compare proxy | ||
| // pointers: those pointers don't change even if the containers are being swapped concurrently (only the | ||
| // proxies' data members change, not their addresses). |
There was a problem hiding this comment.
I guess it's meaningful to say that we don't hold the lock during constant evaluation.
- Clarify that "IDL" refers to _ITERATOR_DEBUG_LEVEL ("level", not
"library"), per frederick-vs-ja's review comment.
- Explain that the separately-allocated _Container_proxy (TRANSITION,
ABI) is why several containers' allocator-extended move operations
aren't unconditionally noexcept, and link to microsoft#169 for the vNext plan.
- Note that the debug lock is skipped during constant evaluation.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.
Suppressed comments (6)
stl/inc/xmemory:1232
- The claim about containers being swapped concurrently reads like a thread-safety guarantee, but
swapracing with iterator operations is still undefined behavior and can involve data races even if proxy addresses are stable. Consider rewording to avoid implying concurrency safety (e.g., focus on the intended invariant: proxy objects aren’t relocated, so address comparisons remain meaningful without taking the debug lock).
// isn't usable there. The only other things we do outside of the lock at runtime are things like iterator
// compatibility checks that compare proxy pointers: those pointers don't change even if the containers are
// being swapped concurrently (only the proxies' data members change, not their addresses).
stl/inc/xmemory:1215
- This bullet is phrased as applying to 'Every container', but the machinery here is specifically about containers participating in IDL (i.e., those using
_Container_base12/ the proxy mechanism). To prevent readers from overgeneralizing, consider scoping the statement (e.g., 'Every IDL-enabled container' or 'Every container derived from_Container_base12').
// * Every container owns a dynamically allocated _Container_proxy at all times, including in its
stl/inc/xmemory:1216
- The inline 'This is TRANSITION, ABI:' reads awkwardly and is inconsistent with typical tag-style notes. Consider restructuring this into a standalone
// TRANSITION, ABI:line (or rephrasing the sentence) so the rationale is clearer and the tag is easier to scan.
// default-constructed and moved-from states. This is TRANSITION, ABI: allocating the proxy separately (instead
stl/inc/xmemory:1211
- Several of these comment lines are quite long. If this repo follows a column limit for headers, wrapping these lines would improve readability and keep formatting consistent with surrounding code/comments.
// The machinery below implements iterator debugging (informally "IDL", after the _ITERATOR_DEBUG_LEVEL macro,
stl/inc/xmemory:1217
- Several of these comment lines are quite long. If this repo follows a column limit for headers, wrapping these lines would improve readability and keep formatting consistent with surrounding code/comments.
// of, say, storing it inline) is the major reason many containers' allocator-extended move constructors and
stl/inc/xmemory:1227
- Several of these comment lines are quite long. If this repo follows a column limit for headers, wrapping these lines would improve readability and keep formatting consistent with surrounding code/comments.
// * The proxy's _Myfirstiter, together with each iterator's _Mynextiter, forms an intrusive singly linked list of
// iterators rooted at the proxy. Every valid iterator belonging to a container is reachable through this list;
// there are no valid "free-floating" iterators.
// * Whenever the proxies and the intrusive list are manipulated at runtime, the debug lock (_Lockit(_LOCK_DEBUG))
Closes #2084.
The iterator debugging (IDL) proxy/iterator machinery in
<xmemory>(
_Container_proxy,_Container_base12,_Iterator_base12) wasn'tcommented at all, even though its invariants are non-obvious (intrusive
linked list of iterators, mutual container/proxy pointers, locking rules).
Stephan T. Lavavej (@StephanTLavavej) explained these invariants in the issue thread (originally
via a linked Discord screenshot from 2021), so this PR just captures that
explanation as a comment next to the code it describes, so it doesn't only
live in a screenshot linked from a 4-year-old issue comment.
I verified the described invariants (mutual _Myproxy/_Mycont pointers, the
_Myfirstiter/_Mynextiter intrusive list, the _Lockit(_LOCK_DEBUG) locking
rule) still match the current code exactly before writing this up.
No behavior change -- comment-only.
Disclosure: This change was drafted with AI assistance (Claude Code), at
my direction, using only the existing issue thread and the current header
as source material.