Skip to content

Fix document order in REXML::XPathParser.sort - #356

Merged
kou merged 1 commit into
ruby:masterfrom
naitoh:fix_xpath_sort_document_order
Aug 16, 2026
Merged

Fix document order in REXML::XPathParser.sort#356
kou merged 1 commit into
ruby:masterfrom
naitoh:fix_xpath_sort_document_order

Conversation

@naitoh

@naitoh naitoh commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

REXML::XPathParser.sort keys each node on its index under each ancestor, but two different nodes could end up with the same key, and ties are then broken arbitrarily by an unstable sort.

The walk stopped at the root element, so the root and everything outside it -- comments and PIs at document level -- all keyed on an empty array:

    <!--c1--><?pi1?><root/><!--c2--><?pi2?>
//comment() | //processing-instruction()
  was: c1, c2, pi1, pi2   want: c1, pi1, c2, pi2

Walk up to the document instead, so those nodes are ordered against each other.

An attribute borrowed the key of the element carrying it, so the element and its attributes tied:

    <root><a x="1" y="2"><b/></a></root>
//a/@* | //a | //a/*
  was: @x, @y, <a>, <b>   want: <a>, @x, @y, <b>

Extend an attribute's key past its element's with ATTRIBUTE_POSITION. A child index is never negative, so -1 lands the attributes after <a> and ahead of <b>, which is where document order wants them.

The attributes of one element tied with each other too, which went unnoticed because a small enough sort leaves its input alone:

<a z="1" m="2" b="3"/>       //a/@*  ->  z, m, b
<a z="1" ... 26 attributes/> //a/@*  ->  scrambled

XPath 1.0 leaves their relative order implementation dependent, but document order is a total ordering, so it has to be decided. Key them on where they were written, which is the order the small case already appeared to have. Finding that out means walking the attribute list, so index the whole list at once and remember it for the rest of the sort; asking per attribute would make sorting quadratic in the number of attributes an element carries, which is something a document gets to choose.

The ancestor walk becomes a method of its own, since both branches of the key need it. It, attribute_position and ATTRIBUTE_POSITION are private rather than public names marked :nodoc:, sort being the only part callers outside the class use.

@naitoh naitoh changed the title Fix document order of nodes that tie on their ancestor indexes Fix document order in REXML::XPathParser.sort Aug 14, 2026
@naitoh
naitoh requested a review from kou August 14, 2026 14:47
@kou
kou requested a lite review from Copilot August 14, 2026 22:58

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes REXML::XPathParser.sort so XPath results are reliably returned in true document order, including for document-level nodes (comments/PIs outside the root) and for attributes (which must be totally ordered even though XPath 1.0 leaves attribute order implementation-dependent).

Changes:

  • Extend the ancestor-walk key generation up to the document node so document-level siblings are ordered correctly.
  • Disambiguate attribute ordering by extending an attribute’s sort key past its owning element’s key using a dedicated attribute position marker and a per-element attribute index cache.
  • Add targeted regression tests covering top-level nodes, descendant-or-self from the document, element-vs-attribute ordering, attribute list ordering, and mixed child node types.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
lib/rexml/xpath_parser.rb Reworks XPathParser.sort keying to include document-level ordering and stable attribute ordering via cached attribute positions.
test/xpath/test_base.rb Adds regression tests to validate document order for top-level nodes, attributes, and mixed child content.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread lib/rexml/xpath_parser.rb Outdated
Comment thread test/xpath/test_base.rb Outdated
Comment thread test/xpath/test_base.rb Outdated
Comment thread test/xpath/test_base.rb Outdated
Comment thread test/xpath/test_base.rb Outdated
Comment thread test/xpath/test_base.rb Outdated
@naitoh
naitoh force-pushed the fix_xpath_sort_document_order branch from bdaf164 to 10ddfec Compare August 16, 2026 09:14
@naitoh
naitoh requested review from kou and tompng August 16, 2026 09:17
@naitoh

naitoh commented Aug 16, 2026

Copy link
Copy Markdown
Contributor Author

I have addressed the issues pointed out by PR
Please review it again.

Comment thread test/xpath/test_base.rb Outdated
@naitoh
naitoh force-pushed the fix_xpath_sort_document_order branch from 10ddfec to ff4d81d Compare August 16, 2026 10:19
@naitoh
naitoh requested a review from kou August 16, 2026 10:22
Comment thread test/xpath/test_base.rb Outdated
REXML::XPathParser.sort keys each node on its index under each ancestor,
but two different nodes could end up with the same key, and ties are
then broken arbitrarily by an unstable sort.

The walk stopped at the root element, so the root and everything outside
it -- comments and PIs at document level -- all keyed on an empty array:

    <!--c1--><?pi1?><root/><!--c2--><?pi2?>

    //comment() | //processing-instruction()
      was: c1, c2, pi1, pi2   want: c1, pi1, c2, pi2

Walk up to the document instead, so those nodes are ordered against each
other.

An attribute borrowed the key of the element carrying it, so the element
and its attributes tied:

    <root><a x="1" y="2"><b/></a></root>

    //a/@* | //a | //a/*
      was: @x, @y, <a>, <b>   want: <a>, @x, @y, <b>

Extend an attribute's key past its element's with ATTRIBUTE_POSITION.  A
child index is never negative, so -1 lands the attributes after <a> and
ahead of <b>, which is where document order wants them.

The attributes of one element tied with each other too, which went
unnoticed because a small enough sort leaves its input alone:

    <a z="1" m="2" b="3"/>       //a/@*  ->  z, m, b
    <a z="1" ... 26 attributes/> //a/@*  ->  scrambled

XPath 1.0 leaves their relative order implementation dependent, but
document order is a total ordering, so it has to be decided.  Key them
on where they were written, which is the order the small case already
appeared to have.  Finding that out means walking the attribute list, so
index the whole list at once and remember it for the rest of the sort;
asking per attribute would make sorting quadratic in the number of
attributes an element carries, which is something a document gets to
choose.

The ancestor walk becomes a method of its own, since both branches of
the key need it.  It, attribute_position and ATTRIBUTE_POSITION are
private rather than public names marked :nodoc:, sort being the only
part callers outside the class use.
@naitoh
naitoh force-pushed the fix_xpath_sort_document_order branch from ff4d81d to aa27972 Compare August 16, 2026 12:36
@naitoh
naitoh requested a review from kou August 16, 2026 12:39
@kou
kou requested a lite review from Copilot August 16, 2026 20:16

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

@kou
kou merged commit 85ac579 into ruby:master Aug 16, 2026
71 checks passed
@kou

kou commented Aug 16, 2026

Copy link
Copy Markdown
Member

Thanks.

@naitoh
naitoh deleted the fix_xpath_sort_document_order branch August 17, 2026 01:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants