Skip to content

Don't use Regexp match for substring-after() XPath function - #355

Merged
kou merged 1 commit into
ruby:masterfrom
voxik:substring-after-no-regexp
Aug 17, 2026
Merged

Don't use Regexp match for substring-after() XPath function#355
kou merged 1 commit into
ruby:masterfrom
voxik:substring-after-no-regexp

Conversation

@voxik

@voxik voxik commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

This is the current behavior:

$ irb
irb(main):001> require "rexml/document"
=> true
irb(main):002> xml = "<root><x>helloworld</x></root>"
=> "<root><x>helloworld</x></root>"
irb(main):003> doc = REXML::Document.new(xml)
=> <UNDEFINED> ... </>
irb(main):004> 
irb(main):005> xpath1 = "//x[substring-after(text(), 'hello')='world']"
=> "//x[substring-after(text(), 'hello')='world']"
irb(main):006> REXML::XPath.match(doc, xpath1)
=> [<x> ... </>]
irb(main):007> 
irb(main):008> xpath2 = "//x[substring-after(text(), 'hel.o')='world']"
=> "//x[substring-after(text(), 'hel.o')='world']"
irb(main):009> REXML::XPath.match(doc, xpath2)
=> [<x> ... </>]
irb(main):010> REXML::VERSION
=> "3.4.5"

However, XPath expects string match according to specification:

https://www.w3.org/TR/1999/REC-xpath-19991116/#function-substring-after

therefore the match for xpath2 should actually return:

irb(main):008> xpath2 = "//x[substring-after(text(), 'hel.o')='world']"
=> "//x[substring-after(text(), 'hel.o')='world']"
irb(main):009> REXML::XPath.match(doc, xpath2)
=> []

Just for comparison, this is similar test case in Python:

$ python3
Python 3.15.0b4 (3.15.0~b4-1.fc45.x86_64, Jul 18 2026, 00:00:00) [GCC 16.1.1 20260703 (Red Hat 16.1.1-4)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from lxml import etree
... 
... xml = b'<root><x>helloworld</x></root>'
... doc = etree.fromstring(xml)
... 
... xpath1 = "//x[substring-after(text(), 'hello')='world']"
... result1 = doc.xpath(xpath1)
... print('Test 1 (hello):', result1)
... 
... xpath2 = "//x[substring-after(text(), 'hel.o')='world']"
... result2 = doc.xpath(xpath2)
... print('Test 2 (hel.o):', result2)
... 
Test 1 (hello): [<Element x at 0x7fcdbd972b00>]
Test 2 (hel.o): []

@voxik

voxik commented Aug 14, 2026

Copy link
Copy Markdown
Contributor Author

Not sure if the test case is in the right spot, or if it should not actually use the full REXML::XPath.match

@voxik

voxik commented Aug 14, 2026

Copy link
Copy Markdown
Contributor Author

BTW trying to dig into history, the RegExp behavior was introduced by 0d1b4da . But that is just some squashed commit. Trying to find the specific reasons for the RegExp, I was able to find the original commit here:

https://web.archive.org/web/20060901105224/http://www.germane-software.com/projects/rexml/changeset/1136

And this is roughly reconstruction of the patch:

From b41dec417658c7804ccbc7db25e929afc81a4f8d Mon Sep 17 00:00:00 2001
From: anonymous
Date: Thu, 15 Aug 2005 05:17:37 +0200
Subject: [PATCH] Fixes roundup issue 43: substring-after bug.

See: http://www.germane-software.com/cgi-bin/roundup/rexml/issue43
---
 lib/rexml/functions.rb | 9 +++------
 test/functions.rb      | 5 +++++
 2 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/lib/rexml/functions.rb b/lib/rexml/functions.rb
index 2550cb3..33895ca 100644
--- a/lib/rexml/functions.rb
+++ b/lib/rexml/functions.rb
@@ -158,12 +158,9 @@ module REXML
     # Kouhei fixed this too
     def Functions::substring_after( string, test )
       ruby_string = string(string)
-      ruby_index = ruby_string.index(string(test))
-      if ruby_index.nil?
-        ""
-      else
-        ruby_string[ ruby_index+1..-1 ]
-      end
+      test_string = string(test)
+      return $1 if ruby_string =~ /#{test}(.*)/
+      ""
     end
 
     # Take equal portions of Mike Stok and Sean Russell; mix
diff --git a/test/functions.rb b/test/functions.rb
index 16dec57..82e08a6 100644
--- a/test/functions.rb
+++ b/test/functions.rb
@@ -100,4 +100,9 @@
 end
 
+def test_substring_angrez
+   testString = REXML::Functions::substring_after("helloworld","hello")
+   assert_equal( 'world', testString )
+end
+
 def test_translate 
    source = <<-EOF

But it seems the original Roundup tracker with the details is lost :(

@voxik

voxik commented Aug 14, 2026

Copy link
Copy Markdown
Contributor Author

I also think that this implementation might be faster then the RegExp, but I have not tried and it was not the reason for this PR

@kou kou changed the title Don't use RegExp match for substring-after() XPath function Don't use Regexp match for substring-after() XPath function Aug 14, 2026
Comment thread lib/rexml/functions.rb Outdated
Comment on lines +208 to +211
needle = string(test)
index = ruby_string.index(needle)
return "" if index.nil?
ruby_string[(index + needle.length)..-1]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you use the same style?

Suggested change
needle = string(test)
index = ruby_string.index(needle)
return "" if index.nil?
ruby_string[(index + needle.length)..-1]
ruby_test = string(test)
ruby_index = ruby_string.index(ruby_test)
if ruby_index.nil?
""
else
ruby_string[(ruby_index + ruby_test.length)..-1]
end

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. Done

XPath expects `string` match according to specification:

https://www.w3.org/TR/1999/REC-xpath-19991116/#function-substring-after

So these two should not provide the same result (while they previously
did):

~~~
REXML::Functions::substring_after("helloworld","hello")
REXML::Functions::substring_after("helloworld","hel.o")
~~~
@voxik
voxik force-pushed the substring-after-no-regexp branch from 4dbc86b to 2e9903c Compare August 17, 2026 08:19
@kou
kou merged commit 1dfc59a into ruby:master Aug 17, 2026
71 checks passed
@kou

kou commented Aug 17, 2026

Copy link
Copy Markdown
Member

Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants