Skip to content

Match namespace-wildcard attribute values individually - #207

Merged
wRAR merged 1 commit into
scrapy:masterfrom
JackyHsu-X:fix/namespace-wildcard-attribute-values
Oct 20, 2026
Merged

wRAR merged 1 commit into
scrapy:masterfrom
JackyHsu-X:fix/namespace-wildcard-attribute-values

Conversation

@JackyHsu-X

Copy link
Copy Markdown
Contributor

When a namespace-wildcard attribute selector matches more than one attribute on an XML element, XPath 1.0 string functions inspect only the first attribute. For example, [*|label^="prefix"] misses <item a:label="other" b:label="prefix-token"/>. Matching depends on attribute order; ASCII case-insensitive equality has the same problem.

Apply standard value comparisons in a predicate on each matching attribute, then test whether any attribute remains. Regression tests cover all six standard value operators, attribute order, namespaced and unnamespaced attributes, case flags, empty values, and combinations with :not(), :is() and :has(). Presence selectors and the nonstandard != operator keep their existing translation.

Validation in an isolated environment on Windows/CPython 3.12.8:

  • New regression cases against the original translator: 53 failing subcases; all pass with the fix.
  • Complete source/unit/documentation-example suite: 33 passed, 76 subtests passed; xpath.py has 100% statement and branch coverage.
  • Independent per-attribute Python oracle: 16,362 element/selector comparisons passed.
  • Ruff lint/format checks, strict mypy for source and tests, and pylint passed.
  • Sphinx HTML build with warnings as errors, sdist/wheel builds, and Twine package checks passed.

The full platform/Python matrix and hosted CI have not been run.

@AdrianAtZyte AdrianAtZyte left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, trim the tests down: a few xpath() assertions next to the existing e[*|foo] one (e.g. ^=, = with i, and an empty value) plus a single small document-based test showing the second attribute matching would cover it. The loops over every operator × flag × order, and the :not()/:is()/:has() and non-ASCII cases, test things that are not specific to this change.

@codecov

codecov Bot commented Oct 19, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.79%. Comparing base (ec61e4b) to head (b5d8440).

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #207   +/-   ##
=======================================
  Coverage   99.79%   99.79%           
=======================================
  Files           3        3           
  Lines         987      994    +7     
  Branches      162      164    +2     
=======================================
+ Hits          985      992    +7     
  Misses          1        1           
  Partials        1        1           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@wRAR
wRAR merged commit a18ab0e into scrapy:master Oct 20, 2026
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants