Skip to content

A comment still silences tests_referenced_check, through #2445's selftest_named #2464

Closed

Description

@bong-water-water-bong

A comment still silences tests_referenced_check, through #2445's selftest_named

#2460 made the gate require an invocation instead of a mention, and while doing so found that _statements()
counted parentheses inside comments. That is fixed on main — but the fix covers the paths that go through
_statements()/invokes(). #2445 landed a second path that reads raw text.

for t in runner_text:                     # runner_text.append(path.read_text(...)) -- comments included
    for m in re.finditer(r"tools/ci/([A-Za-z0-9_]+)\.py\s+--self[-_]?test", t):
        selftest_named.add(m.group(1) + "_test.py")
...
if q.name in imported or q.name in selftest_named:
    continue                              # the finding is SUPPRESSED

selftest_named only ever removes a finding, so a comment is enough to make the check go quiet about a test
nothing runs. Measured on the merged regex:

input:  '# note: we could add tools/ci/helper.py --self-test here one day'
match:  ['helper']      -> selftest_named gets helper_test.py -> the finding is dropped

The same block also substring-matches q.name in t against the same raw text.

This is the same defect as #2456, one layer out: a mention satisfying a gate. The tool half is already safe,
because invokes() runs over _statements(), which now drops comments. The test half needs the same treatment.

Fix: strip comments from the text collected for the test half (_strip_line_comments already exists for this), and
add the arm — a comment naming the tool must not clear tests/<tool>_test.py.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions