Skip to content

fix: batch discover prefilter equivalence + comma-generic selector round-trip (US-075, US-076) - #21

Merged
sting8k merged 6 commits into
mainfrom
us-075-batch-prefilter-equivalence
Sep 6, 2026
Merged

sting8k merged 6 commits into
mainfrom
us-075-batch-prefilter-equivalence

Conversation

@sting8k

@sting8k sting8k commented Sep 6, 2026

Copy link
Copy Markdown
Owner

Two bounded fixes to the same class of defect: srcwalk emitted or accepted a query that its own parser then handled differently from the single-target path.

US-075 — batch discover dropped definitions (251a920, 83b5535)

discover 'a,b' --as symbol lost definitions the same terms find alone. Two defects in the batch prefilter:

  1. Qualified terms: the Aho-Corasick gate scanned the raw literal (NextAction.new) while the single path scans the plain name (new).
  2. find_iter is non-overlapping, so search masked search_batch.

Fix (11 lines): extract definition_scan_needle() from the single path and use it for batch; switch to find_overlapping_iter. Usage path, qualifier matching, ranking, Text OR untouched.

US-076 — emitted comma-generic selectors could not be replayed (ad9d334, 714fd0a, 4cc0825)

discover printed > Next: srcwalk show 'src/cache.rs:Cache<K, V>.get', but show/context/--section split target lists at every comma, so the copied command failed with Cache<K not resolved. Pre-existing on main.

Fix: one shared split_target_list() (src/format.rs) that splits only at depth-zero commas outside balanced <...>, wired into show, context, the show hint normaliser, section lists, and the artifact section guard. Markdown headings are resolved before framing so ## Compare A < B, C keeps working. Unbalanced <...> in a list fails once before any target executes. Generic parameters are kept in emission because Cache.get does not resolve and generics uniquely disambiguate overloads.

Discover/query batching, Text OR, intent routing, --skip-hubs, and other CSV/content parsers are untouched (25 comma sites inventoried: 9 changed, 2 inspected, 14 out of class).

Verification

  • Regression tests fail on base, pass after each fix; oracles compare definition identity (path + anchor + kind), not counts.
  • Byte parity on clean builds: US-075 12/12 single-query commands identical; US-076 24 commands / 72 stream files identical (including US-075 batch, Text OR, comma-path, heading controls).
  • cargo test --locked 1498 pass / 0 fail / 1 ignored; clippy --locked -- -D warnings; fmt --check; build --release --locked.
  • Perf (US-075, median of 9): plain batches unchanged; qualified batches 75→145ms / 142→203ms because more files are correctly admitted. Correctness costs work; not a speedup claim.
  • Windows: tests/windows_paths.rs gains cfg(windows) cases for both stories — CI on this PR is the gate.

Docs

GUIDE.md drops the dotted-in-batch workaround and documents that commas inside emitted <...> are selector data; README examples; CHANGELOG Unreleased / Fixed. No version bump.

Deferred (pre-existing, separate story)

show '<path,with,comma>' --section X — the emitted form for comma paths — is rejected by the CLI multi-target branch even though the resolver accepts it. Identical on base and candidate; tracked separately.

Batching supported symbol queries dropped definitions that the same queries
found individually, through two defects in one seam — the Aho-Corasick gate
in find_definitions_batch:

- the gate matched the raw query bytes, while the single search pre-scans with
  the plain name of a `Q.N` query, so a receiver/container-qualified term never
  admitted any file and the existing qualified matcher was never reached;
- the gate used non-overlapping iteration, so a shorter term masked an
  overlapping longer one and the longer term's hit bit was never set.

Both are fixed at the shared policy: definitions.rs now exports
definition_scan_needle(), used by single and batch admission alike, and the
batch gate iterates overlapping matches. Needles stay 1:1 with the queries,
duplicates included, so a pattern id remains a query index. The original query
still decides qualification in the matcher, usage matching is untouched, and
single-symbol output is byte-for-byte unchanged.

Adds candidate-level and CLI regressions for qualified terms, prefix overlap,
shared needles, comment independence, ambiguity and no-match, plus batch cases
in the US-071 selector and Windows path suites.
Adds bounded rows to the existing single-vs-batch definition oracle rather
than new fixtures or helpers:

- reversed Rust and Python batches, plus a qualified term batched with its own
  plain name, so the plain name keeps both definitions and the qualified term
  keeps only its own;
- suffix/contained overlap (`extra` inside `helper_extra`) and a three-term
  overlap chain (`help`, `helper`, `helper_extra`) in both orders, each name
  owning a file that contains no other query name;
- the self-hosted `search`/`search_batch` repro shape that first exposed the
  masked needle;
- a Go generic receiver (`Store[T]`) batched with a plain receiver sharing the
  same method needle.

AC-6 now batches five terms so the shared plain name keeps both file
identities while a wrong qualifier and an absent name stay empty.

The suffix row passes on the base binary as well: contained needles were
already reported, so it guards the semantics rather than reproducing the bug.
Every other added row fails on 39c6360.
@greptile-apps

greptile-apps Bot commented Sep 6, 2026

Copy link
Copy Markdown

Greptile Summary

This PR restores equivalence between batched and single-symbol definition discovery and makes emitted generic selectors containing commas replayable across navigation commands.

  • Shares qualified-definition scan-needle derivation between single and batch search and admits overlapping Aho-Corasick matches.
  • Adds balanced-angle target-list framing across show, context, section reads, hints, and artifact handling.
  • Adds cross-command, heading, generic-selector, batch-equivalence, and Windows regression coverage.
  • Synchronizes release metadata and user/agent documentation for version 1.8.1.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
src/format.rs Adds shared balanced-angle framing so commas nested in generic selectors remain part of one target.
src/cli_run.rs Applies shared framing to show and context while preserving target-count and budget bounds.
src/read/section.rs Uses shared framing for section lists while resolving exact Markdown headings before interpreting commas or angle characters.
src/artifact.rs Allows generic selectors through the artifact single-section path while deferring lists and malformed framing to the shared reader.
src/search/symbol/batch.rs Aligns batch definition admission with single-symbol search and reports overlapping or duplicate pattern hits independently.
src/search/symbol/definitions.rs Extracts the qualified-query scan-needle rule for consistent reuse by single and batch definition searches.
tests/us075_batch_prefilter_equivalence.rs Verifies batch discovery preserves the same definition identities as independent qualified and overlapping queries.
tests/us076_comma_selector_round_trip.rs Verifies emitted comma-bearing generic selectors replay across affected navigation and section commands.
tests/windows_paths.rs Extends Windows-specific coverage for qualified batch discovery and generic-selector replay.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[CLI target or section input] --> B[Resolve exact Markdown heading when applicable]
    B -->|Resolved| C[Read one heading section]
    B -->|Not resolved or non-heading| D[Split at depth-zero commas]
    D --> E[One framed target]
    D --> F[Multiple framed targets]
    E --> G[Resolve generic selector unchanged]
    F --> H[Execute bounded target list]
    I[Batch symbol queries] --> J[Derive definition scan needle]
    J --> K[Overlapping Aho-Corasick prefilter]
    K --> L[Run structural matcher with original query]
    L --> M[Results equivalent to single-target search]
Loading

Reviews (3): Last reviewed commit: "release: 1.8.1" | Re-trigger Greptile

srcwalk prints `> Next: srcwalk show '<path>:Cache<K, V>.get'` and promises the
printed command works unchanged, but every target-list consumer split that
string at each comma, so the fragment `<path>:Cache<K` was resolved instead.
The comma is a grammar collision: it separates targets in a list and is also
legal inside a parser-backed generic selector.

`format::split_target_list` frames a list once, splitting only at a comma whose
generic angle depth is zero and rejecting unbalanced `<...>` before any target
runs. Input without a comma is returned unvalidated, so nothing that parses
today can start failing. The four framing sites now share it: show and context
target lists, the show rejection-hint normalization, and section/range lists.
The artifact JS/TS reader stops refusing every comma-bearing symbol and refuses
only a real depth-zero list.

Emission is unchanged, including generic parameters and the quoted path plus
`--section` form chosen for comma paths; replaying that comma-path command still
fails exactly as it does on the base, which is recorded rather than asserted as
a working round-trip. Discover and query comma grammar is untouched.
Review finding: eager target-list framing ran before full-heading resolution,
so an existing heading carrying both a comma and a literal `<` or `>` started
failing as an unbalanced list (`show doc.md --section '## Compare A < B, C'`
resolved on the base and exited 3 here).

Framing is now lazy for the heading route: the full heading is matched first,
exactly as before, and the list is framed only when no heading matches.
Non-heading section input stays validated up front, so a malformed list still
fails before any target runs.

Also narrows the changelog wording: only a comma-separated target list is
rejected for unbalanced angles; comma-free input is deliberately handed to the
resolver unchanged for byte parity.
Review found two evidence gaps, no production behavior finding.

The artifact reader's changed guard had no direct test. Its condition moves
into a named private classifier so the boundary can be asserted without a
JS/TS outline that emits generic arguments: a nested-comma selector reaches
symbol lookup, while a real depth-zero list, a heading and a line range stay
out of scope. Malformed input stays unclassified there on purpose, because the
shared section framing rejects it before any body is read.

The emitted-target contract also lacked language controls, so a parameterized
CLI test drives TypeScript, Java and C# generic containers. Those outlines
currently emit a non-generic selector, so the test asserts the actual emitted
string and replays it, rather than assuming a generic spelling.
@sting8k sting8k changed the title fix: share the definition scan needle between single and batch discover (US-075) fix: batch discover prefilter equivalence + comma-generic selector round-trip (US-075, US-076) Sep 6, 2026
@sting8k
sting8k merged commit f046cbf into main Sep 6, 2026
7 checks passed
@sting8k
sting8k deleted the us-075-batch-prefilter-equivalence branch September 6, 2026 10:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant