fix: batch discover prefilter equivalence + comma-generic selector round-trip (US-075, US-076) - #21
Merged
Merged
Conversation
Batching supported symbol queries dropped definitions that the same queries found individually, through two defects in one seam — the Aho-Corasick gate in find_definitions_batch: - the gate matched the raw query bytes, while the single search pre-scans with the plain name of a `Q.N` query, so a receiver/container-qualified term never admitted any file and the existing qualified matcher was never reached; - the gate used non-overlapping iteration, so a shorter term masked an overlapping longer one and the longer term's hit bit was never set. Both are fixed at the shared policy: definitions.rs now exports definition_scan_needle(), used by single and batch admission alike, and the batch gate iterates overlapping matches. Needles stay 1:1 with the queries, duplicates included, so a pattern id remains a query index. The original query still decides qualification in the matcher, usage matching is untouched, and single-symbol output is byte-for-byte unchanged. Adds candidate-level and CLI regressions for qualified terms, prefix overlap, shared needles, comment independence, ambiguity and no-match, plus batch cases in the US-071 selector and Windows path suites.
Adds bounded rows to the existing single-vs-batch definition oracle rather than new fixtures or helpers: - reversed Rust and Python batches, plus a qualified term batched with its own plain name, so the plain name keeps both definitions and the qualified term keeps only its own; - suffix/contained overlap (`extra` inside `helper_extra`) and a three-term overlap chain (`help`, `helper`, `helper_extra`) in both orders, each name owning a file that contains no other query name; - the self-hosted `search`/`search_batch` repro shape that first exposed the masked needle; - a Go generic receiver (`Store[T]`) batched with a plain receiver sharing the same method needle. AC-6 now batches five terms so the shared plain name keeps both file identities while a wrong qualifier and an absent name stay empty. The suffix row passes on the base binary as well: contained needles were already reported, so it guards the semantics rather than reproducing the bug. Every other added row fails on 39c6360.
Greptile SummaryThis PR restores equivalence between batched and single-symbol definition discovery and makes emitted generic selectors containing commas replayable across navigation commands.
Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains.
|
| Filename | Overview |
|---|---|
| src/format.rs | Adds shared balanced-angle framing so commas nested in generic selectors remain part of one target. |
| src/cli_run.rs | Applies shared framing to show and context while preserving target-count and budget bounds. |
| src/read/section.rs | Uses shared framing for section lists while resolving exact Markdown headings before interpreting commas or angle characters. |
| src/artifact.rs | Allows generic selectors through the artifact single-section path while deferring lists and malformed framing to the shared reader. |
| src/search/symbol/batch.rs | Aligns batch definition admission with single-symbol search and reports overlapping or duplicate pattern hits independently. |
| src/search/symbol/definitions.rs | Extracts the qualified-query scan-needle rule for consistent reuse by single and batch definition searches. |
| tests/us075_batch_prefilter_equivalence.rs | Verifies batch discovery preserves the same definition identities as independent qualified and overlapping queries. |
| tests/us076_comma_selector_round_trip.rs | Verifies emitted comma-bearing generic selectors replay across affected navigation and section commands. |
| tests/windows_paths.rs | Extends Windows-specific coverage for qualified batch discovery and generic-selector replay. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[CLI target or section input] --> B[Resolve exact Markdown heading when applicable]
B -->|Resolved| C[Read one heading section]
B -->|Not resolved or non-heading| D[Split at depth-zero commas]
D --> E[One framed target]
D --> F[Multiple framed targets]
E --> G[Resolve generic selector unchanged]
F --> H[Execute bounded target list]
I[Batch symbol queries] --> J[Derive definition scan needle]
J --> K[Overlapping Aho-Corasick prefilter]
K --> L[Run structural matcher with original query]
L --> M[Results equivalent to single-target search]
Reviews (3): Last reviewed commit: "release: 1.8.1" | Re-trigger Greptile
srcwalk prints `> Next: srcwalk show '<path>:Cache<K, V>.get'` and promises the printed command works unchanged, but every target-list consumer split that string at each comma, so the fragment `<path>:Cache<K` was resolved instead. The comma is a grammar collision: it separates targets in a list and is also legal inside a parser-backed generic selector. `format::split_target_list` frames a list once, splitting only at a comma whose generic angle depth is zero and rejecting unbalanced `<...>` before any target runs. Input without a comma is returned unvalidated, so nothing that parses today can start failing. The four framing sites now share it: show and context target lists, the show rejection-hint normalization, and section/range lists. The artifact JS/TS reader stops refusing every comma-bearing symbol and refuses only a real depth-zero list. Emission is unchanged, including generic parameters and the quoted path plus `--section` form chosen for comma paths; replaying that comma-path command still fails exactly as it does on the base, which is recorded rather than asserted as a working round-trip. Discover and query comma grammar is untouched.
Review finding: eager target-list framing ran before full-heading resolution, so an existing heading carrying both a comma and a literal `<` or `>` started failing as an unbalanced list (`show doc.md --section '## Compare A < B, C'` resolved on the base and exited 3 here). Framing is now lazy for the heading route: the full heading is matched first, exactly as before, and the list is framed only when no heading matches. Non-heading section input stays validated up front, so a malformed list still fails before any target runs. Also narrows the changelog wording: only a comma-separated target list is rejected for unbalanced angles; comma-free input is deliberately handed to the resolver unchanged for byte parity.
Review found two evidence gaps, no production behavior finding. The artifact reader's changed guard had no direct test. Its condition moves into a named private classifier so the boundary can be asserted without a JS/TS outline that emits generic arguments: a nested-comma selector reaches symbol lookup, while a real depth-zero list, a heading and a line range stay out of scope. Malformed input stays unclassified there on purpose, because the shared section framing rejects it before any body is read. The emitted-target contract also lacked language controls, so a parameterized CLI test drives TypeScript, Java and C# generic containers. Those outlines currently emit a non-generic selector, so the test asserts the actual emitted string and replays it, rather than assuming a generic spelling.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two bounded fixes to the same class of defect: srcwalk emitted or accepted a query that its own parser then handled differently from the single-target path.
US-075 — batch discover dropped definitions (
251a920,83b5535)discover 'a,b' --as symbollost definitions the same terms find alone. Two defects in the batch prefilter:NextAction.new) while the single path scans the plain name (new).find_iteris non-overlapping, sosearchmaskedsearch_batch.Fix (11 lines): extract
definition_scan_needle()from the single path and use it for batch; switch tofind_overlapping_iter. Usage path, qualifier matching, ranking, Text OR untouched.US-076 — emitted comma-generic selectors could not be replayed (
ad9d334,714fd0a,4cc0825)discoverprinted> Next: srcwalk show 'src/cache.rs:Cache<K, V>.get', butshow/context/--sectionsplit target lists at every comma, so the copied command failed withCache<Knot resolved. Pre-existing onmain.Fix: one shared
split_target_list()(src/format.rs) that splits only at depth-zero commas outside balanced<...>, wired intoshow,context, the show hint normaliser, section lists, and the artifact section guard. Markdown headings are resolved before framing so## Compare A < B, Ckeeps working. Unbalanced<...>in a list fails once before any target executes. Generic parameters are kept in emission becauseCache.getdoes not resolve and generics uniquely disambiguate overloads.Discover/query batching, Text OR, intent routing,
--skip-hubs, and other CSV/content parsers are untouched (25 comma sites inventoried: 9 changed, 2 inspected, 14 out of class).Verification
cargo test --locked1498 pass / 0 fail / 1 ignored;clippy --locked -- -D warnings;fmt --check;build --release --locked.tests/windows_paths.rsgains cfg(windows) cases for both stories — CI on this PR is the gate.Docs
GUIDE.md drops the dotted-in-batch workaround and documents that commas inside emitted
<...>are selector data; README examples; CHANGELOGUnreleased / Fixed. No version bump.Deferred (pre-existing, separate story)
show '<path,with,comma>' --section X— the emitted form for comma paths — is rejected by the CLI multi-target branch even though the resolver accepts it. Identical on base and candidate; tracked separately.