Release/v1.2.0 - #2
Merged
Merged
Conversation
Ships Workstream A in full: statement-boundary hardening, number lexing, the dialect_ambiguous guardrail, and bracket/block balance diagnostics with correct document/region scope. - dialects: semicolon-free statements now split by grammar and control keywords instead of requiring a newline; SQLite SELECT RAISE(...) stays one statement; detectDialect reports detection ties - tokenizer: lex 0x hex, underscore digit separators, and 1./.5 forms - ir: dialect_ambiguous (document) and unexpected_end (region) diagnostics; token attribution marks by token identity so every body token is accounted for - tests: boundary.ts range fixtures for all four dialects (terminated and unterminated statements), number/detection/balance/findBody assertions, and fuzz invariants for attribution, edge kinds, and provenance - docs: README, PR_NOTE_1.2.0, RELEASE_NOTE_1.2.0, RELEASE_NOTE_v1.2.0 Correctness gate green: typecheck, build, test:file, golden 156/156, fuzz 400/400, ui 13/13, local-only check, dist committed in sync.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR: v1.2.0 — Trust statement boundaries
Summary
This PR implements the v1.2.0 — Trust statement boundaries milestone from
ROADMAP.md. It ships Workstream A (statement-boundary hardening) in full,plus the E boundary diagnostics (
dialect_ambiguous, bracket/block balance)with correct document/region scope. No existing golden changed: the fixture
corpus grew and every prior case remains structurally identical.
What's included
Statement boundaries (
src/dialects.ts)control keywords; a newline is no longer required. Previously
newStatementHeredemanded a newline, so one-line semicolon-free statementssuch as
SET @x = 1 PRINT @x SELECT @xcollapsed into a single statement.They now parse into three statements.
SELECT RAISE(...)idiom stays one statement (RAISEdoes notsplit when it directly follows
SELECT).INSERT…SELECT,UPDATE…OUTPUT,MERGE, andCASE-expression continuation guards are preserved.Number lexing (
src/tokenizer.ts)0x/0Xhexadecimal literals (0x1F,0x1_000).1_000,1_000_000)..5) and trailing-point forms (1.) besides theexisting
1.5exponent forms.dbo.t) and..ranges are unaffected.Diagnostics (
src/ir.ts,src/dialects.ts)dialect_ambiguousdocument-scoped warning on low-confidence detectionties (auto mode, several dialects tied). It carries no artificial span.
unexpected_endregion-scoped warning when a block-terminating keyword(
END,ELSE, …,UNTIL) has no matching opener, reported alongsideunconsumed_input.unterminated_identifier, andmissing_enddiagnostics are asserted with region scope and valid spans.
Detection guardrail (
src/dialects.ts,src/types.d.ts)detectDialectnow reportstied(top scores equal), which drives thedialect_ambiguousguardrail.Token-attribution fix (
src/ir.ts)span-based marking could hide filtered semicolons inside an unresolved
region, so attribution did not always account for every body token. The
fuzz suite now asserts the accounting invariant: resolved + ignored +
unresolved + opaque always equals the body-token total, every edge has a
recognised kind, and every node carries provenance.
Fixtures (
tests/tests.ts,tests/boundary.ts,tests/fuzz.ts)tests/boundary.tswith:PROCFLOW_RANGE_FIXTURES— exact statement source ranges for terminatedand valid unterminated statements across all four dialects, including the
one-line semicolon-free cases;
PROCFLOW_FIXTURESforCREATE VIEW … WITH, CTE-starting viewbodies,
ALTER TRIGGER, and semicolon-free one-line T-SQL.tests/tests.tsgained range-running plus direct assertions for numberlexing,
dialect_ambiguous, bracket/block balance diagnostics, and thehardened
findBodyheaders (CREATE VIEW … WITH,ALTERobject headers,multi-object scripts without
GO).tests/fuzz.tsadded the token-attribution, edge-kind, and provenanceinvariants.
Verification
npm run typecheck— passesnpm run build— passesnpm run test:file— passesunattributed-token rate 4.95 % (down from 5.05 % on the prior corpus);
zero unknown-node fallbacks.
Deferred (per roadmap)