Skip to content

original-operator family (!): bypass any overload; tuple-destructure shadowing check; class-finalize module pin - #3684

Merged
borisbat merged 5 commits into
masterfrom
bbatkin/original-operators
Aug 10, 2026
Merged

original-operator family (!): bypass any overload; tuple-destructure shadowing check; class-finalize module pin#3684
borisbat merged 5 commits into
masterfrom
bbatkin/original-operators

Conversation

@borisbat

Copy link
Copy Markdown
Collaborator

The ! original-operator family

! in front of an overloadable access or test operator yields the original operator, never the overload:

Raw spelling Original operation
a!.x field access (skips operator .)
a!?.x null-safe field access (skips operator ?.)
a![i] indexing (skips operator [])
a!?[i] null-safe indexing (skips operator ?[])
a !?? b pointer null-coalescing (skips operator ??)
a !is x variant / type check (skips operator is)
a !as x variant access (skips operator as)
a !?as x safe variant access (skips operator ?as)

The four access forms are ergonomic spellings of the existing no_promotion bypass, whose dot-prefixed spellings (a. .x, a.?.x, a.[i], a.?[i]) remain valid and equivalent. The variant and coalescing forms close a real hole: is/as/?as/is type/?? promoted to generic operators and variant macros unconditionally, before regular infer, so a matching user overload could shadow real variant access or pointer coalescing with no escape spelling. The new forms carry no_promotion on ExprIs/ExprNullCoalescing (new flag) and on the ExprField-derived variant nodes (existing flag), gated in infer ahead of both the generic-operator promotion and the variantMacros preamble. Bypassing variant macros too is deliberate: a macro substitution is exactly as "overwritten" as an operator, and generated code escaping to the original needs to escape both.

Grammar: zero conflicts (%expect 0 holds; validated with bison on a scratch copy before touching the tree). Lexer: the word forms use match-plus-unput guards so !isfoo stays !(isfoo); ![/!?[ maintain the square-brace counter like ?[. The only re-lexed spelling is !.5 (previously !(0.5), which never typechecked). Both AST printers and the source formatter learned the spellings; the formatter's token templates deliberately exclude !is/!as (no word boundary in template matching — they would steal the prefix of !is_unary(...)-style expressions; the formatter formatting its own source caught this) and keep !?as, which cannot precede a letter in valid source and breaks without a template.

daslib/match deliberately stays on plain is/as: its expr is ExprField patterns on AST nodes depend on the ast_boost variant macros. Raw ops are for code that wants builtin variant semantics specifically.

Tuple destructuring now enforces let shadowing rules (doc-sweep ruling)

let (ok, a) = p1(); let (ok, b) = p2() compiled and silently rebound ok (the destructured names are assume aliases on a hidden compound local, and the newest alias won in ExprVar resolution) — while plain let x twice was error[30704]. expandTupleName now checks new names against existing aliases, locals, function/block arguments, with-scopes, and duplicates within one pattern; covers for ((k, v) in ...) too; respects can_shadow and the allow_local_variable_shadowing policy. Fallout: zero in-tree (full suite green), zero doc-corpus (doc-verify 402 green / 0 red).

Generated class finalizer pins to the defining module (doc-sweep ruling)

A class whose name matches a class in a required module failed its own generated-method resolution — error[30810] _::X'__finalize ambiguous with the twin's. makeClassFinalize now emits __::X'__finalize (strict this-module), since the generated finalizer always lives with the class. Named residual: template classes stay on open _:: — their stamped instances infer in the consumer module while the stamped finalizer may live elsewhere (tests/typemacro/test_template_structure_class caught the strict pin), so a template class name colliding with a required module's class keeps the old ambiguity error. The lambda/generator _:: sites are untouched: lambda names embed the module name, so they are collision-proof by construction.

ManagedVectorAnnotation doc fix (user report)

embedding/cpp_api.rst showed addAnnotation(new ManagedVectorAnnotation(...)) over the element type — born-wrong twice: the template parameter is the vector type, and plain addAnnotation registers none of the container functions. Now shows addVectorAnnotation over vector of int32_t with (this, lib, "IntVector") and says why. It is a cpp block, outside doc-verify's das compile gate, which is how it survived the sweep.

Tests

  • tests/language/operators.das: raw-operator suite with deliberately lying overloads (operator [] on array of float returning -999 etc.) proving each ! form reaches the builtin; legacy . . equivalence pinned.
  • tests/typer_errors/failed_tuple_destructure_shadowing.das: all five bypass shapes now error 30704 (between destructures, within one pattern, over a local, over an argument, in a for-loop).
  • tests/language/class_name_collision.das (+ helper module): same-named class compiles, cross-module derivation instantiates and finalizes at runtime.
  • Full tree 13,344 tests green (interp), operators/collision suites green under -jit, doc-verify 402/0, tree-wide formatter verify clean on tracked files.
  • Preflight --full: 15/17 first pass; the two docs-gate failures (positional handmade docs for the new reflected fields; sphinx cascade from the das2rst panic) fixed and re-validated with --only docs (6/6). Pushed with --no-verify per the fix-after-full-run policy.

Docs: expressions.rst (family table, primary spelling), functions.rst, CLAUDE.md gen2 bullet, the distributable skills/daslang bundle; ledger updates in plans/doc-sweep.md.

🤖 Generated with Claude Code

borisbat and others added 4 commits August 10, 2026 07:09
…ator bypasses the overload

`a!.x`, `a!?.x`, `a![i]`, `a!?[i]`, `a !?? b`, `a !is x`, `a !as x`, `a !?as x` -
one rule for the whole family: the original operator, never the overload. The four
access forms are ergonomic spellings of the existing no_promotion bypass (whose
dot-prefixed spellings `. .`, `.?.`, `.[`, `.?[` remain valid); the variant and
coalescing forms close a real hole: `is`/`as`/`?as`/`??`/`is type` promoted to
generic operators (and variant macros) unconditionally, so a matching user overload
could shadow real variant access or pointer coalescing with no escape spelling.
The new forms carry no_promotion on ExprIs/ExprNullCoalescing (new flag) and the
ExprField-derived variant nodes (existing flag), gated in infer ahead of both the
generic-operator promotion and the variantMacros preamble.

Zero grammar conflicts (%expect 0 holds); lexer guards keep `!isfoo` lexing as
`!(isfoo)`; both AST printers and the source formatter learned the new spellings.
Tests: raw-operator suite in tests/language/operators.das with shadowing overloads
proving each `!` form reaches the builtin (interp + jit green, full tests/language
1554/1554); docs in expressions.rst/functions.rst (doc-verify green), CLAUDE.md,
and the distributable daslang skill.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…finalize module pin, ManagedVectorAnnotation doc fix

Three items from the 0.6.4 doc-sweep ledger:

- Tuple destructuring now enforces the same shadowing rules as `let`. The
  destructured names are assume aliases on a hidden compound local, and nothing
  checked a new alias against existing aliases, locals, function/block arguments,
  with-scopes, or duplicates within one pattern - the newest alias silently won in
  ExprVar resolution while plain `let x` twice was error[30704]. expandTupleName
  now checks all of it (covers `for ((k, v) in ...)` too; respects `can_shadow`
  and the allow_local_variable_shadowing policy). Zero in-tree fallout (full
  suite green) and zero doc-corpus fallout (doc-verify 402/0).

- A class whose name matches a class in a required module no longer fails its own
  generated finalizer resolution (error[30810] `_::X'__finalize` ambiguous with the
  twin's). makeClassFinalize pins the generated address to the defining module
  (`__::`) - the finalizer always lives with the class. Named residual: template
  classes stay on open `_::` because their stamped instances infer in the consumer
  module while the stamped finalizer may live elsewhere
  (tests/typemacro/test_template_structure_class caught the strict pin).

- embedding/cpp_api.rst ManagedVectorAnnotation block was born-wrong (user report):
  plain addAnnotation registers the type but none of the container functions, and
  the template parameter is the vector type, not the element. Now shows
  addVectorAnnotation<vector<int32_t>>(this, lib, "IntVector"). It is a cpp block,
  outside doc-verify's das compile gate, which is how it survived the sweep.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…-caught)

The formatter's token templates have no word boundary, so the new `!is`/`!as`
entries stole the prefix of `!is_unary(...)`-style unary-not expressions - the
formatter formatting its own source caught it by breaking itself. The letter-
ending templates are out (they format fine as `!` + keyword); `!?as` stays,
since `!?as<letter>` cannot appear in valid source and without the template the
`?` gets spaced apart. Pre-existing STYLE037/038/PERF030 debt in the touched
files takes the sanctioned nolint (flat token matchers, printer-coverage
fixture).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
das2rst validates handmade docs positionally, so the new reflected field needs
its description line in both annotation files.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings August 10, 2026 15:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends daslang’s expression syntax and type inference to support an “original-operator” bypass family using ! (e.g. a!.x, a![i], a !?? b, a !is x), closes shadowing holes in tuple destructuring name binding, and fixes generated class finalizer resolution to be module-local to avoid cross-module name collisions. It also updates formatter/printers, documentation, and adds targeted regression tests.

Changes:

  • Add lexer/parser support and AST/infer plumbing for ! original-operator spellings (including bypassing generic-operator promotion and variant macros where applicable).
  • Enforce let-style shadowing rules for tuple destructuring (including for ((k,v) in ...) tuple iterators).
  • Pin generated class __finalize address lookup to the defining module (__::) to prevent ambiguity when required modules define same-named classes; update docs and tests accordingly.

Reviewed changes

Copilot reviewed 27 out of 28 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/typer_errors/failed_tuple_destructure_shadowing.das New negative test ensuring tuple destructuring respects shadowing/redeclaration rules (30704).
tests/language/operators.das Adds tests (and deliberately “lying” overloads) validating each ! raw operator reaches builtin semantics.
tests/language/class_name_collision.das Regression test for same-named classes across modules and generated finalizer resolution.
tests/language/_finalize_collision_helper.das Helper module defining a colliding class name for the finalizer-resolution test.
src/parser/lex2.yy.h Regenerated lexer header line markers after lexer changes.
src/parser/ds2_parser.ypp Introduces tokens/precedence and grammar productions for ! raw operator spellings.
src/parser/ds2_parser.hpp Regenerated parser token enum to include new NOT* tokens.
src/parser/ds2_lexer.lpp Adds lexing rules for !., !?., ![, !?[, !??, !is, !as, !?as with boundary guards.
src/parser/ds2_lexer.cpp Regenerated flex output reflecting the new lexer rules/state tables.
src/builtin/module_builtin_ast_annotations_2.cpp Exposes no_promotion fields for ExprNullCoalescing and ExprIs to the AST annotation API.
src/ast/ast.cpp Ensures no_promotion is preserved through cloning for relevant AST nodes.
src/ast/ast_print.cpp Prints the new raw operator spellings when no_promotion is set on relevant nodes.
src/ast/ast_infer_type.cpp Gates generic-operator promotion and variant-macro substitution based on no_promotion.
src/ast/ast_infer_type_helper.cpp Implements tuple-destructure shadowing checks in expandTupleName (now accepts canShadow).
src/ast/ast_generate.cpp Pins generated class finalizer address to __:: for non-template classes; retains _:: for templates.
include/daScript/ast/ast_infer_type.h Updates expandTupleName signature to include canShadow (defaulted).
include/daScript/ast/ast_expressions.h Adds no_promotion fields to ExprNullCoalescing and ExprIs.
doc/source/stdlib/handmade/structure_annotation-ast-ExprNullCoalescing.rst Documents the new no_promotion field meaning for ExprNullCoalescing (!??).
doc/source/stdlib/handmade/structure_annotation-ast-ExprIs.rst Documents the new no_promotion field meaning for ExprIs (!is).
doc/source/reference/language/functions.rst Updates operator-overload docs to use !. and references the original-operator family.
doc/source/reference/language/expressions.rst Adds a dedicated “Original Operator Access (!)” section and family table; notes legacy equivalences.
doc/source/reference/embedding/cpp_api.rst Fixes ManagedVectorAnnotation example to use addVectorAnnotation<vector<T>> and explains why.
skills/daslang/references/functions.md Updates shipped skill docs to prefer !. and documents the full ! original-operator family.
daslib/das_source_formatter.das Teaches formatter tokenization/spacing about new ! operator spellings; adds nolint notes for large dispatch.
daslib/ast_print.das Updates das-side AST pretty-printer to emit new raw operator spellings based on flags.
plans/doc-sweep.md Updates doc-sweep ledger items to reflect fixes landed on this branch.
CLAUDE.md Updates gen2/language guidance to document the new original-operator access family.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread daslib/ast_print.das Outdated
…, swizzle flag, generated-variant raw ops, tooling parity

Fixes from the opus /code-review round plus the Copilot comment and the CI tutorial
fallout, per-item approved:

- `a![i]` now bypasses on the WRITE path too: the three ExprAt write-side promotion
  sites (assignment-to-property, compound-op, move) gate on no_promotion like the
  read path; write/compound arms added to the raw-operator tests.
- The AST serializer carries both new no_promotion bools (ExprNullCoalescing, ExprIs)
  and the version bumps 108 -> 109 - a cached generic with `!??`/`!is type<>` no
  longer re-promotes on cache-hit compiles. Ser/deser sweep green.
- `__::` is now strict in findFuncAddr: module-name-filtered foreach also walks the
  named module's public dependencies, so a `require X public` twin still made the
  generated finalizer ambiguous. New public-require collision test.
- `_` is the discard in tuple destructuring (Boris-ruled): binds nothing, repeats
  freely - destructuring forces a name per position, unlike plain `let _`, which
  stays strict. The alias-collision checks (same-scope replacement, not shadowing)
  now run unconditionally, mirroring the sibling checks; the for-loop form reports
  30708 like plain iterator shadowing; duplicate reports dedupe. doc-verify's
  renamer leaves destructure `_` verbatim. Both CI-red tutorials fixed (one rename,
  one healed by the discard) plus a discard demo in the tuples tutorial and docs.
- ExprSwizzle carries no_promotion (flags word - serialized wholesale), the
  ExprField->ExprSwizzle rewrite propagates it, and both printers spell it, so
  `v!.xyz` round-trips through printed ASTs.
- Generated variant clone/finalize code marks its `is`/field accesses raw - a lying
  `operator is` can no longer hijack `:=` of a variant into an empty payload.
- `obj!.method()` parses (NOTDOT twins for the invoke arms; %expect 0 still holds);
  makeInvokeMethod threads no_promo onto the inner field.
- tree-sitter-daslang learned all 8 raw spellings (+ method-call arm, corpus test,
  45/45; dylib rebuilt) so MCP parse-aware tools see real nodes, not ERROR.
- Formatter: `![`/`!?[` count as square-bracket openers and unary context; the
  variant-macro dispatch triplication in infer is factored into runVariantMacros.
- das ast_print: `is type<...>` gets its leading space (Copilot).

Full tree green, tutorials 220/220, doc-verify 402/0, docs gates 6/6, lint clean on
all three rails, cpp-syntax 177-TU sweep clean. Deferred by agreement: flags-word
unification for the two loose bools (review finding 15).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 10, 2026 22:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 45 out of 47 changed files in this pull request and generated no new comments.

Suppressed comments (1)

src/ast/ast_print.cpp:1049

  • ExprSafeField pretty-print currently emits the non-raw operator as .? (e.g. obj.?field), which doesn’t match the language spelling ?. and won’t round-trip through the parser. Since this function is being updated for the raw !?. form, it’s a good time to correct the normal form to ?. as well (keeping !?. for no_promotion).
        virtual ExpressionPtr visit ( ExprSafeField * field ) override {
            if ( printRef && field->r2v ) ss << "@";
            if ( printRef && field->r2cr ) ss << "$";
            if ( printRef && field->write ) ss << "#";
            ss << (field->no_promotion ? "!?." : ".?") << field->name;
            return Visitor::visit(field);

@borisbat
borisbat merged commit b6a3400 into master Aug 10, 2026
38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants