Skip to content

perf(treesitter): compile AST rules into lookup sets and cache pointers - #118

Merged
HarshK97 merged 1 commit into
mainfrom
perf/rules-lookup-optimization
Aug 19, 2026
Merged

perf(treesitter): compile AST rules into lookup sets and cache pointers#118
HarshK97 merged 1 commit into
mainfrom
perf/rules-lookup-optimization

Conversation

@HarshK97

Copy link
Copy Markdown
Owner

Problem

  1. When building an AST, buildASTWithRules was running slices.Contains across 5 different slices (Ignored, Keywords, LabelIgnored, Unordered, Flattened) on every single node, so on larger files with thousands of nodes, doing linear scans over and over added unnecessary overhead.
  2. AreTypesEquivalent was looping through nested 2D slices on every node comparison during matching.
  3. GetRules stored and returned Rules by value, so it copied the whole struct on every language lookup.

What Changed

  • Precomputed map[string]struct{} lookup sets for Flattened, Ignored, LabelIgnored, Keywords, Blocks, and Unordered at startup in compileSets(), turning linear slice checks into O(1) map lookups while keeping slice fallback for uncompiled rules.
  • Indexed EquivalentTypes into an equivGroups map (map[string][]int), so type equivalence lookups just check matching group IDs directly instead of scanning every group.
  • Added nil-safe helper methods on *Rules (IsIgnored, IsKeyword, IsLabelIgnored, IsUnordered, IsFlattened, IsBlock, Alias) to keep all rule lookups clean and encapsulated in buildASTWithRules.
  • Changed rulesCache to store and return *Rules pointers directly, stopping all the struct copying on GetRules calls.
  • Added an isomorphic fast path in engine.Match so identical subtrees pair up immediately when root node hashes match.
  • Added unit tests in internal/treesitter/rules_test.go covering compiled sets, uncompiled fallback, and nil receiver safety across all helper methods.

- treesitter: compile rules slices into lookup sets on init to replace O(N) slice scans with O(1) set checks
- treesitter: cache *Rules pointers in rulesCache to eliminate per-call struct copying and allocations in GetRules
- treesitter: add nil-safe helper query methods with fallback for uncompiled rules
- engine: fast-path matching when root ASTs have identical hashes and are isomorphic
- tests: add unit tests for Rules query helper methods across compiled, uncompiled, and nil receivers
@HarshK97
HarshK97 merged commit 4fb597e into main Aug 19, 2026
16 checks passed
@HarshK97
HarshK97 deleted the perf/rules-lookup-optimization branch August 19, 2026 08:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant