Skip to content

Language Service: Implement Semantic Token Provider - #2227

Draft
marcoroth wants to merge 1 commit into
mainfrom
semantic-tokens
Draft

Language Service: Implement Semantic Token Provider#2227
marcoroth wants to merge 1 commit into
mainfrom
semantic-tokens

Conversation

@marcoroth

Copy link
Copy Markdown
Owner

Opening as WIP.

This pull request implements textDocument/semanticTokens, so an editor colors HTML+ERB from the parsed template rather than from a TextMate/Tree Sitter grammar guessing at it with regexes.

CleanShot 2026-08-14 at 02 35 22@2x

A grammar has to decide what <div class="a <%= b %>"> is by looking at characters. Herb already knows, so the tokens it hands back stay right where nesting gets awkward, and they carry things no grammar could work out at all, like whether a helper came from Action View or from the application.

What it claims, and what it leaves alone

Token Type
Tag names type
Attribute names property
Attribute values string quoted and unquoted
< > </ /> <% %> macro output modifier on <%=
<!-- --> and <%# %> comment
<!DOCTYPE> keyword
Ruby keywords keyword
Action View helpers function defaultLibrary modifier
Names in a locals: declaration parameter

The Ruby inside a tag is deliberately almost untouched. A Ruby language server describes it far better.

@github-actions github-actions Bot added documentation Improvements or additions to documentation javascript JavaScript source across the javascript/ packages vscode VS Code editor support language-server @herb-tools/language-server and Language Server Protocol support typescript TypeScript source across the javascript/ packages highlighter @herb-tools/highlighter syntax highlighting and diagnostic rendering playground The Herb playground web app core @herb-tools/core shared AST nodes, interfaces, and utilities language-service @herb-tools/language-service HTML+ERB language service labels Aug 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core @herb-tools/core shared AST nodes, interfaces, and utilities documentation Improvements or additions to documentation highlighter @herb-tools/highlighter syntax highlighting and diagnostic rendering javascript JavaScript source across the javascript/ packages language-server @herb-tools/language-server and Language Server Protocol support language-service @herb-tools/language-service HTML+ERB language service playground The Herb playground web app typescript TypeScript source across the javascript/ packages vscode VS Code editor support

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant