Intern token/node type strings and short token values

[joelhawksley]

What

Intern the strings that the C extension attaches to every Herb::Token and AST
node, so identical bytes are shared instead of re-allocated on every parse/lex.

Two categories are interned into Ruby's global fstring table via
rb_enc_interned_str:

• Token & node type names ("TOKEN_NEWLINE", "AST_HTML_TEXT_NODE", …).
  These come from fixed enums, so previously every token/node re-allocated an
  identical String.
• Short token values ("\n", "<", ">", "%>", "=", quotes, tag and
  attribute names). The overwhelming majority of token values are short,
  structural literals drawn from a tiny vocabulary that repeats across most
  tokens.

Long token values (ERB code, prose text runs) are effectively unique, so they
keep allocating a fresh, mutable String as before — a length threshold
(HERB_MAX_INTERNED_TOKEN_VALUE_LENGTH = 16) draws the line and keeps the
fstring table bounded.

Why

Building the Ruby token/AST objects allocated one fresh String per node type,
one per token type, and one per token value — even though ~96% of those bytes
are duplicates from tiny vocabularies. Interning collapses them onto shared
frozen instances.

Frozen-value safety

Interned strings are frozen, so the two in-repo consumers that mutated a token
value in place are made copy-on-write:

• Engine::Compiler whitespace-trim helpers take a mutable copy before trimming
  (text = +@tokens.last[1] / +token[1] — a no-op unless the value is frozen,
  so it only allocates on the rare trim path when the value is shared).
• Token#tree_inspect coerces a copy for display instead of calling
  force_encoding on the value in place.

The full test suite passes (2386 runs, 5462 assertions, 0 failures, 0 errors).

Benchmarks

Compiling a large real-world .erb view corpus (~6,000 templates, ~10 MB, from
the github.com monolith) with Herb::Engine, on Ruby 4.0.5, measuring
GC.stat(:total_allocated_objects):

path	metric	before	after	Δ
lex	T_STRING allocations	103,332	6,645	−93.6%
parse	T_STRING allocations	98,097	21,756	−77.8%
parse	total objects	9.84M	7.75M	−21.2%
lex	total objects	11.35M	8.24M	−27.4%
Herb::Engine compile	total objects	20.99M	18.78M	−10.5%

(T_STRING figures measured over a 600-template subset with GC disabled;
totals over the full corpus.)

Measured end-to-end through the actual Rails view precompiler (github.com's
ViewPrecompiler.precompile, which compiles .html.erb via Herb::Engine
behind reactionview), compiling 4,056 templates on Ruby 4.0.5:

metric	before	after (with the small match? change too)	Δ
objects allocated	21,154,648	19,201,428	−9.2%
wall time	~13.3s	~13.8s	roughly neutral

The allocation reduction is deterministic. Because the type names are cached
by enum value (an O(1) array for tokens, a pinned per-builder static for nodes),
the hot path is both allocation-free and fstring-probe-free, so the win comes at
roughly neutral wall time — an isolated Herb::Engine-over-the-corpus loop
actually times a hair faster than stock (~6.24s vs ~6.28s). Only the variable
token values still hit the fstring table, and only when short.

Notes for reviewers

• The node-type change is in templates/ext/herb/nodes.c.erb; the generated
  ext/herb/nodes.c is a build artifact.
• Type names are cached by enum (probe-free after first use); rb_enc_interned_str
  is used directly only for the variable, short token values.