✨ feat(clean): transform trees before serialization

[gaborbernat]

Callers converting HTML to XML in #791 need collapsed whitespace before traversing the tree. Minifying during serialization leaves the stored text unchanged, so they must serialize and parse again before reading normalized XML fragments. ✨ Allow cleanup on the existing tree and configurable child output without a wrapper, removing that extra round trip from this workflow.

Compose cleanup with clean.transform_node(node, *steps) using ordinary synchronous callables. Applications can mix functions, bound methods, and callable objects without registration or a transformer base class. Each step receives the current root once and owns any descendant traversal it needs. Returning a Node selects the next root; returning None retains it, supporting existing mutators. Invalid results raise TypeError with the stage number, and a pipeline with no steps returns its input.

Make ownership explicit when combining stages. collapse_whitespace_node, strip_comments_node, and the existing linkify_node mutate and return the same root; sanitize_node returns a copy. Callers must retain the composed result when a step can replace the root. Exceptions stop execution and preserve earlier mutations. Native operations lock their own tree, but composition holds no lock across callbacks and provides no transaction across stages. Applications must coordinate shared-tree access and choose stage order with care: a custom edit after sanitization can introduce content outside the sanitizer policy.

Whitespace cleanup collapses ASCII whitespace runs to one space, including leading and trailing runs, while retaining NBSP and other Unicode spaces. Preserve preformatted and raw-text contexts, check ancestors for subtree calls, and skip foreign subtrees. Visit template content without entering attached shadow trees. Reject XML-mode trees because their consumers may assign significance to whitespace; CSS layout rules remain outside this HTML policy. Comment removal supports HTML and XML, retains the context root, and leaves references to removed comments valid as detached nodes. Removing comments before collapsing whitespace can join runs separated by a comment.

Add keyword-only inner=True to Node.serialize, Node.encode, and Node.serialize_iter. Retain the selected node as serialization context while omitting its wrapper, so raw-text and preserved-whitespace rules survive child output. Existing calls keep outer serialization. Compact and indented output support streaming; minified output uses serialize or encode. A leaf root produces empty inner output. Callers needing well-formed XML fragments should continue to use inner_xml.

Keep dispatch and tree mutation in C, with Python providing exports and typing. Traverse without creating a Python wrapper for each visited node, preserve text-node identity, and retain borrowed source storage when text needs no change. Changed text uses replacement storage and preserves mutation-observer old values. Indented output reserves each indentation run once and fills it by doubling the copied prefix, avoiding a capacity check and small copy at each depth. Tree arenas retain allocations until the caller releases the tree, so cleanup does not promise to reduce the resident memory of a long-lived tree.

Document the workflow across tutorials, how-to recipes, reference contracts, and the ownership explanation, with migration guidance for 21 libraries. Extend the shared benchmark suite and CodSpeed registrations for cleanup, composition, and child output. The performance tables include competitor adapters and the html5lib filter/reparse workflow, with parsing and cyclic-garbage collection outside mutation timings. The dispatcher table includes a Python implementation of the same Node/None contract alongside the unchecked callable loop. They distinguish Python traversal adapters from native operations, record differing preservation and pretty-print policies, and label parse5/jsdom subprocess measurements as Python integration costs. Retain existing migration measurements and commit only JSON feeds consumed by documentation.

[codspeed-hq]

Merging this PR will not alter performance

✅ 132 untouched benchmarks
🆕 22 new benchmarks
⏩ 32 skipped benchmarks¹

Performance Changes

	Benchmark	BASE	HEAD	Efficiency
🆕	test_feature[collapse-whitespace]	N/A	2.6 ms	N/A
🆕	test_feature[serialize-inner]	N/A	5.5 ms	N/A
🆕	test_feature[strip-comments]	N/A	694.9 µs	N/A
🆕	test_feature[transform-tree]	N/A	2.8 ms	N/A
🆕	test_feature[collapse-whitespace-dense]	N/A	16.1 ms	N/A
🆕	test_feature[collapse-whitespace-unchanged]	N/A	3.7 ms	N/A
🆕	test_feature[encode-inner-indent]	N/A	9.2 ms	N/A
🆕	test_feature[encode-inner-minify]	N/A	8.8 ms	N/A
🆕	test_feature[encode-inner]	N/A	6.7 ms	N/A
🆕	test_feature[iterate-inner-indent]	N/A	7 ms	N/A
🆕	test_feature[iterate-inner]	N/A	5.1 ms	N/A
🆕	test_feature[serialize-inner-indent]	N/A	7.4 ms	N/A
🆕	test_feature[serialize-inner-minify]	N/A	7.8 ms	N/A
🆕	test_feature[transform-dispatch-1]	N/A	21.3 µs	N/A
🆕	test_feature[transform-dispatch-16]	N/A	28.2 µs	N/A
🆕	test_feature[transform-dispatch-4]	N/A	22.7 µs	N/A
🆕	test_feature[transform-dispatch]	N/A	21 µs	N/A
🆕	test_feature[linkify-node-fresh-spec]	N/A	9.1 ms	N/A
🆕	test_feature[parse-inner-encode]	N/A	17.6 ms	N/A
🆕	test_feature[parse-inner]	N/A	16.4 ms	N/A
...	...	...	...	...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

---

_{Comparing gaborbernat:feat/tree-transforms (d740508) with main (6bd00f3)}

Footnotes

1. 32 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩