Feat/ts optimizer package

packages/ts-optimizer/DESIGN.md

@@ -0,0 +1,3 @@
+The idea is that we should hold operations performed in a virtual representation, then at the end we do a write with magic string. This allows us to hold all the AST information and only need to parse once.
+
+We should try to avoid cases where we have to reparse or do strange cases of looping over characters and words to make diffs. On the other hand, we should avoid using a printer approach with codegen that is exhaustive and leads to complexity and difficult changes.

packages/ts-optimizer/LICENSE

@@ -0,0 +1,22 @@
+MIT License
+
+Copyright (c) 2024 QwikDev
+Copyright (c) 2021 BuilderIO
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

packages/ts-optimizer/README.md

@@ -0,0 +1,170 @@
+# @qwik.dev/ts-optimizer
+
+A TypeScript implementation of the [Qwik](https://qwik.dev) optimizer.
+
+It takes a Qwik source file, finds every `$()` closure, lifts each one into its
+own lazy-loadable module, and rewrites the original file so the closures become
+`qrl(() => import(...))` references — the runtime then loads each chunk only
+when the user actually triggers it.
+
+> **Status: experimental.** This is a from-scratch TypeScript port of the Rust
+> SWC optimizer that ships with Qwik core, built on [OXC](https://oxc.rs). It
+> currently matches **~96%** of the reference optimizer's snapshot suite. APIs
+> may change before a 1.0.
+
+## Why
+
+The optimizer is the heart of Qwik's resumability model: every `$()` boundary
+becomes a separately loadable chunk, so the browser downloads code lazily as
+interaction demands it instead of hydrating the whole app up front. This package
+makes that transform available as a pure-TypeScript, [OXC](https://oxc.rs)-based
+library — no native SWC binding required for the transform logic itself (the
+parser/transformer it builds on, `oxc-parser` / `oxc-transform`, do ship native
+bindings).
+
+## Install
+
+```sh
+npm install @qwik.dev/ts-optimizer
+# or
+pnpm add @qwik.dev/ts-optimizer
+```
+
+**Requires Node `>=22`** — `oxc-parser`'s raw-transfer path throws on Node 20.
+ESM-only.
+
+## Usage
+
+There are two entry points: a synchronous core (`transformModule`) and an
+async, SWC-NAPI-compatible factory (`createOptimizer`) for drop-in use inside a
+bundler.
+
+### `transformModule` — synchronous core
+
+```ts
+import { transformModule, mkFilePath, mkSourceText } from '@qwik.dev/ts-optimizer';
+
+const result = transformModule({
+  input: [
+    {
+      path: mkFilePath('components/counter.tsx'),
+      code: mkSourceText(`
+        import { component$, useSignal } from '@qwik.dev/core';
+        export const Counter = component$(() => {
+          const count = useSignal(0);
+          return <button onClick$={() => count.value++}>{count.value}</button>;
+        });
+      `),
+    },
+  ],
+  srcDir: mkFilePath('/app/src/'),
+  entryStrategy: { type: 'segment' },
+  minify: 'simplify',
+});
+
+for (const mod of result.modules) {
+  console.log(mod.path, mod.kind); // 'parent' | 'segment'
+  // mod.code — emitted source
+  // mod.segment — segment metadata (name, hash, ctxKind, captures, …) on segments
+}
+
+console.log(result.diagnostics); // [] when clean
+```
+
+Each input file is transformed independently into **one parent module** (the
+original file rewritten so its `$()` calls become `qrl(...)` references) plus
+**zero or more segment modules** (each lifted closure body as its own
+lazy-loadable file).
+
+### `createOptimizer` — async, bundler-facing
+
+Mirrors the surface of Qwik's SWC optimizer (`@qwik.dev/optimizer`'s
+`createOptimizer().transformModules(...)`), so it can be swapped in by a bundler
+adapter. Takes and returns plain strings (no branded types at the boundary):
+
+```ts
+import { createOptimizer } from '@qwik.dev/ts-optimizer';
+
+const optimizer = createOptimizer();
+const output = await optimizer.transformModules({
+  srcDir: '/app/src',
+  input: [{ path: 'components/counter.tsx', code: source }],
+  entryStrategy: { type: 'smart' },
+});
+```
+
+## What it emits — before / after
+
+**Input** (`test.tsx`):
+
+```tsx
+import { $, component } from '@qwik.dev/core';
+
+export const renderHeader = $(() => {
+  return <div onClick={$((ctx) => console.log(ctx))} />;
+});
+```
+
+**Output** — a rewritten parent plus one segment per `$()` body:
+
+```ts
+// test.tsx  (parent — closures replaced by lazy QRL references)
+import { qrl } from '@qwik.dev/core';
+const q_renderHeader_jMxQsjbyDss = /*#__PURE__*/ qrl(
+  () => import('./test.tsx_renderHeader_jMxQsjbyDss'),
+  'renderHeader_jMxQsjbyDss'
+);
+export const renderHeader = q_renderHeader_jMxQsjbyDss;
+```
+
+```ts
+// test.tsx_renderHeader_jMxQsjbyDss.tsx  (segment — the lifted body)
+export const renderHeader_jMxQsjbyDss = () => {
+  return <div onClick={q_renderHeader_div_onClick_USi8k1jUb40} />;
+};
+```
+
+The nested `onClick` closure is lifted again into its own leaf segment, loaded
+only after the user clicks. That granularity — each `$()` boundary a separate
+chunk — is the whole point.
+
+## How it works
+
+`transformModule` runs each file through a fixed pipeline:
+
+| Phase                     | Does                                                                                                |
+| ------------------------- | --------------------------------------------------------------------------------------------------- |
+| **0 — Prepare**           | Repair recoverable parse errors, flatten destructures, parse once with OXC                          |
+| **1 — Extract**           | Walk the AST, find every `$(...)` / marker call, capture each closure body + naming context         |
+| **2 — Captures**          | Determine which outer-scope variables each closure closes over                                      |
+| **3 — Migrate**           | Decide where each module-level binding lives: stay in the parent, move into a segment, or re-export |
+| **4 — Rewrite parent**    | Replace each `$(closure)` with a generated `qrl(...)` reference; apply migration                    |
+| **5 — Generate segments** | Emit one lazy-loadable module per extracted closure                                                 |
+| **6 — Post-process**      | TypeScript strip, dead-code elimination, unused-import cleanup                                      |
+
+Symbol names are content-addressed: a closure's exported name is composed from
+its call-site context (`renderHeader_div_onClick`) plus an 11-character
+SipHash-1-3 suffix, so the same source always produces the same chunk names the
+runtime will fetch.
+
+The marker family the optimizer recognizes (`component$`, `useTask$`,
+`useStyles$`, `useVisibleTask$`, `server$`, event handlers like `onClick$`, …)
+is detected **structurally** — any call whose callee's imported name ends in `$`
+extracts — so library-defined `name$` functions work automatically.
+
+## Public API
+
+| Export                                                             | Kind      | Purpose                                         |
+| ------------------------------------------------------------------ | --------- | ----------------------------------------------- |
+| `transformModule(options)`                                         | function  | Synchronous core transform → `TransformOutput`  |
+| `createOptimizer()`                                                | function  | Async, SWC-NAPI-compatible optimizer instance   |
+| `mk*` (e.g. `mkFilePath`, `mkSourceText`)                          | functions | Smart constructors for the branded string types |
+| `TransformModulesOptions`, `TransformOutput`, `TransformModule`, … | types     | Input/output contracts                          |
+| `SymbolName`, `Hash`, `FilePath`, `SourceText`, …                  | types     | Branded identifier types                        |
+
+All other modules are internal and not part of the public contract.
+
+## License
+
+[MIT](./LICENSE) — matching [Qwik](https://github.com/QwikDev/qwik) core, of
+which this is a derivative. Copyright © QwikDev and BuilderIO.

packages/ts-optimizer/match-these-snaps/qwik_core__test__component_level_self_referential_qrl.snap

@@ -0,0 +1,178 @@
+---
+source: packages/optimizer/core/src/test.rs
+assertion_line: 5854
+expression: output
+---
+==INPUT==
+
+
+import { component$, useAsync$ } from '@qwik.dev/core';
+
+// Component-level self-referential component
+export const Foo = component$((props) => {
+	const sig = useAsync$(async ({cleanup}) => {
+		const timer = setInterval(() => {
+			sig.value++;
+		}, 1000);
+		cleanup(() => clearInterval(timer));
+		return 0;
+	});
+	const other = useAsync$(async ({cleanup}) => {
+		const timer = setInterval(() => {
+			other.value++;
+		}, 900);
+		cleanup(() => clearInterval(timer));
+		return 0;
+	});
+	return (
+		<div>
+			{other.value}
+		</div>
+	);
+});
+
+============================= test.tsx_Foo_component_HTDRsvUbLiE.tsx (ENTRY POINT)==
+
+import { qrl } from "@qwik.dev/core";
+import { useAsyncQrl } from "@qwik.dev/core";
+//
+const q_Foo_component_other_useAsync_fsHooibmyyE = /*#__PURE__*/ qrl(()=>import("./test.tsx_Foo_component_other_useAsync_fsHooibmyyE"), "Foo_component_other_useAsync_fsHooibmyyE");
+const q_Foo_component_sig_useAsync_f0BGwWm4eeY = /*#__PURE__*/ qrl(()=>import("./test.tsx_Foo_component_sig_useAsync_f0BGwWm4eeY"), "Foo_component_sig_useAsync_f0BGwWm4eeY");
+//
+export const Foo_component_HTDRsvUbLiE = (props)=>{
+    const _ref = {};
+    _ref.sig = useAsyncQrl(q_Foo_component_sig_useAsync_f0BGwWm4eeY.w([
+        _ref.sig
+    ]));
+    const { sig } = _ref;
+    _ref.other = useAsyncQrl(q_Foo_component_other_useAsync_fsHooibmyyE.w([
+        _ref.other
+    ]));
+    const { other } = _ref;
+    return <div>
+			{other.value}
+		</div>;
+};
+
+
+Some("{\"version\":3,\"sources\":[\"/user/qwik/src/test.tsx\"],\"names\":[],\"mappings\":\";;;;;;yCAI8B,CAAC;IAC9B;eAAY;;;IAAZ,QAAM;iBAOQ;;;IAAd,QAAM;IAON,QACE,IAAI;GACJ,CAAC,MAAM,KAAK,CAAC;EACd,EAAE;AAEJ\"}")
+/*
+{
+  "origin": "test.tsx",
+  "name": "Foo_component_HTDRsvUbLiE",
+  "entry": null,
+  "displayName": "test.tsx_Foo_component",
+  "hash": "HTDRsvUbLiE",
+  "canonicalFilename": "test.tsx_Foo_component_HTDRsvUbLiE",
+  "path": "",
+  "extension": "tsx",
+  "parent": null,
+  "ctxKind": "function",
+  "ctxName": "component$",
+  "captures": false,
+  "loc": [
+    137,
+    534
+  ],
+  "paramNames": [
+    "props"
+  ]
+}
+*/
+============================= test.tsx_Foo_component_other_useAsync_fsHooibmyyE.tsx (ENTRY POINT)==
+
+import { _captures } from "@qwik.dev/core";
+//
+export const Foo_component_other_useAsync_fsHooibmyyE = async (_rawProps)=>{
+    const other = _captures[0];
+    const timer = setInterval(()=>{
+        other.value++;
+    }, 900);
+    _rawProps.cleanup(()=>clearInterval(timer));
+    return 0;
+};
+
+
+Some("{\"version\":3,\"sources\":[\"/user/qwik/src/test.tsx\"],\"names\":[],\"mappings\":\";;wDAYyB;;IACvB,MAAM,QAAQ,YAAY;QACzB,MAAM,KAAK;IACZ,GAAG;IACH,UAJ+B,QAIvB,IAAM,cAAc;IAC5B,OAAO\"}")
+/*
+{
+  "origin": "test.tsx",
+  "name": "Foo_component_other_useAsync_fsHooibmyyE",
+  "entry": null,
+  "displayName": "test.tsx_Foo_component_other_useAsync",
+  "hash": "fsHooibmyyE",
+  "canonicalFilename": "test.tsx_Foo_component_other_useAsync_fsHooibmyyE",
+  "path": "",
+  "extension": "tsx",
+  "parent": "Foo_component_HTDRsvUbLiE",
+  "ctxKind": "function",
+  "ctxName": "useAsync$",
+  "captures": true,
+  "loc": [
+    341,
+    482
+  ],
+  "paramNames": [
+    "_rawProps"
+  ],
+  "captureNames": [
+    "other"
+  ]
+}
+*/
+============================= test.tsx ==
+
+import { componentQrl } from "@qwik.dev/core";
+import { qrl } from "@qwik.dev/core";
+//
+const q_Foo_component_HTDRsvUbLiE = /*#__PURE__*/ qrl(()=>import("./test.tsx_Foo_component_HTDRsvUbLiE"), "Foo_component_HTDRsvUbLiE");
+// Component-level self-referential component
+//
+export const Foo = /*#__PURE__*/ componentQrl(q_Foo_component_HTDRsvUbLiE);
+
+
+Some("{\"version\":3,\"sources\":[\"/user/qwik/src/test.tsx\"],\"names\":[],\"mappings\":\";;;;AAGA,6CAA6C;;AAC7C,OAAO,MAAM,oBAAM,0CAoBhB\"}")
+============================= test.tsx_Foo_component_sig_useAsync_f0BGwWm4eeY.tsx (ENTRY POINT)==
+
+import { _captures } from "@qwik.dev/core";
+//
+export const Foo_component_sig_useAsync_f0BGwWm4eeY = async (_rawProps)=>{
+    const sig = _captures[0];
+    const timer = setInterval(()=>{
+        sig.value++;
+    }, 1000);
+    _rawProps.cleanup(()=>clearInterval(timer));
+    return 0;
+};
+
+
+Some("{\"version\":3,\"sources\":[\"/user/qwik/src/test.tsx\"],\"names\":[],\"mappings\":\";;sDAKuB;;IACrB,MAAM,QAAQ,YAAY;QACzB,IAAI,KAAK;IACV,GAAG;IACH,UAJ6B,QAIrB,IAAM,cAAc;IAC5B,OAAO\"}")
+/*
+{
+  "origin": "test.tsx",
+  "name": "Foo_component_sig_useAsync_f0BGwWm4eeY",
+  "entry": null,
+  "displayName": "test.tsx_Foo_component_sig_useAsync",
+  "hash": "f0BGwWm4eeY",
+  "canonicalFilename": "test.tsx_Foo_component_sig_useAsync_f0BGwWm4eeY",
+  "path": "",
+  "extension": "tsx",
+  "parent": "Foo_component_HTDRsvUbLiE",
+  "ctxKind": "function",
+  "ctxName": "useAsync$",
+  "captures": true,
+  "loc": [
+    173,
+    313
+  ],
+  "paramNames": [
+    "_rawProps"
+  ],
+  "captureNames": [
+    "sig"
+  ]
+}
+*/
+== DIAGNOSTICS ==
+
+[]