|
113 | 113 | * could not read the tree" are different verdicts and a reader should not have |
114 | 114 | * to guess which one they got. Both are non-zero, so CI fails either way. |
115 | 115 | * |
| 116 | + * ## A FOURTH door, for a different question: text THIS PROCESS synthesised |
| 117 | + * |
| 118 | + * The three doors above all answer "could I read this tree?", and a refusal is |
| 119 | + * right for that question because the tree belongs to the gate's own author. |
| 120 | + * `packages/lint/src/checked-parse.ts` records the other side of the same axis |
| 121 | + * in as many words: it answers the same question by REPORTING rather than |
| 122 | + * refusing, "because a `scripts/**` gate audits a tree its own author controls" |
| 123 | + * while "a publish-time validator is handed metadata by someone else". |
| 124 | + * |
| 125 | + * A third position was never written down, and a gate in this tree was standing |
| 126 | + * on it. `tenant-audit-census.mjs` reads a receiver's declared type out of a |
| 127 | + * source that PARSED, stores that type's text whitespace-collapsed, and later |
| 128 | + * re-parses the stored text as a synthetic type alias |
| 129 | + * (`type CensusReceiver = <the stored text>;`) to ask whether it declares a |
| 130 | + * write door. When the collapse loses a member separator -- a type literal may |
| 131 | + * separate its members by a newline alone, which is legal TypeScript and |
| 132 | + * collapses to nothing -- the synthetic alias does not parse, and the census |
| 133 | + * takes `EXIT_UNPARSEABLE` for the whole run. |
| 134 | + * |
| 135 | + * Measured on 2026-09-18 against TypeScript 6.0.3: |
| 136 | + * |
| 137 | + * type text alias parses? |
| 138 | + * ---------------------------------------------------- ------------- |
| 139 | + * { insert(o: string): Promise<void> yes (0 diags) |
| 140 | + * find(o: string): Promise<void> } <- authored |
| 141 | + * { insert(o: string): Promise<void> NO (1 diag, |
| 142 | + * find(o: string): Promise<void> } <- collapsed "';' expected") |
| 143 | + * { insert(o: string): Promise<void>; yes (0 diags) |
| 144 | + * find(o: string): Promise<void>; } <- lit control |
| 145 | + * |
| 146 | + * The refusal's own text is what makes the misfit legible: it names |
| 147 | + * `census-receiver-type.ts`, a file that does not exist in the tree, and its |
| 148 | + * reasoning -- "a file the gate could not read, reported as a file with nothing |
| 149 | + * to report" -- is FALSE here. The gate read the file. What did not round-trip |
| 150 | + * is the gate's own re-serialisation of a fragment of it, and that is a fact |
| 151 | + * about the synthesiser, never about the corpus. Ending the process on it turns |
| 152 | + * one site's unanswerable question into no answer for any site. |
| 153 | + * |
| 154 | + * So {@link parseDerivedText} answers "did the text I synthesised parse?" and |
| 155 | + * hands the verdict BACK, and the floor is held by making that door unreachable |
| 156 | + * for the question the refusal exists for: it takes an `origin` -- a |
| 157 | + * `ts.SourceFile` this module has already certified -- and the only way to hold |
| 158 | + * one is {@link parseSourceFile} returning, or a Program {@link |
| 159 | + * createProgramChecked} vouched for. Both of those EXIT on a source that does |
| 160 | + * not parse. ⇒ a source the gate could not read cannot reach the returnable |
| 161 | + * door: it hits the refusal first, by construction rather than by review. |
| 162 | + * Handing an uncertified origin is itself a refusal ({@link |
| 163 | + * EXIT_DERIVED_MISUSE}), for the reason the section above gives: a throw there |
| 164 | + * would be one `catch` away from the silent skip. |
| 165 | + * |
| 166 | + * Two deliberate divergences from the `packages/lint` sibling, both in the |
| 167 | + * strict direction, because this is `scripts/**`: |
| 168 | + * |
| 169 | + * • it returns NO tree on failure (`sourceFile: null`), where the sibling |
| 170 | + * always returns the recovered one. The sibling has callers that already |
| 171 | + * report findings off a partial tree; here there are none, and a recovered |
| 172 | + * tree is the thing a caller walks before scoring it clean. A caller that |
| 173 | + * forgets to branch gets a TypeError, which is loud and non-zero. |
| 174 | + * • the failure is not swallowable into a PASS. It is data, so a caller can |
| 175 | + * attribute it to one site -- but there is no verdict in it that reads as |
| 176 | + * "nothing to report", and `parseCensus()` counts it as a `rejection`, |
| 177 | + * never as a `refusal`: the run continued, and a census that conflated the |
| 178 | + * two would be lying about its own numerator. |
| 179 | + * |
116 | 180 | * ## The knobs that are NOT knobs |
117 | 181 | * |
118 | 182 | * `ScriptTarget.Latest` and `setParentNodes: true` are fixed here because all |
@@ -269,26 +333,54 @@ function batteryFloorFailures() { |
269 | 333 | */ |
270 | 334 | export const EXIT_UNPARSEABLE = 3; |
271 | 335 |
|
272 | | -/** Parses attempted, the distinct file names, and the refusals. */ |
273 | | -const census = { parses: 0, programs: 0, transpiles: 0, files: new Set(), refusals: 0 }; |
| 336 | +/** Parses attempted, the distinct file names, the refusals and the rejections. */ |
| 337 | +const census = { |
| 338 | + parses: 0, programs: 0, transpiles: 0, derived: 0, files: new Set(), refusals: 0, rejections: 0, |
| 339 | +}; |
274 | 340 |
|
275 | 341 | /** |
276 | 342 | * A snapshot of what this module has been asked to parse in this process. |
277 | 343 | * |
278 | | - * `parses` counts every source that reached the parser through ANY of the three |
| 344 | + * `parses` counts every source that reached the parser through ANY of the four |
279 | 345 | * entry points, so it stays the numerator of "how much of this run was read"; |
280 | | - * `programs` and `transpiles` say which door they came through. |
| 346 | + * `programs`, `transpiles` and `derived` say which door they came through. |
| 347 | + * |
| 348 | + * `refusals` and `rejections` are deliberately separate totals rather than one |
| 349 | + * "failures" number. A refusal ENDED THE RUN, so every source after it went |
| 350 | + * unread and the numerator above is short by an unknown amount; a rejection is |
| 351 | + * a verdict {@link parseDerivedText} handed back about text this process |
| 352 | + * synthesised, with the run still going and every later source still read. A |
| 353 | + * single total would make those two indistinguishable in the one report whose |
| 354 | + * job is to say how much of the run was actually measured. |
281 | 355 | */ |
282 | 356 | export function parseCensus() { |
283 | 357 | return { |
284 | 358 | parses: census.parses, |
285 | 359 | programs: census.programs, |
286 | 360 | transpiles: census.transpiles, |
| 361 | + derived: census.derived, |
287 | 362 | files: census.files.size, |
288 | 363 | refusals: census.refusals, |
| 364 | + rejections: census.rejections, |
289 | 365 | }; |
290 | 366 | } |
291 | 367 |
|
| 368 | +/** |
| 369 | + * The `ts.SourceFile`s this module has CERTIFIED as parseable. |
| 370 | + * |
| 371 | + * Membership is what {@link parseDerivedText} requires of its `origin`, and it |
| 372 | + * is the whole floor argument for that door: a source the parser could not read |
| 373 | + * never becomes a member, because the two doors that add members |
| 374 | + * ({@link parseSourceFile}, {@link createProgramChecked}) end the process |
| 375 | + * instead of returning. So the one door in this module whose failure is |
| 376 | + * RETURNABLE is unreachable for the input the refusal exists for. |
| 377 | + * |
| 378 | + * A `WeakSet` so a long scan does not retain every tree it has read, and so |
| 379 | + * `has()` answers `false` for a non-object rather than throwing -- an |
| 380 | + * uncertified origin must reach the misuse refusal, not a TypeError. |
| 381 | + */ |
| 382 | +const vouchedSources = new WeakSet(); |
| 383 | + |
292 | 384 | let censusReportArmed = false; |
293 | 385 |
|
294 | 386 | /** |
@@ -323,7 +415,8 @@ function armCensusReport() { |
323 | 415 | const c = parseCensus(); |
324 | 416 | process.stderr.write( |
325 | 417 | `[ts-parse census] ${c.parses} parse(s) over ${c.files} distinct file name(s) ` |
326 | | - + `(${c.programs} program(s), ${c.transpiles} transpile(s)); ${c.refusals} refusal(s)\n`, |
| 418 | + + `(${c.programs} program(s), ${c.transpiles} transpile(s), ${c.derived} derived); ` |
| 419 | + + `${c.refusals} refusal(s), ${c.rejections} rejection(s)\n`, |
327 | 420 | ); |
328 | 421 | }); |
329 | 422 | } |
@@ -484,6 +577,7 @@ export function parseSourceFile(fileName, text, scriptKind) { |
484 | 577 | process.stderr.write(refusalReport(fileName, scriptKind, diagnostics)); |
485 | 578 | process.exit(EXIT_UNPARSEABLE); |
486 | 579 | } |
| 580 | + vouchedSources.add(sourceFile); |
487 | 581 | return sourceFile; |
488 | 582 | } |
489 | 583 |
|
@@ -574,6 +668,13 @@ export function createProgramChecked(rootNames, options, host) { |
574 | 668 | process.stderr.write(programRefusalReport(roots, rows)); |
575 | 669 | process.exit(EXIT_UNPARSEABLE); |
576 | 670 | } |
| 671 | + // Every file this Program pulled in has just been through the syntactic |
| 672 | + // check above -- transitively, which is the point of checking the whole |
| 673 | + // Program rather than its roots -- so each one is certified for |
| 674 | + // {@link parseDerivedText}. A gate that reads its corpus through a Program |
| 675 | + // can therefore synthesise from it on the same terms as one that parses file |
| 676 | + // by file. |
| 677 | + for (const sf of program.getSourceFiles()) vouchedSources.add(sf); |
577 | 678 | return program; |
578 | 679 | } |
579 | 680 |
|
@@ -619,6 +720,154 @@ export function transpileChecked(fileName, text, transpileOptions = {}) { |
619 | 720 | return result; |
620 | 721 | } |
621 | 722 |
|
| 723 | +/** |
| 724 | + * The exit status of a MISUSE of {@link parseDerivedText} -- an origin this |
| 725 | + * module never certified. |
| 726 | + * |
| 727 | + * Distinct from {@link EXIT_UNPARSEABLE} for the reason that code is distinct |
| 728 | + * from 1: "this gate found violations", "I could not read the tree" and "this |
| 729 | + * call site is asking the wrong door" are three verdicts, and a reader should |
| 730 | + * not have to guess which one they got. All are non-zero, so CI fails either |
| 731 | + * way. |
| 732 | + */ |
| 733 | +export const EXIT_DERIVED_MISUSE = 4; |
| 734 | + |
| 735 | +/** |
| 736 | + * The refusal text for a derived parse whose origin was never certified. |
| 737 | + * |
| 738 | + * Separate from the exit so a self-test case can read it, exactly as |
| 739 | + * {@link refusalReport} is. |
| 740 | + */ |
| 741 | +export function derivedMisuseReport(fileName, origin) { |
| 742 | + const what = origin === null ? 'null' |
| 743 | + : origin === undefined ? 'undefined' |
| 744 | + : typeof origin === 'object' ? `an object with fileName ${JSON.stringify(origin.fileName ?? '(none)')}` |
| 745 | + : `a ${typeof origin}`; |
| 746 | + return [ |
| 747 | + `x ts-parse — REFUSING a derived parse whose ORIGIN this module never certified.`, |
| 748 | + ``, |
| 749 | + ` derived text ${fileName}`, |
| 750 | + ` origin ${what}`, |
| 751 | + ``, |
| 752 | + ` parseDerivedText answers a different question from parseSourceFile:`, |
| 753 | + ` "did the text I synthesised parse?", not "could I read this tree?". Its`, |
| 754 | + ` verdict is returnable ONLY because the tree behind it has already been`, |
| 755 | + ` read, so the origin must be a ts.SourceFile this module returned from`, |
| 756 | + ` parseSourceFile, or one a Program from createProgramChecked was built`, |
| 757 | + ` over. Both of those end the process on a source that does not parse.`, |
| 758 | + ``, |
| 759 | + ` Reading a source off disk and handing its text here would route the one`, |
| 760 | + ` door whose failure is NOT a refusal at exactly the input the refusal`, |
| 761 | + ` exists for — a file the gate could not read, scored as a file with`, |
| 762 | + ` nothing to report. Parse it with parseSourceFile, and pass the tree`, |
| 763 | + ` that call returns as the origin of anything you synthesise from it.`, |
| 764 | + ``, |
| 765 | + ` This is an exit rather than a throw for the reason the refusals are:`, |
| 766 | + ` a throw here is one \`catch\` away from the silent skip.`, |
| 767 | + ``, |
| 768 | + ].join('\n'); |
| 769 | +} |
| 770 | + |
| 771 | +/** |
| 772 | + * The verdict text for derived text that did not parse. NOT a refusal: it names |
| 773 | + * the origin, says the run continues, and is meant to be printed by the caller |
| 774 | + * against the site the text came from. |
| 775 | + */ |
| 776 | +export function derivedFailureReport(fileName, originName, scriptKind, rows) { |
| 777 | + return [ |
| 778 | + `! ts-parse — derived text does not parse. The source it came FROM does.`, |
| 779 | + ``, |
| 780 | + ` derived text ${fileName}`, |
| 781 | + ` derived from ${originName}`, |
| 782 | + ` parsed as ${describeScriptKind(scriptKind)}`, |
| 783 | + ` errors ${rows.length} parse diagnostic(s) from TypeScript ${ts.version}`, |
| 784 | + ``, |
| 785 | + ...locationLines(fileName, rows), |
| 786 | + ``, |
| 787 | + ` This is a fact about the text THIS PROCESS SYNTHESISED, not about the`, |
| 788 | + ` source above: that source was read, and certified, before this text was`, |
| 789 | + ` built from it. So the run is NOT aborted — the verdict is returned, and`, |
| 790 | + ` the caller attributes it to the one site it belongs to instead of`, |
| 791 | + ` losing every other site to a process exit.`, |
| 792 | + ``, |
| 793 | + ` ⛔ It is still not a pass. No tree comes back with this (sourceFile is`, |
| 794 | + ` null), so there is no recovered wreckage to walk and no reading of this`, |
| 795 | + ` result that says "nothing to report". If your synthesis is supposed to`, |
| 796 | + ` round-trip, this is the bug in the synthesis.`, |
| 797 | + ``, |
| 798 | + ].join('\n'); |
| 799 | +} |
| 800 | + |
| 801 | +/** |
| 802 | + * Parse text THIS PROCESS SYNTHESISED from a source it has already read, and |
| 803 | + * hand the verdict back instead of ending the run. |
| 804 | + * |
| 805 | + * ⚠️ ⛔ NOT a general escape from the refusal, and ⛔ not for a source read off |
| 806 | + * disk: see the `origin` parameter. The header section "A FOURTH door" carries |
| 807 | + * the whole argument, the measurement behind it, and why this is the only door |
| 808 | + * here whose failure is returnable. |
| 809 | + * |
| 810 | + * @param {ts.SourceFile} origin The tree `text` was derived from, as returned |
| 811 | + * by {@link parseSourceFile} or pulled from a {@link createProgramChecked} |
| 812 | + * Program. Anything else ends the process with {@link EXIT_DERIVED_MISUSE}. |
| 813 | + * @param {string} fileName What to call the synthesised text. Its extension |
| 814 | + * picks the ScriptKind when `scriptKind` is omitted, exactly as for |
| 815 | + * {@link parseSourceFile}; a synthesised source has no real path, so give it |
| 816 | + * a `.ts`/`.tsx` name that says what it is. |
| 817 | + * @param {string} text The synthesised source. |
| 818 | + * @param {ts.ScriptKind} [scriptKind] Omit to let `fileName` decide. |
| 819 | + * @returns {{ sourceFile: ts.SourceFile|null, failure: null|{ message: string, |
| 820 | + * line: number, column: number, count: number, |
| 821 | + * rows: { line: number, column: number, message: string }[], report: string } }} |
| 822 | + * `failure` null ⇒ it parsed and `sourceFile` is the tree. Otherwise |
| 823 | + * `sourceFile` is null — there is deliberately no recovered tree to walk — |
| 824 | + * and `failure.report` is the text to print against the site. |
| 825 | + */ |
| 826 | +export function parseDerivedText(origin, fileName, text, scriptKind) { |
| 827 | + armCensusReport(); |
| 828 | + if (!vouchedSources.has(origin)) { |
| 829 | + process.stderr.write(derivedMisuseReport(fileName, origin)); |
| 830 | + process.exit(EXIT_DERIVED_MISUSE); |
| 831 | + } |
| 832 | + census.parses += 1; |
| 833 | + census.derived += 1; |
| 834 | + census.files.add(fileName); |
| 835 | + |
| 836 | + // The same fixed knobs the three doors above pass, for the reason the header |
| 837 | + // gives: a call site that needs a different pair says so once, here. Pinned |
| 838 | + // behaviourally by the self-test rather than by this comment -- a derived |
| 839 | + // tree has its parents set and reads modern syntax, or the knobs drifted. |
| 840 | + const sourceFile = ts.createSourceFile( |
| 841 | + fileName, |
| 842 | + text, |
| 843 | + ts.ScriptTarget.Latest, |
| 844 | + /* setParentNodes */ true, |
| 845 | + scriptKind, |
| 846 | + ); |
| 847 | + |
| 848 | + const rows = describeDiagnostics(sourceFile); |
| 849 | + if (rows.length === 0) { |
| 850 | + // Derived text that parsed is itself a tree this module has read, so a |
| 851 | + // second-order synthesis (a fragment of a fragment) can name it as origin. |
| 852 | + vouchedSources.add(sourceFile); |
| 853 | + return { sourceFile, failure: null }; |
| 854 | + } |
| 855 | + |
| 856 | + census.rejections += 1; |
| 857 | + const first = rows[0]; |
| 858 | + return { |
| 859 | + sourceFile: null, |
| 860 | + failure: { |
| 861 | + message: first.message, |
| 862 | + line: first.line, |
| 863 | + column: first.column, |
| 864 | + count: rows.length, |
| 865 | + rows, |
| 866 | + report: derivedFailureReport(fileName, origin.fileName, scriptKind, rows), |
| 867 | + }, |
| 868 | + }; |
| 869 | +} |
| 870 | + |
622 | 871 | // --------------------------------------------------------------------------- |
623 | 872 | // Self-test -- real child processes, because the refusal IS a process exit |
624 | 873 | // --------------------------------------------------------------------------- |
|
0 commit comments