Skip to content

Commit 37183a2

Browse files
committed
fix(tooling): a derived parse's verdict comes BACK, so one unparseable synthesis stops taking the whole run
`scripts/ts-parse.mjs` answers "could I read this tree?" by ending the process, which is right for a tree the gate's own author controls. A gate that re-parses text it SYNTHESISED from a source it already read is asking a different question, and it was getting that answer: `tenant-audit-census.mjs` stores a declared type's text whitespace-collapsed and re-parses it as a synthetic type alias, so a receiver whose members are separated by a newline alone -- legal TypeScript, 0 diagnostics as authored -- collapses to an unparseable alias and takes the entire census down with EXIT_UNPARSEABLE, naming a file that does not exist in the tree. `parseDerivedText` answers the synthesis question and returns the verdict. The floor is held by construction rather than by review: it requires an `origin` this module has certified, and the only two doors that certify one exit on a source that does not parse -- so a source the gate could not read cannot reach the returnable door. An uncertified origin is itself a refusal. On failure no tree comes back, so there is no recovered wreckage to walk and no reading of the result that says "nothing to report". The three existing refusals are untouched: every line from each diagnostics read through its `process.exit` is byte-identical, and the only change inside an existing door is one `vouchedSources.add` on a success path, after the refusal has already been decided. Claude-Session: https://claude.ai/code/session_019srGWGCBBCBHqcDoRZpQRh Co-authored-by: Claude <noreply@anthropic.com>
1 parent 07c6f82 commit 37183a2

1 file changed

Lines changed: 254 additions & 5 deletions

File tree

‎scripts/ts-parse.mjs‎

Lines changed: 254 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -113,6 +113,70 @@
113113
* could not read the tree" are different verdicts and a reader should not have
114114
* to guess which one they got. Both are non-zero, so CI fails either way.
115115
*
116+
* ## A FOURTH door, for a different question: text THIS PROCESS synthesised
117+
*
118+
* The three doors above all answer "could I read this tree?", and a refusal is
119+
* right for that question because the tree belongs to the gate's own author.
120+
* `packages/lint/src/checked-parse.ts` records the other side of the same axis
121+
* in as many words: it answers the same question by REPORTING rather than
122+
* refusing, "because a `scripts/**` gate audits a tree its own author controls"
123+
* while "a publish-time validator is handed metadata by someone else".
124+
*
125+
* A third position was never written down, and a gate in this tree was standing
126+
* on it. `tenant-audit-census.mjs` reads a receiver's declared type out of a
127+
* source that PARSED, stores that type's text whitespace-collapsed, and later
128+
* re-parses the stored text as a synthetic type alias
129+
* (`type CensusReceiver = <the stored text>;`) to ask whether it declares a
130+
* write door. When the collapse loses a member separator -- a type literal may
131+
* separate its members by a newline alone, which is legal TypeScript and
132+
* collapses to nothing -- the synthetic alias does not parse, and the census
133+
* takes `EXIT_UNPARSEABLE` for the whole run.
134+
*
135+
* Measured on 2026-09-18 against TypeScript 6.0.3:
136+
*
137+
* type text alias parses?
138+
* ---------------------------------------------------- -------------
139+
* { insert(o: string): Promise<void> yes (0 diags)
140+
* find(o: string): Promise<void> } <- authored
141+
* { insert(o: string): Promise<void> NO (1 diag,
142+
* find(o: string): Promise<void> } <- collapsed "';' expected")
143+
* { insert(o: string): Promise<void>; yes (0 diags)
144+
* find(o: string): Promise<void>; } <- lit control
145+
*
146+
* The refusal's own text is what makes the misfit legible: it names
147+
* `census-receiver-type.ts`, a file that does not exist in the tree, and its
148+
* reasoning -- "a file the gate could not read, reported as a file with nothing
149+
* to report" -- is FALSE here. The gate read the file. What did not round-trip
150+
* is the gate's own re-serialisation of a fragment of it, and that is a fact
151+
* about the synthesiser, never about the corpus. Ending the process on it turns
152+
* one site's unanswerable question into no answer for any site.
153+
*
154+
* So {@link parseDerivedText} answers "did the text I synthesised parse?" and
155+
* hands the verdict BACK, and the floor is held by making that door unreachable
156+
* for the question the refusal exists for: it takes an `origin` -- a
157+
* `ts.SourceFile` this module has already certified -- and the only way to hold
158+
* one is {@link parseSourceFile} returning, or a Program {@link
159+
* createProgramChecked} vouched for. Both of those EXIT on a source that does
160+
* not parse. ⇒ a source the gate could not read cannot reach the returnable
161+
* door: it hits the refusal first, by construction rather than by review.
162+
* Handing an uncertified origin is itself a refusal ({@link
163+
* EXIT_DERIVED_MISUSE}), for the reason the section above gives: a throw there
164+
* would be one `catch` away from the silent skip.
165+
*
166+
* Two deliberate divergences from the `packages/lint` sibling, both in the
167+
* strict direction, because this is `scripts/**`:
168+
*
169+
* • it returns NO tree on failure (`sourceFile: null`), where the sibling
170+
* always returns the recovered one. The sibling has callers that already
171+
* report findings off a partial tree; here there are none, and a recovered
172+
* tree is the thing a caller walks before scoring it clean. A caller that
173+
* forgets to branch gets a TypeError, which is loud and non-zero.
174+
* • the failure is not swallowable into a PASS. It is data, so a caller can
175+
* attribute it to one site -- but there is no verdict in it that reads as
176+
* "nothing to report", and `parseCensus()` counts it as a `rejection`,
177+
* never as a `refusal`: the run continued, and a census that conflated the
178+
* two would be lying about its own numerator.
179+
*
116180
* ## The knobs that are NOT knobs
117181
*
118182
* `ScriptTarget.Latest` and `setParentNodes: true` are fixed here because all
@@ -269,26 +333,54 @@ function batteryFloorFailures() {
269333
*/
270334
export const EXIT_UNPARSEABLE = 3;
271335

272-
/** Parses attempted, the distinct file names, and the refusals. */
273-
const census = { parses: 0, programs: 0, transpiles: 0, files: new Set(), refusals: 0 };
336+
/** Parses attempted, the distinct file names, the refusals and the rejections. */
337+
const census = {
338+
parses: 0, programs: 0, transpiles: 0, derived: 0, files: new Set(), refusals: 0, rejections: 0,
339+
};
274340

275341
/**
276342
* A snapshot of what this module has been asked to parse in this process.
277343
*
278-
* `parses` counts every source that reached the parser through ANY of the three
344+
* `parses` counts every source that reached the parser through ANY of the four
279345
* entry points, so it stays the numerator of "how much of this run was read";
280-
* `programs` and `transpiles` say which door they came through.
346+
* `programs`, `transpiles` and `derived` say which door they came through.
347+
*
348+
* `refusals` and `rejections` are deliberately separate totals rather than one
349+
* "failures" number. A refusal ENDED THE RUN, so every source after it went
350+
* unread and the numerator above is short by an unknown amount; a rejection is
351+
* a verdict {@link parseDerivedText} handed back about text this process
352+
* synthesised, with the run still going and every later source still read. A
353+
* single total would make those two indistinguishable in the one report whose
354+
* job is to say how much of the run was actually measured.
281355
*/
282356
export function parseCensus() {
283357
return {
284358
parses: census.parses,
285359
programs: census.programs,
286360
transpiles: census.transpiles,
361+
derived: census.derived,
287362
files: census.files.size,
288363
refusals: census.refusals,
364+
rejections: census.rejections,
289365
};
290366
}
291367

368+
/**
369+
* The `ts.SourceFile`s this module has CERTIFIED as parseable.
370+
*
371+
* Membership is what {@link parseDerivedText} requires of its `origin`, and it
372+
* is the whole floor argument for that door: a source the parser could not read
373+
* never becomes a member, because the two doors that add members
374+
* ({@link parseSourceFile}, {@link createProgramChecked}) end the process
375+
* instead of returning. So the one door in this module whose failure is
376+
* RETURNABLE is unreachable for the input the refusal exists for.
377+
*
378+
* A `WeakSet` so a long scan does not retain every tree it has read, and so
379+
* `has()` answers `false` for a non-object rather than throwing -- an
380+
* uncertified origin must reach the misuse refusal, not a TypeError.
381+
*/
382+
const vouchedSources = new WeakSet();
383+
292384
let censusReportArmed = false;
293385

294386
/**
@@ -323,7 +415,8 @@ function armCensusReport() {
323415
const c = parseCensus();
324416
process.stderr.write(
325417
`[ts-parse census] ${c.parses} parse(s) over ${c.files} distinct file name(s) `
326-
+ `(${c.programs} program(s), ${c.transpiles} transpile(s)); ${c.refusals} refusal(s)\n`,
418+
+ `(${c.programs} program(s), ${c.transpiles} transpile(s), ${c.derived} derived); `
419+
+ `${c.refusals} refusal(s), ${c.rejections} rejection(s)\n`,
327420
);
328421
});
329422
}
@@ -484,6 +577,7 @@ export function parseSourceFile(fileName, text, scriptKind) {
484577
process.stderr.write(refusalReport(fileName, scriptKind, diagnostics));
485578
process.exit(EXIT_UNPARSEABLE);
486579
}
580+
vouchedSources.add(sourceFile);
487581
return sourceFile;
488582
}
489583

@@ -574,6 +668,13 @@ export function createProgramChecked(rootNames, options, host) {
574668
process.stderr.write(programRefusalReport(roots, rows));
575669
process.exit(EXIT_UNPARSEABLE);
576670
}
671+
// Every file this Program pulled in has just been through the syntactic
672+
// check above -- transitively, which is the point of checking the whole
673+
// Program rather than its roots -- so each one is certified for
674+
// {@link parseDerivedText}. A gate that reads its corpus through a Program
675+
// can therefore synthesise from it on the same terms as one that parses file
676+
// by file.
677+
for (const sf of program.getSourceFiles()) vouchedSources.add(sf);
577678
return program;
578679
}
579680

@@ -619,6 +720,154 @@ export function transpileChecked(fileName, text, transpileOptions = {}) {
619720
return result;
620721
}
621722

723+
/**
724+
* The exit status of a MISUSE of {@link parseDerivedText} -- an origin this
725+
* module never certified.
726+
*
727+
* Distinct from {@link EXIT_UNPARSEABLE} for the reason that code is distinct
728+
* from 1: "this gate found violations", "I could not read the tree" and "this
729+
* call site is asking the wrong door" are three verdicts, and a reader should
730+
* not have to guess which one they got. All are non-zero, so CI fails either
731+
* way.
732+
*/
733+
export const EXIT_DERIVED_MISUSE = 4;
734+
735+
/**
736+
* The refusal text for a derived parse whose origin was never certified.
737+
*
738+
* Separate from the exit so a self-test case can read it, exactly as
739+
* {@link refusalReport} is.
740+
*/
741+
export function derivedMisuseReport(fileName, origin) {
742+
const what = origin === null ? 'null'
743+
: origin === undefined ? 'undefined'
744+
: typeof origin === 'object' ? `an object with fileName ${JSON.stringify(origin.fileName ?? '(none)')}`
745+
: `a ${typeof origin}`;
746+
return [
747+
`x ts-parse — REFUSING a derived parse whose ORIGIN this module never certified.`,
748+
``,
749+
` derived text ${fileName}`,
750+
` origin ${what}`,
751+
``,
752+
` parseDerivedText answers a different question from parseSourceFile:`,
753+
` "did the text I synthesised parse?", not "could I read this tree?". Its`,
754+
` verdict is returnable ONLY because the tree behind it has already been`,
755+
` read, so the origin must be a ts.SourceFile this module returned from`,
756+
` parseSourceFile, or one a Program from createProgramChecked was built`,
757+
` over. Both of those end the process on a source that does not parse.`,
758+
``,
759+
` Reading a source off disk and handing its text here would route the one`,
760+
` door whose failure is NOT a refusal at exactly the input the refusal`,
761+
` exists for — a file the gate could not read, scored as a file with`,
762+
` nothing to report. Parse it with parseSourceFile, and pass the tree`,
763+
` that call returns as the origin of anything you synthesise from it.`,
764+
``,
765+
` This is an exit rather than a throw for the reason the refusals are:`,
766+
` a throw here is one \`catch\` away from the silent skip.`,
767+
``,
768+
].join('\n');
769+
}
770+
771+
/**
772+
* The verdict text for derived text that did not parse. NOT a refusal: it names
773+
* the origin, says the run continues, and is meant to be printed by the caller
774+
* against the site the text came from.
775+
*/
776+
export function derivedFailureReport(fileName, originName, scriptKind, rows) {
777+
return [
778+
`! ts-parse — derived text does not parse. The source it came FROM does.`,
779+
``,
780+
` derived text ${fileName}`,
781+
` derived from ${originName}`,
782+
` parsed as ${describeScriptKind(scriptKind)}`,
783+
` errors ${rows.length} parse diagnostic(s) from TypeScript ${ts.version}`,
784+
``,
785+
...locationLines(fileName, rows),
786+
``,
787+
` This is a fact about the text THIS PROCESS SYNTHESISED, not about the`,
788+
` source above: that source was read, and certified, before this text was`,
789+
` built from it. So the run is NOT aborted — the verdict is returned, and`,
790+
` the caller attributes it to the one site it belongs to instead of`,
791+
` losing every other site to a process exit.`,
792+
``,
793+
` ⛔ It is still not a pass. No tree comes back with this (sourceFile is`,
794+
` null), so there is no recovered wreckage to walk and no reading of this`,
795+
` result that says "nothing to report". If your synthesis is supposed to`,
796+
` round-trip, this is the bug in the synthesis.`,
797+
``,
798+
].join('\n');
799+
}
800+
801+
/**
802+
* Parse text THIS PROCESS SYNTHESISED from a source it has already read, and
803+
* hand the verdict back instead of ending the run.
804+
*
805+
* ⚠️ ⛔ NOT a general escape from the refusal, and ⛔ not for a source read off
806+
* disk: see the `origin` parameter. The header section "A FOURTH door" carries
807+
* the whole argument, the measurement behind it, and why this is the only door
808+
* here whose failure is returnable.
809+
*
810+
* @param {ts.SourceFile} origin The tree `text` was derived from, as returned
811+
* by {@link parseSourceFile} or pulled from a {@link createProgramChecked}
812+
* Program. Anything else ends the process with {@link EXIT_DERIVED_MISUSE}.
813+
* @param {string} fileName What to call the synthesised text. Its extension
814+
* picks the ScriptKind when `scriptKind` is omitted, exactly as for
815+
* {@link parseSourceFile}; a synthesised source has no real path, so give it
816+
* a `.ts`/`.tsx` name that says what it is.
817+
* @param {string} text The synthesised source.
818+
* @param {ts.ScriptKind} [scriptKind] Omit to let `fileName` decide.
819+
* @returns {{ sourceFile: ts.SourceFile|null, failure: null|{ message: string,
820+
* line: number, column: number, count: number,
821+
* rows: { line: number, column: number, message: string }[], report: string } }}
822+
* `failure` null ⇒ it parsed and `sourceFile` is the tree. Otherwise
823+
* `sourceFile` is null — there is deliberately no recovered tree to walk —
824+
* and `failure.report` is the text to print against the site.
825+
*/
826+
export function parseDerivedText(origin, fileName, text, scriptKind) {
827+
armCensusReport();
828+
if (!vouchedSources.has(origin)) {
829+
process.stderr.write(derivedMisuseReport(fileName, origin));
830+
process.exit(EXIT_DERIVED_MISUSE);
831+
}
832+
census.parses += 1;
833+
census.derived += 1;
834+
census.files.add(fileName);
835+
836+
// The same fixed knobs the three doors above pass, for the reason the header
837+
// gives: a call site that needs a different pair says so once, here. Pinned
838+
// behaviourally by the self-test rather than by this comment -- a derived
839+
// tree has its parents set and reads modern syntax, or the knobs drifted.
840+
const sourceFile = ts.createSourceFile(
841+
fileName,
842+
text,
843+
ts.ScriptTarget.Latest,
844+
/* setParentNodes */ true,
845+
scriptKind,
846+
);
847+
848+
const rows = describeDiagnostics(sourceFile);
849+
if (rows.length === 0) {
850+
// Derived text that parsed is itself a tree this module has read, so a
851+
// second-order synthesis (a fragment of a fragment) can name it as origin.
852+
vouchedSources.add(sourceFile);
853+
return { sourceFile, failure: null };
854+
}
855+
856+
census.rejections += 1;
857+
const first = rows[0];
858+
return {
859+
sourceFile: null,
860+
failure: {
861+
message: first.message,
862+
line: first.line,
863+
column: first.column,
864+
count: rows.length,
865+
rows,
866+
report: derivedFailureReport(fileName, origin.fileName, scriptKind, rows),
867+
},
868+
};
869+
}
870+
622871
// ---------------------------------------------------------------------------
623872
// Self-test -- real child processes, because the refusal IS a process exit
624873
// ---------------------------------------------------------------------------

0 commit comments

Comments
 (0)