Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,25 @@
# Changelog

## 0.4.0

- **Representability repair loop** (Phase-2): an a2ui `EmitSurfaceError` — the
emitter refusing a contract-legal-but-unrepresentable surface (declared
casualties, transparent-dissolution donation boundaries, Collect join key
violations) — now rides the bounded repair loop instead of finalizing
`failed-gate` on first refusal. The refusal text becomes the repair turn,
with one class-targeted hint; exhausted budget keeps the original terminal
semantics (`failed-gate`, exit 3, `emitted.refusal`). New additive report
field `attempts[].representability = { pass: false, refusal }`.
- **Casualty-free generation view**: with `RunOptions.emitProfile` set,
generation compiles from the contract minus the profile's declared
`casualtyComponents` (system-prompt vocabulary, generation schema, few-shot
examples). S-gates and the report's contract digest keep the ratified
original contract.
- **Ollama `think: false`**: structured-output generation cannot budget
reasoning tokens — thinking models burned the whole `num_predict` on
reasoning and returned "empty model output"; harmless on non-thinking
models.

## 0.3.2

- `./browser` export: a supported browser-safe boundary — `runPipeline`,
Expand Down
5 changes: 5 additions & 0 deletions docs/AUDIT.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,11 @@ no guarantees.
the target-equivalent emitter-gate failure. Outcome is `failed-gate`; `emitted.validations`
is empty and `emitted.surfaceMessages` absent in that case. Reports written before this
field existed never carried refusals (the pipeline crashed instead — the flaw this fixed).
- `attempts[].representability` (2026-08-10, Phase-2): `{ pass: false, refusal }` on each
attempt whose lint-clean surface the active emit profile refused. Refusals ride the
bounded repair loop now, so a `passed` run may carry refused attempts on its trail;
`emitted.refusal` still marks the terminal case (budget exhausted). The matching repair
turn appears verbatim in `repairMessages[]` at the attempt's index.

Breaking changes bump `reportVersion` and get a new schema file; version "1" documents stay
valid against the "1" schema forever.
7 changes: 4 additions & 3 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@aestheticfunction/dspack-gen",
"version": "0.3.2",
"version": "0.4.0",
"description": "Generation + governance pipeline for dspack contracts: prompt/context compiler, surface gates S1–S3, bounded repair, protocol emission, audit reports.",
"type": "module",
"license": "Apache-2.0",
Expand Down Expand Up @@ -66,7 +66,7 @@
"test:pack": "bash scripts/pack-test.sh"
},
"dependencies": {
"@aestheticfunction/dspack-emit": "^0.3.1 || ^0.4.0 || ^0.5.0 || ^0.6.0",
"@aestheticfunction/dspack-emit": "^0.3.1 || ^0.4.0 || ^0.5.0 || ^0.6.0 || ^0.7.0",
"@anthropic-ai/sdk": "^0.109.1",
"ajv": "^8.17.1",
"ajv-formats": "^3.0.1",
Expand Down
16 changes: 16 additions & 0 deletions schemas/audit-report.v1.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,22 @@
"adapterError": {
"type": "string"
},
"representability": {
"type": "object",
"description": "Phase-2 representability (additive in v1): the active emit profile's typed refusal of this attempt's lint-clean surface. Refusals are repair-loop events, so a passed run may carry refused attempts.",
"required": [
"pass",
"refusal"
],
"properties": {
"pass": {
"const": false
},
"refusal": {
"type": "string"
}
}
},
"gates": {
"type": "array",
"items": {
Expand Down
3 changes: 3 additions & 0 deletions src/adapters/adapters.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,9 @@ describe("OllamaAdapter", () => {
expect(capture.url).toBe("http://ollama.test/api/chat");
expect(capture.body!.format).toEqual(context.schema); // depth-unrolled schema round-trips
expect(capture.body!.stream).toBe(false);
// Thinking models must not spend the structured-output budget on reasoning
// (qwen3.6 returned "empty model output"); harmless on non-thinking models.
expect(capture.body!.think).toBe(false);
expect((capture.body!.messages as unknown[]).length).toBe(request.messages.length + 1); // + system
expect(result.json).toEqual(workedSurface);
expect(result.usage).toEqual({ inputTokens: 100, outputTokens: 50 });
Expand Down
6 changes: 6 additions & 0 deletions src/adapters/ollama.ts
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,12 @@ export class OllamaAdapter implements GenerationAdapter {
model: this.model,
stream: false,
format: request.jsonSchema,
// Structured-output generation cannot budget reasoning tokens: thinking
// models (qwen 3.6 — spelled with a space; the no-default-model guard
// scans this file) burn the whole `num_predict` on reasoning and return
// empty content ("empty model output"). Empirically verified harmless
// on non-thinking models (gemma4 ignores it).
think: false,
options: { temperature: request.params?.temperature ?? 0.2 },
messages: [{ role: "system", content: request.system }, ...request.messages],
};
Expand Down
8 changes: 8 additions & 0 deletions src/audit/report.ts
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,14 @@ export interface AttemptRecord {
/** Surface gates S1/S2/S3, independently reported. */
gates?: GateReport[];
findings?: Finding[];
/**
* Phase-2 representability (additive in v1): the active emit profile's
* typed refusal of this attempt's lint-clean surface. Recorded per attempt
* because refusals are repair-loop events now, not only terminal ones — a
* passed run can carry refused attempts on its trail. Run-layer concern:
* the S-gates above are untouched by it.
*/
representability?: { pass: false; refusal: string };
}

export interface EmittedValidation {
Expand Down
53 changes: 53 additions & 0 deletions src/run/casualty-view.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
/**
* Casualty-free generation view (Phase-2 representability mechanism, part 2).
*
* A profile's `casualtyComponents` are contract components the active emit
* target cannot represent — the emitter refuses them by design. Compiling the
* generation context from the FULL contract steers the model straight into
* those refusals: the casualty sits in the system-prompt vocabulary, in the
* generation schema, and (worst) in few-shot examples the model is told to
* imitate. This view removes them from what GENERATION sees, and nothing else.
*
* Run-layer on purpose: `core` stays protocol-neutral and profile-unaware.
* The caller (orchestrator) uses the view ONLY for compileContext — the
* S-gates (lintSurface) and the report's contract digest keep the ratified
* original, so governance and report identity never vary with the profile.
*/
import type { Profile } from "@aestheticfunction/dspack-emit";
import type { Contract, ExampleEntry } from "../core/contract.js";

/**
* The contract minus the profile's declared casualties: casualty component
* entries are dropped, and every example whose surface tree uses a casualty
* is dropped with them (a few-shot exemplar of a refused component is a
* steering bug, not a teaching aid). Shallow-copied; the original contract is
* never mutated. Without a profile — or with no declared casualties — the
* SAME contract object is returned, so identity checks and digests are
* trivially unaffected.
*/
export function casualtyFreeView(contract: Contract, profile?: Profile): Contract {
const casualtyIds = new Set((profile?.casualtyComponents ?? []).map((c) => c.dspackId));
if (casualtyIds.size === 0) return contract;

const view: Contract = { ...contract };
if (contract.components) {
view.components = Object.fromEntries(
Object.entries(contract.components).filter(([id]) => !casualtyIds.has(id)),
);
}
if (contract.examples) {
view.examples = contract.examples.filter((example: ExampleEntry) => !usesCasualty(example.surface, casualtyIds));
}
return view;
}

/** Walk any object/array shape for a node with `component` ∈ casualtyIds. */
function usesCasualty(value: unknown, casualtyIds: Set<string>): boolean {
if (Array.isArray(value)) return value.some((entry) => usesCasualty(entry, casualtyIds));
if (value !== null && typeof value === "object") {
const record = value as Record<string, unknown>;
if (typeof record.component === "string" && casualtyIds.has(record.component)) return true;
return Object.values(record).some((entry) => usesCasualty(entry, casualtyIds));
}
return false;
}
82 changes: 75 additions & 7 deletions src/run/orchestrator.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,11 @@
* failed-adapter (exit 1) — produces a complete audit report. The system
* prompt is immutable across attempts; the only delta between attempts is
* the model's own output plus the rendered repair feedback.
*
* Phase-2 representability: an a2ui EmitSurfaceError (the emitter refusing a
* contract-legal surface) rides the same bounded repair loop — the refusal
* text becomes the repair turn — and generation compiles from a casualty-free
* view of the contract so the model is not steered into refusals at all.
*/
import {
buildCatalogModel,
Expand All @@ -25,6 +30,7 @@ import {
} from "@aestheticfunction/dspack-emit";
import type { Contract } from "../core/contract.js";
import { applicableRules, compileContext, type CompileOptions } from "../core/compiler.js";
import { casualtyFreeView } from "./casualty-view.js";
import { lintSurface, type Finding, type GateReport } from "../core/lint/index.js";
import { AdapterOutputError, type GenerateMessage, type GenerationAdapter } from "../adapters/types.js";
import { renderRepairMessage, type RepairTemplate } from "../repair/render.js";
Expand Down Expand Up @@ -107,14 +113,59 @@ const A_GATE: Record<string, "A1" | "A2" | "A3"> = {
instance: "A3",
};

/**
* Refusal-class detection over the emitter's message text: one targeted hint
* per class, matched in declaration order. The emitter's refusal strings are
* typed API surface in spirit (dspack-emit pins them in its own tests), so
* matching on them is the honest seam short of structured refusal codes.
*/
const REFUSAL_HINTS: ReadonlyArray<{ pattern: RegExp; hint: string }> = [
{
pattern: /declared casualty/,
hint: "Do not use that component with this profile — express the same meaning with other approved components.",
},
{
pattern: /donation boundary/,
hint: "Each transparent form wrapper (e.g. 'field') must contain exactly one labeled control.",
},
{
pattern: /carries no key|dangling counterpart|join/,
hint: "Give every collected sub-component item a unique `id`, and make paired items (e.g. tabs-trigger/tabs-content) use matching ids.",
},
];

/**
* The representability repair turn (Phase-2): the emitter refusal verbatim —
* it names the offending component/path precisely — plus at most one
* class-targeted hint. Unlike S3 repair messages this is not rendered from
* findings (there are none: the surface is lint-clean); the refusal IS the
* finding.
*/
function representabilityRepairMessage(refusal: string): string {
const base =
"The surface passed all governance gates but cannot be represented by the active emit profile. " +
`Emitter refusal: ${refusal}. Correct the composition and return the complete corrected JSON object.`;
const hint = REFUSAL_HINTS.find(({ pattern }) => pattern.test(refusal))?.hint;
return hint ? `${base} ${hint}` : base;
}

export async function runPipeline(options: RunOptions): Promise<RunResult> {
const { contract, intent, prompt, adapter } = options;
const maxRepairs = options.maxRepairs ?? 2;
const repairTemplate = options.repairTemplate ?? "standard";
const now = options.now ?? (() => new Date());
const startedAt = now();

const context = compileContext(contract, intent, options.compile);
// Generation compiles from the casualty-free view of the contract: the
// active emit profile's declared casualties leave the system-prompt
// vocabulary, the generation schema, and the few-shot exemplars, so the
// model is never steered into components the emitter must refuse. ONLY
// generation sees the view — lintSurface below keeps the ORIGINAL contract
// (the S-gates govern the contract as ratified; S2 vocabulary is unchanged)
// and so does contractDigest (report identity must not vary with the
// emit profile).
const generationContract = casualtyFreeView(contract, options.emitProfile);
const context = compileContext(generationContract, intent, options.compile);
const conversation: GenerateMessage[] = [
...context.fewshot,
...(options.conversation ?? []),
Expand Down Expand Up @@ -231,17 +282,29 @@ export async function runPipeline(options: RunOptions): Promise<RunResult> {
}

// The emitter can REFUSE a lint-clean surface outright (typed
// EmitSurfaceError — e.g. a sub-component outside its compound parent:
// in-vocabulary for S2, ungoverned by S3, unprojectable by the
// profile). That is the emitter-gate failure class ("target
// equivalent" in the exit-code table), not a crash: outcome
// failed-gate, exit 3, refusal recorded in the report (ADR-D1 family
// evidence, same as an A3 rejection).
// EmitSurfaceError — declared casualties, transparent-dissolution
// donation boundaries, Collect join key violations: in-vocabulary for
// S2, ungoverned by S3, unprojectable by the profile). That is the
// emitter-gate failure class ("target equivalent" in the exit-code
// table), not a crash — and since Phase-2 it is a REPAIRABLE one: the
// refusal text is a precise repair instruction, so while repair budget
// remains it becomes the next repair turn instead of dying terminal.
// Exhausted budget keeps the original semantics: outcome failed-gate,
// exit 3, refusal recorded in the report (ADR-D1 family evidence, same
// as an A3 rejection).
let emission: EmitSurfaceResult;
try {
emission = emitSurface(surface, doc, options.emitProfile ? { profile: options.emitProfile } : {});
} catch (error) {
if (error instanceof EmitSurfaceError) {
attempts[attempts.length - 1].representability = { pass: false, refusal: error.message };
if (index < maxRepairs) {
const repair = representabilityRepairMessage(error.message);
repairMessages.push(repair);
conversation.push({ role: "assistant", content: generated.raw }, { role: "user", content: repair });
emit({ type: "repair", index, message: repair });
continue;
}
const emitted = { target: "a2ui" as const, refusal: error.message, warnings: [], validations: [] };
emit({ type: "emitted", validations: [], warnings: [] });
const result = finalize("failed-gate", 3, {}, emitted);
Expand All @@ -268,6 +331,11 @@ export async function runPipeline(options: RunOptions): Promise<RunResult> {

const emitted = { target: "a2ui" as const, surfaceMessages: { messages }, warnings, validations };
emit({ type: "emitted", validations, warnings });
// The post-emit validations-fail branch stays TERMINAL (no repair):
// emit-side self-validation (dspack-emit ≥0.7 gates its own output
// before returning) makes this branch unreachable from emit output —
// it remains as the guard for older emitters and caller-supplied
// a2uiVersions the profile was never validated against.
const result = gatesPass
? finalize("passed", 0, { surface, surfaceMessages: { messages } }, emitted)
: finalize("failed-gate", 3, {}, emitted);
Expand Down
13 changes: 11 additions & 2 deletions src/run/pipeline.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,7 @@ describe("failure paths are first-class artifacts", () => {
expect(validateReport(JSON.parse(JSON.stringify(result.report)))).toBe(true);
});

it("emitter REFUSAL: lint-clean surface the emitter cannot project at all → failed-gate, exit 3, refusal recorded", async () => {
it("emitter REFUSAL with no repair budget → failed-gate, exit 3, refusal recorded", async () => {
// The live-eval discovery (2026-07-03, qwen): a sub-component outside its
// compound parent was in-vocabulary (S2), ungoverned (S3), but the a2ui
// profile cannot emit it standalone — EmitSurfaceError. That is the
Expand All @@ -161,6 +161,11 @@ describe("failure paths are first-class artifacts", () => {
// shipped profile; 'dialog' is intent-forbidden by S3 here) —
// in-vocabulary, ungoverned in this surface, and refused by the emitter
// with the casualty reason.
//
// Phase-2 note: refusals are REPAIRABLE now (representability.test.ts
// covers the loop); maxRepairs 0 pins the preserved TERMINAL semantics —
// refusal with the budget exhausted stays failed-gate/exit 3 with the
// refusal recorded.
const refusalBreaker: Surface = {
dspackSurface: "0.1",
system: "shadcn/ui",
Expand All @@ -174,10 +179,14 @@ describe("failure paths are first-class artifacts", () => {
},
};
const adapter = new ScriptedAdapter([{ output: refusalBreaker }]);
const result = await runPipeline({ ...baseOptions, adapter });
const result = await runPipeline({ ...baseOptions, adapter, maxRepairs: 0 });
expect(result.report.outcome).toBe("failed-gate");
expect(result.exitCode).toBe(3);
expect(result.report.emitted!.refusal).toContain("dropdown-menu");
expect(result.report.attempts[0].representability).toEqual({
pass: false,
refusal: expect.stringContaining("dropdown-menu"),
});
expect(result.report.emitted!.validations).toEqual([]);
expect(result.surfaceMessages).toBeUndefined();
expect(validateReport(JSON.parse(JSON.stringify(result.report)))).toBe(true);
Expand Down
Loading
Loading