Skip to content

Examples and API coverage contract #14

Description

@dvcolomban

Current executed-evidence checkpoint, 2026-10-04

The owner accepted the bounded inspector proof and Portable contribution proof is closed. This resolves its human walkthrough gate, not the broader SDK matrix. Documentation reconciliation commit 241fd1e updates that status in the inventory, removes stale hardcoded narrative counts and describes the actual common browser/debugger receipt checks.

Current local inventory accounts for 814 public paths, 460 linked paths and 38 groups with remaining gaps. It validates 110 browser references across 65 receipts and 621 distinct browser checks, plus 366 unit-test references. The two new same-host native path-repair cases execute routed actions, continuing native peer mutations and actual disk restoration on both backends. Scoped checks and all 18 affected package cases pass. Separate live IAB observations are explicitly manual evidence and do not change automated browser counts. Current full CI is queued/running. New actual-browser evidence includes pending native RPC suspension in both browsers, ordinary production-panel navigation and closure, and malformed Firefox saved-state rejection/recovery. The state checkpoints retain separate commit references, native-only bug reproduction, exact commands, receipts and limits. The prior independently inspected CI passed at 241fd1e. Independent inspection of its downloaded artifact verifies all 110 browser references, 65 receipts, 621 referenced checks and 364 unit-test references; every linked unit case passed. The 38 broader gap groups remain open.

The conformance completion gate still requires remaining host/mode and API obligations. No public API, release promise or coverage gap is declared complete solely by owner prototype acceptance or these bounded browser slices.

Built public Devframe client declarations, 2026-10-03

Commit ac59e94 adds two test files using built package exports and strict TypeScript 7, with no source aliases and skipLibCheck false. No SDK or runtime change.

const ordinary: ActionBinding = { action: incrementAction };
const routed: ActionBinding = { action: incrementAction, routing: { realm: 'webext', provider: 'example.extension' } };
const broadcast: ActionBinding = { action: incrementAction, selection: [{ realm: 'devserver' }] };
// Each is checked separately through createActionCall; duplicate action IDs are not passed together.

Definition of Ready: six exported type entries lacked an exact compiler result. The maintained package already builds the public client subpath.

Definition of Done for this slice:

  • Compile ActionBinding, its action kind, ActionCallOptions, ProviderRpcClient, RpcProviderConnection and RpcProviderConnectionOptions through the actual manifest.
  • Eight expected errors are consumed: conflicting routing/selection, executable-action kind, missing portable action client, missing native collector, missing realm, wrong numeric input/result and wrong capability input. Positive native composition and action/capability inference pass.
  • All eleven affected package tests, strict lint, TS7, formatting and the executed inventory gate pass. Its six paths now have explicit compiler evidence.
  • Full CI at ac59e94 passed. Downloaded artifacts independently confirm all eleven Devframe package cases, including the exact public client compiler case; 46 native references to35 payloads and400 unique checks also pass review.

The local matrix now has 814 paths, 431 linked paths and 36 partial gap groups. The separate all-stage frame/CSP proof expands existing payloads to 400 unique native checks across 35 payloads and 46 references. Selected Bundler declarations do not prove all optional/generic members, NodeNext, auth, catalogs, connection lifetime or browser routing. Those remain with their native evidence groups.

Action-binding contract routing proof, 2026-10-03

Commit 64f02b5 adds one focused native MessageChannel/router/provider case. Four real local providers have matching IDs across two realms, so selecting only the provider ID cannot accidentally satisfy the test. Native calls retain their separate argument path. No runtime/API change.

Client default: devserver / second
  → contract default: webext / first
  → explicit binding override: devserver / second
Broadcast selection: devserver → both devserver providers, independently of defaults
Missing explicit override → unavailable-provider; no default fallback

Definition of Ready: three imported action-routing paths lacked direct binding evidence. The existing native call/router fixture supplies it.

Definition of Done for this slice:

  • Five direct assertions pass: contract precedence, native passthrough, binding override, independent broadcast and missing-override rejection.
  • Full affected package execution passes ten tests; strict lint, TS7 and formatting pass. Canonical evidence includes the exact new test name.
  • The inventory links the three action.routing paths. The executed local gate has 814/425 paths, 288 exact native checks across 35 payloads and 36 gap groups after the separate native-stage slice.
  • Full CI at 64f02b5 passed. The downloaded native unit report independently confirms all ten package cases, including the exact routing case; every referenced browser payload/check also passes review.

Browser-host contract defaults, other routing-policy forms at this binding boundary, complete type contracts remain separate obligations. The built-client fixture below now directly asserts the action kind field. The broad Examples/API ticket remains open.

Built public runtime declaration proof, 2026-10-03

Commit d440a18 executes two strict TypeScript 7 consumers against manifest-built @devkit/runtime and @devkit/core exports. The old result fixture used a test helper's overloads; the new fixture calls the actual public factory. No runtime or API changes.

Fixture Positive contract Consumed negative assertions
Installation results Default/explicit strict, false and boolean factory configuration; service/plugin install, replace and startup result inference Six: strict has no envelope; relaxed is not a direct handle; skipped has no handle; relaxed/configured startup cannot assume handles; boolean configuration requires narrowing.
Native lookup Descriptor value type and optional result survive provider options, local invocation context and invocation requests Four: wrong value types reject at all three paths, and optional lookup cannot be treated as always present.
Maintained fixture → temporary consumer under package
  → manifest-built declarations + strict TS7 / skipLibCheck false / no source aliases
  → positive compilation and consumed @ts-expect-error checks
  → exact passed Vitest names → API inventory links

Definition of Ready: nine inventory paths had no executed compiler evidence; the installed public exports supply the proof without a source/API workaround.

Definition of Done for this slice:

  • Both compiler cases pass and consume all ten expected errors. Temporary consumers are removed after every case.
  • All 94 runtime tests, its build, strict type-aware lint, TS7 and formatting pass. Independent inspection confirms both exact compiler names in the canonical JSON report.
  • The API gate verifies 814 paths and 422 linked paths. Nine runtime paths gain evidence and 26 remain unmapped. Two narrowly defined type-evidence groups retain their own limitations, giving 36 gap groups rather than silently treating partial type proof as full conformance.
  • Full CI at d440a18 passed. Independent artifact review verifies all 94 runtime cases and both exact compiler cases, plus the 16 new Linux Firefox renderer checks. All 42 references to 31 distinct native payloads verify 248 exact checks, with 814/422 paths and 36 explicit gap groups.

This proves only the checked type contracts under pinned TS7/Bundler resolution. Actual native availability, permissions, transport/lifecycle behavior, unexamined type members, packed NodeNext consumers and other compiler versions remain separate obligations. The same combined head includes the Firefox renderer proof, with 16 additional exact native checks and a retained receipt.

Complete example unit-report retention, 2026-10-03

Commit ad88dbb adds Vitest's native JSON reporter to the debugger and Vite-host example commands. These were the only maintained examples whose executed unit results existed only in console output. CI already collects examples/*/.conformance/vitest.json; no new reporting infrastructure or runtime behavior is added.

Existing scoped Vitest commands → console output + native JSON reports
  → existing CI artifact collection → exact case names and outcomes retained

Definition of Ready: the omission was verified on the previous commit and its successful CI log; both workspaces use the same reporters as the other examples.

Definition of Done for this slice:

  • Scoped rerun passes all 26 debugger and 28 Vite-host tests. Both generated reports record success, exact counts and zero failed, pending or todo cases.
  • Both affected strict lint and TypeScript 7 commands pass; formatting and diff checks pass.
  • Full CI passed. Independent download confirms the two new reports contain exactly 26 and 28 passing cases, with zero failed/pending/todo outcomes. All 41 browser references to 30 distinct payloads and 232 exact checks also pass; the executed matrix remains 814/413 paths and 34 gap groups at that commit.

These unit reports supplement native browser receipts. They do not close any remaining host, lifecycle or public-API coverage gap.

Native script-context evidence reconciliation, 2026-10-02

Injection and transform contract records commit de473a7 and its native observations. The maintained example exposes native allFrames; both real browser drivers use the actual contribution UI and registration lifecycle.

  • Four production/development receipts contain 14 exact checks each. Independent review validates all 56 named checks, native registrations, absent localhost permission and 288 document observations.
  • Both native builds, all 30 affected unit tests, strict lint, TS7, formatting and executed API gate pass locally.
  • Matrix retains its 814/413 API paths and 34 gap groups, with 232 exact native checks across 30 distinct receipts. Broader frame/stage/CSP obligations remain explicit.
  • Full CI passed. Its actual executed report and all 41 references to 30 distinct retained payloads were independently checked: 814/413 API paths, 34 gap groups, 232 exact checks and 288 new document observations.

Core schema-envelope evidence reconciliation, 2026-10-02

Commit c5dcc71 links six directly asserted public fields that were still in the unmapped core group: OperationDefinition.input/output, CapabilityDescriptor.operations, and ActionDescriptor.operation plus its input/output. It changes only the inventory. Existing tests establish the behavior; there is no production or API change.

Direct evidence Established behavior
snapshots.test.ts and definitions.test.ts Caller mutation cannot replace captured input/output schema objects; copied maps/envelopes are frozen while caller objects stay mutable. Async/transforming schema identity is preserved.
invalid-definitions.test.ts Missing output, unsupported Standard Schema version and missing validator/vendor reject declaration admission.
runtime/tests/invocation.test.ts Original input/return values survive transforming validators; invalid input never dispatches; invalid return rejects and a thrown validator's original cause stays local.
Original schema objects → copied declaration envelope
  → validate original call input → original input reaches handler
  → validate original return → original return reaches caller
Caller mutation cannot replace the captured schema objects.

Definition of Ready

  • Six exact inventory paths have direct assertions, rather than inferred coverage from an imported containing type.
  • Every linked test name exists uniquely in actual passing Vitest output.
  • Accepted native guard-only semantics require no new owner decision.

Definition of Done for this slice

  • Narrow reruns pass: 43 core declaration/snapshot/rejection tests and 7 runtime invocation tests.
  • All 13 core/runtime references in the expanded existing group are verified against these actual generated reports.
  • Root tooling lint/types, formatting and the API evidence gate pass: 814 paths, 413 linked and 34 gap groups. The broad unmapped core group falls from 352 to 346 paths.
  • Full CI for the metadata-only commit passed all workspace/native browser gates. Its retained executed report independently verifies 814/413 paths and 34 remaining gap groups.

Focused commands: pnpm --filter @devkit/core exec vitest run tests/definitions.test.ts tests/snapshots.test.ts tests/invalid-definitions.test.ts and pnpm --filter @devkit/runtime exec vitest run tests/invocation.test.ts. The inventory remains an index to exact execution, not a claim that all SDK or host obligations are complete. These are local declaration/invocation checks; native serialization/browser behavior, all async outcomes, type-only aliases and remaining host/mode cells stay explicitly open.

Built public server declaration proof, 2026-10-02

Commit 9da3e51 closes the executed-declaration gap for ten server type paths. The existing compiler fixtures previously checked repository source and had no linked executed result. Four maintained Vitest cases now compile copies against the manifest's built @devkit/server and @devkit/server/client exports using strict TypeScript 7, skipLibCheck: false and no source aliases. Each temporary consumer is removed afterward. No runtime or public contract changes.

Fixture Accepted contract Rejected contract
Provider composition Devframe/DevTools options, exposure, handle; strict, relaxed and dynamic installation return types Old two-argument startup, executable plugin used as exposed action contract, Hub context used as DevTools context
Remote client Both genuine native RPC client types; numeric action input/result; option and connection aliases Wrong action input; raw backend primitives accessed through remote binding
Native contexts Base, Hub and DevTools descriptors preserve their native context types A context treated as a string or as a more specialized context
Connection isolation Native shared/discovered/prepared connection configuration Wrong flag type, legacy standalone toggle and partial prepared metadata
Maintained source fixtures
  → copy and change only adapter imports to package names
  → TypeScript resolves built manifest exports
  → valid declarations compile; each @ts-expect-error must reject
  → exact passing Vitest result is linked by the API gate

Definition of Ready

  • Existing source fixtures and the ten unmapped declaration paths are identified.
  • Public exports and native types supply the checks; no adapter mechanism is needed.
  • Type evidence is kept distinct from native runtime/browser behavior.

Definition of Done for this slice

  • All four built-export compiler cases and all 54 server tests pass, including real loopback HTTP/WebSocket hosts.
  • Affected strict lint, TypeScript 7 and formatting pass. The sandbox-only localhost EPERM run is not represented as a source regression; the real-host rerun passes.
  • The API gate verifies exact passing results: 814 paths, 407 linked, 34 groups retain gaps. This clears only the declaration group.
  • Full CI passed at the combined head 0da5f55.

Run pnpm --filter @devkit/server test, pnpm --filter @devkit/server lint and pnpm --filter @devkit/server typecheck after building the server dependency graph. The maintained compiler runner exercises built exports; real-host/browser matrix gaps remain with their named groups.

Full CI at 0da5f55 passed every workspace and native production/development gate. All 26 retained native receipts were downloaded and independently checked against every exact referenced scenario: 176 native checks, 814 API paths, 407 linked paths and 34 gap groups. This establishes acceptance of the declaration/capture slices; it does not identify the earlier intermittent failures.

Native cancellation evidence checkpoint, 2026-10-02

Injection and transform contract now records native Firefox cancellation acceptance in production and development. Both maintained commands and strict affected checks pass. Actual generated/retained outcomes match; the inventory requires the two exact cases in both receipts. It now links 176 native scenarios across 26 receipts, with 814 discovered/397 linked API paths and 35 gap groups. The preceding full CI at f03f4cc passed, and all its 26 referenced payloads were downloaded and independently verified. Full CI for the cancellation additions: Attempt 1 passed the workspace, production checks and both new Firefox cancellation cases in development, then failed navigating to the extension panel after native background reload (firefox-development.ts:171). The downloaded Firefox production/development response receipts each pass all eight exact referenced checks. Attempt 2 stopped earlier at the final Chromium idle screenshot (Page.captureScreenshot timeout), after its functional assertions. Both failure investigations remain open; no reload/capture fix or complete CI acceptance is claimed. The final executed-browser gate ran in neither attempt. Complete new CI acceptance is not yet claimed. Native running-background and bounded-observation limits remain explicit.

Routing diagnostic selection evidence, 2026-10-02

Commit 899fc5c strengthens the maintained ambiguity scenario and links three previously unlinked public paths: RoutingError, RoutingError.name and RoutingError.candidates. It changes no runtime or contract.

Two available providers satisfy the preferred dev-server route; a WebExtension provider is the lower-priority fallback. The failed call exposes the public error class, clear retry message and exactly the two preferred candidates. Its frozen diagnostic list retains the failed call's availability after one service is disabled. A new explicit call using the reported realm/provider pair reaches the chosen provider once, with no fallback dispatch.

Caller-owned selection uses the already implemented API:

// Illustrative picker composition; no automatic SDK retry or new picker API.
const selected = await chooseProvider(error.candidates);
return client.actions.invoke({
  action,
  input,
  routing: { realm: selected.realm.id, provider: selected.id },
});

Definition of Ready for this slice

  • The current inventory identifies the missing class/name/candidate evidence.
  • An existing real local-provider routing scenario supplies the full failure-to-explicit-selection flow.

Definition of Done for this slice

  • Existing scenario uses exactly ten Vitest assertions and passes without weakening strict lint.
  • All 31 client tests, affected lint/types and formatting pass.
  • Maintained examples/server-contexts/checks/routing.ts passes against genuine Devframe and DevTools hosts through built public exports.
  • Executed inventory now accounts for 814 paths and links 397; it still retains 35 gap groups and 172 exact native scenarios across 26 receipts.
  • Full CI for this commit and receipt retention passes at f03f4cc, including the strengthened routing test and final executed-evidence gate.

The diagnostic test exercises local providers; native-browser presentation, transport/disclosure guarantees and remaining error variants are not inferred from it. The existing live two-server check confirms composition, not an implemented UI picker. Full SDK conformance remains open.

Combined validation checkpoint, 2026-10-02

The first attempt of combined CI at bc5baa5 passed the workspace gate and preceding Chromium checks, then timed out taking the final screenshot after the natural-idle recovery assertions. The unchanged scenario passes locally; no cause or fix is claimed from that run. Its second attempt was superseded by a main push. Full CI at 0b533e8 passed every workspace, native production, persistence, idle, quota, debugger, trust, concurrent development and executed-evidence gate. This head only removes expired dependency-age exclusions; it changes no runtime or browser fixture. The original screenshot timeout remains an unexplained intermittent failure, with no fix claimed.

Native encoding and quota evidence, 2026-10-02

Encoding commit adds one exact aggregate five-case byte/header proof to each generated Firefox production/development response receipt. Quota commit adds six actual native failure/recovery/cleanup checks from a maintained reduced-limit command, and CI executes that command before the combined browser gate.

The inspected inventory now links172 distinct native scenarios across26 generated receipts, up from164/25. It still discovers814 public paths, links394 and retains35 groups with unresolved coverage gaps. Maintained examples, actual native runs, exact schema checks, all30 affected workspace tests and strict lint/types/format pass. Combined full CI passed at 0b533e8. Historical failed pilot receipts do not count as successful automated evidence. Native storage remains example-owned, and native encoded headers are not normalized to transformed-byte length.

State acceptance and limits and transform acceptance and limits own the detailed context, pseudocode, commands and DoR/DoD. No checker weakening, copied native protocol or SDK API change is introduced. Full conformance remains incomplete.

Public declaration evidence, 2026-10-02

Public core declaration checks add five accepted and 19 rejected compiled consumers through the built export, with 99/99 core tests and scoped lint/type checks passing. Current associations are 343 of 769 public API paths, 203 distinct Vitest tests, and 48 browser scenarios from five generated receipts. Full CI passed for 1e64fb6033433e25a13778d16b7d0d7747fcb06b, including the native browser suites and combined API-evidence gate. Compiler/mode, runtime and host coverage gaps remain explicit; this is not completion of this contract.

Native persistence recovery evidence, 2026-10-02

The exact reference and validation record links confirmed-write recovery after forced Chromium worker termination and explicit Firefox extension reload. The inventory now associates 320 of 769 public API paths, 179 distinct Vitest tests, and 48 browser scenarios from five generated receipts. Native example storage is not counted as a new SDK API. Natural suspension, interrupted writes, browser restart and Firefox quota behavior remain explicit gaps.

Full CI on the combined final state passed at 34e301f77bb9edfd767419e1506855f0ce00b3cb, including workspace validation, both native persistence tests, every production browser suite, concurrent development transitions and the combined executed API-evidence gate. The underlying Chromium persistence implementation already passed its full CI. Both affected references pass locally against fresh native receipts; the evidence checker tests and scoped tooling types/lint pass. These associations do not complete host/feature conformance.

Native-auth evidence and reconciled CI, 2026-10-02

The server-auth follow-up links native connection-error propagation to an executed real-socket revocation test. The inventory now links 320 of 769 public API paths to 179 distinct executed tests; the existing 44 browser scenarios remain unchanged. These are evidence associations, not complete host/feature coverage.

Full CI passed at ce546ce68e7df120a1adec6141768a8ca44a24d0, including the full workspace checks, production browser suites, concurrent development transitions and executed API-evidence gate. The previous Firefox reload failure and narrow test correction remain recorded. Full conformance and owner decisions stay open.

Runnable custom kinds and guard evidence, 2026-10-02

The implementation and validation record adds a custom native command contribution to both runnable server demos. Commit 781f054 uses existing declaration factories, the host-supplied kind installer and activation cleanup. Both real headless hosts prove the command's initial result, disable/unregistration, reenable and disposal through built package exports.

The inventory now links 318 of 769 public API paths to 178 distinct executed tests, with the existing 44 browser scenarios unchanged. New references cover six custom-kind paths and the portable operation-error guard; local runtime failure cases and actual native-host results retain separate scopes. All remaining generic type, browser, reload and failure obligations stay explicit. Local build/types/lint, both demos, affected tests and exact report checks passed. That CI run was superseded by the next debugger commit. Full CI on the combined main-branch state failed in the Firefox development check while reading a reloading DevTools panel. The new debugger scenarios and preceding workspace checks passed; the Firefox observation race is being investigated under Live preview and reload contract.

Port declaration and worker-lifetime evidence, 2026-10-01

Conformance implementation f31c51954b77b20b09891ff75d49e7a8d52ea9a0 links six executed strict TypeScript consumers and five native Chromium worker scenarios. The inventory now associates 311 of 769 public API paths with 165 distinct executed tests and 44 distinct browser scenarios from the same three generated browser receipts. No public SDK API was added.

The declaration consumers run where the browser types are installed, in the maintained WebExtension example. They resolve the built @devkit/webext export map normally: no source aliases, copied vendor declarations or test-only production hooks. TypeScript 7 checks Chrome and WXT Ports against RuntimePort/PortChannelOptions, composes the returned PortChannel with native Devframe RPC client/server channels, and checks the unchanged native client channel assignment. Each of five malformed consumers must emit exactly its intended compiler diagnostic, so an unrelated import failure cannot count as successful rejection. Exact complete test names are retained in Vitest's native JSON report.

WXT's declarations are Chrome-derived. This proves those installed consumer types; native Firefox execution is separate evidence. Other declaration sources and browser/version combinations remain unverified.

The five worker scenarios come from State scope and recovery: real Chromium worker termination, pending rejection/local unmount, both independent native server backends surviving, full explicit teardown after background loss, and fresh provider state without replay. They are linked to the successful production receipt, with zero captured page errors. Forced termination is not natural idle suspension or Firefox event-page evidence.

Validation: all nine WebExtension example tests (including six compiler cases), strict scoped lint/type/format checks and the seven conformance scanner/gate tests pass. The affected evidence groups were checked directly against fresh generated Vitest and native browser reports. Chromium production passes 52 scenarios, Firefox 51, and concurrent development suites pass 15 Chromium and 11 Firefox scenarios. Full CI passed workspace validation and all production browser suites, then failed in Firefox development while opening the native sidebar (toggleSidebar: button was null). The final combined evidence gate was skipped. The scoped test correction waits for the actual Firefox sidebar control before one native click; Firefox production and both concurrent development suites pass locally. Full CI passed at 8b8af184412f00522d1e545c17801337a88aae2b, including workspace build/types/strict lint/tests, all production browser suites, both concurrent development suites and the combined executed API-evidence gate.

The matrix and guide retain 29 groups with unfinished obligations. This checkpoint adds verified references and resolves one example ownership bug; it does not close this issue or establish release conformance.

Native capability evidence linked, 2026-10-01

Implementation e45f7c017d54ac5cf129e5cdbdfb91d37fc84a3e connects the existing capability-broadcast group to the new actual-browser proof in Provider discovery and routing. Four exact scenarios run on both Chromium and Firefox, separately from action broadcast. The inventory now names 39 distinct native scenarios across the same three generated receipts; API and unit-reference totals remain unchanged.

The new assertions verify independent values and provider identities across the three native hosts, realm/provider selection, unavailable-service outcomes and recovery, and successful siblings after server disconnection. The current unit references remain; the matrix explicitly records that their deduplication/cancellation assertions use actions. Browser unmatched-selector details, overlapping selectors, handler failures, cancellation, lifecycle races, development HMR and production-server preview remain gaps.

The changed group validates against both freshly generated native receipts and its four exact unit references from a fresh 31-test client run. The computed distinct native-scenario count is 39; JSON/whitespace checks pass. Existing scanner/evidence-gate tests and tooling lint/types passed earlier in this implementation session and their code has not changed. Full CI passed for the final pushed head, including workspace validation, native Chromium/Firefox production tests, concurrent development transitions and the combined API evidence gate.

The inventory still retains 29 groups with unresolved coverage. This is additional executed evidence, not release-conformance completion.

Native JSON action evidence linked, 2026-10-01

Implementation 4232360e59972b24ef17fd47bd291698cfdcfb1f links the 17 public portable action-call binding paths to maintained native browser assertions. The existing four unit references remain. The browser references now cover native rendering, outgoing realm/provider selection, provider-owned applicability, fulfilled not-applicable outcomes, successful siblings after partial failure, disconnection without rerouting, reconnect without replay, native input/success/error callbacks and retry-error recovery.

The inventory now names 31 distinct native scenarios across the same three generated receipts, up from 19. It still discovers 769 public paths and links 308 paths to 159 distinct executed unit tests. No export or executable checker changed. The browser references distinguish production-extension tests from those using native development-server backends. Firefox's lack of global page-error capture remains explicit.

The changed group was checked against fresh successful Chromium/Firefox production receipts, not committed snapshots. Scoped tooling TypeScript, strict Oxlint, seven scanner/evidence-gate tests and diff whitespace checks passed. Full repository CI passed for the final pushed head, including workspace checks, native production and concurrent development browser suites, and the combined API evidence gate.

The matrix and evidence guide retain the gaps: contract routing defaults, complete native built-ins, browser malformed/duplicate bindings, output-validation parity, the Firefox detached domain-action case and mixed-backend development/preview. The 29 groups with unresolved obligations remain. This increases traceability of executed behavior; it does not satisfy release conformance or close this issue.

Executable inventory and native browser checkpoint, 2026-10-01

Implementation 44e4a48 expands the compiler-discovered inventory to link 308 of 769 public API paths to 159 distinct executed tests. Runtime activation, admission, invocation and provider lifecycle and native Devframe exposure/action bindings now have exact references. The eight package entry points and discovered API set are unchanged. The matrix retains every remaining gap; the guide explains evidence scope.

The checker also verifies 19 exact scenarios from three generated native browser receipts after the renderer retry backport added two native-alert recovery regressions: Chromium and Firefox production Port composition/disconnection and action broadcast across extension and native development-server providers. It requires each scenario exactly once, a recorded browser version and empty captured page errors. Firefox's lack of global page-error capture remains explicit. Native action broadcast is separated from capability broadcast; the former does not establish the latter. Node MessageChannel or headless server tests do not count as browser/preview execution.

conformance:check leaves browser references not-checked; conformance:check --browser verifies generated receipts after successful current browser commands. CI runs that second gate after native browser suites and retains its report and receipts. conformance:release requires both evidence kinds and rejects unresolved gaps. Receipt modification time is recorded, but is not presented as a freshness guarantee. Historical committed evidence/ snapshots are not accepted by the browser checker.

Validation for this commit:

  • 79 runtime tests, six Devframe tests and seven scanner/evidence-gate tests passed, with scoped strict lint/type/format checks.
  • Fresh Chromium 153.0.8010.12 and Firefox 157.0 production runs each passed 39 scenarios; Chromium JSON actions passed 12. The browser evidence gate then passed against those freshly generated files.
  • With the JSON-action receipt removed, the browser gate failed rather than reusing historical evidence. Unit gate tests reject missing, duplicate, failed, malformed and page-error receipts.
  • Release conformance still fails on 29 explicit groups with unresolved obligations. These groups are finer classifications, not 29 newly introduced failures.

Full CI passed for this conformance commit. Earlier inventory/evidence commits 0adc44f and bacceed6 passed full CI. Type-only contracts, remaining core fields, native host/mode parity and still-unimplemented feature contracts keep this issue open. Chrome prompt confirmation is separately pending under Permissions and trust; it is not counted as passing evidence here.

Reconciled status, 2026-09-30

The example catalogue now includes real native server/Port composition, native JSON action routing, reference/custom rendering, Chromium/Firefox extension surfaces, and the recorded development reload cases. The extension example and renderer example own the commands and receipts. JSON routing and optional CDB interception ownership are settled.

The catalogue is still incomplete. Native server view-index composition now has maintained Chromium and Firefox example/tests in ef43f0f. Its remaining browser/HMR matrix, remaining permission/lifecycle cases, generic script/transform contracts and exhaustive public API/hook/mode coverage remain open. This reconciliation updates evidence and dependencies; it does not close the issue or claim release conformance.

Earlier checkpoints below are dated evidence. Their then-pending items are superseded by this status and later accepted resolutions; they do not reopen settled architecture choices.

Custom renderer coverage checkpoint

Commit d410852 supplies the required custom-renderer example through native local registration. The unchanged JSON counter runs in reference and framework-free DOM renderers against both actual server backends. Seven maintained Chromium groups per host cover action/state/error behavior, detached-view cleanup, remount, return to reference rendering, unsupported component failure/recovery and disconnect. Six package tests, strict local checks, browser-bundle exclusions and live acceptance pass. Full evidence · Receipt and scope. Full CI passed, including the new browser command and all maintained Chromium/Firefox and development checks. Mixed-provider JSON routing and interception coverage still need the pending owner decisions. No public SDK contract or new patch was added; the broader map remains open.

Maintained packaged-script reload proof

Commit a7ba2eb adds the script-edit scenario to both existing WXT development suites. It changes an imported source file only in the owned temporary fixture and lets WXT choose its native rebuild/reload path. There is no production adapter change, new patch, manual reload event or hidden re-registration.

Chromium 153.0.8010.12 and Firefox 156.0.1 both pass:

Stage Observed result
Register MAIN, then navigate Packaged script executes before the first page script.
Change imported script source WXT rebuilds the unlisted file and reloads the extension; old extension page closes and provider changes.
Observe the existing page Exact timeOrigin, first-script snapshot and old global remain; new revision is absent.
Read native registrations The owned persistAcrossSessions:false registration is gone.
Navigate without re-registering New document receives no bootstrap marker.
Explicitly re-register and navigate Next document receives the updated revision and the early-script snapshot.
Cleanup Registration, source page, native browser and owned fixture server are closed.

The Chromium runner waits for its independent CDP observer to report disconnection after native WXT stop. The initial immediate flag read raced that notification; the final check uses Playwright's bounded assertion, with no production cleanup mechanism.

Maintained recipe and limits, Chromium receipt, Firefox receipt.

Strict TS7, type-aware Oxlint and Oxfmt pass. Both full affected development suites pass locally, including their ordinary UI/module, HTML, background and configuration changes. Full CI passes.

This establishes native reload behavior for MAIN and temporary registrations, not in-place script HMR, automatic startup recovery, mutation rollback, browser-restart persistence or ISOLATED-world reload semantics. Shared script contribution design remains under #11, and broader #14 DoD stays open.

Popup observer reload ordering correction

Commit 68d758d corrects the test sequence exposed by CI 36553885590. The native HTML edit reloads both the actual toolbar popup and its observing options page. Polling popup text through page.evaluate before the options navigation finishes can reject with Execution context was destroyed.

The test now first waits for the options page's new heading through Playwright's navigation-aware locator, then reads the popup through that stable context. The existing 30-second reload allowance is retained. There is no browser retry controller, swallowed error or production behavior change.

Strict type-aware lint, TS7 and formatting pass. The actual concurrent Chromium/Firefox WXT suites pass locally, including the packaged script checks and popup HTML reload. Full CI passes on this correction plus the following issue11 Vite example.

Latest implementation checkpoint, 2026-09-29

The native integration is now installed in the workspace, with no upstream checkout required. Port binding 3fb91fd, RPC/state backport 5ebcb5a, renderer/view backport 631db2b, and maintained example 86b24b3 are separate ticket commits on main.

examples/webext consumes built public exports from the pinned, patched 1.0 dependency graph. Its real Chromium test passes 12 scenarios with zero page errors. The same test now runs in full SDK CI, which passes. The old temporary prototype is replaced by this maintained example.

The upstream review is split into draft RPC/state #410, JSON view/renderer #411, and independent baseline snapshot repair #412. The renderer declaration shim and implementation file moves are removed.

The maintained example has standard build/type/lint/test commands, a browser graph guard and a real-browser CI gate. Chrome types 0.3.0 and Playwright 1.63.0 were verified as current registry releases; the seven-day release policy remains unchanged. Native behavior is tested, while portable provider/catalog composition, complete browser surfaces, content/page scripts, debugger, Firefox and HMR still need their own examples and conformance evidence. The full API matrix gate remains open.


Native JSON renderer evidence, 2026-09-28

The renderer delivery now includes commit 1e45712 with native view/state/action integration on both server hosts, five automated tests and live two-tab mount/unmount/disconnect confirmation. Commands and limitations. Extension surfaces, cross-provider rendering and automated browser conformance remain open; no new state or rendering framework was introduced.

Native state lifetime follow-up, 2026-09-28

State scope and recovery now records the accepted native lifetime/write policy. The follow-up commit adds two real-host replacement tests, six native state tests total, and a replace command to the browser launcher. Live confirmation on both hosts verifies retained value, rejected old action binding, fresh attachment and continuing updates in an old native observer. The SDK adds no state API or enforcement layer. Extension and JSON-renderer cells remain open.

Native state example and live confirmation, 2026-09-27

Implementation commit adds native shared-state observation to the maintained browser example. It uses sharedState.get, state.value() and state.on('updated', ...) directly; no SDK state API or upstream patch is added. Commands still return typed results, shown separately from the observed state.

Four automated real-socket tests cover both native hosts: peer updates, separate host values under the same key, listener removal, fresh-client recovery, retained backend identity and native client writes. Live in-app browser checks on both hosts confirmed 0 → 1 in both tabs; a disconnected peer stayed at 1 while the active client reached 2; reload read 2 with the same incarnation; after closing the first tab, the remaining peer reached 3. Actual DevTools host shutdown marked the view stale and disabled actions. Detailed evidence and limits.

This proves explicit fresh-client recovery, not automatic reconnect, persistence, atomic snapshot/subscription ordering, JSON rendering or extension conformance. The native state object remains writable under the example's host policy.

Part of Design a portable contribution SDK and WebExtension runtime.

Latest native Vite host deliverable: 82a1265 runs the shared counter through both released native Vite plugins. Twelve real-host tests and full CI pass, covering HTTP/context startup, delayed cleanup/restart identity, failed close, shutdown before listening, real config watching and retained backend identity/state after client-module invalidation. Later maintained preview/watch delivery is recorded in issue 13, and simultaneous local routing in issue 7. The unpatched Vite cleanup gap is accepted; browser HMR and remote/renderer/browser-host conformance remain open.

Earlier headless native-context examples: 0873f1d reuses the same portable contracts through genuine Devframe hub and DevTools kit contexts, with executable built-export checks and strict TS7/Oxc/build validation. The API proof matrix records exactly this local scope; full Vite/renderer/browser-host coverage remains open.

Upstream integration boundary, 2026-09-27

Compose the same contribution fixtures with real upstream contexts and public exports. The inventory covers SDK-owned promises and adapter boundaries; native escape hatches need exposure/type/ownership tests, not exhaustive replicas of vendor test suites. Reuse scenarios rather than building example-only runtimes.

See the implementation and map review. This narrows implementation mechanisms without dropping the accepted feature or real-host coverage requirements.

Question

What runnable example catalogue and automated conformance system will prove that every public SDK API, lifecycle hook, and feature works through each supported host? Define the catalogue, coverage inventory, failure checks, and CI rules before treating the SDK as implementation-ready.

Context and current behavior

This is an explicit completion requirement: the monorepo must ship working examples of every piece, including contributions, a replaceable renderer, a standalone devframe host, a devtools host, and Chromium/Firefox extension hosts. Every public SDK API/hook/feature must be demonstrated and fully tested in its supported environments.

The original Vue template has been replaced on main by the strict Oxc workspace, portable core and local provider lifecycle. The first maintained UI-free contribution example and its clean packed-consumer proof are available in the implementation deliverable, commit 57f3381. These published commits do not establish the full catalogue, renderer or real server/browser host conformance. Existing neighboring examples remain evidence for their own integrations. The complete release must implement and execute the entire resulting coverage matrix.

The API inventory concerns SDK-owned contracts, wrappers, hooks, and promises. Raw browser/Vite/devframe escape hatches require tests of exposure, typing, ownership, availability, and disposal; this requirement does not mean reimplementing or exhaustively retesting every vendor API internals.

Current native browser example, 2026-09-27

dfef733 adds demo:browser devframe and demo:browser devtools through the shared router and built native adapter export. Manual browser checks on each host prove 0 → 1 and reload retaining value/incarnation. An automated Vite production-bundle check rejects Node and backend-runtime imports. Affected builds/types/lint/format, server/example integrations and 24 Vite-host tests pass. Full repository CI passed at dfef733: builds, types, strict lint, formatting, packed-consumer checks and tests.

This diagnostic page proves transport composition, not the JSON-renderer, extension-surface or complete HMR/coverage requirements below. Example commands and boundaries.

Requirements and scope

  • Maintain independent, runnable examples for contribution authoring, renderer replacement, devframe hosting, devtools hosting, and extension hosting. Examples may compose shared fixtures so the same contribution is visibly reused across hosts.
  • Include UI-only, action-only, state-only, transform-only, realm-specific, and combined contribution forms; all routing strategies; state scopes/recovery; injection/HTTP behavior; native debugger/optional CDB; and dev/watch-preview workflows.
  • Cover background, isolated content, page-world, popup, options, DevTools, and side-panel/sidebar lifetimes where supported.
  • Use public exports and built/packed dependencies in downstream-consumer proof. An example must not work only because a workspace path alias bypasses package exports.
  • Each example has prerequisites, exact start/build/test commands, expected visible behavior, browser setup, cleanup, and deterministic test entrypoints.
  • Every supported API/host/mode combination must have assertions of its specified behavior. Unsupported combinations must assert the declared unavailable result and reason. Permission denial and temporary disconnection are separate cases.
  • Tests include success, errors, cleanup, repeated mount/dispose, navigation, disconnect/reconnect, and persistence according to each API contract. Use real browsers for browser-owned behavior, while unit tests isolate pure logic at genuine module/I/O boundaries.
  • Follow repository conventions: exact expect.assertions(N) in Vitest, no coverage-ignore pragmas, no production DI added only for tests, package-scoped local validation, and full CI validation.

Options and recommendation

Compare a single large showcase with isolated examples, and a handwritten checklist with a machine-checked feature inventory. Recommend a small set of composable examples plus a versioned API-to-example-to-test matrix verified in CI. This keeps examples understandable while catching new APIs that have no executable proof.

Line/branch coverage can reveal missing cases, but cannot replace behavior and host coverage. The inventory must be checked against public exports, registries, hooks, and documented feature promises so adding an API cannot silently omit a matrix entry.

Proposed API or experiment

Illustrative inventory shape and verification algorithm; exact storage format is a decision in this ticket:

const coverageEntry = {
  api: 'actions.dispatch',
  scenarios: ['selected-provider', 'broadcast-partial-failure', 'disconnect'],
  examples: ['portable-contribution', 'multi-provider-host'],
  hostExpectations: {
    devframe: 'supported',
    devtools: 'supported',
    'chromium-extension': 'supported',
    'firefox-extension': 'supported',
  },
  requiredNegativeScenarios: ['permission-denied', 'disconnected-provider'],
  tests: ['action-contract', 'multi-provider-browser'],
}

for (const publicApi of collectPublicApiInventory()) {
  const entry = requireCoverageEntry(publicApi)
  requireRunnableExamples(entry)
  requireExecutableTests(entry)
  for (const hostMode of declaredHostModes(publicApi)) {
    assertEvidenceForExpectedBehavior(entry, hostMode)
  }
}

Tests must exercise outcomes, not merely verify that inventory strings or test files exist. The CI report should identify the API, scenario, host, mode, example, test run, and result. Build matrix expansion must be deliberate: an irrelevant host cell is explicitly classified, not silently skipped.

Scenarios and acceptance criteria

  1. Add a public lifecycle hook without an example/test mapping. The inventory check fails with its public identifier.
  2. Remove a mapped test or replace its behavior assertion with an unexecuted placeholder. CI cannot report the feature as covered; establish how test execution evidence is tied to scenario IDs.
  3. Run one contribution unchanged in standalone devframe, devtools, and both extension builds. Observe the same JSON view and action semantics through the real transport.
  4. Replace the reference renderer through public registration. Its mount/update/dispose contract works without importing renderer internals.
  5. On Firefox, a Chromium-only debugger capability returns its explicit unavailable contract; the test does not fake successful debugger support.
  6. Force worker suspension, close/reopen a surface, navigate the document, deny permission, and interrupt a provider during an action. The tests assert the documented state and cleanup after each transition.
  7. Install packed SDK packages into an isolated example consumer and run its supported commands without sibling checkout links.
  8. Start the watched production-build example, change a contribution, and verify the specified live-preview update/reload and retained state.

Dependencies

This decision defines the coverage machinery and catalogue categories early. Later API decisions populate its matrix before the Portable contribution proof and Release and conformance contract can finish. Do not create a dependency cycle by waiting for the full implementation here.

Definition of Ready

  • Each example lists the public upstream entry points and the SDK-specific behavior it proves.

  • Blockers provide an initial public API inventory, host/mode list, and feasible browser test tooling.

  • The working-example requirement and every required example category are included.

  • A small proposed inventory demonstrates both supported and unsupported operations.

  • The distinction between decision/prototype completion and complete SDK release coverage is explicit.

Definition of Done

  • One shared fixture proves equivalent contracts across hosts, and dependency upgrades rerun the affected native integration/type checks.

  • A concrete example catalogue assigns every SDK piece a runnable owner and reuse relationship.

  • The matrix schema, public-API discovery/checking method, and test-execution evidence format are specified.

  • CI failure rules catch unmapped public APIs/hooks/features, missing examples, unexecuted cases, and broken supported combinations.

  • Real-browser versus unit/contract test responsibilities are explicit and runtime limitations are accurately represented.

  • Each example has a required README/command/cleanup contract and a packed-consumer check where applicable.

  • The implementation completion gate requires every declared matrix cell to pass its expected outcome: successful execution for supported cells and explicit assertions for unsupported, denial, failure, and lifecycle scenarios. No placeholders, unexplained skips, or compile-only evidence count as completion.

  • The user has confirmed the catalogue and coverage standard, and downstream decisions reference it.

Resolution record

Record the catalogue, matrix schema, enforcement algorithm, representative populated entries, and release gate in the resolution comment. Subsequent implementation work must deliver and run these artifacts before claiming the SDK or its examples complete.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

wayfinder:grillingDecision requiring discussion with the project owner

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions