Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,17 @@
# Changelog

## 0.10.0 — in development (not yet released)
## 0.10.0 — 2026-10-04

- Browser artifact streaming with caller-owned file/OPFS sinks and distinct
received versus stored verification.
- Opt-in selected-file uploads with bounded private staging, current plugin/write
authority, chunk/full hashes, explicit restart recovery and conditional commit.
- Caller-owned mixed transfer queue with chunk scheduling, connection/resource/rate
limits, pause/resume, bounded transient chunk retries and distinct verified progress.
- Worker OPFS checkpoints with full stored identity, fresh host authority, independent
prefix rehash, metadata crash recovery and exclusive ownership across tabs.
- Native browser examples cover reload, host restart, cross-tab conflict and recovery
after owner-tab termination. Other browser/storage environments require qualification.
- Keep project, text-file, job and artifact v1 contracts and released 0.3–0.9
plugin package metadata verification. See [uploads](docs/UPLOADS.md).

Expand Down
45 changes: 27 additions & 18 deletions PUBLIC_SURFACE.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,53 +2,54 @@
"schemaVersion": 1,
"description": "Explicitly reviewed files for the public DDS Plugin SDK package. Changes require publication and rights review.",
"packageFiles": [
"bin/dds-plugin.mjs",
"CHANGELOG.md",
"CONTRIBUTING.md",
"LICENSE",
"NOTICE",
"PUBLIC_SURFACE.json",
"README.md",
"SECURITY.md",
"bin/dds-plugin.mjs",
"docs/api.md",
"docs/ARTIFACT_STORAGE.md",
"docs/BINARY_ARTIFACTS.md",
"docs/BROWSER_ARTIFACTS.md",
"docs/BROWSER_RESUME.md",
"docs/COMPATIBILITY.md",
"docs/CONSENT.md",
"docs/DEVTOOLS.md",
"docs/FILE_REVISIONS.md",
"docs/JOBS.md",
"docs/host-contract.md",
"docs/JOB_HISTORY.md",
"docs/JOB_STORAGE.md",
"docs/JOBS.md",
"docs/LANGUAGE.md",
"docs/LICENSING.md",
"docs/OBSERVATION.md",
"docs/PROJECTS.md",
"docs/performance.md",
"docs/PROJECT_QUERIES.md",
"docs/PROJECT_TOOLS.md",
"docs/PROJECT_WATCH.md",
"docs/PROJECTS.md",
"docs/PUBLIC_SCOPE.md",
"docs/PUBLISHING.md",
"docs/testing.md",
"docs/THEMES.md",
"docs/TRANSFER_QUEUE.md",
"docs/UPLOADS.md",
"docs/VERIFYING.md",
"docs/WORKSPACES.md",
"docs/api.md",
"docs/host-contract.md",
"docs/performance.md",
"docs/testing.md",
"docs/workspace-protocol.md",
"docs/WORKSPACES.md",
"examples/binary-artifacts/run.mjs",
"examples/browser-artifacts/browser.html",
"examples/browser-artifacts/browser.mjs",
"examples/browser-artifacts/host.mjs",
"examples/browser-artifacts/run.mjs",
"examples/browser-artifacts/serve.mjs",
"examples/browser-artifacts/worker.mjs",
"examples/browser-resume/browser.html",
"examples/browser-resume/browser.mjs",
"examples/browser-resume/host.mjs",
"examples/browser-resume/serve.mjs",
"examples/browser-resume/worker.mjs",
"examples/command-jobs/run.mjs",
"examples/consent/README.md",
"examples/consent/prompt.mjs",
"examples/consent/README.md",
"examples/durable-jobs/run.mjs",
"examples/hello-diagnostics/plugin.mjs",
"examples/hello-diagnostics/run.mjs",
Expand All @@ -66,13 +67,13 @@
"examples/project-tools/run.mjs",
"examples/project-tools/serve.mjs",
"examples/project-watch/run.mjs",
"examples/publishable-plugin/LICENSE",
"examples/publishable-plugin/NOTICE",
"examples/publishable-plugin/README.md",
"examples/publishable-plugin/dds-package.json",
"examples/publishable-plugin/disclosure.json",
"examples/publishable-plugin/LICENSE",
"examples/publishable-plugin/NOTICE",
"examples/publishable-plugin/plugin.json",
"examples/publishable-plugin/plugin.mjs",
"examples/publishable-plugin/README.md",
"examples/release-verification/README.md",
"examples/release-verification/run.mjs",
"examples/stored-artifacts/run.mjs",
Expand All @@ -85,15 +86,19 @@
"examples/uploads/browser.mjs",
"examples/uploads/run.mjs",
"examples/uploads/serve.mjs",
"examples/user-workspace/LICENSE",
"examples/user-workspace/client.mjs",
"examples/user-workspace/LICENSE",
"examples/user-workspace/plugin.mjs",
"examples/user-workspace/private-manifest.json",
"examples/user-workspace/project/design.sv",
"examples/user-workspace/server.mjs",
"examples/user-workspace/tool.mjs",
"examples/workspace-observation/run.mjs",
"LICENSE",
"NOTICE",
"package.json",
"PUBLIC_SURFACE.json",
"README.md",
"schemas/artifact-sink-capabilities.schema.json",
"schemas/artifact-storage-capabilities.schema.json",
"schemas/artifact-transfer-receipt.schema.json",
Expand All @@ -102,6 +107,7 @@
"schemas/binary-artifact-list.schema.json",
"schemas/binary-artifact-reference.schema.json",
"schemas/binary-artifact.schema.json",
"schemas/browser-artifact-checkpoint.schema.json",
"schemas/command.schema.json",
"schemas/diagnostics-request.schema.json",
"schemas/diagnostics-result.schema.json",
Expand Down Expand Up @@ -146,8 +152,11 @@
"schemas/workspace-file-revision.schema.json",
"schemas/workspace-project-capabilities.schema.json",
"schemas/workspace-project-poll.schema.json",
"SECURITY.md",
"src/artifact-download-node.d.mts",
"src/artifact-download-node.mjs",
"src/artifact-resume-browser.d.mts",
"src/artifact-resume-browser.mjs",
"src/artifact-storage-node.d.mts",
"src/artifact-storage-node.mjs",
"src/artifact-storage-session.mjs",
Expand Down
16 changes: 8 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,21 +10,21 @@ a small ESM package, TypeScript declarations and versioned contracts. Connect
your own workspace and tools with the Node.js host and HTTP client. The SDK has
**zero runtime dependencies**.

The next 0.10 line is in development: [browser artifact streaming](docs/BROWSER_ARTIFACTS.md)
adds bounded sinks, incremental whole-file hashes, explicit storage readback and
main-thread/Worker examples and [selected-file uploads](docs/UPLOADS.md) with private
The 0.10 line adds [browser artifact streaming](docs/BROWSER_ARTIFACTS.md)
with bounded sinks, incremental whole-file hashes, explicit storage readback and
main-thread/Worker examples, and [selected-file uploads](docs/UPLOADS.md) with private
staging, explicit restart recovery and verified conditional commit. A caller-owned
[transfer queue](docs/TRANSFER_QUEUE.md) schedules mixed uploads/downloads with
chunk fairness, pause/resume, rate budgets and bounded safe retries. The checkout
uses 0.10.0 for protocol development; these additions are not in the published
0.9 archive.
chunk fairness, pause/resume, rate budgets and bounded safe retries.
[Worker OPFS checkpoints](docs/BROWSER_RESUME.md) allow explicit recovery after
reload or host restart, with prefix hashes and cross-tab exclusive ownership.

This is the intentionally public SDK for external plugin developers. The DDS
application, private engines/services and production signing systems are outside
this repository. Read the [public scope](docs/PUBLIC_SCOPE.md) and
[compatibility table](docs/COMPATIBILITY.md) for exact boundaries.

Version **0.9.0** supports Node 22 and 24. Manifest v1 diagnostic plugins remain
Version **0.10.0** supports Node 22 and 24. Manifest v1 diagnostic plugins remain
compatible. Manifest v2 adds commands, permission-scoped workspace and backend
APIs, and open-source or proprietary distribution metadata. Themes export to
the XML format already supported by DDS. The workspace host runs trusted plugins
Expand Down Expand Up @@ -84,7 +84,7 @@ mkdir my-dds-plugin
cd my-dds-plugin
npm init -y
npm pkg set type=module
npm install --ignore-scripts --save-exact https://github.com/Altifigence/dds-plugin-sdk/releases/download/v0.9.0/altifigence-dds-plugin-sdk-0.9.0.tgz
npm install --ignore-scripts --save-exact https://github.com/Altifigence/dds-plugin-sdk/releases/download/v0.10.0/altifigence-dds-plugin-sdk-0.10.0.tgz
```

Copy the example's `plugin.mjs` and `run.mjs` into this directory, then run:
Expand Down
21 changes: 11 additions & 10 deletions docs/BROWSER_ARTIFACTS.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Browser artifact streaming (0.10 development)
# Browser artifact streaming (0.10)

The public 0.9 archive does not contain these APIs yet. This foundation is part of
the next 0.10 release. Existing Node file downloads remain available unchanged.
These APIs are included in 0.10. Existing Node file downloads remain available.
For reload-safe downloads, use the [Worker checkpoint sink](BROWSER_RESUME.md).

`/artifact-transfer` imports no Node built-ins. `streamJobBinaryArtifact()` uses
the existing 0.7 binary protocol; `streamStoredJobArtifact()` uses the 0.8 snapshot
Expand Down Expand Up @@ -54,8 +54,9 @@ event and returned receipt set `verified: true`. Callback exceptions fail the
operation, even if commit has already happened.

The trusted caller sink contract advertises `seek`, `readback`, `persistence`
(`none`, `on-commit`, `per-checkpoint`) and abort behavior. The current core writes
sequentially from zero; advertising seek or checkpoints does not enable resume.
(`none`, `on-commit`, `per-checkpoint`) and abort behavior. Explicit `resume: true`
requires seek, readback and per-checkpoint persistence plus a session `recovery`
reader. The core independently verifies its prefix before requesting remaining bytes.
Implementations must bound retained data and cooperate with cancellation. The
helper waits for in-flight writes before cleanup so late writes are not detached.

Expand All @@ -72,8 +73,8 @@ staging stream on pre-commit failure. They do not provide a persistent partial
download after a reload. Normal intervening file changes are checked using size
and last-modified values; browser APIs provide no atomic compare-and-swap against
external programs. Reservations prevent concurrent use of the same entry only
within this module instance/realm. Cross-tab persistent recovery is a separate
0.10 integration step and is not claimed by this foundation.
within this module instance/realm. Use the separate [checkpoint adapter](BROWSER_RESUME.md)
for persistent recovery and cross-tab exclusion.

## Examples, support and resource evidence

Expand Down Expand Up @@ -124,10 +125,10 @@ Semantics follow the [File System Standard](https://fs.spec.whatwg.org/) and the

## 한국어 요약

0.10 개발 단계 기능이다. 호출자가 선택한 파일/OPFS 디렉터리에만 저장하며
0.10에 포함된 기능이다. 호출자가 선택한 파일/OPFS 디렉터리에만 저장하며
권한을 자동 요청하거나 다른 위치로 우회하지 않는다. 청크별 검증·저장을
기다린 뒤 다음 청크를 읽고, 수신 전체 해시와 저장 파일 재읽기 해시를 구분한다.
재읽기 없는 sink는 디스크 검증을 주장하지 않는다. 전체 바이트를 받았다는
진행률만으로 완료 처리하지 않는다. 취소·용량 초과·권한 철회·해시 오류에는
부분 상태와 확정 여부를 반환한다. 이 foundation의 비동기 저장은 새로고침 후
부분 파일 재개를 제공하지 않으며 영속 복구·탭 간 독점은 뒤의 0.10 통합 범위다.
부분 상태와 확정 여부를 반환한다. 비동기 저장은 확정 전에 중단되면 staging을
버리며, 새로고침 후 재개·탭 간 독점에는 별도 Worker checkpoint adapter를 쓴다.
129 changes: 129 additions & 0 deletions docs/BROWSER_RESUME.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
# Browser checkpoint recovery (0.10)

`/artifact-resume-browser` supplies a persistent OPFS sink for **stored artifacts**.
Use it in a dedicated Worker with a directory and UUID explicitly selected by the
application. A new or reconnected client must first obtain the same retained
snapshot with current host authority. The SDK never restores tokens, grants,
connections, commands, pickers or a previous host generation from local storage.

```js
// Inside an application-owned dedicated Worker:
import {createOpfsArtifactSink} from '@altifigence/dds-plugin-sdk/artifact-resume-browser';
import {streamStoredJobArtifact} from '@altifigence/dds-plugin-sdk/artifact-transfer';

await client.connect();
const reference = await client.getStoredJobArtifact(pluginId, jobId, artifactId, pluginPin);
const sink = createOpfsArtifactSink({directory: selectedDirectory, partialId, reference});
const receipt = await streamStoredJobArtifact(client, reference, {
sink,
resume: true, // false/omitted creates a new partial; an existing ID is a conflict
signal: controller.signal,
});
showVerifiedHash(receipt.storedSha256);
```

The same options work with `queue.enqueueStoredDownload(client, reference, options)`.
The application selects its own directory and partial ID again after reload; the
sink does not enumerate origin storage or guess a previous download. The example
remembers only those two selections. An application must decide its own consent,
retention, storage persistence requests and deletion UI.

## Checkpoint format and ordering

`getOpfsArtifactPartialNames(partialId)` returns exactly two names:
`dds-partial-<UUID>.data` and `dds-partial-<UUID>.json`. Creation refuses an existing
data or metadata entry. Explicit recovery requires both entries; eviction or
deletion never causes a silent replacement download.
An interrupted first creation can leave an empty or orphan entry. Explicitly
inspect and remove that selected fixture, or choose a new ID; it is never reused
as an implicitly empty download.

The data file uses an exclusive `FileSystemSyncAccessHandle`. Each accepted chunk
is at most 64 KiB and is flushed before the small checksummed metadata is saved.
Metadata uses a separate writable stream, closed before acknowledging the chunk.
It records full immutable store/workspace/plugin-pin/job/snapshot/artifact
identity, timestamps, state and current/previous prefix offsets and hashes. It
contains no token, grants or old connection scope. `parseBrowserArtifactCheckpoint`
validates this bounded format; the JSON schema describes its structural contract.
The checksum detects accidental damage, not malicious changes by origin scripts.

On recovery the sink rehashes the recorded prefix in bounded slices. If the file
has extra uncheckpointed bytes, it truncates them only after that prefix matches.
If the current checkpoint is beyond the actual file size, the previous checkpoint
is considered only when its complete bytes exist and its hash matches. A corrupt
prefix, missing file, mismatched identity or invalid metadata fails without
resetting the download. The host's current reference must match the complete saved
snapshot; a new host generation is permitted through a fresh connection and grant.

The transfer helper independently rehashes the recovery prefix before asking for
remaining network bytes. Its final hash covers prefix plus new bytes, and its
stored verification re-reads the complete saved file. Recovering an already
complete partial still validates the host's EOF and rechecks saved bytes.
`resumedBytes` counts verified local prefix bytes; `receivedBytes` counts only this
attempt's network bytes. `verifying-prefix` is distinct from receive/commit/readback.
Only the returned verified receipt means that the whole transfer completed.

## Ownership, cancellation and bounds

An origin-wide exclusive Web Lock named `dds-artifact-partial:<UUID>` is requested
with `ifAvailable`. A competing tab receives `conflict`; no waiting queue, lock
stealing or takeover is attempted. The native data handle is also exclusive. Both
remain held through final readback, and cancellation closes them while retaining
the last checkpoint. Worker/tab termination releases native ownership; a later
explicit recovery validates the files again. This is cooperative same-origin
coordination, not an isolation boundary against other code executing in the origin.

Four open sink sessions per module instance, 1 GiB per file, 64 KiB per data
operation and 16 KiB per metadata record are hard maxima. The default local
retention is one hour; callers may choose up to 24 hours, capped by the retained
host snapshot's expiry. Every operation rechecks expiry. The transfer's separate
30-minute deadline includes queue wait and pause. Paused sessions retain locks,
and the fifth open checkpoint sink is rejected until a session closes.

There is no total origin-disk quota manager. Browser quota/eviction policy still
applies and can remove data. A data write or metadata-close failure poisons that
session; explicitly reopen to inspect whichever old/new checkpoint actually
persisted. Cancellation is not rollback. Cleanup belongs to the caller after
closing its session; no error path deletes unknown files or expired partials.

## Runtime and evidence

`getBrowserArtifactResumeSupport()` reports dedicated Worker, sync-access and Web
Locks presence. Native main-thread use is unsupported. Structural directory and
lock ports are trusted implementations useful for deterministic contract tests;
they do not establish native browser acceptance.

Run `npm run example:browser-resume`. The disposable host captures a 2 MiB result
and deletes its original source. In the page:

1. Create a checkpoint and stop; reload, restart the host, then resume.
2. Remove that completed fixture, create a new partial and hold its lock.
3. Open the same URL in a second tab and try recovery: it must report conflict.
4. Close the owning tab; retry recovery in the second tab and verify the full hash.
5. Remove the selected fixture files and stop the host.

Windows embedded Chromium 154 was exercised for checkpoint cancellation, page
reload, host restart, remaining-byte recovery, two-tab contention and owner-tab
termination. Other browsers and power-loss durability are not qualified by these
checks. Deterministic tests cover quota failures, metadata-close ambiguity,
truncated or damaged data, missing entries, expiry and identity changes; real HTTP
tests also cover fresh authority, revoked grants and source deletion.
The 2,097,169-byte reload/restart case reused 65,536 bytes and received 2,031,633
bytes in 32 chunks, with matching full SHA-256 in 3.67 seconds. The owner-tab
termination case took 4.31 seconds after selecting recovery. These are local
fixture observations, not portable latency guarantees.

The storage primitives follow the [File System Standard](https://fs.spec.whatwg.org/)
and [Web Locks specification](https://w3c.github.io/web-locks/). Writable metadata
becomes visible at close; sync data writes operate in place. These API semantics
do not promise operating-system power-loss durability or permanent browser storage.

## 한국어 요약

전용 Worker에서 사용자가 선택한 OPFS 디렉터리와 partial UUID로 중단된
다운로드를 재개한다. 새 연결이 현재 권한으로 얻은 저장 결과 참조와 로컬
메타데이터의 전체 identity를 비교하고, 실제 prefix를 다시 해시한 뒤 남은
바이트만 받는다. 손상·삭제·만료·다른 결과는 자동 초기화 없이 거부한다.
동일 partial의 다른 탭은 conflict를 받고, 소유 탭 종료 뒤 명시적으로 다시
복구한다. 취소는 검증된 체크포인트를 보관하며 token이나 이전 grants를
저장하지 않는다. 전체 수신·저장 재검증이 끝나야 완료로 표시한다.
Loading