Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 31 additions & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -210,6 +210,28 @@ jobs:
name: npm-tarball
path: .

- name: What the registry will be asked to trust
# The claims npm compares with the trusted publisher configured on npmjs.com, printed so that
# a refusal can be read against them. Only the claims: the token itself is a credential and
# never leaves this step. Measured on 27 September: 0.1.0-rc.1 was refused twice with
# nothing in this log but ENEEDAUTH, because npm reports why an exchange failed only at
# verbose level. The cause was that the package had no trust configuration at all (`npm trust
# list`: "No trust configurations found"); the website had not saved it.
run: |
set -euo pipefail
SDE_ID_TOKEN=$(curl -sSf -H "Authorization: Bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" \
"$ACTIONS_ID_TOKEN_REQUEST_URL&audience=npm:registry.npmjs.org" \
| python3 -c 'import json, sys; print(json.load(sys.stdin)["value"])')
export SDE_ID_TOKEN
python3 - <<'CLAIMS'
import base64, json, os
payload = os.environ["SDE_ID_TOKEN"].split(".")[1]
claims = json.loads(base64.urlsafe_b64decode(payload + "=" * (-len(payload) % 4)))
for key in ("repository", "repository_owner", "workflow_ref", "job_workflow_ref",
"environment", "ref", "event_name", "runner_environment"):
print(f"{key}: {claims.get(key)}")
CLAIMS

- name: Publish
# No NODE_AUTH_TOKEN and no .npmrc. Trusted publishing generates the provenance attestation
# by itself, so `--provenance` is not passed: npm does it and it cannot be forgotten. The tag
Expand All @@ -220,7 +242,15 @@ jobs:
set -euo pipefail
test -n "$DIST_TAG"
tarball=$(ls ./*.tgz)
npm publish "$tarball" --access public --tag "$DIST_TAG"
npm publish "$tarball" --access public --tag "$DIST_TAG" || {
# The OIDC exchange never throws: npm logs why it got no token into its debug log, at
# verbose level, and the console shows only ENEEDAUTH. Those lines carry the registry's
# message and no credential.
echo "::group::npm's own account of the OIDC exchange"
grep -h "oidc" "$HOME"/.npm/_logs/*-debug-0.log || true
echo "::endgroup::"
exit 1
}

- name: It is actually there, under its tag
env:
Expand Down
6 changes: 3 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ not make them agree. What does is the conformance suite and

## `smart-data-engine-sdk` 0.1.0rc1 and `@smart-data-engines/sde` 0.1.0-rc.1

These are the first release candidates published through the release workflow. They cover
everything since `0.1.0.dev0`, the development release that claimed the PyPI name on 12 September
2026.
Published on 27 September 2026. These are the first release candidates published through the
release workflow, with attestations on PyPI and provenance on npm. They cover everything since
`0.1.0.dev0`, the development release that claimed the PyPI name on 12 September 2026.

**Runtimes.** Python 3.11 to 3.14 and Node 18 to 26, every version tested in CI
([`docs/platforms.md`](docs/platforms.md)).
Expand Down
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,8 +107,9 @@ artifacts under mixed traffic and checks scheduled latency, native recovery and
[Session lifetime](docs/session-lifecycle.md) defines borrowed/owned connections and transaction
ownership for concurrent applications.
The [Weather starter](docs/weather-starter.md) provides a local setup, restricted runtime clients in
Python/TypeScript, measured telemetry, operator handoffs and an ownership-checked reset. It is
unreleased; the runbook uses built artifacts and distinguishes this demo from production qualification.
Python/TypeScript, measured telemetry, operator handoffs and an ownership-checked reset. It ships in the
first release candidates, `smart-data-engine-sdk` 0.1.0rc1 and `@smart-data-engines/sde` 0.1.0-rc.1,
and the runbook distinguishes this demo from production qualification.


What does not exist yet is a library in any other language: `java/` and `rust/` are the next two and
Expand Down
12 changes: 7 additions & 5 deletions docs/implementations.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,11 +76,13 @@ The tier in the table above is checked against the library's own `TIER` constant
list that says one thing while the code says another is the failure requirement 17.6 exists to
prevent, and prose does not fail.

**`pip install smart-data-engine-sdk` installs this library**, published to PyPI on 12 September
2026. `npm install @smart-data-engines/sde` does not install anything yet: the scope is ours, held by
the `smart-data-engines` organisation from the same day, so that name cannot become somebody else's
package — but on npm a scope is a reservation and the package under it is a separate act, and the
first publish there will come from CI with provenance rather than from a laptop.
**`pip install smart-data-engine-sdk` installs this library.** The name was claimed on PyPI on
12 September 2026 with a development release. `0.1.0rc1` followed on 27 September, the first release
candidate, published by the release workflow with attestations. **`npm install @smart-data-engines/sde`
installs `0.1.0-rc.1`**, published by the same workflow the same day, with provenance. The one version
before it, `0.1.0-dev.0`, was published by hand. That publish was forced: npm configures trusted
publishing only on a package that already exists, so the first publish cannot come from CI
([`publishing.md`](publishing.md) §5.3).

The suffix is not decoration. `smart-data-engine` is **refused** by PyPI as "too similar to an
existing project" — `smartdata-engine`, registered by somebody else with no releases — so the
Expand Down
41 changes: 31 additions & 10 deletions docs/publishing.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,8 +112,9 @@ Verified on 8 September 2026: `python -m build` succeeds and `twine check` passe
## 1. npm — the scope is ours ✅ (12 September 2026)

**Done.** The `smart-data-engines` organisation exists, `krzysztof-smartdataengines` owns it, and
`npm org ls smart-data-engines` is what says so rather than a screenshot. Nothing is published under
the scope and nothing should be — see step 4. Ten minutes, as estimated.
`npm org ls smart-data-engines` is what says so rather than a screenshot. Two versions are
published under the scope. `0.1.0-dev.0` was published by hand, as the bootstrap that trusted
publishing needs. `0.1.0-rc.1` came from the release workflow (§5.5). Ten minutes, as estimated.

The steps are kept below rather than deleted, because the next scope this organisation reserves
follows exactly this path and the two warnings in it are the part worth having again.
Expand Down Expand Up @@ -523,13 +524,20 @@ rotate, nothing to forget to delete. That matters more here than the convenience
account-scoped PyPI token publishes to every project that account owns, forever, and section 2.4
exists because the alternative was keeping one.

**Nothing has been released through it yet, and that is the first thing to know before trusting it.**
As of 12 September 2026 this repository has **zero tags** and this workflow has **never run** —
`gh run list --workflow=release.yml` returns nothing. Every refusal in section 5.2 is covered by a
test or by `check_contexts.py`, and nineteen mutations were used to show each one goes red when it
should. What is *not* covered is the half only a real run exercises: whether PyPI accepts the OIDC
token, whether the environment gate actually pauses, whether `npm publish` of a prebuilt tarball
behaves the way its documentation says.
**It first ran on 27 September 2026, and three things surfaced that the gate's tests could not
see.** Until then every refusal in section 5.2 was covered by a test or by `check_contexts.py`, and
nineteen mutations showed each one going red when it should. The half that talks to a registry was
covered by nothing:
- **npm 11 refuses a prerelease without `--tag`.** This was found before the run, by reading the
CLI's source, and fixed in #93.
- **The registry caches the package document for five minutes**, which outlived the one-minute
check after publishing. Fixed in #95.
- **A trust configuration that does not exist fails as `ENEEDAUTH`**, with the registry's reason
only in npm's verbose log. The publishing job now prints the claims it presents and npm's own
account of the exchange.

`python-v0.1.0rc1` published to PyPI on its first run. `typescript-v0.1.0-rc.1` published on its third
(§5.5).

**It cannot be rehearsed, and that is a property of the design rather than an omission.** The only
trigger is a tag push; tags are immutable under the ruleset; neither registry reuses a version
Expand Down Expand Up @@ -680,7 +688,7 @@ client would actually pin is attested** — the gap lands on the one release nob

### 5.5 The first release through the workflow: `0.1.0rc1` and `0.1.0-rc.1`

These are release candidates, and that is deliberate. The pipeline has never run, and a candidate is
These are release candidates, and that is deliberate. The pipeline had never run, and a candidate is
the number this section says to spend on the first run (§5). Do the steps in this order, because each
one needs the one before it.

Expand Down Expand Up @@ -716,6 +724,19 @@ one needs the one before it.

Then approve each deployment: Actions → the release run → Review deployments. On npm the
candidate becomes `latest`, because no final version exists yet (§5.2).

**What happened on 27 September.**
- `python-v0.1.0rc1`, on `ebc89be`, published to PyPI with attestations on its first run.
- `typescript-v0.1.0-rc.1`, on `1ec940f`, was refused twice at the trusted-publishing exchange
with only `ENEEDAUTH` in the log. The package had no trust configuration at all:
`npm trust list` answered "No trust configurations found", after the website had appeared to
save it twice.
- `npm trust github ... --allow-publish --yes`, run in an interactive session with 2FA, created
the configuration. The third run then published with provenance. Re-running the failed job
was enough; the version number was never spent.

**Read the configuration back before tagging:**
`npx npm@11.15.0 trust list @smart-data-engines/sde`.
5. **Verification from the registries**, which is ours. A clean environment installs
`smart-data-engine-sdk==0.1.0rc1` from PyPI and `@smart-data-engines/sde@0.1.0-rc.1` from npm. It
runs the shared conformance vectors against the installed packages and the Weather starter end to
Expand Down
21 changes: 14 additions & 7 deletions docs/weather-starter.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,19 +4,26 @@ This is a local, synthetic demonstration for PostgreSQL and ClickHouse. Python p
setup and runs the existing SDK operator. Python and TypeScript applications use independent
runtime connections. They read `state/active-map.json`; a running controller is not required.

The starter is unreleased: the published Python development version predates this command, and
npm has no published package yet. Use reviewed build artifacts. Do not publish a release merely
to try the demo. [Publishing](publishing.md) describes the separate release procedure.
The starter ships in the first release candidates, `smart-data-engine-sdk` 0.1.0rc1 on PyPI and
`@smart-data-engines/sde` 0.1.0-rc.1 on npm. Do not publish a release merely to try the demo.
[Publishing](publishing.md) describes the separate release procedure.

## Install artifacts and obtain trusted metadata
## Install the packages and obtain trusted metadata

Use Python 3.11-3.14, Node 18-26 and a local POSIX filesystem. An operator supplies the reviewed
wheel and npm tarball. In fresh application directories:
Use Python 3.11-3.14, Node 18-26 and a local POSIX filesystem. In fresh application directories,
from the registries, pinned:

```sh
python3 -m venv .venv
.venv/bin/python -m pip install './smart_data_engine_sdk-0.1.0rc1-py3-none-any.whl[signed,postgres,clickhouse]'
.venv/bin/python -m pip install 'smart-data-engine-sdk[signed,postgres,clickhouse]==0.1.0rc1'
npm init -y
npm install @smart-data-engines/sde@0.1.0-rc.1 pg
```

Or from reviewed build artifacts, which an operator supplies for a commit after the candidates:

```sh
.venv/bin/python -m pip install './smart_data_engine_sdk-0.1.0rc1-py3-none-any.whl[signed,postgres,clickhouse]'
npm install ./smart-data-engines-sde-0.1.0-rc.1.tgz pg
```

Expand Down
5 changes: 3 additions & 2 deletions examples/weather/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,9 @@ The starter is shipped in the Python wheel (`sde-weather`) and npm tarball (`sde
It writes and reads synthetic weather observations through the logical SDK API. The controller
supplies signed metadata; it never runs this data application or receives its credentials.

This starter is **unreleased**. The existing PyPI development release predates it, and the npm
package has not been published. Use the reviewed wheel/npm artifacts supplied for the demo.
The starter is in the first release candidates: `smart-data-engine-sdk` 0.1.0rc1 on PyPI and
`@smart-data-engines/sde` 0.1.0-rc.1 on npm. Pin those versions, or use reviewed artifacts built from
a later commit.
The [runbook](../../docs/weather-starter.md) covers setup, both clients, telemetry, local operator
handoffs, recovery and reset. [model.json](model.json) is the model to declare in the controller;
[generator.json](generator.json) pins an exact synthetic example in both languages.
Expand Down
22 changes: 22 additions & 0 deletions python/tests/test_release.py
Original file line number Diff line number Diff line change
Expand Up @@ -486,3 +486,25 @@ def test_the_registry_check_waits_longer_than_the_registry_caches() -> None:
pause = re.search(r"sleep (\d+)", loop.group(2))
assert pause is not None
assert int(loop.group(1)) * int(pause.group(1)) > 300 + 60


def test_the_publishing_job_says_what_it_presented_and_why_npm_refused() -> None:
"""ENEEDAUTH alone is not a reason, and 0.1.0-rc.1 was refused twice with nothing else.

npm's OIDC helper never throws: it logs why the exchange produced no token at verbose level,
into its debug log, and the console shows only that no credential exists. So the job prints the
claims the registry compares with the trusted publisher, and on a failed publish the `oidc`
lines of npm's debug log - never the token itself.
"""
workflow = (ROOT / ".github" / "workflows" / "release.yml").read_text(encoding="utf-8")
publish = workflow.split(" publish-npm:", 1)[1]
claims = publish.split("What the registry will be asked to trust", 1)[1]
claims = claims.split("- name: Publish", 1)[0]
for claim in ("repository", "repository_owner", "workflow_ref", "environment"):
assert f'"{claim}"' in claims, claim
assert "audience=npm:registry.npmjs.org" in claims
assert 'print(f"{key}: {claims.get(key)}")' in claims
# The token is decoded, never printed.
assert "SDE_ID_TOKEN" in claims
assert not re.search(r"(echo|print)\W+\$?\{?SDE_ID_TOKEN", claims)
assert 'grep -h "oidc" "$HOME"/.npm/_logs/*-debug-0.log' in publish
Loading