From 704b5215712d7835a2f9542d9a3fe381c35cf5da Mon Sep 17 00:00:00 2001 From: Krzysztof Macewicz Date: Sun, 27 Sep 2026 23:06:18 +0200 Subject: [PATCH 1/2] docs: the documents describe the 0.1.0 release; main moves to 0.1.1.dev0 0.1.0 is on both registries. The install commands pin it, and the pages that still spoke of candidates, of an unpublished npm package or of a publish workflow yet to be written now say what happened on 27 September. main moves to 0.1.1.dev0 and 0.1.1-dev.0. Otherwise every later commit builds an artefact carrying the released number, and pip treats a local install of one as the release: measured with a wheel of f4d57c4 plus one comment, `pip install smart-data-engine-sdk==0.1.0` answered "Requirement already satisfied" and the comment stayed installed. The test that holds the security document's count of required checks matched only "N required checks", so "Eleven required status checks" stayed wrong from twelve checks to fourteen. It now matches every order the document uses, and went red on the stale sentence before the sentence was fixed. Co-Authored-By: Claude Opus 5.5 (1M context) --- CHANGELOG.md | 3 ++ README.md | 4 +- docs/github-security.md | 21 +++++----- docs/implementations.md | 11 +++--- docs/publishing.md | 76 +++++++++++++++++++++++------------- docs/weather-starter.md | 19 ++++----- examples/weather/README.md | 6 +-- python/src/sde/__init__.py | 2 +- python/tests/test_release.py | 7 +++- typescript/package-lock.json | 4 +- typescript/package.json | 2 +- 11 files changed, 94 insertions(+), 61 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5f9bf72..b1ac3e8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,9 @@ not make them agree. What does is the conformance suite and ## `smart-data-engine-sdk` 0.1.0 and `@smart-data-engines/sde` 0.1.0 +Published on 27 September 2026, each tag on its first run of the release workflow, with attestations +on PyPI and provenance on npm. On npm it is `latest`. + The first final release. The libraries are the release candidates below: apart from the version itself, nothing in `python/src`, `typescript/src` or `typescript/bin` changed after `python-v0.1.0rc1` and `typescript-v0.1.0-rc.1`. diff --git a/README.md b/README.md index 11d3ea3..ba08d68 100644 --- a/README.md +++ b/README.md @@ -108,8 +108,8 @@ artifacts under mixed traffic and checks scheduled latency, native recovery and ownership for concurrent applications. The [Weather starter](docs/weather-starter.md) provides a local setup, restricted runtime clients in Python/TypeScript, measured telemetry, operator handoffs and an ownership-checked reset. It ships in the -first release candidates, `smart-data-engine-sdk` 0.1.0rc1 and `@smart-data-engines/sde` 0.1.0-rc.1, -and the runbook distinguishes this demo from production qualification. +first release, `smart-data-engine-sdk` 0.1.0 and `@smart-data-engines/sde` 0.1.0, and the runbook +distinguishes this demo from production qualification. What does not exist yet is a library in any other language: `java/` and `rust/` are the next two and diff --git a/docs/github-security.md b/docs/github-security.md index 4f4e834..54c9c69 100644 --- a/docs/github-security.md +++ b/docs/github-security.md @@ -307,8 +307,8 @@ for it: so `--provenance` is deliberately *not* passed — a flag is something a future edit can drop in silence. PyPI attestations come the same way. The one gap is named in `publishing.md` §5.4: the first npm publish has to happen by hand, because npm requires a package to exist before a trusted - publisher can be configured for it, so `0.1.0-dev.0` will carry no attestation and every version a - client would pin will. + publisher can be configured for it, so `0.1.0-dev.0` carries no attestation and every version a + client would pin does: `0.1.0-rc.1` and `0.1.0` were published with provenance. - **Two GitHub Environments with a required reviewer** ✅, not a repository secret — and each one is locked to its own tag pattern, so nothing but a `python-v*` tag can ask to use the PyPI one. A merge cannot become a publish without a person. @@ -439,7 +439,7 @@ malicious or broken code onto `main` and it reaches a release, (b) a published a forged, either through a leaked registry token or through the distribution name never having been registered, (c) a dependency is compromised and lands in a client's application through our extras, (d) a credential from a client engagement is committed by accident, (e) a leaked maintainer token is -used to rewrite history or publish a fake release. Eleven required status checks on a protected branch +used to rewrite history or publish a fake release. Fourteen required status checks on a protected branch with no bypass actors handle (a); trusted publishing with provenance, an environment with a reviewer, tag protection and — first of all — **registering the names** handle (b); Dependabot with a committed lockfile handles (c); secret scanning with push protection handles (d); 2FA, signed commits and tag @@ -451,7 +451,8 @@ distribution name is the one an attacker needs no access at all to exploit. ## Checklist ``` -✅ ci.yml: lint, mypy --strict, tests on 3 Pythons against a real PostgreSQL, tsc + vitest on 3 Nodes +✅ ci.yml: lint, mypy --strict, tests on every supported Python, tsc + vitest on every supported Node, + against a real PostgreSQL and ClickHouse (the versions: docs/platforms.md) ✅ ci.yml: contract job — PEP 561 marker in the wheel, openssl-verified digests, frozen vector ✅ codeql.yml — python, javascript-typescript, actions; default query suite on purpose ✅ dependabot.yml — actions, pip (python/), npm (typescript/); codeql-action grouped @@ -463,19 +464,21 @@ distribution name is the one an attacker needs no access at all to exploit. ✅ rulesets kept as JSON in .github/rulesets/ ✅ branch ruleset on main: PR required, no force push, no deletion, linear history, no bypass actors ✅ branch ruleset: all fourteen status checks required, strict -✅ tag ruleset on refs/tags/v* +✅ tag ruleset on refs/tags/v*, python-v* and typescript-v* ✅ secret scanning + push protection ✅ Dependabot alerts + security updates ✅ private vulnerability reporting ✅ Actions: read-only default token, cannot approve PRs ✅ Actions: fork PR approval required for all external contributors ✅ merge commits off — squash and rebase only, consistent with required_linear_history -✅ smart-data-engine-sdk on PyPI, published 12 September 2026 -✅ @smart-data-engines scope on npm, held by the organisation since 12 September 2026 +✅ smart-data-engine-sdk on PyPI, claimed 12 September 2026, 0.1.0 released 27 September 2026 +✅ @smart-data-engines scope on npm, held by the organisation since 12 September 2026; + @smart-data-engines/sde 0.1.0 released 27 September 2026 ✅ organisation defaults for new repositories: scanning, push protection, Dependabot, dep graph +✅ registry accounts with 2FA: PyPI requires it, and npm asked for the second factor on 27 September +✅ release.yml: OIDC trusted publishing, attestations and provenance, an environment with a reviewer + per registry; first run 27 September 2026 ⚙️ org-wide 2FA on Smart-Data-Engines — UI only, the API reports success and changes nothing -⚙️ registry accounts with 2FA, before the first publish ⚙️ SSH/GPG signing key registered as a *signing* key, then required_signatures in the ruleset ⚙️ non-provider secret patterns + validity checks (organisation-level Secret Protection) -⚙️ when a publish workflow is written: OIDC trusted publishing, provenance, environment with reviewer ``` diff --git a/docs/implementations.md b/docs/implementations.md index 9696748..6a05501 100644 --- a/docs/implementations.md +++ b/docs/implementations.md @@ -76,11 +76,12 @@ The tier in the table above is checked against the library's own `TIER` constant list that says one thing while the code says another is the failure requirement 17.6 exists to prevent, and prose does not fail. -**`pip install smart-data-engine-sdk` installs this library.** The name was claimed on PyPI on -12 September 2026 with a development release. `0.1.0rc1` followed on 27 September, the first release -candidate, published by the release workflow with attestations. **`npm install @smart-data-engines/sde` -installs `0.1.0-rc.1`**, published by the same workflow the same day, with provenance. The one version -before it, `0.1.0-dev.0`, was published by hand. That publish was forced: npm configures trusted +**`pip install smart-data-engine-sdk` installs `0.1.0`**, the first release, published by the release +workflow on 27 September 2026 with attestations. The name was claimed on PyPI on 12 September with a +development release, and the release candidate `0.1.0rc1` came through the same workflow earlier on +27 September. **`npm install @smart-data-engines/sde` installs `0.1.0`**, published by the same +workflow the same day, with provenance, after the candidate `0.1.0-rc.1`. The one version before +those, `0.1.0-dev.0`, was published by hand. That publish was forced: npm configures trusted publishing only on a package that already exists, so the first publish cannot come from CI ([`publishing.md`](publishing.md) §5.3). diff --git a/docs/publishing.md b/docs/publishing.md index 089436f..1a34279 100644 --- a/docs/publishing.md +++ b/docs/publishing.md @@ -11,7 +11,7 @@ what decides which ones are worth a visit today and which are worth nothing unti | Name | Registry | State | How it gets claimed | |---|---|---|---| | `smart-data-engine-sdk` | PyPI | **ours since 12 September 2026** | the upload that claimed it | -| `@smart-data-engines/sde` | npm | **scope ours since 12 September 2026**, package not yet published | creating the organisation granted the scope | +| `@smart-data-engines/sde` | npm | **scope ours since 12 September 2026**, package published since 27 September 2026 | creating the organisation granted the scope | | `com.smartdataengines` | Maven Central | **ours since 12 September 2026** | a DNS TXT record; nothing published, ever | | `smart-data-engine` | PyPI | **refused** | too similar to `smartdata-engine`; see §2.0 | | `sde` | PyPI | taken by someone else | — which is why the distribution and the import differ | @@ -63,8 +63,9 @@ the moment something has already failed, to run `pip install 'smart-data-engine- worst possible moment to point somebody at a name a stranger controls: they are debugging, they will copy the command, and the instruction came from inside code they had already decided to trust. -Today the name resolves to nothing and the command simply fails, which is safe. It stops being safe -the moment anybody else registers it, and nothing warns us when that happens. +Until 12 September 2026 the name resolved to nothing and the command simply failed, which was safe. +It would have stopped being safe the moment anybody else registered it, and nothing would have warned +us. The upload in §2 closed that: the command now installs this library. ## What cannot be undone @@ -112,9 +113,10 @@ Verified on 8 September 2026: `python -m build` succeeds and `twine check` passe ## 1. npm — the scope is ours ✅ (12 September 2026) **Done.** The `smart-data-engines` organisation exists, `krzysztof-smartdataengines` owns it, and -`npm org ls smart-data-engines` is what says so rather than a screenshot. Two versions are +`npm org ls smart-data-engines` is what says so rather than a screenshot. Three versions are published under the scope. `0.1.0-dev.0` was published by hand, as the bootstrap that trusted -publishing needs. `0.1.0-rc.1` came from the release workflow (§5.5). Ten minutes, as estimated. +publishing needs. `0.1.0-rc.1` and `0.1.0` came from the release workflow (§5.5). Ten minutes, as +estimated. The steps are kept below rather than deleted, because the next scope this organisation reserves follows exactly this path and the two warnings in it are the part worth having again. @@ -152,23 +154,24 @@ else can publish anything under `@smart-data-engines/`. **If the name is taken, stop and tell me** — six files cite the scope (`typescript/package.json`, its lockfile, `typescript/README.md`, and three documents), and they would all have to change together. That is a code change, not a rename. -4. That is the whole reservation. Do not publish. The library is at `0.1.0-dev.0`, the first publish - should carry provenance from CI, and publishing by hand would spend that version number to prove - something the scope already guarantees. +4. That is the whole reservation. Do not publish to hold the name: the scope already guarantees it, + and a publish spends a version number to prove it. The first real publish still has to be by + hand, because npm configures trusted publishing only on a package that exists (§5.3, item 2). -**Nothing else on npm is urgent, and the distinction that makes that true is worth keeping straight: -the scope is ours and the package is unpublished, which are two different facts.** `npm install -@smart-data-engines/sde` still installs nothing — and cannot install somebody else's package either, -which is the whole protection the reservation buys. `REGISTRIES` in `_claims.py` therefore reads the -npm entry as registered while the package does not exist, and that is correct: the flag is about who -owns the name, not about whether anything has been shipped under it. +**The scope being ours and the package being published are two different facts, and for fifteen +days only the first was true.** From 12 to 27 September `npm install @smart-data-engines/sde` +installed nothing — and could not install somebody else's package either, which is the whole +protection the reservation buys. `REGISTRIES` in `_claims.py` read the npm entry as registered while +the package did not exist, and that was correct: the flag is about who owns the name, not about +whether anything has been shipped under it. ## 2. PyPI — published ✅ (12 September 2026) **Done.** `smart-data-engine-sdk` `0.1.0.dev0` is on PyPI, and what says so is `pip install smart-data-engine-sdk` in an empty virtualenv rather than the page rendering: it imports, the wheel carries `licenses/LICENSE`, `licenses/NOTICE` and `py.typed`, and -`[signed,postgres]` resolves `cryptography` and `psycopg` from the real index. +`[signed,postgres]` resolves `cryptography` and `psycopg` from the real index. `0.1.0rc1` and +`0.1.0` followed on 27 September, from the release workflow with attestations (§5.5). It took two attempts and the first one is §2.0, which is the part of this section worth reading. The steps are kept below because the second distribution this repository publishes walks the same @@ -537,7 +540,8 @@ covered by nothing: account of the exchange. `python-v0.1.0rc1` published to PyPI on its first run. `typescript-v0.1.0-rc.1` published on its third -(§5.5). +(§5.5). The final release, `python-v0.1.0` and `typescript-v0.1.0`, published on the first run of +each, the same evening. **It cannot be rehearsed, and that is a property of the design rather than an omission.** The only trigger is a tag push; tags are immutable under the ruleset; neither registry reuses a version @@ -560,6 +564,13 @@ Then approve the deployment on GitHub. The publish job waits on an environment w reviewer, so a merge cannot become a publish without a person — and the environments are scoped to their own tag pattern, so nothing but a `python-v*` tag can even ask to use the PyPI one. +**After the release, `main` moves to a development version**, in its own pull request: `0.1.1.dev0` +and `0.1.1-dev.0` followed `0.1.0`. Otherwise every later commit builds an artefact carrying the +released number, and pip treats a local install of one as the release: `pip install +smart-data-engine-sdk==0.1.0` answers "Requirement already satisfied" and fetches nothing. Measured +on 27 September with a wheel of `f4d57c4` plus one comment: the comment stayed installed. The next +release bumps from there to whatever number it takes. + **The tags are per-language, and the reason is not tidiness.** One shared tag would publish an artefact byte-identical to its predecessor, with an empty changelog, every time the *other* language moved — at a version number neither registry ever hands back. What makes the two libraries agree is @@ -597,9 +608,9 @@ that the registry shows the version under it. reported absent. A checker stuck on "present" would find every required file and report a flawless package, which is the shape of good news worth distrusting. -### 5.3 Three things still need you +### 5.3 Three things that needed the owner ✅ (27 September 2026) -1. **PyPI: add the trusted publisher** ⚙️ — . +1. **PyPI: add the trusted publisher** ✅ — . Four fields, and all four must match exactly or the token is refused: | Field | Value | @@ -612,7 +623,7 @@ package, which is the shape of good news worth distrusting. The environment field is optional at PyPI and is filled in deliberately: with it, a token minted by any other job in this repository is rejected at the registry rather than trusted. -2. **npm: the first publish has to be by hand** ⚙️, and this is npm's constraint rather than a +2. **npm: the first publish has to be by hand** ✅, and this is npm's constraint rather than a shortcut. Trusted publishing there is configured on a package's own settings page, and `npm trust` (npm ≥ 11.15.0) says the same thing in its documentation: "The package you're configuring must already exist on the npm registry." So the sequence is fixed: @@ -657,12 +668,12 @@ package, which is the shape of good news worth distrusting. **Do not put a token in an Actions secret to avoid this.** It would buy one attestation and leave behind a credential that publishes under our scope for as long as nobody remembers it is there. - **And then add the npm link to the landing page.** The product page links the PyPI package and the - repository from its "Read the Code" section and deliberately does *not* link npm, because a 404 is - worse than an absence. The file is + **And then add the npm link to the landing page** ✅. Until the package existed, the product page + linked the PyPI package and the repository from its "Read the Code" section and deliberately did + *not* link npm, because a 404 is worse than an absence. The file is `smart-data-engine-landing-page/frontend/smart-data-engine/index.html`, the block is - `