Skip to content

feat(sanitize): materialize an approved derivative from its archive - #418

Merged
abrichr merged 2 commits into
mainfrom
codex/materialize-approved
Aug 28, 2026
Merged

feat(sanitize): materialize an approved derivative from its archive#418
abrichr merged 2 commits into
mainfrom
codex/materialize-approved

Conversation

@abrichr

@abrichr abrichr commented Aug 28, 2026

Copy link
Copy Markdown
Member

The gap

approve_derivative computes the approval over the archive's own SHA-256, so _write_deterministic_archive skips APPROVAL_NAME. That is correct, and it has a consequence: an archive unzipped anywhere else is a valid derivative carrying no approval.

ARCHIVE members           : ['.openadapt-sanitization.json', 'events.jsonl', 'frames/before.png', 'meta.json']
CLOUD load_valid_approval : FAILED: Sanitized artifact has not been approved

openadapt-cloud's requalification action does exactly that (unzip recording.approved.zip -d .../recording), so its production deploy has failed since 2026-08-18. compile hides the same gap one step earlier: it sets source_recording_sha256 only if (recording / ".openadapt-approval.json").is_file(), so the runner built a bundle with null provenance and only validate-hosted complained.

What this adds

openadapt-flow materialize-approved \
  --archive recording.approved.zip \
  --approval recording.approval.json \
  --out recording/ \
  --expect-archive-sha256 <digest the caller already trusts>

It extracts the archive into a new empty directory, places the reviewer's existing approval record beside it, and runs load_valid_approval unchanged. Nothing is auto-approved, no check is skipped, and load_valid_approval is not relaxed.

--approval takes either shape a real deployment has:

  • the derivative's .openadapt-approval.json, used verbatim;
  • the openadapt.sanitization/v1 ingest envelope, which is what a cloud deployment actually persists. Every field carrying authority comes from that stored envelope: archive SHA-256, archive size, reviewer, approval time, approval method. The two tree hashes are functions of the extracted bytes that the archive SHA-256 already binds, so they are recomputed rather than transported, and the envelope is cross-checked against the extracted manifest on kind, policy version, execution semantics, and coverage. A derivative whose manifest records no approval_verification rescan is refused: it was never approved.

An envelope recording an automatic approval must carry its policy key id and MAC. The MAC is verified wherever OPENADAPT_SANITIZATION_POLICY_KEY is configured, reusing the exact canonicalization build_ingest_manifest signs (factored into _policy_signature_message, one definition now instead of two).

Extraction trusts no member name. Absolute paths, .. segments, backslashes, directory entries, symlinks, and any member named .openadapt-approval.json are refused before a byte is written. A failure leaves no approval file behind.

On signing

The approval record is unsigned today and stays unsigned here. That is a deliberate call, not an oversight, and the docs say so plainly.

Signing buys something when the approval and the artifact it approves travel on different channels, so the verifier does not already trust whoever hands it the bundle. That is not this shape. The record moves on the same channel as the archive, under the same access control, and the caller pins the archive digest from a reviewed source. Anyone who could forge the record could equally swap the archive and its pin, which defeats the gate whether or not a signature exists.

The available key would also be the wrong key. The workflow's reviewer is ${GITHUB_ACTOR}@github-actions, and a signing key held in Actions secrets attests "CI had the secret", not "a named person reviewed this". A signature there would invite more trust than the fact deserves.

What changes the answer: the moment a record is accepted from a lower-trust source (a customer upload, an ingest request body, a bucket with wider write access, another org's runner), or the moment the reviewer name becomes evidence shown to a third party rather than a description. The codebase already draws this line correctly for automatic approvals, which cross into cloud ingest and therefore carry a MAC.

Tests

tests/test_sanitized_artifact.py, committed before the implementation (15d8e2d then 40dbccf). At the test-only commit the module cannot import and no test runs; the implementation commit turns all of them green. They cover the reproduction, the round trip, the envelope path, compile provenance (asserting null before and bound after), a tampered archive, an unexpected digest, an approval for another artifact, a traversing member, a member carrying its own approval, a non-empty destination, a policy envelope missing its signature, a bad MAC, and the CLI.

Paired with openadapt-cloud PR (linked below). Cloud needs a Flow release carrying this command — 1.34.0 does not have it, and cloud must not pin a version that does not exist.

approve_derivative computes the approval over the archive's own SHA-256, so
_write_deterministic_archive must skip APPROVAL_NAME. Unzipping the archive
elsewhere therefore yields the reviewed bytes without the review:
load_valid_approval refuses the tree, and compile silently emits a bundle whose
source_recording_sha256 is null.

These tests state the contract a portable materialize path must satisfy. They
fail until that path exists.
Add `openadapt-flow materialize-approved --archive X --approval Y --out DIR`.
It extracts the archive into a new empty directory, places the reviewer's
existing approval record beside it, and then runs `load_valid_approval`
unchanged. It approves nothing and relaxes nothing: the record must already
exist and must match the extracted bytes.

`--approval` takes either the derivative's `.openadapt-approval.json` or the
`openadapt.sanitization/v1` ingest envelope a deployment persisted for that
artifact. The envelope carries every field that holds authority -- archive
SHA-256 and size, reviewer, approval time, approval method. The two tree hashes
are functions of the extracted bytes that the archive SHA-256 already binds, so
they are recomputed rather than transported, and a derivative whose manifest
records no approval rescan is refused outright. An envelope recording an
automatic approval must carry its policy key id and MAC; the MAC is verified
wherever OPENADAPT_SANITIZATION_POLICY_KEY is configured, reusing the same
canonicalization `build_ingest_manifest` signs.

Extraction trusts no member name: absolute paths, `..` segments, backslashes,
directory entries, symlinks, and any member named `.openadapt-approval.json`
are refused before a byte is written. `--expect-archive-sha256` pins the
archive against a digest the caller already trusts, independently of the
record. A failure leaves no approval file behind.

The record stays unsigned, exactly as it is on the reviewing machine, so it is
only as trustworthy as the channel that delivered it. That channel should be
the one that already delivers the archive.
@abrichr

abrichr commented Aug 28, 2026

Copy link
Copy Markdown
Member Author

Paired with OpenAdaptAI/openadapt-cloud#340, which wires the requalification action to this command. Flow merges and releases first.

@abrichr
abrichr merged commit 80b33d0 into main Aug 28, 2026
16 checks passed
@abrichr
abrichr deleted the codex/materialize-approved branch August 28, 2026 03:31
abrichr added a commit that referenced this pull request Aug 28, 2026
`feat(sanitize): materialize an approved derivative from its archive` merged to
main while this branch was in review. It is unreleased, so it belongs in
v1.34.0. Regenerated the entry from the merged history rather than appending by
hand, so it stays exactly what the generator produces.

Date the entry 2026-08-28, the day it lands.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
abrichr added a commit that referenced this pull request Aug 28, 2026
…detect the gap (#419)

* fix(release): restore changelog generation and backfill the v1.x gap

python-semantic-release 9.15.2 -> 10.6.1 landed on 2026-07-14 in a grouped
dependency bump (#87). Version 10 changed the default changelog mode from
"init" to "update". In "update" mode the generator inserts each new version
after `insertion_flag` and writes nothing at all when that flag is absent: it
exits 0, emits no warning, and leaves no diff. This file had been written
entirely under version 9, so it carried no flag.

The timing is exact. v0.26.0 was released at 17:25 UTC that day and is the last
entry the generator ever wrote. The bump merged at 17:44. v1.0.0 was released
at 17:47 with no entry, and every release through v1.33.0 did the same: six
weeks, 34 published versions, no changelog and no signal.

Restore generation by adding the insertion flag, and pin `mode` and
`insertion_flag` explicitly so the next major bump of the generator cannot
change them silently. The regression was caused by depending on a default.

Backfill v1.0.0 through v1.33.0 by re-running that same generator over the same
tagged commits. Nothing here is hand-written; the entries are the tool's own
output and re-running it reproduces them. A note at the top of the file records
that the range was reconstructed, on what date, and why, and states that
version 10's default template emits the commit summary line only, so the
reconstructed entries carry no commit bodies. `mode = "init"` would have
regenerated the whole file under that same template and discarded the bodies in
the v0.x entries, so the backfill was spliced in above them instead: all 30
v0.x entries are byte-identical to what was committed at the time.

Mark v1.13.0 and v1.14.0 as yanked on PyPI.

Add the v1.34.0 entry, which the release workflow's changelog gate requires.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(release-health): detect published versions that have no changelog entry

The changelog stopped being written on 2026-07-14 and nothing reported it for
six weeks. The release workflow does gate on a changelog entry, but that gate
was added on 2026-08-26, after all 34 of the affected releases, so its first
evaluation was also the first report of the problem, and it surfaced as a
blocked release rather than as a finding.

Comparing published tags against changelog headings would have alerted on
2026-07-14, on the next scheduled run after v1.0.0 shipped. This adds that
comparison as a fourth detector in the lane that already watches for unreleased
work and incomplete publishes.

It is state-based like its siblings: the alarm is "a published version is
undocumented", not "a generator run did something unexpected". That distinction
matters here because this repository has already moved once from
generator-written entries to entries prepared in a reviewed pull request, and
the detector has to survive that kind of change. It reuses the lane's existing
publish grace window so a tag minted moments ago is not called undocumented,
warns rather than alerts when the file cannot be read, stays silent for a lane
that declares no `changelog_file`, and matches the same heading shape the
release workflow greps for so the gate and the detector cannot disagree.

Tag dates come from the releases already fetched rather than one API call per
tag: a six-week gap is 34 tags, and the detector must not spend 34 requests to
notice it.

Verified in both directions against the live repository: it reports 52
undocumented versions against the pre-fix changelog and reports clean against
the restored one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(changelog): include #418 in the pending v1.34.0 entry

`feat(sanitize): materialize an approved derivative from its archive` merged to
main while this branch was in review. It is unreleased, so it belongs in
v1.34.0. Regenerated the entry from the merged history rather than appending by
hand, so it stays exactly what the generator produces.

Date the entry 2026-08-28, the day it lands.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant