From 70a094574675e2720ee38972464feae7f3331704 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Fri, 24 Jul 2026 16:33:52 +0100 Subject: [PATCH 01/26] docs(artifacts): reconcile submission bundle plan --- .agent-loop/REVIEW_LOG.md | 19 +- .../AUTH_HANDOFF.md | 66 +++++++ .../CHUNK_MAP.md | 55 ++++-- .../DECISIONS.md | 104 +++++++++-- .../INTENT.md | 34 +++- .../PLAN.md | 121 +++++++------ .../RISKS.md | 13 +- .../STATUS.md | 102 +++++------ .../WS-ART-001-03-guide-source-cutover.md | 10 +- ...WS-ART-001-03A-guide-source-byte-ingest.md | 73 ++++++++ ...3B-guide-source-binding-materialization.md | 72 ++++++++ ...1-03C-guide-source-cutover-continuation.md | 75 ++++++++ ...S-ART-001-04A-upload-inspection-sealing.md | 142 ++++++++------- .../WS-ART-001-04B-pre-submit-admission.md | 136 +++++++------- ...4C-verified-submission-bundle-admission.md | 105 +++++++++++ ...-ART-001-05-submission-artifact-cutover.md | 128 +++++-------- ...T-001-06A-checker-input-materialization.md | 13 +- .../WS-ART-001-07-recovery-live-proof.md | 14 +- ...-PLAN2-submission-bundle-reconciliation.md | 122 +++++++++++++ ...-ART-001-PLAN2-internal-review-evidence.md | 55 ++++++ .../WS-ART-001-PLAN2-pr-trust-bundle.md | 56 ++++++ .../merge-intents/WS-ART-001-PLAN2.json | 9 + docs/glossary.md | 34 ++-- docs/spec_artifact_storage_service.md | 168 ++++++++---------- docs/spec_authorization_service.md | 13 +- docs/template_submission_artifact_policy.md | 4 +- docs/template_submission_packet.md | 19 +- scripts/test_agent_gates.py | 59 +++--- 28 files changed, 1295 insertions(+), 526 deletions(-) create mode 100644 .agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/AUTH_HANDOFF.md create mode 100644 .agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03A-guide-source-byte-ingest.md create mode 100644 .agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B-guide-source-binding-materialization.md create mode 100644 .agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03C-guide-source-cutover-continuation.md create mode 100644 .agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04C-verified-submission-bundle-admission.md create mode 100644 .agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-PLAN2-submission-bundle-reconciliation.md create mode 100644 .agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-internal-review-evidence.md create mode 100644 .agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-pr-trust-bundle.md create mode 100644 .agent-loop/merge-intents/WS-ART-001-PLAN2.json diff --git a/.agent-loop/REVIEW_LOG.md b/.agent-loop/REVIEW_LOG.md index 37b8596b..d3489e22 100644 --- a/.agent-loop/REVIEW_LOG.md +++ b/.agent-loop/REVIEW_LOG.md @@ -2844,7 +2844,6 @@ inline finding. Its unexplained 37.61 percent docstring warning is non-actionable because the unchanged repository gate independently passes the same head at 87.6 percent against the 80 percent floor. Human review and explicit approval of PR #178 remain. - ## 2026-07-23 - WS-AUTH-001-10C Internal Implementation Review AUTH-10C exact implementation SHA @@ -2883,3 +2882,21 @@ retryability. Full GitHub shards then exposed and closed a stale linked-event matrix classification plus legacy-fixture trigger/constraint cleanup residue; the exact isolated audit case and sequential upgrade/downgrade matrices pass. fresh GitHub Backend, Agent Gates, CodeRabbit, and human review remain. + +# WS-ART-001-PLAN2 Submission Bundle Reconciliation + +- Signed automation run `30100940860` cancelled the rejected combined + `WS-ART-001-03` implementation before runtime edits. +- Human intent locks one outer ZIP, bounded process-local scratch through every + pre-submit gate, no candidate retention/provider namespace, conservative + existing limits, existing recovery reuse, and full integrity recomputation on + downstream byte streams. +- Initial planning review failed on stale ART history, incomplete guide chunks, + duplicate upload-session/artifact-set contracts, partial scratch surfaces, + pre-submit evidence privacy, nested-archive ambiguity, AUTH sequencing, and + failing governance/coverage/e2e gates. +- The repaired candidate splits 03A-C and 04A-C/05, reconciles canonical specs + and templates, defines the exact AUTH handoff, preserves existing + scratch/admission/recovery abstractions, and passes all deterministic gates. +- Final senior, architecture, QA/test, security/auth, product/ops, reuse/dedup, + CI integrity, test-delta, and docs reviews pass with no open finding. diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/AUTH_HANDOFF.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/AUTH_HANDOFF.md new file mode 100644 index 00000000..157e445e --- /dev/null +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/AUTH_HANDOFF.md @@ -0,0 +1,66 @@ +# WS-ART-001 Authorization Handoff + +ART owns hidden artifact behavior, canonical product resource facts, lifecycle +guards, surface manifests, and feature tests. AUTH owns ActionId/PermissionId +catalogues, service identities, fixed matrices, evaluator integration, grants, +activation custody, and availability. + +## Guide Source Sequence + +1. Existing guide-source actions remain planned and unavailable. +2. ART-03A implements hidden `artifact.guide_source.ingest` behavior and its + exact resource/guard/surface manifest. +3. AUTH activates only that exact action through a separately reviewed AUTH + contract after consuming ART evidence. +4. ART-03B implements hidden `artifact.guide_source.read` and + `artifact.guide_source.binding.create` behavior; binding maps to fixed + permission `artifact.binding.create`. +5. AUTH activates only those exact actions after consuming 03B evidence. +6. ART-03C performs the legacy clean cut. No ART chunk writes availability. + +## Submission Bundle Sequence + +Before ART-04A starts, AUTH must merge a separately reviewed registration +contract, provisionally named +`WS-AUTH-001-ART-SUBMISSION-BUNDLE-REGISTRATION`, that: + +- registers planned ActionId `artifact.submission_bundle.prepare`; +- maps it only to existing human PermissionId `submission.create`; +- limits candidates to the assigned contributor for the exact task/project; +- names ART-owned canonical facts for actor, identity link, project, task, + active assignment, locked policy context, and operation generation; +- keeps the action unavailable and adds no grant or evaluator activation; +- records parity evidence in AUTH's closed catalogue/constraint/owner manifests; +- explicitly retires the unused planned multi-step upload-session ActionIds or + proves they are unavailable and have no route/command manifest entry. + +Until that AUTH contract merges, current agent-gate catalogue assertions retain +those strings only as an exact planned/unavailable discovery baseline. Their +presence in the closed catalogue is not an active design, grant, route, or +permission to implement a second intake path, and PLAN2 does not edit AUTH-owned +catalogue or parity assertions. + +ART-04A through 04C then implement one hidden continuous surface and publish +its exact route/resource/guard manifest. After 04C, a separate reviewed AUTH +activation contract may integrate the evaluator and change only +`artifact.submission_bundle.prepare` to active. ART-05 cannot start until that +activation merges. + +The continuous contributor action never implies the fixed service actions: + +- `artifact.checker_input.materialize`; +- `artifact.verification.execute`; +- `artifact.pending_work.scan`; +- `artifact.put_attempt.resolve`; +- `artifact.binding.create`. + +Each fixed service action retains its canonical provisioned service identity, +matrix row, resource facts, terminal reauthorization, and separate activation +evidence. No human grant supplies fixed service authority. + +## Fail-Closed Rule + +An ART implementation contract stops if its required AUTH registration or +activation contract is absent, unmerged, inactive, differently mapped, or +targets a different resource fact shape. Planned catalogue presence, a local +action string, or hidden feature code is never executable authority. diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/CHUNK_MAP.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/CHUNK_MAP.md index b90c0b63..1272658d 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/CHUNK_MAP.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/CHUNK_MAP.md @@ -12,13 +12,18 @@ Each chunk is one PR. No later chunk starts automatically. | `WS-ART-001-02A3` | Replace ArtifactStore v1 with byte-only v2, activate API-startup and Celery Beat scratch cleanup, migrate schema/callers/factory, and remove `flow_node` in one atomic clean cut. | L1 | Merged through PR #141 as `a10d901` on 2026-07-18 | | `WS-ART-001-02B1` | Implement the S3-compatible adapter, MinIO integration, and AWS S3 production profile. | L1 | Merged through PR #151 as `1b5422fc` on 2026-07-19 | | `WS-ART-001-02C1` | Add the generic durable-byte admission ledger and durable put-attempt state foundation without provider execution. | L1 | Merged through PR #154 as `44f2467c` on 2026-07-19 | -| `WS-ART-001-02C2` | Add put resolution, verification publication, complete-object observation, immutable receipts, and PostgreSQL execution fencing without recovery attempts or routes. | L1 | Active after PR #154 and explicit user start on 2026-07-19 | -| `WS-ART-001-02C3` | Add the recovery-attempt model and exact idempotent source-job to retry-job chain without public or Operator routes. | L1 | Proposed after 02C2 | -| `WS-ART-001-02D` | Add hidden Operator content/job/retry/recovery/audit APIs, canonical resource composition, and production-readiness checks while actions and provider profiles remain inactive. | L1 | Proposed after 02C3, AUTH-09E, and `WS-AUTH-001-ART-CUSTODY` | -| `WS-ART-001-03` | Store and bind guide-source bytes; add same-snapshot setup recovery through the authorized artifact reader. | L1 | Proposed after 02D | -| `WS-ART-001-04A` | Add task-scoped upload sessions/items, trusted archive inspection, independent verification, immutable sealing, and artifact-set manifests. | L1 | Proposed after 03 | -| `WS-ART-001-04B` | Execute authoritative pre-submit against sealed artifact sets and persist exact admissions with bounded infrastructure continuation. | L1 | Proposed after 04A | -| `WS-ART-001-05` | Atomically bind admitted artifact sets to submissions and remove legacy URI/hash/finalization contracts. | L1 | Proposed after 04B | +| `WS-ART-001-02C2` | Add put resolution, verification publication, complete-object observation, immutable receipts, and PostgreSQL execution fencing without recovery attempts or routes. | L1 | Merged through PR #159 as `bc5e6a42` | +| `WS-ART-001-02C3` | Add the recovery-attempt model and exact idempotent source-job to retry-job chain without public or Operator routes. | L1 | Merged through PR #174 as `92b8a7aa` | +| `WS-ART-001-02D` | Add hidden Operator content/job/retry/recovery/audit APIs, canonical resource composition, and production-readiness checks while actions and provider profiles remain inactive. | L1 | Merged through PR #177 as `93c14181` | +| `WS-ART-001-03` | Original combined guide-source cutover. | L1 | Cancelled before implementation; no runtime changes | +| `WS-ART-001-PLAN2` | Reconcile guide and one-ZIP submission planning with bounded scratch, existing immutable admission/recovery, exact AUTH sequencing, and downstream ownership. | L1 | Planning-only successor proposed after cancellation | +| `WS-ART-001-03A` | Add hidden guide-source byte ingest through existing preparation, admission, verification, and publication. | L1 | Proposed after PLAN2 | +| `WS-ART-001-03B` | Bind verified guide-source content and provide authorized integrity-checking setup materialization. | L1 | Proposed after 03A and exact AUTH activation | +| `WS-ART-001-03C` | Remove legacy guide-source identity and add exact same-generation setup continuation. | L1 | Proposed after 03B and exact AUTH activation | +| `WS-ART-001-04A` | Accept one outer ZIP in bounded scratch, safely inspect its tree, produce canonical identities, and reject unchanged work before provider I/O. | L1 | Proposed after 03C and AUTH planned action registration | +| `WS-ART-001-04B` | Run mandatory platform and locked Project Guide pre-submit checks against the same scratch-bound tree without durable storage. | L1 | Proposed after 04A | +| `WS-ART-001-04C` | Admit the passing ZIP once through existing ArtifactStore, independently verify it, and publish one bindable admission. | L1 | Proposed after 04B; AUTH activation follows hidden completion | +| `WS-ART-001-05` | Atomically bind one verified admission to one immutable Submission and remove legacy caller transport authority. | L1 | Proposed after 04C and exact AUTH activation | | `WS-ART-001-06A` | Persist checker input snapshots and materialize authorized immutable bytes into bounded checker workspaces. | L1 | Proposed after 05 | | `WS-ART-001-06B` | Ingest checker logs/outputs as artifacts, persist checker completion facts, and preserve existing checker-owned routing without creating review aggregates. | L1 | Proposed after 06A | | `WS-ART-001-07` | Prove Local/MinIO plus AWS S3 readiness, Operator recovery, and exact-byte guide/pre/post-submit behavior through real APIs. | L1 | Proposed after 06B | @@ -35,9 +40,17 @@ OBJECT-STORAGE-AMENDMENT -> 02C2 put resolution, verification publication, and fencing -> 02C3 recovery attempt and idempotency chain -> 02D Operator and production readiness --> 03 guide source cutover --> 04A upload/inspection/sealing --> 04B pre-submit admission and outage continuation +-> PLAN2 planning reconciliation +-> 03A guide-source byte ingest +-> AUTH activation for exact 03A actions +-> 03B guide-source binding/materialization +-> AUTH activation for exact 03B actions +-> 03C guide-source clean cut/continuation +-> AUTH planned registration of `artifact.submission_bundle.prepare` +-> 04A one-ZIP scratch intake/inspection/manifest/change gate +-> 04B scratch-bound platform/project pre-submit checks +-> 04C one-time immutable admission/verification +-> AUTH activation of exact complete contributor surface -> 05 submission cutover -> 06A checker input/materialization -> 06B checker output/post-submit routing @@ -48,10 +61,14 @@ OBJECT-STORAGE-AMENDMENT deferred. It has no active chunk, runtime profile, credential service, or configuration value in v0.1. `ReviewPacketManifest` and `ReviewEvidenceArtifact` remain owned by WS-REV. -Physical deletion and semantic search require separate approved initiatives. +Physical deletion, temporary provider retention, candidate object storage, and +semantic search require separate approved initiatives. ## Cross-Initiative Handoffs +The exact authorization sequence and stop conditions are recorded in +`AUTH_HANDOFF.md`. + - Artifact actions follow AUTH planned registration -> hidden ART behavior and canonical resource composition -> AUTH evaluator integration and activation. `WS-AUTH-001-ART-CUSTODY` first transfers the 25 current ART actions to eight @@ -64,7 +81,10 @@ Physical deletion and semantic search require separate approved initiatives. - WS-REV owns `ReviewPacketManifest` and `ReviewEvidenceArtifact`. Review code receives verified Workstream `ArtifactBinding` IDs through a narrow review-facing capability; it must not receive provider references, scratch - paths, or concrete adapters. + paths, or concrete adapters. REV also owns reviewer decisions and + note/findings for + the exact `Submission`; `needs_revision` authorizes a later contributor ZIP + but contains no reviewer-uploaded artifact. - A future optional contribution-evidence projection requires separately approved ART-owned read/write capabilities and AUTH action activation. Core ContributionRecord creation makes no ART capability/provider call and is not @@ -73,8 +93,17 @@ Physical deletion and semantic search require separate approved initiatives. - Cross-initiative terminology must use ART's canonical `resource_type`, `resource_id`, and `logical_role`, or define an explicit integration mapping; product initiatives must not create a second binding vocabulary implicitly. +- The existing immutable `Submission` row is the version aggregate. TASK/REV + jointly own the exact `needs_revision` response relation and indexed + latest/current/accepted access; no initiative creates a competing + `SubmissionVersion` table. +- Reviewer and delivery streams consume an ART-owned integrity-checking read + capability that recomputes full SHA-256 and byte count. ART does not own the + review decision, ContributionRecord, compensation, reputation, or delivery + lifecycle that consumes that capability. ## Checkpoint Before Checker Expansion Do not resume checker feature expansion until `WS-ART-001-06B` proves pre-submit -and post-submit consume the same immutable artifact-set commitment. +evidence and post-submit execution name the same archive identity, +semantic-manifest hash, verified admission, and exact binding. diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/DECISIONS.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/DECISIONS.md index d36712f2..388aa8bc 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/DECISIONS.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/DECISIONS.md @@ -149,23 +149,25 @@ Operators receive bounded read-only admission-usage visibility and alerts. Quota expansion is a reviewed configuration/runbook operation; it never releases or edits an authoritative charge. -## D19 - Upload-Session Slot Expiry +## D19 - Submission-Bundle Scratch Capacity -Contributor open-session slots are separate from retained-byte admission. -Chunk 04A owns PostgreSQL-clock periodic Celery expiry and lazy expiry before -new-session admission. Both paths use the fixed system permission -`artifact.upload_session.expire` and one atomic terminal transition that -releases the slot exactly once. Cancellation or expiry never releases a -completed durable-byte charge. +The earlier durable multi-step upload-session slot design is superseded before +activation. One continuous submission-bundle preparation reserves bounded +scratch capacity through the existing `ArtifactScratchManager`. No durable +upload session/item is a candidate store, no scratch path crosses a process or +Celery boundary, and process loss requires reupload. ART-04A must remove or make +unreachable every unused upload-session/item model and action path; no alias or +parallel contributor intake remains. ## D20 - Closed Materialization Sources -The provider-neutral materializer accepts exactly one sealed upload artifact -set whose items are all `ready` before submission creation, or immutable -`ArtifactBinding` IDs after submission creation. Staging never creates a -premature product binding. Both forms resolve through phase-specific -authorization, stream exact provider bytes, and recompute SHA-256 and byte -count before a checker receives a read-only workspace. +Before durable admission, the checker receives only one internal +`PreparedArtifact`/`ArtifactScratchManager`-owned submission-bundle workspace in +the same process-local orchestration. After admission, the provider-neutral +materializer accepts only immutable `ArtifactBinding` IDs. Neither form creates +a second handle, lease, ledger, quota, cleanup path, or premature product +binding. Both recompute the relevant SHA-256 and byte counts before a checker +receives a read-only workspace. ## D21 - Configured Storage Namespace Fence @@ -254,3 +256,79 @@ defined canonically in `WS-AUTH-001-ART-CUSTODY` chunk contract. Operator verification retry remains an independently authorized human action. Service identity, Celery executor identity, and execution-generation fencing never substitute for one another. + +## D29 - One Outer ZIP Per Submission + +Every contributor Submission version accepts exactly one outer ZIP. Workstream +recursively inspects the directory/file tree represented by that archive so a +locked Project Guide checker can require entries such as `task.toml`. An archive +entry that is itself a ZIP remains one opaque ordinary file. Nested archive +unpacking is outside v0.1 and requires a separate reviewed capability with +cumulative safety limits before a Project Guide may request it. + +## D30 - Scratch Before Durable Admission + +Unchecked contributor bytes remain only in bounded private scratch through +format/safety inspection, canonical manifest generation, unchanged-work +comparison, mandatory platform gates, and locked project pre-submit checks. +Failed, unsafe, unchanged, abandoned, or checker-failing attempts never enter +object storage. Process or scratch loss before durable intent requires reupload. + +No candidate provider namespace, retention window, promotion/copy, or physical +cleanup API is added. A passing ZIP enters the existing immutable +`ArtifactStore` path once. Ambiguous durable effects reuse existing put attempt, +observation, verification, receipt, scanner, and recovery abstractions. + +## D31 - Canonical Submission Bundle Identity + +Workstream records both exact archive identity (server-computed SHA-256 and byte +count) and semantic tree identity. The semantic manifest commits to normalized +file/directory path, entry type, and each file's SHA-256 and byte count. It does +not commit to archive timestamps, compression method/level, comments, ownership, +or platform permission metadata. Symlinks and unsupported special entries are +rejected. Exact archive equality or semantic-manifest equality with the +immediate prior immutable Submission rejects before provider I/O. + +## D32 - Existing Submission Is The Version Aggregate + +The current immutable `Submission` row plus its `version` and +`supersedes_submission_id` is the canonical version aggregate. There is no +competing `SubmissionVersion` table. A reviewer attaches only one of `accept`, +`needs_revision`, or `reject` plus note/findings to the exact Submission. A +contributor response to `needs_revision` uploads a new complete ZIP, creates the +next Submission, and records the exact prior-Submission and review relationship +through the TASK/REV joint contract. + +Indexed latest/current/accepted queries or projections serve normal reads; +immutable relationships retain complete history and audit reconstruction. + +## D33 - Integrity Is Rechecked On Egress + +Historical verification is necessary but not sufficient for a later full-byte +read. Checker materialization, reviewer download, and downstream delivery use an +ART-owned provider-neutral read capability that recomputes complete SHA-256 and +byte count while streaming and fails closed on missing or mismatched bytes. +Consumer initiatives own their lifecycle decisions and reference the exact +Submission/binding; ART owns only byte identity, integrity, manifests, binding, +and access capability. + +## D34 - Conservative Limits Remain + +The existing configured upload, scratch, duration, and provider limits remain +unchanged. Any larger ZIP, extracted-byte, per-file, entry-count, depth, +compression-ratio, scratch, or time limit requires a separate reviewed change +with concurrency, capacity, runtime, cost, and abuse evidence. + +## D35 - One Contributor Preparation Action + +The prior multi-step upload-session action proposal does not govern the new +continuous scratch-bound flow. AUTH first registers planned ActionId +`artifact.submission_bundle.prepare` mapped to existing PermissionId +`submission.create`. ART-04A through 04C then provide the complete hidden route, +canonical task/assignment/actor resource facts, guards, and manifest. AUTH alone +integrates and activates it afterward. + +Contributor authority never implies fixed service authority. Checker +materialization, verification execution, pending-work scanning, ambiguous-put +resolution, and binding retain their distinct fixed actions. No compatibility +alias exposes the obsolete multi-step session actions. diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/INTENT.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/INTENT.md index 28fc9331..82e50192 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/INTENT.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/INTENT.md @@ -36,13 +36,31 @@ authorized bytes -> content-addressed S3-compatible object key -> independent complete-object verification -> immutable ArtifactContent and ArtifactReplica facts --> sealed artifact-set commitment +-> verified archive identity and canonical semantic-manifest commitment -> ArtifactBinding to guide, submission, checker input, log, or output ``` PostgreSQL stores metadata, bindings, operation receipts, lifecycle state, audit, and recovery coordination. The object provider stores bytes only. +For contributor work, the mandatory v0.1 invariant is: + +```text +one immutable Submission row/version +-> one uploaded outer ZIP +-> one safely inspected internal file/directory tree +-> one canonical semantic manifest +-> mandatory platform and locked Project Guide prechecks +-> one ArtifactStore admission and complete read-back verification +-> one exact ArtifactBinding +-> the same bytes checked, reviewed, accepted, recorded, and delivered +``` + +The outer ZIP may contain one file, a codebase, evidence, datasets, nested +directories, or any other content allowed by the locked Project Guide. A ZIP +entry inside the outer archive remains an ordinary file. Nested archive +unpacking is outside v0.1 even when a guide would otherwise request it. + ## First-Principle Constraints - Workstream computes and verifies canonical SHA-256 and byte count. @@ -59,6 +77,13 @@ audit, and recovery coordination. The object provider stores bytes only. retention references, or object metadata. - v0.1 performs no physical object deletion. Release and garbage collection require a later approved deletion-policy initiative. +- v0.1 has no candidate/quarantine object store or temporary provider + retention. Unchecked contributor bytes remain only in bounded private scratch + and enter the existing immutable store once, after every pre-submit check + passes. +- The existing `Submission` row is the immutable version aggregate. Reviewers + attach only a decision and note/findings to that exact version; a contributor + response to `needs_revision` is another complete ZIP and immutable Submission. - Local storage is forbidden in staging and production. - No compatibility alias or dual provider-construction path is retained. @@ -83,6 +108,9 @@ v0.1 contribution lifecycle. - no provider-side legal hold, pin, retain, or release API; - no physical deletion or garbage collection; - no semantic search; +- no candidate storage namespace, promotion copy, or temporary provider + retention window; +- no second artifact recovery aggregate; - no review packet or reviewer evidence implementation, which remains WS-REV; - no payment, reputation, blockchain, or marketplace expansion. @@ -101,6 +129,10 @@ v0.1 contribution lifecycle. Celery under PostgreSQL generation fencing; - guide, pre-submit, post-submit, and reviewer-facing records resolve the same immutable content commitment; +- exact-archive and semantic-manifest equality reject unchanged resubmissions + before provider I/O; +- checker, reviewer, and delivery reads recompute SHA-256 and byte count while + streaming the exact bound bytes; - the final real API drill runs without direct database inspection. ## Human Decisions diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md index 60db8577..94cb7f88 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md @@ -29,7 +29,7 @@ Those product capabilities are closed and operation-specific: ```text GuideArtifactIngestPort -ContributorArtifactUploadPort +SubmissionBundlePreparationPort ArtifactBindingPort ArtifactMaterializationPort CheckerArtifactOutputPort @@ -184,15 +184,16 @@ durable acknowledgement-unknown attempt rather than committing a terminal fact. computes digest and size, and rejects a mismatched client commitment before provider I/O. 3. PostgreSQL atomically reserves unique-byte charges at every applicable task, - producer, project, and deployment scope. Contributor upload-session slots - are a separate 04A reservation. Any exceeded scope fails before provider - I/O. Provisional and completed byte charges count; exact replay and + producer, project, and deployment scope. For contributor submissions this + durable reservation occurs only after all scratch-bound pre-submit gates + pass. Any exceeded scope fails before provider I/O. Provisional and + completed byte charges count; exact replay and concurrent same-content reservations cannot double-charge or oversubscribe. Product callers do not assemble this set; artifact orchestration derives it - from authoritative actor/service, project, task, upload-session or + from authoritative actor/service, project, task, submission-bundle operation or checker-run, and deployment records. A missing required relationship fails before provider I/O. -4. Transaction A reserves the upload item and commits the server-computed +4. Transaction A reserves the producer-specific admission and commits the server-computed digest, size, media type, operation identity, request digest, and CAS values. 5. Workstream passes the sealed `CommittedArtifactSource` to the injected adapter outside the transaction. @@ -201,7 +202,7 @@ durable acknowledgement-unknown attempt rather than committing a terminal fact. the adapter conditionally stores under the content-addressed key or resolves an exact replay candidate. 7. Transaction B records provider acknowledgement, completes the provisional - admission charges, sets the item to `stored_pending_verification`, and + admission charges, sets the producer attempt to `stored_pending_verification`, and creates the replica with pending verification and unknown availability/integrity; no binding exists. 8. A durable verification job is committed in PostgreSQL and published to @@ -221,8 +222,8 @@ resolver repeats a provider write. Exact replay after absence must atomically reacquire capacity before another provider call. Workstream never stores upload bytes in Postgres, Redis, or Celery payloads. -Before binding, an object confirmed missing returns its reserved upload item to -`replay_required`, and only the original authorized uploader may replay the +Before binding, an object confirmed missing returns its reserved producer attempt to +`replay_required`, and only the original authorized producer may replay the same bytes under the same operation identity. That pre-binding replay resets the same replica from `missing/unavailable/unknown` to `pending/unknown/unknown`, appends a receipt, and creates a new verification @@ -233,9 +234,9 @@ replaces or rebinds the bytes. A digest/size mismatch on an existing key is also unrecoverable in v0.1 and becomes a security incident; Workstream never overwrites that key. -Contributor uploads, authorized caller-supplied guide bytes, and generated -checker logs/outputs -all use the same bounded two-pass `PreparedArtifact` scratch boundary. The first +Contributor submission ZIPs, authorized caller-supplied guide bytes, and +generated checker logs/outputs all use the same bounded two-pass +`PreparedArtifact` scratch boundary. The first pass uses private ephemeral local scratch to hash/count; the second pass exposes a sealed `CommittedArtifactSource` to ArtifactStore. No caller can pass an arbitrary expected digest beside an arbitrary stream. Scratch is never @@ -274,30 +275,41 @@ Physical deletion, garbage collection, legal hold, and retention windows are a later explicit initiative. This removes destructive storage behavior from the first production cutover. -## Server-Generated Artifact Set - -The server seals an upload session before authoritative pre-submit execution. -Trusted inspection derives normalized display names, logical roles, media -types, archive members, path safety, individual sizes, total size, and -Workstream SHA-256 values. Canonical ordering and JSON produce -`artifact_set_hash`. - -The pre-submit admission record binds actor, task, sealed session, artifact-set -hash, locked guide/policy/checker context, result, expiry, and canonical -submission input hash. Submission creation row-locks and consumes that exact -admission and session. No checked-versus-submitted byte swap is possible. - -Pre-submit artifact reads use a bounded Workstream-owned transient retry budget. -If it is exhausted, the API returns HTTP 503 with -`pre_submission_infrastructure_unavailable`; this is neither a checker failure -nor a product review decision. The exact precheck attempt, sealed upload -session, and artifact set remain unconsumed and reusable until normal expiry. -Only infrastructure-attempt and audit facts are written. Idempotency binds -actor, task, session/artifact set, locked context, client key, and canonical -request digest: exact replay continues or returns the same attempt, changed -replay conflicts, and concurrent replay creates no duplicate. After retry-after -or infrastructure recovery, the contributor continues the same exact attempt -without Project Manager or Operator approval. +## Server-Generated Submission Bundle Identity + +Every Submission version has exactly one outer ZIP. Workstream keeps that ZIP +only in bounded private scratch while it walks the complete normalized +file/directory tree, enforces archive-safety/resource limits, hashes each file, +and derives two identities: + +```text +archive identity = SHA-256 and byte count of the exact outer ZIP +semantic identity = canonical hash of normalized path, entry type, + file SHA-256, and file byte count +``` + +ZIP timestamps, compression settings, comments, ownership, and platform +permission metadata do not change semantic identity. Explicit empty directories +and synthetic parents use one documented canonical representation. Symlinks and +special entries are rejected. A ZIP entry within the outer archive is opaque +in v0.1; recursive inspection means walking the outer archive tree, not opening +nested archives. A later capability requires separate cumulative safety proof. + +Both identities are compared with the immediate prior immutable `Submission`. +Exact archive equality or semantic equality rejects before checker and provider +I/O. Mandatory Workstream gates and the task's locked Project Guide checker then +consume the same read-only scratch tree. A project may narrow platform limits +but cannot disable gates or raise limits. Checker failure creates findings only +and destroys scratch without durable artifact, Submission, or review state. + +A passing result stays bound to the same process-local scratch generation and is +consumed immediately by the normal durable admission path. It is never a +provider object or reusable candidate. Process/scratch loss requires reupload. +The durable path writes the outer ZIP once, independently reads it completely, +and publishes a bindable admission only after the observed archive identity +matches. Existing put-attempt, observation, verification, receipt, scanner, and +recovery behavior resolves ambiguity. No candidate namespace, promotion copy, +retention window, physical deletion, or second recovery aggregate exists. ## Durable Verification And Recovery @@ -394,20 +406,29 @@ exact resource/content/job/attempt filters. ## Product Cutover -1. Guide source capture stores exact bytes and binds the immutable source - snapshot to content. -2. Task-scoped upload sessions accept contributor artifacts and seal one exact - server artifact set. -3. Authoritative pre-submit executes against sealed verified bytes. - One authorized artifact materializer allocates through the canonical scratch - manager, recomputes every digest and byte count from the provider read, seals - the verified workspace read-only, and is reused unchanged by post-submit. -4. Submission creation consumes the same admission and creates immutable - bindings; contributor finalization is not a manager action. -5. Post-submit dispatch reads the same content commitment and stores checker - inputs, logs, and outputs as artifacts. -6. Review packet/evidence integration remains WS-REV, but it consumes the same - `ArtifactBinding` model. +1. Guide-source delivery is split into hidden byte ingest, AUTH activation, + verified snapshot binding/materialization, AUTH activation, and the legacy + identity/continuation clean cut. +2. Contributor intake accepts one outer ZIP, inspects and manifests its complete + tree in bounded private scratch, and rejects exact or semantic unchanged work. +3. Mandatory platform gates and locked Project Guide pre-submit checks execute + against that same scratch tree. Failure produces no durable bytes. +4. Passing bytes enter the existing immutable store once. Complete read-back + verification publishes one bindable admission; ambiguity uses existing ART + recovery. +5. Submission creation consumes that exact admission and creates one immutable + `Submission` version and binding; contributor finalization is not a manager + action and no competing `SubmissionVersion` table is introduced. +6. Post-submit dispatch materializes the same binding, recomputes integrity, and + stores checker logs/outputs as separately bound artifacts. +7. Review packet/evidence integration remains WS-REV. REV attaches a decision + (`accept`, `needs_revision`, or `reject`) and note/findings to the exact + Submission; the reviewer uploads no revision artifact. +8. A contributor response to `needs_revision` is another complete ZIP and + immutable Submission linked to the prior Submission and exact Review. +9. CON and future delivery own their lifecycle records but reference the same + accepted Submission/binding. Full-byte streams recompute SHA-256 and byte + count and fail closed on mismatch. The existing misleading `/submissions/{id}/finalize` endpoint is not retained as a normal handoff. Its genuine exceptional behavior moves to diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/RISKS.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/RISKS.md index fa254d12..8d92acf2 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/RISKS.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/RISKS.md @@ -27,11 +27,11 @@ | Concurrent prepared artifacts exhaust worker disk | High | Full-max pre-reservation, dedicated quota, cross-process ledger, bounded concurrency, free-space floor, and deadline-before-TTL cleanup. | | Any guide, contributor, or checker producer creates unbounded retained bytes | Critical | One generic admission service atomically caps cumulative unique provisional/completed bytes at every applicable task, producer, project, and deployment scope before provider I/O; only authoritative absence releases a provisional charge. | | Operators cannot diagnose or expand exhausted storage capacity | High | Bounded admission-usage API, scope metrics/alerts, and a configuration-driven expansion/rollback runbook; no charge deletion or database editing. | -| Expired upload sessions retain open-session capacity forever | High | PostgreSQL-clock periodic Celery expiry plus lazy expiry before admission perform one atomic terminal transition and release the session slot exactly once. | +| Legacy planned upload-session actions accidentally become a second intake path | Critical | Keep them unavailable and route-less until the AUTH registration handoff retires them; ART-04A removes or makes their models/routes unreachable before the single bundle action can activate. | | Recovery replay or ancestor reuse creates multiple jobs | High | Canonical request digest plus a lifetime unique source-job constraint return original IDs on exact replay, reject different-key/concurrent reuse, and require chain progression through the immediate retry job. | | Operator cannot observe retry completion | High | Authorized recovery-attempt GET includes immutable source-job and current retry-job status plus audit IDs. | | Storage failure becomes contributor failure or review decision | Critical | Stable infrastructure errors; no accept/needs_revision/reject, contribution, payment, or reputation effect. | -| Pre-submit outage strands or duplicates a contributor attempt | High | Bounded Workstream retry, stable 503, sealed unconsumed artifact set, exact request-digest idempotency, and contributor continuation without manager/operator approval. | +| Pre-submit process or scratch loss strands an attempt | High | Return a stable infrastructure outcome, clean scratch, and require reupload without manager/operator approval; only post-intent ambiguity uses existing durable ART recovery. | | Pre-existing lifecycle rule deletes evidence | Critical | Read-only AWS lifecycle inspection blocks activation for any deletion/expiration rule intersecting the completed-object prefix. | | Local adapter reaches production | High | Closed backend enum and production startup rejection. | | MinIO proof is mistaken for AWS readiness | High | Shared conformance runs on LocalStorage/MinIO, while AWS has a separate private-bucket, workload-identity, lifecycle, and anonymous-read live proof. | @@ -39,3 +39,12 @@ | R2 remains a hidden v0.1 dependency | High | No active R2 chunk/configuration/runtime path; stale-contract gate rejects reintroduction outside a separately approved initiative. | | Route tests overclaim fleet atomicity | Medium | Per-image proof only; deployment barrier documented separately. | | Large chunks become unreviewable | High | Preparation, v2 cutover, provider profiles, verification, recovery, upload/sealing, pre-submit admission, checker input, and checker output/routing are separate checkpoints with explicit allowed files and stop conditions. | +| Temporary candidate storage consumes capacity or creates a second lifecycle | Critical | Candidate storage is rejected for v0.1. Unchecked bytes remain only in bounded scratch and passing bytes enter the existing immutable store once. | +| Outer ZIP tree cannot be checked against the Project Guide | Critical | Safely walk every normalized file/directory entry in the outer ZIP, hash files during bounded materialization, and give the locked checker the exact read-only tree. Nested ZIP entries stay opaque throughout v0.1. | +| Repackaging unchanged work wastes reviewer capacity | High | Compare both exact archive identity and canonical semantic-manifest identity with the immediate prior immutable Submission before checker/provider I/O. | +| Scratch path is sent to another worker or survives process loss | Critical | Keep 04A-04C one process-local orchestrated operation; never serialize scratch paths. Process/scratch loss invalidates the attempt and requires reupload. | +| Verified bytes become unbound after process/transaction loss | High | Persist exact durable intent and use idempotent retry plus existing put/verification recovery; create no Submission until verified content and admission are bindable. | +| Reviewer note is confused with a revision artifact | High | REV attaches decision and note/findings to the exact Submission. Only a contributor uploads the next complete ZIP after `needs_revision`. | +| Full history requires expensive recursive traversal | Medium | Preserve immutable links but add indexed latest/current/accepted repository queries or projections maintained under lifecycle transactions. | +| Stored content changes after initial verification | Critical | Recompute full SHA-256 and byte count during checker materialization, reviewer full download, and downstream delivery; fail closed and record an artifact incident on mismatch. | +| Speculative larger limits exhaust disk or worker time | High | Preserve current configured limits until separately reviewed capacity, concurrency, runtime, cost, and abuse evidence justifies a change. | diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/STATUS.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/STATUS.md index 3bb7ed80..ea1977a3 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/STATUS.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/STATUS.md @@ -1,72 +1,60 @@ # Status: WS-ART-001 Immutable Artifact Storage -## Current State +## Completed Foundation -Original planning merged through PR #97, artifact/LocalStorage foundation -merged through PR #101, the AWS-first object-storage amendment merged through -PR #120 as `4408256`, the external-service adapter foundation merged through -PR #127 as `f64a8e5`, committed-source preparation merged through PR #129 as -`9a04434`, the ArtifactStore v2 Local clean cut merged through PR #141 as -`a10d901`, and S3-compatible MinIO/AWS preparation merged through PR #151 as -`1b5422fc` on 2026-07-19. ART admission and the put-attempt foundation then -merged through PR #154 as `44f2467c`, and the user explicitly started -`WS-ART-001-02C2` on 2026-07-19. +Planning and the artifact foundation merged through PR #97 and PR #101. The +AWS-first object-storage amendment and typed adapter clean cut merged through +PRs #120, #127, #129, #141, and #151. Durable admission, put attempts, +verification/publication, recovery idempotency, and hidden Operator operations +merged through PRs #154, #159, #174, and #177 (`WS-ART-001-02D`). -The planning-only cross-initiative boundary reconciliation merged through -PR #139 as `5d353b6`, and AUTH's owner reconciliation merged through PR #140 as -`d541521`. ART now consumes AUTH's canonical activation-custody and prepared -mutation contracts without editing or activating AUTH runtime behavior. -AUTH-09D-A merged through PR #148 as `99ae4c9`, AUTH-09D-B merged through PR -#152 as `93dd392`, the contributor foundation merged through PR #153 as -`8d5eb15b`, AUTH-09E merged through PR #157 as `42a89b2d`, and the -availability-neutral ART custody transfer merged through PR #158 as -`be2a79a2`, and the unrelated availability-neutral REV custody transfer merged -through PR #160 as `fe0e4492`. Backend CI sharding and timeout repair merged -through PRs #163 and #164 as `b0f9ad64` and `61bc0390`; signed-start loop-memory -planning merged through PR #165 as `58d0514a`; and AUTH-PREP merged through PR -#162 as `c559d556`. All are integrated into the ART candidate. AUTH-PREP adds no -ART consumer or activation. The three ART internal feature actions remain -assigned to future -`WS-AUTH-001-ART-02D-INTERNAL` activation custody, but remain planned and -inactive. +The current v0.1 provider direction remains AWS S3 in production, MinIO for +local/CI protocol proof, and LocalStorage for development/focused tests. Flow +Node and R2 remain deferred. Completed objects have no physical deletion path. +AUTH's owner reconciliation merged through PR #140 as +`d541521`; PLAN2 preserves AUTH ownership and proposes no availability edit. -The Flow Node-focused amendment candidate `6cc422d` passed deterministic checks -but failed internal review on recovery/API completeness. Before repair, the user -approved a first-principle change on 2026-07-14: +## Cancelled Work -```text -v0.1 production bytes -> S3CompatibleArtifactStore -> AWS S3 -local/CI proof bytes -> S3CompatibleArtifactStore -> MinIO -development bytes -> LocalStorageAdapter -future optional bytes -> Flow Node adapter initiative -``` +`WS-ART-001-03` received a signed implementation start on current history, but +mandatory preimplementation review rejected the combined contract before any +runtime edit. The user authorized cancellation, and signed automation run +`30100940860` recorded `stopped_after_cancel` on 2026-07-24. The rejected +contract combined guide byte ingest, binding, materialization, setup recovery, +migration, and inactive AUTH dependencies without a safe executable boundary. + +## Current Planning Reconciliation -The failed Flow Node candidate and every reviewer session are closed. It is not -approval or reusable evidence. Its source remains on branch -`codex/ws-art-001-fn01-isolation-amendment` for the deferred Flow Node plan. +`WS-ART-001-PLAN2` is planning-only. It incorporates the human-approved +submission invariant: -## Current Work +```text +one outer ZIP +-> bounded private scratch inspection and canonical manifest +-> exact/semantic unchanged rejection +-> mandatory platform and locked Project Guide prechecks +-> one existing ArtifactStore admission and complete read-back verification +-> one immutable Submission binding +-> the same bytes checked, reviewed, accepted, recorded, and delivered +``` -`WS-ART-001-02C2` is active after the user's explicit start on 2026-07-19. It -adds caller-only committed put execution, read-only ambiguous-put resolution, -durable verification jobs and typed receipts, PostgreSQL executor/generation -fencing, bounded publication scanning, and complete-read deadlines. Production -composition remains deny-only: the three internal artifact actions stay -planned, no 02C2 Beat schedule is active, and AUTH retains sole activation -custody. Recovery attempts, Operator routes, product cutovers, deletion, and -background write replay remain out of scope. +There is no candidate store, temporary provider retention, promotion copy, +physical deletion, second recovery aggregate, speculative capacity increase, or +competing `SubmissionVersion` table. Reviewers attach a decision plus +note/findings to the +exact `Submission`; contributors answer `needs_revision` with another complete +ZIP and immutable Submission. ## Next Proposed Chunk -`02C3` may add the recovery-attempt and idempotency chain only after 02C2 merges -and receives a separate explicit start. No deferred provider has a v0.1 chunk. +After this planning package merges, `WS-ART-001-03A` is the only declared ART +successor. It adds hidden guide-source byte ingest through the existing artifact +preparation/admission/verification path. It requires a separate signed start and +does not activate its own AUTH action. ## Gate -The candidate integrates trusted main `c559d556`, the sharded Backend workflow, -and merged AUTH-PREP without activating ART. Exact-SHA internal evidence is -complete for reviewed code `59fbab56`; hosted checks must rerun on the published -evidence head. The current gate is external CI/review followed by explicit human -merge approval. -Production activation and recovery remain in later owning chunks. No later -artifact chunk starts automatically, and only the user may approve merge. +Planning evidence and all required internal reviewer tracks must pass before a +PR. The planning merge starts no successor. Every implementation chunk retains +its separate signed start, exact AUTH activation sequence, internal review, CI, +CodeRabbit, human checkpoint, and automated merge-memory stop. diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03-guide-source-cutover.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03-guide-source-cutover.md index 05324f14..a4b69daa 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03-guide-source-cutover.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03-guide-source-cutover.md @@ -1,6 +1,12 @@ -# Chunk Contract: WS-ART-001-03 - Guide Source Artifact Cutover +# Cancelled Contract: WS-ART-001-03 - Guide Source Artifact Cutover -Initiative: `WS-ART-001` | Risk: L1 | Status: Proposed after 02D +Initiative: `WS-ART-001` | Risk: L1 | Status: Cancelled before implementation; superseded by 03A-03C + +This contract was signed and then cancelled after mandatory preimplementation +review found that it combined guide byte intake, binding, setup-agent reads, +recovery continuation, migration, and inactive AUTH dependencies without an +implementable byte-transport or activation sequence. It authorizes no work and +is retained only as history. Artifact contract phase: `guide_source_cutover` diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03A-guide-source-byte-ingest.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03A-guide-source-byte-ingest.md new file mode 100644 index 00000000..5d2462bb --- /dev/null +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03A-guide-source-byte-ingest.md @@ -0,0 +1,73 @@ +# Chunk Contract: WS-ART-001-03A Guide Source Byte Ingest + +Initiative: `WS-ART-001` | Risk: L1 | Status: Proposed after PLAN2 + +Artifact contract phase: `guide_source_cutover` + +## Goal + +Add hidden, authorized guide-source byte ingest through the existing preparation, +admission, put-attempt, verification, and content-publication path without guide +snapshot binding, setup-agent reads, or public activation. + +## Allowed Files + +- project guide-source ingest models, one migration, repository, schemas, + service, and router; +- ART guide ingest capability, canonical resource facts/guards, and composition; +- existing prepared-source/admission call sites only for this producer; +- focused migration, authorization, ingest, ambiguity, and API tests; +- backend workflow/agent-gate files only to add and preserve exact coverage; +- related docs and chunk memory. + +## Not Allowed + +- snapshot binding/activation, setup materialization/continuation, or legacy + schema removal; +- task/submission/checker/review changes; +- AUTH-owned catalogue/evaluator/grant/identity/matrix/availability edits; +- provider/factory changes, compatibility aliases, or network URL fetching. + +## Acceptance Criteria + +- authorized caller-supplied/import-pipeline bytes cross `PreparedArtifact` and + generic admission once; caller hashes and provider references are never truth; +- `ArtifactContent` is published only after complete read-back verification; +- exact retry and ambiguity use existing put/recovery state; +- legacy hash-only/provider-object guide ingest is unavailable on the hidden + path, but destructive schema removal waits for 03C; +- `artifact.guide_source.ingest` behavior remains fail-closed until AUTH + activates the exact action after this hidden implementation merges; +- changed subsystem coverage is at least 90 percent and repository coverage + remains at least 78 percent. + +## Exact CI Coverage Gates + +```bash +coverage report --include='app/adapters/artifacts/*,app/core/cancellation.py,app/core/file_locks.py,app/interfaces/artifact_operations.py,app/interfaces/artifacts.py,app/modules/artifacts/*' --precision=2 --fail-under=90 +coverage report --include='app/interfaces/external_services.py' --precision=2 --fail-under=90 +coverage report --include='app/core/config.py' --precision=2 --fail-under=90 +coverage report --include='app/workers/*' --precision=2 --fail-under=90 +coverage report --include='app/main.py' --precision=2 --fail-under=90 +coverage report --include='app/adapters/artifacts/s3_compatible.py' --precision=2 --fail-under=90 +coverage report --include='app/core/s3_validation.py' --precision=2 --fail-under=90 +coverage report --include='app/modules/audit/*' --precision=2 --fail-under=90 +coverage report --include='app/api/router.py' --precision=2 --fail-under=90 +coverage report --include='app/modules/projects/*' --precision=2 --fail-under=90 +``` + +## Verification + +```bash +docker compose up -d --wait postgres redis minio +(cd backend && WORKSTREAM_TEST_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/workstream_test .venv/bin/pytest tests/test_alembic.py tests/test_guide_artifacts.py tests/test_artifact_admission.py tests/test_artifact_put_resolution.py tests/test_artifact_verification.py -q --cov=app.modules.projects --cov=app.modules.artifacts --cov-report=term-missing --cov-fail-under=90) +(metadata_dir="$(mktemp -d)" && trap 'rm -rf "$metadata_dir"' EXIT && (cd backend && WORKSTREAM_TEST_ADMIN_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/postgres .venv/bin/python scripts/run_isolated_tests.py --metadata-json "$metadata_dir/result.json" --timeout-seconds 12600 -- .venv/bin/python -m pytest -q --ignore=tests/test_isolated_database_runner.py --cov=app --cov-report=term-missing --cov-fail-under=78)) +(cd backend && .venv/bin/ruff check app tests) +python3 scripts/check_stale_artifact_contracts.py +python3 scripts/test_agent_gates.py +``` + +## Required Reviewers + +Senior engineering, architecture, QA/test, security/auth, product/ops, +reuse/dedup, CI integrity, test delta, and docs. diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B-guide-source-binding-materialization.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B-guide-source-binding-materialization.md new file mode 100644 index 00000000..d015559a --- /dev/null +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B-guide-source-binding-materialization.md @@ -0,0 +1,72 @@ +# Chunk Contract: WS-ART-001-03B Guide Source Binding And Materialization + +Initiative: `WS-ART-001` | Risk: L1 | Status: Proposed after 03A and exact AUTH activation + +Artifact contract phase: `guide_source_cutover` + +## Goal + +Bind verified guide-source content to snapshot items and give setup agents one +authorized, provider-neutral, integrity-checking read/materialization capability +without performing the legacy schema cutover. + +## Allowed Files + +- guide snapshot/item binding models, one migration, repository, schemas, and service; +- ART guide read/binding capability and canonical resource facts/guards; +- existing artifact scratch manager/materializer integration for setup input; +- setup-agent input assembly consuming the typed ART capability; +- focused migration, authorization, binding, materialization, and integrity tests; +- backend workflow/agent-gate files only to add and preserve exact coverage; +- related docs and chunk memory. + +## Not Allowed + +- contributor submission/checker/review behavior; +- direct provider access, URL fetching, a second materializer/scratch manager, + setup policy decisions, or destructive legacy-field removal; +- AUTH-owned catalogue/evaluator/grant/identity/matrix/availability edits. + +## Acceptance Criteria + +- snapshot items bind exact verified `ArtifactContent` without circularly making + the item both the source and authority for that content; +- snapshot/bundle identity commits to ordered content identities and descriptors; +- setup agents receive bounded bytes/manifests only through ART and every full + read recomputes SHA-256 and byte count; +- missing/mismatched bound bytes are artifact incidents, not guide insufficiency; +- fixed guide read/binding actions remain fail-closed until AUTH activation; +- changed subsystem coverage is at least 90 percent and repository coverage + remains at least 78 percent. + +## Exact CI Coverage Gates + +```bash +coverage report --include='app/adapters/artifacts/*,app/core/cancellation.py,app/core/file_locks.py,app/interfaces/artifact_operations.py,app/interfaces/artifacts.py,app/modules/artifacts/*' --precision=2 --fail-under=90 +coverage report --include='app/interfaces/external_services.py' --precision=2 --fail-under=90 +coverage report --include='app/core/config.py' --precision=2 --fail-under=90 +coverage report --include='app/workers/*' --precision=2 --fail-under=90 +coverage report --include='app/main.py' --precision=2 --fail-under=90 +coverage report --include='app/adapters/artifacts/s3_compatible.py' --precision=2 --fail-under=90 +coverage report --include='app/core/s3_validation.py' --precision=2 --fail-under=90 +coverage report --include='app/modules/audit/*' --precision=2 --fail-under=90 +coverage report --include='app/api/router.py' --precision=2 --fail-under=90 +coverage report --include='app/modules/projects/*' --precision=2 --fail-under=90 +coverage report --include='app/adapters/project_agents/*,app/interfaces/project_agents.py' --precision=2 --fail-under=90 +``` + +## Verification + +```bash +docker compose up -d --wait postgres redis minio +(cd backend && WORKSTREAM_TEST_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/workstream_test .venv/bin/pytest tests/test_alembic.py tests/test_guide_artifacts.py tests/test_project_setup.py tests/test_checker_materialization.py tests/test_artifact_scratch_manager.py -q --cov=app.modules.projects --cov=app.modules.artifacts --cov-report=term-missing --cov-fail-under=90) +(metadata_dir="$(mktemp -d)" && trap 'rm -rf "$metadata_dir"' EXIT && (cd backend && WORKSTREAM_TEST_ADMIN_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/postgres .venv/bin/python scripts/run_isolated_tests.py --metadata-json "$metadata_dir/result.json" --timeout-seconds 12600 -- .venv/bin/python -m pytest -q --ignore=tests/test_isolated_database_runner.py --cov=app --cov-report=term-missing --cov-fail-under=78)) +(cd backend && .venv/bin/ruff check app tests) +python3 scripts/check_stale_artifact_contracts.py +python3 scripts/test_agent_gates.py +``` + +## Required Reviewers + +Senior engineering, architecture, QA/test, security/auth, product/ops, +reuse/dedup, CI integrity, test delta, and docs. diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03C-guide-source-cutover-continuation.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03C-guide-source-cutover-continuation.md new file mode 100644 index 00000000..5b10ca86 --- /dev/null +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03C-guide-source-cutover-continuation.md @@ -0,0 +1,75 @@ +# Chunk Contract: WS-ART-001-03C Guide Source Cutover And Continuation + +Initiative: `WS-ART-001` | Risk: L1 | Status: Proposed after 03B and exact AUTH activation + +Artifact contract phase: `guide_source_cutover` + +## Goal + +Remove legacy guide-source identity, make verified ART bindings authoritative, +and resume the exact same setup generation after artifact recovery without +adding a Project Manager resume command. + +## Allowed Files + +- guide-source/snapshot models, one clean-cut migration, repository, schemas, + service, and router; +- exact same-generation setup continuation record/worker and affected setup tasks; +- `backend/scripts/api_contract_e2e.py` only for the guide clean cut; +- focused refusal, continuation, recovery, API, and real-flow tests; +- stale-contract/agent-gate/backend workflow files only for the exact phase and + preserved coverage gates; +- related docs and chunk memory. + +## Not Allowed + +- task/submission/checker/review cutover, synthetic backfill, URL fetching, + provider/factory changes, or Project Manager resume command; +- AUTH-owned catalogue/evaluator/grant/identity/matrix/availability edits. + +## Acceptance Criteria + +- migration refuses unsafe populated legacy rows rather than fabricating bytes; +- caller `content_cid`, provider-object schemes, and hash-only identity are + removed without aliases after verified replacement is available; +- recovery continuation references a persisted setup run/generation and resumes + only when project, guide, snapshot, policy, and generation still match; +- Operator recovery authorization and automatic setup continuation remain + distinct audit facts; neither approves guide sufficiency; +- changed guide bytes create a new snapshot/setup generation, while locked tasks + retain their prior context unless explicitly rebased; +- changed subsystem coverage is at least 90 percent and repository coverage + remains at least 78 percent. + +## Exact CI Coverage Gates + +```bash +coverage report --include='app/adapters/artifacts/*,app/core/cancellation.py,app/core/file_locks.py,app/interfaces/artifact_operations.py,app/interfaces/artifacts.py,app/modules/artifacts/*' --precision=2 --fail-under=90 +coverage report --include='app/interfaces/external_services.py' --precision=2 --fail-under=90 +coverage report --include='app/core/config.py' --precision=2 --fail-under=90 +coverage report --include='app/workers/*' --precision=2 --fail-under=90 +coverage report --include='app/main.py' --precision=2 --fail-under=90 +coverage report --include='app/adapters/artifacts/s3_compatible.py' --precision=2 --fail-under=90 +coverage report --include='app/core/s3_validation.py' --precision=2 --fail-under=90 +coverage report --include='app/modules/audit/*' --precision=2 --fail-under=90 +coverage report --include='app/api/router.py' --precision=2 --fail-under=90 +coverage report --include='app/modules/projects/*' --precision=2 --fail-under=90 +coverage report --include='app/adapters/project_agents/*,app/interfaces/project_agents.py' --precision=2 --fail-under=90 +``` + +## Verification + +```bash +docker compose up -d --wait postgres redis minio +(cd backend && WORKSTREAM_TEST_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/workstream_test .venv/bin/pytest tests/test_alembic.py tests/test_projects.py tests/test_project_setup.py tests/test_guide_artifacts.py tests/test_artifact_recovery.py -q --cov=app.modules.projects --cov=app.modules.artifacts --cov=app.workers --cov-report=term-missing --cov-fail-under=90) +(metadata_dir="$(mktemp -d)" && trap 'rm -rf "$metadata_dir"' EXIT && (cd backend && WORKSTREAM_TEST_ADMIN_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/postgres .venv/bin/python scripts/run_isolated_tests.py --metadata-json "$metadata_dir/result.json" --timeout-seconds 12600 -- .venv/bin/python -m pytest -q --ignore=tests/test_isolated_database_runner.py --cov=app --cov-report=term-missing --cov-fail-under=78)) +(cd backend && WORKSTREAM_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/workstream_test .venv/bin/python scripts/api_contract_e2e.py) +(cd backend && .venv/bin/ruff check app tests) +python3 scripts/check_stale_artifact_contracts.py +python3 scripts/test_agent_gates.py +``` + +## Required Reviewers + +Senior engineering, architecture, QA/test, security/auth, product/ops, +reuse/dedup, CI integrity, test delta, and docs. diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04A-upload-inspection-sealing.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04A-upload-inspection-sealing.md index e07b2d90..a529839e 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04A-upload-inspection-sealing.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04A-upload-inspection-sealing.md @@ -1,85 +1,83 @@ -# Chunk Contract: WS-ART-001-04A Upload, Inspection, And Sealing +# Chunk Contract: WS-ART-001-04A One-ZIP Scratch Intake And Manifest -Initiative: `WS-ART-001` | Risk: L1 | Status: Proposed after 03 +Initiative: `WS-ART-001` | Risk: L1 | Status: Proposed after 03C and AUTH planned registration Artifact contract phase: `guide_source_cutover` ## Goal -Add contributor-authorized task upload sessions/items, trusted archive -inspection, independent verification, immutable sealing, and artifact-set -manifests without running pre-submit or creating submissions. +Accept exactly one contributor outer ZIP into bounded private scratch, safely +inspect its complete internal tree, and produce authoritative archive and +semantic-manifest commitments without provider I/O, project checker execution, +or Submission creation. ## Allowed Files -- artifact upload-session/item/set models and one migration; -- artifact repository/service/router/schemas for upload, inspect, seal, and - logical cancellation APIs and internal upload-session expiry; -- trusted archive/media inspection helpers; -- artifact call sites consuming exact decisions already delivered by the - approved WS-AUTH dependency; no Authorization Service owner files; -- focused migration, concurrency, upload, inspection, sealing, and real-API tests; -- `.github/workflows/backend.yml` only to expand the exact 90 percent scoped gate; -- `scripts/test_agent_gates.py` only to assert that backend CI retains this - chunk's exact scoped coverage sources and fail-closed 90 percent threshold; -- related docs and chunk memory. +- ART-owned scratch-intake, ZIP inspection, normalization, and manifest code; +- existing `ArtifactUploadSession`/`ArtifactUploadItem` models, migration, + repositories, schemas, and routes only to remove them or make them unreachable + as contributor intake; no retained provider/content-result staging semantics; +- task-scoped hidden upload surface and ART canonical resource facts/guards; +- one migration for the bounded intake/manifest control plane if required; +- focused archive, quota, concurrency, cleanup, fuzz/regression, API, coverage, + docs, and chunk-memory changes; +- CI files only to add and preserve the exact scoped 90 percent gate. ## Not Allowed -- pre-submit checker execution or admission records; -- submission creation/finalization changes; -- post-submit checker or review changes; -- presigned/direct provider upload or provider identifiers in responses; -- free-form client policy/checker overrides; -- physical provider deletion on cancellation. +- `ArtifactStore.put`, `ArtifactContent`, replica, binding, or provider I/O; +- project pre-submit checker execution or admission; +- Submission, Review, Contribution, compensation, reputation, or delivery work; +- multi-item submission sets, direct/presigned uploads, candidate storage, + retention windows, or physical deletion; +- AUTH-owned catalogue, evaluator, grant, identity, matrix, or activation edits; +- raising current configured limits. ## Acceptance Criteria -- only the assigned contributor with exact task authorization can create or - mutate a live task upload session; -- create, read, item write, seal, and cancel call the distinct task-scoped - permissions `artifact.upload_session.create`, - `artifact.upload_session.read`, `artifact.upload_item.write`, - `artifact.upload_session.seal`, and `artifact.upload_session.cancel` - through the central AUTH kernel with ART-owned canonical resource facts and - guards; this chunk first supplies hidden behavior while actions remain planned, - and AUTH alone activates them afterward; none implies another; -- every ID-addressed read/mutation uses concealed deny/not-found behavior and - is tested across actors, projects, revocation, terminal states, and random IDs; -- upload requires a server-approved logical role, bounded display name, client - SHA-256 commitment, expected byte count, and streamed file; -- Workstream prepares and hashes the complete stream, rejects mismatch before - provider I/O, and derives the key only from the server commitment; -- PostgreSQL atomically enforces configured open-session slots at task, actor, - project, and deployment scope; contributor item bytes use the generic 02C1 - admission service at task, authenticated actor, project, and deployment - scope after canonical hashing and before provider I/O; -- PostgreSQL-clock expiry is owned by a bounded periodic Celery scanner and is - also applied lazily before new-session admission. Both paths require the - fixed internal permission `artifact.upload_session.expire`; one atomic - terminal transition releases each open-session slot exactly once, and stale - scanner execution cannot release it again; -- cancellation and expiry release only the open-session slot. Provisional and - completed byte charges follow the generic admission state machine; - cancelled, expired, and unbound completed bytes remain charged in v0.1, - while exact deduplicated content is charged once per applicable scope; -- every item is independently verified before it can be sealed; -- limits cover item count, aggregate/per-file bytes, archive expansion, entry - count, path safety, nesting, and compression ratio; -- sealing row-locks the session/items, requires verified replicas, creates a - deterministic manifest/hash, and is idempotent under exact replay; -- cancellation is logical and cannot delete shared completed bytes; -- race tests cover upload/seal/cancel/expiry conflicts, repeated-session quota - exhaustion, concurrent quota admission without oversubscription, terminal - session-slot release by periodic and lazy expiry, retained charges for - cancelled/expired/unbound content, - and first-writer and cross-project digest-key poisoning attempts; -- changed subsystem coverage is at least 90 percent and repository coverage - remains at least 78 percent; -- backend CI installs this chunk's exact focused 90 percent gate, preserves - every earlier scoped 90 percent gate, and retains the exact 78 percent - repository command below; `scripts/test_agent_gates.py` fails on workflow - command, source-set, threshold, or cumulative-retention drift. +- a contributor attempt accepts exactly one outer ZIP and rejects every other + archive form or additional item; +- Workstream streams, sizes, and hashes the exact ZIP while reserving bounded + aggregate scratch through the canonical `ArtifactScratchManager`; +- the outer ZIP tree is walked recursively without `extractall`; ZIP entries + contained inside that tree remain opaque ordinary files by default; +- path normalization rejects absolute/drive/UNC paths, `.`/`..`, NUL/control + characters, backslashes, symlinks/special entries, duplicate normalized paths, + Unicode-normalization collisions, and configured case-fold collisions; +- bounded streaming enforces current upload, extracted-byte, per-file, entry, + path-depth/path-length, compression-ratio, deadline, concurrency, and free-space + limits without allocating from untrusted declared sizes; +- the canonical semantic manifest records normalized file and directory paths, + entry type, and file SHA-256/byte count; timestamps, compression, comments, + ownership, and platform permission metadata do not affect its hash; +- explicit empty directories and deterministic synthetic parents have one + documented canonical representation; +- exact archive hash and semantic manifest hash are compared with the immediate + prior immutable Submission under task/version locking; equality returns a + stable unchanged error before checker or provider I/O; +- the server-computed ZIP commitment is authoritative; an optional client + commitment can only cause early mismatch rejection; +- successful output is a process-local, lease-bound prepared bundle handle that + contains no path/provider reference in an API response and is unusable after + cleanup, expiry, process loss, or generation mismatch; +- that handle is metadata around the existing `ArtifactScratchManager`, + `PreparedArtifact`, and `CommittedArtifactSource` lifecycle, not a second + handle/lease/ledger/quota/cleanup abstraction; +- 04A exposes only internal hidden library seams and tests. It composes no + contributor route until 04C completes the same-process orchestration; +- every old multi-step upload-session/item route and action path is removed or + statically unreachable; no multi-item or provider-result compatibility path + remains; +- process loss requires reupload; normal, error, cancellation, timeout, and + crash cleanup release scratch quota without deleting durable artifacts; +- the route declares only planned action `artifact.submission_bundle.prepare`, + mapped by AUTH to `submission.create`, with ART-owned task/assignment/actor + facts and guards; the old multi-step upload-session actions are not accepted + as aliases or alternate routes; +- hidden behavior remains unavailable until AUTH activates that exact action + only after 04A-04C publish the complete surface/guard manifest; +- focused subsystem coverage is at least 90 percent and repository coverage + remains at least 78 percent. ## Exact CI Coverage Gates @@ -100,8 +98,8 @@ coverage report --include='app/adapters/project_agents/*,app/interfaces/project_ ## Verification ```bash -docker compose up -d --wait postgres redis minio -(cd backend && WORKSTREAM_TEST_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/workstream_test .venv/bin/pytest tests/test_alembic.py tests/test_artifact_upload_api.py -q --cov=app.modules.artifacts --cov-report=term-missing --cov-fail-under=90) +docker compose up -d --wait postgres redis +(cd backend && WORKSTREAM_TEST_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/workstream_test .venv/bin/pytest tests/test_alembic.py tests/test_submission_bundle_intake.py tests/test_submission_zip_safety.py tests/test_submission_bundle_manifest.py tests/test_artifact_scratch_manager.py -q --cov=app.modules.artifacts --cov-report=term-missing --cov-fail-under=90) (metadata_dir="$(mktemp -d)" && trap 'rm -rf "$metadata_dir"' EXIT && (cd backend && WORKSTREAM_TEST_ADMIN_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/postgres .venv/bin/python scripts/run_isolated_tests.py --metadata-json "$metadata_dir/result.json" --timeout-seconds 12600 -- .venv/bin/python -m pytest -q --ignore=tests/test_isolated_database_runner.py --cov=app --cov-report=term-missing --cov-fail-under=78)) (cd backend && .venv/bin/ruff check app tests) python3 scripts/check_stale_artifact_contracts.py @@ -115,6 +113,6 @@ reuse/dedup, CI integrity, test delta, and docs. ## Human Review Focus -- Can a client choose a provider key or bypass independent verification? -- Is the sealed artifact-set commitment deterministic and immutable? -- Are archive and resource-exhaustion boundaries explicit and tested? +- Can more than one outer ZIP or unchecked bytes escape scratch? +- Is recursive inspection clearly the outer archive tree, not nested unpacking? +- Does semantic equality ignore packaging-only changes without ignoring files? diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04B-pre-submit-admission.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04B-pre-submit-admission.md index 3da38010..bbe191ea 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04B-pre-submit-admission.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04B-pre-submit-admission.md @@ -1,4 +1,4 @@ -# Chunk Contract: WS-ART-001-04B Pre-Submit Admission +# Chunk Contract: WS-ART-001-04B Scratch-Bound Pre-Submission Checks Initiative: `WS-ART-001` | Risk: L1 | Status: Proposed after 04A @@ -6,81 +6,71 @@ Artifact contract phase: `upload_admission` ## Goal -Execute the task's locked project pre-submit checker against one sealed -artifact set and persist an exact admission, including bounded continuation -when artifact infrastructure is temporarily unavailable. +Run mandatory Workstream archive gates and the task's locked Project Guide +pre-submission checker against the exact prepared ZIP tree while it remains in +bounded private scratch. Persist checker evidence, but create no durable +artifact content, provider object, or Submission. ## Allowed Files -- pre-submit attempt/admission models and one migration; -- task/checker/artifact input assembly, execution, repository, service, router, - schemas, and stable infrastructure error mapping; -- the canonical artifact scratch manager and one authorized, bounded, - digest-verifying checker materializer shared by both checker phases; -- artifact/task/checker call sites consuming exact decisions already delivered - by the approved WS-AUTH dependency; no Authorization Service owner files; -- focused migration, idempotency, concurrency, outage, checker, and real-API tests; -- `.github/workflows/backend.yml` only to expand the exact 90 percent scoped gate; -- `scripts/test_agent_gates.py` only to assert that backend CI retains this - chunk's exact scoped coverage sources and fail-closed 90 percent threshold; -- `scripts/check_stale_artifact_contracts.py` and its focused tests only to - advance the marker to `upload_admission`; -- related docs and chunk memory. +- ART/checker/task input assembly for one prepared bundle; +- the shared bounded, read-only scratch workspace/materialization capability; +- pre-submit attempt/result control-plane models and one migration; +- hidden task/checker surfaces with ART-owned canonical resource facts/guards; +- focused checker, timeout, cleanup, concurrency, API, coverage, docs, and + chunk-memory changes; +- CI files only to add and preserve the exact scoped 90 percent gate. ## Not Allowed -- submission creation or consumption of an admission; -- post-submit execution or review routing; -- any storage outage translated into checker failure, `needs_revision`, reject, - payment, contribution, or reputation state; -- Project Manager or Operator approval to continue a contributor precheck; -- client policy/checker/provider overrides. +- provider I/O, candidate storage, `ArtifactContent`, replica, or binding; +- passing admission publication or Submission creation; +- post-submit routing or Review/Contribution/delivery changes; +- checker interpretation as `accept`, `needs_revision`, or `reject`; +- manager/operator approval for an ordinary contributor retry; +- AUTH-owned activation changes or larger configured limits. ## Acceptance Criteria -- authoritative pre-submit executes the task's locked project checker against - the exact sealed artifact-set hash; -- checker workspace creation declares - `artifact.pre_submit.checker_input.materialize`, mapped to the fixed service - permission `artifact.checker_input.materialize`; contributor upload - permissions do not authorize materialization; -- pre-submit passes a provider-neutral materialization request for the sealed - upload artifact set and resolves only its verified `ready` items; it does not - require or create a product binding before submission creation. Post-submit - later uses the same materializer with immutable binding IDs; -- the shared materializer reserves its complete checker workspace in the - canonical scratch ledger and recomputes every SHA-256 and byte count while - reading the exact provider bytes; -- a mismatch fails closed as an artifact incident before checker execution; - verified files are sealed read-only before the checker receives the workspace - handle, and the handle cannot expose provider references or credentials; -- the canonical manager owns aggregate quota, private/no-follow paths, normal - completion and cancellation cleanup, crash/stale cleanup, and fail-closed - exhaustion for both source preparation and checker workspace allocations; -- a passing admission binds actor, task, session, artifact set, locked guide/ - policy/checker context, canonical request digest, checker result, and expiry; -- checker failure returns structured results and creates no admission, - submission, product review decision, or contributor outcome; -- Workstream owns a bounded transient retry budget for artifact reads; -- exhausted transient storage failure returns HTTP 503 with stable code - `pre_submission_infrastructure_unavailable`, not checker failure; -- on infrastructure unavailability the sealed session and artifact set remain - sealed, unconsumed, and reusable until normal expiry; only infrastructure - attempt and audit facts are persisted; -- idempotency scope includes actor, task, sealed session/artifact set, locked - context, client key, and canonical request digest; exact replay continues or - returns the same attempt, changed replay conflicts, and concurrent replay - creates no duplicate admission or attempt; -- after retry-after or infrastructure recovery, the contributor can continue - the same exact attempt without manager/operator approval; -- changed subsystem coverage is at least 90 percent and repository coverage - remains at least 78 percent; -- backend CI installs this chunk's exact focused 90 percent gate, preserves - every earlier scoped 90 percent gate, and retains the exact 78 percent - repository command below; `scripts/test_agent_gates.py` fails on workflow - command, source-set, threshold, or cumulative-retention drift; -- the stale-contract phase advances to `upload_admission` only after the sealed - artifact admission path is active and the phase scan passes. +- platform-owned format, safety, manifest, change, and scratch-integrity gates + cannot be disabled or weakened by a Project Guide; +- the project checker uses the exact task-locked guide/policy/checker context and + may require content such as `task.toml`, directories, evidence, configuration, + or tests from the safely materialized outer-ZIP tree; +- project policy may narrow platform limits but cannot raise them; +- checker input records the archive commitment, manifest ID/hash, task, actor, + locked policy context, and scratch generation used; +- durable attempt/evidence records are actor/task/attempt scoped, contain no raw + bytes, filenames, scratch/provider references, credentials, or unbounded + checker output, and follow the existing audit retention policy; +- v0.1 exposes no ID-addressed pre-submit-evidence read route. The authorized + contributor receives only the bounded redacted result of the same continuous + request; existing authorized audit/Operator surfaces may diagnose metadata, + while reviewers, CON, and delivery cannot consume it before Submission binding; +- the checker receives a read-only private workspace and no provider reference, + credential, mutable intake path, or arbitrary host path; +- checker workspace creation uses the distinct fixed service action + `artifact.checker_input.materialize`; contributor preparation authority never + grants or substitutes for that service authority; +- a checker failure creates structured findings only: no durable artifact, + admission, Submission, Review decision, contribution, payment, reputation, or + provider charge; +- infrastructure failure remains a stable retryable infrastructure outcome and + never becomes contributor blame or product review state; +- successful checks create a short-lived, single-use passing result bound to the + same scratch handle and exact predecessor observed in 04A; they do not make + bytes authoritative or durable; +- 04B remains an internal hidden library seam with no separately invocable + contributor route; 04C composes the one continuous same-process endpoint; +- the complete operation remains process-local through 04C; process loss or + scratch loss invalidates the result and requires reupload rather than routing + a later distributed worker to a local path; +- completion, failure, cancellation, deadline, and abandoned-work cleanup are + bounded and idempotent; +- the continuous contributor surface remains hidden; this chunk does not + activate the planned contributor action or change fixed service grants; +- focused subsystem coverage is at least 90 percent and repository coverage + remains at least 78 percent. ## Exact CI Coverage Gates @@ -103,8 +93,8 @@ coverage report --include='app/modules/checkers/*' --precision=2 --fail-under=90 ## Verification ```bash -docker compose up -d --wait postgres redis minio -(cd backend && WORKSTREAM_TEST_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/workstream_test .venv/bin/pytest tests/test_alembic.py tests/test_submission_precheck.py tests/test_artifact_precheck_outage.py tests/test_checker_materialization.py tests/test_artifact_scratch_manager.py -q --cov=app.modules.artifacts --cov=app.modules.tasks --cov=app.modules.checkers --cov-report=term-missing --cov-fail-under=90) +docker compose up -d --wait postgres redis +(cd backend && WORKSTREAM_TEST_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/workstream_test .venv/bin/pytest tests/test_alembic.py tests/test_submission_precheck.py tests/test_submission_precheck_scratch.py tests/test_checker_materialization.py tests/test_artifact_scratch_manager.py -q --cov=app.modules.artifacts --cov=app.modules.checkers --cov-report=term-missing --cov-fail-under=90) (metadata_dir="$(mktemp -d)" && trap 'rm -rf "$metadata_dir"' EXIT && (cd backend && WORKSTREAM_TEST_ADMIN_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/postgres .venv/bin/python scripts/run_isolated_tests.py --metadata-json "$metadata_dir/result.json" --timeout-seconds 12600 -- .venv/bin/python -m pytest -q --ignore=tests/test_isolated_database_runner.py --cov=app --cov-report=term-missing --cov-fail-under=78)) (cd backend && .venv/bin/ruff check app tests) python3 scripts/check_stale_artifact_contracts.py @@ -118,8 +108,6 @@ reuse/dedup, CI integrity, test delta, and docs. ## Human Review Focus -- Is the exact staged payload bound to one admission? -- Can infrastructure failure ever become contributor blame or product state? -- Are retry and idempotency semantics exact under concurrency and replay? -- Does pre-submit execute only the read-only bytes whose digest and size it - recomputed through the shared materializer? +- Can a failed or unchecked bundle reach object storage? +- Can an async/distributed worker receive an unusable local scratch path? +- Are checker findings separate from reviewer decisions? diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04C-verified-submission-bundle-admission.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04C-verified-submission-bundle-admission.md new file mode 100644 index 00000000..f95f2231 --- /dev/null +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04C-verified-submission-bundle-admission.md @@ -0,0 +1,105 @@ +# Chunk Contract: WS-ART-001-04C Verified Submission Bundle Admission + +Initiative: `WS-ART-001` | Risk: L1 | Status: Proposed after 04B + +Artifact contract phase: `upload_admission` + +## Goal + +Consume one passing scratch-bound pre-submit result, write the exact outer ZIP +once through the existing immutable `ArtifactStore` admission path, independently +verify it, and publish one bindable submission-bundle admission. + +## Allowed Files + +- ART admission orchestration and one submission-bundle admission model/migration; +- existing put-attempt, replica, receipt, verification-job, recovery, scanner, + and scratch-source integration needed for this producer; +- hidden submission-bundle admission surfaces and ART resource facts/guards; +- focused ambiguity, idempotency, verification, cleanup, API, coverage, docs, + and chunk-memory changes; +- CI files only to add and preserve the exact scoped 90 percent gate. + +## Not Allowed + +- a second provider namespace, candidate store, promotion/copy, retention + window, physical deletion, or new recovery aggregate; +- Submission creation or binding consumption; +- Review, Contribution, compensation, reputation, or delivery behavior; +- AUTH-owned activation changes or larger configured limits. + +## Acceptance Criteria + +- admission requires the exact unexpired passing result, prepared ZIP + commitment, semantic manifest, locked task context, actor, and predecessor; +- the predecessor is rechecked under task/version locking before provider I/O; + drift invalidates the attempt and requires a fresh upload/check; +- the existing generic admission service derives and reserves authoritative + byte scopes before one conditional `ArtifactStore.put`; +- Workstream persists the put attempt before I/O and uses existing observation, + receipts, verification jobs, scanners, generation fencing, and recovery; +- background verification, pending-work publication, and ambiguous-put + resolution retain their distinct fixed service actions + `artifact.verification.execute`, `artifact.pending_work.scan`, and + `artifact.put_attempt.resolve`; contributor preparation authority implies none; +- provider acknowledgement alone creates no bindable admission; +- complete provider read-back recomputes ZIP SHA-256 and byte count; only an + exact match publishes `ArtifactContent`, a ready replica, and one immutable + admission bound to the manifest and checker evidence; +- provider absence, mismatch, ambiguity, or unavailability creates no + Submission and never becomes checker failure or review state; +- exact replay is idempotent and concurrent replay creates one provider/business + effect; no `ArtifactOutboxRecoveryAttempt` or receipt-lookup port is added; +- scratch is cleaned after safe source handoff or terminal outcome; if process + loss occurs before durable put intent, reupload is required; after durable + intent, existing ART recovery resolves the operation; +- unbound verified content can result only from a transaction/process failure + after successful admission and is resolved through exact idempotent retry; + v0.1 adds no deletion path; +- after 04A-04C merge their complete hidden manifest, AUTH alone may activate + `artifact.submission_bundle.prepare`; ART changes no action availability; +- this chunk alone composes the hidden contributor endpoint across the 04A + intake, 04B checks, and 04C durable handoff in one process-local operation; + no scratch handle/path is serialized into a later request or Celery payload; +- focused subsystem coverage is at least 90 percent and repository coverage + remains at least 78 percent. + +## Exact CI Coverage Gates + +```bash +coverage report --include='app/adapters/artifacts/*,app/core/cancellation.py,app/core/file_locks.py,app/interfaces/artifact_operations.py,app/interfaces/artifacts.py,app/modules/artifacts/*' --precision=2 --fail-under=90 +coverage report --include='app/interfaces/external_services.py' --precision=2 --fail-under=90 +coverage report --include='app/core/config.py' --precision=2 --fail-under=90 +coverage report --include='app/workers/*' --precision=2 --fail-under=90 +coverage report --include='app/main.py' --precision=2 --fail-under=90 +coverage report --include='app/adapters/artifacts/s3_compatible.py' --precision=2 --fail-under=90 +coverage report --include='app/core/s3_validation.py' --precision=2 --fail-under=90 +coverage report --include='app/modules/audit/*' --precision=2 --fail-under=90 +coverage report --include='app/api/router.py' --precision=2 --fail-under=90 +coverage report --include='app/modules/projects/*' --precision=2 --fail-under=90 +coverage report --include='app/adapters/project_agents/*,app/interfaces/project_agents.py' --precision=2 --fail-under=90 +coverage report --include='app/modules/tasks/*' --precision=2 --fail-under=90 +coverage report --include='app/modules/checkers/*' --precision=2 --fail-under=90 +``` + +## Verification + +```bash +docker compose up -d --wait postgres redis minio +(cd backend && WORKSTREAM_TEST_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/workstream_test .venv/bin/pytest tests/test_alembic.py tests/test_submission_bundle_admission.py tests/test_artifact_put_resolution.py tests/test_artifact_verification.py tests/test_artifact_recovery.py -q --cov=app.modules.artifacts --cov-report=term-missing --cov-fail-under=90) +(metadata_dir="$(mktemp -d)" && trap 'rm -rf "$metadata_dir"' EXIT && (cd backend && WORKSTREAM_TEST_ADMIN_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/postgres .venv/bin/python scripts/run_isolated_tests.py --metadata-json "$metadata_dir/result.json" --timeout-seconds 12600 -- .venv/bin/python -m pytest -q --ignore=tests/test_isolated_database_runner.py --cov=app --cov-report=term-missing --cov-fail-under=78)) +(cd backend && .venv/bin/ruff check app tests) +python3 scripts/check_stale_artifact_contracts.py +python3 scripts/test_agent_gates.py +``` + +## Required Reviewers + +Senior engineering, architecture, QA/test, security/auth, product/ops, +reuse/dedup, CI integrity, test delta, and docs. + +## Human Review Focus + +- Is the exact checked ZIP written once and independently verified? +- Does every ambiguous effect reuse existing ART recovery? +- Can a Submission be created before bindable verification? diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-05-submission-artifact-cutover.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-05-submission-artifact-cutover.md index c66fc262..609d4c11 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-05-submission-artifact-cutover.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-05-submission-artifact-cutover.md @@ -1,91 +1,63 @@ -# Chunk Contract: WS-ART-001-05 Submission Artifact Cutover +# Chunk Contract: WS-ART-001-05 Submission Bundle Binding Cutover -Initiative: `WS-ART-001` | Risk: L1 | Status: Proposed after 04B +Initiative: `WS-ART-001` | Risk: L1 | Status: Proposed after 04C and exact AUTH activation Artifact contract phase: `submission_cutover` ## Goal -Make submission creation atomically consume the exact passing admission and -sealed artifact set, create immutable submission bindings, and automatically -enter `evaluation_pending` without manager finalization. +Atomically consume one verified submission-bundle admission, create the next +immutable `Submission` row and exact artifact binding, and remove caller-owned +package transport/hash/manifest authority. ## Allowed Files -- submission/task models, migration, repository, schemas, service, and router -- artifact admission consumption and binding service -- project policy/compiler and checker contract files only to remove the legacy - caller storage-scheme, manifest/hash, and compiler-primitive fields owned by - this clean cut -- Celery post-submit dispatch handoff -- task/artifact call sites consuming exact decisions already delivered by the - approved WS-AUTH dependency; no Authorization Service owner files -- focused migration, race, API, and real-flow tests -- `.github/workflows/backend.yml` only to expand the exact 90 percent scoped gate -- `scripts/test_agent_gates.py` only to assert the exact workflow command, - source set, threshold, and cumulative retention -- `backend/scripts/week2_api_e2e.py` to migrate its complete upload, - pre-submit, and submission flow from legacy transport fields and the removed - `/finalize` route -- `backend/scripts/api_contract_e2e.py` only to migrate project-policy, - submission request, and response assertions from legacy storage fields to - upload-session/admission semantics -- `examples/terminal_benchmark/terminal_benchmark_api_e2e.py` only to migrate - its private local fixture and manual API example from removed transport and - storage-policy fields; this sanitized local example is not CI or acceptance - proof, and Chunk 07 owns standalone executable proof -- `scripts/check_stale_artifact_contracts.py` and its focused tests only to - advance the marker to `submission_cutover` -- related docs/chunk memory +- task/submission models, migration, repository, schemas, service, and router; +- ART admission consumption and binding capability; +- project/checker contracts only for the exact legacy transport-field cutover; +- post-submit dispatch handoff and affected real-API examples; +- `backend/scripts/api_contract_e2e.py` only to migrate the exact submission + request/response flow to verified submission-bundle admission; +- focused migration, concurrency, history, API, coverage, docs, and chunk memory; +- CI files only to add and preserve the exact scoped 90 percent gate. ## Not Allowed -- reviewer decision/revision implementation; -- manager/admin finalization of ordinary contributor submissions; -- caller package URI, package hash, artifact manifest, evidence IDs, provider - reference, or artifact-set hash; +- a competing `SubmissionVersion` table or alias; +- reviewer decision/note implementation, reviewer-uploaded revision files, or + Review/Contribution/delivery ownership; +- caller package URI/hash/manifest/evidence IDs, provider references, content + IDs, or artifact-set hashes; - direct provider calls from task/submission modules; -- checker-result interpretation as a human review decision. -- Authorization Service implementation, permission registration, or policy - changes. +- AUTH-owned activation changes or checker results interpreted as review. ## Acceptance Criteria -- immutable submission binding creation declares - `artifact.submission.binding.create`, mapped to the fixed service permission - `artifact.binding.create`; submission creation authority does not imply it; - -- request body contains only summary, contributor attestation, and upload - session ID. -- assigned contributor can create their own submission after exact pre-submit - admission passes; no manager action is required. -- one transaction locks and consumes admission/session/items once, allocates - server submission version, creates immutable bindings, stamps exact locked - policy/checker/artifact-set context, and transitions to - `evaluation_pending`. -- the exact payload checked is the payload submitted; changed summary, - attestation, task, actor, policy, checker, session, or artifact set fails. -- automatic post-submit dispatch is durable and duplicate-safe. -- concurrent/replayed creation yields one submission/business effect. -- the misleading `/submissions/{id}/finalize` route is replaced by the exact - Operator-only `/operator/submissions/{id}/pre-review-gate-repair` command; - existing API-only drill consumers move in the same clean cut. -- pre-review-gate repair requires exact authorization, reason, audit, and a - failed/stale automatic dispatch; it cannot re-finalize healthy submissions or - create another checker run. -- storage failure leaves infrastructure state and never blames contributor. -- no live project/task/checker contract retains the removed caller transport, - storage-policy, or compiler-primitive fields; immutable Alembic history and - test fixture literals are not rewritten merely to satisfy wording scans. -- changed subsystem coverage is at least 90 percent and repository coverage - does not decrease. -- backend CI installs this chunk's exact focused 90 percent gate, preserves - every earlier scoped 90 percent gate, and retains the exact 78 percent - repository command below; `scripts/test_agent_gates.py` fails on workflow - command, source-set, threshold, or cumulative-retention drift. -- the stale-contract phase advances to `submission_cutover`; active docs and - project/task/checker contracts contain no caller-owned provider URI - references and the phase scan passes. +- one `Submission` row is one immutable version and binds exactly one verified + outer ZIP plus its canonical semantic manifest and checker evidence; +- request body contains only contributor-authored product fields and the + opaque verified admission identifier; all artifact identity is server-owned; +- one transaction locks task/assignment/admission, rechecks predecessor and + locked context, consumes admission once, allocates the next version, creates + the immutable binding, and enters the existing post-submit lifecycle; +- binding creation uses fixed service action `artifact.binding.create`; the + contributor's active `artifact.submission_bundle.prepare` action and + `submission.create` permission do not imply internal binding authority; +- version `N+1` records `supersedes_submission_id = N`; the exact relationship + to a `needs_revision` Review is added only through the reviewed REV/TASK joint + contract and cannot be fabricated from a note or client ID; +- exact replay/concurrency creates one Submission and one business effect; +- latest/current/accepted access uses indexed repository queries or projections, + while the immutable version/review graph remains fully queryable; +- no manager finalization is required for an ordinary passing contributor + submission, and infrastructure failure never blames the contributor; +- all active callers and docs lose legacy package URI, caller hash, and caller + artifact-manifest authority without fabricated backfill for historical rows; +- migration explicitly preserves readable historical rows, refuses any unsafe + authoritative backfill, and documents rollback/cutover ordering; +- hidden behavior remains unavailable until AUTH activates its exact actions; +- focused subsystem coverage is at least 90 percent and repository coverage + remains at least 78 percent. ## Exact CI Coverage Gates @@ -109,11 +81,9 @@ coverage report --include='app/modules/checkers/*' --precision=2 --fail-under=90 ```bash docker compose up -d --wait postgres redis minio -(cd backend && WORKSTREAM_TEST_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/workstream_test .venv/bin/pytest tests/test_alembic.py tests/test_submission_api.py tests/test_submission_concurrency.py tests/test_projects.py tests/test_checkers.py -q --cov=app.modules.tasks --cov=app.modules.artifacts --cov=app.modules.projects --cov=app.modules.checkers --cov=app.adapters.project_agents --cov=app.interfaces.project_agents --cov=app.workers --cov-report=term-missing --cov-fail-under=90) +(cd backend && WORKSTREAM_TEST_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/workstream_test .venv/bin/pytest tests/test_alembic.py tests/test_submission_api.py tests/test_submission_concurrency.py tests/test_submission_history.py tests/test_projects.py tests/test_checkers.py -q --cov=app.modules.tasks --cov=app.modules.artifacts --cov=app.modules.projects --cov=app.modules.checkers --cov-report=term-missing --cov-fail-under=90) (metadata_dir="$(mktemp -d)" && trap 'rm -rf "$metadata_dir"' EXIT && (cd backend && WORKSTREAM_TEST_ADMIN_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/postgres .venv/bin/python scripts/run_isolated_tests.py --metadata-json "$metadata_dir/result.json" --timeout-seconds 12600 -- .venv/bin/python -m pytest -q --ignore=tests/test_isolated_database_runner.py --cov=app --cov-report=term-missing --cov-fail-under=78)) -(cd backend && WORKSTREAM_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/workstream_test .venv/bin/python scripts/week2_api_e2e.py) (cd backend && WORKSTREAM_DATABASE_URL=postgresql+asyncpg://workstream:workstream@localhost:5433/workstream_test .venv/bin/python scripts/api_contract_e2e.py) -(cd backend && .venv/bin/python -m py_compile ../examples/terminal_benchmark/terminal_benchmark_api_e2e.py) (cd backend && .venv/bin/ruff check app tests) python3 scripts/check_stale_artifact_contracts.py python3 scripts/test_agent_gates.py @@ -126,8 +96,6 @@ reuse/dedup, CI integrity, test delta, and docs. ## Human Review Focus -- Does contributor intent produce and lock exactly one submission version? -- Is all manager finalization removed from the normal path? -- Is pre-submit payload binding authoritative and race-safe? -- Are all active API callers migrated without treating the private local - Terminal Benchmark example as reproducible proof? +- Does exactly one checked and verified ZIP become exactly one Submission? +- Is the current `Submission` aggregate preserved without a duplicate model? +- Is historical migration honest and fail closed? diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-06A-checker-input-materialization.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-06A-checker-input-materialization.md index 85147cf3..1a4d8002 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-06A-checker-input-materialization.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-06A-checker-input-materialization.md @@ -37,14 +37,15 @@ bounded isolated checker workspaces. ## Acceptance Criteria -- `CheckerInputSnapshot` commits to submission version, artifact set, exact - binding/content IDs, hashes/sizes, locked policy/checker versions, and checker +- `CheckerInputSnapshot` commits to submission version, submission-bundle + manifest, exact binding/content IDs, hashes/sizes, locked policy/checker versions, and checker implementation identity; -- pre-submit and post-submit input prove the same artifact-set hash; +- pre-submit evidence and post-submit input prove the same archive commitment, + semantic-manifest hash, and exact binding; - the post-submit runner receives only authorized immutable Workstream binding - IDs and reuses the canonical materializer that pre-submit invoked with a - sealed-ready upload-set source; both are closed forms of the same typed - materialization request; + IDs and reuses the same scratch-manager/materialization primitives without + accepting a pre-submit scratch handle; pre-submit was process-local and + post-submit is binding-only; - the fixed checker service principal declares `artifact.post_submit.checker_input.materialize`, mapped to `artifact.checker_input.materialize`; it does not authorize any binding-create diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-07-recovery-live-proof.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-07-recovery-live-proof.md index adeb5ec1..a023129b 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-07-recovery-live-proof.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-07-recovery-live-proof.md @@ -39,9 +39,11 @@ without direct database inspection or Terminal Benchmark product coupling. - a standalone project guide is ingested, verified, snapshotted, and processed through setup using real APIs and visible async status. -- a contributor uploads exact artifacts, seals them, runs pre-submit, creates a - submission, and automatically enters `evaluation_pending`. -- post-submit reads the same immutable artifact-set commitment and stores +- a contributor uploads one outer ZIP; Workstream inspects and checks it in + bounded scratch, admits and verifies it once, creates a submission, and + automatically enters `evaluation_pending`. +- post-submit reads the same immutable binding/archive/semantic-manifest + commitment and stores checker logs/outputs as verified artifacts. - post-submit remains `evaluation_pending` while execution or infrastructure retry is active. Durable completion preserves the existing checker-owned @@ -50,9 +52,9 @@ without direct database inspection or Terminal Benchmark product coupling. assignment, or decision. - provider outage produces infrastructure pending/retry, not contributor or review outcomes; restart recovers through the periodic scanner. -- pre-submit outage exhausts to `pre_submission_infrastructure_unavailable`, - preserves the exact sealed unconsumed attempt, and continues on contributor - retry without manager/operator approval; post-submit outage separately keeps +- pre-submit process/scratch loss requires reupload without manager/operator + approval; durable-admission ambiguity is resolved by existing ART recovery; + post-submit outage separately keeps `evaluation_pending` and uses checker retry infrastructure. - changed/truncated/missing bytes and stale/duplicate Celery executions are proven safely. diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-PLAN2-submission-bundle-reconciliation.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-PLAN2-submission-bundle-reconciliation.md new file mode 100644 index 00000000..3305a9ed --- /dev/null +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-PLAN2-submission-bundle-reconciliation.md @@ -0,0 +1,122 @@ +# Chunk Contract: WS-ART-001-PLAN2 Submission Bundle Reconciliation + +Initiative: `WS-ART-001` | Risk: L1 | Status: Planning only + +## Goal + +Reconcile the remaining artifact initiative with current `main` and lock the +smallest v0.1 path that proves one contributor ZIP is safely inspected in +bounded private scratch, checked under the task's locked Project Guide, admitted +once through the existing immutable `ArtifactStore`, and referenced unchanged +by the resulting Submission and every downstream consumer. + +This chunk changes planning and contracts only. It does not activate ART, +AUTH, task, checker, review, contribution, or delivery behavior. + +## Allowed Files + +- WS-ART-001 intent, discovery, plan, decisions, risks, status, chunk map, and + remaining unimplemented chunk contracts; +- the WS-ART-001 authorization handoff naming exact registration, hidden + behavior, activation, fixed-service separation, and stop conditions; +- the canonical artifact-storage specification, shared glossary, and one + availability-neutral authorization-spec clarification needed to remove stale + multi-step upload guidance without changing AUTH runtime ownership; +- submission artifact-policy and submission-packet templates needed to replace + caller hash/manifest and multi-step session guidance; +- ART planning review evidence and PR trust bundle; +- `scripts/test_agent_gates.py` only to replace stale ART status/chunk-map + assertions with the exact merged/cancelled planning projection, bind new + successor phases/coverage contracts, and preserve the signed-automation-only + authority warning instead of treating copied root projections as live state; +- cross-initiative handoff prose that names dependencies without modifying + another initiative's owned runtime contract; +- exactly one merge intent for this planning chunk. + +## Not Allowed + +- backend, frontend, migration, workflow, provider, or deployment changes; +- Authorization Service catalogue, evaluator, grant, identity, or activation + changes; +- Review, Contribution, compensation, reputation, or delivery implementation; +- candidate/quarantine object storage, temporary provider namespaces, physical + deletion, retention windows, or a second artifact recovery aggregate; +- larger upload, expansion, scratch, entry-count, or duration limits without + separately reviewed capacity evidence; +- starting any successor automatically. + +## Acceptance Criteria + +- every Submission version accepts exactly one outer ZIP; its contents are + governed by the locked Project Guide; +- Workstream recursively walks the normalized directory/file tree represented + by the outer ZIP, while a ZIP entry inside that tree remains an ordinary file; + nested archive unpacking is explicitly outside v0.1; +- bounded private scratch is the only pre-admission custody; failed, unsafe, + unchanged, abandoned, or checker-failing attempts never enter object storage; +- the canonical semantic manifest commits to normalized file/directory paths, + entry type, and each file's SHA-256 and byte count while excluding packaging, + timestamps, ownership, compression, and platform permission metadata; +- exact archive equality and manifest equality are compared with the immediate + prior immutable Submission and both reject before provider I/O; +- all mandatory platform gates and locked project pre-submit checks pass before + the existing `ArtifactStore` admission path is called; +- passing bytes are written once, independently read back, verified, published + as `ArtifactContent`, and bound to the Submission through existing ART + identity, put-attempt, verification-job, receipt, and recovery abstractions; +- the plan creates no candidate store, promotion, retention policy, duplicate + provider write, `ArtifactOutboxRecoveryAttempt`, or provider delete path; +- process loss before durable admission requires reupload; an ambiguous durable + put uses existing observation/recovery and never creates a Submission until + verified content is bindable; +- current conservative configured limits remain unchanged; +- the current `Submission` row remains the immutable version aggregate; a new + version links through `supersedes_submission_id` and, when responding to + `needs_revision`, through an exact review relationship owned jointly with + REV rather than a competing `SubmissionVersion` table; +- reviewer decision values remain exactly `accept`, `needs_revision`, and + `reject`; reviewer note/findings attach to the exact Submission and contain + no reviewer-uploaded revision artifact; +- normal reads use indexed latest/current/accepted projections while immutable + Submission/Review relationships preserve complete history; +- checker materialization and downstream reviewer/delivery streaming resolve + the exact binding and recompute full SHA-256 and byte count while streaming; +- ART owns identity, integrity, manifests, binding, and byte access; AUTH, REV, + CON, and delivery consume explicit capabilities through their own reviewed + contracts and activation sequence; +- the obsolete multi-step upload-session action plan is not reused blindly: + AUTH first registers one planned contributor action + `artifact.submission_bundle.prepare` mapped to `submission.create`, ART-04A-C + then implement one hidden continuous orchestration, and AUTH alone activates + that action after exact manifest/guard proof; existing fixed service actions + remain distinct for verification, pending-work scanning, put resolution, + checker materialization, and binding; +- every implementation successor is PR-sized, names exact allowed/forbidden + files, and requires a separate signed start. + +## Verification + +```bash +git diff --check +python3 scripts/check_stale_artifact_contracts.py +python3 scripts/test_agent_gates.py +python3 scripts/check_markdown_links.py +rg -n "candidate storage|seven-day|7 days|ArtifactOutboxRecoveryAttempt|SubmissionVersion table" .agent-loop/initiatives/WS-ART-001-immutable-artifact-storage docs +``` + +The wording scan may match an explicit rejection in this planning contract or +decision log; it must not find an active design that implements those concepts. + +## Required Reviewers + +Senior engineering, architecture, QA/test, security/auth, product/ops, +reuse/dedup, CI integrity, test delta, and docs. + +## Human Review Focus + +- Is one outer ZIP the only contributor submission boundary? +- Can any failed or unchecked payload reach immutable object storage? +- Is scratch clearly ephemeral rather than a second artifact store? +- Does the plan preserve exact bytes across checks, review, acceptance, + contribution, and delivery without moving ownership into ART? +- Are AUTH activation and cross-initiative handoffs explicit and fail closed? diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-internal-review-evidence.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-internal-review-evidence.md new file mode 100644 index 00000000..92325b30 --- /dev/null +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-internal-review-evidence.md @@ -0,0 +1,55 @@ +# Internal Review Evidence: WS-ART-001-PLAN2 + +## Candidate + +Planning-only reconciliation after signed cancellation of `WS-ART-001-03`. +No backend, frontend, migration, workflow, provider, or deployment behavior is +implemented or activated. + +## Deterministic Evidence + +- `git diff --check` — PASS +- `python3 scripts/check_stale_artifact_contracts.py` — PASS at + `artifact_store_cutover` +- `python3 scripts/check_markdown_links.py` — PASS +- `python3 scripts/test_agent_gates.py` — PASS, 100 tests +- targeted stale wording scan — only explicit rejected/legacy/unavailable + references remain + +## Reviewer Results + +| Track | Result | Resolved findings | +|---|---|---| +| Senior engineering | PASS after repair | Reconciled status/map history, completed 03A-C contracts, made 04A-C internal composition executable, restored isolated coverage/e2e proof. | +| Architecture | PASS after repair | Removed duplicate session/candidate paths, reused existing scratch/admission/recovery, and made cross-subsystem ownership explicit. | +| QA/test | PASS after repair | Added exact verification commands, 78/90 percent gates, nested-archive closure, process-loss behavior, and migration/history proof. | +| Security/auth | PASS after repair | Added exact AUTH handoff, fixed-service separation, pre-submit evidence privacy, and fail-closed scratch/provider boundaries. | +| Product/ops | PASS after repair | Corrected merged lineage, reviewer note/findings semantics, and Submission/revision/downstream ownership. | +| Reuse/dedup | PASS after repair | Reused `ArtifactScratchManager`, `PreparedArtifact`, `CommittedArtifactSource`, and existing admission/put/verification/recovery paths. | +| CI integrity | PASS | No workflow weakening; cumulative scoped 90 percent and repository 78 percent contracts remain enforced. | +| Test delta | PASS after repair | Replaced stale authored-state assertions without treating root projections as canonical; new phases and e2e ownership remain guarded. | +| Docs | PASS after repair | Reconciled storage/auth specs, glossary, artifact-policy template, and submission-packet template. | + +## Material Repairs + +- removed candidate storage, retention, promotion, duplicate provider writes, + physical deletion, and a second recovery aggregate; +- locked one outer ZIP, safe outer-tree inspection, opaque nested archives, + exact archive identity, and canonical semantic-manifest identity; +- kept failed/unchecked bytes only in bounded process-local scratch; +- split guide cutover into 03A/03B/03C and submission work into + 04A/04B/04C/05; +- retired the future multi-step session design behind an exact AUTH-owned + registration/activation handoff; +- preserved existing immutable `Submission` as the version aggregate and REV's + decision/note ownership; +- required integrity recomputation for checker, reviewer, and delivery streams. + +## Accepted Risks + +- Current AUTH catalogue/gate rows for upload-session actions remain as exact + trusted-main planned/unavailable discovery state until the separately reviewed + AUTH registration contract retires them. They have no route or activation and + block ART-04A until reconciled. +- Process loss before durable artifact intent requires contributor reupload in + v0.1; no temporary provider custody is introduced to mask that tradeoff. diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-pr-trust-bundle.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-pr-trust-bundle.md new file mode 100644 index 00000000..e0dc9bc8 --- /dev/null +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-pr-trust-bundle.md @@ -0,0 +1,56 @@ +# PR Trust Bundle: WS-ART-001-PLAN2 + +## Intent + +Replace the rejected combined ART-03/future upload design with a bounded plan +that proves the same contributor ZIP is safely checked, immutably stored, +reviewed, accepted, recorded, and delivered. + +## Design + +```text +one outer ZIP +-> bounded process-local scratch +-> safe outer-tree inspection and canonical manifest +-> exact/semantic unchanged rejection +-> mandatory platform and locked-guide checks +-> one existing ArtifactStore admission +-> complete read-back verification +-> one immutable Submission binding +``` + +Nested archives remain opaque in v0.1. Failed or unchecked bytes never enter +object storage. Existing admission, put-attempt, verification, receipt, scanner, +and recovery abstractions are reused. + +## Scope + +Planning, successor contracts, canonical artifact documentation/templates, and +the exact agent-gate assertions required by the new phase map. No runtime code, +migration, provider, workflow, or authorization availability changes. + +Exactly one merge intent is added: +`.agent-loop/merge-intents/WS-ART-001-PLAN2.json`, naming only same-initiative +successor `WS-ART-001-03A` with a separate explicit start. + +## Evidence + +- diff, stale-contract, markdown-link, and 100 agent-gate tests pass; +- all nine required reviewer tracks pass after repair; +- detailed findings and resolutions are recorded in + `WS-ART-001-PLAN2-internal-review-evidence.md`. + +## Human Review Focus + +- Confirm one outer ZIP and no candidate retention are the intended v0.1 UX. +- Confirm process loss before durable intent requiring reupload is acceptable. +- Confirm AUTH must retire planned session actions and register/activate the + exact bundle action before ART-04A/05 can proceed. +- Confirm ART owns bytes/identity/access capabilities while REV, CON, and + delivery retain their lifecycle decisions. + +## Gate + +This planning merge starts no implementation. `WS-ART-001-03A` requires a +fresh signed explicit start after human merge approval and automated merge +memory reconciliation. diff --git a/.agent-loop/merge-intents/WS-ART-001-PLAN2.json b/.agent-loop/merge-intents/WS-ART-001-PLAN2.json new file mode 100644 index 00000000..679f1f70 --- /dev/null +++ b/.agent-loop/merge-intents/WS-ART-001-PLAN2.json @@ -0,0 +1,9 @@ +{ + "chunk_id": "WS-ART-001-PLAN2", + "chunk_title": "Submission Bundle Reconciliation", + "initiative_id": "WS-ART-001", + "next_chunk_id": "WS-ART-001-03A", + "next_chunk_title": "Guide Source Byte Ingest", + "next_requires_explicit_start": true, + "schema_version": 2 +} diff --git a/docs/glossary.md b/docs/glossary.md index 63c74a01..f98df4d7 100644 --- a/docs/glossary.md +++ b/docs/glossary.md @@ -68,11 +68,13 @@ actor IDs or grants. The active-or-revoked link between one canonical issuer/opaque subject and one ActorProfile. Raw tokens, provider credentials, and full claims are not stored. -## ArtifactUploadSession +## SubmissionBundlePreparation -The task- or guide-scoped operational staging record that authorizes bounded -artifact upload, sealing, expiry, and single-use consumption. It is not a -submission and grants no review authority. +The bounded process-local operation that receives one contributor outer ZIP, +inspects and checks its internal tree in private scratch, and hands passing bytes +once to immutable artifact admission. It is not durable candidate storage, a +Submission, or review authority; process loss before durable intent requires +reupload. ## ArtifactContent @@ -82,10 +84,9 @@ protocol observations are replica details, not this record's identity. ## ArtifactUploadItem -The per-item mutable upload-operation ledger inside an -`ArtifactUploadSession`. It owns byte reservation, logical role, scoped -idempotency, request digest, CAS state, opaque `provider_object_ref`, and the -resulting `ArtifactContent`. It is not an `ArtifactBinding`. +Legacy unavailable multi-item staging metadata. ART-04A must remove it or make +it statically unreachable before submission-bundle preparation activates. It is +not the v0.1 contributor intake, `ArtifactContent`, or an `ArtifactBinding`. ## ArtifactBinding @@ -105,20 +106,19 @@ states. Logical Workstream references are represented only by ## ArtifactOperationReceipt Append-only Workstream evidence for one immutable put acknowledgement. It links -the exact upload item and replica and records operation, idempotency key, +the exact producer admission/attempt and replica and records operation, idempotency key, `request_digest`, opaque `provider_object_ref`, replay observation, bounded outcome/details, attempt number, correlation ID, and creation time. It contains no response digest or provider receipt. -## ArtifactSetManifest +## SubmissionBundleManifest -The server-generated canonical description of a sealed upload session's content -IDs, logical roles, trusted file facts, SHA-256 values, and byte counts. Every -entry has a server-derived identity over all semantic fields, exact duplicate -entries are rejected, and a total ordering makes the hash deterministic. Its -hash commits only to that exact set. A separate pre-submit admission record -binds the hash to actor, task, policy, checker, exact submission input, result, -and expiry. +The server-generated canonical semantic description of the file/directory tree +inside one contributor outer ZIP. It commits to normalized paths, entry types, +and each file's SHA-256 and byte count while excluding ZIP packaging metadata. +It is distinct from the exact outer-ZIP SHA-256/byte count. Together they bind +pre-submit evidence, verified admission, the immutable Submission, and its exact +artifact binding. ## ReviewPacketManifest diff --git a/docs/spec_artifact_storage_service.md b/docs/spec_artifact_storage_service.md index 0603bdc4..5b6c627d 100644 --- a/docs/spec_artifact_storage_service.md +++ b/docs/spec_artifact_storage_service.md @@ -53,22 +53,20 @@ product role, product authorization, or proof of stored content. ## Canonical Records -### ArtifactUploadSession +### SubmissionBundlePreparation -Fields include id, actor, project, optional task/guide, permitted logical roles, -state, item/byte limits and totals, sealed artifact-set hash, expiry, consumed -timestamp, database timestamps, and CAS version. +One contributor operation accepts exactly one outer ZIP. The ZIP remains in the +existing bounded `ArtifactScratchManager` through safe tree inspection, +canonical archive/semantic-manifest identity, unchanged-work comparison, and +mandatory platform/locked-guide checks. It is not candidate storage, carries no +provider reference, cannot cross a process boundary, and is discarded on +failure or process loss. -States are `open`, `sealed`, `consumed`, `expired`, and `cancelled`. - -### ArtifactUploadItem - -One per staged object. Fields include session, logical role, normalized display -name, expected SHA-256, expected byte count, canonical request digest, scoped -idempotency key, CAS version, content ID, failure code, and timestamps. - -States are `reserved`, `uploading`, `replay_required`, -`stored_pending_verification`, `ready`, `failed`, and `cancelled`. +Only a passing preparation enters generic durable admission. Existing +`ArtifactUploadSession`/`ArtifactUploadItem` rows and planned actions are legacy +unavailable staging input and must be removed or made statically unreachable by +ART-04A plus the separately reviewed AUTH registration handoff before the new +surface activates. ### ArtifactStorageAdmissionLedger @@ -107,7 +105,7 @@ configuration for task, producer, project, and deployment scopes; a missing, malformed, or nonpositive required limit fails startup rather than creating an unbounded scope. Product callers never supply scope collections. They submit one of the closed producer requests defined below; artifact orchestration loads -the authoritative project, task, upload-session, checker-run, actor/service, +the authoritative project, task, submission-bundle operation, checker-run, actor/service, and deployment facts, derives the complete `AdmissionScopeSet` internally, and commits all reservations or none. A missing canonical relationship fails before provider I/O rather than silently omitting a quota scope. @@ -121,7 +119,7 @@ manufacture capacity. ### ArtifactPutAttempt One durable attempt owns the gap between Transaction A and Transaction B. -Fields include producer request type/reference, upload item or system output +Fields include producer request type/reference, producer admission or system output reference, content commitment, deterministic object identity, admission-charge references, operation/request digests, status, next-run time, executor UUID, database-clock lease expiry, execution generation, bounded terminal result, @@ -140,19 +138,16 @@ A bounded PostgreSQL scanner publishes `prepared`, `acknowledgement_unknown`, and expired `put_in_flight` attempts. The resolver uses read-only `observe_put_result`, then opens and hashes any observed object. Matching bytes complete the original charges and Transaction B facts exactly -once. Authoritative absence releases charges and moves the item to +once. Authoritative absence releases charges and moves the producer attempt to `replay_required`. Mismatched bytes remain charged, quarantine the key, and create an incident. Terminal writes require matching executor and generation. No background resolver performs another provider write. -Contributor upload sessions also reserve open-session slots at task, actor, -project, and deployment scope. PostgreSQL-clock expiry is owned by one bounded -periodic scanner and is also applied lazily before new session admission. The -scanner and lazy path use the fixed system permission -`artifact.upload_session.expire`; the atomic terminal transition releases the -slot once. Completed byte charges remain after cancellation, expiry, or absence -of a product binding because v0.1 has no physical deletion. Provider I/O never -starts when any applicable scope lacks capacity. +Submission-bundle preparation reserves bounded scratch through the existing +scratch ledger, not a durable open-session slot. Failed or lost preparations +release scratch without altering completed durable-byte charges. Provider I/O +never starts until every pre-submit gate passes and every applicable durable +scope has capacity. ### ArtifactContent @@ -175,20 +170,20 @@ availability_state = unknown | available | unavailable integrity_state = unknown | valid | invalid ``` -Transaction B sets the upload item to `stored_pending_verification` and creates -the replica as `pending/unknown/unknown`. A matching complete read sets the item -to `ready` and replica to `verified/available/valid`. A provider-unavailable or +Transaction B sets the producer attempt to `stored_pending_verification` and creates +the replica as `pending/unknown/unknown`. A matching complete read makes the +admission bindable and replica `verified/available/valid`. A provider-unavailable or conflict job result does not fabricate a replica observation. A confirmed absent object sets the replica to `missing/unavailable/unknown`. Before any binding exists, and only while the -original upload session/item remains eligible, an exact replay by the original -authorized uploader may move that same replica to `pending/unknown/unknown`, -set the item back to `stored_pending_verification`, append a new operation +original producer attempt remains eligible, an exact replay by the original +authorized producer may move that same replica to `pending/unknown/unknown`, +set the attempt back to `stored_pending_verification`, append a new operation receipt, and create a new verification job. After a binding exists, the missing replica is terminal and cannot return to pending in v0.1. A digest/size mismatch sets the replica to `integrity_mismatch/available/invalid`, fails the pre-binding -item when applicable, and quarantines access; it never returns to pending. +attempt when applicable, and quarantines access; it never returns to pending. Provider references are internal. They never appear in contributor, reviewer, project-owner, or public API responses. @@ -270,7 +265,7 @@ preserves history. ### ArtifactOperationReceipt Append-only Workstream evidence for one immutable put acknowledgement. It links -the exact upload item and replica and records operation, idempotency key, +the exact producer admission/attempt and replica and records operation, idempotency key, `request_digest`, opaque `provider_object_ref`, replay observation, bounded outcome/details, attempt number, correlation ID, and `created_at`. Adapter and namespace identity resolve through the linked replica. A response digest, @@ -356,9 +351,9 @@ Product modules receive only these narrow asynchronous capabilities: ```text GuideArtifactIngestPort.ingest(GuideArtifactIngestRequest) -ContributorArtifactUploadPort.create/read/write/seal/cancel(...) +SubmissionBundlePreparationPort.prepare(SubmissionBundlePreparationRequest) ArtifactBindingPort.bind_verified(ArtifactBindingCreateRequest) -ArtifactMaterializationPort.materialize_ready_upload_set(ReadyUploadSetRequest) +ArtifactMaterializationPort.materialize_prepared_bundle(PreparedBundleMaterializationRequest) ArtifactMaterializationPort.materialize_bindings(BindingMaterializationRequest) CheckerArtifactOutputPort.store(CheckerOutputArtifactRequest) ArtifactOperatorReadPort.list_bindings/list_replicas/list_receipts/ @@ -368,8 +363,10 @@ ArtifactOperatorRecoveryPort.retry_verification(ArtifactRecoveryRequest) `GuideArtifactIngestRequest` contains the authenticated actor context, project and guide-source snapshot item IDs, logical role, authorized byte source, and -optional client commitment. `ReadyUploadSetRequest` contains task ID, sealed -upload-session ID, locked policy/checker context, and authorization context. +optional client commitment. `SubmissionBundlePreparationRequest` contains the +authorized contributor/task/assignment context and one outer ZIP byte source. +`PreparedBundleMaterializationRequest` is internal and process-local; it wraps +only the current `PreparedArtifact` generation and locked policy/checker context. `BindingMaterializationRequest` contains task/submission/checker-run context and immutable binding IDs. `CheckerOutputArtifactRequest` contains the fixed service actor, task/submission/checker-run IDs, logical role, and generated byte @@ -884,11 +881,11 @@ The same canonical `ArtifactScratchManager` also owns checker-workspace allocations. Authoritative pre-submit introduces one authorized artifact materializer, and post-submit reuses it without a second workspace manager. The materializer exposes two separate methods, not one caller-selected source -union: `materialize_ready_upload_set` accepts a sealed upload artifact set whose -items are all `ready` before a submission binding exists, and -`materialize_bindings` accepts immutable `ArtifactBinding` IDs after submission -creation. Source resolution is phase-specific and authorized; staging never -creates a premature product binding. The materializer reserves +union: an internal pre-submit method accepts only the current process-local +`PreparedArtifact` submission-bundle workspace, and `materialize_bindings` +accepts immutable `ArtifactBinding` IDs after submission creation. No scratch +path or handle crosses an API/Celery boundary, and staging never creates a +premature product binding. The materializer reserves the complete workspace against the same aggregate ledger and quotas, streams exact provider bytes into private no-follow paths, and recomputes SHA-256 and byte count for every file. Any mismatch becomes an artifact incident before @@ -1021,20 +1018,17 @@ those stable Workstream IDs. No route returns provider object references, bucket/key, endpoint, credentials, signed URLs, or raw provider responses. Pagination is bounded and stable. -## Exact Artifact-Set Admission +## Exact Submission-Bundle Admission -Sealing generates `ArtifactSetManifest` from trusted server facts. Entries are -deterministically ordered and commit to logical role, normalized display name, -content ID, SHA-256, and byte count. Exact duplicates are rejected. +One contributor operation receives one outer ZIP into bounded private scratch. +Workstream computes its exact SHA-256/byte count, walks the complete safe outer +archive tree, and generates a deterministic `SubmissionBundleManifest` from +normalized file/directory paths, entry type, and file SHA-256/byte count. Nested +archives remain opaque in v0.1. Exact archive or semantic-manifest equality with +the immediate prior immutable Submission rejects before checker/provider I/O. -The pre-submit admission binds the artifact-set hash to actor, task, effective -project policy, project pre-submit checker, summary, contributor attestation, -expiry, and upload session. Submission creation locks and consumes that exact -admission and session in one transaction. A changed field or changed artifact -requires a new precheck. - -Artifact reads during authoritative pre-submit have a bounded Workstream-owned -transient retry budget. Exhaustion returns: +Mandatory platform and locked Project Guide checks consume that same read-only +scratch tree. Infrastructure exhaustion returns: ```text HTTP 503 @@ -1042,26 +1036,17 @@ code = pre_submission_infrastructure_unavailable ``` This is infrastructure state, not a checker result, review decision, or -contributor outcome. It creates no admission, submission, checker finding, -compensation, contribution, or reputation effect. The sealed upload session and -artifact set remain sealed, unconsumed, and reusable until normal expiry; only -the infrastructure attempt and audit are persisted. Idempotency scope is actor, -task, sealed session/artifact set, locked context, client key, and canonical -request digest. Exact replay continues or returns the same attempt, changed -replay conflicts, and concurrent replay cannot create duplicate attempts or -admissions. After retry-after or storage recovery, the contributor continues -that same exact attempt without Project Manager or Operator approval. - -Before authoritative pre-submit invokes the checker, the shared authorized -materializer resolves every `ready` item in the sealed upload artifact set, -reads its content through the referenced verified replica into a canonical -scratch-manager workspace, recomputes SHA-256 and byte count, and compares them -with the upload-item/content commitment. No `ArtifactBinding` exists at this -stage. The checker executes only that verified read-only workspace. Post-submit -materialization instead resolves immutable binding IDs and compares bytes with -their bound content commitments. A mismatch is an artifact incident; quota -exhaustion or transient read failure is infrastructure state and creates no -checker result or admission. +contributor outcome. It creates no durable artifact, admission, Submission, +compensation, contribution, or reputation effect. Scratch is cleaned and +process loss requires reupload without manager/operator approval. + +Only a passing result is handed immediately to generic durable admission in the +same process. Workstream writes the outer ZIP once, independently reads it back, +and publishes a bindable admission only after exact verification. Existing put +attempt, observation, receipt, verification, scanner, and recovery records own +durable ambiguity. Submission creation locks and consumes the exact admission; +post-submit materialization resolves its immutable binding and recomputes +integrity before checker execution. After cutover, the public request is: @@ -1069,33 +1054,27 @@ After cutover, the public request is: { "summary": "...", "contributor_attestation": "...", - "upload_session_id": "..." + "submission_bundle_admission_id": "..." } ``` Clients do not provide package URI, provider reference, canonical digest -manifest, artifact-set hash, or server content IDs. +manifest, semantic-manifest hash, or server content IDs. -### Product Upload APIs +### Product Submission-Bundle APIs ```text -POST /api/v1/tasks/{task_id}/artifact-upload-sessions -GET /api/v1/artifact-upload-sessions/{session_id} -POST /api/v1/artifact-upload-sessions/{session_id}/artifacts -POST /api/v1/artifact-upload-sessions/{session_id}/seal -DELETE /api/v1/artifact-upload-sessions/{session_id} -POST /api/v1/tasks/{task_id}/submission-precheck +POST /api/v1/tasks/{task_id}/submission-bundle-preparations +GET /api/v1/tasks/{task_id}/submission-bundle-preparations/{operation_id} POST /api/v1/tasks/{task_id}/submissions ``` -The upload APIs return Workstream IDs, canonical SHA-256, byte count, detected -media type, verification/readiness, and bounded artifact summaries. They never -return provider internals. Session cancellation is logical staging cleanup; it -does not physically delete a completed content-addressed object. - -Every ID-addressed session read or mutation uses concealed deny/not-found -behavior. Cross-actor, cross-project, revoked, expired, cancelled, consumed, -and random IDs cannot be used as an existence oracle. +The preparation POST performs scratch intake, inspection, checks, and durable +handoff as one process-local orchestration. It returns only a Workstream +operation/admission identity and bounded redacted status; polling occurs only +after durable intent and never resolves a scratch path. APIs never return +provider internals. Every ID-addressed status read uses concealed deny/not-found +behavior across actors, projects, revocation, terminal state, and random IDs. ## Guide And Checker Binding @@ -1114,9 +1093,10 @@ policy. Recovery authorization and automatic setup continuation write separate audit events. A changed snapshot creates a new setup run and cannot resume the old one. -`CheckerInputSnapshot` references binding/content IDs, digest, byte count, -artifact-set hash, locked policy/checker identities, and checker implementation -identity. Pre-submit and post-submit consume the same sealed artifact-set hash. +`CheckerInputSnapshot` references binding/content IDs, outer-ZIP digest/byte +count, semantic-manifest hash, locked policy/checker identities, and checker +implementation identity. Pre-submit evidence and post-submit execution name the +same verified admission and exact binding. Checker logs and generated outputs become artifact bindings. Transient post-submit storage unavailability leaves the task in diff --git a/docs/spec_authorization_service.md b/docs/spec_authorization_service.md index f1252a0a..7b95f154 100644 --- a/docs/spec_authorization_service.md +++ b/docs/spec_authorization_service.md @@ -362,12 +362,21 @@ The paired artifact hidden-behavior matrix is closed: |---|---| | `WS-ART-001-02D` | Operator binding/replica/receipt/verification-job/recovery-attempt/audit reads; the operations-domain `operations.artifact_storage_admission.read` action mapped to `operations.status.read`; verification retry; `artifact.verification.execute`; `artifact.pending_work.scan`; and `artifact.put_attempt.resolve` | | `WS-ART-001-03` | `artifact.guide_source.ingest`, `artifact.guide_source.read`, and `artifact.guide_source.binding.create` mapped to `artifact.binding.create` | -| `WS-ART-001-04A` | upload-session create/read/seal/cancel/expire and upload-item write | +| `WS-ART-001-04A` legacy unavailable baseline | planned upload-session create/read/seal/cancel/expire and upload-item write have no route/command and must be retired by the separate AUTH registration contract before 04A starts | +| `WS-ART-001-04A` through `04C` after AUTH registration | one hidden `artifact.submission_bundle.prepare` surface mapped to `submission.create`; remains unavailable until 04C evidence and a later AUTH activation contract | | `WS-ART-001-04B` | `artifact.pre_submit.checker_input.materialize` mapped to `artifact.checker_input.materialize` | | `WS-ART-001-05` | `artifact.submission.binding.create` mapped to `artifact.binding.create` | | `WS-ART-001-06A` | `artifact.post_submit.checker_input.materialize` mapped to `artifact.checker_input.materialize` | | `WS-ART-001-06B` | `artifact.checker_output.write` and `artifact.checker_output.binding.create` mapped to `artifact.binding.create` using the checker-run resource | +The `WS-ART-001-04A` row records the current planned/unavailable catalogue +baseline only. ART PLAN2 proposes a separate AUTH-owned registration contract +that retires those unused multi-step actions and registers planned +`artifact.submission_bundle.prepare -> submission.create`. No ART implementation +may use that new ActionId before the AUTH contract merges; no action activates +until ART-04A-C publish the complete hidden surface and a later AUTH activation +contract consumes its exact evidence. + Every row requires AUTH-07A's registry and AUTH-07B's kernel first. A row with an Operator principal also requires its AUTH-08 grant definition; a row with a fixed service principal also requires AUTH-09A's static matrix, AUTH-09B's provisioned service @@ -435,7 +444,7 @@ and unavailable, and add no migration. | `artifact.verification.execute` | `artifact.verification.execute` | fixed verifier service | verification job | `02D` | | `artifact.pending_work.scan` | `artifact.pending_work.scan` | fixed scheduler service | system pending-work scope | `02D` | | `artifact.put_attempt.resolve` | `artifact.put_attempt.resolve` | fixed put-resolver service | put attempt | `02D` | -| `artifact.pre_submit.checker_input.materialize` | `artifact.checker_input.materialize` | fixed materializer service | sealed upload session and task | `04B` | +| `artifact.pre_submit.checker_input.materialize` | `artifact.checker_input.materialize` | fixed materializer service | task plus current process-local prepared-bundle generation; no scratch path/handle is serialized | `04B` | | `artifact.post_submit.checker_input.materialize` | `artifact.checker_input.materialize` | fixed materializer service | checker run and immutable bindings | `06A` | | `artifact.checker_output.write` | `artifact.checker_output.write` | fixed checker-output service | checker run | `06B` | diff --git a/docs/template_submission_artifact_policy.md b/docs/template_submission_artifact_policy.md index 7de999d9..977ff691 100644 --- a/docs/template_submission_artifact_policy.md +++ b/docs/template_submission_artifact_policy.md @@ -98,8 +98,8 @@ Default artifact rules: Default storage rules: -- clients submit bytes through Workstream upload sessions and receive only - Workstream artifact IDs +- clients submit exactly one outer ZIP through Workstream submission-bundle + preparation and receive only Workstream operation/admission IDs - persisted product references are immutable Workstream artifact bindings - signed URLs, raw local filesystem paths, provider references, credentials, query strings, bucket secrets, and token-bearing references are rejected diff --git a/docs/template_submission_packet.md b/docs/template_submission_packet.md index bd481175..ac22406d 100644 --- a/docs/template_submission_packet.md +++ b/docs/template_submission_packet.md @@ -16,16 +16,20 @@ Assigned by Workstream after blocking pre-submit checks pass. The contributor do Briefly describe what was completed. -## Output +## Submission Bundle -List files, links, packages, or deliverables. +Upload exactly one outer ZIP. Its internal files and directories must satisfy +the locked Project Guide. Workstream safely inspects that tree and computes all +artifact identity; the contributor does not submit links or storage references. ## Provenance -- package hash: -- artifact hash manifest: - generated at: +Workstream records the server-computed outer-ZIP SHA-256/byte count, canonical +semantic manifest, verified admission, and immutable binding. Those values are +server output, not contributor input. + Workstream derives the locked project guide version, locked guide-source snapshot id/hash, effective project submission artifact policy id/hash, generated project pre-submit checker policy id/bundle hash, post-submit @@ -46,11 +50,10 @@ submission row, no submission version, and no submission-created audit event, and do not return review decision values: `accept`, `needs_revision`, or `reject`. -## Artifact Hash Manifest +## Submission Bundle Manifest -| Artifact | Hash | Size | Notes | -| --- | --- | ---: | --- | -| `` | `sha256:<64 lowercase hex>` | `` | `` | +Workstream generates this manifest from normalized directory/file paths, entry +type, and each file's SHA-256/byte count. Nested archives remain opaque in v0.1. ## Evidence diff --git a/scripts/test_agent_gates.py b/scripts/test_agent_gates.py index 68819fc5..41b2971c 100644 --- a/scripts/test_agent_gates.py +++ b/scripts/test_agent_gates.py @@ -36,9 +36,12 @@ "02C2", "02C3", "02D", - "03", + "03A", + "03B", + "03C", "04A", "04B", + "04C", "05", "06A", "06B", @@ -77,19 +80,23 @@ "02D": ( "coverage report --include='app/api/router.py' --precision=2 --fail-under=90", ), - "03": ( + "03A": ( "coverage report --include='app/modules/projects/*' " "--precision=2 --fail-under=90", + ), + "03B": ( "coverage report " "--include='app/adapters/project_agents/*,app/interfaces/project_agents.py' " "--precision=2 --fail-under=90", ), + "03C": (), "04A": (), "04B": ( "coverage report --include='app/modules/tasks/*' --precision=2 --fail-under=90", "coverage report --include='app/modules/checkers/*' " "--precision=2 --fail-under=90", ), + "04C": (), "05": (), "06A": (), "06B": (), @@ -163,7 +170,7 @@ def artifact_contract_phase_for(coverage_phase: str) -> str: phase = "foundation" if active_index >= ARTIFACT_COVERAGE_ORDER.index("02A3"): phase = "artifact_store_cutover" - if active_index >= ARTIFACT_COVERAGE_ORDER.index("03"): + if active_index >= ARTIFACT_COVERAGE_ORDER.index("03A"): phase = "guide_source_cutover" if active_index >= ARTIFACT_COVERAGE_ORDER.index("04B"): phase = "upload_admission" @@ -5658,7 +5665,7 @@ def test_auth_spec_orders_service_admission_before_project_roles() -> None: def test_parallel_initiative_status_matches_trusted_main() -> None: - """Auth prerequisites and the current ART gate remain internally consistent.""" + """Auth history and authored ART planning remain internally consistent.""" auth_map = Path( ".agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/" "CHUNK_MAP.md" @@ -5678,6 +5685,18 @@ def test_parallel_initiative_status_matches_trusted_main() -> None: ).read_text(encoding="utf-8") work_queue = Path(".agent-loop/WORK_QUEUE.md").read_text(encoding="utf-8") loop_state = Path(".agent-loop/LOOP_STATE.md").read_text(encoding="utf-8") + contribution_guide = Path("CONTRIBUTING.md").read_text(encoding="utf-8") + loop_guide = Path(".agent-loop/README.md").read_text(encoding="utf-8") + + # Authored main snapshots below remain historical discovery input. Live + # active/cancel state is read only from independently verified signed + # automation, so ART assertions use its initiative status/map rather than + # pretending copied root projections are current authority. + canonical_warning = ( + "Only independently verified signed automation state is canonical authority" + ) + assert canonical_warning in " ".join(contribution_guide.split()) + assert canonical_warning in " ".join(loop_guide.split()) assert "Merged through PR #131 as `aa0fdcd`" in auth_map assert "Merged through PR #143 as `053242b`" in auth_map @@ -5711,7 +5730,6 @@ def test_parallel_initiative_status_matches_trusted_main() -> None: "as `8d5eb15` on 2026-07-19" in work_queue ) assert "Active implementation chunk: `WS-AUTH-001-CONTRIBUTOR-FOUNDATION`" not in loop_state - assert "Active ART implementation chunk: `WS-ART-001-02C2`" in loop_state assert "Merged through PR #157 as `42a89b2d`" in work_queue assert "ActionIds, with 17 active actions" in loop_state assert "candidate total of 17" not in loop_state @@ -5742,28 +5760,20 @@ def test_parallel_initiative_status_matches_trusted_main() -> None: assert "Merged through PR #141 as `a10d901`" in artifact_map assert "Merged through PR #151 as `1b5422fc`" in artifact_map assert "Merged through PR #154 as `44f2467c`" in artifact_map - assert "Active after PR #154 and explicit user start" in artifact_map - assert "Status: Active after explicit start on 2026-07-19" in artifact_contract + assert "Merged through PR #159 as `bc5e6a42`" in artifact_map + assert "Merged through PR #174 as `92b8a7aa`" in artifact_map + assert "Merged through PR #177 as `93c14181`" in artifact_map assert ( "AUTH's owner reconciliation merged through PR #140 as\n" "`d541521`" in artifact_status ) - assert "`WS-ART-001-02C2` is active" in artifact_status - assert "The current gate is external CI/review followed by explicit human" in ( - artifact_status.replace("\n", " ") - ) - assert "No later artifact chunk starts automatically" in artifact_status.replace( - "\n", " " - ) + assert "`WS-ART-001-03` received a signed implementation start" in artifact_status + assert "recorded `stopped_after_cancel`" in artifact_status + assert "The planning merge starts no successor" in artifact_status assert ( "| `WS-AUTH-001-09C` | Actor And Identity-Link Administration Reads | L1 | " "Merged through PR #146 as `0ffdabf`" in work_queue ) - assert ( - "| `WS-ART-001-02C2` | Verification Publication And Fencing | L1 | " - "Latest-main, sharded-CI, AUTH-PREP, and outbox-repair reconciliation complete" - in work_queue - ) assert ( "| `WS-AUTH-001-ART-CUSTODY` | ART Activation Custody Transfer | L1 | " "Merged through PR #158 as `be2a79a2`" in work_queue @@ -5779,12 +5789,9 @@ def test_parallel_initiative_status_matches_trusted_main() -> None: "Merged through PR #162 as `c559d556`" in work_queue ) assert "no feature consumer or activation" in work_queue - assert "Current ART gate: latest-main integration" in loop_state - assert "hosted checks, and explicit human merge approval remain" in " ".join( - loop_state.split() - ) - assert "hosted reruns, and explicit human merge approval remain" in work_queue - assert "PR #154 then merged `WS-ART-001-02C1`" in loop_state + assert "| `WS-ART-001-PLAN2` |" in artifact_map + assert "Planning-only successor proposed after cancellation" in artifact_map + assert "`WS-ART-001-03A` is the only declared ART" in artifact_status def test_stale_authorization_discovery_includes_new_untracked_docs() -> None: @@ -7161,7 +7168,7 @@ def test_artifact_chunk_verification_commands_are_isolated_and_rerunnable() -> N assert "run_isolated_tests.py" in metadata_command assert metadata_command.endswith("))") assert "alembic upgrade head" not in verification - if phase in {"03", "05"}: + if phase in {"03C", "05"}: assert "backend/scripts/api_contract_e2e.py" in contract assert "scripts/api_contract_e2e.py" in verification expected_contract_phase = artifact_contract_phase_for(phase) From 8fd03205414bf512b07874737ff439943ffb271f Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Fri, 24 Jul 2026 17:23:23 +0100 Subject: [PATCH 02/26] docs(artifacts): address external plan review --- .agent-loop/REVIEW_LOG.md | 9 ++++ .../INTENT.md | 5 ++- .../PLAN.md | 12 ++++-- .../RISKS.md | 2 +- .../STATUS.md | 2 +- .../WS-ART-001-03-guide-source-cutover.md | 2 +- ...WS-ART-001-03A-guide-source-byte-ingest.md | 2 +- ...3B-guide-source-binding-materialization.md | 2 +- ...1-03C-guide-source-cutover-continuation.md | 2 +- ...S-ART-001-04A-upload-inspection-sealing.md | 13 +++--- .../WS-ART-001-04B-pre-submit-admission.md | 2 +- ...4C-verified-submission-bundle-admission.md | 6 ++- ...-ART-001-05-submission-artifact-cutover.md | 2 +- ...T-001-06A-checker-input-materialization.md | 2 +- .../WS-ART-001-07-recovery-live-proof.md | 2 +- ...-PLAN2-submission-bundle-reconciliation.md | 2 +- ...-ART-001-PLAN2-external-review-response.md | 43 +++++++++++++++++++ .../WS-ART-001-PLAN2-pr-trust-bundle.md | 7 +++ docs/spec_artifact_storage_service.md | 16 ++++--- docs/template_submission_artifact_policy.md | 6 ++- scripts/test_agent_gates.py | 4 +- 21 files changed, 110 insertions(+), 33 deletions(-) create mode 100644 .agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-external-review-response.md diff --git a/.agent-loop/REVIEW_LOG.md b/.agent-loop/REVIEW_LOG.md index d3489e22..2159c384 100644 --- a/.agent-loop/REVIEW_LOG.md +++ b/.agent-loop/REVIEW_LOG.md @@ -2900,3 +2900,12 @@ fresh GitHub Backend, Agent Gates, CodeRabbit, and human review remain. scratch/admission/recovery abstractions, and passes all deterministic gates. - Final senior, architecture, QA/test, security/auth, product/ops, reuse/dedup, CI integrity, test-delta, and docs reviews pass with no open finding. + +## 2026-07-24 - WS-ART-001-PLAN2 External Review Reconciliation + +CodeRabbit found seven valid inconsistencies in phase ownership, hidden route +composition, canonical ZIP paths, AUTH retirement custody, successor wording, +put-attempt state vocabulary, and server-owned manifest input. Hosted CI also +found noncanonical chunk headings and a successor-title mismatch. The repair +resolves all findings without runtime behavior or AUTH activation; refreshed +internal review, hosted checks, and human approval remain required. diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/INTENT.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/INTENT.md index 82e50192..80342717 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/INTENT.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/INTENT.md @@ -36,8 +36,11 @@ authorized bytes -> content-addressed S3-compatible object key -> independent complete-object verification -> immutable ArtifactContent and ArtifactReplica facts +-> ArtifactBinding to guide, checker input, log, or output + +submission outer ZIP only -> verified archive identity and canonical semantic-manifest commitment --> ArtifactBinding to guide, submission, checker input, log, or output +-> exact Submission binding ``` PostgreSQL stores metadata, bindings, operation receipts, lifecycle state, diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md index 94cb7f88..a3522680 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md @@ -291,9 +291,15 @@ semantic identity = canonical hash of normalized path, entry type, ZIP timestamps, compression settings, comments, ownership, and platform permission metadata do not change semantic identity. Explicit empty directories and synthetic parents use one documented canonical representation. Symlinks and -special entries are rejected. A ZIP entry within the outer archive is opaque -in v0.1; recursive inspection means walking the outer archive tree, not opening -nested archives. A later capability requires separate cumulative safety proof. +special entries are rejected. Canonical paths use `/` separators, contain only +relative non-empty segments, and normalize Unicode to NFC. The identity remains +case-sensitive, but Workstream rejects exact duplicates, NFC collisions, and +Unicode case-fold collisions so the checked tree cannot vary by filesystem. +Collision detection completes before an entry enters the manifest, any project +precheck or materializer observes the tree, or provider I/O starts. A ZIP entry +within the outer archive is opaque in v0.1; recursive inspection means walking +the outer archive tree, not opening nested archives. A later capability requires +separate cumulative safety proof. Both identities are compared with the immediate prior immutable `Submission`. Exact archive equality or semantic equality rejects before checker and provider diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/RISKS.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/RISKS.md index 8d92acf2..f47ec460 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/RISKS.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/RISKS.md @@ -27,7 +27,7 @@ | Concurrent prepared artifacts exhaust worker disk | High | Full-max pre-reservation, dedicated quota, cross-process ledger, bounded concurrency, free-space floor, and deadline-before-TTL cleanup. | | Any guide, contributor, or checker producer creates unbounded retained bytes | Critical | One generic admission service atomically caps cumulative unique provisional/completed bytes at every applicable task, producer, project, and deployment scope before provider I/O; only authoritative absence releases a provisional charge. | | Operators cannot diagnose or expand exhausted storage capacity | High | Bounded admission-usage API, scope metrics/alerts, and a configuration-driven expansion/rollback runbook; no charge deletion or database editing. | -| Legacy planned upload-session actions accidentally become a second intake path | Critical | Keep them unavailable and route-less until the AUTH registration handoff retires them; ART-04A removes or makes their models/routes unreachable before the single bundle action can activate. | +| Legacy planned upload-session actions accidentally become a second intake path | Critical | ART-04A removes only ART-side reachability. The AUTH registration contract must retire, or prove unavailable and route-less, the AUTH-owned ActionIds and routes before AUTH may activate the single bundle action. | | Recovery replay or ancestor reuse creates multiple jobs | High | Canonical request digest plus a lifetime unique source-job constraint return original IDs on exact replay, reject different-key/concurrent reuse, and require chain progression through the immediate retry job. | | Operator cannot observe retry completion | High | Authorized recovery-attempt GET includes immutable source-job and current retry-job status plus audit IDs. | | Storage failure becomes contributor failure or review decision | Critical | Stable infrastructure errors; no accept/needs_revision/reject, contribution, payment, or reputation effect. | diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/STATUS.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/STATUS.md index ea1977a3..3c84b475 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/STATUS.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/STATUS.md @@ -47,7 +47,7 @@ ZIP and immutable Submission. ## Next Proposed Chunk -After this planning package merges, `WS-ART-001-03A` is the only declared ART +After this planning package merges, `WS-ART-001-03A` is the only immediate ART successor. It adds hidden guide-source byte ingest through the existing artifact preparation/admission/verification path. It requires a separate signed start and does not activate its own AUTH action. diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03-guide-source-cutover.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03-guide-source-cutover.md index a4b69daa..6e61a7a1 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03-guide-source-cutover.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03-guide-source-cutover.md @@ -1,4 +1,4 @@ -# Cancelled Contract: WS-ART-001-03 - Guide Source Artifact Cutover +# Chunk Contract: WS-ART-001-03 - Guide Source Artifact Cutover Initiative: `WS-ART-001` | Risk: L1 | Status: Cancelled before implementation; superseded by 03A-03C diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03A-guide-source-byte-ingest.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03A-guide-source-byte-ingest.md index 5d2462bb..ce9e7f13 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03A-guide-source-byte-ingest.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03A-guide-source-byte-ingest.md @@ -1,4 +1,4 @@ -# Chunk Contract: WS-ART-001-03A Guide Source Byte Ingest +# Chunk Contract: WS-ART-001-03A - Guide Source Byte Ingest Initiative: `WS-ART-001` | Risk: L1 | Status: Proposed after PLAN2 diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B-guide-source-binding-materialization.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B-guide-source-binding-materialization.md index d015559a..7a177641 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B-guide-source-binding-materialization.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03B-guide-source-binding-materialization.md @@ -1,4 +1,4 @@ -# Chunk Contract: WS-ART-001-03B Guide Source Binding And Materialization +# Chunk Contract: WS-ART-001-03B - Guide Source Binding And Materialization Initiative: `WS-ART-001` | Risk: L1 | Status: Proposed after 03A and exact AUTH activation diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03C-guide-source-cutover-continuation.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03C-guide-source-cutover-continuation.md index 5b10ca86..d1bd49d6 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03C-guide-source-cutover-continuation.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03C-guide-source-cutover-continuation.md @@ -1,4 +1,4 @@ -# Chunk Contract: WS-ART-001-03C Guide Source Cutover And Continuation +# Chunk Contract: WS-ART-001-03C - Guide Source Cutover And Continuation Initiative: `WS-ART-001` | Risk: L1 | Status: Proposed after 03B and exact AUTH activation diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04A-upload-inspection-sealing.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04A-upload-inspection-sealing.md index a529839e..01f85279 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04A-upload-inspection-sealing.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04A-upload-inspection-sealing.md @@ -1,8 +1,8 @@ -# Chunk Contract: WS-ART-001-04A One-ZIP Scratch Intake And Manifest +# Chunk Contract: WS-ART-001-04A - One-ZIP Scratch Intake And Manifest Initiative: `WS-ART-001` | Risk: L1 | Status: Proposed after 03C and AUTH planned registration -Artifact contract phase: `guide_source_cutover` +Artifact contract phase: `upload_admission` ## Goal @@ -70,12 +70,9 @@ or Submission creation. remains; - process loss requires reupload; normal, error, cancellation, timeout, and crash cleanup release scratch quota without deleting durable artifacts; -- the route declares only planned action `artifact.submission_bundle.prepare`, - mapped by AUTH to `submission.create`, with ART-owned task/assignment/actor - facts and guards; the old multi-step upload-session actions are not accepted - as aliases or alternate routes; -- hidden behavior remains unavailable until AUTH activates that exact action - only after 04A-04C publish the complete surface/guard manifest; +- 04A declares no contributor route or routable action manifest entry; the + planned `artifact.submission_bundle.prepare` action and AUTH mapping are + composed only by 04C after the complete internal operation exists; - focused subsystem coverage is at least 90 percent and repository coverage remains at least 78 percent. diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04B-pre-submit-admission.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04B-pre-submit-admission.md index bbe191ea..923daab8 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04B-pre-submit-admission.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04B-pre-submit-admission.md @@ -1,4 +1,4 @@ -# Chunk Contract: WS-ART-001-04B Scratch-Bound Pre-Submission Checks +# Chunk Contract: WS-ART-001-04B - Scratch-Bound Pre-Submission Checks Initiative: `WS-ART-001` | Risk: L1 | Status: Proposed after 04A diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04C-verified-submission-bundle-admission.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04C-verified-submission-bundle-admission.md index f95f2231..7c6515f8 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04C-verified-submission-bundle-admission.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04C-verified-submission-bundle-admission.md @@ -1,4 +1,4 @@ -# Chunk Contract: WS-ART-001-04C Verified Submission Bundle Admission +# Chunk Contract: WS-ART-001-04C - Verified Submission Bundle Admission Initiative: `WS-ART-001` | Risk: L1 | Status: Proposed after 04B @@ -58,6 +58,10 @@ verify it, and publish one bindable submission-bundle admission. v0.1 adds no deletion path; - after 04A-04C merge their complete hidden manifest, AUTH alone may activate `artifact.submission_bundle.prepare`; ART changes no action availability; +- 04C alone declares the hidden contributor route and routable + `artifact.submission_bundle.prepare` action, mapped by AUTH to + `submission.create`, with ART-owned task/assignment/actor facts and guards; + old upload-session actions are never aliases or alternate routes; - this chunk alone composes the hidden contributor endpoint across the 04A intake, 04B checks, and 04C durable handoff in one process-local operation; no scratch handle/path is serialized into a later request or Celery payload; diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-05-submission-artifact-cutover.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-05-submission-artifact-cutover.md index 609d4c11..5b376d17 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-05-submission-artifact-cutover.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-05-submission-artifact-cutover.md @@ -1,4 +1,4 @@ -# Chunk Contract: WS-ART-001-05 Submission Bundle Binding Cutover +# Chunk Contract: WS-ART-001-05 - Submission Bundle Binding Cutover Initiative: `WS-ART-001` | Risk: L1 | Status: Proposed after 04C and exact AUTH activation diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-06A-checker-input-materialization.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-06A-checker-input-materialization.md index 1a4d8002..3b54b881 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-06A-checker-input-materialization.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-06A-checker-input-materialization.md @@ -1,4 +1,4 @@ -# Chunk Contract: WS-ART-001-06A Checker Input And Materialization +# Chunk Contract: WS-ART-001-06A - Checker Input And Materialization Initiative: `WS-ART-001` | Risk: L1 | Status: Proposed after 05 diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-07-recovery-live-proof.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-07-recovery-live-proof.md index a023129b..c8151334 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-07-recovery-live-proof.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-07-recovery-live-proof.md @@ -1,4 +1,4 @@ -# Chunk Contract: WS-ART-001-07 Real API And Provider Proof +# Chunk Contract: WS-ART-001-07 - Real API And Provider Proof Initiative: `WS-ART-001` | Risk: L1 | Status: Proposed after 06B diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-PLAN2-submission-bundle-reconciliation.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-PLAN2-submission-bundle-reconciliation.md index 3305a9ed..890acfbd 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-PLAN2-submission-bundle-reconciliation.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-PLAN2-submission-bundle-reconciliation.md @@ -1,4 +1,4 @@ -# Chunk Contract: WS-ART-001-PLAN2 Submission Bundle Reconciliation +# Chunk Contract: WS-ART-001-PLAN2 - Submission Bundle Reconciliation Initiative: `WS-ART-001` | Risk: L1 | Status: Planning only diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-external-review-response.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-external-review-response.md new file mode 100644 index 00000000..afed980d --- /dev/null +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-external-review-response.md @@ -0,0 +1,43 @@ +# External Review Response: WS-ART-001-PLAN2 + +## Comments addressed + +- moved 04A to `upload_admission`, kept it internal and non-routable, and made + 04C the sole owner of the hidden contributor route/action declaration; +- defined relative `/`-separated NFC paths and fail-closed exact, NFC, and + Unicode case-fold collision rejection before checks, materialization, or + provider I/O; +- clarified that ART removes only ART-side legacy reachability while the AUTH + registration contract owns ActionId retirement/unavailable proof; +- described 03A as the only immediate successor; +- reconciled `ArtifactPutAttempt` states with the implemented closed enum and + separated them from legacy producer-item and replica states; +- removed the contributor-supplied hash-manifest field and documented the + manifest as server-generated; +- scoped archive and semantic-manifest identities to Submission outer ZIPs; +- canonicalized every changed chunk heading and aligned the PLAN2 successor + title so repository governance gates can parse the contracts; +- expanded the PR description to the canonical trust-bundle structure. + +## Comments deferred + +None. + +## Human decisions needed + +None beyond the normal human review and explicit merge approval for PR #197. + +## Commands rerun + +- `git diff --check` +- `python3 scripts/check_stale_artifact_contracts.py` +- `python3 scripts/check_markdown_links.py` +- `python3 scripts/test_agent_gates.py` — 100 tests +- `python3 scripts/check_internal_review_evidence.py` +- `python3 scripts/update_post_merge_memory.py validate-merge-intent --base-ref origin/main` + +## Remaining risks + +Hosted GitHub checks and refreshed CodeRabbit review must validate the pushed +repair. No backend runtime behavior changed, so slow backend suites remain +hosted rather than local. diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-pr-trust-bundle.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-pr-trust-bundle.md index e0dc9bc8..308a102d 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-pr-trust-bundle.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-pr-trust-bundle.md @@ -54,3 +54,10 @@ successor `WS-ART-001-03A` with a separate explicit start. This planning merge starts no implementation. `WS-ART-001-03A` requires a fresh signed explicit start after human merge approval and automated merge memory reconciliation. + +## External Review + +CodeRabbit reported seven valid planning/specification inconsistencies. All +were addressed in the smallest planning-only repair, alongside the two +governance failures exposed by hosted CI. The detailed disposition is recorded +in `WS-ART-001-PLAN2-external-review-response.md`. diff --git a/docs/spec_artifact_storage_service.md b/docs/spec_artifact_storage_service.md index 5b6c627d..cf5a1b5c 100644 --- a/docs/spec_artifact_storage_service.md +++ b/docs/spec_artifact_storage_service.md @@ -139,7 +139,8 @@ A bounded PostgreSQL scanner publishes `prepared`, uses read-only `observe_put_result`, then opens and hashes any observed object. Matching bytes complete the original charges and Transaction B facts exactly once. Authoritative absence releases charges and moves the producer attempt to -`replay_required`. Mismatched bytes remain charged, quarantine the key, and +`absent_replay_required`; a legacy contributor upload item, while it exists, +moves to `replay_required`. Mismatched bytes remain charged, quarantine the key, and create an incident. Terminal writes require matching executor and generation. No background resolver performs another provider write. @@ -170,8 +171,9 @@ availability_state = unknown | available | unavailable integrity_state = unknown | valid | invalid ``` -Transaction B sets the producer attempt to `stored_pending_verification` and creates -the replica as `pending/unknown/unknown`. A matching complete read makes the +Transaction B sets the `ArtifactPutAttempt` to `object_confirmed`, sets a legacy +contributor upload item, while it exists, to `stored_pending_verification`, and +creates the replica as `pending/unknown/unknown`. A matching complete read makes the admission bindable and replica `verified/available/valid`. A provider-unavailable or conflict job result does not fabricate a replica observation. @@ -179,8 +181,9 @@ A confirmed absent object sets the replica to `missing/unavailable/unknown`. Before any binding exists, and only while the original producer attempt remains eligible, an exact replay by the original authorized producer may move that same replica to `pending/unknown/unknown`, -set the attempt back to `stored_pending_verification`, append a new operation -receipt, and create a new verification job. After a binding exists, the missing +set the attempt back to `object_confirmed`, set a legacy contributor upload item, +while it exists, to `stored_pending_verification`, append a new operation receipt, +and create a new verification job. After a binding exists, the missing replica is terminal and cannot return to pending in v0.1. A digest/size mismatch sets the replica to `integrity_mismatch/available/invalid`, fails the pre-binding attempt when applicable, and quarantines access; it never returns to pending. @@ -733,7 +736,8 @@ in the v2 clean cut. No compatibility adapter or dual format remains. stores or resolves an exact replay candidate. 6. Transaction B validates the reservation/CAS, completes every provisional charge, records content, replica, and operation receipt, sets the item to - `stored_pending_verification`, and sets the replica to + `object_confirmed`, sets a legacy contributor upload item, while it exists, + to `stored_pending_verification`, and sets the replica to `pending/unknown/unknown`. 7. The transaction creates an outbox/publication obligation for one verification job. diff --git a/docs/template_submission_artifact_policy.md b/docs/template_submission_artifact_policy.md index 977ff691..020defa3 100644 --- a/docs/template_submission_artifact_policy.md +++ b/docs/template_submission_artifact_policy.md @@ -86,9 +86,13 @@ Every project inherits Workstream default submission artifact rules. Project pol Default required packet fields: - summary -- artifact hash manifest - contributor attestation +Workstream generates the archive commitment and semantic artifact manifest +after safely inspecting the submitted outer ZIP. Clients do not supply an +artifact hash manifest as packet input; later APIs may expose an immutable +server-generated manifest reference. + Default artifact rules: - artifact paths must be relative diff --git a/scripts/test_agent_gates.py b/scripts/test_agent_gates.py index 41b2971c..057b5b16 100644 --- a/scripts/test_agent_gates.py +++ b/scripts/test_agent_gates.py @@ -172,7 +172,7 @@ def artifact_contract_phase_for(coverage_phase: str) -> str: phase = "artifact_store_cutover" if active_index >= ARTIFACT_COVERAGE_ORDER.index("03A"): phase = "guide_source_cutover" - if active_index >= ARTIFACT_COVERAGE_ORDER.index("04B"): + if active_index >= ARTIFACT_COVERAGE_ORDER.index("04A"): phase = "upload_admission" if active_index >= ARTIFACT_COVERAGE_ORDER.index("05"): phase = "submission_cutover" @@ -5791,7 +5791,7 @@ def test_parallel_initiative_status_matches_trusted_main() -> None: assert "no feature consumer or activation" in work_queue assert "| `WS-ART-001-PLAN2` |" in artifact_map assert "Planning-only successor proposed after cancellation" in artifact_map - assert "`WS-ART-001-03A` is the only declared ART" in artifact_status + assert "`WS-ART-001-03A` is the only immediate ART" in artifact_status def test_stale_authorization_discovery_includes_new_untracked_docs() -> None: From 525182bd644ee4c1de63157d060678f4fb0c092d Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Fri, 24 Jul 2026 17:33:47 +0100 Subject: [PATCH 03/26] docs(artifacts): close review state boundaries --- .../PLAN.md | 14 ++++---- ...4C-verified-submission-bundle-admission.md | 3 ++ docs/spec_artifact_storage_service.md | 33 ++++++++++--------- 3 files changed, 29 insertions(+), 21 deletions(-) diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md index a3522680..e3c58eaf 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md @@ -202,9 +202,10 @@ durable acknowledgement-unknown attempt rather than committing a terminal fact. the adapter conditionally stores under the content-addressed key or resolves an exact replay candidate. 7. Transaction B records provider acknowledgement, completes the provisional - admission charges, sets the producer attempt to `stored_pending_verification`, and + admission charges, sets the `ArtifactPutAttempt` to `object_confirmed`, and creates the replica with pending verification and unknown - availability/integrity; no binding exists. + availability/integrity; while a legacy contributor upload item exists, it + alone moves to `stored_pending_verification`. No binding exists. 8. A durable verification job is committed in PostgreSQL and published to Celery after commit. A periodic scanner republishes pending work within the configured SLA. @@ -216,14 +217,15 @@ Provider acknowledgement loss keeps the durable put attempt and admission charges provisional. A PostgreSQL scanner publishes ambiguous and expired in-flight attempts; a fixed service principal runs read-only `observe_put_result` plus a complete hash. Matching bytes complete Transaction B -once, authoritative absence releases charges and makes the item -`replay_required`, and mismatched bytes quarantine the key. No background +once, authoritative absence releases charges and moves the put attempt to +`absent_replay_required`; while a legacy contributor upload item exists, it +alone moves to `replay_required`. Mismatched bytes quarantine the key. No background resolver repeats a provider write. Exact replay after absence must atomically reacquire capacity before another provider call. Workstream never stores upload bytes in Postgres, Redis, or Celery payloads. -Before binding, an object confirmed missing returns its reserved producer attempt to -`replay_required`, and only the original authorized producer may replay the +Before binding, an object confirmed missing returns its reserved put attempt to +`absent_replay_required`, and only the original authorized producer may replay the same bytes under the same operation identity. That pre-binding replay resets the same replica from `missing/unavailable/unknown` to `pending/unknown/unknown`, appends a receipt, and creates a new verification diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04C-verified-submission-bundle-admission.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04C-verified-submission-bundle-admission.md index 7c6515f8..8b254710 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04C-verified-submission-bundle-admission.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04C-verified-submission-bundle-admission.md @@ -65,6 +65,9 @@ verify it, and publish one bindable submission-bundle admission. - this chunk alone composes the hidden contributor endpoint across the 04A intake, 04B checks, and 04C durable handoff in one process-local operation; no scratch handle/path is serialized into a later request or Celery payload; +- v0.1 exposes no separate preparation-status GET route; an exact idempotent + POST retry may return only the same bounded durable operation/admission state + under the original `artifact.submission_bundle.prepare` authorization; - focused subsystem coverage is at least 90 percent and repository coverage remains at least 78 percent. diff --git a/docs/spec_artifact_storage_service.md b/docs/spec_artifact_storage_service.md index cf5a1b5c..b052d33c 100644 --- a/docs/spec_artifact_storage_service.md +++ b/docs/spec_artifact_storage_service.md @@ -735,8 +735,8 @@ in the v2 clean cut. No compatibility adapter or dual format remains. 5. The claimed attempt is set to `put_in_flight`; the adapter conditionally stores or resolves an exact replay candidate. 6. Transaction B validates the reservation/CAS, completes every provisional - charge, records content, replica, and operation receipt, sets the item to - `object_confirmed`, sets a legacy contributor upload item, while it exists, + charge, records content, replica, and operation receipt, sets the + `ArtifactPutAttempt` to `object_confirmed`, sets a legacy contributor upload item, while it exists, to `stored_pending_verification`, and sets the replica to `pending/unknown/unknown`. 7. The transaction creates an outbox/publication obligation for one @@ -760,10 +760,11 @@ Acknowledgement loss leaves the durable put attempt and charges provisional. The attempt scanner publishes resolution after an ambiguous outcome or expired execution lease. Resolution performs a fresh read-only observation and full hash. A confirmed object completes the same charges and Transaction B facts. -Fresh authoritative absence releases them and moves the item to exact replay; -replay must atomically reacquire every applicable charge before another provider -call. Integrity-mismatched or quarantined existing bytes remain completed and -charged. +Fresh authoritative absence releases them and moves the `ArtifactPutAttempt` to +`absent_replay_required`; while a legacy contributor upload item exists, it +alone moves to `replay_required`. Replay must atomically reacquire every +applicable charge before another provider call. Integrity-mismatched or +quarantined existing bytes remain completed and charged. ### Prepared Byte Sources @@ -1069,16 +1070,15 @@ manifest, semantic-manifest hash, or server content IDs. ```text POST /api/v1/tasks/{task_id}/submission-bundle-preparations -GET /api/v1/tasks/{task_id}/submission-bundle-preparations/{operation_id} POST /api/v1/tasks/{task_id}/submissions ``` The preparation POST performs scratch intake, inspection, checks, and durable handoff as one process-local orchestration. It returns only a Workstream -operation/admission identity and bounded redacted status; polling occurs only -after durable intent and never resolves a scratch path. APIs never return -provider internals. Every ID-addressed status read uses concealed deny/not-found -behavior across actors, projects, revocation, terminal state, and random IDs. +operation/admission identity and bounded redacted status. An exact idempotent +POST retry may observe the same durable operation after durable intent; v0.1 +exposes no separate preparation-status GET route. No retry resolves a scratch +path, and APIs never return provider internals. ## Guide And Checker Binding @@ -1114,10 +1114,13 @@ mismatch on an existing content-addressed key is likewise unrecoverable in v0.1: the replica remains quarantined, evaluation stays blocked, and a security incident is required. Recovery never overwrites the poisoned key. -Pre-submit and post-submit outage continuation are distinct contracts. The -former preserves a sealed, unconsumed attempt and returns the stable 503 above; -the latter preserves `evaluation_pending` and uses checker retry -infrastructure. Neither path fabricates `accept`, `needs_revision`, or `reject`. +Pre-submit and post-submit outage continuation are distinct contracts. Before +durable artifact intent, the former cleans scratch, preserves no reusable +attempt, returns the stable 503 above, and requires reupload after scratch or +process loss. After durable handoff, existing ART put/verification recovery +owns ambiguity. Post-submit checker outage preserves `evaluation_pending` and +uses checker retry infrastructure. No path fabricates `accept`, +`needs_revision`, or `reject`. ## Authorization Dependencies From 4111218c1c876bfcabe0795c21b8b53b426831d8 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Fri, 24 Jul 2026 17:39:08 +0100 Subject: [PATCH 04/26] docs(artifacts): align submission manifest templates --- docs/operations_queue_policy.md | 2 +- docs/template_preflight_evidence.md | 2 +- docs/template_project_guide.md | 8 +++++--- 3 files changed, 7 insertions(+), 5 deletions(-) diff --git a/docs/operations_queue_policy.md b/docs/operations_queue_policy.md index 84ec28eb..b1706fb1 100644 --- a/docs/operations_queue_policy.md +++ b/docs/operations_queue_policy.md @@ -254,7 +254,7 @@ Every operating day starts with: | `DRAFT -> SCREENING` | project id, locked guide candidate, task source/description fields, acceptance and rejection criteria | | `SCREENING -> READY` | screening decision, guide version lock, guide source snapshot id/hash lock, acceptance criteria, effective project submission artifact policy hash lock, project `PreSubmitCheckerPolicy` compiled bundle hash lock, approved generated project `PostSubmitCheckerPolicy` with matching provenance, review policy, revision policy | | `READY -> CLAIMED` | active published ContributionPolicyVersion whose `accepted_submission` rule is explicit; TaskAssignment freezes that version | -| `IN_PROGRESS -> SUBMITTED` | blocking pre-submit checks passed, submission packet, artifact hash manifest, evidence references, contributor attestation | +| `IN_PROGRESS -> SUBMITTED` | blocking pre-submit checks passed against the exact outer ZIP, submission packet, server-generated submission-bundle manifest and verified admission binding, evidence references, contributor attestation | | `SUBMITTED -> EVALUATION_PENDING` | immutable submission version, locked post-submit checker policy id/version/hash/body copied from the task context | | `EVALUATION_PENDING -> REVIEW_PENDING` | durable, final, current CheckerRun for the exact Submission with outcome exactly `allow_review`, verified artifact bindings, no blocking failures | | `EVALUATION_PENDING -> NEEDS_REVISION` | durable, final, current CheckerRun for the exact Submission with outcome exactly `needs_revision`, verified artifact bindings, outcome source `auto_checker`, contributor-visible checker failures with severity, message, suggested fix | diff --git a/docs/template_preflight_evidence.md b/docs/template_preflight_evidence.md index c3af85f7..c459287f 100644 --- a/docs/template_preflight_evidence.md +++ b/docs/template_preflight_evidence.md @@ -38,7 +38,7 @@ State what was checked before review. - package hash: - artifact count: - required files present: -- artifact hash manifest: +- server-generated submission-bundle manifest reference/hash: - evidence ids bound to artifact hashes: ## Review Readiness diff --git a/docs/template_project_guide.md b/docs/template_project_guide.md index 6ff1aed9..9b80e28d 100644 --- a/docs/template_project_guide.md +++ b/docs/template_project_guide.md @@ -80,13 +80,15 @@ Define prohibited behavior, tools, copied material, generated artifacts, confide ## Required Submission Fields - summary -- output files or package +- exactly one outer ZIP containing every required output/evidence file - evidence - revision replay when applicable - contributor attestation -- artifact hash manifest -Workstream assigns submission version server-side after blocking pre-submit checks pass. The contributor does not provide a submission version or any guide/policy version. +Workstream generates the archive commitment and semantic submission-bundle +manifest, then assigns the submission version server-side after blocking +pre-submit checks pass. The contributor does not provide a manifest, submission +version, or guide/policy version. ## Submission Expectations Summary From 080a4275147cf3dd7203a47e7cb5c8b6e2f2e19e Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Fri, 24 Jul 2026 17:42:02 +0100 Subject: [PATCH 05/26] docs(artifacts): clarify preflight bundle evidence --- docs/template_preflight_evidence.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/docs/template_preflight_evidence.md b/docs/template_preflight_evidence.md index c459287f..8e91cc27 100644 --- a/docs/template_preflight_evidence.md +++ b/docs/template_preflight_evidence.md @@ -32,14 +32,15 @@ State what was checked before review. | --- | --- | --- | --- | --- | | `` | `` | `` | `` | `` | -## Package Evidence +## Submission Bundle Evidence -- package path or URI: -- package hash: -- artifact count: +- server-generated outer-ZIP SHA-256: +- server-generated outer-ZIP byte count: +- inspected entry count: - required files present: - server-generated submission-bundle manifest reference/hash: -- evidence ids bound to artifact hashes: +- verified admission/binding IDs: +- evidence IDs bound to the verified admission and manifest: ## Review Readiness From 58b766ef83138081d46e57a53f39a1dd4d7dedec Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Fri, 24 Jul 2026 17:49:47 +0100 Subject: [PATCH 06/26] docs(agent-loop): finalize PLAN2 review evidence --- ...-ART-001-PLAN2-internal-review-evidence.md | 37 +++++++++++++------ 1 file changed, 26 insertions(+), 11 deletions(-) diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-internal-review-evidence.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-internal-review-evidence.md index 92325b30..41a6a2f3 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-internal-review-evidence.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-internal-review-evidence.md @@ -1,5 +1,16 @@ # Internal Review Evidence: WS-ART-001-PLAN2 +Reviewed code SHA: `12d326869608e5d55cab3da5766c00fff8719803` + +Reviewed against trusted main: `bcf1292e1a591e3e84bf8ee212ee7191d80741fa` + +Reviewed at: `2026-07-24T16:48:03Z` + +Reviewer run IDs: `art_plan2_ext_senior`, `art_plan2_ext_arch`, +`art_plan2_ext_qa`, `art_plan2_ext_security`, `art_plan2_ext_product`, +`art_plan2_ext_docs`, `art_plan2_ext_reuse`, `art_plan2_ext_ci`, +`art_plan2_ext_test_delta` + ## Candidate Planning-only reconciliation after signed cancellation of `WS-ART-001-03`. @@ -18,17 +29,17 @@ implemented or activated. ## Reviewer Results -| Track | Result | Resolved findings | -|---|---|---| -| Senior engineering | PASS after repair | Reconciled status/map history, completed 03A-C contracts, made 04A-C internal composition executable, restored isolated coverage/e2e proof. | -| Architecture | PASS after repair | Removed duplicate session/candidate paths, reused existing scratch/admission/recovery, and made cross-subsystem ownership explicit. | -| QA/test | PASS after repair | Added exact verification commands, 78/90 percent gates, nested-archive closure, process-loss behavior, and migration/history proof. | -| Security/auth | PASS after repair | Added exact AUTH handoff, fixed-service separation, pre-submit evidence privacy, and fail-closed scratch/provider boundaries. | -| Product/ops | PASS after repair | Corrected merged lineage, reviewer note/findings semantics, and Submission/revision/downstream ownership. | -| Reuse/dedup | PASS after repair | Reused `ArtifactScratchManager`, `PreparedArtifact`, `CommittedArtifactSource`, and existing admission/put/verification/recovery paths. | -| CI integrity | PASS | No workflow weakening; cumulative scoped 90 percent and repository 78 percent contracts remain enforced. | -| Test delta | PASS after repair | Replaced stale authored-state assertions without treating root projections as canonical; new phases and e2e ownership remain guarded. | -| Docs | PASS after repair | Reconciled storage/auth specs, glossary, artifact-policy template, and submission-packet template. | +| Reviewer | Result | Blocking findings | Notes | +|---|---|---|---| +| senior engineering | PASS AFTER FIXES | none | Reconciled status, state vocabulary, outage behavior, route ownership, and evidence accuracy. | +| architecture | PASS AFTER FIXES | none | Removed duplicate session/candidate paths, reused existing scratch/admission/recovery, and separated generic attempt from legacy item state. | +| QA/test | PASS AFTER FIXES | none | Proved exact verification commands, 78/90 percent gates, nested-archive closure, process-loss behavior, and governance parsing. | +| security/auth | PASS AFTER FIXES | none | Kept 04A non-routable, removed the unmapped GET, preserved AUTH activation custody, and closed canonical-path collisions. | +| product/ops | PASS AFTER FIXES | none | Corrected reviewer/revision semantics and separated pre-intent reupload from durable ART/checker recovery. | +| reuse/dedup | PASS | none | Reused `ArtifactScratchManager`, `PreparedArtifact`, `CommittedArtifactSource`, and existing admission/put/verification/recovery paths. | +| CI integrity | PASS AFTER FIXES | none | Canonical headings/merge intent parse; cumulative 90 percent and repository 78 percent gates remain enforced. | +| test delta | PASS | none | No removed/skipped tests or weakened assertions; 100 agent-gate tests pass. | +| docs | PASS AFTER FIXES | none | Reconciled storage/auth specs and all live submission/manifest templates. | ## Material Repairs @@ -53,3 +64,7 @@ implemented or activated. block ART-04A until reconciled. - Process loss before durable artifact intent requires contributor reupload in v0.1; no temporary provider custody is introduced to mask that tradeoff. + +Valid findings addressed: yes + +Open sub-agent sessions: none From 1eec271247ddc6e56f1732168011d5467df75ae3 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Fri, 24 Jul 2026 18:02:59 +0100 Subject: [PATCH 07/26] fix(agent-gates): recognize technical worker modules --- .../WS-ART-001-immutable-artifact-storage/RISKS.md | 4 ++-- .../WS-ART-001-03C-guide-source-cutover-continuation.md | 2 +- .../chunks/WS-ART-001-04B-pre-submit-admission.md | 4 ++-- scripts/check_stale_authorization_docs.py | 2 +- scripts/test_agent_gates.py | 1 + 5 files changed, 7 insertions(+), 6 deletions(-) diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/RISKS.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/RISKS.md index f47ec460..d06a2c3a 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/RISKS.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/RISKS.md @@ -42,9 +42,9 @@ | Temporary candidate storage consumes capacity or creates a second lifecycle | Critical | Candidate storage is rejected for v0.1. Unchecked bytes remain only in bounded scratch and passing bytes enter the existing immutable store once. | | Outer ZIP tree cannot be checked against the Project Guide | Critical | Safely walk every normalized file/directory entry in the outer ZIP, hash files during bounded materialization, and give the locked checker the exact read-only tree. Nested ZIP entries stay opaque throughout v0.1. | | Repackaging unchanged work wastes reviewer capacity | High | Compare both exact archive identity and canonical semantic-manifest identity with the immediate prior immutable Submission before checker/provider I/O. | -| Scratch path is sent to another worker or survives process loss | Critical | Keep 04A-04C one process-local orchestrated operation; never serialize scratch paths. Process/scratch loss invalidates the attempt and requires reupload. | +| Scratch path is sent to another process or survives process loss | Critical | Keep 04A-04C one process-local orchestrated operation; never serialize scratch paths. Process/scratch loss invalidates the attempt and requires reupload. | | Verified bytes become unbound after process/transaction loss | High | Persist exact durable intent and use idempotent retry plus existing put/verification recovery; create no Submission until verified content and admission are bindable. | | Reviewer note is confused with a revision artifact | High | REV attaches decision and note/findings to the exact Submission. Only a contributor uploads the next complete ZIP after `needs_revision`. | | Full history requires expensive recursive traversal | Medium | Preserve immutable links but add indexed latest/current/accepted repository queries or projections maintained under lifecycle transactions. | | Stored content changes after initial verification | Critical | Recompute full SHA-256 and byte count during checker materialization, reviewer full download, and downstream delivery; fail closed and record an artifact incident on mismatch. | -| Speculative larger limits exhaust disk or worker time | High | Preserve current configured limits until separately reviewed capacity, concurrency, runtime, cost, and abuse evidence justifies a change. | +| Speculative larger limits exhaust disk or processing time | High | Preserve current configured limits until separately reviewed capacity, concurrency, runtime, cost, and abuse evidence justifies a change. | diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03C-guide-source-cutover-continuation.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03C-guide-source-cutover-continuation.md index d1bd49d6..4a8380f6 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03C-guide-source-cutover-continuation.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-03C-guide-source-cutover-continuation.md @@ -14,7 +14,7 @@ adding a Project Manager resume command. - guide-source/snapshot models, one clean-cut migration, repository, schemas, service, and router; -- exact same-generation setup continuation record/worker and affected setup tasks; +- exact same-generation setup continuation record/service and affected setup tasks; - `backend/scripts/api_contract_e2e.py` only for the guide clean cut; - focused refusal, continuation, recovery, API, and real-flow tests; - stale-contract/agent-gate/backend workflow files only for the exact phase and diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04B-pre-submit-admission.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04B-pre-submit-admission.md index 923daab8..fc78fccb 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04B-pre-submit-admission.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04B-pre-submit-admission.md @@ -64,7 +64,7 @@ artifact content, provider object, or Submission. contributor route; 04C composes the one continuous same-process endpoint; - the complete operation remains process-local through 04C; process loss or scratch loss invalidates the result and requires reupload rather than routing - a later distributed worker to a local path; + a later distributed service to a local path; - completion, failure, cancellation, deadline, and abandoned-work cleanup are bounded and idempotent; - the continuous contributor surface remains hidden; this chunk does not @@ -109,5 +109,5 @@ reuse/dedup, CI integrity, test delta, and docs. ## Human Review Focus - Can a failed or unchecked bundle reach object storage? -- Can an async/distributed worker receive an unusable local scratch path? +- Can an async/distributed service receive an unusable local scratch path? - Are checker findings separate from reviewer decisions? diff --git a/scripts/check_stale_authorization_docs.py b/scripts/check_stale_authorization_docs.py index 69297841..6a7c15ca 100644 --- a/scripts/check_stale_authorization_docs.py +++ b/scripts/check_stale_authorization_docs.py @@ -385,7 +385,7 @@ def technical_worker_match(text: str, match: re.Match[str]) -> bool: ) or ( re.search(r"(?:^|[^A-Za-z0-9_.-])app\.$", prefix) is not None - and suffix.startswith(".") + and (suffix.startswith(".") or not suffix or suffix[0] in " '`,)") ) ) exact_technical_cli_flag = ( diff --git a/scripts/test_agent_gates.py b/scripts/test_agent_gates.py index 057b5b16..03e1bad7 100644 --- a/scripts/test_agent_gates.py +++ b/scripts/test_agent_gates.py @@ -5452,6 +5452,7 @@ def test_stale_authorization_rule_examples_are_rejected() -> None: "See app/workers/tasks.py.", "coverage report --include='app/workers/*' --precision=2 --fail-under=90", "ruff check app/workers/reviews.py", + "pytest --cov=app.workers --cov-report=term-missing", "review_lifecycle_live_drill.py --start-api-worker-beat --require-workers", ) for sample in technical_worker_statements: From 5934be55a1f26f9215fd823aee7526decedeb082 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Fri, 24 Jul 2026 18:10:30 +0100 Subject: [PATCH 08/26] fix(agent-gates): reject authority in worker paths --- scripts/check_stale_authorization_docs.py | 20 ++++++++++++++++++-- scripts/test_agent_gates.py | 7 +++++++ 2 files changed, 25 insertions(+), 2 deletions(-) diff --git a/scripts/check_stale_authorization_docs.py b/scripts/check_stale_authorization_docs.py index 6a7c15ca..4f2b4f65 100644 --- a/scripts/check_stale_authorization_docs.py +++ b/scripts/check_stale_authorization_docs.py @@ -377,6 +377,7 @@ def technical_worker_match(text: str, match: re.Match[str]) -> bool: prefix = text[max(0, worker_offset - 120) : worker_offset] token_end = worker_offset + len(token.group(0)) suffix = text[token_end : token_end + 20] + authority_suffix = text[token_end : token_end + 120] exact_code_path = token.group(0).lower() == "workers" and ( ( re.search(r"(?:^|[^A-Za-z0-9_.-])(?:backend/)?app/$", prefix) @@ -385,9 +386,23 @@ def technical_worker_match(text: str, match: re.Match[str]) -> bool: ) or ( re.search(r"(?:^|[^A-Za-z0-9_.-])app\.$", prefix) is not None - and (suffix.startswith(".") or not suffix or suffix[0] in " '`,)") + and suffix.startswith(".") ) ) + exact_coverage_module = ( + token.group(0).lower() == "workers" + and re.search(r"(?:^|\s)--cov=app\.$", prefix) is not None + and (not suffix or suffix[0].isspace()) + ) + after_code_reference = re.sub( + r"^(?:[/.][A-Za-z0-9_.-]+)+", "", authority_suffix + ) + code_path_claims_product_authority = exact_code_path and re.search( + r"\b(?:submits?|reviews?|approves?|grants?|revokes?|manages?|accepts?|" + r"rejects?|requests?\s+revision)\b", + after_code_reference, + re.IGNORECASE, + ) is not None exact_technical_cli_flag = ( token.group(0).lower() == "worker" and prefix.endswith("--start-api-") @@ -407,7 +422,8 @@ def technical_worker_match(text: str, match: re.Match[str]) -> bool: ) return bool( TECHNICAL_WORKER_PREFIX.search(prefix) - or exact_code_path + or (exact_code_path and not code_path_claims_product_authority) + or exact_coverage_module or exact_celery_cli or exact_technical_cli_flag ) diff --git a/scripts/test_agent_gates.py b/scripts/test_agent_gates.py index 03e1bad7..6a348a4f 100644 --- a/scripts/test_agent_gates.py +++ b/scripts/test_agent_gates.py @@ -5496,6 +5496,13 @@ def test_stale_authorization_rule_examples_are_rejected() -> None: "review_lifecycle_live_drill.py --require-workers-extra", "maliciousapp/workers/reviews.py", "maliciousapp.workers.reviews", + "app.workers submit task packets.", + "app.workers approves project work.", + "The app.workers module submits contributor packets.", + "app.workers may review a contributor submission.", + "app.workers approves project guides.", + "app.workers manages contributor grants.", + "backend/app/workers/reviews.py may review a contributor submission.", ) for sample in human_worker_statements: assert gate.scan_text("docs/new_active_doc.md", sample), sample From 12638db35cf7edb53277a36c4c173331f3a4c994 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Fri, 24 Jul 2026 18:14:16 +0100 Subject: [PATCH 09/26] test(agent-gates): reject worker path task claims --- scripts/check_stale_authorization_docs.py | 2 +- scripts/test_agent_gates.py | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/scripts/check_stale_authorization_docs.py b/scripts/check_stale_authorization_docs.py index 4f2b4f65..d589bcce 100644 --- a/scripts/check_stale_authorization_docs.py +++ b/scripts/check_stale_authorization_docs.py @@ -398,7 +398,7 @@ def technical_worker_match(text: str, match: re.Match[str]) -> bool: r"^(?:[/.][A-Za-z0-9_.-]+)+", "", authority_suffix ) code_path_claims_product_authority = exact_code_path and re.search( - r"\b(?:submits?|reviews?|approves?|grants?|revokes?|manages?|accepts?|" + r"\b(?:claims?|submits?|reviews?|approves?|grants?|revokes?|manages?|accepts?|" r"rejects?|requests?\s+revision)\b", after_code_reference, re.IGNORECASE, diff --git a/scripts/test_agent_gates.py b/scripts/test_agent_gates.py index 6a348a4f..4dae85ee 100644 --- a/scripts/test_agent_gates.py +++ b/scripts/test_agent_gates.py @@ -5503,6 +5503,8 @@ def test_stale_authorization_rule_examples_are_rejected() -> None: "app.workers approves project guides.", "app.workers manages contributor grants.", "backend/app/workers/reviews.py may review a contributor submission.", + "backend/app/workers/reviews.py claims a task.", + "backend/app/workers/tasks.py can claim a task.", ) for sample in human_worker_statements: assert gate.scan_text("docs/new_active_doc.md", sample), sample From 2c210b80b4464ec3b7d30c5827a9a9db30e4541c Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Fri, 24 Jul 2026 18:18:41 +0100 Subject: [PATCH 10/26] fix(agent-gates): restrict worker paths to references --- scripts/check_stale_authorization_docs.py | 24 ++++++++++++++++------- scripts/test_agent_gates.py | 5 +++++ 2 files changed, 22 insertions(+), 7 deletions(-) diff --git a/scripts/check_stale_authorization_docs.py b/scripts/check_stale_authorization_docs.py index d589bcce..14153c6d 100644 --- a/scripts/check_stale_authorization_docs.py +++ b/scripts/check_stale_authorization_docs.py @@ -397,12 +397,22 @@ def technical_worker_match(text: str, match: re.Match[str]) -> bool: after_code_reference = re.sub( r"^(?:[/.][A-Za-z0-9_.-]+)+", "", authority_suffix ) - code_path_claims_product_authority = exact_code_path and re.search( - r"\b(?:claims?|submits?|reviews?|approves?|grants?|revokes?|manages?|accepts?|" - r"rejects?|requests?\s+revision)\b", - after_code_reference, - re.IGNORECASE, - ) is not None + exact_coverage_path = bool( + exact_code_path + and re.search(r"coverage\s+report\s+--include=['\"]app/$", prefix) + and re.match(r"/\*['\"](?:\s|$)", suffix) + ) + exact_celery_module_path = bool( + exact_code_path + and re.search(r"(?:^|\n)[^;\n]*\bcelery\s+-A\s+app\.$", prefix) + and re.match(r"\s+worker\s+--", after_code_reference) + ) + code_path_has_prose = bool( + exact_code_path + and not exact_coverage_path + and not exact_celery_module_path + and re.search(r"[A-Za-z]", after_code_reference) + ) exact_technical_cli_flag = ( token.group(0).lower() == "worker" and prefix.endswith("--start-api-") @@ -422,7 +432,7 @@ def technical_worker_match(text: str, match: re.Match[str]) -> bool: ) return bool( TECHNICAL_WORKER_PREFIX.search(prefix) - or (exact_code_path and not code_path_claims_product_authority) + or (exact_code_path and not code_path_has_prose) or exact_coverage_module or exact_celery_cli or exact_technical_cli_flag diff --git a/scripts/test_agent_gates.py b/scripts/test_agent_gates.py index 4dae85ee..b432c7bd 100644 --- a/scripts/test_agent_gates.py +++ b/scripts/test_agent_gates.py @@ -5505,6 +5505,11 @@ def test_stale_authorization_rule_examples_are_rejected() -> None: "backend/app/workers/reviews.py may review a contributor submission.", "backend/app/workers/reviews.py claims a task.", "backend/app/workers/tasks.py can claim a task.", + "backend/app/workers/tasks.py creates a project.", + "backend/app/workers/tasks.py can create a project.", + "app/workers/tasks.py creates a project.", + "backend/app/workers/tasks.py uses submitter authority.", + "backend/app/workers/tasks.py is a human product role.", ) for sample in human_worker_statements: assert gate.scan_text("docs/new_active_doc.md", sample), sample From 72042613ce5870842ddf71c17b40b97f7d5ee04b Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Fri, 24 Jul 2026 18:25:02 +0100 Subject: [PATCH 11/26] docs(agent-loop): authorize scanner gate repair --- .../WS-ART-001-PLAN2-submission-bundle-reconciliation.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-PLAN2-submission-bundle-reconciliation.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-PLAN2-submission-bundle-reconciliation.md index 890acfbd..db93639f 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-PLAN2-submission-bundle-reconciliation.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-PLAN2-submission-bundle-reconciliation.md @@ -28,7 +28,12 @@ AUTH, task, checker, review, contribution, or delivery behavior. - `scripts/test_agent_gates.py` only to replace stale ART status/chunk-map assertions with the exact merged/cancelled planning projection, bind new successor phases/coverage contracts, and preserve the signed-automation-only - authority warning instead of treating copied root projections as live state; + authority warning instead of treating copied root projections as live state, + plus focused regression assertions required by an in-scope documentation + gate repair; +- `scripts/check_stale_authorization_docs.py` only for the narrow technical + `app.workers` module/path recognition repair required to scan the changed + contracts without re-admitting human/product worker-authority vocabulary; - cross-initiative handoff prose that names dependencies without modifying another initiative's owned runtime contract; - exactly one merge intent for this planning chunk. From ad1ffdf14601d9ce2e544a716f5c82fe2da057b3 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Fri, 24 Jul 2026 18:31:33 +0100 Subject: [PATCH 12/26] docs(agent-loop): clarify scanner repair scope --- .../WS-ART-001-PLAN2-submission-bundle-reconciliation.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-PLAN2-submission-bundle-reconciliation.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-PLAN2-submission-bundle-reconciliation.md index db93639f..dce4aabc 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-PLAN2-submission-bundle-reconciliation.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-PLAN2-submission-bundle-reconciliation.md @@ -32,8 +32,8 @@ AUTH, task, checker, review, contribution, or delivery behavior. plus focused regression assertions required by an in-scope documentation gate repair; - `scripts/check_stale_authorization_docs.py` only for the narrow technical - `app.workers` module/path recognition repair required to scan the changed - contracts without re-admitting human/product worker-authority vocabulary; + background-service module/path recognition repair required to scan the changed + contracts without re-admitting deprecated human product-role vocabulary; - cross-initiative handoff prose that names dependencies without modifying another initiative's owned runtime contract; - exactly one merge intent for this planning chunk. From f36d237791e5192c7969d4add9749eb98125fbca Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Fri, 24 Jul 2026 18:40:00 +0100 Subject: [PATCH 13/26] docs(agent-loop): refresh PLAN2 repair evidence --- .../reviews/WS-ART-001-PLAN2-internal-review-evidence.md | 6 +++--- .../reviews/WS-ART-001-PLAN2-pr-trust-bundle.md | 2 ++ 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-internal-review-evidence.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-internal-review-evidence.md index 41a6a2f3..6196dc2d 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-internal-review-evidence.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-internal-review-evidence.md @@ -1,10 +1,10 @@ # Internal Review Evidence: WS-ART-001-PLAN2 -Reviewed code SHA: `12d326869608e5d55cab3da5766c00fff8719803` +Reviewed code SHA: `afb883dd4e8ab52fad3d7301b5e865d3a25885a7` Reviewed against trusted main: `bcf1292e1a591e3e84bf8ee212ee7191d80741fa` -Reviewed at: `2026-07-24T16:48:03Z` +Reviewed at: `2026-07-24T17:35:13Z` Reviewer run IDs: `art_plan2_ext_senior`, `art_plan2_ext_arch`, `art_plan2_ext_qa`, `art_plan2_ext_security`, `art_plan2_ext_product`, @@ -37,7 +37,7 @@ implemented or activated. | security/auth | PASS AFTER FIXES | none | Kept 04A non-routable, removed the unmapped GET, preserved AUTH activation custody, and closed canonical-path collisions. | | product/ops | PASS AFTER FIXES | none | Corrected reviewer/revision semantics and separated pre-intent reupload from durable ART/checker recovery. | | reuse/dedup | PASS | none | Reused `ArtifactScratchManager`, `PreparedArtifact`, `CommittedArtifactSource`, and existing admission/put/verification/recovery paths. | -| CI integrity | PASS AFTER FIXES | none | Canonical headings/merge intent parse; cumulative 90 percent and repository 78 percent gates remain enforced. | +| CI integrity | PASS AFTER FIXES | none | Canonical headings/merge intent parse; stale-auth scanner remains fail-closed; cumulative 90 percent and repository 78 percent gates remain enforced. | | test delta | PASS | none | No removed/skipped tests or weakened assertions; 100 agent-gate tests pass. | | docs | PASS AFTER FIXES | none | Reconciled storage/auth specs and all live submission/manifest templates. | diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-pr-trust-bundle.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-pr-trust-bundle.md index 308a102d..6b6007a5 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-pr-trust-bundle.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-pr-trust-bundle.md @@ -37,6 +37,8 @@ successor `WS-ART-001-03A` with a separate explicit start. - diff, stale-contract, markdown-link, and 100 agent-gate tests pass; - all nine required reviewer tracks pass after repair; +- the stale-authorization documentation gate and its technical-module + regression repair pass without reopening deprecated product-role vocabulary; - detailed findings and resolutions are recorded in `WS-ART-001-PLAN2-internal-review-evidence.md`. From 57a7d1e6016cdf142c7f84fa44ec78f407250fe6 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Sat, 25 Jul 2026 09:55:43 +0100 Subject: [PATCH 14/26] docs(artifacts): close PLAN2 admission lifecycle gaps --- .../AUTH_HANDOFF.md | 21 ++++ .../CHUNK_MAP.md | 10 +- .../DECISIONS.md | 36 ++++++- .../INTENT.md | 8 ++ .../PLAN.md | 62 +++++++++++- .../RISKS.md | 4 + .../STATUS.md | 6 ++ ...S-ART-001-04A-upload-inspection-sealing.md | 20 +++- .../WS-ART-001-04B-pre-submit-admission.md | 7 ++ ...4C-verified-submission-bundle-admission.md | 38 +++++-- ...-ART-001-05-submission-artifact-cutover.md | 34 ++++++- ...T-001-06A-checker-input-materialization.md | 6 ++ ...-PLAN2-submission-bundle-reconciliation.md | 15 ++- docs/glossary.md | 12 ++- docs/spec_artifact_storage_service.md | 99 ++++++++++++++++++- docs/template_submission_artifact_policy.md | 6 ++ docs/template_submission_packet.md | 9 +- scripts/test_agent_gates.py | 28 ++++++ 18 files changed, 392 insertions(+), 29 deletions(-) diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/AUTH_HANDOFF.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/AUTH_HANDOFF.md index 157e445e..e6361990 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/AUTH_HANDOFF.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/AUTH_HANDOFF.md @@ -46,6 +46,27 @@ activation contract may integrate the evaluator and change only `artifact.submission_bundle.prepare` to active. ART-05 cannot start until that activation merges. +The preparation surface authorizes before scratch intake, but the initial +decision cannot authorize the later durable mutation. Immediately before +capacity reservation and `ArtifactPutAttempt` creation, 04C must consume an +AUTH-owned transaction-local prepared capability whose canonical facts cover +the current actor, exact identity link, project authority, assignment, task, +predecessor, locked task/guide/policy context, action availability, and +operation generation. AUTH and the owning product services reload/lock their +own facts; ART receives only the typed capability and never imports AUTH-owned +repositories. Authorization evidence, capacity reservation, and durable put +intent commit atomically before provider I/O. + +ART-05 requires a new human authorization decision for `submission.create` and +a separately prepared fixed-service capability for +`artifact.binding.create`. Both are consumed in the one transaction that locks +the ready admission and TASK-owned context, creates Submission and binding, and +marks the admission consumed. Human authority implies no service authority. +Revocation after durable put intent does not cancel verification/recovery, but +the resulting admission remains unbound until a fresh 05 decision succeeds. +Authorization denial precedes admission-detail errors so unrelated actors +cannot distinguish missing, ready, stale, or consumed admissions. + The continuous contributor action never implies the fixed service actions: - `artifact.checker_input.materialize`; diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/CHUNK_MAP.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/CHUNK_MAP.md index 1272658d..8af1f995 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/CHUNK_MAP.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/CHUNK_MAP.md @@ -20,11 +20,11 @@ Each chunk is one PR. No later chunk starts automatically. | `WS-ART-001-03A` | Add hidden guide-source byte ingest through existing preparation, admission, verification, and publication. | L1 | Proposed after PLAN2 | | `WS-ART-001-03B` | Bind verified guide-source content and provide authorized integrity-checking setup materialization. | L1 | Proposed after 03A and exact AUTH activation | | `WS-ART-001-03C` | Remove legacy guide-source identity and add exact same-generation setup continuation. | L1 | Proposed after 03B and exact AUTH activation | -| `WS-ART-001-04A` | Accept one outer ZIP in bounded scratch, safely inspect its tree, produce canonical identities, and reject unchanged work before provider I/O. | L1 | Proposed after 03C and AUTH planned action registration | -| `WS-ART-001-04B` | Run mandatory platform and locked Project Guide pre-submit checks against the same scratch-bound tree without durable storage. | L1 | Proposed after 04A | -| `WS-ART-001-04C` | Admit the passing ZIP once through existing ArtifactStore, independently verify it, and publish one bindable admission. | L1 | Proposed after 04B; AUTH activation follows hidden completion | -| `WS-ART-001-05` | Atomically bind one verified admission to one immutable Submission and remove legacy caller transport authority. | L1 | Proposed after 04C and exact AUTH activation | -| `WS-ART-001-06A` | Persist checker input snapshots and materialize authorized immutable bytes into bounded checker workspaces. | L1 | Proposed after 05 | +| `WS-ART-001-04A` | Accept one outer ZIP in bounded scratch, safely inspect its tree, normalize executable intent, produce canonical identities, and reject unchanged work before provider I/O. | L1 | Proposed after 03C and AUTH planned action registration | +| `WS-ART-001-04B` | Run mandatory platform and locked Project Guide pre-submit checks against the same scratch-bound tree and executable semantics without durable storage. | L1 | Proposed after 04A | +| `WS-ART-001-04C` | Reauthorize at durable intent, admit/verify the passing ZIP once, and publish one capacity-charged `ready` admission that may remain unbound. | L1 | Proposed after 04B; AUTH activation follows hidden completion | +| `WS-ART-001-05` | Freshly authorize and atomically consume one ready admission into one immutable Submission/binding, or terminally stale it on proven context drift. | L1 | Proposed after 04C and exact AUTH activation | +| `WS-ART-001-06A` | Persist checker input snapshots and materialize authorized immutable bytes with identical normalized executable semantics. | L1 | Proposed after 05 | | `WS-ART-001-06B` | Ingest checker logs/outputs as artifacts, persist checker completion facts, and preserve existing checker-owned routing without creating review aggregates. | L1 | Proposed after 06A | | `WS-ART-001-07` | Prove Local/MinIO plus AWS S3 readiness, Operator recovery, and exact-byte guide/pre/post-submit behavior through real APIs. | L1 | Proposed after 06B | diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/DECISIONS.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/DECISIONS.md index 388aa8bc..68b83c57 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/DECISIONS.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/DECISIONS.md @@ -283,9 +283,13 @@ observation, verification, receipt, scanner, and recovery abstractions. Workstream records both exact archive identity (server-computed SHA-256 and byte count) and semantic tree identity. The semantic manifest commits to normalized -file/directory path, entry type, and each file's SHA-256 and byte count. It does +file/directory path, entry type, each file's SHA-256 and byte count, and the +normalized executable flag for regular files. It does not commit to archive timestamps, compression method/level, comments, ownership, -or platform permission metadata. Symlinks and unsupported special entries are +or ownership, group, read/write/special bits, or other platform permission +metadata. Valid Unix execute bits are normalized into the sole semantic +permission flag; non-Unix/invalid mode metadata defaults it to false. Symlinks +and unsupported special entries are rejected. Exact archive equality or semantic-manifest equality with the immediate prior immutable Submission rejects before provider I/O. @@ -319,7 +323,33 @@ unchanged. Any larger ZIP, extracted-byte, per-file, entry-count, depth, compression-ratio, scratch, or time limit requires a separate reviewed change with concurrency, capacity, runtime, cost, and abuse evidence. -## D35 - One Contributor Preparation Action +## D35 - Verified Admissions May Remain Unbound + +`SubmissionBundleAdmission` has the closed lifecycle `ready -> consumed|stale`. +Client abandonment is valid and remains charged to existing completed-byte +capacity. Consumption and Submission/binding creation are one transaction with +database uniqueness. Proven task/predecessor/locked-context drift makes a ready +admission stale; authority loss alone does not. No expiry, release, deletion, +retention process, or cleanup lifecycle exists in v0.1. + +## D36 - Authorization Is Fresh At Both Durable Mutations + +Initial route authorization is insufficient. 04C consumes an AUTH-owned +transaction-local prepared capability in the durable-intent transaction before +provider I/O. 05 obtains and consumes new human submission and fixed-service +binding capabilities in the atomic Submission/binding/admission transaction. +ART never reads or locks AUTH-owned persistence directly. + +## D37 - Executable Intent Is Canonical, Not Archive Permission Preservation + +Regular-file executable intent participates in semantic identity. Materializers +project fixed read-only or read-and-execute modes and fixed read-and-traverse +directory modes; they never preserve arbitrary ZIP permissions or automatically +execute a file. The Project Guide/checker policy remains the execution decision +owner. Shared scratch startup rejects inconsistent canonical mode configuration, +and secure cleanup remains owned by `ArtifactScratchManager`. + +## D38 - One Contributor Preparation Action The prior multi-step upload-session action proposal does not govern the new continuous scratch-bound flow. AUTH first registers planned ActionId diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/INTENT.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/INTENT.md index 80342717..374bd689 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/INTENT.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/INTENT.md @@ -55,6 +55,7 @@ one immutable Submission row/version -> one canonical semantic manifest -> mandatory platform and locked Project Guide prechecks -> one ArtifactStore admission and complete read-back verification +-> one capacity-charged ready admission, which may remain unbound -> one exact ArtifactBinding -> the same bytes checked, reviewed, accepted, recorded, and delivered ``` @@ -87,6 +88,13 @@ unpacking is outside v0.1 even when a guide would otherwise request it. - The existing `Submission` row is the immutable version aggregate. Reviewers attach only a decision and note/findings to that exact version; a contributor response to `needs_revision` is another complete ZIP and immutable Submission. +- A verified admission may remain unbound through client abandonment. It stays + capacity-charged and creates no product lifecycle effect; consumption is + atomic with Submission/binding creation and no admission expires or deletes + bytes in v0.1. +- Regular-file executable intent is normalized into semantic identity and + materialized consistently, without preserving arbitrary archive permissions + or granting execution authority. - Local storage is forbidden in staging and production. - No compatibility alias or dual provider-construction path is retained. diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md index e3c58eaf..02a45697 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md @@ -287,11 +287,15 @@ and derives two identities: ```text archive identity = SHA-256 and byte count of the exact outer ZIP semantic identity = canonical hash of normalized path, entry type, - file SHA-256, and file byte count + file SHA-256, file byte count, and normalized executable flag ``` -ZIP timestamps, compression settings, comments, ownership, and platform -permission metadata do not change semantic identity. Explicit empty directories +ZIP timestamps, compression settings, comments, ownership, group, read/write +bits, special bits, and other platform permission metadata do not change +semantic identity. Executable intent is the sole permission-derived semantic: +for a regular entry created with valid Unix mode metadata it is true when any +execute bit is present; otherwise it is false. Directories have no executable +field. Explicit empty directories and synthetic parents use one documented canonical representation. Symlinks and special entries are rejected. Canonical paths use `/` separators, contain only relative non-empty segments, and normalize Unicode to NFC. The identity remains @@ -319,6 +323,58 @@ matches. Existing put-attempt, observation, verification, receipt, scanner, and recovery behavior resolves ambiguity. No candidate namespace, promotion copy, retention window, physical deletion, or second recovery aggregate exists. +### Verified Submission-Bundle Admission + +Verification publishes an immutable, capacity-charged +`SubmissionBundleAdmission` in `ready`. It binds the preparing actor and +identity-link provenance, project, task, assignment, immediate predecessor, +exact locked task/guide/policy context, verified `ArtifactContent`, semantic +manifest, and immutable pre-submit evidence set. Client abandonment is an +accepted quota-bounded v0.1 outcome: a `ready` admission may remain unbound +without creating a Submission, review, contribution, compensation, or +reputation effect. + +The only transitions are `ready -> consumed` and `ready -> stale`. +Consumption occurs in the same transaction that creates the immutable +Submission and binding, and database uniqueness permits one consuming +Submission. Proven task closure, predecessor advancement, or locked-context +replacement makes a still-ready admission `stale` during a consumption attempt. +Authority loss alone does not make it stale; every attempt obtains fresh AUTH +authority, and restored authority may consume a still-compatible admission. +`consumed` and `stale` are terminal. No state expires, releases capacity, or +authorizes provider deletion. Ready, stale, and consumed admissions continue to +count against existing completed-byte scopes. Existing Operator admission-usage +projections add bounded unbound-ready and stale counts/bytes without exposing +content or provider identities. + +Exact preparation replay returns the original admission without another put, +charge, evidence set, or admission. Uniqueness and status/terminal-field check +constraints enforce the lifecycle; the Submission owns a unique admission +reference as the second database fence. + +### Durable Authorization Boundaries + +The contributor request is authorized before scratch intake. Immediately before +durable capacity reservation and `ArtifactPutAttempt` creation, 04C opens the +owning transaction and consumes AUTH's transaction-local prepared capability. +AUTH—not ART—reloads and validates the current ActorProfile, exact identity link, +project authority, assignment, action availability, and canonical resource +facts. TASK/PROJECT owners lock task, predecessor, and locked context through +their typed capabilities. Authorization evidence, capacity reservation, and put +intent commit atomically; failure means no provider I/O and scratch cleanup. + +Durable put intent creates a technical recovery obligation that later human +revocation does not cancel. Verification may finish, but binding remains +impossible until 05 obtains fresh human `submission.create` authority and fresh +fixed-service `artifact.binding.create` authority. 05 consumes both prepared +capabilities in the single task transaction that locks the admission/context, +creates Submission and binding, and marks the admission consumed. Denial, +cancellation, stale execution, or persistence failure rolls back every protected +effect. After authority succeeds, proven task closure, predecessor advancement, +or locked-context replacement may instead commit only the terminal stale +transition and bounded evidence, with no Submission or binding. ART/TASK never +imports or locks AUTH-owned tables. + ## Durable Verification And Recovery Background jobs have exactly one operation class: diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/RISKS.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/RISKS.md index d06a2c3a..c64b8b41 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/RISKS.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/RISKS.md @@ -44,6 +44,10 @@ | Repackaging unchanged work wastes reviewer capacity | High | Compare both exact archive identity and canonical semantic-manifest identity with the immediate prior immutable Submission before checker/provider I/O. | | Scratch path is sent to another process or survives process loss | Critical | Keep 04A-04C one process-local orchestrated operation; never serialize scratch paths. Process/scratch loss invalidates the attempt and requires reupload. | | Verified bytes become unbound after process/transaction loss | High | Persist exact durable intent and use idempotent retry plus existing put/verification recovery; create no Submission until verified content and admission are bindable. | +| Contributor abandons a verified admission | High | Treat unbound `ready` as a valid capacity-charged state, expose bounded Operator counts/bytes, enforce existing quotas, and add no expiry, release, deletion, or retention process. | +| One admission creates multiple Submissions | Critical | Atomic `ready -> consumed` transition, row lock/CAS, unique Submission admission reference, terminal-state constraints, and concurrent-consumption tests. | +| ZIP executable intent is lost or arbitrary permissions escape | High | Normalize only valid Unix execute bits for regular files into the semantic manifest; default other entries false, reject special entries, project fixed sealed modes, and never preserve arbitrary archive permissions. | +| Authority changes during long preparation or before binding | Critical | Consume fresh AUTH-owned prepared capabilities in both durable transactions; commit authorization evidence with the protected mutation, perform no provider I/O before the first commit, and never let human authority substitute for fixed binding authority. | | Reviewer note is confused with a revision artifact | High | REV attaches decision and note/findings to the exact Submission. Only a contributor uploads the next complete ZIP after `needs_revision`. | | Full history requires expensive recursive traversal | Medium | Preserve immutable links but add indexed latest/current/accepted repository queries or projections maintained under lifecycle transactions. | | Stored content changes after initial verification | Critical | Recompute full SHA-256 and byte count during checker materialization, reviewer full download, and downstream delivery; fail closed and record an artifact incident on mismatch. | diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/STATUS.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/STATUS.md index 3c84b475..3bb2d701 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/STATUS.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/STATUS.md @@ -45,6 +45,12 @@ note/findings to the exact `Submission`; contributors answer `needs_revision` with another complete ZIP and immutable Submission. +PLAN2 also treats client-abandoned verified admissions as valid, bounded, +capacity-charged `ready` facts with terminal `consumed|stale` outcomes; it +normalizes regular-file executable intent into the semantic manifest; and it +requires fresh AUTH prepared capabilities at durable put intent and atomic +Submission/binding consumption. + ## Next Proposed Chunk After this planning package merges, `WS-ART-001-03A` is the only immediate ART diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04A-upload-inspection-sealing.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04A-upload-inspection-sealing.md index 01f85279..efe56046 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04A-upload-inspection-sealing.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04A-upload-inspection-sealing.md @@ -48,8 +48,13 @@ or Submission creation. path-depth/path-length, compression-ratio, deadline, concurrency, and free-space limits without allocating from untrusted declared sizes; - the canonical semantic manifest records normalized file and directory paths, - entry type, and file SHA-256/byte count; timestamps, compression, comments, - ownership, and platform permission metadata do not affect its hash; + entry type, file SHA-256/byte count, and `executable` for regular files; + valid Unix mode metadata normalizes any execute bit to true, while non-Unix + or invalid mode metadata defaults false and directories record null; +- timestamps, compression, comments, ownership, group, read/write/special bits, + and other platform permission metadata do not affect the manifest hash; +- same bytes with a changed normalized executable flag produce different + semantic hashes, while timestamp/compression-only repackaging does not; - explicit empty directories and deterministic synthetic parents have one documented canonical representation; - exact archive hash and semantic manifest hash are compared with the immediate @@ -70,6 +75,17 @@ or Submission creation. remains; - process loss requires reupload; normal, error, cancellation, timeout, and crash cleanup release scratch quota without deleting durable artifacts; +- ZIP inspection never blindly applies archive modes; the scratch manager seals + regular files to fixed read-only or read-and-execute modes and directories to + a fixed read-and-traverse mode, with startup parity across every process that + shares the scratch root and secure cleanup owned by the manager; cleanup may + restore only manager-owned directory write access under an exclusive valid + lease after no-follow ownership/type checks; +- executable intent alone never causes execution; the locked Project Guide and + checker policy remain the execution decision owners; +- focused tests prove false-to-true execute-bit changes alter semantic hash, + timestamp/compression-only changes do not, non-Unix entries default false, + and symlink/special entries are rejected; - 04A declares no contributor route or routable action manifest entry; the planned `artifact.submission_bundle.prepare` action and AUTH mapping are composed only by 04C after the complete internal operation exists; diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04B-pre-submit-admission.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04B-pre-submit-admission.md index fc78fccb..3789f742 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04B-pre-submit-admission.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04B-pre-submit-admission.md @@ -49,6 +49,10 @@ artifact content, provider object, or Submission. while reviewers, CON, and delivery cannot consume it before Submission binding; - the checker receives a read-only private workspace and no provider reference, credential, mutable intake path, or arbitrary host path; +- the workspace projects the manifest's normalized executable flag using the + same fixed sealed modes as post-submit materialization; it never preserves + arbitrary ZIP permission metadata or executes a file merely because the flag + is true; - checker workspace creation uses the distinct fixed service action `artifact.checker_input.materialize`; contributor preparation authority never grants or substitutes for that service authority; @@ -67,6 +71,9 @@ artifact content, provider object, or Submission. a later distributed service to a local path; - completion, failure, cancellation, deadline, and abandoned-work cleanup are bounded and idempotent; +- focused tests prove pre-submit and post-submit views expose identical + executable semantics for Unix, non-Unix, symlink/special, and permission-only + revision cases; - the continuous contributor surface remains hidden; this chunk does not activate the planned contributor action or change fixed service grants; - focused subsystem coverage is at least 90 percent and repository coverage diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04C-verified-submission-bundle-admission.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04C-verified-submission-bundle-admission.md index 8b254710..31ca953e 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04C-verified-submission-bundle-admission.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04C-verified-submission-bundle-admission.md @@ -12,7 +12,8 @@ verify it, and publish one bindable submission-bundle admission. ## Allowed Files -- ART admission orchestration and one submission-bundle admission model/migration; +- ART admission orchestration and one submission-bundle admission model/migration + with the closed `ready|consumed|stale` lifecycle and database constraints; - existing put-attempt, replica, receipt, verification-job, recovery, scanner, and scratch-source integration needed for this producer; - hidden submission-bundle admission surfaces and ART resource facts/guards; @@ -34,6 +35,14 @@ verify it, and publish one bindable submission-bundle admission. commitment, semantic manifest, locked task context, actor, and predecessor; - the predecessor is rechecked under task/version locking before provider I/O; drift invalidates the attempt and requires a fresh upload/check; +- initial request authorization occurs before scratch intake; immediately before + durable capacity/put intent, the owning transaction consumes AUTH's fresh + prepared capability and typed TASK/PROJECT context capabilities without ART + importing or locking AUTH-owned tables; +- current actor/identity, project authority, assignment, task, predecessor, + locked context, canonical resource facts, authorization evidence, capacity, + and put intent validate/commit atomically; denial or drift causes no provider + I/O, durable admission, or Submission and cleans scratch; - the existing generic admission service derives and reserves authoritative byte scopes before one conditional `ArtifactStore.put`; - Workstream persists the put attempt before I/O and uses existing observation, @@ -45,7 +54,11 @@ verify it, and publish one bindable submission-bundle admission. - provider acknowledgement alone creates no bindable admission; - complete provider read-back recomputes ZIP SHA-256 and byte count; only an exact match publishes `ArtifactContent`, a ready replica, and one immutable - admission bound to the manifest and checker evidence; + `ready` admission bound to preparation actor/identity provenance, project, + task, assignment, predecessor, exact locked context, manifest, and immutable + checker-evidence set; +- the admission lifecycle is exactly `ready -> consumed|stale`; only 05 may + perform either terminal transition and both terminal states are irreversible; - provider absence, mismatch, ambiguity, or unavailability creates no Submission and never becomes checker failure or review state; - exact replay is idempotent and concurrent replay creates one provider/business @@ -53,9 +66,14 @@ verify it, and publish one bindable submission-bundle admission. - scratch is cleaned after safe source handoff or terminal outcome; if process loss occurs before durable put intent, reupload is required; after durable intent, existing ART recovery resolves the operation; -- unbound verified content can result only from a transaction/process failure - after successful admission and is resolved through exact idempotent retry; - v0.1 adds no deletion path; +- client abandonment may leave a valid unbound `ready` admission; it creates no + product lifecycle effect and remains charged to existing completed-byte + scopes alongside terminal `stale`/`consumed` admissions; +- exact preparation replay returns the original admission with no new provider + write, charge, evidence set, or admission; v0.1 adds no expiry, release, + deletion, retention process, or cleanup route; +- Operator capacity projections expose bounded counts/bytes for unbound ready + and stale admissions through existing admission-usage authority; - after 04A-04C merge their complete hidden manifest, AUTH alone may activate `artifact.submission_bundle.prepare`; ART changes no action availability; - 04C alone declares the hidden contributor route and routable @@ -67,7 +85,15 @@ verify it, and publish one bindable submission-bundle admission. no scratch handle/path is serialized into a later request or Celery payload; - v0.1 exposes no separate preparation-status GET route; an exact idempotent POST retry may return only the same bounded durable operation/admission state - under the original `artifact.submission_bundle.prepare` authorization; + after a fresh `artifact.submission_bundle.prepare` decision; it never relies + on the original human authorization evidence; +- revocation after committed put intent does not cancel fixed-service technical + verification/recovery, but no later Submission consumption inherits the old + human decision; +- crossed-state tests revoke authority during upload and checker execution and + prove no durable put/provider I/O; revocation after durable intent permits + verification only; exact replay and abandoned-ready capacity projections are + single-effect and bounded; - focused subsystem coverage is at least 90 percent and repository coverage remains at least 78 percent. diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-05-submission-artifact-cutover.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-05-submission-artifact-cutover.md index 5b376d17..638b034f 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-05-submission-artifact-cutover.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-05-submission-artifact-cutover.md @@ -37,9 +37,24 @@ package transport/hash/manifest authority. outer ZIP plus its canonical semantic manifest and checker evidence; - request body contains only contributor-authored product fields and the opaque verified admission identifier; all artifact identity is server-owned; -- one transaction locks task/assignment/admission, rechecks predecessor and - locked context, consumes admission once, allocates the next version, creates - the immutable binding, and enters the existing post-submit lifecycle; +- before revealing admission state/context, the route obtains a fresh human + authorization decision; AUTH denial remains concealed from unrelated actors; +- one transaction consumes fresh prepared human `submission.create` and fixed + service `artifact.binding.create` capabilities, locks task, assignment, + admission and predecessor, and recomposes final resource facts without TASK + or ART importing AUTH-owned repositories; +- the transaction requires `ready`, matches actor profile, task, project, + assignment, predecessor and exact locked context, allocates the next version, + creates the immutable Submission and binding, and changes admission to + `consumed` before one commit; +- the admission records preparation identity-link provenance, while consumption + records its new decision/identity evidence and requires the same canonical + actor rather than an obsolete identity-link row; +- database status/terminal-field checks and a unique Submission admission + reference guarantee at most one consumer; `consumed` and `stale` are terminal; +- proven task closure, predecessor advancement, or locked-context replacement + changes a still-ready admission to `stale`; authority loss alone does not, so + restored authority may later consume a still-compatible ready admission; - binding creation uses fixed service action `artifact.binding.create`; the contributor's active `artifact.submission_bundle.prepare` action and `submission.create` permission do not imply internal binding authority; @@ -47,6 +62,19 @@ package transport/hash/manifest authority. to a `needs_revision` Review is added only through the reviewed REV/TASK joint contract and cannot be fabricated from a note or client ID; - exact replay/concurrency creates one Submission and one business effect; +- concurrent consumption yields one success and one exact replay or stable + conflict; cancellation, denial, service-action unavailability, context drift, + stale execution, or persistence failure creates no Submission/binding; +- after successful authorization, proven task closure, predecessor advancement, + or locked-context replacement may commit only the terminal stale transition + and bounded evidence; every other failed mutation rolls back authorization + evidence and admission transition together; +- stable authorized outcomes distinguish already-consumed, context-changed, and + stale admissions, but unauthorized callers learn no admission existence/state; +- crossed-state tests cover authority restoration for a compatible ready + admission, predecessor advancement to stale, disabled binding service action, + token-valid but revoked identity/grant/assignment, stale execution, and two + concurrent requests producing one business effect; - latest/current/accepted access uses indexed repository queries or projections, while the immutable version/review graph remains fully queryable; - no manager finalization is required for an ordinary passing contributor diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-06A-checker-input-materialization.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-06A-checker-input-materialization.md index 3b54b881..3ba3ab3d 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-06A-checker-input-materialization.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-06A-checker-input-materialization.md @@ -58,6 +58,12 @@ bounded isolated checker workspaces. - files are sealed read-only after digest/size verification and before checker access; success, failure, cancellation, and crash/stale paths cannot orphan confidential bytes; +- materialization reproduces each manifest entry's normalized executable flag + with the same fixed read-only/read-and-execute/read-and-traverse modes used by + pre-submit scratch; arbitrary archive modes are never restored and executable + intent alone never authorizes execution; +- parity tests prove identical pre/post materialization semantics and reject any + mode configuration mismatch across processes sharing a scratch root; - transient unavailability leaves the post-submit task in `evaluation_pending`; missing/integrity mismatch blocks execution as an artifact incident and never becomes a review decision; diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-PLAN2-submission-bundle-reconciliation.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-PLAN2-submission-bundle-reconciliation.md index dce4aabc..e92cf33b 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-PLAN2-submission-bundle-reconciliation.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-PLAN2-submission-bundle-reconciliation.md @@ -60,8 +60,9 @@ AUTH, task, checker, review, contribution, or delivery behavior. - bounded private scratch is the only pre-admission custody; failed, unsafe, unchanged, abandoned, or checker-failing attempts never enter object storage; - the canonical semantic manifest commits to normalized file/directory paths, - entry type, and each file's SHA-256 and byte count while excluding packaging, - timestamps, ownership, compression, and platform permission metadata; + entry type, each file's SHA-256/byte count, and normalized regular-file + executable intent while excluding packaging, timestamps, ownership, group, + read/write/special bits, compression, and other permission metadata; - exact archive equality and manifest equality are compared with the immediate prior immutable Submission and both reject before provider I/O; - all mandatory platform gates and locked project pre-submit checks pass before @@ -74,6 +75,16 @@ AUTH, task, checker, review, contribution, or delivery behavior. - process loss before durable admission requires reupload; an ambiguous durable put uses existing observation/recovery and never creates a Submission until verified content is bindable; +- a verified admission may remain unbound through client abandonment; its + closed lifecycle is `ready -> consumed|stale`, every state stays charged to + existing completed-byte scopes, and v0.1 adds no expiry/release/delete process; +- fresh AUTH-owned prepared capabilities guard durable put intent and later + Submission/binding consumption in their respective transactions; ART never + reads AUTH persistence directly and old human authority never implies fixed + service binding authority; +- pre-submit and post-submit materializers project identical fixed executable + semantics without preserving arbitrary archive modes or automatically + executing files; - current conservative configured limits remain unchanged; - the current `Submission` row remains the immutable version aggregate; a new version links through `supersedes_submission_id` and, when responding to diff --git a/docs/glossary.md b/docs/glossary.md index f98df4d7..d33dc5b5 100644 --- a/docs/glossary.md +++ b/docs/glossary.md @@ -82,6 +82,15 @@ Workstream's provider-neutral immutable content identity: server-computed SHA-256, byte count, and bounded media metadata. Opaque provider identifiers and protocol observations are replica details, not this record's identity. +## SubmissionBundleAdmission + +The capacity-charged verified result of one passing submission-bundle +preparation. Its closed lifecycle is `ready -> consumed|stale`. A ready admission +may remain unbound through client abandonment; consumption is atomic with one +immutable Submission and binding, while proven task/predecessor/locked-context +drift may make it stale. No state expires, releases storage capacity, or +authorizes deletion in v0.1. + ## ArtifactUploadItem Legacy unavailable multi-item staging metadata. ART-04A must remove it or make @@ -115,7 +124,8 @@ no response digest or provider receipt. The server-generated canonical semantic description of the file/directory tree inside one contributor outer ZIP. It commits to normalized paths, entry types, -and each file's SHA-256 and byte count while excluding ZIP packaging metadata. +each file's SHA-256/byte count, and normalized regular-file executable intent +while excluding other ZIP packaging and permission metadata. It is distinct from the exact outer-ZIP SHA-256/byte count. Together they bind pre-submit evidence, verified admission, the immutable Submission, and its exact artifact binding. diff --git a/docs/spec_artifact_storage_service.md b/docs/spec_artifact_storage_service.md index b052d33c..824b7b8a 100644 --- a/docs/spec_artifact_storage_service.md +++ b/docs/spec_artifact_storage_service.md @@ -1028,10 +1028,25 @@ signed URLs, or raw provider responses. Pagination is bounded and stable. One contributor operation receives one outer ZIP into bounded private scratch. Workstream computes its exact SHA-256/byte count, walks the complete safe outer archive tree, and generates a deterministic `SubmissionBundleManifest` from -normalized file/directory paths, entry type, and file SHA-256/byte count. Nested +normalized file/directory paths, entry type, file SHA-256/byte count, and the +normalized executable flag for each regular file. Nested archives remain opaque in v0.1. Exact archive or semantic-manifest equality with the immediate prior immutable Submission rejects before checker/provider I/O. +For a regular Unix-created ZIP entry with valid mode metadata, `executable` is +true when any execute bit is present. It defaults false for non-Unix or invalid +mode metadata and is null for directories. Ownership, group, read/write bits, +setuid/setgid/sticky bits, timestamps, compression, and comments are excluded. +Symlinks/special entries remain rejected. Materialization never restores raw +archive modes: fixed startup-validated modes produce sealed read-only files, +read-and-execute files, and read-and-traverse directories. The flag never +authorizes execution by itself. Pre-submit and post-submit materialization use +the same projection and `ArtifactScratchManager` retains secure cleanup custody. +The exact numeric modes are one startup configuration shared by every process +using the scratch root. Cleanup under an exclusive generation/lease may restore +only the manager-owned directory write bit after no-follow ownership/type +validation, then remove the tree; archive-supplied modes are never applied. + Mandatory platform and locked Project Guide checks consume that same read-only scratch tree. Infrastructure exhaustion returns: @@ -1047,12 +1062,89 @@ process loss requires reupload without manager/operator approval. Only a passing result is handed immediately to generic durable admission in the same process. Workstream writes the outer ZIP once, independently reads it back, -and publishes a bindable admission only after exact verification. Existing put +and publishes a `ready` bindable admission only after exact verification. Existing put attempt, observation, receipt, verification, scanner, and recovery records own durable ambiguity. Submission creation locks and consumes the exact admission; post-submit materialization resolves its immutable binding and recomputes integrity before checker execution. +### SubmissionBundleAdmission + +One immutable verified preparation with closed status: + +```text +status = ready | consumed | stale +actor_profile_id = UUID +preparation_identity_link_id = UUID +project_id = UUID +task_id = UUID +assignment_id = UUID +predecessor_submission_id = UUID | null +locked_task_context_id/hash = immutable canonical context reference +artifact_content_id = UUID +submission_bundle_manifest_id = UUID +pre_submit_evidence_set_id = UUID +ready_at = timestamp +consumed_at = timestamp | null +consumed_by_submission_id = UUID | null +stale_at = timestamp | null +stale_reason = bounded token | null +``` + +It records preparation actor/profile and identity-link provenance, project, +task, assignment, immediate predecessor, exact locked task/guide/policy context, +`ArtifactContent`, `SubmissionBundleManifest`, immutable pre-submit evidence-set +identity, timestamps, terminal reason, and optional consuming Submission. +Status-field constraints require terminal facts only in their matching state. +A unique Submission admission reference plus locked/CAS transition permits at +most one consumer. + +`ready -> consumed` occurs only in the transaction creating Submission and +binding. `ready -> stale` occurs only when consumption proves task closure, +predecessor advancement, or locked-context replacement. Authority loss alone +does not stale the admission. Both terminal states are irreversible. Client +abandonment may leave `ready` unbound indefinitely; it has no Submission, +review, contribution, compensation, or reputation effect. + +All states count against existing completed-byte task/actor/project/deployment +capacity. Existing Operator admission-usage projections expose bounded unbound +ready and stale counts/bytes. No admission expiry, release, deletion, provider +cleanup, retention process, or new recovery aggregate exists. Exact preparation +replay returns the original admission without another provider write, charge, +evidence set, or admission. + +### Authorization At Durable Boundaries + +The preparation route authorizes `artifact.submission_bundle.prepare` before +accepting bytes. Immediately before capacity reservation and put-attempt +creation, the owning transaction consumes AUTH's transaction-local prepared +capability. AUTH and typed TASK/PROJECT capabilities reload/lock their own actor, +identity-link, project authority, assignment, task, predecessor, locked context, +action availability, and canonical resource facts. ART does not access +AUTH-owned persistence. Authorization evidence, capacity, and durable put intent +commit atomically; provider I/O starts only afterward. + +Revocation after durable put intent does not cancel fixed-service verification +or recovery. It does prevent later binding until Submission creation receives a +new human decision. That transaction consumes both fresh human +`submission.create` and fixed-service `artifact.binding.create` capabilities, +locks and matches admission/task/assignment/predecessor/context, creates the +immutable Submission and binding, and marks admission consumed. Any denial, +cancellation, stale execution, or persistence failure rolls back all effects. +After authority succeeds, proven task closure, predecessor advancement, or +locked-context replacement may commit only `ready -> stale` plus bounded +evidence, never a Submission/binding. Authorization/concealment precedes +detailed admission status errors. + +Stable outcomes for an already authorized matching resource include: + +```text +409 submission_bundle_admission_already_consumed +409 submission_bundle_admission_context_changed +409 submission_bundle_admission_stale +403 +``` + After cutover, the public request is: ```json @@ -1078,7 +1170,8 @@ handoff as one process-local orchestration. It returns only a Workstream operation/admission identity and bounded redacted status. An exact idempotent POST retry may observe the same durable operation after durable intent; v0.1 exposes no separate preparation-status GET route. No retry resolves a scratch -path, and APIs never return provider internals. +path, every retry obtains fresh preparation authority, and APIs never return +provider internals. ## Guide And Checker Binding diff --git a/docs/template_submission_artifact_policy.md b/docs/template_submission_artifact_policy.md index 020defa3..85d50485 100644 --- a/docs/template_submission_artifact_policy.md +++ b/docs/template_submission_artifact_policy.md @@ -99,12 +99,18 @@ Default artifact rules: - artifact paths must not contain empty, `.`, or `..` segments - uploaded artifacts and storage-backed evidence require `sha256:<64 lowercase hex>` hashes in production - test fixtures may use deterministic placeholder hash tokens only in explicit local test paths +- Workstream normalizes regular-file executable intent from valid Unix ZIP mode + metadata; non-Unix/invalid metadata defaults to non-executable +- executable intent participates in semantic identity, but never by itself + authorizes a checker or service to execute the file Default storage rules: - clients submit exactly one outer ZIP through Workstream submission-bundle preparation and receive only Workstream operation/admission IDs - persisted product references are immutable Workstream artifact bindings +- verified admissions may remain unbound and capacity-charged in `ready` or + terminal `stale`; clients cannot expire, release, or delete them - signed URLs, raw local filesystem paths, provider references, credentials, query strings, bucket secrets, and token-bearing references are rejected before persistence diff --git a/docs/template_submission_packet.md b/docs/template_submission_packet.md index ac22406d..e28d729f 100644 --- a/docs/template_submission_packet.md +++ b/docs/template_submission_packet.md @@ -27,9 +27,16 @@ artifact identity; the contributor does not submit links or storage references. - generated at: Workstream records the server-computed outer-ZIP SHA-256/byte count, canonical -semantic manifest, verified admission, and immutable binding. Those values are +semantic manifest including normalized regular-file executable intent, verified +admission, and immutable binding. Those values are server output, not contributor input. +A successful preparation may return a verified `ready` admission before a +Submission exists. Submission creation obtains fresh authority and atomically +consumes that admission with the immutable Submission/binding. An abandoned +ready admission has no review, contribution, compensation, or reputation +effect, remains capacity-charged, and has no client expiry/release/delete path. + Workstream derives the locked project guide version, locked guide-source snapshot id/hash, effective project submission artifact policy id/hash, generated project pre-submit checker policy id/bundle hash, post-submit diff --git a/scripts/test_agent_gates.py b/scripts/test_agent_gates.py index b432c7bd..40498ad9 100644 --- a/scripts/test_agent_gates.py +++ b/scripts/test_agent_gates.py @@ -5809,6 +5809,34 @@ def test_parallel_initiative_status_matches_trusted_main() -> None: assert "`WS-ART-001-03A` is the only immediate ART" in artifact_status +def test_artifact_plan2_closes_submission_bundle_lifecycle_gaps() -> None: + """PLAN2 locks admission, executable, and durable-auth semantics.""" + initiative = ROOT / ".agent-loop/initiatives/WS-ART-001-immutable-artifact-storage" + plan = (initiative / "PLAN.md").read_text(encoding="utf-8") + decisions = (initiative / "DECISIONS.md").read_text(encoding="utf-8") + auth_handoff = (initiative / "AUTH_HANDOFF.md").read_text(encoding="utf-8") + contract_04a = (initiative / "chunks/WS-ART-001-04A-upload-inspection-sealing.md").read_text(encoding="utf-8") + contract_04c = (initiative / "chunks/WS-ART-001-04C-verified-submission-bundle-admission.md").read_text(encoding="utf-8") + contract_05 = (initiative / "chunks/WS-ART-001-05-submission-artifact-cutover.md").read_text(encoding="utf-8") + contract_06a = (initiative / "chunks/WS-ART-001-06A-checker-input-materialization.md").read_text(encoding="utf-8") + storage_spec = (ROOT / "docs/spec_artifact_storage_service.md").read_text(encoding="utf-8") + + for text in (plan, decisions, contract_04c, storage_spec): + assert "ready -> consumed" in text + assert "stale" in text + assert "expiry" in text.lower() + assert "Client abandonment" in plan + assert "existing completed-byte scopes" in plan + assert "transaction-local prepared capability" in auth_handoff + assert "never imports AUTH-owned repositories" in contract_05 + assert "authorization evidence" in contract_04c + assert "fixed-service `artifact.binding.create`" in auth_handoff + assert "normalized executable flag" in contract_04a + assert "same fixed" in contract_06a + assert "non-Unix" in storage_spec + assert "submission_bundle_admission_already_consumed" in storage_spec + + def test_stale_authorization_discovery_includes_new_untracked_docs() -> None: """A new active doc fails without being added to a hardcoded corpus.""" gate = load_module( From 07cea2252500f0a64f4e30ce6f3503d8b0b6492c Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Sat, 25 Jul 2026 10:02:17 +0100 Subject: [PATCH 15/26] docs(artifacts): complete executable manifest template --- docs/template_submission_packet.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/template_submission_packet.md b/docs/template_submission_packet.md index e28d729f..95a83def 100644 --- a/docs/template_submission_packet.md +++ b/docs/template_submission_packet.md @@ -60,7 +60,10 @@ and do not return review decision values: `accept`, `needs_revision`, or ## Submission Bundle Manifest Workstream generates this manifest from normalized directory/file paths, entry -type, and each file's SHA-256/byte count. Nested archives remain opaque in v0.1. +type, each file's SHA-256/byte count, and normalized executable intent for +regular files. Valid Unix execute bits normalize to true; non-Unix or invalid +mode metadata defaults false, and directories have no executable value. Other +archive permission metadata is excluded. Nested archives remain opaque in v0.1. ## Evidence From cbeaffbf0948f52f3e5fe334333a1741f48426cb Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Sat, 25 Jul 2026 10:22:15 +0100 Subject: [PATCH 16/26] docs(artifacts): enforce canonical admission gates --- .../AUTH_HANDOFF.md | 12 ++++--- .../PLAN.md | 3 +- .../WS-ART-001-04B-pre-submit-admission.md | 3 +- ...-ART-001-05-submission-artifact-cutover.md | 15 ++++---- docs/spec_artifact_storage_service.md | 6 ++-- scripts/test_agent_gates.py | 34 +++++++++++++++++-- 6 files changed, 57 insertions(+), 16 deletions(-) diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/AUTH_HANDOFF.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/AUTH_HANDOFF.md index e6361990..f7b48574 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/AUTH_HANDOFF.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/AUTH_HANDOFF.md @@ -59,9 +59,10 @@ intent commit atomically before provider I/O. ART-05 requires a new human authorization decision for `submission.create` and a separately prepared fixed-service capability for +ActionId `artifact.submission.binding.create`, mapped to PermissionId `artifact.binding.create`. Both are consumed in the one transaction that locks -the ready admission and TASK-owned context, creates Submission and binding, and -marks the admission consumed. Human authority implies no service authority. +the ready admission and TASK-owned context, creates Submission and binding, +and marks the admission consumed. Human authority implies no service authority. Revocation after durable put intent does not cancel verification/recovery, but the resulting admission remains unbound until a fresh 05 decision succeeds. Authorization denial precedes admission-detail errors so unrelated actors @@ -69,11 +70,14 @@ cannot distinguish missing, ready, stale, or consumed admissions. The continuous contributor action never implies the fixed service actions: -- `artifact.checker_input.materialize`; +- `artifact.pre_submit.checker_input.materialize` and + `artifact.post_submit.checker_input.materialize`, both mapped to PermissionId + `artifact.checker_input.materialize`; - `artifact.verification.execute`; - `artifact.pending_work.scan`; - `artifact.put_attempt.resolve`; -- `artifact.binding.create`. +- `artifact.submission.binding.create`, mapped to PermissionId + `artifact.binding.create`. Each fixed service action retains its canonical provisioned service identity, matrix row, resource facts, terminal reauthorization, and separate activation diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md index 02a45697..8dddb785 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/PLAN.md @@ -366,7 +366,8 @@ intent commit atomically; failure means no provider I/O and scratch cleanup. Durable put intent creates a technical recovery obligation that later human revocation does not cancel. Verification may finish, but binding remains impossible until 05 obtains fresh human `submission.create` authority and fresh -fixed-service `artifact.binding.create` authority. 05 consumes both prepared +fixed-service ActionId `artifact.submission.binding.create` authority, mapped +to PermissionId `artifact.binding.create`. 05 consumes both prepared capabilities in the single task transaction that locks the admission/context, creates Submission and binding, and marks the admission consumed. Denial, cancellation, stale execution, or persistence failure rolls back every protected diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04B-pre-submit-admission.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04B-pre-submit-admission.md index 3789f742..8adc3cdb 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04B-pre-submit-admission.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-04B-pre-submit-admission.md @@ -53,7 +53,8 @@ artifact content, provider object, or Submission. same fixed sealed modes as post-submit materialization; it never preserves arbitrary ZIP permission metadata or executes a file merely because the flag is true; -- checker workspace creation uses the distinct fixed service action +- checker workspace creation uses fixed-service ActionId + `artifact.pre_submit.checker_input.materialize`, mapped to PermissionId `artifact.checker_input.materialize`; contributor preparation authority never grants or substitutes for that service authority; - a checker failure creates structured findings only: no durable artifact, diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-05-submission-artifact-cutover.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-05-submission-artifact-cutover.md index 638b034f..4fcfe8a6 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-05-submission-artifact-cutover.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-05-submission-artifact-cutover.md @@ -40,9 +40,10 @@ package transport/hash/manifest authority. - before revealing admission state/context, the route obtains a fresh human authorization decision; AUTH denial remains concealed from unrelated actors; - one transaction consumes fresh prepared human `submission.create` and fixed - service `artifact.binding.create` capabilities, locks task, assignment, - admission and predecessor, and recomposes final resource facts without TASK - or ART importing AUTH-owned repositories; + service ActionId `artifact.submission.binding.create` capabilities (mapped to + PermissionId `artifact.binding.create`), locks task, assignment, admission and + predecessor, and recomposes final resource facts without TASK or ART importing + AUTH-owned repositories; - the transaction requires `ready`, matches actor profile, task, project, assignment, predecessor and exact locked context, allocates the next version, creates the immutable Submission and binding, and changes admission to @@ -55,9 +56,11 @@ package transport/hash/manifest authority. - proven task closure, predecessor advancement, or locked-context replacement changes a still-ready admission to `stale`; authority loss alone does not, so restored authority may later consume a still-compatible ready admission; -- binding creation uses fixed service action `artifact.binding.create`; the - contributor's active `artifact.submission_bundle.prepare` action and - `submission.create` permission do not imply internal binding authority; +- binding creation uses fixed-service ActionId + `artifact.submission.binding.create`, mapped to PermissionId + `artifact.binding.create`; the contributor's active + `artifact.submission_bundle.prepare` ActionId and `submission.create` + permission do not imply internal binding authority; - version `N+1` records `supersedes_submission_id = N`; the exact relationship to a `needs_revision` Review is added only through the reviewed REV/TASK joint contract and cannot be fabricated from a note or client ID; diff --git a/docs/spec_artifact_storage_service.md b/docs/spec_artifact_storage_service.md index 824b7b8a..8cb26a62 100644 --- a/docs/spec_artifact_storage_service.md +++ b/docs/spec_artifact_storage_service.md @@ -1127,8 +1127,10 @@ commit atomically; provider I/O starts only afterward. Revocation after durable put intent does not cancel fixed-service verification or recovery. It does prevent later binding until Submission creation receives a new human decision. That transaction consumes both fresh human -`submission.create` and fixed-service `artifact.binding.create` capabilities, -locks and matches admission/task/assignment/predecessor/context, creates the +`submission.create` and fixed-service ActionId +`artifact.submission.binding.create` capabilities, mapped to PermissionId +`artifact.binding.create`, locks and matches +admission/task/assignment/predecessor/context, creates the immutable Submission and binding, and marks admission consumed. Any denial, cancellation, stale execution, or persistence failure rolls back all effects. After authority succeeds, proven task closure, predecessor advancement, or diff --git a/scripts/test_agent_gates.py b/scripts/test_agent_gates.py index 40498ad9..7b37e7c1 100644 --- a/scripts/test_agent_gates.py +++ b/scripts/test_agent_gates.py @@ -5824,13 +5824,43 @@ def test_artifact_plan2_closes_submission_bundle_lifecycle_gaps() -> None: for text in (plan, decisions, contract_04c, storage_spec): assert "ready -> consumed" in text assert "stale" in text - assert "expiry" in text.lower() + no_lifecycle_process = { + "plan": "No state expires, releases capacity, or authorizes provider deletion.", + "decisions": ( + "No expiry, release, deletion, retention process, or cleanup lifecycle " + "exists in v0.1." + ), + "04c": ( + "v0.1 adds no expiry, release, deletion, retention process, or cleanup " + "route;" + ), + "storage_spec": ( + "No admission expiry, release, deletion, provider cleanup, retention " + "process, or new recovery aggregate exists." + ), + } + for name, text in { + "plan": plan, + "decisions": decisions, + "04c": contract_04c, + "storage_spec": storage_spec, + }.items(): + normalized = " ".join(text.split()) + clause = no_lifecycle_process[name] + assert clause in normalized + assert clause not in normalized.replace(clause, "Admission expiry is enabled.") assert "Client abandonment" in plan assert "existing completed-byte scopes" in plan assert "transaction-local prepared capability" in auth_handoff assert "never imports AUTH-owned repositories" in contract_05 assert "authorization evidence" in contract_04c - assert "fixed-service `artifact.binding.create`" in auth_handoff + assert "ActionId `artifact.submission.binding.create`" in auth_handoff + assert "PermissionId `artifact.binding.create`" in auth_handoff + assert "`artifact.pre_submit.checker_input.materialize`" in auth_handoff + assert "`artifact.post_submit.checker_input.materialize`" in auth_handoff + assert "mapped to PermissionId\n `artifact.checker_input.materialize`" in auth_handoff + assert "fixed service action\n `artifact.checker_input.materialize`" not in contract_04a + assert "fixed service action `artifact.binding.create`" not in contract_05 assert "normalized executable flag" in contract_04a assert "same fixed" in contract_06a assert "non-Unix" in storage_spec From dea69550656748f95d8bce8dd297255fc9f8393b Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Sat, 25 Jul 2026 10:34:01 +0100 Subject: [PATCH 17/26] test(agent-gates): guard canonical materializer action --- scripts/test_agent_gates.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/scripts/test_agent_gates.py b/scripts/test_agent_gates.py index 7b37e7c1..acd11937 100644 --- a/scripts/test_agent_gates.py +++ b/scripts/test_agent_gates.py @@ -5816,6 +5816,7 @@ def test_artifact_plan2_closes_submission_bundle_lifecycle_gaps() -> None: decisions = (initiative / "DECISIONS.md").read_text(encoding="utf-8") auth_handoff = (initiative / "AUTH_HANDOFF.md").read_text(encoding="utf-8") contract_04a = (initiative / "chunks/WS-ART-001-04A-upload-inspection-sealing.md").read_text(encoding="utf-8") + contract_04b = (initiative / "chunks/WS-ART-001-04B-pre-submit-admission.md").read_text(encoding="utf-8") contract_04c = (initiative / "chunks/WS-ART-001-04C-verified-submission-bundle-admission.md").read_text(encoding="utf-8") contract_05 = (initiative / "chunks/WS-ART-001-05-submission-artifact-cutover.md").read_text(encoding="utf-8") contract_06a = (initiative / "chunks/WS-ART-001-06A-checker-input-materialization.md").read_text(encoding="utf-8") @@ -5859,7 +5860,7 @@ def test_artifact_plan2_closes_submission_bundle_lifecycle_gaps() -> None: assert "`artifact.pre_submit.checker_input.materialize`" in auth_handoff assert "`artifact.post_submit.checker_input.materialize`" in auth_handoff assert "mapped to PermissionId\n `artifact.checker_input.materialize`" in auth_handoff - assert "fixed service action\n `artifact.checker_input.materialize`" not in contract_04a + assert "fixed service action\n `artifact.checker_input.materialize`" not in contract_04b assert "fixed service action `artifact.binding.create`" not in contract_05 assert "normalized executable flag" in contract_04a assert "same fixed" in contract_06a From 822b43d9de3b6a8354532dcbd7a3ac52d17da4f8 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Sat, 25 Jul 2026 11:07:38 +0100 Subject: [PATCH 18/26] docs(agent-loop): record PLAN2 amendment review --- .agent-loop/REVIEW_LOG.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/.agent-loop/REVIEW_LOG.md b/.agent-loop/REVIEW_LOG.md index 2159c384..421df411 100644 --- a/.agent-loop/REVIEW_LOG.md +++ b/.agent-loop/REVIEW_LOG.md @@ -2909,3 +2909,16 @@ put-attempt state vocabulary, and server-owned manifest input. Hosted CI also found noncanonical chunk headings and a successor-title mismatch. The repair resolves all findings without runtime behavior or AUTH activation; refreshed internal review, hosted checks, and human approval remain required. + +## 2026-07-25 - WS-ART-001-PLAN2 Admission And Authorization Amendment + +The final planning candidate now closes verified-but-unbound admission +lifecycle, canonical executable intent, and fresh authorization at durable put +intent and Submission consumption. Review repair replaced shared PermissionIds +misnamed as service actions with the canonical phase-specific AUTH ActionIds +and strengthened the governance test to require exact no-expiry, no-release, +no-deletion, no-cleanup, and no-retention-process clauses. Exact material SHA +`91c2c95900a59cf1e5aa4c0524b48e0e218aeed9` is rebased on trusted main +`f3ece23e0f128258947137764b39b7d59dd7b2a8`; all nine internal reviewer tracks +pass with no open finding. Hosted external checks and human merge approval +remain required. From cd1dd1204a293daea609317247dfeb2635033721 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Sat, 25 Jul 2026 11:12:25 +0100 Subject: [PATCH 19/26] docs(agent-loop): bind PLAN2 amendment evidence --- ...-ART-001-PLAN2-external-review-response.md | 13 +++++ ...-ART-001-PLAN2-internal-review-evidence.md | 48 +++++++++++++------ .../WS-ART-001-PLAN2-pr-trust-bundle.md | 9 ++++ 3 files changed, 56 insertions(+), 14 deletions(-) diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-external-review-response.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-external-review-response.md index afed980d..1d5852b2 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-external-review-response.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-external-review-response.md @@ -18,6 +18,18 @@ - canonicalized every changed chunk heading and aligned the PLAN2 successor title so repository governance gates can parse the contracts; - expanded the PR description to the canonical trust-bundle structure. +- defined abandoned verified admissions as capacity-charged `ready` records + with only terminal `consumed` or `stale` outcomes and no expiry, release, + deletion, cleanup, or retention process; +- made normalized regular-file executable intent part of semantic identity and + fixed read-only materialization without preserving arbitrary ZIP permissions; +- required fresh AUTH-owned prepared capabilities at durable put intent and at + atomic Submission/admission consumption; +- used canonical phase-specific fixed-service ActionIds and kept + `artifact.binding.create` and `artifact.checker_input.materialize` as their + shared PermissionIds; +- strengthened agent-gate assertions so lifecycle and ActionId/PermissionId + regressions fail closed. ## Comments deferred @@ -31,6 +43,7 @@ None beyond the normal human review and explicit merge approval for PR #197. - `git diff --check` - `python3 scripts/check_stale_artifact_contracts.py` +- `python3 scripts/check_stale_authorization_docs.py` - `python3 scripts/check_markdown_links.py` - `python3 scripts/test_agent_gates.py` — 100 tests - `python3 scripts/check_internal_review_evidence.py` diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-internal-review-evidence.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-internal-review-evidence.md index 6196dc2d..e5dd7859 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-internal-review-evidence.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-internal-review-evidence.md @@ -1,15 +1,15 @@ # Internal Review Evidence: WS-ART-001-PLAN2 -Reviewed code SHA: `afb883dd4e8ab52fad3d7301b5e865d3a25885a7` +Reviewed code SHA: `25bae9792208697366bd828230d8f321d819586d` -Reviewed against trusted main: `bcf1292e1a591e3e84bf8ee212ee7191d80741fa` +Reviewed against trusted main: `f3ece23e0f128258947137764b39b7d59dd7b2a8` -Reviewed at: `2026-07-24T17:35:13Z` +Reviewed at: `2026-07-25T09:10:36Z` -Reviewer run IDs: `art_plan2_ext_senior`, `art_plan2_ext_arch`, -`art_plan2_ext_qa`, `art_plan2_ext_security`, `art_plan2_ext_product`, -`art_plan2_ext_docs`, `art_plan2_ext_reuse`, `art_plan2_ext_ci`, -`art_plan2_ext_test_delta` +Reviewer run IDs: `art_plan2_ext_senior`, `art_plan2_amend_arch`, +`art_plan2_ext_qa`, `art_plan2_amend_security`, `art_plan2_ext_product`, +`art_plan2_amend_docs`, `art_plan2_final_reuse`, `art_plan2_final_ci`, +`art_plan2_final_test_delta` ## Candidate @@ -22,7 +22,10 @@ implemented or activated. - `git diff --check` — PASS - `python3 scripts/check_stale_artifact_contracts.py` — PASS at `artifact_store_cutover` +- `python3 scripts/check_stale_authorization_docs.py` — PASS - `python3 scripts/check_markdown_links.py` — PASS +- `python3 scripts/update_post_merge_memory.py validate-merge-intent + --base-ref origin/main` — PASS - `python3 scripts/test_agent_gates.py` — PASS, 100 tests - targeted stale wording scan — only explicit rejected/legacy/unavailable references remain @@ -31,15 +34,15 @@ implemented or activated. | Reviewer | Result | Blocking findings | Notes | |---|---|---|---| -| senior engineering | PASS AFTER FIXES | none | Reconciled status, state vocabulary, outage behavior, route ownership, and evidence accuracy. | -| architecture | PASS AFTER FIXES | none | Removed duplicate session/candidate paths, reused existing scratch/admission/recovery, and separated generic attempt from legacy item state. | -| QA/test | PASS AFTER FIXES | none | Proved exact verification commands, 78/90 percent gates, nested-archive closure, process-loss behavior, and governance parsing. | -| security/auth | PASS AFTER FIXES | none | Kept 04A non-routable, removed the unmapped GET, preserved AUTH activation custody, and closed canonical-path collisions. | -| product/ops | PASS AFTER FIXES | none | Corrected reviewer/revision semantics and separated pre-intent reupload from durable ART/checker recovery. | +| senior engineering | PASS AFTER FIXES | none | The only final finding was this expected evidence refresh; admission, executable, and reauthorization contracts are maintainable and operationally bounded. | +| architecture | PASS | none | Preserved ART/AUTH/TASK/REV ownership, reused existing scratch/admission/recovery, and resolved latest-main review-history integration. | +| QA/test | PASS AFTER FIXES | none | Closed executable-template coverage and proved lifecycle, authorization crossed states, concurrency, and governance gates. | +| security/auth | PASS | none | Preserved prepared-capability ordering, fixed-service separation, concealment, and ready/consumed/stale integrity fencing. | +| product/ops | PASS | none | Preserved one-ZIP contributor flow, reviewer decision vocabulary, and zero downstream effect before immutable Submission consumption. | | reuse/dedup | PASS | none | Reused `ArtifactScratchManager`, `PreparedArtifact`, `CommittedArtifactSource`, and existing admission/put/verification/recovery paths. | | CI integrity | PASS AFTER FIXES | none | Canonical headings/merge intent parse; stale-auth scanner remains fail-closed; cumulative 90 percent and repository 78 percent gates remain enforced. | | test delta | PASS | none | No removed/skipped tests or weakened assertions; 100 agent-gate tests pass. | -| docs | PASS AFTER FIXES | none | Reconciled storage/auth specs and all live submission/manifest templates. | +| docs | PASS AFTER FIXES | none | Reconciled storage/auth specs and templates, including canonical executable-intent semantics in the submission packet. | ## Material Repairs @@ -54,7 +57,24 @@ implemented or activated. registration/activation handoff; - preserved existing immutable `Submission` as the version aggregate and REV's decision/note ownership; -- required integrity recomputation for checker, reviewer, and delivery streams. +- required integrity recomputation for checker, reviewer, and delivery streams; +- defined verified-but-unbound admissions as capacity-charged + `ready -> consumed|stale` records with no expiry, deletion, release, retention + worker, or downstream product effect before atomic consumption; +- included normalized regular-file executable intent in semantic identity while + excluding arbitrary permission preservation and execution authority; +- required fresh transaction-local AUTH capabilities immediately before durable + put intent and again during atomic Submission/admission consumption; +- corrected fixed-service contracts to use canonical phase-specific ActionIds, + with shared `artifact.binding.create` and + `artifact.checker_input.materialize` named only as PermissionIds; +- strengthened the governance regression to require exact no-expiry, no-release, + no-deletion, no-cleanup, and no-retention-process admission clauses; +- corrected the materializer regression assertion to inspect its owning 04B + contract rather than the adjacent 04A contract; +- rebased onto trusted main `f3ece23e` and preserved AUTH-10C, AUTH-002, and + PLAN2 review history. +- recorded the final amendment review outcome in the canonical review log. ## Accepted Risks diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-pr-trust-bundle.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-pr-trust-bundle.md index 6b6007a5..a754db43 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-pr-trust-bundle.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-pr-trust-bundle.md @@ -16,6 +16,8 @@ one outer ZIP -> mandatory platform and locked-guide checks -> one existing ArtifactStore admission -> complete read-back verification +-> one capacity-charged ready SubmissionBundleAdmission +-> fresh human and fixed-service authorization -> one immutable Submission binding ``` @@ -37,6 +39,7 @@ successor `WS-ART-001-03A` with a separate explicit start. - diff, stale-contract, markdown-link, and 100 agent-gate tests pass; - all nine required reviewer tracks pass after repair; +- exact reviewed candidate `25bae979` is based on trusted main `f3ece23e`; - the stale-authorization documentation gate and its technical-module regression repair pass without reopening deprecated product-role vocabulary; - detailed findings and resolutions are recorded in @@ -50,6 +53,12 @@ successor `WS-ART-001-03A` with a separate explicit start. exact bundle action before ART-04A/05 can proceed. - Confirm ART owns bytes/identity/access capabilities while REV, CON, and delivery retain their lifecycle decisions. +- Confirm abandoned ready admissions remain quota-bounded and capacity-charged + in v0.1, with no expiry, deletion, release, or retention worker. +- Confirm executable intent is semantic manifest data, not permission + preservation or authority to execute contributor content. +- Confirm authority is freshly revalidated at durable put intent and again at + atomic Submission consumption through AUTH-owned prepared capabilities. ## Gate From cd19af55a87a08871eb78afdc61910c4e840b53d Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Sat, 25 Jul 2026 11:20:50 +0100 Subject: [PATCH 20/26] test(agent-gates): prove typed AUTH boundary --- ...S-ART-001-05-submission-artifact-cutover.md | 4 ++-- scripts/test_agent_gates.py | 18 +++++++++++------- 2 files changed, 13 insertions(+), 9 deletions(-) diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-05-submission-artifact-cutover.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-05-submission-artifact-cutover.md index 4fcfe8a6..da548d96 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-05-submission-artifact-cutover.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/WS-ART-001-05-submission-artifact-cutover.md @@ -42,8 +42,8 @@ package transport/hash/manifest authority. - one transaction consumes fresh prepared human `submission.create` and fixed service ActionId `artifact.submission.binding.create` capabilities (mapped to PermissionId `artifact.binding.create`), locks task, assignment, admission and - predecessor, and recomposes final resource facts without TASK or ART importing - AUTH-owned repositories; + predecessor, and recomposes final resource facts through typed capabilities; + ART and TASK never import AUTH-owned repositories; - the transaction requires `ready`, matches actor profile, task, project, assignment, predecessor and exact locked context, allocates the next version, creates the immutable Submission and binding, and changes admission to diff --git a/scripts/test_agent_gates.py b/scripts/test_agent_gates.py index acd11937..7ccb2700 100644 --- a/scripts/test_agent_gates.py +++ b/scripts/test_agent_gates.py @@ -5821,6 +5821,7 @@ def test_artifact_plan2_closes_submission_bundle_lifecycle_gaps() -> None: contract_05 = (initiative / "chunks/WS-ART-001-05-submission-artifact-cutover.md").read_text(encoding="utf-8") contract_06a = (initiative / "chunks/WS-ART-001-06A-checker-input-materialization.md").read_text(encoding="utf-8") storage_spec = (ROOT / "docs/spec_artifact_storage_service.md").read_text(encoding="utf-8") + normalized_auth_handoff = " ".join(auth_handoff.split()) for text in (plan, decisions, contract_04c, storage_spec): assert "ready -> consumed" in text @@ -5852,14 +5853,17 @@ def test_artifact_plan2_closes_submission_bundle_lifecycle_gaps() -> None: assert clause not in normalized.replace(clause, "Admission expiry is enabled.") assert "Client abandonment" in plan assert "existing completed-byte scopes" in plan - assert "transaction-local prepared capability" in auth_handoff - assert "never imports AUTH-owned repositories" in contract_05 + assert "transaction-local prepared capability" in normalized_auth_handoff + assert "ART and TASK never import AUTH-owned repositories" in contract_05 assert "authorization evidence" in contract_04c - assert "ActionId `artifact.submission.binding.create`" in auth_handoff - assert "PermissionId `artifact.binding.create`" in auth_handoff - assert "`artifact.pre_submit.checker_input.materialize`" in auth_handoff - assert "`artifact.post_submit.checker_input.materialize`" in auth_handoff - assert "mapped to PermissionId\n `artifact.checker_input.materialize`" in auth_handoff + assert "ActionId `artifact.submission.binding.create`" in normalized_auth_handoff + assert "PermissionId `artifact.binding.create`" in normalized_auth_handoff + assert "`artifact.pre_submit.checker_input.materialize`" in normalized_auth_handoff + assert "`artifact.post_submit.checker_input.materialize`" in normalized_auth_handoff + assert ( + "mapped to PermissionId `artifact.checker_input.materialize`" + in normalized_auth_handoff + ) assert "fixed service action\n `artifact.checker_input.materialize`" not in contract_04b assert "fixed service action `artifact.binding.create`" not in contract_05 assert "normalized executable flag" in contract_04a From b839d2572d64657c5759ecbf234a20a650f46bf5 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Sat, 25 Jul 2026 11:24:03 +0100 Subject: [PATCH 21/26] docs(agent-loop): record hosted PLAN2 gate repair --- .agent-loop/REVIEW_LOG.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/.agent-loop/REVIEW_LOG.md b/.agent-loop/REVIEW_LOG.md index 421df411..692007b6 100644 --- a/.agent-loop/REVIEW_LOG.md +++ b/.agent-loop/REVIEW_LOG.md @@ -2918,7 +2918,14 @@ intent and Submission consumption. Review repair replaced shared PermissionIds misnamed as service actions with the canonical phase-specific AUTH ActionIds and strengthened the governance test to require exact no-expiry, no-release, no-deletion, no-cleanup, and no-retention-process clauses. Exact material SHA -`91c2c95900a59cf1e5aa4c0524b48e0e218aeed9` is rebased on trusted main +`8c23ab7edec9b3caa73ab2d6c40a5cac58cfe1be` is rebased on trusted main `f3ece23e0f128258947137764b39b7d59dd7b2a8`; all nine internal reviewer tracks pass with no open finding. Hosted external checks and human merge approval remain required. + +GitHub Agent Gates then exercised the full pytest collection that the direct +script entry point does not run and exposed two line-wrap-sensitive assertions. +The repair makes the 05 typed-capability/no-AUTH-repository boundary explicit +and normalizes handoff whitespace before semantic ActionId/PermissionId checks. +The exact hosted-equivalent command passes 300 tests with 90.46 percent branch +coverage for the loop-memory updater. From 7c88bbfb13bc090adde7caeef4845d56634b15c7 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Sat, 25 Jul 2026 11:27:56 +0100 Subject: [PATCH 22/26] docs(agent-loop): bind hosted PLAN2 repair evidence --- .../WS-ART-001-PLAN2-external-review-response.md | 5 +++++ .../WS-ART-001-PLAN2-internal-review-evidence.md | 15 ++++++++++----- .../reviews/WS-ART-001-PLAN2-pr-trust-bundle.md | 2 +- 3 files changed, 16 insertions(+), 6 deletions(-) diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-external-review-response.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-external-review-response.md index 1d5852b2..e2da03cd 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-external-review-response.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-external-review-response.md @@ -30,6 +30,9 @@ shared PermissionIds; - strengthened agent-gate assertions so lifecycle and ActionId/PermissionId regressions fail closed. +- repaired the hosted pytest-only failure by asserting the explicit typed + capability/no-AUTH-repository boundary and normalizing Markdown whitespace + before semantic ActionId/PermissionId checks. ## Comments deferred @@ -46,6 +49,8 @@ None beyond the normal human review and explicit merge approval for PR #197. - `python3 scripts/check_stale_authorization_docs.py` - `python3 scripts/check_markdown_links.py` - `python3 scripts/test_agent_gates.py` — 100 tests +- hosted-equivalent loop-memory branch-coverage command — 300 tests, 90.46 + percent - `python3 scripts/check_internal_review_evidence.py` - `python3 scripts/update_post_merge_memory.py validate-merge-intent --base-ref origin/main` diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-internal-review-evidence.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-internal-review-evidence.md index e5dd7859..91479cd5 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-internal-review-evidence.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-internal-review-evidence.md @@ -1,10 +1,10 @@ # Internal Review Evidence: WS-ART-001-PLAN2 -Reviewed code SHA: `25bae9792208697366bd828230d8f321d819586d` +Reviewed code SHA: `648dd31117ab5bbd4a6c25d7067f89e4edb61d85` Reviewed against trusted main: `f3ece23e0f128258947137764b39b7d59dd7b2a8` -Reviewed at: `2026-07-25T09:10:36Z` +Reviewed at: `2026-07-25T10:27:17Z` Reviewer run IDs: `art_plan2_ext_senior`, `art_plan2_amend_arch`, `art_plan2_ext_qa`, `art_plan2_amend_security`, `art_plan2_ext_product`, @@ -27,6 +27,8 @@ implemented or activated. - `python3 scripts/update_post_merge_memory.py validate-merge-intent --base-ref origin/main` — PASS - `python3 scripts/test_agent_gates.py` — PASS, 100 tests +- hosted-equivalent loop-memory coverage command — PASS, 300 tests and 90.46 + percent branch coverage - targeted stale wording scan — only explicit rejected/legacy/unavailable references remain @@ -36,12 +38,12 @@ implemented or activated. |---|---|---|---| | senior engineering | PASS AFTER FIXES | none | The only final finding was this expected evidence refresh; admission, executable, and reauthorization contracts are maintainable and operationally bounded. | | architecture | PASS | none | Preserved ART/AUTH/TASK/REV ownership, reused existing scratch/admission/recovery, and resolved latest-main review-history integration. | -| QA/test | PASS AFTER FIXES | none | Closed executable-template coverage and proved lifecycle, authorization crossed states, concurrency, and governance gates. | +| QA/test | PASS AFTER FIXES | none | Closed executable-template and hosted-collection gaps; proved lifecycle, authorization crossed states, concurrency, and governance gates. | | security/auth | PASS | none | Preserved prepared-capability ordering, fixed-service separation, concealment, and ready/consumed/stale integrity fencing. | | product/ops | PASS | none | Preserved one-ZIP contributor flow, reviewer decision vocabulary, and zero downstream effect before immutable Submission consumption. | | reuse/dedup | PASS | none | Reused `ArtifactScratchManager`, `PreparedArtifact`, `CommittedArtifactSource`, and existing admission/put/verification/recovery paths. | -| CI integrity | PASS AFTER FIXES | none | Canonical headings/merge intent parse; stale-auth scanner remains fail-closed; cumulative 90 percent and repository 78 percent gates remain enforced. | -| test delta | PASS | none | No removed/skipped tests or weakened assertions; 100 agent-gate tests pass. | +| CI integrity | PASS AFTER FIXES | none | Canonical headings/merge intent parse; stale-auth scanner remains fail-closed; hosted 300-test collection and coverage floor pass. | +| test delta | PASS AFTER FIXES | none | No removed/skipped tests or weakened assertions; semantic guards are whitespace-robust and inspect the owning contracts. | | docs | PASS AFTER FIXES | none | Reconciled storage/auth specs and templates, including canonical executable-intent semantics in the submission packet. | ## Material Repairs @@ -72,6 +74,9 @@ implemented or activated. no-deletion, no-cleanup, and no-retention-process admission clauses; - corrected the materializer regression assertion to inspect its owning 04B contract rather than the adjacent 04A contract; +- made 05's typed-capability boundary and prohibition on ART/TASK imports of + AUTH-owned repositories explicit, then proved the exact hosted pytest + collection rather than relying only on the direct script entry point; - rebased onto trusted main `f3ece23e` and preserved AUTH-10C, AUTH-002, and PLAN2 review history. - recorded the final amendment review outcome in the canonical review log. diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-pr-trust-bundle.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-pr-trust-bundle.md index a754db43..6f84751b 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-pr-trust-bundle.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-pr-trust-bundle.md @@ -39,7 +39,7 @@ successor `WS-ART-001-03A` with a separate explicit start. - diff, stale-contract, markdown-link, and 100 agent-gate tests pass; - all nine required reviewer tracks pass after repair; -- exact reviewed candidate `25bae979` is based on trusted main `f3ece23e`; +- exact reviewed candidate `648dd311` is based on trusted main `f3ece23e`; - the stale-authorization documentation gate and its technical-module regression repair pass without reopening deprecated product-role vocabulary; - detailed findings and resolutions are recorded in From 1169992db3d8b59b1c4b29e6ee726b91f6288e61 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Sat, 25 Jul 2026 17:31:06 +0100 Subject: [PATCH 23/26] docs(agent-loop): record latest-main PLAN2 integration --- .agent-loop/REVIEW_LOG.md | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/.agent-loop/REVIEW_LOG.md b/.agent-loop/REVIEW_LOG.md index 692007b6..6988e7d8 100644 --- a/.agent-loop/REVIEW_LOG.md +++ b/.agent-loop/REVIEW_LOG.md @@ -2918,8 +2918,8 @@ intent and Submission consumption. Review repair replaced shared PermissionIds misnamed as service actions with the canonical phase-specific AUTH ActionIds and strengthened the governance test to require exact no-expiry, no-release, no-deletion, no-cleanup, and no-retention-process clauses. Exact material SHA -`8c23ab7edec9b3caa73ab2d6c40a5cac58cfe1be` is rebased on trusted main -`f3ece23e0f128258947137764b39b7d59dd7b2a8`; all nine internal reviewer tracks +`b839d2572d64657c5759ecbf234a20a650f46bf5` is rebased on trusted main +`bba4ba5f171a4438b072740707a5cf8bde49d9af`; all nine internal reviewer tracks pass with no open finding. Hosted external checks and human merge approval remain required. @@ -2929,3 +2929,8 @@ The repair makes the 05 typed-capability/no-AUTH-repository boundary explicit and normalizes handoff whitespace before semantic ActionId/PermissionId checks. The exact hosted-equivalent command passes 300 tests with 90.46 percent branch coverage for the loop-memory updater. + +The later trusted-main reconciliation incorporates PR #198's exact CI custody +and Ruff/orchestration repairs without changing the 22-commit PLAN2 patch +series; range-diff equivalence and fresh exact-base integration review are the +required evidence before republishing PR #197. From 3185af57055f8e7b2411a2090671a9707a26dc58 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Sat, 25 Jul 2026 17:46:42 +0100 Subject: [PATCH 24/26] fix(agent-gates): remove obsolete ART contract read --- scripts/test_agent_gates.py | 4 ---- 1 file changed, 4 deletions(-) diff --git a/scripts/test_agent_gates.py b/scripts/test_agent_gates.py index 7ccb2700..e9a657df 100644 --- a/scripts/test_agent_gates.py +++ b/scripts/test_agent_gates.py @@ -5694,10 +5694,6 @@ def test_parallel_initiative_status_matches_trusted_main() -> None: artifact_status = Path( ".agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/STATUS.md" ).read_text(encoding="utf-8") - artifact_contract = Path( - ".agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/chunks/" - "WS-ART-001-02C2-verification-publication-fencing.md" - ).read_text(encoding="utf-8") work_queue = Path(".agent-loop/WORK_QUEUE.md").read_text(encoding="utf-8") loop_state = Path(".agent-loop/LOOP_STATE.md").read_text(encoding="utf-8") contribution_guide = Path("CONTRIBUTING.md").read_text(encoding="utf-8") From 1dbc0d4e52c262844106ef9434c69139e38b33e9 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Sat, 25 Jul 2026 17:47:14 +0100 Subject: [PATCH 25/26] docs(agent-loop): record Ruff integration repair --- .agent-loop/REVIEW_LOG.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/.agent-loop/REVIEW_LOG.md b/.agent-loop/REVIEW_LOG.md index 6988e7d8..69ebbb06 100644 --- a/.agent-loop/REVIEW_LOG.md +++ b/.agent-loop/REVIEW_LOG.md @@ -2918,7 +2918,7 @@ intent and Submission consumption. Review repair replaced shared PermissionIds misnamed as service actions with the canonical phase-specific AUTH ActionIds and strengthened the governance test to require exact no-expiry, no-release, no-deletion, no-cleanup, and no-retention-process clauses. Exact material SHA -`b839d2572d64657c5759ecbf234a20a650f46bf5` is rebased on trusted main +`3185af57055f8e7b2411a2090671a9707a26dc58` is rebased on trusted main `bba4ba5f171a4438b072740707a5cf8bde49d9af`; all nine internal reviewer tracks pass with no open finding. Hosted external checks and human merge approval remain required. @@ -2934,3 +2934,8 @@ The later trusted-main reconciliation incorporates PR #198's exact CI custody and Ruff/orchestration repairs without changing the 22-commit PLAN2 patch series; range-diff equivalence and fresh exact-base integration review are the required evidence before republishing PR #197. + +The exact Ruff 0.15.22 integration review found one branch-owned `F841`: a +historical ART contract read remained after its assertion was retired. The +repair removes only that dead read; focused Ruff passes, all 104 collected +agent-gate pytest cases pass, and the 100 direct regression cases pass. From be62b740e8476c11cca70b6dc66a8ad5dfd6cf92 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Sat, 25 Jul 2026 17:56:57 +0100 Subject: [PATCH 26/26] docs(agent-loop): bind latest-main PLAN2 evidence --- ...WS-ART-001-PLAN2-external-review-response.md | 6 ++++++ ...WS-ART-001-PLAN2-internal-review-evidence.md | 17 ++++++++++++----- .../reviews/WS-ART-001-PLAN2-pr-trust-bundle.md | 2 +- 3 files changed, 19 insertions(+), 6 deletions(-) diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-external-review-response.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-external-review-response.md index e2da03cd..0191c8b1 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-external-review-response.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-external-review-response.md @@ -33,6 +33,10 @@ - repaired the hosted pytest-only failure by asserting the explicit typed capability/no-AUTH-repository boundary and normalizing Markdown whitespace before semantic ActionId/PermissionId checks. +- rebased without conflict onto `bba4ba5f`, including PR #198's exact CI/Ruff + custody repairs; range-diff confirms the complete PLAN2 series is unchanged. +- removed one obsolete `artifact_contract` read exposed by Ruff `F841`; no live + assertion or coverage obligation was removed. ## Comments deferred @@ -51,6 +55,8 @@ None beyond the normal human review and explicit merge approval for PR #197. - `python3 scripts/test_agent_gates.py` — 100 tests - hosted-equivalent loop-memory branch-coverage command — 300 tests, 90.46 percent +- `ruff 0.15.22 check app tests scripts` — PASS on the reconciled tree +- focused agent-gate pytest collection — 104 PASS - `python3 scripts/check_internal_review_evidence.py` - `python3 scripts/update_post_merge_memory.py validate-merge-intent --base-ref origin/main` diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-internal-review-evidence.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-internal-review-evidence.md index 91479cd5..5a443f78 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-internal-review-evidence.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-internal-review-evidence.md @@ -1,10 +1,10 @@ # Internal Review Evidence: WS-ART-001-PLAN2 -Reviewed code SHA: `648dd31117ab5bbd4a6c25d7067f89e4edb61d85` +Reviewed code SHA: `1dbc0d4e52c262844106ef9434c69139e38b33e9` -Reviewed against trusted main: `f3ece23e0f128258947137764b39b7d59dd7b2a8` +Reviewed against trusted main: `bba4ba5f171a4438b072740707a5cf8bde49d9af` -Reviewed at: `2026-07-25T10:27:17Z` +Reviewed at: `2026-07-25T16:41:40Z` Reviewer run IDs: `art_plan2_ext_senior`, `art_plan2_amend_arch`, `art_plan2_ext_qa`, `art_plan2_amend_security`, `art_plan2_ext_product`, @@ -77,9 +77,16 @@ implemented or activated. - made 05's typed-capability boundary and prohibition on ART/TASK imports of AUTH-owned repositories explicit, then proved the exact hosted pytest collection rather than relying only on the direct script entry point; -- rebased onto trusted main `f3ece23e` and preserved AUTH-10C, AUTH-002, and - PLAN2 review history. +- rebased onto trusted main `bba4ba5f` and preserved AUTH-10C, AUTH-002, CI, + and PLAN2 review history; - recorded the final amendment review outcome in the canonical review log. +- proved all 22 PLAN2 commits patch-equivalent across the latest-main rebase and + confirmed the merged exact Ruff/CI custody changes introduce no ART conflict; + Ruff 0.15.22, the 300-test hosted-equivalent suite, and all deterministic + planning gates pass; +- removed the obsolete ART contract read exposed by the new Ruff `F841` rule; + focused Ruff, 104 collected pytest cases, and the 100 direct regression cases + pass without weakening an assertion. ## Accepted Risks diff --git a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-pr-trust-bundle.md b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-pr-trust-bundle.md index 6f84751b..305bdb8d 100644 --- a/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-pr-trust-bundle.md +++ b/.agent-loop/initiatives/WS-ART-001-immutable-artifact-storage/reviews/WS-ART-001-PLAN2-pr-trust-bundle.md @@ -39,7 +39,7 @@ successor `WS-ART-001-03A` with a separate explicit start. - diff, stale-contract, markdown-link, and 100 agent-gate tests pass; - all nine required reviewer tracks pass after repair; -- exact reviewed candidate `648dd311` is based on trusted main `f3ece23e`; +- exact reviewed candidate `1dbc0d4e` is based on trusted main `bba4ba5f`; - the stale-authorization documentation gate and its technical-module regression repair pass without reopening deprecated product-role vocabulary; - detailed findings and resolutions are recorded in