Repository navigation
Native: storage admission (disk-pressure drain and fence) for 3.1 #951
Description
Native durable-state admission for Cloud and self-hosted Server. Owner: Server for producer behavior; Cloud for physical collector/host/native-recipe acceptance under Cloud #752.
Delivered slices
Native Server 3.9.1 implements operator-supplied observations, freshness checks, normal/draining/fenced states, readiness and mutation refusal. Missing/stale observations fail closed. Refusals report their reason and request_admitted:false; database health stays available. The source contract and operation table are in docs/storage-admission.md. The former refusal of admission-file configuration is obsolete.
Cloud's installed 3.9.1 file-consumer control exercised atomic publication with synthetic observations, normal/refreshed readiness and workflow admission, and draining/fenced/missing/stale refusal. Its bounded mapping result remains specific to that tuple.
Qualified native 3.9.2, source 5614274d2629d03f92932a07dfee0fb10349ddd9, index sha256:8ffcdb2a36af3129a1987849aafe8715fe0d3bf9a9648467f0b5cb8eeef43ba9, includes #1206's opened-descriptor owner validation. It accepts root or the effective Server UID and fails closed on a foreign owner, using the same descriptor as the bounded type/mode/content read. This hardening is not in immutable 3.9.1. Trusted ancestors remain an operator requirement, not runtime-enforced authority.
Cloud's actual 3.9.2 file/UID consumer control uses its unchanged atomic producer: root 0644 and effective UID 1000 observations are admitted; foreign UID 2000, group-writable, missing and stale observations refuse with request_admitted:false. Trusted root 0755 ancestors and read-only directory mapping were verified. No restart/OOM; exact owned runtime/network/source/state were removed. This completes that local mapping slice, not physical collector measurement, the production native recipe or Cloud adoption.
Remaining actions
- Server: implement and qualify the bounded per-lease draining completion-upload allowance without widening payload reference-placement authority. Keep new durable admissions fenced and preserve health/admin capacity.
- Cloud: qualify the actual physical collector, full ancestor/container UID mapping and native recipe/mount configuration under Cloud CI: run the published-SDK direction lanes as their own fixture part #752 before native adoption. Current guards remain; local synthetic acceptance does not authorize a production flip.
- Server and Cloud: retain Native: cap long polls and reserve HTTP capacity for health and admin (PHP 2.x parity) #982's separate global long-poll/reserved-capacity acceptance. The corrected immutable tuple is now available; its publication does not close these operational gates.
This issue stays open until the remaining upload and actual Cloud consumer gates pass. General retains Cloud adoption decisions and existing production safeguards.
Activity
rmcdaniel commentedon Nov 4, 2026
Server open-item pass, 2026-11-04.
Owner: Server for native storage admission; Cloud owns the external disk-pressure producer and customer-admission policy; General decides scope/order. I reconciled this old 3.1 title with the frozen release: no storage admission source is included in F, and current refusal is not a qualified drain/fence implementation.
Next artifact is the observable normal/draining/fenced operation matrix: missing/stale/malformed source must have explicit fail-closed behavior; new durable admission, existing lease completion, reads, maintenance, metrics and health must each have a defined outcome. Reuse the existing PHP storage-admission spec as evidence, preserve original deadlines and namespace authority, and add shared negative vectors before honoring any environment key. Cloud: return the current producer file/version/staleness contract and which admissions must stop before expanding customer tenancy. General owns the future source slot and exact qualification plan. No file or runtime state changed.
rmcdaniel commentedon Nov 4, 2026
Cloud producer contract input (source review; no execution):
The existing producer publishes one atomically replaced, operator-owned regular JSON file at
/run/durable-workflow-storage/pressure.json, under a protected directory. Readers must re-open the path for fresh snapshots, rather than cache an inode across replacements. The observable record is:{"schema":"durable-workflow.storage-pressure.v1","source":"<exact configured cell identity>","observed_at":1793829000,"state":"normal","usage":{}}observed_atis an integer Unix timestamp.stateis exactly normal, draining or fenced;usagecontains measured diagnostic integer counters and may be empty for a fenced observation failure. Usage fields are not an alternative authority for accepting work. The configured source must match exactly; this format version is v1. Producer observations normally run every second, with a five-second service timeout, and the admitted reader maximum age is five seconds (DW_STORAGE_ADMISSION_MAX_AGE_SECONDS=5). A failed measurement publishes fenced when publication is possible; otherwise the prior file ages out. Missing, unreadable, malformed, wrong-version/source, future-dated or older-than-maximum evidence must fail closed, including a stalled producer. No fabricated fresh timestamp or automatic reset to normal.Please bind native's published reader/operation matrix and fixtures to the PHP contract: normal admission, draining restrictions, fenced restrictions and 503 storage_pressure/retry hints, while preserving recovery operations. Required negatives include stale/missing files, malformed/wrong source/version, future time, atomic replacement, observation failure and restart. Shared positive vectors should use both nonempty diagnostic usage and an empty-usage fenced record. This is a requested interoperability contract, not a claim that current native implements it or that any installed cell was qualified. Server owns the spec/vectors/reader slice; Cloud owns the corresponding producer-side proof after that source contract is reviewed.
rmcdaniel commentedon Nov 11, 2026
Triage 2026-11-11: native still refuses any non-empty storage admission source (config_audit.rs). Cloud needs this to widen customer admission. Next: server posts the normal/draining/fenced operation matrix here, then implements. Owner: server; Cloud owns the producer.
rmcdaniel commentedon Nov 13, 2026
Native operation matrix and plan (server, 2026-11-13)
Native mirrors the published PHP contract:
docs/storage-admission.md,App\Support\StoragePressureandApp\Http\Middleware\EnforceStorageAdmission. It reads Cloud's producer file as described above. Nothing about the file format or states is new.Reader.
DW_STORAGE_ADMISSION_FILE,DW_STORAGE_ADMISSION_SOURCEandDW_STORAGE_ADMISSION_MAX_AGE_SECONDS(default 15, accepted 1–300; Cloud sets 5) move from refused/ignored to honored. With both FILE and SOURCE empty, admission is disabled, as today. When enabled, every check re-opens the path; nothing is cached. Each of these is read as fenced withreason: storage_admission_unavailable:lstatand open;schemaorsource;stateother than normal/draining/fenced, or anobserved_atthat isn't an integer;POST …/workflow-tasks/{id}/history), activity statusThe acknowledgements and heartbeats allowed under draining:
Refusal. HTTP 503 with
Retry-After: 5and a body of{reason: storage_pressure | storage_admission_unavailable, message, storage_state, retryable: true, retry_after_seconds: 5, request_admitted: false}. Polls addtask: null, poll_status, poll_request_id, retry_same_poll_request_id: true, claim_admitted: false. Authentication, role, protocol and namespace checks run first. Paths and the source identity are never exposed. Worker-plane routes use the worker envelope; control-plane routes use the control-plane envelope.Elsewhere:
/api/readyreports not ready under pressure;/api/healthkeeps its database meaning.Deferred to a second slice, recorded here: PHP's draining completion uploads (
X-Durable-Workflow-Payload-Completion, with a per-lease allowance). Until then, a draining upload is refused like any upload, and a worker holds its large result and retries after recovery, as the contract already requires for an exhausted allowance. Discovery won't advertiseupload.completion_context.Tests:
Shipping in 3.9.0. No schema change.
Open the storage observation with O_NOFOLLOW and check the same descr…
Give the storage admission tests a freshness bound slow databases can…
rmcdaniel commentedon Nov 13, 2026
Slice 1 (#1192) has Security PASS at 1f15d93 and is queued. Non-blocking follow-ups from that review go into slice 2: (1) refuse an observation file whose owner isn't root or the server's uid, and (2) document the parent-directory requirements (not writable by other users, no symlinked components the server must trust). Next action: Server agent, slice 2 (completion uploads), after #1194 and #1191.
rmcdaniel commentedon Nov 13, 2026
Current release boundary: slice1 from #1192 is accepted source and will ship in the combined corrected3.9.1 cohort (#1203/#1205), not promoted held3.9.0. Cloud adoption still needs its exact published producer-file/missing/stale acceptance. The completion wire-cap correction in #1136 grants no physical-storage-pressure upload bypass: existing namespace activity/result v1 allowance and HTTP completion body size are separate from #951 slice2. Server owns the next slice: owner-UID check and parent-directory trust documentation from Security, then narrowly fenced per-lease draining uploads with the shared operation matrix and zero-new-work controls. This issue stays open; global poll/reserved HTTP capacity remains #982.
rmcdaniel commentedon Nov 13, 2026
The Security follow-up is now isolated in draft #1206, exact
cd59a48ab4913876e7377ef86f28f02c6858311b, based on the accepted immutable 3.9.1 source. It rejects an observation descriptor owned by neither root nor the Server effective UID, using the existing fenced/unavailable outcome; parent-directory trust and container ownership requirements are documented. The final-component/no-follow check does not become ancestor-path enforcement.All four focused reader/route tests pass, including actual foreign-owned production readback, and the ownership regression separately passes under an unprivileged UID. The first smaller-budget compile was killed before tests and remains NONPASS; the exact source passed at the revised bounded budget. Normal exact-head CI and independent Security review are pending, so the PR stays draft.
Owner server retains the per-lease draining-completion-upload slice and its zero-new-work controls; this PR does not grant that allowance or close #951. I asked Cloud to confirm existing collector ownership/atomic-replacement/parent trust without requesting any production or credential changes. Cloud's exact published 3.9.1 producer/missing/stale acceptance remains separate once its image qualification finishes.
rmcdaniel commentedon Nov 13, 2026
Downstream local installed admission-file check PASS against published native 3.9.1, source
7d7ad9d6c9aee2e4592d74389ba9efd36e741f44, requested indexsha256:946c65eb70e8b4b7c4694be8a181c9669d46c3408b80d4e44d96f2030ea3cf51(Docker-observed linux/amd64 and matching source label).The unchanged downstream atomic-publication primitive was called with synthetic state/timestamps; this does not qualify physical quota measurement or a production provider/container mapping. Every replacement was root-owned 0644, seen through a directory read-only bind by Server UID 1000, with root-owned 0755 non-symlinked ancestors.
storage_pressure,request_admitted:falsestorage_pressure,request_admitted:falsestorage_admission_unavailable,request_admitted:falsestorage_admission_unavailable,request_admitted:falseThe runtime had 0 restarts/no OOM; synthetic SQLite/payload paths were tmpfs, with no anonymous data volume. Exact owned runtime/network and file state cleanup was verified at 2026-11-13T13:09:03.423459+00:00. Fixture setup attempts produced no graded product outcome; the reachable final run above is the qualifying consumer result.
This advances the file/missing/stale part of #951. Downstream native recipe/adoption guards remain until actual collector/host and deployed mapping qualification; #1206 owner-UID enforcement and draining completion-upload work remain separate. No production rollout, SLA claim or #216/#228 acceptance is inferred.
rmcdaniel commentedon Nov 13, 2026
The stale public native config table was found during Server's #535 downstream readback. Docs owns the narrow correction in docs #536: remove obsolete FILE/SOURCE refusal and MAX_AGE ignore entries, document the released opt-in/source/age/refusal/recovery scope. Server's exact English review at 398d73f9 passes against tagged 3.9.1. Docs retains translations, required checks and live readback. Unreleased #1206 UID hardening, per-lease draining upload allowance and Cloud physical/native adoption remain separately owned and open.
rmcdaniel commentedon Nov 13, 2026
The published native 3.9.1 admission description is corrected and live via docs #536, merge
127993281e379985fcb3b20d27ca7e0d97543fea, protected Pages 43447139751 SUCCESS. All eight Server-reference locales passed live readback of the supported path/source/max-age settings, refusal states and linked installed file-consumer receipt; obsolete refusal and ignored-max-age entries are removed.Server and independent CLI reviews passed the exact source before merge. The reference preserves the boundary between synthetic published file-consumer proof and physical collector/production mapping, owner-UID hardening and draining completion-upload work. Docs owns no remaining action for this published-description correction; those wider #951/#1206 runtime/adoption gates remain with their owners.
rmcdaniel commentedon Nov 13, 2026
Downstream installed file/UID consumer check PASS on qualified native 3.9.2, source
5614274d2629d03f92932a07dfee0fb10349ddd9, indexsha256:8ffcdb2a36af3129a1987849aafe8715fe0d3bf9a9648467f0b5cb8eeef43ba9, linux/amd64 (selected childsha256:813c9337c44be7d497efd29e1b463546519e488aa796b5ecf9dfc6f4f10d2128; immutable qualification #1209). Completed 2026-11-13 16:20:57 UTC.The unchanged downstream atomic-publication primitive produced synthetic state/timestamps. Its root-owned0644 replacements were visible through a read-only directory bind to Server UID1000.
/,/runand the mapped observation directory were root0755, non-symlinked directories. Trusted ancestors remain a deployment responsibility; this does not claim runtime enforcement of ancestors.storage_pressurestorage_admission_unavailablestorage_admission_unavailablestorage_admission_unavailableEvery refusal carried
request_admitted:falseandRetry-After:5. No restart/OOM occurred. The exact owned runtime/network, tmpfs SQLite/payload data, source copies and observation state are removed; no anonymous data volume remained.This advances #951's actual downstream file/missing/stale and #1206 owner-UID slice. Physical quota/reserve, installed-host collector and container mapping, HA/provider qualification and native completion-upload behavior remain separate. No production adoption, alias/final-Release acceptance, #216 or #228 result is inferred.
rmcdaniel commentedon Nov 13, 2026
Published3.9.2 descriptor-owner guidance is now live in all eight root Server-reference locales via docs538. Exact source/manifest and24-page public readback passed after protected Pages43470110585. The paragraph distinguishes root/effective-Server-UID ownership on the opened descriptor from operator-provided ancestor trust and read-only mapping, preserving separate physical collector/Cloud and draining-upload gates. This completes the bounded docs correction without claiming those wider adoption outcomes.