Repository navigation
Description
Implementation progress — 27 October 2026
Unresolved implementation; only a narrow related repair is published. #265 preserves unknown custody across the pre-ready launch window.
The real reparented/detached descendant reservation-release gap remains unresolved. The pre-ready repair does not prove complete post-launch descendant settlement or measured-capacity admission acceptance. Polling disappearance, monitor exit and private unvalidated cohort observations must not authorize release.
The referenced candidates are open and unmerged in stack #253. This progress map is non-closing; issue acceptance remains authoritative. Historical text below calling #238 open is superseded: #238 merged as c43cf0b.
Problem and evidence
Launching several deliveries together overloaded the local host during the reliability incident. A fixed count of active roles does not account for each role's process tree, parallel test workers, browsers, local services and dependency preparation.
Existing slot and native cleanup safeguards (including #77 and #150) remain necessary. Add resource-aware admission without removing those ownership checks.
Required behavior
- Measure current host memory pressure and owned process-tree usage, including child test/browser workers, before admitting more work.
- Reserve a configurable, explainable resource budget per workload and leave host headroom. Queue work when safe capacity is insufficient; display the resource reason and queue progress.
- Include nested test parallelism and preparation in the budget; do not allow each role to independently consume the entire host.
- Release reservations only after verified settlement, and reconcile stale reservations after restart without counting unknown processes as safely gone.
- Distinguish healthy long-running work from unavailable observability or dead ownership. Do not restore the removed controller-milestone “no progress” cancellation heuristic from fix: prevent false no-progress cancellation and disable limits live #229.
- Persist enough admission/peak/pressure evidence to diagnose overload and tune estimates, without storing unrelated process contents or private command arguments.
Acceptance
Exercise real owned process trees and representative bounded check/browser workloads. Demonstrate that pressure queues new work before an unsafe launch, queued work resumes when resources settle, and cancellation/restart neither leaks slots nor double-admits.
Report resource peaks, actual concurrency and completed useful work; a higher test count or lower wall time by dropping coverage is not evidence of success.
Dependencies
No blocking prerequisite in this issue set. This work can proceed in parallel with the other independent foundations; integrated qualification will verify their composition.
Blocks:
- test(runtime): qualify recovery, pause and cleanup through public interfaces #246 — qualify recovery, pause and cleanup through public interfaces.
Activity
ebarti commentedon Oct 26, 2026
Implementation checkpoint — confirmed P1 settlement gap; this issue remains unresolved.
Trigger and reproduction: an owned controller helper spawns a detached child, closes inherited descriptors/stdio, exits, and reparents the child before the final controller-tree observation. On local candidate e9478cc,
test_reparented_controller_descendant_cannot_release_reservation(runtime/tests/test_delivery_resource_capacity.py:519) fails while that child is still live. The fixture subsequently verifies cleanup of its child. This is an actual process reproduction, not a mocked successful settlement.Code evidence:
ResourceLease.mark_unmonitored(False)(runtime/src/devflow_temporal/delivery_resource_capacity.py:737–761) relies on the current controller descendants. The escaped child is absent from that snapshot, allowingunmonitored_activeto be cleared and the reservation to become releasable. Expected: retain HELD/unknown until the workload's supported custody protocol proves settlement; no competing admission may treat it as gone. Observed: the current normal-return path can clear the marker while work remains alive.The owning cause is using a point-in-time ancestry observation as a lifetime proof. Repeating the snapshot, process-group membership, or an inherited descriptor that a child can close does not prove arbitrary detached-child closure. The existing native monitor receipt is scoped to observed identity-bound descendants and owned ports; that useful safeguard must not be relabeled as a stronger proof. A host-wide process baseline also introduces unrelated-process holds and does not establish ownership.
Acceptance additions:
Related work: #77 and #150 remain required custody safeguards. This defect blocks the native-resource composition of #245 and the final qualification in #246. The current #242 branch is intentionally unpublished and contains the failing regression. Earlier passing resource/queue tests and measured peaks do not establish this missing settlement invariant. No installed runtime or active delivery was modified.
ebarti commentedon Oct 26, 2026
Additional owned macOS custody qualification confirms the unresolved blocker recorded above.
The keeper fixture preserved a unique launchd resource cohort while a real child double-forked, called setsid, closed its descriptors and was reparented to PID 1. Private coalition counters changed while the detached child was alive and after it exited; an unrelated owned process had a different cohort. After the final process exited, the query returned ESRCH. That disappearance remains an unknown outcome, not proof of an empty cohort. The accounting layout is a private, unvalidated ABI for the installed kernel; these diagnostic counts cannot authorize resource settlement.
The production regression still demonstrates premature reservation release while the detached child is alive. No safe release repair is qualified yet, and the failing candidate has not been published as a fix. A generic process group, inherited descriptor, polling loop or wrapper exit does not establish this invariant for arbitrary Git, pnpm and browser descendants.
Acceptance remains: prove the actual owned execution cohort is settled before releasing its reservation, retain an unknown reservation after observer/keeper failure, and preserve normal cleanup and supported dependency/browser flows. #245 native pause and #246 combined recovery qualification genuinely depend on this custody boundary. All experiments used disposable owned jobs; no installed runtime, settings or active delivery changed.
ebarti commentedon Oct 26, 2026
Additional confirmed pre-GO custody failure within the same native settlement scope (P1): an allocated launch with no ready.json does not prove that no child exists. native_child.py invokes ps before publishing ready identity; the monitor can die after Popen while that child and ps remain alive.
An owned real-process reproduction paused that child before ready, killed its worker and detached monitor, then called public reconcile_process. The original code returned observed-native-confirmed while the child was alive. Later child disappearance also cannot manufacture the missing join or stream-drain evidence. Current published source shows the inference in delivery_native_process._reconcile_interrupted around lines105–116; native_child.py invokes check_output at line16.
The isolated repair leaves interrupted allocated journals unknown, retains the resource, denies reattachment GO and public prelaunch retry, and rejects an interrupted authorized journal that merely labels its provider not_started. Explicit server-proven no-launch and authenticated EOF/join/drained paths remain available. Fifteen focused real/negative/positive cases passed on the isolated candidate; a clean backport to the published stack is in progress. This does not resolve the separate arbitrary reparented descendant gap already recorded here.
Acceptance for this sequence: a live pre-ready child must never produce confirmed teardown; ambiguous history remains unknown even after polling stops seeing it; valid independently proven no-launch/no-GO keeps working; no provider effect or second GO occurs during reconciliation. Owned process groups were cleaned up; active deliveries and the installed runtime were untouched. Related integration: #243, #245.
fix(admission): prepare and verify the execution environment (#263)
fix(native): retain custody across the pre-ready launch window (#265)
fix(checkpoint): journal restoration and preserve uncertain custody (#…
fix(recovery): verify retained workspaces before acknowledging recove…
fix(install): preserve managed runtime generations across updates (#268)
fix(recovery): verify current checkpoint custody on readback (#275)