reliability: prevent timed-out DNS workers from exhausting resolver capacity

[seonghobae]

Buyer-visible availability boundary

EgressWeave bounds caller-facing DNS waits and process-wide resolver concurrency, but an already-running platform socket.getaddrinfo(...) call cannot be safely force-cancelled by Python. The accepted design therefore prevents repeated overlapping validation of the same canonical (hostname, port) from multiplying live resolver workers/slots while retaining a fresh DNS lookup after a live flight completes. This is a bounded availability/resource-amplification boundary, not an SSRF-authority bypass.

Current canonical implementation

Historical #109/#156 and their predecessor heads remain TDD/review history only. The single live reconstruction owner is Draft PR #205 — reliability: reconstruct DNS single-flight on protected main.

Fresh exact identity:

• protected/live main: 7faf7a3b8a47980113982914000e724ab6a6cda5;
• predecessor reliability: reconstruct DNS single-flight on integrated validation-result boundary #156 head: f95f75747a555b95a44879f09f28489ee6d41121 (historical only; its checks/reviews do not transfer);
• reliability: reconstruct DNS single-flight on protected main #205 exact current head: 39332783589c8571f15d77ea60170d918fb82461;
• reliability: reconstruct DNS single-flight on protected main #205 is open / Draft / mergeable, based on exact protected main 7faf7a3b8a47980113982914000e724ab6a6cda5;
• reliability: reconstruct DNS single-flight on protected main #205 is 0 commits behind that protected base and changes only the eight DNS single-flight paths recorded in the PR, preserving the protected release workflow/runbook/evidence contracts.

Runtime contract

One canonical (hostname, port) may share only one live raw DNS lookup. Each caller retains its own finite deadline and independently reapplies address scope, local-development rules, deduplication, and max_resolved_addresses. Completed results are never cached. Worker-start failure releases the resolver slot and removes the live flight without swallowing KeyboardInterrupt or SystemExit; ordinary public failures remain behind the generic non-leaking denial boundary.

Exact-current-head evidence

On unchanged 39332783589c8571f15d77ea60170d918fb82461:

• repository ci run 31598859748: completed success;
• required SAST Semgrep run 31598859828: completed success;
• required Security Scan run 31598859864: aggregate success, but job 94120959500 shows the actual immutable-pinned Dependency review step as skipped; aggregate green is therefore non-passing supply-chain evidence under security: fail closed when dependency review is unavailable #76;
• formal reviews: none at the latest exact-head refetch.

The separately governed organization repair remains read-only ContextualWisdomLab/.github#897, exact head 59505c1d89eb7ea816e921b6da38079c736608c2, open / Ready / mergeable / unmerged at the latest refetch. Do not add an EgressWeave-local substitute, weaken the gate, invent credentials, or churn this clean source head merely to reproduce the same skipped result.

Acceptance criteria

• Reproduce same-authority slow-resolver amplification deterministically without public DNS.
• Establish public sync/async RED evidence before production repair.
• Keep finite caller waits and finite process-wide resolver work.
• Share only a live same-authority resolver worker/slot; do not cache completed DNS results.
• Reapply caller-specific address scope, local-development rules, deduplication, and max_resolved_addresses after shared raw resolution.
• Count async executor scheduling delay inside the public DNS deadline while preserving caller-directed cancellation.
• Fail closed on capacity exhaustion, pre-start exhaustion, worker-start failure, empty/incomplete result, resolver failure, and caller timeout.
• Normalize dependency/resolver/worker-start failures to the stable generic public denial without private exception provenance.
• Preserve ValidatedEgressURL integrity, exact authority, DNS pinning through connect, TLS identity, proxy isolation, request/response controls, and deterministic cleanup.
• Reconstruct the unique DNS single-flight delta directly on exact protected main as reliability: reconstruct DNS single-flight on protected main #205 without force-push/destructive rebase and obtain exact-head repository CI/SAST success.
• Integrate the corrected separately governed central dependency-review repair and obtain a fresh unchanged reliability: reconstruct DNS single-flight on protected main #205 Security Scan whose actual immutable-pinned Dependency review action executes and succeeds.
• Require every other live organization-required workflow to be genuinely passing on the unchanged exact head; queued/skipped/neutral/backend-unavailable/wrapper-only/status/model/predecessor evidence does not pass by inference.
• Obtain current semantic/source review with zero valid unresolved findings; formal approval is required only if the live ruleset actually requires it.
• Immediately before integration refetch exact head, live base, ancestry, ruleset, required workflows, unresolved findings/reviews/threads, and writer evidence; merge only the unchanged accepted tree.
• Verify the DNS live-flight/resource contract from protected main before closure.

Non-goals

• No custom recursive DNS resolver or DNSSEC claim.
• No claim that a Python library can instantly terminate a blocked operating-system resolver.
• No completed-result cache or stale DNS reuse for availability.
• No destination-policy, TLS, proxy, request/response, credential, workflow-publisher, release, tag/ref, package-publication, or persistence widening.
• No EgressWeave-local replacement for organization-owned required workflows.

[seonghobae]

RCA / implementation handoff from protected main 10d0c51daf2ad278d66f43be479df8cf6b08ba6d (source mutation remains deferred while PR #105 owns validation.py).

The first causal boundary is confirmed in _resolve_all_global_addresses(...): one caller acquires _DNS_RESOLUTION_SLOTS, starts a daemon worker, and waits on a per-call Queue only until its caller deadline. On queue.Empty the caller returns a generic denial, but the worker releases the semaphore only in its later finally. Therefore the caller timeout bounds waiting, not resolver-slot lifetime. Repeated overlapping calls for the same hostile authority can acquire independent slots and start independent getaddrinfo calls until the 32-slot process ceiling is exhausted. The async API inherits the same behavior through asyncio.to_thread(...); this is not a separate async root cause.

Smallest remedy selected for implementation after #105 clears: in-flight single-flight resolution keyed by the canonical (hostname, port) authority, with no completed-result cache. One shared worker owns one semaphore slot and one socket.getaddrinfo call. Concurrent callers for that exact authority join the same in-flight record but retain independent dns_timeout_seconds deadlines; caller timeout does not detach or duplicate resolver work. When the worker finishes, it atomically publishes the raw resolved address set/error, wakes all waiters, removes only its own in-flight entry, and releases its slot. A later request after completion performs a fresh DNS resolution, preserving rebinding/freshness semantics. Policy-specific address validation and max_resolved_addresses enforcement remain per caller rather than being cached across policies.

Rejected as the primary fix: (1) Python thread cancellation—getaddrinfo cannot be safely killed in-process; (2) timeout-only negative cache—reduces retries but does not prevent an overlapping first-wave slot fan-out; (3) detached worker replacement—would weaken the finite-work boundary; (4) custom recursive/c-ares-style resolver—too large a compatibility/security change for this defect; (5) deployment-only resolver timeout—useful defense in depth but does not repair the library's own repeated-call amplification.

Deterministic RED plan when the branch lease clears: use a controlled blocking getaddrinfo with events, never public DNS. Prove N overlapping public sync validations for one exact authority all hit their caller deadline while exactly one resolver worker/slot remains live; prove the equivalent async public path; prove a second distinct authority is still governed by the global slot ceiling; release the blocking resolver and prove the slot/in-flight record is reclaimed and a later request performs a fresh resolution. Also prove distinct authorities remain independently bounded, every returned address is revalidated under each caller's policy, and no stale result survives worker completion.

This narrows #108 to a testable root-cause repair without touching #105 or changing authority, local-development, TLS, DNS-pinning, max_resolved_addresses, error, workflow, credential, or release semantics.

[seonghobae]

Canonical implementation path is now stacked Draft PR #109 on exact predecessor #105 head c60d1b4eae45085c0e5c6e6e7fae8152456c3a54. Immutable behavioral RED is exact test-only head f361bf58816285c3c76950ab2183ea0464ed4fff, CI 31333539340: package acceptance and Ruff succeeded; all Python 3.10–3.13 lanes failed exactly the sync+async single-flight regressions, with Python 3.13.14 at 2 failed, 751 passed; each reproduction observed four live same-authority resolver calls instead of one. A single write-capable OpenCode handoff is active on #109 from that exact RED head; do not race that branch. Other EgressWeave lanes remain independently executable.

[seonghobae]

Fresh exact-state refresh after the latest #156 head movement:

• protected main: 52a5c6a0862e0d035b0698b79a0e8556269e0b00;
• current Ready PR reliability: reconstruct DNS single-flight on integrated validation-result boundary #156 head: b81af75e84abe5964edce9dafc7bebc9ee44dd2f on main;
• exact-head CI 31537564654, SAST Semgrep 31537564591, and aggregate Security Scan 31537564733 are terminal success;
• Security Scan dependency-review job 93932226869 is not accepted dependency-review evidence: its actual pinned Dependency review step 4 completed skipped while the wrapper job concluded success;
• current reliability: reconstruct DNS single-flight on integrated validation-result boundary #156 review threads are all resolved. Historical CodeRabbit/GitHub Code Quality findings on worker-start cleanup and broad exception handling are therefore not open findings on this exact head.

Do not transfer the older 4ff0f10.../35649392... check or review identities as current acceptance. The implementation lane remains unchanged and should not be churned merely to retrigger the centrally owned supply-chain gate. The read-only organization repair remains .github#897; after that repair is protected-main integrated, require a fresh unchanged-head EgressWeave Security Scan in which the real pinned Dependency review action executes and succeeds before accepting this lane.