Repository navigation
Back off persistent worker poll failures - #203
Merged
Merged
Conversation
rmcdaniel
marked this pull request as ready for review
October 23, 2026 07:25
Member
Author
|
The original final-head CI run 39918720501 is retained as failed: all three Python versions reject the new corpus fixture because I omitted its workflow from the standalone corpus runner's explicit registration list. The focused worker test exercised it, but did not cover that separate consumer. Test-only correction |
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Workers currently retry persistent authentication, HTTP and transport poll failures every second, multiplying requests and warnings during outages. Workflow, activity and query polling now back off independently with fresh jitter, a ceiling that doubles from 1 to 30 seconds, and reset after any successful response. HTTP 401/403 diagnostics name credential/permission failures; unchanged warnings repeat at most once per minute per poller while every failure still records an error metric.
Typed capacity waits and storage-admission propagation retain their existing paths. Failure waits release poll reservations and remain interruptible by shutdown. Worker documentation describes the limits and recovery.
Addresses #189. Initial unchanged-source evidence: 21 behavior controls fail; three storage-admission controls pass. Current focused qualification: 59 tests pass, including real Client/HTTP transport mapping, task and empty recovery, jitter, cap, warning limits, reservation release and shutdown. Ruff passes; mypy passes all 36 source files on functional candidate dceb721. Full local Python 3.12 functional qualification passes 3,183 tests / two skips / 105 integration deselections in 158.64 seconds; final focused worker/storage/backoff qualification passes 399 tests / two skips. The later cold-history compatibility control passes independently; these full/focused counts belong to their earlier exact revisions, not a whole-final-head claim. Fresh hosted normal CI remains required. A test-only Sonar warning about multiple possible invocations inside an exception assertion was corrected without changing the gate.
Publication and installed-package acceptance remain required before closing #189. The existing registry-side Trusted Publisher dependency in #197 still applies; this PR neither creates a release tag nor changes the prepared security-only version metadata.
The original hosted corpus refusal is retained. A new, actually exercised cold-history fixture proves recovery preserves the original task attempt/run and recorded activity result without rescheduling the activity. The unchanged corpus validator now accepts growth from 65 to 66 replay fixtures, and all 54 corpus guard tests pass. No guard or timeout was weakened.