MariaDB held activity poll can time out worker re-registration in native tests

Observed failure

Exact required Actions run 43405494921, MariaDB10.11 native-tests-b, job131820856947, failed worker_poll_fence::held_activity_polls_end_when_the_registration_changes at rust/tests/execution/worker_poll_fence.rs:141. Re-registering act-replaced while its activity long poll waits returned HTTP503 storage_unavailable with the closed diagnostic error_kind: pool_timeout, where the test expects HTTP201. The cell otherwise passed403 tests (one ignored). No original SQL/credentials are needed to reproduce the report.

The test, HTTP routing, poll-fence implementation, pool policy, transaction wrapper, Cargo inputs and Actions resource configuration are byte-identical between main f315a01 and #1202 head cb7158a. That scenario starts/polls/completes activities and registers workers; it does not call the changed retained signal/update lookup. This establishes a separate observed failure location, not a resolved root cause or a passing qualification.

Acceptance

Reproduce the held-activity-poll / re-registration schedule on the pinned MariaDB10.11 image with the existing CI CPU/memory and two-test-thread bounds; compare baseline and candidate without treating repeated green retries as a fix.
Capture bounded pool/connection/lock diagnostics when the refusal occurs and establish whether the cause is runtime cancellation/reset, pool ownership or fixture cleanup/load.
Correct the owning cause with a focused deterministic control. Preserve the existing pool bound, acquisition deadline and registration fence; do not hide HTTP503 with blanket test retries or raise timeouts without causal evidence.
Obtain normal current-head source/backend checks and independent review for any runtime change.

Owner: Server. The pinned baseline investigation ran three declared two-test pairs at the runtime-equivalent later baseline and CI limits: six passes, no reproduction, all owned resources/state removed. No fix is claimed. Store::begin can also emit PoolTimedOut while waiting for its FIFO transition mutex, so the original HTTP diagnostic does not identify the phase.

The single original 405-test shard attempt is runner-blocked: its declared 900-second bound expired after 210 explicit success lines, no failed-test lines/OOM/pool-timeout, before the reported held-activity case executed. This is incomplete evidence, not full-shard acceptance; all owned runtime/DB/network/state is removed.

Current diagnostic source is e75db9fb, a three-file failure-observation overlay on the later runtime-equivalent cb7158a0 baseline. It distinguishes transition-queue, MySQL pool/begin, ownership-row lock and session-reset failures without changing bounds or ordering. The instrumented pair passed both cases, and the single nine-test neighborhood passed all nine cases. The latter recorded an ownership-lock refusal consistent with its existing intentional shutdown-pressure control, not the original typed pool timeout. Its original substring-classifier false positive is preserved and explicitly corrected in the derived analysis. All owned resources/state are absent. None of these passes fixes the defect.

The original complete Actions artifact is now independently bound: 14027583636, ZIP SHA-256 50c4ecf63c8d362f6d2e25195cd7fde3f269d919461749d0c00f6c4e5dc98809. Its exact 405-test membership matches the local selection. The historical executed merge is 1ecdfb1fda1b37a8723f1de6f3be5ad0e41b0d43 (PR head 6d7afc6a); the later baseline is not relabeled as that execution. Their entire Rust difference is two additional assertions in an unrelated update-validation test. All runtime code, Cargo inputs and the targeted test bodies are identical. Original DB evidence confirms its 1-GiB cap and no OOM, but supplies no per-test start/acquisition timestamps.

The timestamped original-context attempt at diagnostic 688c1cdd is also runner-blocked under the same 900-second limit: 208 distinct success lines, no failed-test or acquisition-phase failure lines, and the reported registration case never started. The 13 added test-only logging lines preserve test identities and registration-call boundaries; Docker stream timestamps are retained together. Runtime/Cargo/assertion/deadline/fence inputs are unchanged from the prior observer. The fresh executable 2ab90d42 matched before/after execution; actual listing still exactly matches the original 405 identities. All captured compiler/runner/DB/network IDs and scoped inventories are independently absent, with no OOM event; private DB/temp state is removed. This establishes another local context cutoff, not an original-failure cause or shard acceptance.

The complete hosted source/profile is now prepared for review at 227984958e8f26b6860de497b08040a4bcdd4f9c, tree 702ad11d6cdf08dd9eea5aab451627ad35a4f0df, with the observer 688c1cdd Rust subtree unchanged. Its manual-only diagnostic branch contains the exact 405 identities, fixed original pre-test DB/layout bootstrap, unchanged test resources/two threads/900-second bound, executable identity and complete cleanup/evidence paths. Seven offline controls, unchanged isolation policy, source preflight and YAML/syntax checks pass. The cold-cache/current-hosted-VM differences are explicit. This is source/profile preparation, not a dispatched capture or production fix.

Independent exact source/profile review passes. Server recorded one owner-selected invocation after immediate branch/workflow/input readback. Actual Actions run 43571681464, attempt 1 selects exact profile 22798495 and observer 688c1cdd. Its complete terminal packet records 404 successful tests plus one declared ignore, exact 405 membership in 455.089 seconds, the target registration actually passing in 14.151 ms, one expected ownership-pressure observation with pool_timeout=false, no original typed pool timeout, and passing primary/fallback cleanup. Artifact 14104074793 matches API/actual ZIP SHA-256 7b7eb6d982add67fc73d22e802776708f2a71ba0e7dd87f0d67c6fec40995990. This is a non-reproducing baseline, not a demonstrated cause or fix. No retry, source move, branch merge or larger deadline is selected.

Independent terminal evidence/cleanup counter-review passes at the exact sole run and raw artifact. This closes the one selected diagnostic observation, with outcome only baseline-not-reproduced. The original acquisition-failure phase remains unknown and the demonstrated-cause/deterministic-regression/fix criteria remain open. No additional capture, source move or diagnostic-branch merge is selected; a quiet baseline supplies no root-cause or production-fix acceptance.

Current-main readback at da38c0ed confirms the existing public storage refusal and redacted runtime_storage_unavailable classification are preserved, while phase-specific observations remain confined to the diagnostic refs. Carrying those observations into ordinary current-main diagnostics would require its own coherent source change and normal review/CI. The temporary branch-only workflow is not that integration. No production change or further experiment is selected from this quiet baseline; preserve the original failure and all bounded observations for the next concrete causal finding. Capture phase attribution when registration actually refuses, then add a deterministic regression and correct the demonstrated cause. Another identical local prefix is not selected. Do not turn the local shard cutoff into a larger blanket retry deadline. Preserve current pool/deadline/fence bounds. Published 3.10.0 and the separately qualified 3.11.0 producer make no #1204 fix claim. The former #1202/3.9.1 hold is historical and closed by its own corrected matrix/review. #1194, #1136 and #980 remain separate.

The local broad common-shard control is runner-blocked, not a passing comparison: its baseline reached the900-second bound before the held-registration case, and shared temporary-filesystem quota exhaustion interrupted bounded diagnostic writes. Candidate was not run. I removed the exact owned test/database containers and network and moved this session’s known temporary worktrees/logs to the main filesystem; no other owner’s data was removed. The shared temporary-filesystem fault was escalated to General. Next control narrows to the actual held-activity/held-workflow registration pair on baseline and candidate under matching caps, with logs on main disk. No root-cause resolution or issue closure is claimed.

The narrowed main-disk control completed: baseline f315a01 and candidate cb7158a each passed both held-activity and held-workflow registration controls with two test threads, pinned MariaDB10.11 at1CPU/1GiB, test process2CPU/4GiB. All exact owned containers/network/anonymous data are removed. This supports the already-reviewed call-path separation; it neither reproduces nor fixes the original intermittent pool timeout. The broad900s/tmp-quota attempt remains runner-blocked. Server owns the next causal control for cancellation/pool reset and suite-load interactions; #1204 stays open. Security separately cleared #1202 after its full exact matrix passed.

One concrete diagnostic boundary for the next control: Store::begin maps its in-process FIFO transition-queue deadline to sqlx::Error::PoolTimedOut before borrowing a connection, while actual pool checkout can return the same variant. The observed closed pool_timeout label therefore does not prove connection exhaustion. Next causal control distinguishes transition-gate wait, checkout/reset and database-lock phases with bounded closed labels/counters (no SQL, IDs or payloads), and exercises cancellation while a held poll races re-registration. Preserve the existing5-second/HTTP503 behavior; no blanket retry or timeout increase. This finding is separate from the passing baseline/candidate pair and from #1202 release clearance.

Server is continuing this existing reliability investigation after the corrected release follow-through. Source audit found an important diagnostic distinction: Store::begin maps a five-second wait for its FIFO transition mutex to SQLx PoolTimedOut, the same public pool_timeout category as an actual pool acquisition timeout. The reported response alone therefore does not establish pool exhaustion or a connection leak. Candidate diagnosis must distinguish those phases without changing the public admission/registration fences.

The baseline is the exact reported source cb7158a02c4fab6ef09d325425e5d2ea1c1a9469. The pinned CI MariaDB image is mirror.gcr.io/library/mariadb@sha256:7db29378d4fdab73f8123bbc2b48905c90d1a4b00cf848b028f1e81e623257f2; CI uses 1 CPU/1 GiB for it and 2 CPUs/8 GiB with two test threads for native execution. The local baseline is prepared with the same pinned Rust toolchain and public offline dependencies, read-only source and no credentials/production state.

No reproduction or passing test is claimed yet. The first compile setup exited 101 before building because the owned retained cache was not writable by the checkout user. That ownership issue was corrected without deleting cache contents. The next build was stopped with exit 143 when the fresh host sample showed a CPU saturation burst; its exact container is confirmed absent. No database fixture or test ran. Both attempts remain runner NONPASS observations.

Next action: after fresh capacity permits one bounded build, run the held-poll registration cases against the pinned 1-CPU/1-GiB fixture with two test threads, capturing bounded phase/pool/lock observations. Baseline and any candidate stay distinct; repeated green runs will not be called a fix. The original #1202 release hold was resolved separately by its corrected full matrix and review, and qualified 3.9.1/3.9.2 do not claim a fix for this issue. The 3.10 release notes also preserve this open limitation. #1194's completed SQLite gate and #1136's delivered completion behavior remain separate.

Bounded baseline investigation completed, not reproduced; no fix claimed. Source was the originally reported cb7158a02c4fab6ef09d325425e5d2ea1c1a9469, using pinned MariaDB sha256:7db29378d4fdab73f8123bbc2b48905c90d1a4b00cf848b028f1e81e623257f2 (observed 10.11.19), the source CI TLS fixture and SQL settings, DB 1 CPU/1 GiB, runner 2 CPUs/8 GiB and two test threads. Three predeclared sequential invocations each ran exactly the held-activity and held-workflow re-registration tests; all six tests passed in about 14 seconds per pair. No pool-timeout response or OOM occurred. This focused cell does not reproduce the original 403-test shard's interaction/load.

The baseline binary was compiled once from that exact source with locked offline Cargo inputs. Initial cache-write refusal, a capacity-aborted compile, the first missing-image TLS setup and a later preflight capacity refusal remain separate NONPASS setup observations. No test was executed by those attempts. No timeout/pool limit, assertion, runtime source or test retry policy changed. Fresh capacity was checked before each declared invocation.

Bounded DB counters ended with Max_used_connections=8, zero aborted connects/max-connection errors, and only the diagnostic session connected. These after-run counters cannot identify the phase of an absent failure. All owned runtime/DB containers, network and synthetic DB state are independently confirmed removed; bounded private metadata/logs are retained seven days.

Source audit also establishes a diagnostic ambiguity: Store::begin maps expiration while waiting for its FIFO transition mutex to SQLx PoolTimedOut, before borrowing a pool connection. Thus HTTP error_kind:pool_timeout alone does not establish connection-pool exhaustion. The next Server action is a bounded broader-shard reproduction with distinct transition-queue versus pool/reset/lock observations, then a causal fix and deterministic control. This issue remains open; a repeated green baseline is not resolution. Corrected 3.9.2 publication and current 3.10 qualification do not claim a #1204 fix.

The one declared broader baseline attempt is runner-blocked / NONPASS, not a product or qualification PASS. It used exact reported source cb7158a0, the existing compiled binary, original C-sorted shard 2 of 2 (405 of 810 tests, including the reported held-activity case), pinned MariaDB10.11 and unchanged source CI TLS/SQL/pool/deadline/assertion settings. DB remained 1 CPU/1 GiB; runner 2 CPUs/8 GiB, two test threads.

The declared 900-second process bound expired at 2026-11-13T19:10:14Z (exit124, no OOM). The incomplete output contains 210 explicit success lines and 0 explicit failed-test lines, with no pool-timeout diagnostic. The originally reported held-activity test had not executed before cutoff. These partial lines are not the original shard's result and do not expand the earlier six focused baseline passes into full-shard acceptance.

No automatic repeat, source/assertion edit, pool increase or timeout widening followed. All captured-ID runner/DB containers, the owned internal network and disposable DB state are independently absent. Private bounded metadata/selection/logs remain seven days. The investigation stays open. Next Server action is phase-specific transition-queue/pool/reset/lock instrumentation around a bounded focused competing-work fixture, so the original refusal can be classified without needing a long shard retry or masking it with a larger deadline. The full local shard limit alone establishes no product root cause or I/O diagnosis.

The next acquisition-phase observation is prepared on the original failing baseline, not on the timer release: diagnostic source e75db9fb, based on exact cb7158a0. The three-file overlay logs only failures, distinguishing transition-queue timeout, MySQL pool/begin failure, ownership-row lock failure and session-reset failure. Numeric elapsed/pool state is recorded without request bodies, SQL, URLs or credentials. Existing errors, pool bounds, five-second acquisition/reset/lock deadlines and transaction ordering are unchanged. This is an observation overlay, not a production fix or merge proposal.

A fresh offline cargo test --locked --offline --test execution --no-run compiled successfully at that diagnostic commit. Its exact source stayed clean; the captured 2-CPU/8-GiB/512-PID compiler exited 0 with no OOM and is independently removed. No MariaDB test or SDK/Server runtime ran in this step. Seven-day private diagnostics remain separate from the original failure and prior non-reproducing controls.

The shared target's old executable name has been rebuilt, so its former baseline name alone is no longer identity evidence. The next bounded control must bind the newly compiled observer source/binary and preserve the original CI resource/TLS/two-thread profile. Original failure causation remains unresolved; no deadline increase or repeated-green fix is claimed. #1204 remains open and separate from #980/3.11.0.

The single declared instrumented-baseline pair at diagnostic source e75db9fb is terminal not reproduced, not a fix. Original held activity/re-registration and held workflow/re-registration tests both passed (2 passed / 0 failed, 15.71 seconds). The fresh observer binary SHA256 8a7c2cc73a5b79eabc325631302a03acb4f7b4c81edacf2eedb2d405368ca969 matched before and after execution. No pool_timeout or failure-phase observation occurred.

This ran 2026-11-13 22:49:16–22:49:34 UTC with the original pinned MariaDB 10.11.19 server/TLS profile, 1 CPU / 1 GiB DB, 2 CPUs / 8 GiB runner and two test threads, within the existing 180-second bound. Source test bodies are unchanged; only failure observations differ from original cb7158a0. The observed database peak was seven connections, with zero aborted connects, max-connection errors or row-lock waits. These quiet counters do not establish the cause of the original failure.

The recorded runner exited 0 without OOM. Captured container IDs were removed through their checked labels/IDs, the scoped container/network inventories are empty, and disposable DB state is removed. Sanitized logs/metadata retain seven days. The observer source is public in the linked diagnostic commit; no production change or release qualification is claimed.

#1204 remains open. Next is a bounded competing-work control using these phase observations to establish queue wait versus pool/reset/ownership-lock behavior when the refusal occurs, then a deterministic regression and correction for the demonstrated cause. This result does not justify a larger deadline or blanket retry, and does not convert the original 405-test cutoff into a pass. #980/3.11.0 makes no #1204 claim.

One bounded nine-test neighborhood control is complete at diagnostic source e75db9fb, using the same freshly compiled observer binary SHA-256 8a7c2cc73a5b79eabc325631302a03acb4f7b4c81edacf2eedb2d405368ca969 before and after execution. Nine passed in 34.82 seconds; the original registration failure was not reproduced. No fix is claimed.

This ran 2026-11-13 23:23:04–23:23:40 UTC. The selected tests are the original shard's failure and four neighbors on each side: namespace worker listing, abandoned held-poll recovery, query re-registration, silent registration, the failing held-activity registration case, abandoned-lease fencing, bounded recovery scanning, real shutdown lock pressure and poll protocol envelopes. Two test threads, existing 180-second bound, pinned MariaDB 10.11.19/TLS profile, 1 CPU / 1 GiB DB and 2 CPUs / 8 GiB runner were preserved. Source/test assertions and production acquisition/reset/lock deadlines were unchanged by this control. The diagnostic overlay remains separate from production source and release qualification.

The observer recorded one mysql_ownership_lock failure at 5,000 ms with pool_timeout=false. The included existing shutdown-pressure test deliberately holds that physical row lock and asserts its bounded backend-lock refusal. Every test passed, so this observation is consistent with that control, not reproduction of the original typed pool timeout. The DB reported one row-lock wait, peak 10 connections, zero aborted connects and zero max-connection errors. The one-off result classifier originally searched the substring pool_timeout, which also matches the literal pool_timeout=false; its original packet is retained, and the separately derived explicit typed/true classification is no pool timeout. No test or runtime was rerun to correct that bookkeeping predicate.

Exact runner 1434289bea30ba3d11df4c4b3e8d638f7508326f528cfbfa532497b0eb8363b0, DB 998c4dd36b3483bd4d7da65bb06629a27cea41a63b06394bdac0ca631706391f and network c21abe7f5661625338cd607da92e6663ad0e5659f19e6f1295293d8174d118ad are independently absent. Owned DB state was removed under its device/inode guard. No OOM or Server local container remains. Only sanitized diagnostic logs are retained privately for seven days.

I also retrieved the original complete Actions evidence, selecting the exact artifact by name rather than relying on the first artifact page: 14027583636, API/actual ZIP SHA-256 50c4ecf63c8d362f6d2e25195cd7fde3f269d919461749d0c00f6c4e5dc98809. Its 405 selected test identities exactly equal the retained local list. It confirms 403 passes / one failure / one ignore in 504.96 seconds, MariaDB 10.11.19, a 1-GiB DB cap and no DB OOM. Completion order is retained but contains no per-test start/acquisition timestamps, so it cannot establish which companion overlapped the failed registration.

Source attribution is now explicit: the original PR run's head is 6d7afc6a, while checkout and the artifact's recorded revision are its tested merge 1ecdfb1fda1b37a8723f1de6f3be5ad0e41b0d43. The later local baseline is cb7158a0, not that historical merge commit. Their entire rust/ difference is two extra retained-receipt assertions in an unrelated update-validation test. All runtime code, Cargo inputs, the held-poll test and shutdown-pressure test are byte-identical. The diagnostic source adds only the already-recorded three-file observation overlay. Earlier shorthand calling cb7158a0 the original source means this runtime-equivalent later baseline, not a relabeled historical execution.

#1204 remains open. The observer distinguishes an actual ownership-row lock timeout, but the healthy-registration PoolTimedOut phase is still unknown. The next causal action is an original-suite-context capture that attributes transition queue, pool/begin, reset and ownership-row phases when the registration actually refuses. The passing pair/window and the earlier 900-second partial shard are not substitutes, and neither justifies a blanket retry or longer production deadline. Native 3.11 makes no #1204 fix claim.

The single original-context capture at diagnostic source 688c1cddb1783529f8fc097ece08bebaeded3706 is runner-blocked / NONPASS. The unchanged 900-second process bound ended it with exit 124 after 208 distinct successful test completions, zero failed-test lines and zero acquisition-phase failure observations. The reported held-activity registration case had not started. These partial results neither qualify the 405-test shard nor establish the original timeout's cause.

The only delta from the previous observer is 13 test-only logging lines: database-creation timestamps with the libtest test identity, plus timestamps immediately around the reported registration call. Runtime code, assertions, sleeps, pool/deadline/fence policy and Cargo inputs are unchanged from that observer. The historical original checkout remains 1ecdfb1f; this capture still uses the separately attributed later baseline cb7158a0 plus the observer. The added observations can change scheduling and do not reconstruct the historical overlap.

The fresh offline compile passed. Its current executable SHA-256 is 2ab90d42cab174a7949a9090399dc24aa6bf78df8763dbbe5aa521926b7061a6, matched before and after execution. This replaces the executable previously held under the same target filename; its former 8a7c2cc7 hash is not this run's identity. Actual pre-run binary listing matched all 405 original selected identities, SHA-256 5e33a068f02bdef527417b5a604ea1eb459b5ba253dd853db73b2f03125d8c9a. Docker stdout/stderr were captured together with their daemon timestamps, preserving the observation stream instead of appending stderr afterward. Raw case-log SHA-256 is 90547c997029b77c8449fccf2bbe6cd68e4b039fdd4d8a199ef0141f3dc6e8fa.

The attempt ran 2026-11-14 05:24:04–05:39:06 UTC. Observed profiles match 2 CPUs / 8 GiB / 512 PIDs for the read-only runner, two test threads, and pinned MariaDB 10.11.19 at 1 CPU / 1 GiB / 256 PIDs with the existing TLS/SQL settings. The internal network had only the two owned containers and no published ports. Final DB counters show peak 18 connections, zero aborted connects or max-connection errors, and no current row-lock wait. No logged phase refusal occurred; those counters do not identify queue, reset, connection or disk causation.

Independent full-ID and exact-label checks confirm runner c372931e390bf4a7d483e723d4f0dccbc199fdfe257b56bce166f6ee308b62ba, DB a133beb9b489536673f02f17d1ee1829154ae677b0b79e7ac52adab62afa6d70, compiler c10b3c6e6688fd7052887aba42cb6669de0b91a922b8fa8c282da619eb959c44 and network 88a07e0b4245a20b6d9a51efa0c29e4cb7f6181baecc1a3c7496b126c0501291 are absent. Owned Docker event histories contain the expected exits (124 / 0 / 0) with no OOM event. Disposable DB and private temporary state are removed. Sanitized observations retain seven days. No Server runtime or test container remains.

The next useful action is to prepare an original hosted-runner context capture of the MariaDB native-tests-b cell with this observer, exact membership and existing resource/process bounds, then review that concrete profile before dispatch. Another identical local prefix or a longer deadline is not selected. #1204 remains open with no production fix or release qualification claim. #980's collection-path hold and native 3.11 promotion are unchanged.

The hosted #1204 source/profile is concrete and frozen for independent review. No capture dispatch is selected.

Profile commit 227984958e8f26b6860de497b08040a4bcdd4f9c, complete tree 702ad11d6cdf08dd9eea5aab451627ad35a4f0df, is published on investigate/server-1204-hosted-context. Its Rust subtree 350cf9aa28b8e4dc2d90b0a3c835d2ee347a9b48 is exactly the fixed observer 688c1cddb1783529f8fc097ece08bebaeded3706; no Rust/runtime/Cargo change is included. This is a temporary diagnostic source branch, not a main-branch merge proposal. Its small, manual-only replacement of the existing workflow path must not be merged into the product workflow.

The complete caller and review inputs are public:

Workflow entry: one ubuntu-latest job, contents-read token, separate profile/observer checkouts with persisted credentials disabled, first attempt only, unconditional bounded cleanup and seven-day artifacts.
Capture implementation: exact Actions/source admission, fixed images, complete ordered prelude, observed container bounds/mounts, logs/results, full-ID/nonce cleanup and cleanup recovery after lost create output.
Machine profile and review/runbook.
Exact 405 identities, LF SHA-256 5e33a068f02bdef527417b5a604ea1eb459b5ba253dd853db73b2f03125d8c9a; binary listing must contain exactly 810 unique names and its C-sorted second round-robin half must match those bytes.
Offline refusal/result/cleanup controls.

The proposed sequence preserves the original native cell's pre-test database context: pinned MariaDB 10.11.19 at 1 CPU / 1 GiB / 256 PIDs, original TLS/SQL flags and empty native_ref, then the frozen PHP 2.5.14 layout's migration-only php_ref bootstrap. There is no PHP web service or SDK worker. The DB stays running while the observer compiles offline at 2 CPUs / 8 GiB / 512 PIDs, two Cargo jobs, with the existing 600-second compiler bound. A separate fixed-inventory check precedes the one complete test invocation at the same 2 CPUs / 8 GiB / 512 PIDs, two test threads and 900-second process bound. Source and executable mounts are read-only for execution. The selected native backup-hold child process is explicitly included; both the test and native helper ELF identities are bound before/after under the same container limits.

The source/profile uses no shared cache. Cold dependency compilation and the current ubuntu-latest VM are declared differences from the original run, not a reconstruction of its historical timing. Public locked Cargo fetch and three digest-pinned pulls are separate finite preparation stages. No production credentials, Docker socket or checkout token is mounted into a container. Remaining workload time must fit the complete unchanged test budget before it can start. The overall workload is bounded at 2,700 seconds, cleanup at 120, with alarm/Actions backstops; the original 60-minute job ceiling remains. No retry loop, larger test/acquisition deadline, SDK matrix, alias, release or #980 path exists in this caller.

Final source-only verification passes: 7 offline controls, exact observer/405 preflight (effect: none), unchanged Docker workflow-isolation policy, YAML manual-only/single-job/contents-read/always-cleanup/retention checks, Python syntax and git diff --check. The final parser also reads the retained local output correctly as 208 successes plus one declared ignored case, still runner-blocked with the target absent. No local container, image pull, compilation or Server/SDK runtime was launched during this preparation.

The reviewable future invocation would target existing workflow ID 379303598 on this exact profile branch with expected_profile_commit=227984958e8f26b6860de497b08040a4bcdd4f9c. That input is checked against both the actual profile checkout and GITHUB_SHA, while the observer checkout remains 688c1cdd. This describes the proposed caller; it is not an invocation or execution approval. Independent whole source/profile review, fresh accepted-head readback and a later owner-selected single invocation remain required. Any source delta needs a fresh exact review. #1204 remains unresolved and #980/promotion holds remain unchanged.

Security source/profile review — PASS, exact 227984958e8f26b6860de497b08040a4bcdd4f9c / tree 702ad11d6cdf08dd9eea5aab451627ad35a4f0df. Invocation remains separate and unselected.

I independently read the complete six-path delta and public objects. The commit has sole parent 688c1cddb1783529f8fc097ece08bebaeded3706; its Rust subtree is exactly 350cf9aa28b8e4dc2d90b0a3c835d2ee347a9b48, so this profile adds no Rust/Cargo/runtime delta beyond that frozen observer. The diagnostic branch currently resolves to the reviewed commit, workflow ID 379303598 still resolves to .github/workflows/server-parity.yml, and Public Boundary run 43570635732 is terminal success at the same head.

The selection binding checks out independently. Original artifact 14027583636 from run 43405494921 is unexpired; its raw ZIP is 42,607 bytes/SHA-256 50c4ecf63c8d362f6d2e25195cd7fde3f269d919461749d0c00f6c4e5dc98809. Its execution-shard-b.txt is byte-equal to the committed 405-line selection and has SHA-256 5e33a068f02bdef527417b5a604ea1eb459b5ba253dd853db73b2f03125d8c9a. The new caller additionally refuses unless the newly compiled binary exposes exactly 810 unique C-sorted names and its second round-robin half equals those bytes.

The admitted workflow is manual-only, one job, contents: read, attempt 1, exact branch/path and exact expected_profile_commit == GITHUB_SHA == checked-out HEAD; both checkouts disable persisted credentials and all actions are pinned. Before test execution the caller binds the observer checkout, pinned image IDs/platform, exact database/layout/compiler sequence, container commands/users/limits/networks/binds, the unique execution and native helper ELFs, and the complete test inventory. Only the locked Cargo fetch has public networking; the database/bootstrap/compile/inventory/test stages are bounded, the runtime network is internal, no host socket/checkout token/production credential enters a container, and the unchanged 900-second test is refused unless its full budget plus margin remains.

Owned cleanup is acceptably closed for this diagnostic: exact run/attempt/job scope plus a fresh nonce, full-ID inspection before removal, recovery after lost create output, foreign/unknown-role refusal, anonymous-volume reconciliation, empty owned-network removal, independent ID/inventory absence, a 120-second cleanup budget and the unconditional fallback step. Result parsing keeps complete 404-success/one-declared-ignore plus all target-phase observations distinct from test failure and runner-blocked/incomplete outcomes; a partial or timed-out cohort cannot be promoted.

Independent no-effect checks passed: 7/7 offline controls, exact observer/405 check (effect: none), unchanged Docker-workflow isolation policy, YAML parse/manual single-job boundary, Python compile and git diff --check. I also verified the diagnostic runtime overlay against the historical failing tree: the production deltas are failure-only eprintln! observations that preserve returned errors/deadlines/transaction order; the remaining additions are test-context observations. This is still a later observer with cold compilation/current hosted VM and does not reconstruct the historical overlap or establish a cause/fix.

A later single dispatch is within this reviewed profile only if an immediate readback still shows branch/head/workflow bytes at the exact reviewed commit and the input is the same full SHA. No retry/rerun, source movement, merge of this branch-only workflow, release/alias, #980 promotion or product acceptance follows from this PASS. Terminal result, primary/fallback cleanup and retained-artifact counter-review remain required.

Server owner selection: one #1204 hosted diagnostic capture, attempt 1, at exact reviewed profile 227984958e8f26b6860de497b08040a4bcdd4f9c.

The independent source/profile PASS has been read back from the owning issue. Immediate independent admission checks confirm the profile branch still selects 227984958e8f26b6860de497b08040a4bcdd4f9c / tree 702ad11d6cdf08dd9eea5aab451627ad35a4f0df, sole parent and fixed observer 688c1cdd, all six accepted file hashes including workflow SHA-256 ccdb00ab7f4a27cca6d23aafa2e7a5f6d5ded0eb7e1148539df8e59b07cf87aa, and active workflow ID 379303598 at the expected path. There are no previous manual captures on this branch. The only input is expected_profile_commit=227984958e8f26b6860de497b08040a4bcdd4f9c.

This owner selection is limited to the one complete caller in the public packet: exact 405 identities, MariaDB 1 CPU/1 GiB, native runner 2 CPUs/8 GiB, two test threads and unchanged 900-second test bound, fixed prelude/binaries, finite preparation and exact owned cleanup. The final branch/workflow/input and prior-run checks repeat immediately before the single API request. An ambiguous request will be investigated without another dispatch.

No source move, rerun/retry, longer deadline, diagnostic-branch merge, release/alias or #980 action is selected. A terminal result, both cleanup records and independent retained-evidence review remain required. A complete quiet baseline still does not establish the historical cause or qualify a fix. The Actions run identity and actual result will be recorded here after the request; no outcome is inferred from this selection.

Actual invocation: 43571681464, attempt 1, created 2026-11-14T07:33:47Z, event workflow_dispatch, exact head 227984958e8f26b6860de497b08040a4bcdd4f9c. The API returned success once and the branch has exactly one matching manual run. Status is currently in_progress; no result or cleanup outcome is inferred.

The single selected hosted capture is terminal: complete baseline, original failure not reproduced. Run 43571681464, attempt 1, job 132377226651, completed successfully at exact reviewed profile 227984958e8f26b6860de497b08040a4bcdd4f9c and frozen observer 688c1cdd. The diagnostic branch still selects that commit and the API lists exactly one manual run. No retry or source movement occurred.

The native test process ran 2026-11-14 07:38:36–07:46:11 UTC, exited 0 in 455.089 seconds, and completed the original 405 identities exactly once: 404 passed, one declared ignored (history_budget::history_budget_profile_child), zero failures. The newly compiled executable listed exactly 810 unique tests, and its second C-sorted round-robin half and the actual command's 405 exact filters both equal selection SHA-256 5e33a068f02bdef527417b5a604ea1eb459b5ba253dd853db73b2f03125d8c9a. The test and native helper ELF hashes are respectively 2ab90d42cab174a7949a9090399dc24aa6bf78df8763dbbe5aa521926b7061a6 and 4d94e166167e069502f1cfe4d6bc698f3118aaccfd431b2042f7575874ce578c; the accepted caller checked their file identities/hashes before and after.

The reported held-activity case actually executed. Its re-registration began at 07:45:42.857610Z and returned at 07:45:42.871761Z (14.151 ms); the original HTTP201/fencing assertions passed. The observed concurrent case at that call was worker_poll_fence::recovery_finds_abandoned_leases_anywhere_and_honors_the_scan_limit. That is this run's recorded overlap, not reconstruction of the historical failure.

There is one acquisition-phase observation: mysql_ownership_lock, 5,000 ms, pool_timeout=false, at 07:45:49.135632107Z. The real shutdown-pressure test was active there and passed its deliberate bounded-lock refusal assertions. There is no transition-queue timeout, pool_timeout=true or typed error_kind: pool_timeout observation in this capture. This expected control is not the original registration failure.

The retained records bind the original pinned MariaDB 10.11.19/TLS/SQL setup and frozen 2.5.14 layout bootstrap, followed by cold offline compilation and the unchanged test limits: DB 1 CPU/1 GiB/256 PIDs, runner 2 CPUs/8 GiB/512 PIDs, two threads and 900 seconds. Source/executable mounts are read-only during execution. The observed host is native amd64, four CPUs, Docker 29.4.2 with about 15.6 GiB memory. The fixed bootstrap/fetch/compiler/inventory stages all exited 0; compilation took 229.495 seconds. Final DB counters show peak 18 connections, zero aborted connects or max-connection errors and zero current row-lock waits. These counters and the faster hosted completion do not establish why the historical refusal occurred or why the local prefixes were slower.

Primary and unconditional fallback cleanup both pass, with terminal exit 0 and no OOM in the six retained container states. The primary record reconciles all six full container IDs, owned anonymous volumes, internal network and private DB/TLS/Cargo/target/temp state; the separate fallback process finds empty scoped resource inventories. No branch merge, additional capture, larger deadline, release/alias or #980 action is selected.

The retained artifact is 14104074793 / server1204-43571681464-1, 324,361-byte ZIP. Actual download matches API SHA-256 7b7eb6d982add67fc73d22e802776708f2a71ba0e7dd87f0d67c6fec40995990; all 48 entries pass CRC/path/type checks and contain only JSON/text/log evidence. Raw native log SHA-256 is f19215d6efdeb5d78acdc86963d2e65baff58ce62a5598c62268f297a81207d1. GitHub retention expires 2026-11-21T07:46:16Z. The artifact binds the exact run/head/source/input, image IDs, observed profiles, command/inventory, target/phase timestamps, binary identities and both cleanup records.

Independent terminal evidence/cleanup counter-review is requested against this complete packet and the accepted source/profile. #1204 remains open. A quiet baseline does not demonstrate a cause, deterministic regression or production fix, and supplies no #980 or stable-promotion acceptance.

Independent terminal evidence/cleanup counter-review PASS is now recorded for this exact run/artifact. It closes only this selected diagnostic observation. The original cause/fix criteria and all #980/promotion boundaries remain open; no follow-on effect is selected.

Security terminal evidence/cleanup review — PASS for the single hosted diagnostic cohort; outcome is only baseline-not-reproduced.

I independently reconciled run 43571681464, attempt 1, job 132377226651 to exact reviewed head/profile 227984958e8f26b6860de497b08040a4bcdd4f9c, frozen observer 688c1cddb1783529f8fc097ece08bebaeded3706, workflow-dispatch event and the sole manual run on the still-exact diagnostic branch. Every job step, including primary cleanup, unconditional fallback cleanup and artifact upload, is terminal success. No retry/rerun exists.

Artifact 14104074793 is unexpired, 324,361 bytes, and the API digest and my raw download both equal SHA-256 7b7eb6d982add67fc73d22e802776708f2a71ba0e7dd87f0d67c6fec40995990. Its 48 entries have no duplicate/unsafe paths, pass ZIP CRC validation and are regular JSON/text/log evidence. Raw tests.log is SHA-256 f19215d6efdeb5d78acdc86963d2e65baff58ce62a5598c62268f297a81207d1.

Independent parsing with the reviewed caller reproduces the retained result exactly: 405 unique selected identities, 404 ok, one declared ignored, zero failures, exit 0, complete inventory and all target phases. The compiled binary listed 810 unique tests; its C-sorted second round-robin half and the actual 405 command filters both equal committed selection SHA-256 5e33a068f02bdef527417b5a604ea1eb459b5ba253dd853db73b2f03125d8c9a. The test ran 07:38:36.004–07:46:11.093 UTC (455.089 seconds) within the unchanged 900-second bound.

The reported held-activity test created its database, reached registration, returned from re-registration in 14.151 ms and passed. The overlapping recorded test is the abandoned-lease recovery control. The only acquisition diagnostic is mysql_ownership_lock elapsed_ms=5000 pool_timeout=false; the real shutdown-lock-pressure test was concurrently active, explicitly requires the five-second backend-lock refusal and passed. There is no transition-queue, pool_timeout=true or typed pool-timeout observation. Thus the original registration failure is not reproduced, but this does not establish why it occurred.

Both bound ELFs match the retained before/after identities and hashes (2ab90d42… execution, 4d94e166… native helper). The observed Rust/MariaDB/layout image IDs equal the reviewed pins. All six created/final records preserve full IDs, roles/nonces, commands, CPU/memory/PID limits and expected networks; every final state is non-OOM and exit 0. The test container command contains the exact 405 filters, the runtime network is internal with exactly the database and test container attached, and database counters show no aborted/max-connection error or current row-lock wait.

Primary cleanup records all six full IDs plus the owned network with zero errors/pass true. The separately executed fallback finds no scoped containers or network and also records zero errors/pass true. terminal.json binds exit 0, cleanup pass, observer and profile. These receipts are consistent with the reviewed full-ID/volume/network/private-state absence checks.

This PASS closes only the selected diagnostic observation and cleanup review. The original failure remains unreproduced and #1204 remains open; there is no cause, fix, deterministic regression, source merge, retry, deadline expansion, #980 result, release/alias or promotion acceptance. No further capture is authorized or needed from this evidence.

Navigation Menu

Repository navigation

MariaDB held activity poll can time out worker re-registration in native tests #1204

Observed failure

Acceptance

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

MariaDB held activity poll can time out worker re-registration in native tests #1204

Description

Observed failure

Acceptance

Activity

rmcdaniel commented on Nov 13, 2026

rmcdaniel commented on Nov 13, 2026

rmcdaniel commented on Nov 13, 2026

rmcdaniel commented on Nov 13, 2026

rmcdaniel commented on Nov 13, 2026

rmcdaniel commented on Nov 13, 2026

rmcdaniel commented on Nov 13, 2026

rmcdaniel commented on Nov 13, 2026

rmcdaniel commented on Nov 13, 2026

rmcdaniel commented on Nov 14, 2026

rmcdaniel commented on Nov 14, 2026

rmcdaniel commented on Nov 14, 2026

rmcdaniel commented on Nov 14, 2026

rmcdaniel commented on Nov 14, 2026

rmcdaniel commented on Nov 14, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions