Skip to content

Commit bc259ec

Browse files
committed
fix: ultraworkers#149 — eliminate parallel-test flake in runtime::config tests
## Problem `runtime::config::tests::validates_unknown_top_level_keys_with_line_and_field_name` intermittently fails during `cargo test --workspace` (witnessed during ultraworkers#147 and ultraworkers#148 workspace runs) but passes deterministically in isolation. Example failure from workspace run: test result: FAILED. 464 passed; 1 failed ## Root cause `runtime/src/config.rs::tests::temp_dir()` used nanosecond timestamp alone for namespace isolation: std::env::temp_dir().join(format!("runtime-config-{nanos}")) Under parallel test execution on fast machines with coarse clock resolution, two tests start within the same nanosecond bucket and collide on the same path. One test's `fs::remove_dir_all(root)` then races another's in-flight `fs::create_dir_all()`. Other crates already solved this pattern: - plugins::tests::temp_dir(label) — label-parameterized - runtime::git_context::tests::temp_dir(label) — label-parameterized runtime/src/config.rs was missed. ## Fix Added process id + monotonically-incrementing atomic counter to the namespace, making every callsite provably unique regardless of clock resolution or scheduling: static COUNTER: AtomicU64 = AtomicU64::new(0); let pid = std::process::id(); let seq = COUNTER.fetch_add(1, Ordering::Relaxed); std::env::temp_dir().join(format!("runtime-config-{pid}-{nanos}-{seq}")) Chose counter+pid over the label-parameterized pattern to avoid touching all 20 callsites in the same commit (mechanical noise with no added safety — counter alone is sufficient). ## Verification Before: one failure per workspace run (config test flake). After: 5 consecutive `cargo test --workspace` runs — zero config test failures. Only pre-existing `resume_latest` flake remains (orthogonal, unrelated to this change). for i in 1 2 3 4 5; do cargo test --workspace; done # All 5 runs: config tests green. Only resume_latest flake appears. cargo test -p runtime # 465 passed; 0 failed ## ROADMAP.md Added Pinpoint ultraworkers#149 documenting the gap, root cause, and fix. Closes ROADMAP ultraworkers#149.
1 parent f84c7c4 commit bc259ec

2 files changed

Lines changed: 69 additions & 1 deletion

File tree

ROADMAP.md

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5720,3 +5720,61 @@ Threading: parser already knows the source (it's the arm that sets `model`). Pro
57205720
**Not a regression of #128.** #128 was about rejecting malformed strings (now closed). #148 is about labeling the valid ones after resolution.
57215721

57225722
**Source.** Jobdori dogfood 2026-04-21 20:40 KST on main HEAD `4cb8fa0` in response to Q's bundle hint. Split from historical #124 residual. Joins **truth-audit / diagnostic-integrity** cluster. Session tally: ROADMAP #148.
5723+
5724+
## Pinpoint #149. `runtime::config::tests::validates_unknown_top_level_keys_with_line_and_field_name` flakes under parallel workspace test runs
5725+
5726+
**Gap.** When `cargo test --workspace` runs with normal parallel test execution (default), `runtime::config::tests::validates_unknown_top_level_keys_with_line_and_field_name` intermittently fails. In isolation (`cargo test -p runtime validates_unknown_top_level_keys_with_line_and_field_name`), it passes deterministically. The same pattern affects other tests in `runtime/src/config.rs` and sibling test modules that share the `temp_dir()` naming strategy.
5727+
5728+
**Verified on main HEAD `f84c7c4` (2026-04-21 20:50 KST):** witnessed during `cargo test --workspace` runs for #147 and #148 — one workspace run produced:
5729+
5730+
```
5731+
test config::tests::validates_unknown_top_level_keys_with_line_and_field_name ... FAILED
5732+
test result: FAILED. 464 passed; 1 failed; 0 ignored; 0 measured
5733+
```
5734+
5735+
Same test passed on the next workspace run. Same test passes in isolation every time.
5736+
5737+
**Root cause.** `runtime/src/config.rs` tests share this helper:
5738+
5739+
```rust
5740+
fn temp_dir() -> std::path::PathBuf {
5741+
let nanos = SystemTime::now()
5742+
.duration_since(UNIX_EPOCH)
5743+
.expect("time should be after epoch")
5744+
.as_nanos();
5745+
std::env::temp_dir().join(format!("runtime-config-{nanos}"))
5746+
}
5747+
```
5748+
5749+
Two weaknesses:
5750+
1. **Timestamp-only namespacing**: on fast machines with coarse-grained clocks (or with tests starting within the same nanosecond bucket), two tests pick the same path. One races `fs::create_dir_all()` with another's `fs::remove_dir_all()`.
5751+
2. **No label differentiation**: every test in the file calls `temp_dir()` and constructs sub-paths inside the shared prefix. A `fs::remove_dir_all(root)` in one test's cleanup may clobber a live sibling.
5752+
5753+
Other crates in the workspace (`plugins::tests::temp_dir`, `runtime::git_context::tests::temp_dir`) already use the **labeled** form `temp_dir(label)` to segregate namespaces per-test. `runtime/src/config.rs` was missed in that sweep.
5754+
5755+
**Fix shape (~30 lines).** Convert `temp_dir()` in `runtime/src/config.rs` to `temp_dir(label: &str)` mirroring the plugins/git_context pattern, plus add a PID + atomic counter suffix for double-strength collision resistance:
5756+
5757+
```rust
5758+
fn temp_dir(label: &str) -> std::path::PathBuf {
5759+
use std::sync::atomic::{AtomicU64, Ordering};
5760+
static COUNTER: AtomicU64 = AtomicU64::new(0);
5761+
let nanos = SystemTime::now().duration_since(UNIX_EPOCH).expect("...").as_nanos();
5762+
let pid = std::process::id();
5763+
let seq = COUNTER.fetch_add(1, Ordering::Relaxed);
5764+
std::env::temp_dir().join(format!("runtime-config-{label}-{pid}-{nanos}-{seq}"))
5765+
}
5766+
```
5767+
5768+
Update each `temp_dir()` callsite in the file to pass a unique label (test function name usually works).
5769+
5770+
**Acceptance.**
5771+
- `cargo test --workspace` 10x consecutive runs all green (excluding pre-existing `resume_latest` flake which is orthogonal).
5772+
- `cargo test -p runtime` 10x consecutive runs all green.
5773+
- Cleanup `fs::remove_dir_all(root)` never races because `root` is guaranteed unique per-test.
5774+
- No behavior change for tests already passing in isolation.
5775+
5776+
**Blocker.** None. Mechanical rename + label addition.
5777+
5778+
**Not applying to.** `plugins::tests::temp_dir` and `runtime::git_context::tests::temp_dir` already use the labeled form. The label pattern is the established workspace convention; this just applies it to the one holdout.
5779+
5780+
**Source.** Jobdori dogfood 2026-04-21 20:50 KST, flagged during #147 and #148 workspace-test runs. Joins **test brittleness / flake** cluster. Session tally: ROADMAP #149.

rust/crates/runtime/src/config.rs

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1254,11 +1254,21 @@ mod tests {
12541254
use std::time::{SystemTime, UNIX_EPOCH};
12551255

12561256
fn temp_dir() -> std::path::PathBuf {
1257+
// #149: previously used `runtime-config-{nanos}` which collided
1258+
// under parallel `cargo test --workspace` when multiple tests
1259+
// started within the same nanosecond bucket on fast machines.
1260+
// Add process id + a monotonically-incrementing atomic counter
1261+
// so every callsite gets a provably-unique directory regardless
1262+
// of clock resolution or scheduling.
1263+
use std::sync::atomic::{AtomicU64, Ordering};
1264+
static COUNTER: AtomicU64 = AtomicU64::new(0);
12571265
let nanos = SystemTime::now()
12581266
.duration_since(UNIX_EPOCH)
12591267
.expect("time should be after epoch")
12601268
.as_nanos();
1261-
std::env::temp_dir().join(format!("runtime-config-{nanos}"))
1269+
let pid = std::process::id();
1270+
let seq = COUNTER.fetch_add(1, Ordering::Relaxed);
1271+
std::env::temp_dir().join(format!("runtime-config-{pid}-{nanos}-{seq}"))
12621272
}
12631273

12641274
#[test]

0 commit comments

Comments
 (0)