TESTING - NEW FEATURE - Test thread invariance of the parallel BVP path - #362
Open
d-burg wants to merge 1 commit into
Open
TESTING - NEW FEATURE - Test thread invariance of the parallel BVP path#362d-burg wants to merge 1 commit into
d-burg wants to merge 1 commit into
Conversation
ForceFreeStatesControl documents that the parallel FM/BVP path produces bit-identical
Delta' across thread counts, but nothing tested it by varying threads, and the example
decks disagreed about whether to rely on it: the SLAYER deck pins parallel_threads = 1
"to keep the regression Delta' (and hence gamma) reproducible", while the DIII-D-like
ideal deck — whose Delta' the regression harness tracks — runs parallel_threads = 2.
A golden Delta' has to be a property of the physics rather than of the machine it was
measured on, so this settles the question by measurement.
Measured on the DIII-D-like deck at parallel_threads = 1, 2, 4 and 16: the Delta' matrix
diagonal and et[1] are bit-identical throughout. The 16-thread configuration is the
informative one — it lifts 4*effective_threads above the min_bvp_intervals floor and so
produces a genuinely different decomposition (64 chunks vs 53, with different boundaries),
which reassociates the propagator products without moving the result. Invariance therefore
holds across decomposition, not merely across scheduling.
- test/runtests_thread_invariance.jl compares parallel_threads 1 vs 2 on the Solovev
and DIII-D-like decks, asserting exact equality rather than a tolerance: the code
claims bit-identity, and a tolerance would mask the reassociation the test exists to
catch. It also asserts the chunk boundaries are unchanged at these caps, so that a
future decomposition change surfaces as a failure instead of silently turning the
comparison into a stronger claim than intended. Skipped when the session has one
thread, where effective_threads collapses to 1 and every comparison is vacuous.
- The test workflow gains a multi-threaded leg (JULIA_NUM_THREADS = 4). The suite had
only ever run single-threaded, so the parallel paths were exercised solely in their
degenerate form. Existing job names are preserved byte-identically because branch
protection names them as required checks.
- balance_integration_chunks' docstring gave target_n as
max(2*msing + 3, 4*Threads.nthreads()), omitting both the parallel_threads cap and the
min_bvp_intervals term that actually dominates. runtests_parallel_integration.jl
mirrored the same stale formula and would fail on a machine with more threads than the
cap; it passed only because CI is single-threaded.
- CLAUDE.md's single-test-file invocation could not work (runtests.jl passes ARGS to
include, which resolves relative to test/), and two of the listed files do not exist.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR strengthens the ForceFreeStates parallel FM/BVP verification pipeline by adding explicit thread-invariance tests, updating CI to exercise multi-threaded execution, and aligning documentation/tests with the actual chunk-balancing formula used for the parallel BVP path.
Changes:
- Add
runtests_thread_invariance.jlto assert parallel BVP results are bit-identical betweenparallel_threads=1and2when running multi-threaded. - Add a multi-threaded CI matrix leg (
JULIA_NUM_THREADS=4) without renaming the existing branch-protection-required job contexts. - Update the
balance_integration_chunksdocstring and the parallel-integration test to reflect the effective thread cap logic.
Note: Project policy still requires third-party human review before merge.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
test/runtests.jl |
Adds the new thread-invariance test file to the default test suite. |
test/runtests_thread_invariance.jl |
New tests to verify thread invariance of Δ′/et across parallel_threads caps. |
test/runtests_parallel_integration.jl |
Fixes stale formula in the test to use effective thread cap, matching implementation intent. |
src/ForceFreeStates/EulerLagrange.jl |
Updates balance_integration_chunks docstring to describe current targeting behavior (but needs a small alignment fix). |
CLAUDE.md |
Corrects documented test invocation and updates the list of example test files. |
.github/workflows/test.yaml |
Adds a multi-threaded test leg by setting JULIA_NUM_THREADS via a matrix include. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+103
to
+106
| for j in 1:serial.msing | ||
| @test threaded.dpm[j, j] === serial.dpm[j, j] | ||
| end | ||
| @test threaded.dpm == serial.dpm |
Comment on lines
+80
to
+82
| if Threads.nthreads() < 2 | ||
| @info "Thread-invariance tests skipped: effective_threads collapses to 1 in a single-threaded session. Run with `julia -t 4` (CI covers this in its multi-threaded matrix leg)." | ||
| @test true |
Comment on lines
+70
to
+71
| `max(2*msing + 3, 4*effective_threads, 8*(msing + 1) + msing)`, where | ||
| `effective_threads = min(Threads.nthreads(), ctrl.parallel_threads)`. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
ForceFreeStatesControldocuments that "the parallel path produces bit-identical Δ′ across thread counts", but nothing tested that by varying threads — and the example decks disagree about whether to rely on it:parallel_threadsDIIID-like_SLAYER_exampleDIIID-like_ideal_exampleSolovev_ideal_example,DIIID-like_gal_resistive_exampleTwo decks, two incompatible beliefs about whether threaded BVP Δ′ is reproducible. Since converged golden Δ′ values have to be a property of the physics rather than of the machine they were measured on, this settles it by measurement instead of by assertion.
Additionally,
test.yamlsets noJULIA_NUM_THREADS, so the entire suite has only ever run at Julia's default of one thread —effective_threads = min(Threads.nthreads(), parallel_threads)collapses to 1 there, meaning every parallel path has been exercised solely in its degenerate serial form.Measurement
DIII-D-like ideal deck, one session per configuration, only the BVP thread cap varied:
parallel_threadsdpm[1,1]7.703609465590473 -0.00034869054124432364im7.703609465590473 -0.00034869054124432364im7.703609465590473 -0.00034869054124432364imjulia -t 16)7.703609465590473 -0.00034869054124432364imAll five diagonal elements and
et[1]match to the last bit in every configuration.The 16-thread row is the informative one.
balance_integration_chunkstargetsmax(2·msing+3, 4·effective_threads, 8·(msing+1)+msing); withmsing = 5themin_bvp_intervalsfloor is 53, so caps of 1/2/4 all yield the same decomposition and only scheduling varies. At a cap of 16,4·effective_threads = 64takes over and the decomposition genuinely changes — verified, not assumed: the chunk-boundary hashes differ (90b7b3bf…vsdabc83e3…) and the mid-chunk interval moves from0.7704..0.7808to0.8922..0.8926. That reassociates the propagator products, and Δ′ still does not move.So invariance holds across decomposition, not merely across scheduling. The docstring's claim is correct.
What this adds
test/runtests_thread_invariance.jl— comparesparallel_threads1 vs 2 on the Solovev and DIII-D-like decks. 11 tests, ~72 s at 4 threads.===, notisapprox. The code claims bit-identity; a tolerance would mask exactly the reassociation the test exists to catch.effective_threadscollapses to 1 there and every comparison is trivially true.A multi-threaded CI leg (
JULIA_NUM_THREADS: 4, via a matrixinclude). Existing job names are preserved byte-identically and there is a comment saying why: branch protection namesruntests 1.11 - ubuntu-latestandruntests 1.x - ubuntu-latestas required checks, so renaming them would leave every PR waiting on a status that never reports — the failure mode in #360. The new leg carries a distinct name and is purely additive.Two stale-formula fixes.
balance_integration_chunks' docstring gavetarget_nasmax(2*msing + 3, 4 * Threads.nthreads()), omitting both theparallel_threadscap and themin_bvp_intervalsterm that actually dominates.runtests_parallel_integration.jl:139mirrored the same stale formula with the raw thread count, so it would fail on any machine with more threads than the cap — it passes today only because CI is single-threaded.CLAUDE.md — the documented single-file invocation cannot work (
runtests.jlpassesARGStoinclude, which resolves relative totest/, producingtest/test/…), and two of the listed test files do not exist. Corrected, and the multi-threaded invocation documented.Not included, deliberately
The decomposition axis is not in CI. Reaching it needs
effective_threads > (9·msing+8)/4— about 14 threads for this deck — which would mean heavily oversubscribing a 4-core runner and slowing the whole suite. It belongs in the nightly harness on hardware with real thread counts, and will be wired there with the harness CI work.No change to the SLAYER deck's
parallel_threads = 1. Its floor arithmetic (msing = 3→ 35) says the pin is unnecessary, but that deck has a different code path to γ and has not been measured. Proposing the change on analogy alone is exactly the reasoning this PR exists to replace. Worth a follow-up measurement.Verification
runtests_thread_invariance.jl: 11/11 pass,julia -t 4, 72.5 s.runtests_parallel_integration.jl: 106/106 pass atjulia -t 4after theeffective_threadsfix.src/change is docstring-only; no executable code in the package is modified.🤖 Generated with Claude Code