Skip to content

TESTING - NEW FEATURE - Test thread invariance of the parallel BVP path - #362

Open
d-burg wants to merge 1 commit into
developfrom
feature/thread-invariance
Open

TESTING - NEW FEATURE - Test thread invariance of the parallel BVP path#362
d-burg wants to merge 1 commit into
developfrom
feature/thread-invariance

Conversation

@d-burg

@d-burg d-burg commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

Why

ForceFreeStatesControl documents that "the parallel path produces bit-identical Δ′ across thread counts", but nothing tested that by varying threads — and the example decks disagree about whether to rely on it:

Deck parallel_threads Comment in the deck
DIIID-like_SLAYER_example 1 "serial/bit-deterministic BVP — keeps the regression Δ' (and hence γ) reproducible"
DIIID-like_ideal_example 2 — (and this is the deck whose Δ′ the harness pins)
Solovev_ideal_example, DIIID-like_gal_resistive_example 2

Two decks, two incompatible beliefs about whether threaded BVP Δ′ is reproducible. Since converged golden Δ′ values have to be a property of the physics rather than of the machine they were measured on, this settles it by measurement instead of by assertion.

Additionally, test.yaml sets no JULIA_NUM_THREADS, so the entire suite has only ever run at Julia's default of one thread — effective_threads = min(Threads.nthreads(), parallel_threads) collapses to 1 there, meaning every parallel path has been exercised solely in its degenerate serial form.

Measurement

DIII-D-like ideal deck, one session per configuration, only the BVP thread cap varied:

parallel_threads chunks dpm[1,1] Δ′ bit-identical to serial
1 53 7.703609465590473 -0.00034869054124432364im baseline
2 53 7.703609465590473 -0.00034869054124432364im yes
4 53 7.703609465590473 -0.00034869054124432364im yes
16 (julia -t 16) 64 7.703609465590473 -0.00034869054124432364im yes

All five diagonal elements and et[1] match to the last bit in every configuration.

The 16-thread row is the informative one. balance_integration_chunks targets max(2·msing+3, 4·effective_threads, 8·(msing+1)+msing); with msing = 5 the min_bvp_intervals floor is 53, so caps of 1/2/4 all yield the same decomposition and only scheduling varies. At a cap of 16, 4·effective_threads = 64 takes over and the decomposition genuinely changes — verified, not assumed: the chunk-boundary hashes differ (90b7b3bf… vs dabc83e3…) and the mid-chunk interval moves from 0.7704..0.7808 to 0.8922..0.8926. That reassociates the propagator products, and Δ′ still does not move.

So invariance holds across decomposition, not merely across scheduling. The docstring's claim is correct.

What this adds

test/runtests_thread_invariance.jl — compares parallel_threads 1 vs 2 on the Solovev and DIII-D-like decks. 11 tests, ~72 s at 4 threads.

  • Asserts ===, not isapprox. The code claims bit-identity; a tolerance would mask exactly the reassociation the test exists to catch.
  • Skips when the session has one thread rather than passing vacuously — effective_threads collapses to 1 there and every comparison is trivially true.
  • Also asserts the chunk boundaries are unchanged at these caps. That reads backwards, so the reasoning is in the file: at caps 1 and 2 the floor fixes the count, so only scheduling varies. If that assertion ever fails, the decomposition moved and the Δ′ comparison silently became a stronger claim than this testset intends — better surfaced as a failure than left unnoticed.

A multi-threaded CI leg (JULIA_NUM_THREADS: 4, via a matrix include). Existing job names are preserved byte-identically and there is a comment saying why: branch protection names runtests 1.11 - ubuntu-latest and runtests 1.x - ubuntu-latest as required checks, so renaming them would leave every PR waiting on a status that never reports — the failure mode in #360. The new leg carries a distinct name and is purely additive.

Two stale-formula fixes. balance_integration_chunks' docstring gave target_n as max(2*msing + 3, 4 * Threads.nthreads()), omitting both the parallel_threads cap and the min_bvp_intervals term that actually dominates. runtests_parallel_integration.jl:139 mirrored the same stale formula with the raw thread count, so it would fail on any machine with more threads than the cap — it passes today only because CI is single-threaded.

CLAUDE.md — the documented single-file invocation cannot work (runtests.jl passes ARGS to include, which resolves relative to test/, producing test/test/…), and two of the listed test files do not exist. Corrected, and the multi-threaded invocation documented.

Not included, deliberately

The decomposition axis is not in CI. Reaching it needs effective_threads > (9·msing+8)/4 — about 14 threads for this deck — which would mean heavily oversubscribing a 4-core runner and slowing the whole suite. It belongs in the nightly harness on hardware with real thread counts, and will be wired there with the harness CI work.

No change to the SLAYER deck's parallel_threads = 1. Its floor arithmetic (msing = 3 → 35) says the pin is unnecessary, but that deck has a different code path to γ and has not been measured. Proposing the change on analogy alone is exactly the reasoning this PR exists to replace. Worth a follow-up measurement.

Verification

  • runtests_thread_invariance.jl: 11/11 pass, julia -t 4, 72.5 s.
  • runtests_parallel_integration.jl: 106/106 pass at julia -t 4 after the effective_threads fix.
  • The src/ change is docstring-only; no executable code in the package is modified.

🤖 Generated with Claude Code

ForceFreeStatesControl documents that the parallel FM/BVP path produces bit-identical
Delta' across thread counts, but nothing tested it by varying threads, and the example
decks disagreed about whether to rely on it: the SLAYER deck pins parallel_threads = 1
"to keep the regression Delta' (and hence gamma) reproducible", while the DIII-D-like
ideal deck — whose Delta' the regression harness tracks — runs parallel_threads = 2.
A golden Delta' has to be a property of the physics rather than of the machine it was
measured on, so this settles the question by measurement.

Measured on the DIII-D-like deck at parallel_threads = 1, 2, 4 and 16: the Delta' matrix
diagonal and et[1] are bit-identical throughout. The 16-thread configuration is the
informative one — it lifts 4*effective_threads above the min_bvp_intervals floor and so
produces a genuinely different decomposition (64 chunks vs 53, with different boundaries),
which reassociates the propagator products without moving the result. Invariance therefore
holds across decomposition, not merely across scheduling.

  - test/runtests_thread_invariance.jl compares parallel_threads 1 vs 2 on the Solovev
    and DIII-D-like decks, asserting exact equality rather than a tolerance: the code
    claims bit-identity, and a tolerance would mask the reassociation the test exists to
    catch. It also asserts the chunk boundaries are unchanged at these caps, so that a
    future decomposition change surfaces as a failure instead of silently turning the
    comparison into a stronger claim than intended. Skipped when the session has one
    thread, where effective_threads collapses to 1 and every comparison is vacuous.

  - The test workflow gains a multi-threaded leg (JULIA_NUM_THREADS = 4). The suite had
    only ever run single-threaded, so the parallel paths were exercised solely in their
    degenerate form. Existing job names are preserved byte-identically because branch
    protection names them as required checks.

  - balance_integration_chunks' docstring gave target_n as
    max(2*msing + 3, 4*Threads.nthreads()), omitting both the parallel_threads cap and the
    min_bvp_intervals term that actually dominates. runtests_parallel_integration.jl
    mirrored the same stale formula and would fail on a machine with more threads than the
    cap; it passed only because CI is single-threaded.

  - CLAUDE.md's single-test-file invocation could not work (runtests.jl passes ARGS to
    include, which resolves relative to test/), and two of the listed files do not exist.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings August 12, 2026 21:10

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR strengthens the ForceFreeStates parallel FM/BVP verification pipeline by adding explicit thread-invariance tests, updating CI to exercise multi-threaded execution, and aligning documentation/tests with the actual chunk-balancing formula used for the parallel BVP path.

Changes:

  • Add runtests_thread_invariance.jl to assert parallel BVP results are bit-identical between parallel_threads=1 and 2 when running multi-threaded.
  • Add a multi-threaded CI matrix leg (JULIA_NUM_THREADS=4) without renaming the existing branch-protection-required job contexts.
  • Update the balance_integration_chunks docstring and the parallel-integration test to reflect the effective thread cap logic.

Note: Project policy still requires third-party human review before merge.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
test/runtests.jl Adds the new thread-invariance test file to the default test suite.
test/runtests_thread_invariance.jl New tests to verify thread invariance of Δ′/et across parallel_threads caps.
test/runtests_parallel_integration.jl Fixes stale formula in the test to use effective thread cap, matching implementation intent.
src/ForceFreeStates/EulerLagrange.jl Updates balance_integration_chunks docstring to describe current targeting behavior (but needs a small alignment fix).
CLAUDE.md Corrects documented test invocation and updates the list of example test files.
.github/workflows/test.yaml Adds a multi-threaded test leg by setting JULIA_NUM_THREADS via a matrix include.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +103 to +106
for j in 1:serial.msing
@test threaded.dpm[j, j] === serial.dpm[j, j]
end
@test threaded.dpm == serial.dpm
Comment on lines +80 to +82
if Threads.nthreads() < 2
@info "Thread-invariance tests skipped: effective_threads collapses to 1 in a single-threaded session. Run with `julia -t 4` (CI covers this in its multi-threaded matrix leg)."
@test true
Comment on lines +70 to +71
`max(2*msing + 3, 4*effective_threads, 8*(msing + 1) + msing)`, where
`effective_threads = min(Threads.nthreads(), ctrl.parallel_threads)`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants