From 76636a0680f61d3d50d303bc36afda4ea22e708d Mon Sep 17 00:00:00 2001 From: Matthew Pharr Date: Mon, 17 Aug 2026 13:15:29 -0400 Subject: [PATCH] FFS - REFACTOR - Reorganize ForceFreeStates into subdirectories (pure move) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Split the module into ownership-based subdirectories with zero logic changes: - Riccati/: Riccati.jl split into Types (IntegrationChunk, ChunkPropagator), Propagators (chunk FM integration, renormalization, propagator application), Crossings (singular-surface crossing algorithms), DeltaPrimeBVP (STRIDE BVP assembly/solve + PEST-3 decomposition), and Driver (chunk orchestration). - Surfaces/: singular-surface machinery in one place — Types (SingType, SingAsymptotics), Finding + Asymptotics (split of Sing.jl), Resist (GGJ coefficients only), ResistEval. - Matching/: outer<->inner matching seed — DeltaPrimeData (shared Delta-prime payload) and resonant_match_rpec/ResonantMatchResult (from Resist.jl). - ForceFreeStatesStructs.jl dissolved: FourFitVars -> Fourfit.jl, FreeBoundaryResult -> Free.jl, OdeState/EdgeScanState -> EulerLagrange.jl, module-wide types (ModeSpace, DebugSettings, Internal, Control) -> CoreTypes.jl. - Sing.jl's EL derivative kernel (sing_der!, el_derivatives!, compute_node_xi_s!) moved to EulerLagrange.jl where it belongs. - Include order in ForceFreeStates.jl rearranged so types load before the code that dispatches on them; updated the stability.md autodocs Pages list and stale file-path references in docs/comments. Verified: full test suite passes; gpec.h5 bit-identical vs the pre-move commit on the forward and riccati decks (h5diff; only the date_created attribute and HDF5 object-header timestamps differ); regression harness diiid_n1 47/47 unchanged with zero diff; local Documenter build clean. Co-Authored-By: Claude Fable 5 --- docs/development/architecture.md | 11 +- docs/src/citations.md | 2 +- docs/src/conventions.md | 2 +- docs/src/stability.md | 2 +- src/ForceFreeStates/CoreTypes.jl | 224 +++ src/ForceFreeStates/EulerLagrange.jl | 418 +++- src/ForceFreeStates/ForceFreeStates.jl | 30 +- src/ForceFreeStates/ForceFreeStatesStructs.jl | 680 ------- src/ForceFreeStates/Fourfit.jl | 77 + src/ForceFreeStates/Free.jl | 34 + src/ForceFreeStates/Matching/DeltaPrime.jl | 42 + src/ForceFreeStates/Matching/ResonantMatch.jl | 80 + src/ForceFreeStates/Riccati.jl | 1779 ----------------- src/ForceFreeStates/Riccati/Crossings.jl | 157 ++ src/ForceFreeStates/Riccati/DeltaPrimeBVP.jl | 731 +++++++ src/ForceFreeStates/Riccati/Driver.jl | 376 ++++ src/ForceFreeStates/Riccati/Propagators.jl | 518 +++++ src/ForceFreeStates/Riccati/Types.jl | 52 + .../{Sing.jl => Surfaces/Asymptotics.jl} | 592 +----- src/ForceFreeStates/Surfaces/Finding.jl | 368 ++++ src/ForceFreeStates/{ => Surfaces}/Resist.jl | 81 - .../{ => Surfaces}/ResistEval.jl | 0 src/ForceFreeStates/Surfaces/Types.jl | 64 + 23 files changed, 3174 insertions(+), 3146 deletions(-) create mode 100644 src/ForceFreeStates/CoreTypes.jl delete mode 100644 src/ForceFreeStates/ForceFreeStatesStructs.jl create mode 100644 src/ForceFreeStates/Matching/DeltaPrime.jl create mode 100644 src/ForceFreeStates/Matching/ResonantMatch.jl delete mode 100644 src/ForceFreeStates/Riccati.jl create mode 100644 src/ForceFreeStates/Riccati/Crossings.jl create mode 100644 src/ForceFreeStates/Riccati/DeltaPrimeBVP.jl create mode 100644 src/ForceFreeStates/Riccati/Driver.jl create mode 100644 src/ForceFreeStates/Riccati/Propagators.jl create mode 100644 src/ForceFreeStates/Riccati/Types.jl rename src/ForceFreeStates/{Sing.jl => Surfaces/Asymptotics.jl} (62%) create mode 100644 src/ForceFreeStates/Surfaces/Finding.jl rename src/ForceFreeStates/{ => Surfaces}/Resist.jl (58%) rename src/ForceFreeStates/{ => Surfaces}/ResistEval.jl (100%) create mode 100644 src/ForceFreeStates/Surfaces/Types.jl diff --git a/docs/development/architecture.md b/docs/development/architecture.md index 3d1aed08c..3de87d88f 100644 --- a/docs/development/architecture.md +++ b/docs/development/architecture.md @@ -61,11 +61,14 @@ GPEC consists of **eight main modules** organized in `src/`: - Solves ideal MHD eigenvalue problem with force-free boundary conditions - Identifies singular surfaces where ξ·∇ψ = 0 - Key files: - - `ForceFreeStatesStructs.jl` - Core data structures + - `CoreTypes.jl` - Module-wide types (`ForceFreeStatesControl`, `ForceFreeStatesInternal`) - `Result.jl` - `ForceFreeStatesResult`, the published solve product every downstream stage reads - - `Ode.jl` - ODE solver for Euler-Lagrange equations - - `Sing.jl` - Singular point handling and layer analysis - - `Fourfit.jl` - Fourier fitting routines + - `EulerLagrange.jl` - ODE integration of the Euler-Lagrange equations (`OdeState`, derivative kernel) + - `Surfaces/` - Singular-surface finding, Frobenius asymptotics, and GGJ coefficients + - `Riccati/` - Chunked fundamental-matrix (STRIDE) driver and Δ' boundary-value problem + - `Galerkin/` - RDCON outer-region singular Galerkin Δ' solver + - `Matching/` - Outer↔inner resistive matching (`DeltaPrimeData`, `resonant_match_rpec`) + - `Fourfit.jl` - Fourier fitting routines (`FourFitVars`) - `FixedBoundaryStability.jl` - Fixed boundary analysis - `Free.jl` - Free boundary stability - Status: Stable, core DCON functionality implemented diff --git a/docs/src/citations.md b/docs/src/citations.md index ea5ddaee3..1b0df7dbc 100644 --- a/docs/src/citations.md +++ b/docs/src/citations.md @@ -36,7 +36,7 @@ The primary reference for the `ForceFreeStates` module. Derives the Euler-Lagran > *Physics of Plasmas* **25**, 032507 (2018). > DOI: [10.1063/1.5007042](https://doi.org/10.1063/1.5007042) -Reformulates the DCON eigenvalue problem as a Riccati matrix ODE, enabling parallel integration across singular surfaces and faster computation. Implemented in `src/ForceFreeStates/Riccati.jl` and enabled via `integrator = "riccati"` in `[ForceFreeStates]`. +Reformulates the DCON eigenvalue problem as a Riccati matrix ODE, enabling parallel integration across singular surfaces and faster computation. Implemented in `src/ForceFreeStates/Riccati/` and enabled via `integrator = "riccati"` in `[ForceFreeStates]`. --- diff --git a/docs/src/conventions.md b/docs/src/conventions.md index ad0c917d6..cc1afd539 100644 --- a/docs/src/conventions.md +++ b/docs/src/conventions.md @@ -155,7 +155,7 @@ modes on the high side as `delta_mlow` does on the low side. ### Why Positive ``m`` Is Always Resonant -Resonant surfaces are found by `sing_find!` (`src/ForceFreeStates/Sing.jl`), which locates flux +Resonant surfaces are found by `sing_find!` (`src/ForceFreeStates/Surfaces/Finding.jl`), which locates flux surfaces where ``m = n\,q(\psi)``. Since ``n > 0`` by convention and ``q > 0`` for a standard tokamak, the resonant ``m`` is always positive: diff --git a/docs/src/stability.md b/docs/src/stability.md index 529304a90..87b168d07 100644 --- a/docs/src/stability.md +++ b/docs/src/stability.md @@ -292,7 +292,7 @@ The Galerkin Δ′ solver (`src/ForceFreeStates/Galerkin/`) is documented separa ```@autodocs Modules = [GeneralizedPerturbedEquilibrium.ForceFreeStates] -Pages = ["ForceFreeStates.jl", "ForceFreeStatesStructs.jl", "Result.jl", "Resist.jl", "EulerLagrange.jl", "Sing.jl", "Fourfit.jl", "Kinetic.jl", "FixedBoundaryStability.jl", "Utils.jl", "Free.jl", "Riccati.jl"] +Pages = ["ForceFreeStates.jl", "CoreTypes.jl", "Surfaces/Types.jl", "Riccati/Types.jl", "Matching/DeltaPrime.jl", "Result.jl", "Surfaces/Resist.jl", "Surfaces/ResistEval.jl", "Matching/ResonantMatch.jl", "EulerLagrange.jl", "Surfaces/Finding.jl", "Surfaces/Asymptotics.jl", "Fourfit.jl", "Kinetic.jl", "FixedBoundaryStability.jl", "Utils.jl", "Free.jl", "Riccati/Propagators.jl", "Riccati/Crossings.jl", "Riccati/DeltaPrimeBVP.jl", "Riccati/Driver.jl"] ``` ## Example usage diff --git a/src/ForceFreeStates/CoreTypes.jl b/src/ForceFreeStates/CoreTypes.jl new file mode 100644 index 000000000..98d4807e1 --- /dev/null +++ b/src/ForceFreeStates/CoreTypes.jl @@ -0,0 +1,224 @@ +# Module-wide types: ModeSpace, DebugSettings, ForceFreeStatesInternal, ForceFreeStatesControl. + +""" + ModeSpace + +Supertype for objects that carry the resolved (m, n) mode space — `mlow`, `mhigh`, `mpert`, +`nlow`, `nhigh`, `npert`, `numpert_total`. Both the solve-time scratch +[`ForceFreeStatesInternal`](@ref) and the published [`ForceFreeStatesResult`](@ref) are +`ModeSpace`s, so kernels that need nothing but the mode indexing (`el_derivatives!`, +`materialize_derivative_stores!`, `build_kinetic_metric_matrices`) accept either. +""" +abstract type ModeSpace end + +""" +DebugSettings + +A mutable struct containing settings for debugging and benchmarking output. + +## Fields + + - `output_benchmark_data::Bool` - Flag to output benchmark data for comparison between codes + - `gal_basis_output::Bool` - Write the raw Galerkin outer-region basis functions (per-interval, unconstrained at the rationals) under `GalerkinIntegration/Basis/`. Solver internals for development verification, not physics output. +""" +@kwdef mutable struct DebugSettings + output_benchmark_data::Bool = false + gal_basis_output::Bool = false +end + +""" + ForceFreeStatesInternal + +A mutable struct holding internal state variables for stability calculations. + +## Fields + + - `dir_path::String` - Directory path for input/output files + - `mlow::Int` - Lowest poloidal mode number + - `mhigh::Int` - Highest poloidal mode number + - `mpert::Int` - Number of poloidal modes (mhigh - mlow + 1) + - `nlow::Int` - Lowest toroidal mode number, resolved from `ctrl.nn_low`/`nn_high` + - `nhigh::Int` - Highest toroidal mode number, resolved from `ctrl.nn_low`/`nn_high` + - `npert::Int` - Number of toroidal modes (nhigh - nlow + 1) + - `numpert_total::Int` - Total number of modes (mpert × npert) + - `msing::Int` - Number of ideal singular surfaces + - `kmsing::Int` - Number of kinetic singular surfaces (det(F̄) near-zeros) + - `sing::Vector{SingType}` - Vector of ideal singular surface data + - `kinsing::Vector{SingType}` - Vector of kinetic singular surface data + - `kinsing_scan_psi::Vector{Float64}` - ψ grid used by `find_kinetic_singular_surfaces!` for the cond(F̄) scan (empty unless the finder has run) + - `kinsing_scan_cond::Vector{Float64}` - cond(F̄) values on that grid; the finder locates peaks that exceed `kinsing_scan_threshold` + - `kinsing_scan_threshold::Float64` - Threshold on cond(F̄) used to accept a peak as a kinetic singular surface + - `psilim::Float64` - Flux limit for integration + - `qlim::Float64` - Safety factor at psilim + - `q1lim::Float64` - Safety factor derivative at psilim + - `wall_settings::Vacuum.WallShapeSettings` - Wall shape settings for vacuum calculations +""" +@kwdef mutable struct ForceFreeStatesInternal <: ModeSpace + dir_path::String = "" + mlow::Int = 0 + mhigh::Int = 0 + mpert::Int = 0 + nlow::Int = 0 + nhigh::Int = 0 + npert::Int = 0 + numpert_total::Int = 0 + msing::Int = 0 + kmsing::Int = 0 + sing::Vector{SingType} = SingType[] + kinsing::Vector{SingType} = SingType[] + kinsing_scan_psi::Vector{Float64} = Float64[] + kinsing_scan_cond::Vector{Float64} = Float64[] + kinsing_scan_threshold::Float64 = 0.0 + psilim::Float64 = 0.0 + psilow::Float64 = 0.0 # lower integration bound; raised above the axis by sing_min! when qlow > qmin (RDCON gal) + qlim::Float64 = 0.0 + q1lim::Float64 = 0.0 + debug_settings::DebugSettings = DebugSettings() + wall_settings::Vacuum.WallShapeSettings = Vacuum.WallShapeSettings() + """ + Inter-surface Δ' matrix of shape (msing × msing) in PEST3 convention. + Computed by `compute_delta_prime_matrix!` (parallel FM path only) using the STRIDE + global BVP with vacuum coupling. The deltap linear combination is applied to the + raw 2msing×2msing BVP solution to produce the PEST3-compatible tearing parameter. + """ + delta_prime_matrix::Matrix{ComplexF64} = Matrix{ComplexF64}(undef, 0, 0) + + """ + Edge coil-response matrix of shape (2msing × numpert_total). Column k is the resonant + small-solution response at each surface side to a unit source on edge poloidal mode k, + built by imposing the Eq. (37) rpec edge boundary condition on the Riccati BVP + (`_solve_bvp_edge_coil`). Empty unless the S-axis vacuum-edge BVP was assembled. + """ + delta_coil::Matrix{ComplexF64} = Matrix{ComplexF64}(undef, 0, 0) + + """ + Raw 2msing × 2msing outer-region matching matrix `D'` from the STRIDE global + BVP, in the side-major ordering `[L_s1, R_s1, L_s2, R_s2, …, L_sm, R_sm]` + (left vs right of each singular surface, interleaved surface-by-surface). + This is the Pletzer–Dewar 1991 outer-region matrix before parity rotation, + and is stored byte-compatibly with the Fortran `rdcon/gal.f::gal_write_delta` + convention (top 2msing×2msing block of `delta_gw.dat`). The PEST3 Δ' matrix + stored in `delta_prime_matrix` is the odd-parity tearing projection of this + raw matrix; the even-parity A' and off-parity B', Γ' blocks are recovered + via `pest3_decompose(dp_raw)` — needed for the full det(D' − D(γ)) = 0 + eigenvalue problem with Glasser stabilization. + + Empty unless the Riccati integrator was used. No ½ prefactor is applied (matches + Fortran rdcon; Pletzer–Dewar paper multiplies by ½). + """ + delta_prime_raw::Matrix{ComplexF64} = Matrix{ComplexF64}(undef, 0, 0) +end + +""" + ForceFreeStatesControl + +An immutable struct containing 'ForceFreeStates' parameters set by the user in +gpec.toml. + +## Fields + + - `verbose::Bool` - Enable verbose output + - `local_stability_flag::Bool` - Enable local stability analysis (`D_I` and ballooning) + - `vac_flag::Bool` - Enable vacuum region calculation + - `mthvac::Int` - Number of vacuum poloidal grid points (corresponds to `mtheta` in VacuumInput) + - `nzvac::Int` - Number of vacuum toroidal grid points (corresponds to `nzeta` in VacuumInput3D) + - `sing_start::Int` - Start integration at the `sing_start`-th singular surface + - `nn_low::Int` - Lower bound for toroidal modes + - `nn_high::Int` - Upper bound for toroidal modes + - `delta_mlow::Int` - Expands lower bound of Fourier harmonics by delta_mlow + - `delta_mhigh::Int` - Expands upper bound of Fourier harmonics by delta_mhigh + - `nstep::Int` - Maximum number of integration steps (not yet implemented) + - `ksing::Int` - Singular surface handling parameter + - `eulerlagrange_tolerance::Float64` - Relative tolerance for ODE integration of Euler-Lagrange equations + - `ucrit::Float64` - Critical value of unorm ratio to trigger solution normalization. In the standard path it triggers Gaussian reduction; in the Riccati path it triggers `renormalize_riccati_inplace!`. Default `1e4` empirically keeps max(|U₁|, |U₂|) in O(1)–O(10⁴) over the integration domain on DIII-D / Solovev sweeps; lower triggers excess renorms without accuracy gain, higher risks overflow before the next renorm. + - `numsteps_init::Int` - Initial array size for ODE data storage + - `numunorms_init::Int` - Initial array size for solution normalization data + - `singfac_min::Float64` - Fractional distance from rational q at which ideal jump condition is enforced + - `set_psilim_via_dmlim::Bool` - Truncate the integration domain at `(last_rational_q + dmlim) / n` rather than at `qhigh` / `psihigh`. Fortran STRIDE found that truncating ~20 % above the outermost rational (`dmlim = 0.2`) avoids a numerical kink instability in δW that appears when the integration ends too close to or just below a rational surface. **For diverted equilibria where q → ∞ at the separatrix** (e.g. DIII-D geqdsks, the bulk of production use) this costs negligible physical domain because rationals get arbitrarily dense near the LCFS — `set_psilim_via_dmlim = true` is the safe and recommended default. **For limited circular / analytical equilibria with finite q at the edge** (Solovev, LAR scans), rationals are sparse and 20 % above the last rational chops off too much edge, so set `set_psilim_via_dmlim = false` and let `qhigh` / `psihigh` control the truncation. Multi-`n` runs are not supported by this truncation (the "outermost rational + dmlim / n" depends on which `n`); when `set_psilim_via_dmlim = true` with `nn_low != nn_high`, `sing_lim!` warns and falls back to `qhigh` / `psihigh`. Default `true`. + - `dmlim::Float64` - Distance beyond last rational surface (normalised ∈ [0,1) in units of 1/n). Only used when `set_psilim_via_dmlim` is true. Fortran STRIDE convention is 0.2 (truncate 20 % of one rational-surface spacing above the last surface), retained here. + - `sing_order::Int` - Order of singular layer (Frobenius) expansion at rational surfaces. Default 6 (Fortran STRIDE convention for Δ' calculations; lower values trade accuracy for speed). + - `qhigh::Float64` - Integration terminated at q limit determined by minimum of qhigh and qa from equil + - `kinetic_source::String` - Kinetic matrix source: "fixed" (X-shaped test matrices scaled by kinetic_factor relative to ideal matrix Frobenius norms; Ak, Dk, Hk Hermitian, Bk, Ck, Ek non-Hermitian), "calculated" (PENTRC — not yet implemented) + - `kinetic_factor::Float64` - Dimensionless scaling factor for kinetic matrices. Zero (the default) disables the kinetic path; any positive value enables it and scales the kinetic matrices: when kinetic_source="fixed", scales X-shaped test matrices relative to ideal matrix norms; when kinetic_source="calculated", applied as uniform post-hoc multiplier to W and T components. + - `qlow::Float64` - Integration terminated at q limit determined by minimum of qlow and q0 from equil + - `psiedge::Float64` - If less than psilim, records a dW(ψ) diagnostic scan over [psiedge, psilim] on odet.edge_scan. The integration domain (psilim) is always controlled by qhigh / psihigh and is not modified by this scan (unless `truncate_at_dW_peak=true`, see caveats below). + - `truncate_at_dW_peak::Bool` - When `true` and `psiedge < psilim`, the edge-dW scan's peak location is adopted as the new physical plasma edge — `intr.psilim`/`intr.qlim`/`odet.u` are pulled back to the peak, AND the FM Δ' chunks/propagators are made self-consistent with the new boundary (the chunk that straddles the peak is rebuilt + re-integrated; any chunks past the peak are dropped). This reproduces the spirit of the original ode_record_edge heuristic from Fortran STRIDE while keeping Δ' and δW well-defined at the new boundary. The Δ' metric is still physically dependent on where the peak falls in the edge band, so use this flag deliberately when you mean to scan against the peak-defined edge (e.g. for studying edge-mode regimes); leave at `false` (default) for the full-domain Δ' at `qhigh` / `psihigh` / `dmlim`. + - `diagnose::Bool` - Enable diagnostic output (not yet implemented) + - `diagnose_ca::Bool` - Enable asymptotic coefficient diagnostics (not yet implemented) + - `write_outputs_to_HDF5::Bool` - Write results to HDF5 format + - `HDF5_filename::String` - Name of HDF5 output file + - `save_interval::Int` - Save every Nth ODE step (1=all, 10=every 10th). Always saves near rational surfaces. (Same as `euler_step` in the Fortran) + - `force_termination::Bool` - Terminate after force-free states (skip perturbed equilibrium calculations) + - `integrator::String` - Which formalism integrates the Euler-Lagrange system. `"forward"` sweeps the plasma serially with Gaussian reduction and returns `u_store` / `du_store` / `xi_s_store` dense in the axis (EL) basis — the only convention PerturbedEquilibrium and FieldReconstruction consume correctly, and the only path that supports `kinetic_factor > 0`. `"riccati"` (default) runs the chunked fundamental-matrix propagator driver (Glasser 2018 Phys. Plasmas 25, 032507): chunks are integrated independently from identity initial conditions and assembled serially with Riccati-style crossings, which is the only way to obtain the singular-surface Δ' matrix for the tearing-mode solvers downstream, but leaves `u_store` as sparse chunk-endpoint Riccati states, so dense ξ profiles are unavailable. `"galerkin"` solves the same Euler-Lagrange system variationally instead of by radial ODE integration — the RDCON outer-region singular Galerkin method (Glasser, Wang & Park 2016 Phys. Plasmas 23, 112506), which discretizes the displacement on packed Hermite-cubic elements and solves one global banded system — producing the resistive Δ′ matrix and, when `gal_match_flag` is set, the RPEC inner-layer-matched ξ; it computes its own vacuum response and returns no free-boundary energies, and does not support `kinetic_factor > 0`. Requires `singfac_min != 0` for `"riccati"`. + - `nchunks::Int` - Target number of Riccati integration chunks. `0` (the default) derives the count from problem structure alone: `max(2·msing + 3, 8·(msing + 1) + msing)`, enough sub-chunks per segment to keep the accumulated propagator products well-conditioned. An explicit value below `2·msing + 3` is clamped up with a warning. Chunk sizing never consults `Threads.nthreads()`, so Riccati outputs are identical whatever thread count `julia -t` provides; threads only change wall-clock. + - `extended_precision_bvp::Bool` - When `true` (default), promote the Δ' BVP linear system to `Complex{Double64}` (~31 digits) for the LU solve and PEST3 combination. Guards against catastrophic cancellation in the PEST3 four-term combination (dp_raw entries can be 10⁴–10⁵× larger than the result; the imaginary part of off-diagonal Δ' is particularly sensitive). Disabling (`false`) saves ~1.5–2× the BVP solve time but on DIIID-class equilibria the imaginary Δ' components can drift by factors of 2–5×; only disable for performance experiments on cases where Float64 has been validated against Double64. +""" +@kwdef struct ForceFreeStatesControl + verbose::Bool = true + local_stability_flag::Bool = false + vac_flag::Bool = false + mthvac::Int = 480 + nzvac::Int = 1 + sing_start::Int = 0 + nn_low::Int = 0 + nn_high::Int = 0 + delta_mlow::Int = 0 + delta_mhigh::Int = 0 + nstep::Int = typemax(Int) + ksing::Int = -1 + eulerlagrange_tolerance::Float64 = 1e-8 + ucrit::Float64 = 1e4 + numsteps_init::Int = 4000 + numunorms_init::Int = 100 + singfac_min::Float64 = 1e-4 # Matches Fortran STRIDE; required nonzero for the Riccati path. + set_psilim_via_dmlim::Bool = true # Safe default for diverted equilibria (most production use); set false for limited/analytical (LAR, Solovev). Auto-skipped for multi-n. See docstring. + dmlim::Float64 = 0.2 + sing_order::Int = 6 + qhigh::Float64 = 1e3 + kinetic_source::String = "fixed" + kinetic_factor::Float64 = 0.0 + qlow::Float64 = 0.0 + psiedge::Float64 = 0.99 + truncate_at_dW_peak::Bool = false # Edge-dW peak becomes new physical edge; Δ' BVP made self-consistent. See docstring. + diagnose::Bool = false + diagnose_ca::Bool = false + write_outputs_to_HDF5::Bool = true + HDF5_filename::String = "gpec.h5" + save_interval::Int = 3 + force_termination::Bool = false + integrator::String = "riccati" # Default: unlocks SingularSurfaces/Delta_prime_matrix (STRIDE BVP Δ′ matrix) used by SLAYER/GGJ downstream. Use "forward" for dense ξ (PerturbedEquilibrium) or kinetic runs. + nchunks::Int = 0 # Riccati chunk-count target; 0 = auto (derived from msing alone, never from Threads.nthreads()). + extended_precision_bvp::Bool = true # Promote Δ' BVP to Complex{Double64}; default on (Float64 drifts the imaginary Δ' by 2–5× on DIIID-class cases). + + # --- RDCON outer-region Galerkin Δ′ solver (gal_solve port); selected by integrator = "galerkin" --- + gal_solver::String = "LU" # "LU" (zgbtrf/zgbtrs) or "cholesky" (zpbtrf/zpbtrs) + gal_nx::Int = 256 # elements per interval between singular surfaces + gal_nq::Int = 6 # Gauss-Lobatto quadrature order per element + gal_pfac::Float64 = 0.001 # grid packing ratio near singular surfaces + gal_dx0::Float64 = 5e-4 # resonant-element integration truncation distance (×1/|n q'|) + gal_dx1::Float64 = 1e-3 # resonant-element size (×1/|n q'|) + gal_dx2::Float64 = 1e-3 # extension-element size (×1/|n q'|) + gal_cutoff::Int = 10 # # of elements carrying the large solution as driving term + gal_tol::Float64 = 1e-10 # resonant-quadrature (QuadGK) tolerance + gal_gnstep::Int = 20000 # max resonant-quadrature evaluations (QuadGK maxevals in gal_resonant!) + gal_dx1dx2_flag::Bool = true # enable special dx1/dx2 treatment for resonant/extension elements + gal_sing_order::Int = 6 # base power-series order for the Galerkin singular asymptotics + gal_sing_order_ceiling::Bool = true # auto-raise order by ceil(2·Re(α)) per surface (high Mercier index) + gal_rpec_flag::Bool = false # append mpert coil-response columns to the Δ′ solve (RDCON rpec_flag): unit boundary sources whose plasma response is recorded; needed for the driven (resistive perturbed-equilibrium) Δ_gw + gal_edge_onesided::Bool = false # pack the two end intervals one-sided toward their single rational end (vs the Fortran symmetric "both" pack); avoids the fine edge cell that inflates cond(A). Default false = faithful to gal.f. + # --- DRIVEN (RPEC) outer↔inner asymptotic matching (rmatch match_rpec port) --- + gal_match_flag::Bool = false # enable the RPEC inner-layer matching: solve the coil-driven matched ξ(ψ) from the gal Δ′ + the inner-layer Δ(Q). Requires gal_rpec_flag=true. + gal_ideal_flag::Bool = false # within the match, build the IDEAL solution: skip the inner-layer Δ, use bare coil columns (cout=0). Mirrors Fortran rmatch coil%ideal_flag (the EL reference). eta/rho/rotation ignored. + gal_inner_solver::String = "ray" # inner-layer Δ backend for the match: "ray" (rotated-contour collocation, certified Δ at the optimal θ = arg(Q)/4; robust for |Q| ≳ 1) or "galerkin" (Hermite-cubic inps; drifts for |Q| ≳ 1) + # --- Inner-layer "galerkin" backend knobs (used only when gal_inner_solver = "galerkin") --- + gal_inner_xfac::Float64 = 10.0 # asymptotic-matching radius multiplier (inps_xfac: xmax × 10) + gal_inner_nx::Int = 1280 # inner-layer grid cells (128 · xfac in the reference) + gal_inner_nq::Int = 5 # quadrature order per cell + gal_inner_cutoff::Int = 5 # cells carrying the large solution as driving term + gal_inner_kmax::Int = 8 # large-x asymptotic series order (↔ order_pow) + gal_eta::Vector{Float64} = Float64[] # per-surface resistivity η (length msing, core→edge); Fortran rmatch `eta` + gal_rho::Vector{Float64} = Float64[] # per-surface mass density ρ [kg/m³] (length msing, core→edge); Fortran rmatch `massden` + gal_rotation::Vector{Float64} = Float64[] # per-surface rotation frequency f [Hz] (length msing, core→edge); forced eigenvalue γ_s = 2πi·n·f. Fortran rmatch `rotation` + gal_gamma::Float64 = 5 / 3 # ratio of specific heats Γ for the resistive-layer coefficients (resist_eval G term) + fixed_axis::Bool = false +end diff --git a/src/ForceFreeStates/EulerLagrange.jl b/src/ForceFreeStates/EulerLagrange.jl index 88c1d9e4a..ab379b12c 100644 --- a/src/ForceFreeStates/EulerLagrange.jl +++ b/src/ForceFreeStates/EulerLagrange.jl @@ -1,3 +1,193 @@ +""" +EdgeScanState + +Holds the state and results for the edge dW stability scan over ψ ∈ [psiedge, psilim]. +Initialized and populated by `findmax_dW_edge!`; results written to HDF5 under `EdgeScan/`. +The energies are generalized (W, N) pencil values: power-normalized and invariant to the +working (Jacobian) coordinate (see `power_norm_matrix!`). + +## Fields + + - `wvmat` - Precomputed wv matrix spline (raw, no singfac); singfac applied analytically in `free_compute_total`. + - `wv_hint::Base.RefValue{Int}` - Search hint for wvmat spline (different grid from equilibrium profiles). + - `psi, q` - ψ and q values at each edge scan step. + - `total_eigenvalue, plasma_energy, vacuum_energy, vacuum_eigenvalue` - Power-normalized energy components at each step (NaN for steps where the wp solve was singular). These drive the truncation choice and are written to `EdgeScan/`. +""" +@kwdef mutable struct EdgeScanState + numpert_total::Int + N_edge::Int + + # Vacuum matrix spline and evaluation infrastructure + wvmat::CubicSeriesInterpolant{Float64,ComplexF64} = _empty_series_interp_complex(numpert_total^2) + wv_hint::Base.RefValue{Int} = Ref(1) + + # Scan results (written to HDF5 under EdgeScan/; NaN where free_compute_total raised SingularException) + psi::Vector{Float64} = Vector{Float64}(undef, N_edge) + q::Vector{Float64} = Vector{Float64}(undef, N_edge) + total_eigenvalue::Vector{ComplexF64} = fill(complex(NaN), N_edge) + plasma_energy::Vector{ComplexF64} = fill(complex(NaN), N_edge) + vacuum_energy::Vector{ComplexF64} = fill(complex(NaN), N_edge) + vacuum_eigenvalue::Vector{Float64} = fill(NaN, N_edge) +end + +EdgeScanState(numpert_total::Int, N_edge::Int) = EdgeScanState(; numpert_total, N_edge) + +""" +OdeState + +A mutable struct to hold the state of the ODE solver used by the ForceFreeStates integration routines. +This struct stores configuration parameters used to allocate arrays, the evolving stored +solution during integration, diagnostic arrays used for normalization / Gaussian reduction, +and a small set of temporary matrices and factors used to compute singular-layer corrections. + +## Fields + + - `numpert_total::Int` - Total number of Fourier mode combinations (m × n) used in the calculation. + + - `numunorms_init::Int` - Initial allocation size for the number of normalization operations recorded. + + - `msing::Int` - Number of singular surfaces in the equilibrium (used to size asymptotic coefficient arrays). + + - `numsteps_init::Int` - Initial allocation size for the number of integration steps to store. + + - `step::Int` - Current integration step index (1-based, like `istep` in the original Fortran). + + - `psi_store::Vector{Float64}` - Stored psi values at each saved integration step (length `numsteps_init`). + + - `q_store::Vector{Float64}` - Stored q values at each saved integration step (length `numsteps_init`). + + - `u_store::Array{ComplexF64,4}` - Stored solution arrays at each saved step with shape + `(numpert_total, numpert_total, 2, numsteps_init)` (complex solution state used by the solver). + + - `du_store::Array{ComplexF64,3}` - dΞ_ψ/dψ (the u₁ block only) at each saved step, shape + `(numpert_total, numpert_total, step)`. Empty until `materialize_derivative_stores!` fills it, + except on the galerkin-matched path which supplies the analytic derivative at construction. + du₂/dψ is never stored densely — its only consumer evaluates it on demand at bracket nodes. + + - `xi_s_store::Array{ComplexF64,3}` - Clebsch displacement Ξ_s at each saved step, eq. 18 of Glasser 2016, + shape `(numpert_total, numpert_total, step)`. Empty until materialized, same as `du_store`. + + - `u_store_el_basis::Bool` - True when `u_store` holds the Euler-Lagrange state `(u₁, u₂)`, so the + derivative kernel can be re-applied to it. False on the sparse parallel path, whose stored columns + are chunk-endpoint Riccati matrices; `materialize_derivative_stores!` refuses to run there. + + - `du_store_populated::Bool` - True once `du_store`/`xi_s_store` hold valid data in the final + (post-transform, post-normalization) basis. Set by `materialize_derivative_stores!` or by the + galerkin-matched constructor; stays false where the stores cannot be materialized, e.g. the + sparse parallel path whose solution is in the Riccati basis. + + - `crit_store::Vector{Float64}` - Stored crit parameter values (smallest eigenvalue of W⁻ꜝ) (length `numsteps_init`). + + - `ca_r::Array{ComplexF64,4}` - Asymptotic coefficients just to the right of each singular surface + with shape `(numpert_total, numpert_total, 2, msing)`. + + - `ca_l::Array{ComplexF64,4}` - Asymptotic coefficients just to the left of each singular surface + with shape `(numpert_total, numpert_total, 2, msing)`. + + - `edge_scan::EdgeScanState` - Edge dW scan state and results. Initialized as a disabled sentinel (N_edge=0) and replaced by `findmax_dW_edge!` when a scan runs. + + - `psifac::Float64` - Current normalized flux coordinate for the integrator. + + - `q::Float64` - Safety factor value at `psifac` (current q during integration). + + - `u::Array{ComplexF64,3}` - Current working solution arrays with shape `(numpert_total, numpert_total, 2)`. + + - `ising_start::Int` - Index of the starting singular surface to be crossed during integration. + + - `psimax::Float64` - Maximum psi value for which the integrator is allowed to run in next integration region. + + - `needs_crossing::Bool` - Flag indicating whether a rational surface needs to be crossed after the current integration region. + + - `nzero::Int` - Count of detected zero crossings (used for diagnostics). + + - `new::Bool` - Flag indicating whether a new `unorm0` should be computed after a fixup. + + # Initialization parameters + + - `unorm::Vector{Float64}` - Current norms of the solution vectors (length `numpert_total`). + + - `unorm0::Vector{Float64}` - Reference/initial norms of the solution vectors (length `numpert_total`). + + # Saved data throughout integration + + - `ifix::Int` - Number of normalization operations performed (index into normalization arrays). + +# Total ODE solver steps taken (all steps, not just saved ones) + + - `index::Array{Int,2}` - Index matrix used for sorting solution norms with shape `(numpert_total, numunorms_init)`. + + - `sing_flag::Vector{Bool}` - Boolean flags indicating which stored normalizations correspond to singular solutions # Edge dW scan state and results (disabled sentinel when psiedge >= psilim, i.e. no edge scan) + (length `numunorms_init`). + + - `zeroed_idx::Vector{Vector{Int}}` - For each ideal rational surface jump, a vector of indices of solutions that were zeroed. # Data for integrator + + - `fixfac::Array{ComplexF64,3}` - Fix-up factors for Gaussian reduction with shape `(numpert_total, numpert_total, numunorms_init)`. + + - `fixstep::Vector{Int64}` - Step indices (psi step positions) at which normalization/fixups were performed (length `numunorms_init`). +""" +@kwdef mutable struct OdeState + # Initialization parameters + numpert_total::Int + numunorms_init::Int + msing::Int + numsteps_init::Int + + # Saved data throughout integration + step::Int = 1 + total_steps::Int = 0 # Total ODE solver steps taken (all steps, not just saved ones) + psi_store::Vector{Float64} = Vector{Float64}(undef, numsteps_init) + q_store::Vector{Float64} = Vector{Float64}(undef, numsteps_init) + u_store::Array{ComplexF64,4} = Array{ComplexF64}(undef, numpert_total, numpert_total, 2, numsteps_init) + du_store::Array{ComplexF64,3} = Array{ComplexF64}(undef, numpert_total, numpert_total, 0) + xi_s_store::Array{ComplexF64,3} = Array{ComplexF64}(undef, numpert_total, numpert_total, 0) + u_store_el_basis::Bool = true + du_store_populated::Bool = false + crit_store::Vector{Float64} = Vector{Float64}(undef, numsteps_init) + ca_r::Array{ComplexF64,4} = Array{ComplexF64}(undef, numpert_total, numpert_total, 2, msing) + ca_l::Array{ComplexF64,4} = Array{ComplexF64}(undef, numpert_total, numpert_total, 2, msing) + + # Edge dW scan state and results (disabled sentinel when psiedge >= psilim, i.e. no edge scan) + edge_scan::EdgeScanState = EdgeScanState(numpert_total, 0) + + # Data for integrator + psifac::Float64 = 0.0 + q::Float64 = 0.0 + u::Array{ComplexF64,3} = zeros(ComplexF64, numpert_total, numpert_total, 2) + ising_start::Int = 0 + psimax::Float64 = 0.0 + needs_crossing::Bool = false + nzero::Int = 0 + + # Used for Gaussian reduction + new::Bool = true + unorm::Vector{Float64} = zeros(Float64, numpert_total) + unorm0::Vector{Float64} = zeros(Float64, numpert_total) + ifix::Int = 0 + index::Array{Int,2} = zeros(Int, numpert_total, numunorms_init) + sing_flag::Vector{Bool} = falses(numunorms_init) + zeroed_idx::Vector{Vector{Int}} = [Int[] for _ in 1:numunorms_init] + fixfac::Array{ComplexF64,3} = zeros(ComplexF64, numpert_total, numpert_total, numunorms_init) + fixstep::Vector{Int64} = zeros(Int64, numunorms_init) + + # Kinetic workspace arrays: evaluated from kwmats/ktmats splines at current psi + kwmat::Array{ComplexF64,3} = zeros(ComplexF64, numpert_total, numpert_total, 6) + ktmat::Array{ComplexF64,3} = zeros(ComplexF64, numpert_total, numpert_total, 6) + + # Shared hint for CubicInterpolant interval search optimization during ODE integration + # All splines evaluated at the same psi can share this hint for O(1) interval lookups + spline_hint::Base.RefValue{Int} = Ref(1) + # Shared 2D hint for CubicInterpolantND (rzphi splines) during ODE integration + # Tuple of (psi_hint, theta_hint) for O(1) interval lookups in 2D bicubic splines + rzphi_hint::Tuple{Base.RefValue{Int},Base.RefValue{Int}} = (Ref(1), Ref(1)) + # Per-thread hint for FourFitVars matrix splines (amats/bmats/cmats/fmats_lower/kmats/gmats + # and kinetic equivalents). Lives on OdeState — which is already cloned per thread in the + # parallel BVP path — so concurrent sing_der! invocations don't race on a shared Ref. + ffit_hint::Base.RefValue{Int} = Ref(1) +end + +OdeState(numpert_total::Int, numsteps_init::Int, numunorms_init::Int, msing::Int) = + OdeState(; numpert_total, numsteps_init, numunorms_init, msing) + """ compute_delta_prime_from_ca!(odet, intr, equil) @@ -230,7 +420,7 @@ function forward_eulerlagrange_integration(ctrl::ForceFreeStatesControl, equil:: # determined solely by qhigh / psihigh / dmlim so Δ' and δW are independent of peak # location. Legacy path (true) reproduces the ode_record_edge heuristic from Fortran # STRIDE — psilim/qlim/u are pulled back to the dW peak. Preserved for experimental - # work; see docstring in ForceFreeStatesStructs.jl for the reliability caveats. + # work; see the ForceFreeStatesControl docstring for the reliability caveats. if ctrl.psiedge < intr.psilim saved_psifac, saved_u = odet.psifac, copy(odet.u) peak_step = findmax_dW_edge!(odet, ctrl, equil, ffit, intr) @@ -1022,3 +1212,229 @@ function transform_u!(odet::OdeState, intr::ForceFreeStatesInternal) jfix = kfix + 1 end end + +""" + sing_der!( + du::Array{ComplexF64,3}, + u::Array{ComplexF64,3}, + params::Tuple{ForceFreeStatesControl, Equilibrium.PlasmaEquilibrium, FourFitVars, ForceFreeStatesInternal, OdeState, IntegrationChunk}, + psieval::Float64 + ) + +Evaluate the derivative of the Euler-Lagrange equations [Glasser Phys. Plasmas 2016 112506 eq. 24]. +This implements du/dψ for both the ideal and kinetic MHD eigenvalue problems. + +This function performs the same role as `sing_der` in the Fortran code, with main differences +coming from hiding LAPACK operations under the hood via Julia's LinearAlgebra package, +so the code is much more straightforward. + +This follows the Julia DifferentialEquations package format for in place updating. + + ode_function!(du, u, p, t) + +From DifferentialEquations.jl docs: Defining your ODE function to be in-place updating +can have performance benefits. What this means is that, instead of writing a function +which outputs its solution, you write a function which updates a vector that is designated +to hold the solution. By doing this, DifferentialEquations.jl's solver packages are able +to reduce the amount of array allocations and achieve better performance. + +Wherever possible, in-place operations on pre-allocated arrays are used to minimize memory allocations. +All LAPACK operations are handled under the hood by Julia's LinearAlgebra package, so we can obtain a much +more simplistic code with similar performance. + +### Arguments + + - `du::Array{ComplexF64,3}`: Pre-allocated array to hold the derivative result, shape (mpert, mpert, 2), updated in-place + - `u::Array{ComplexF64,3}`: Current state array, shape (mpert, mpert, 2) + - `params::Tuple{ForceFreeStatesControl, PlasmaEquilibrium, FourFitVars, ForceFreeStatesInternal, OdeState, IntegrationChunk}`: Tuple of relevant structs + - `psieval::Float64`: Current psi value at which to evaluate the derivative + +The unpacked-argument method carries the arithmetic; this tuple method is the thin adapter the +integrator calls. Ξ_s is *not* computed here — it is a save-point quantity, obtained from +[`compute_node_xi_s!`](@ref) only where it is actually consumed. +""" +function sing_der!(du::Array{ComplexF64,3}, u::Array{ComplexF64,3}, + params::Tuple{ForceFreeStatesControl,Equilibrium.PlasmaEquilibrium, + FourFitVars,ForceFreeStatesInternal,OdeState,IntegrationChunk}, + psieval::Float64) + ctrl, equil, ffit, intr, odet, _ = params + return sing_der!(du, u, ctrl, equil, ffit, intr, odet, psieval) +end + +""" + sing_der!(du, u, ctrl, equil, ffit, intr, odet, psieval) + +Unpacked-argument form of the Euler-Lagrange derivative, using `odet`'s spline hints and +recording q at `psieval` in `odet.q`. Not safe to call concurrently on a shared `odet`; +multi-threaded callers should use [`el_derivatives!`](@ref) with their own hints. +""" +function sing_der!(du::Array{ComplexF64,3}, u::Array{ComplexF64,3}, + ctrl::ForceFreeStatesControl, equil::Equilibrium.PlasmaEquilibrium, ffit::FourFitVars, + intr::ForceFreeStatesInternal, odet::OdeState, psieval::Float64) + odet.q = el_derivatives!(du, u, ctrl.kinetic_factor > 0, equil, ffit, intr, psieval, odet.spline_hint, odet.ffit_hint) + return nothing +end + +""" + el_derivatives!(du, u, kinetic, equil, ffit, intr, psieval, spline_hint, ffit_hint) -> q + +Euler-Lagrange (or, when `kinetic` is true, FKG) derivative kernel: writes du₁/dψ and du₂/dψ at +`psieval` into `du` and returns q there. Holds no state of its own — the two hints are the +caller's interval-search accelerators, so concurrent callers just pass their own. +""" +@with_pool pool function el_derivatives!(du::Array{ComplexF64,3}, u::Array{ComplexF64,3}, + kinetic::Bool, equil::Equilibrium.PlasmaEquilibrium, ffit::FourFitVars, + intr::ModeSpace, psieval::Float64, spline_hint::Base.RefValue{Int}, ffit_hint::Base.RefValue{Int}) + + # Allocate temporary arrays from the pool + Npert = intr.numpert_total + + singfac_vec = acquire!(pool, Float64, Npert) + singfac_mat = reshape(singfac_vec, intr.mpert, intr.npert) + + fmat_lower = acquire!(pool, ComplexF64, Npert, Npert) + kmat = similar!(pool, fmat_lower) + gmat = similar!(pool, fmat_lower) + tmp_mat = similar!(pool, fmat_lower) + + fill!(tmp_mat, zero(ComplexF64)) + u1 = @view(u[:, :, 1]) + u2 = @view(u[:, :, 2]) + du1 = @view(du[:, :, 1]) + du2 = @view(du[:, :, 2]) + + # Compute singfac = 1 / (m - nq) + # Use caller-supplied hint for O(1) interval lookup during sequential ODE integration + q = equil.profiles.q_spline(psieval; hint=spline_hint) + singfac_mat .= 1.0 ./ ((intr.mlow:intr.mhigh) .- q .* (intr.nlow:intr.nhigh)') + + if kinetic + # ---- Kinetic path with pre-computed FKG matrices ---- + # Use the caller's hint, not ffit._hint (shared, racy in the parallel BVP) + # Load FKG sub-matrices (note: reusing fmat_lower/kmat/gmat as workspace) + f0mat = similar!(pool, fmat_lower) + pmat_kin = similar!(pool, fmat_lower) + paat_kin = similar!(pool, fmat_lower) + kkmat_kin = similar!(pool, fmat_lower) + kkaat_kin = similar!(pool, fmat_lower) + r1mat_kin = similar!(pool, fmat_lower) + r2mat_kin = similar!(pool, fmat_lower) + r3mat_kin = similar!(pool, fmat_lower) + gaat_kin = similar!(pool, fmat_lower) + + ffit.f0mats(vec(f0mat), psieval; hint=ffit_hint) + ffit.pmats(vec(pmat_kin), psieval; hint=ffit_hint) + ffit.paats(vec(paat_kin), psieval; hint=ffit_hint) + ffit.kkmats(vec(kkmat_kin), psieval; hint=ffit_hint) + ffit.kkaats(vec(kkaat_kin), psieval; hint=ffit_hint) + ffit.r1mats(vec(r1mat_kin), psieval; hint=ffit_hint) + ffit.r2mats(vec(r2mat_kin), psieval; hint=ffit_hint) + ffit.r3mats(vec(r3mat_kin), psieval; hint=ffit_hint) + ffit.gaats(vec(gaat_kin), psieval; hint=ffit_hint) + + # Build singfac-dependent F̄, K̄, K̄†, Ḡ† matrices (Logan 2015 Appendix C, Eqs C.5-C.11): + # F̄(i,j) = q1*f0*q2 - q1*P - P†'*q2 + R1 + # K̄(i,j) = q1*KK + R2 + # K̄†(i,j) = KK†*q2 + R3 + # where q1 = (m₁ - n*q), q2 = (m₂ - n*q) — direct singfac, NOT 1/(m-nq) as in ideal path + singfac_direct = acquire!(pool, Float64, Npert) + singfac_direct_mat = reshape(singfac_direct, intr.mpert, intr.npert) + singfac_direct_mat .= (intr.mlow:intr.mhigh) .- q .* (intr.nlow:intr.nhigh)' + + # Build F, K, K† with singfac (using fmat_lower, kmat, gmat as workspace for F, K, K†) + kaat_kin = similar!(pool, fmat_lower) # K† matrix + for j in 1:Npert + q2 = singfac_direct[j] + for i in 1:Npert + q1 = singfac_direct[i] + fmat_lower[i, j] = q1 * f0mat[i, j] * q2 - q1 * pmat_kin[i, j] - + conj(paat_kin[j, i]) * q2 + r1mat_kin[i, j] + kmat[i, j] = q1 * kkmat_kin[i, j] + r2mat_kin[i, j] + kaat_kin[i, j] = kkaat_kin[i, j] * q2 + r3mat_kin[i, j] + end + end + # gmat = gaat (already loaded) + gmat .= gaat_kin + + # Kinetic ODE (Logan 2015 Eq 7.46): singfac absorbed into F̄/K̄/K̄†, no explicit Q⁻¹ + # du₁ = F̄⁻¹(u₂ - K̄·u₁) + du1 .= u2 + mul!(tmp_mat, kmat, u1) + du1 .-= tmp_mat + # LU factorize F (non-Hermitian, non-symmetric); direct LAPACK for the same hot-loop reason + _, ipiv2, _ = LAPACK.getrf!(fmat_lower) + LAPACK.getrs!('N', fmat_lower, ipiv2, du1) + + # du₂ = Ḡ†·u₁ + K̄†·du₁ (Logan 2015 Eq C.10-C.11) + mul!(tmp_mat, gmat, u1) + du2 .= tmp_mat + mul!(tmp_mat, kaat_kin, du1) + du2 .+= tmp_mat + + else + # ---- Ideal path ---- + # Evaluate matrix splines at the current psi (hint is the caller's, never shared) + ffit.fmats_lower(vec(fmat_lower), psieval; hint=ffit_hint) + ffit.kmats(vec(kmat), psieval; hint=ffit_hint) + ffit.gmats(vec(gmat), psieval; hint=ffit_hint) + + # See equations 22-24 in Glasser 2016 DCON paper for derivation + # du[1] = - F̄⁻¹ * K̄ * u[1] + F̄⁻¹ * Q⁻¹ * u[2] + du1 .= u2 .* singfac_vec + mul!(tmp_mat, kmat, u1) + du1 .-= tmp_mat + ldiv!(LowerTriangular(fmat_lower), du1) + ldiv!(UpperTriangular(fmat_lower'), du1) + # du[2] = G * u[1] + K̄^† * du[1] = G * u[1] - K̄^† * F̄⁻¹ * K̄ * u[1] + K̄^† * F̄⁻¹ * Q⁻¹ * u[2] + mul!(tmp_mat, gmat, u1) + du2 .= tmp_mat + mul!(tmp_mat, adjoint(kmat), du1) + du2 .+= tmp_mat + # du[1] = - Q⁻¹ * F̄⁻¹ * K̄ * u[1] + Q⁻¹ * F̄⁻¹ * Q⁻¹ * u[2] + du1 .*= singfac_vec + end + return q +end + +""" + compute_node_xi_s!(xi_s, du1, u1, ffit, psieval; kinetic=false, hint=Ref(1)) + +Evaluate Ξ_s = -A⁻¹(B·Ξ′_ψ + C·Ξ_ψ) [Glasser Phys. Plasmas 2016 112506 eq. 18] at `psieval`, +writing into `xi_s`. `du1` and `u1` are the Ξ′_ψ and Ξ_ψ blocks at the same ψ, i.e. slices of a +`el_derivatives!` result and its input state. + +Split out of the derivative kernel because Ξ_s is needed only at saved nodes, not at every +Runge-Kutta stage. Ideal runs factor the Hermitian A by Cholesky; with `kinetic=true` A picks up +non-Hermitian contributions and needs an LU. +""" +@with_pool pool function compute_node_xi_s!(xi_s::AbstractMatrix{ComplexF64}, du1::AbstractMatrix{ComplexF64}, + u1::AbstractMatrix{ComplexF64}, ffit::FourFitVars, psieval::Float64; kinetic::Bool=false, hint::Base.RefValue{Int}=Ref(1)) + + Npert = size(u1, 1) + amat = acquire!(pool, ComplexF64, Npert, Npert) + bmat = similar!(pool, amat) + cmat = similar!(pool, amat) + tmp_mat = similar!(pool, amat) + + ffit.amats(vec(amat), psieval; hint=hint) + ffit.bmats(vec(bmat), psieval; hint=hint) + ffit.cmats(vec(cmat), psieval; hint=hint) + + # Solve bmat = A⁻¹ * bmat, cmat = A⁻¹ * cmat in-place + if kinetic + _, ipiv, _ = LAPACK.getrf!(amat) + LAPACK.getrs!('N', amat, ipiv, bmat) + LAPACK.getrs!('N', amat, ipiv, cmat) + else + LAPACK.potrf!('U', amat) + LAPACK.potrs!('U', amat, bmat) + LAPACK.potrs!('U', amat, cmat) + end + + mul!(tmp_mat, bmat, du1) + xi_s .= .-tmp_mat + mul!(tmp_mat, cmat, u1) + xi_s .-= tmp_mat + return xi_s +end + diff --git a/src/ForceFreeStates/ForceFreeStates.jl b/src/ForceFreeStates/ForceFreeStates.jl index 8981a141a..1dc93e776 100644 --- a/src/ForceFreeStates/ForceFreeStates.jl +++ b/src/ForceFreeStates/ForceFreeStates.jl @@ -22,19 +22,35 @@ using Printf using DoubleFloats import StaticArrays: @MMatrix -# Include all necessary files -include("ForceFreeStatesStructs.jl") -include("Resist.jl") -include("EulerLagrange.jl") -include("Sing.jl") -include("ResistEval.jl") +# Types with cross-subsystem consumers, loaded before the code that dispatches on them +include("Surfaces/Types.jl") +include("CoreTypes.jl") +include("Riccati/Types.jl") include("Fourfit.jl") +include("Matching/DeltaPrime.jl") + +include("EulerLagrange.jl") + +# Singular-surface machinery: finding/filtering, Frobenius asymptotics, GGJ coefficients +include("Surfaces/Finding.jl") +include("Surfaces/Asymptotics.jl") +include("Surfaces/Resist.jl") +include("Surfaces/ResistEval.jl") + +# Outer<->inner resistive matching +include("Matching/ResonantMatch.jl") + include("FixedKineticMatrices.jl") include("Kinetic.jl") include("FixedBoundaryStability.jl") include("Utils.jl") include("Free.jl") -include("Riccati.jl") + +# Chunked fundamental-matrix (Riccati/STRIDE) driver +include("Riccati/Propagators.jl") +include("Riccati/Crossings.jl") +include("Riccati/DeltaPrimeBVP.jl") +include("Riccati/Driver.jl") # RDCON outer-region singular Galerkin Δ′ solver (gal_solve port) include("Galerkin/GalerkinStructs.jl") diff --git a/src/ForceFreeStates/ForceFreeStatesStructs.jl b/src/ForceFreeStates/ForceFreeStatesStructs.jl deleted file mode 100644 index 79735b895..000000000 --- a/src/ForceFreeStates/ForceFreeStatesStructs.jl +++ /dev/null @@ -1,680 +0,0 @@ -""" - ModeSpace - -Supertype for objects that carry the resolved (m, n) mode space — `mlow`, `mhigh`, `mpert`, -`nlow`, `nhigh`, `npert`, `numpert_total`. Both the solve-time scratch -[`ForceFreeStatesInternal`](@ref) and the published [`ForceFreeStatesResult`](@ref) are -`ModeSpace`s, so kernels that need nothing but the mode indexing (`el_derivatives!`, -`materialize_derivative_stores!`, `build_kinetic_metric_matrices`) accept either. -""" -abstract type ModeSpace end - -""" - SingType - -A mutable struct holding data related to the singular surfaces in the equilibrium. - -## Fields - - - `psifac::Float64` - Normalized flux coordinate at the singular surface - - `rho::Float64` - Radial coordinate (√ψ) - - `m::Vector{Int}` - Poloidal mode number(s) - - `n::Vector{Int}` - Toroidal mode number(s) - - `q::Float64` - Safety factor (= m/n) - - `q1::Float64` - Derivative of safety factor with respect to ψ - - `delta_prime::Vector{ComplexF64}` - **STUB (not physically valid)**. Per-surface ca-based Δ' estimate retained for future work / debugging only. The physically valid Δ' is `ForceFreeStatesInternal.delta_prime_matrix`, computed via the STRIDE global BVP (Glasser 2018 PoP 25, 032501). Do not use this field for tearing-stability analysis; do not expect agreement with `delta_prime_matrix`. - - `delta_prime_col::Matrix{ComplexF64}` - **STUB (not physically valid)**. Per-surface ca-based Δ' column retained for future work / debugging only. Shape (numpert_total × n_res_modes); `delta_prime_col[j, i] = (ca_r[j,ipert_res_i,2] - ca_l[j,ipert_res_i,2]) / (4π²·psio)`. The diagonal element matches the (also stubbed) `delta_prime[i]`. Only populated for the Riccati/parallel FM paths. The physically valid Δ' is `ForceFreeStatesInternal.delta_prime_matrix`; this field exists for future development on intra-surface coupling diagnostics, not for production use. -""" -@kwdef mutable struct SingType - psifac::Float64 = 0.0 - rho::Float64 = 0.0 - m::Vector{Int} = Int[] - n::Vector{Int} = Int[] - q::Float64 = 0.0 - q1::Float64 = 0.0 - delta_prime::Vector{ComplexF64} = ComplexF64[] - delta_prime_col::Matrix{ComplexF64} = Matrix{ComplexF64}(undef, 0, 0) - ua_left::Array{ComplexF64,3} = Array{ComplexF64}(undef, 0, 0, 0) # asymptotic basis at left inner-layer boundary - ua_right::Array{ComplexF64,3} = Array{ComplexF64}(undef, 0, 0, 0) # asymptotic basis at right inner-layer boundary - psi_ua_left::Float64 = 0.0 # ψ where ua_left was evaluated (left inner-layer boundary) - psi_ua_right::Float64 = 0.0 # ψ where ua_right was evaluated (right inner-layer boundary) - restype::Any = nothing # ResistGeometry from ResistEval.jl (populated by resist_eval_all!); typed `Any` to avoid a cross-file type reference -end - -""" - SingAsymptotics - -A struct containing asymptotic expansion data for ideal ForceFreeStates calculations at a singular surface. -This data is computed on-demand during singular surface crossings in `cross_ideal_singular_surf!`. - -## Fields - - - `alpha::Vector{ComplexF64}` - Resonant matrix eigenvalues - - `r1::Vector{Int}` - Resonant indices along first index - - `r2::Vector{Int}` - Resonant indices along second index - - `n1::Vector{Int}` - Nonresonant indices along first index - - `n2::Vector{Int}` - Nonresonant indices along second index - - `power::Vector{ComplexF64}` - Power series coefficients - - `vmat::Array{ComplexF64,4}` - Power series of V matrix for asymptotic analysis - - `mmat::Array{ComplexF64,4}` - Power series of M matrix for asymptotic analysis - - `m0mat::Matrix{ComplexF64}` - Zeroth order M matrix projected onto resonant subspace -""" -struct SingAsymptotics - sing_order::Int - alpha::Vector{ComplexF64} - r1::Vector{Int} - r2::Vector{Int} - n1::Vector{Int} - n2::Vector{Int} - power::Vector{ComplexF64} - vmat::Array{ComplexF64,4} - mmat::Array{ComplexF64,4} - m0mat::Matrix{ComplexF64} -end - -""" - IntegrationChunk - -A struct representing a region of integration in the Euler-Lagrange solver. - -## Fields - - - `psi_start::Float64` - Starting ψ coordinate for this integration region - - `psi_end::Float64` - Ending ψ coordinate for this integration region - - `needs_crossing::Bool` - Whether a rational surface crossing is needed after this chunk - - `ising::Int` - Index of the singular surface associated with this chunk (0 if none) - - `direction::Int` - Integration direction: +1 forward (axis→edge), -1 backward (edge→axis). - For `direction=-1` chunks, `psi_start` < `psi_end` but integration proceeds from `psi_end` - toward `psi_start`. The resulting propagator maps state at `psi_end` → state at `psi_start`. - Used in bidirectional parallel FM to produce well-conditioned crossing-chunk propagators: - solutions that grow exponentially forward (toward a singularity) decay when integrated - backward, so the backward propagator is well-conditioned. -""" -@kwdef struct IntegrationChunk - psi_start::Float64 - psi_end::Float64 - needs_crossing::Bool - ising::Int = 0 - direction::Int = 1 # +1 forward, -1 backward -end - -""" - ChunkPropagator - -Fundamental matrix for one integration chunk, stored as two N×N×2 solution blocks. -Represents the propagator Φ(ψ₂,ψ₁) computed by integrating the EL ODE from two -identity-block initial conditions: - - - `block_upper_ic`: result of integrating with IC = (I_N, 0_N) (U₁ = I, U₂ = 0) - - `block_lower_ic`: result of integrating with IC = (0_N, I_N) (U₁ = 0, U₂ = I) - -Applying the propagator to the current state `u_prev`: - -u₁_new = block_upper_ic[:,:,1] · u₁_prev + block_lower_ic[:,:,1] · u₂_prev -u₂_new = block_upper_ic[:,:,2] · u₁_prev + block_lower_ic[:,:,2] · u₂_prev - -Since each chunk starts from a bounded identity IC (rather than the accumulated state), -exponential growth within a chunk does not affect the conditioning of the overall -assembly. This enables `Threads.@threads` parallel integration across all chunks. -""" -struct ChunkPropagator - block_upper_ic::Array{ComplexF64,3} # shape (N, N, 2) — result from IC = (I, 0) - block_lower_ic::Array{ComplexF64,3} # shape (N, N, 2) — result from IC = (0, I) -end -ChunkPropagator(N::Int) = ChunkPropagator(zeros(ComplexF64, N, N, 2), zeros(ComplexF64, N, N, 2)) - -""" -DebugSettings - -A mutable struct containing settings for debugging and benchmarking output. - -## Fields - - - `output_benchmark_data::Bool` - Flag to output benchmark data for comparison between codes - - `gal_basis_output::Bool` - Write the raw Galerkin outer-region basis functions (per-interval, unconstrained at the rationals) under `GalerkinIntegration/Basis/`. Solver internals for development verification, not physics output. -""" -@kwdef mutable struct DebugSettings - output_benchmark_data::Bool = false - gal_basis_output::Bool = false -end - -""" - ForceFreeStatesInternal - -A mutable struct holding internal state variables for stability calculations. - -## Fields - - - `dir_path::String` - Directory path for input/output files - - `mlow::Int` - Lowest poloidal mode number - - `mhigh::Int` - Highest poloidal mode number - - `mpert::Int` - Number of poloidal modes (mhigh - mlow + 1) - - `nlow::Int` - Lowest toroidal mode number, resolved from `ctrl.nn_low`/`nn_high` - - `nhigh::Int` - Highest toroidal mode number, resolved from `ctrl.nn_low`/`nn_high` - - `npert::Int` - Number of toroidal modes (nhigh - nlow + 1) - - `numpert_total::Int` - Total number of modes (mpert × npert) - - `msing::Int` - Number of ideal singular surfaces - - `kmsing::Int` - Number of kinetic singular surfaces (det(F̄) near-zeros) - - `sing::Vector{SingType}` - Vector of ideal singular surface data - - `kinsing::Vector{SingType}` - Vector of kinetic singular surface data - - `kinsing_scan_psi::Vector{Float64}` - ψ grid used by `find_kinetic_singular_surfaces!` for the cond(F̄) scan (empty unless the finder has run) - - `kinsing_scan_cond::Vector{Float64}` - cond(F̄) values on that grid; the finder locates peaks that exceed `kinsing_scan_threshold` - - `kinsing_scan_threshold::Float64` - Threshold on cond(F̄) used to accept a peak as a kinetic singular surface - - `psilim::Float64` - Flux limit for integration - - `qlim::Float64` - Safety factor at psilim - - `q1lim::Float64` - Safety factor derivative at psilim - - `wall_settings::Vacuum.WallShapeSettings` - Wall shape settings for vacuum calculations -""" -@kwdef mutable struct ForceFreeStatesInternal <: ModeSpace - dir_path::String = "" - mlow::Int = 0 - mhigh::Int = 0 - mpert::Int = 0 - nlow::Int = 0 - nhigh::Int = 0 - npert::Int = 0 - numpert_total::Int = 0 - msing::Int = 0 - kmsing::Int = 0 - sing::Vector{SingType} = SingType[] - kinsing::Vector{SingType} = SingType[] - kinsing_scan_psi::Vector{Float64} = Float64[] - kinsing_scan_cond::Vector{Float64} = Float64[] - kinsing_scan_threshold::Float64 = 0.0 - psilim::Float64 = 0.0 - psilow::Float64 = 0.0 # lower integration bound; raised above the axis by sing_min! when qlow > qmin (RDCON gal) - qlim::Float64 = 0.0 - q1lim::Float64 = 0.0 - debug_settings::DebugSettings = DebugSettings() - wall_settings::Vacuum.WallShapeSettings = Vacuum.WallShapeSettings() - """ - Inter-surface Δ' matrix of shape (msing × msing) in PEST3 convention. - Computed by `compute_delta_prime_matrix!` (parallel FM path only) using the STRIDE - global BVP with vacuum coupling. The deltap linear combination is applied to the - raw 2msing×2msing BVP solution to produce the PEST3-compatible tearing parameter. - """ - delta_prime_matrix::Matrix{ComplexF64} = Matrix{ComplexF64}(undef, 0, 0) - - """ - Edge coil-response matrix of shape (2msing × numpert_total). Column k is the resonant - small-solution response at each surface side to a unit source on edge poloidal mode k, - built by imposing the Eq. (37) rpec edge boundary condition on the Riccati BVP - (`_solve_bvp_edge_coil`). Empty unless the S-axis vacuum-edge BVP was assembled. - """ - delta_coil::Matrix{ComplexF64} = Matrix{ComplexF64}(undef, 0, 0) - - """ - Raw 2msing × 2msing outer-region matching matrix `D'` from the STRIDE global - BVP, in the side-major ordering `[L_s1, R_s1, L_s2, R_s2, …, L_sm, R_sm]` - (left vs right of each singular surface, interleaved surface-by-surface). - This is the Pletzer–Dewar 1991 outer-region matrix before parity rotation, - and is stored byte-compatibly with the Fortran `rdcon/gal.f::gal_write_delta` - convention (top 2msing×2msing block of `delta_gw.dat`). The PEST3 Δ' matrix - stored in `delta_prime_matrix` is the odd-parity tearing projection of this - raw matrix; the even-parity A' and off-parity B', Γ' blocks are recovered - via `pest3_decompose(dp_raw)` — needed for the full det(D' − D(γ)) = 0 - eigenvalue problem with Glasser stabilization. - - Empty unless the Riccati integrator was used. No ½ prefactor is applied (matches - Fortran rdcon; Pletzer–Dewar paper multiplies by ½). - """ - delta_prime_raw::Matrix{ComplexF64} = Matrix{ComplexF64}(undef, 0, 0) -end - -""" - ForceFreeStatesControl - -An immutable struct containing 'ForceFreeStates' parameters set by the user in -gpec.toml. - -## Fields - - - `verbose::Bool` - Enable verbose output - - `local_stability_flag::Bool` - Enable local stability analysis (`D_I` and ballooning) - - `vac_flag::Bool` - Enable vacuum region calculation - - `mthvac::Int` - Number of vacuum poloidal grid points (corresponds to `mtheta` in VacuumInput) - - `nzvac::Int` - Number of vacuum toroidal grid points (corresponds to `nzeta` in VacuumInput3D) - - `sing_start::Int` - Start integration at the `sing_start`-th singular surface - - `nn_low::Int` - Lower bound for toroidal modes - - `nn_high::Int` - Upper bound for toroidal modes - - `delta_mlow::Int` - Expands lower bound of Fourier harmonics by delta_mlow - - `delta_mhigh::Int` - Expands upper bound of Fourier harmonics by delta_mhigh - - `nstep::Int` - Maximum number of integration steps (not yet implemented) - - `ksing::Int` - Singular surface handling parameter - - `eulerlagrange_tolerance::Float64` - Relative tolerance for ODE integration of Euler-Lagrange equations - - `ucrit::Float64` - Critical value of unorm ratio to trigger solution normalization. In the standard path it triggers Gaussian reduction; in the Riccati path it triggers `renormalize_riccati_inplace!`. Default `1e4` empirically keeps max(|U₁|, |U₂|) in O(1)–O(10⁴) over the integration domain on DIII-D / Solovev sweeps; lower triggers excess renorms without accuracy gain, higher risks overflow before the next renorm. - - `numsteps_init::Int` - Initial array size for ODE data storage - - `numunorms_init::Int` - Initial array size for solution normalization data - - `singfac_min::Float64` - Fractional distance from rational q at which ideal jump condition is enforced - - `set_psilim_via_dmlim::Bool` - Truncate the integration domain at `(last_rational_q + dmlim) / n` rather than at `qhigh` / `psihigh`. Fortran STRIDE found that truncating ~20 % above the outermost rational (`dmlim = 0.2`) avoids a numerical kink instability in δW that appears when the integration ends too close to or just below a rational surface. **For diverted equilibria where q → ∞ at the separatrix** (e.g. DIII-D geqdsks, the bulk of production use) this costs negligible physical domain because rationals get arbitrarily dense near the LCFS — `set_psilim_via_dmlim = true` is the safe and recommended default. **For limited circular / analytical equilibria with finite q at the edge** (Solovev, LAR scans), rationals are sparse and 20 % above the last rational chops off too much edge, so set `set_psilim_via_dmlim = false` and let `qhigh` / `psihigh` control the truncation. Multi-`n` runs are not supported by this truncation (the "outermost rational + dmlim / n" depends on which `n`); when `set_psilim_via_dmlim = true` with `nn_low != nn_high`, `sing_lim!` warns and falls back to `qhigh` / `psihigh`. Default `true`. - - `dmlim::Float64` - Distance beyond last rational surface (normalised ∈ [0,1) in units of 1/n). Only used when `set_psilim_via_dmlim` is true. Fortran STRIDE convention is 0.2 (truncate 20 % of one rational-surface spacing above the last surface), retained here. - - `sing_order::Int` - Order of singular layer (Frobenius) expansion at rational surfaces. Default 6 (Fortran STRIDE convention for Δ' calculations; lower values trade accuracy for speed). - - `qhigh::Float64` - Integration terminated at q limit determined by minimum of qhigh and qa from equil - - `kinetic_source::String` - Kinetic matrix source: "fixed" (X-shaped test matrices scaled by kinetic_factor relative to ideal matrix Frobenius norms; Ak, Dk, Hk Hermitian, Bk, Ck, Ek non-Hermitian), "calculated" (PENTRC — not yet implemented) - - `kinetic_factor::Float64` - Dimensionless scaling factor for kinetic matrices. Zero (the default) disables the kinetic path; any positive value enables it and scales the kinetic matrices: when kinetic_source="fixed", scales X-shaped test matrices relative to ideal matrix norms; when kinetic_source="calculated", applied as uniform post-hoc multiplier to W and T components. - - `qlow::Float64` - Integration terminated at q limit determined by minimum of qlow and q0 from equil - - `psiedge::Float64` - If less than psilim, records a dW(ψ) diagnostic scan over [psiedge, psilim] on odet.edge_scan. The integration domain (psilim) is always controlled by qhigh / psihigh and is not modified by this scan (unless `truncate_at_dW_peak=true`, see caveats below). - - `truncate_at_dW_peak::Bool` - When `true` and `psiedge < psilim`, the edge-dW scan's peak location is adopted as the new physical plasma edge — `intr.psilim`/`intr.qlim`/`odet.u` are pulled back to the peak, AND the FM Δ' chunks/propagators are made self-consistent with the new boundary (the chunk that straddles the peak is rebuilt + re-integrated; any chunks past the peak are dropped). This reproduces the spirit of the original ode_record_edge heuristic from Fortran STRIDE while keeping Δ' and δW well-defined at the new boundary. The Δ' metric is still physically dependent on where the peak falls in the edge band, so use this flag deliberately when you mean to scan against the peak-defined edge (e.g. for studying edge-mode regimes); leave at `false` (default) for the full-domain Δ' at `qhigh` / `psihigh` / `dmlim`. - - `diagnose::Bool` - Enable diagnostic output (not yet implemented) - - `diagnose_ca::Bool` - Enable asymptotic coefficient diagnostics (not yet implemented) - - `write_outputs_to_HDF5::Bool` - Write results to HDF5 format - - `HDF5_filename::String` - Name of HDF5 output file - - `save_interval::Int` - Save every Nth ODE step (1=all, 10=every 10th). Always saves near rational surfaces. (Same as `euler_step` in the Fortran) - - `force_termination::Bool` - Terminate after force-free states (skip perturbed equilibrium calculations) - - `integrator::String` - Which formalism integrates the Euler-Lagrange system. `"forward"` sweeps the plasma serially with Gaussian reduction and returns `u_store` / `du_store` / `xi_s_store` dense in the axis (EL) basis — the only convention PerturbedEquilibrium and FieldReconstruction consume correctly, and the only path that supports `kinetic_factor > 0`. `"riccati"` (default) runs the chunked fundamental-matrix propagator driver (Glasser 2018 Phys. Plasmas 25, 032507): chunks are integrated independently from identity initial conditions and assembled serially with Riccati-style crossings, which is the only way to obtain the singular-surface Δ' matrix for the tearing-mode solvers downstream, but leaves `u_store` as sparse chunk-endpoint Riccati states, so dense ξ profiles are unavailable. `"galerkin"` solves the same Euler-Lagrange system variationally instead of by radial ODE integration — the RDCON outer-region singular Galerkin method (Glasser, Wang & Park 2016 Phys. Plasmas 23, 112506), which discretizes the displacement on packed Hermite-cubic elements and solves one global banded system — producing the resistive Δ′ matrix and, when `gal_match_flag` is set, the RPEC inner-layer-matched ξ; it computes its own vacuum response and returns no free-boundary energies, and does not support `kinetic_factor > 0`. Requires `singfac_min != 0` for `"riccati"`. - - `nchunks::Int` - Target number of Riccati integration chunks. `0` (the default) derives the count from problem structure alone: `max(2·msing + 3, 8·(msing + 1) + msing)`, enough sub-chunks per segment to keep the accumulated propagator products well-conditioned. An explicit value below `2·msing + 3` is clamped up with a warning. Chunk sizing never consults `Threads.nthreads()`, so Riccati outputs are identical whatever thread count `julia -t` provides; threads only change wall-clock. - - `extended_precision_bvp::Bool` - When `true` (default), promote the Δ' BVP linear system to `Complex{Double64}` (~31 digits) for the LU solve and PEST3 combination. Guards against catastrophic cancellation in the PEST3 four-term combination (dp_raw entries can be 10⁴–10⁵× larger than the result; the imaginary part of off-diagonal Δ' is particularly sensitive). Disabling (`false`) saves ~1.5–2× the BVP solve time but on DIIID-class equilibria the imaginary Δ' components can drift by factors of 2–5×; only disable for performance experiments on cases where Float64 has been validated against Double64. -""" -@kwdef struct ForceFreeStatesControl - verbose::Bool = true - local_stability_flag::Bool = false - vac_flag::Bool = false - mthvac::Int = 480 - nzvac::Int = 1 - sing_start::Int = 0 - nn_low::Int = 0 - nn_high::Int = 0 - delta_mlow::Int = 0 - delta_mhigh::Int = 0 - nstep::Int = typemax(Int) - ksing::Int = -1 - eulerlagrange_tolerance::Float64 = 1e-8 - ucrit::Float64 = 1e4 - numsteps_init::Int = 4000 - numunorms_init::Int = 100 - singfac_min::Float64 = 1e-4 # Matches Fortran STRIDE; required nonzero for the Riccati path. - set_psilim_via_dmlim::Bool = true # Safe default for diverted equilibria (most production use); set false for limited/analytical (LAR, Solovev). Auto-skipped for multi-n. See docstring. - dmlim::Float64 = 0.2 - sing_order::Int = 6 - qhigh::Float64 = 1e3 - kinetic_source::String = "fixed" - kinetic_factor::Float64 = 0.0 - qlow::Float64 = 0.0 - psiedge::Float64 = 0.99 - truncate_at_dW_peak::Bool = false # Edge-dW peak becomes new physical edge; Δ' BVP made self-consistent. See docstring. - diagnose::Bool = false - diagnose_ca::Bool = false - write_outputs_to_HDF5::Bool = true - HDF5_filename::String = "gpec.h5" - save_interval::Int = 3 - force_termination::Bool = false - integrator::String = "riccati" # Default: unlocks SingularSurfaces/Delta_prime_matrix (STRIDE BVP Δ′ matrix) used by SLAYER/GGJ downstream. Use "forward" for dense ξ (PerturbedEquilibrium) or kinetic runs. - nchunks::Int = 0 # Riccati chunk-count target; 0 = auto (derived from msing alone, never from Threads.nthreads()). - extended_precision_bvp::Bool = true # Promote Δ' BVP to Complex{Double64}; default on (Float64 drifts the imaginary Δ' by 2–5× on DIIID-class cases). - - # --- RDCON outer-region Galerkin Δ′ solver (gal_solve port); selected by integrator = "galerkin" --- - gal_solver::String = "LU" # "LU" (zgbtrf/zgbtrs) or "cholesky" (zpbtrf/zpbtrs) - gal_nx::Int = 256 # elements per interval between singular surfaces - gal_nq::Int = 6 # Gauss-Lobatto quadrature order per element - gal_pfac::Float64 = 0.001 # grid packing ratio near singular surfaces - gal_dx0::Float64 = 5e-4 # resonant-element integration truncation distance (×1/|n q'|) - gal_dx1::Float64 = 1e-3 # resonant-element size (×1/|n q'|) - gal_dx2::Float64 = 1e-3 # extension-element size (×1/|n q'|) - gal_cutoff::Int = 10 # # of elements carrying the large solution as driving term - gal_tol::Float64 = 1e-10 # resonant-quadrature (QuadGK) tolerance - gal_gnstep::Int = 20000 # max resonant-quadrature evaluations (QuadGK maxevals in gal_resonant!) - gal_dx1dx2_flag::Bool = true # enable special dx1/dx2 treatment for resonant/extension elements - gal_sing_order::Int = 6 # base power-series order for the Galerkin singular asymptotics - gal_sing_order_ceiling::Bool = true # auto-raise order by ceil(2·Re(α)) per surface (high Mercier index) - gal_rpec_flag::Bool = false # append mpert coil-response columns to the Δ′ solve (RDCON rpec_flag): unit boundary sources whose plasma response is recorded; needed for the driven (resistive perturbed-equilibrium) Δ_gw - gal_edge_onesided::Bool = false # pack the two end intervals one-sided toward their single rational end (vs the Fortran symmetric "both" pack); avoids the fine edge cell that inflates cond(A). Default false = faithful to gal.f. - # --- DRIVEN (RPEC) outer↔inner asymptotic matching (rmatch match_rpec port) --- - gal_match_flag::Bool = false # enable the RPEC inner-layer matching: solve the coil-driven matched ξ(ψ) from the gal Δ′ + the inner-layer Δ(Q). Requires gal_rpec_flag=true. - gal_ideal_flag::Bool = false # within the match, build the IDEAL solution: skip the inner-layer Δ, use bare coil columns (cout=0). Mirrors Fortran rmatch coil%ideal_flag (the EL reference). eta/rho/rotation ignored. - gal_inner_solver::String = "ray" # inner-layer Δ backend for the match: "ray" (rotated-contour collocation, certified Δ at the optimal θ = arg(Q)/4; robust for |Q| ≳ 1) or "galerkin" (Hermite-cubic inps; drifts for |Q| ≳ 1) - # --- Inner-layer "galerkin" backend knobs (used only when gal_inner_solver = "galerkin") --- - gal_inner_xfac::Float64 = 10.0 # asymptotic-matching radius multiplier (inps_xfac: xmax × 10) - gal_inner_nx::Int = 1280 # inner-layer grid cells (128 · xfac in the reference) - gal_inner_nq::Int = 5 # quadrature order per cell - gal_inner_cutoff::Int = 5 # cells carrying the large solution as driving term - gal_inner_kmax::Int = 8 # large-x asymptotic series order (↔ order_pow) - gal_eta::Vector{Float64} = Float64[] # per-surface resistivity η (length msing, core→edge); Fortran rmatch `eta` - gal_rho::Vector{Float64} = Float64[] # per-surface mass density ρ [kg/m³] (length msing, core→edge); Fortran rmatch `massden` - gal_rotation::Vector{Float64} = Float64[] # per-surface rotation frequency f [Hz] (length msing, core→edge); forced eigenvalue γ_s = 2πi·n·f. Fortran rmatch `rotation` - gal_gamma::Float64 = 5 / 3 # ratio of specific heats Γ for the resistive-layer coefficients (resist_eval G term) - fixed_axis::Bool = false -end - -@kwdef mutable struct FourFitVars{S<:CubicSeriesInterpolant,Opts<:NamedTuple} - mpert::Int - numpert_total::Int # = mpert * npert (total series count per matrix = numpert_total^2) - - # Complex-valued CubicSeriesInterpolant for stability matrices - # Each matrix is flattened to (npsi × numpert_total^2) series - # FastInterpolations natively supports complex values: CubicSeriesInterpolant{Tgrid, Tvalue} - # NOTE: itp_opts must precede interpolant fields — @kwdef evaluates defaults in declaration order - itp_opts::Opts = (; extrap=ExtendExtrap()) - - amats::S = _empty_series_interp_complex(numpert_total^2, itp_opts) - bmats::S = _empty_series_interp_complex(numpert_total^2, itp_opts) - cmats::S = _empty_series_interp_complex(numpert_total^2, itp_opts) - # `dmats_prim`, `emats_prim` are the pre-Schur-reduction geometric forms - # (D = χ₁·(g23 + q·g33·m/n); E = (-χ₁/n)·(q'·χ₁·g33 - 2π·i·χ₁·g31·singfac + jθ·I)). - # The `_prim` suffix follows `fmats_prim`. Downstream kinetic FKG Schur complements - # consume these primitive forms; the alternate singular-layer path that would need - # kinetic-added overwrites of D and E is not implemented here (see Kinetic.jl). - dmats_prim::S = _empty_series_interp_complex(numpert_total^2, itp_opts) - emats_prim::S = _empty_series_interp_complex(numpert_total^2, itp_opts) - hmats::S = _empty_series_interp_complex(numpert_total^2, itp_opts) - fmats_lower::S = _empty_series_interp_complex(numpert_total^2, itp_opts) - fmats_prim::S = _empty_series_interp_complex(numpert_total^2, itp_opts) # primitive F before Schur complement (for kinetic) - fmats_gal::S = _empty_series_interp_complex(numpert_total^2, itp_opts) # reduced Hermitian F̄ (un-factored) for the Galerkin solver - kmats::S = _empty_series_interp_complex(numpert_total^2, itp_opts) - gmats::S = _empty_series_interp_complex(numpert_total^2, itp_opts) - - # Ideal A,B,C splines preserved before kinetic overwrite - amats_ideal::S = _empty_series_interp_complex(numpert_total^2, itp_opts) - bmats_ideal::S = _empty_series_interp_complex(numpert_total^2, itp_opts) - cmats_ideal::S = _empty_series_interp_complex(numpert_total^2, itp_opts) - - # Kinetic energy matrix splines: 6 components (A,B,C,D,E,H perturbations) - kwmats::Vector{S} = [_empty_series_interp_complex(numpert_total^2, itp_opts) for _ in 1:6] - # Kinetic torque matrix splines: 6 components - ktmats::Vector{S} = [_empty_series_interp_complex(numpert_total^2, itp_opts) for _ in 1:6] - - kinetic_populated::Bool = false # set by make_kinetic_matrix; the solution then obeys the FKG ODE, not the ideal EL relation - - # Pre-computed FKG kinetic matrices (populated by make_kinetic_matrix) - f0mats::S = _empty_series_interp_complex(numpert_total^2, itp_opts) - pmats::S = _empty_series_interp_complex(numpert_total^2, itp_opts) - paats::S = _empty_series_interp_complex(numpert_total^2, itp_opts) - kkmats::S = _empty_series_interp_complex(numpert_total^2, itp_opts) - kkaats::S = _empty_series_interp_complex(numpert_total^2, itp_opts) - r1mats::S = _empty_series_interp_complex(numpert_total^2, itp_opts) - r2mats::S = _empty_series_interp_complex(numpert_total^2, itp_opts) - r3mats::S = _empty_series_interp_complex(numpert_total^2, itp_opts) - gaats::S = _empty_series_interp_complex(numpert_total^2, itp_opts) - - # Pre-allocated evaluation buffer for matrix output - _mat_out::Matrix{ComplexF64} = Matrix{ComplexF64}(undef, numpert_total, numpert_total) - - # Shared hint for sequential evaluation (all splines evaluated at same psi) - _hint::Base.RefValue{Int} = Ref(1) - - # Jacobian Fourier band ψ-spline (2·mpert−1 conjugate-symmetric coefficients per surface), - # used to assemble the power-normalization matrix N in Free.jl - jmats::S = _empty_series_interp_complex(2 * mpert - 1, itp_opts) -end - -# Helper to create empty complex series interpolant for default initialization -function _empty_series_interp_complex(n_series::Int) - xs = collect(range(0.0, 1.0; length=5)) - Y = zeros(ComplexF64, 5, n_series) - return cubic_interp(xs, Series(Y)) -end - -function _empty_series_interp_complex(n_series::Int, itp_opts::NamedTuple) - xs = collect(range(0.0, 1.0; length=5)) - Y = zeros(ComplexF64, 5, n_series) - return cubic_interp(xs, Series(Y); itp_opts...) -end - -# Convenience constructor -FourFitVars(mpert::Int, numpert_total::Int) = FourFitVars(; mpert, numpert_total) - -""" - FreeBoundaryResult - -Result of the free-boundary calculation, returned by `free_run`. All matrices are in the ξ Fourier -basis and are `numpert_total × numpert_total`; the energies are generalized (W, N) pencil values, -power-normalized and invariant to the working (Jacobian) coordinate. - -## Fields - - - `wt::Matrix{ComplexF64}` - Eigenvector matrix of W·v = λ·N·v. Columns are eigenmodes sorted most-unstable first, normalized to unit power norm v†·N·v = 1. - - `wt0::Matrix{ComplexF64}` - Total-energy matrix W = wp + wv before diagonalisation - - `wp::Matrix{ComplexF64}` - Plasma energy matrix - - `wv::Matrix{ComplexF64}` - Vacuum energy matrix, singfac-scaled at `qlim` - - `ep::Vector{ComplexF64}` - Plasma energy per eigenmode (power quotient v†·wp·v with v†·N·v = 1) - - `ev::Vector{ComplexF64}` - Vacuum energy per eigenmode (power quotient v†·wv·v with v†·N·v = 1) - - `et::Vector{ComplexF64}` - Total energy eigenvalues of the pencil (W, N); et = ep + ev per mode - - `n_tor_idx::Vector{Int}` - 0-based toroidal mode number index of each sorted eigenvalue - - `vacuum_eigenvalue::Float64` - Least stable (minimum) eigenvalue of the pencil (wv, N), clamped to zero - - `plasma_pts`, `wall_pts::Matrix{Float64}` - Cartesian (x, y, z) surface coordinates, `numpoints × 3`, retained for HDF5 output -""" -struct FreeBoundaryResult - wt::Matrix{ComplexF64} - wt0::Matrix{ComplexF64} - wp::Matrix{ComplexF64} - wv::Matrix{ComplexF64} - ep::Vector{ComplexF64} - ev::Vector{ComplexF64} - et::Vector{ComplexF64} - n_tor_idx::Vector{Int} - vacuum_eigenvalue::Float64 - plasma_pts::Matrix{Float64} - wall_pts::Matrix{Float64} -end - -""" -EdgeScanState - -Holds the state and results for the edge dW stability scan over ψ ∈ [psiedge, psilim]. -Initialized and populated by `findmax_dW_edge!`; results written to HDF5 under `EdgeScan/`. -The energies are generalized (W, N) pencil values: power-normalized and invariant to the -working (Jacobian) coordinate (see `power_norm_matrix!`). - -## Fields - - - `wvmat` - Precomputed wv matrix spline (raw, no singfac); singfac applied analytically in `free_compute_total`. - - `wv_hint::Base.RefValue{Int}` - Search hint for wvmat spline (different grid from equilibrium profiles). - - `psi, q` - ψ and q values at each edge scan step. - - `total_eigenvalue, plasma_energy, vacuum_energy, vacuum_eigenvalue` - Power-normalized energy components at each step (NaN for steps where the wp solve was singular). These drive the truncation choice and are written to `EdgeScan/`. -""" -@kwdef mutable struct EdgeScanState - numpert_total::Int - N_edge::Int - - # Vacuum matrix spline and evaluation infrastructure - wvmat::CubicSeriesInterpolant{Float64,ComplexF64} = _empty_series_interp_complex(numpert_total^2) - wv_hint::Base.RefValue{Int} = Ref(1) - - # Scan results (written to HDF5 under EdgeScan/; NaN where free_compute_total raised SingularException) - psi::Vector{Float64} = Vector{Float64}(undef, N_edge) - q::Vector{Float64} = Vector{Float64}(undef, N_edge) - total_eigenvalue::Vector{ComplexF64} = fill(complex(NaN), N_edge) - plasma_energy::Vector{ComplexF64} = fill(complex(NaN), N_edge) - vacuum_energy::Vector{ComplexF64} = fill(complex(NaN), N_edge) - vacuum_eigenvalue::Vector{Float64} = fill(NaN, N_edge) -end - -EdgeScanState(numpert_total::Int, N_edge::Int) = EdgeScanState(; numpert_total, N_edge) - -""" -OdeState - -A mutable struct to hold the state of the ODE solver used by the ForceFreeStates integration routines. -This struct stores configuration parameters used to allocate arrays, the evolving stored -solution during integration, diagnostic arrays used for normalization / Gaussian reduction, -and a small set of temporary matrices and factors used to compute singular-layer corrections. - -## Fields - - - `numpert_total::Int` - Total number of Fourier mode combinations (m × n) used in the calculation. - - - `numunorms_init::Int` - Initial allocation size for the number of normalization operations recorded. - - - `msing::Int` - Number of singular surfaces in the equilibrium (used to size asymptotic coefficient arrays). - - - `numsteps_init::Int` - Initial allocation size for the number of integration steps to store. - - - `step::Int` - Current integration step index (1-based, like `istep` in the original Fortran). - - - `psi_store::Vector{Float64}` - Stored psi values at each saved integration step (length `numsteps_init`). - - - `q_store::Vector{Float64}` - Stored q values at each saved integration step (length `numsteps_init`). - - - `u_store::Array{ComplexF64,4}` - Stored solution arrays at each saved step with shape - `(numpert_total, numpert_total, 2, numsteps_init)` (complex solution state used by the solver). - - - `du_store::Array{ComplexF64,3}` - dΞ_ψ/dψ (the u₁ block only) at each saved step, shape - `(numpert_total, numpert_total, step)`. Empty until `materialize_derivative_stores!` fills it, - except on the galerkin-matched path which supplies the analytic derivative at construction. - du₂/dψ is never stored densely — its only consumer evaluates it on demand at bracket nodes. - - - `xi_s_store::Array{ComplexF64,3}` - Clebsch displacement Ξ_s at each saved step, eq. 18 of Glasser 2016, - shape `(numpert_total, numpert_total, step)`. Empty until materialized, same as `du_store`. - - - `u_store_el_basis::Bool` - True when `u_store` holds the Euler-Lagrange state `(u₁, u₂)`, so the - derivative kernel can be re-applied to it. False on the sparse parallel path, whose stored columns - are chunk-endpoint Riccati matrices; `materialize_derivative_stores!` refuses to run there. - - - `du_store_populated::Bool` - True once `du_store`/`xi_s_store` hold valid data in the final - (post-transform, post-normalization) basis. Set by `materialize_derivative_stores!` or by the - galerkin-matched constructor; stays false where the stores cannot be materialized, e.g. the - sparse parallel path whose solution is in the Riccati basis. - - - `crit_store::Vector{Float64}` - Stored crit parameter values (smallest eigenvalue of W⁻ꜝ) (length `numsteps_init`). - - - `ca_r::Array{ComplexF64,4}` - Asymptotic coefficients just to the right of each singular surface - with shape `(numpert_total, numpert_total, 2, msing)`. - - - `ca_l::Array{ComplexF64,4}` - Asymptotic coefficients just to the left of each singular surface - with shape `(numpert_total, numpert_total, 2, msing)`. - - - `edge_scan::EdgeScanState` - Edge dW scan state and results. Initialized as a disabled sentinel (N_edge=0) and replaced by `findmax_dW_edge!` when a scan runs. - - - `psifac::Float64` - Current normalized flux coordinate for the integrator. - - - `q::Float64` - Safety factor value at `psifac` (current q during integration). - - - `u::Array{ComplexF64,3}` - Current working solution arrays with shape `(numpert_total, numpert_total, 2)`. - - - `ising_start::Int` - Index of the starting singular surface to be crossed during integration. - - - `psimax::Float64` - Maximum psi value for which the integrator is allowed to run in next integration region. - - - `needs_crossing::Bool` - Flag indicating whether a rational surface needs to be crossed after the current integration region. - - - `nzero::Int` - Count of detected zero crossings (used for diagnostics). - - - `new::Bool` - Flag indicating whether a new `unorm0` should be computed after a fixup. - - # Initialization parameters - - - `unorm::Vector{Float64}` - Current norms of the solution vectors (length `numpert_total`). - - - `unorm0::Vector{Float64}` - Reference/initial norms of the solution vectors (length `numpert_total`). - - # Saved data throughout integration - - - `ifix::Int` - Number of normalization operations performed (index into normalization arrays). - -# Total ODE solver steps taken (all steps, not just saved ones) - - - `index::Array{Int,2}` - Index matrix used for sorting solution norms with shape `(numpert_total, numunorms_init)`. - - - `sing_flag::Vector{Bool}` - Boolean flags indicating which stored normalizations correspond to singular solutions # Edge dW scan state and results (disabled sentinel when psiedge >= psilim, i.e. no edge scan) - (length `numunorms_init`). - - - `zeroed_idx::Vector{Vector{Int}}` - For each ideal rational surface jump, a vector of indices of solutions that were zeroed. # Data for integrator - - - `fixfac::Array{ComplexF64,3}` - Fix-up factors for Gaussian reduction with shape `(numpert_total, numpert_total, numunorms_init)`. - - - `fixstep::Vector{Int64}` - Step indices (psi step positions) at which normalization/fixups were performed (length `numunorms_init`). -""" -@kwdef mutable struct OdeState - # Initialization parameters - numpert_total::Int - numunorms_init::Int - msing::Int - numsteps_init::Int - - # Saved data throughout integration - step::Int = 1 - total_steps::Int = 0 # Total ODE solver steps taken (all steps, not just saved ones) - psi_store::Vector{Float64} = Vector{Float64}(undef, numsteps_init) - q_store::Vector{Float64} = Vector{Float64}(undef, numsteps_init) - u_store::Array{ComplexF64,4} = Array{ComplexF64}(undef, numpert_total, numpert_total, 2, numsteps_init) - du_store::Array{ComplexF64,3} = Array{ComplexF64}(undef, numpert_total, numpert_total, 0) - xi_s_store::Array{ComplexF64,3} = Array{ComplexF64}(undef, numpert_total, numpert_total, 0) - u_store_el_basis::Bool = true - du_store_populated::Bool = false - crit_store::Vector{Float64} = Vector{Float64}(undef, numsteps_init) - ca_r::Array{ComplexF64,4} = Array{ComplexF64}(undef, numpert_total, numpert_total, 2, msing) - ca_l::Array{ComplexF64,4} = Array{ComplexF64}(undef, numpert_total, numpert_total, 2, msing) - - # Edge dW scan state and results (disabled sentinel when psiedge >= psilim, i.e. no edge scan) - edge_scan::EdgeScanState = EdgeScanState(numpert_total, 0) - - # Data for integrator - psifac::Float64 = 0.0 - q::Float64 = 0.0 - u::Array{ComplexF64,3} = zeros(ComplexF64, numpert_total, numpert_total, 2) - ising_start::Int = 0 - psimax::Float64 = 0.0 - needs_crossing::Bool = false - nzero::Int = 0 - - # Used for Gaussian reduction - new::Bool = true - unorm::Vector{Float64} = zeros(Float64, numpert_total) - unorm0::Vector{Float64} = zeros(Float64, numpert_total) - ifix::Int = 0 - index::Array{Int,2} = zeros(Int, numpert_total, numunorms_init) - sing_flag::Vector{Bool} = falses(numunorms_init) - zeroed_idx::Vector{Vector{Int}} = [Int[] for _ in 1:numunorms_init] - fixfac::Array{ComplexF64,3} = zeros(ComplexF64, numpert_total, numpert_total, numunorms_init) - fixstep::Vector{Int64} = zeros(Int64, numunorms_init) - - # Kinetic workspace arrays: evaluated from kwmats/ktmats splines at current psi - kwmat::Array{ComplexF64,3} = zeros(ComplexF64, numpert_total, numpert_total, 6) - ktmat::Array{ComplexF64,3} = zeros(ComplexF64, numpert_total, numpert_total, 6) - - # Shared hint for CubicInterpolant interval search optimization during ODE integration - # All splines evaluated at the same psi can share this hint for O(1) interval lookups - spline_hint::Base.RefValue{Int} = Ref(1) - # Shared 2D hint for CubicInterpolantND (rzphi splines) during ODE integration - # Tuple of (psi_hint, theta_hint) for O(1) interval lookups in 2D bicubic splines - rzphi_hint::Tuple{Base.RefValue{Int},Base.RefValue{Int}} = (Ref(1), Ref(1)) - # Per-thread hint for FourFitVars matrix splines (amats/bmats/cmats/fmats_lower/kmats/gmats - # and kinetic equivalents). Lives on OdeState — which is already cloned per thread in the - # parallel BVP path — so concurrent sing_der! invocations don't race on a shared Ref. - ffit_hint::Base.RefValue{Int} = Ref(1) -end - -OdeState(numpert_total::Int, numsteps_init::Int, numunorms_init::Int, msing::Int) = - OdeState(; numpert_total, numsteps_init, numunorms_init, msing) - -""" - DeltaPrimeData - -The solve's Δ′/outer-region matching payload, in one formalism-independent layout. The -Riccati/STRIDE boundary-value problem and the RDCON Galerkin solve compute the same -quantities in the same PEST-3 convention — the four parity blocks are the identical ± -combination of the raw side-major matrix in both (`pest3_decompose`, Riccati.jl, and -`gal_pest3_blocks`, GalerkinSolve.jl, both porting Fortran `gal_write_pest3_data`) — so -consumers never branch on which integrator ran. - -Every matrix is indexed by the singular surfaces the producing formalism actually solved -across, ordered core→edge: `result.surfaces` for Riccati, the in-domain in-band subset of -it for Galerkin (`gal_resonant_surfaces`). Side-major orderings run -`[L_s1, R_s1, L_s2, R_s2, …]`. - -## Fields - - - `matrix::Matrix{ComplexF64}` - Inter-surface Δ′ of shape (msing × msing) in PEST3 - convention, the tearing↔tearing parity projection of `raw`. Same as `Delta` of the - PEST-3 block set. Both formalisms. - - `raw::Matrix{ComplexF64}` - Raw outer-region matching matrix D′ of shape - (2msing × 2msing), side-major on both axes. Both formalisms. - - `coil::Matrix{ComplexF64}` - Edge coil-response matrix of shape - (2msing × numpert_total); column k is the resonant small-solution response at each - surface side to a unit source on edge poloidal mode k. Riccati fills it from the - vacuum-edge BVP, Galerkin from the `gal_rpec_flag` columns (transposed at pack time - from the (numpert_total × 2msing) block the Galerkin solve produces). Empty when - neither ran. - - `A::Union{Nothing,Matrix{ComplexF64}}` - PEST-3 interchange↔interchange block - (msing × msing). Galerkin only; `nothing` for Riccati, which persists only `raw` and - recovers the blocks on demand via `pest3_decompose`. - - `B::Union{Nothing,Matrix{ComplexF64}}` - PEST-3 interchange↔tearing block; see `A`. - - `Gamma::Union{Nothing,Matrix{ComplexF64}}` - PEST-3 tearing↔interchange block; see `A`. -""" -struct DeltaPrimeData - matrix::Matrix{ComplexF64} - raw::Matrix{ComplexF64} - coil::Matrix{ComplexF64} - A::Union{Nothing,Matrix{ComplexF64}} - B::Union{Nothing,Matrix{ComplexF64}} - Gamma::Union{Nothing,Matrix{ComplexF64}} -end diff --git a/src/ForceFreeStates/Fourfit.jl b/src/ForceFreeStates/Fourfit.jl index dc807f9df..39aa3db51 100644 --- a/src/ForceFreeStates/Fourfit.jl +++ b/src/ForceFreeStates/Fourfit.jl @@ -1,3 +1,80 @@ +@kwdef mutable struct FourFitVars{S<:CubicSeriesInterpolant,Opts<:NamedTuple} + mpert::Int + numpert_total::Int # = mpert * npert (total series count per matrix = numpert_total^2) + + # Complex-valued CubicSeriesInterpolant for stability matrices + # Each matrix is flattened to (npsi × numpert_total^2) series + # FastInterpolations natively supports complex values: CubicSeriesInterpolant{Tgrid, Tvalue} + # NOTE: itp_opts must precede interpolant fields — @kwdef evaluates defaults in declaration order + itp_opts::Opts = (; extrap=ExtendExtrap()) + + amats::S = _empty_series_interp_complex(numpert_total^2, itp_opts) + bmats::S = _empty_series_interp_complex(numpert_total^2, itp_opts) + cmats::S = _empty_series_interp_complex(numpert_total^2, itp_opts) + # `dmats_prim`, `emats_prim` are the pre-Schur-reduction geometric forms + # (D = χ₁·(g23 + q·g33·m/n); E = (-χ₁/n)·(q'·χ₁·g33 - 2π·i·χ₁·g31·singfac + jθ·I)). + # The `_prim` suffix follows `fmats_prim`. Downstream kinetic FKG Schur complements + # consume these primitive forms; the alternate singular-layer path that would need + # kinetic-added overwrites of D and E is not implemented here (see Kinetic.jl). + dmats_prim::S = _empty_series_interp_complex(numpert_total^2, itp_opts) + emats_prim::S = _empty_series_interp_complex(numpert_total^2, itp_opts) + hmats::S = _empty_series_interp_complex(numpert_total^2, itp_opts) + fmats_lower::S = _empty_series_interp_complex(numpert_total^2, itp_opts) + fmats_prim::S = _empty_series_interp_complex(numpert_total^2, itp_opts) # primitive F before Schur complement (for kinetic) + fmats_gal::S = _empty_series_interp_complex(numpert_total^2, itp_opts) # reduced Hermitian F̄ (un-factored) for the Galerkin solver + kmats::S = _empty_series_interp_complex(numpert_total^2, itp_opts) + gmats::S = _empty_series_interp_complex(numpert_total^2, itp_opts) + + # Ideal A,B,C splines preserved before kinetic overwrite + amats_ideal::S = _empty_series_interp_complex(numpert_total^2, itp_opts) + bmats_ideal::S = _empty_series_interp_complex(numpert_total^2, itp_opts) + cmats_ideal::S = _empty_series_interp_complex(numpert_total^2, itp_opts) + + # Kinetic energy matrix splines: 6 components (A,B,C,D,E,H perturbations) + kwmats::Vector{S} = [_empty_series_interp_complex(numpert_total^2, itp_opts) for _ in 1:6] + # Kinetic torque matrix splines: 6 components + ktmats::Vector{S} = [_empty_series_interp_complex(numpert_total^2, itp_opts) for _ in 1:6] + + kinetic_populated::Bool = false # set by make_kinetic_matrix; the solution then obeys the FKG ODE, not the ideal EL relation + + # Pre-computed FKG kinetic matrices (populated by make_kinetic_matrix) + f0mats::S = _empty_series_interp_complex(numpert_total^2, itp_opts) + pmats::S = _empty_series_interp_complex(numpert_total^2, itp_opts) + paats::S = _empty_series_interp_complex(numpert_total^2, itp_opts) + kkmats::S = _empty_series_interp_complex(numpert_total^2, itp_opts) + kkaats::S = _empty_series_interp_complex(numpert_total^2, itp_opts) + r1mats::S = _empty_series_interp_complex(numpert_total^2, itp_opts) + r2mats::S = _empty_series_interp_complex(numpert_total^2, itp_opts) + r3mats::S = _empty_series_interp_complex(numpert_total^2, itp_opts) + gaats::S = _empty_series_interp_complex(numpert_total^2, itp_opts) + + # Pre-allocated evaluation buffer for matrix output + _mat_out::Matrix{ComplexF64} = Matrix{ComplexF64}(undef, numpert_total, numpert_total) + + # Shared hint for sequential evaluation (all splines evaluated at same psi) + _hint::Base.RefValue{Int} = Ref(1) + + # Jacobian Fourier band ψ-spline (2·mpert−1 conjugate-symmetric coefficients per surface), + # used to assemble the power-normalization matrix N in Free.jl + jmats::S = _empty_series_interp_complex(2 * mpert - 1, itp_opts) +end + +# Helper to create empty complex series interpolant for default initialization +function _empty_series_interp_complex(n_series::Int) + xs = collect(range(0.0, 1.0; length=5)) + Y = zeros(ComplexF64, 5, n_series) + return cubic_interp(xs, Series(Y)) +end + +function _empty_series_interp_complex(n_series::Int, itp_opts::NamedTuple) + xs = collect(range(0.0, 1.0; length=5)) + Y = zeros(ComplexF64, 5, n_series) + return cubic_interp(xs, Series(Y); itp_opts...) +end + +# Convenience constructor +FourFitVars(mpert::Int, numpert_total::Int) = FourFitVars(; mpert, numpert_total) + """ MetricData diff --git a/src/ForceFreeStates/Free.jl b/src/ForceFreeStates/Free.jl index ce3ac4ca7..1bb7afbb5 100644 --- a/src/ForceFreeStates/Free.jl +++ b/src/ForceFreeStates/Free.jl @@ -1,3 +1,37 @@ +""" + FreeBoundaryResult + +Result of the free-boundary calculation, returned by `free_run`. All matrices are in the ξ Fourier +basis and are `numpert_total × numpert_total`; the energies are generalized (W, N) pencil values, +power-normalized and invariant to the working (Jacobian) coordinate. + +## Fields + + - `wt::Matrix{ComplexF64}` - Eigenvector matrix of W·v = λ·N·v. Columns are eigenmodes sorted most-unstable first, normalized to unit power norm v†·N·v = 1. + - `wt0::Matrix{ComplexF64}` - Total-energy matrix W = wp + wv before diagonalisation + - `wp::Matrix{ComplexF64}` - Plasma energy matrix + - `wv::Matrix{ComplexF64}` - Vacuum energy matrix, singfac-scaled at `qlim` + - `ep::Vector{ComplexF64}` - Plasma energy per eigenmode (power quotient v†·wp·v with v†·N·v = 1) + - `ev::Vector{ComplexF64}` - Vacuum energy per eigenmode (power quotient v†·wv·v with v†·N·v = 1) + - `et::Vector{ComplexF64}` - Total energy eigenvalues of the pencil (W, N); et = ep + ev per mode + - `n_tor_idx::Vector{Int}` - 0-based toroidal mode number index of each sorted eigenvalue + - `vacuum_eigenvalue::Float64` - Least stable (minimum) eigenvalue of the pencil (wv, N), clamped to zero + - `plasma_pts`, `wall_pts::Matrix{Float64}` - Cartesian (x, y, z) surface coordinates, `numpoints × 3`, retained for HDF5 output +""" +struct FreeBoundaryResult + wt::Matrix{ComplexF64} + wt0::Matrix{ComplexF64} + wp::Matrix{ComplexF64} + wv::Matrix{ComplexF64} + ep::Vector{ComplexF64} + ev::Vector{ComplexF64} + et::Vector{ComplexF64} + n_tor_idx::Vector{Int} + vacuum_eigenvalue::Float64 + plasma_pts::Matrix{Float64} + wall_pts::Matrix{Float64} +end + """ power_norm_matrix!(Nmat, jmat, mpert, npert, dV_dpsi) -> Nmat diff --git a/src/ForceFreeStates/Matching/DeltaPrime.jl b/src/ForceFreeStates/Matching/DeltaPrime.jl new file mode 100644 index 000000000..707f58eff --- /dev/null +++ b/src/ForceFreeStates/Matching/DeltaPrime.jl @@ -0,0 +1,42 @@ +""" + DeltaPrimeData + +The solve's Δ′/outer-region matching payload, in one formalism-independent layout. The +Riccati/STRIDE boundary-value problem and the RDCON Galerkin solve compute the same +quantities in the same PEST-3 convention — the four parity blocks are the identical ± +combination of the raw side-major matrix in both (`pest3_decompose`, Riccati/DeltaPrimeBVP.jl, and +`gal_pest3_blocks`, GalerkinSolve.jl, both porting Fortran `gal_write_pest3_data`) — so +consumers never branch on which integrator ran. + +Every matrix is indexed by the singular surfaces the producing formalism actually solved +across, ordered core→edge: `result.surfaces` for Riccati, the in-domain in-band subset of +it for Galerkin (`gal_resonant_surfaces`). Side-major orderings run +`[L_s1, R_s1, L_s2, R_s2, …]`. + +## Fields + + - `matrix::Matrix{ComplexF64}` - Inter-surface Δ′ of shape (msing × msing) in PEST3 + convention, the tearing↔tearing parity projection of `raw`. Same as `Delta` of the + PEST-3 block set. Both formalisms. + - `raw::Matrix{ComplexF64}` - Raw outer-region matching matrix D′ of shape + (2msing × 2msing), side-major on both axes. Both formalisms. + - `coil::Matrix{ComplexF64}` - Edge coil-response matrix of shape + (2msing × numpert_total); column k is the resonant small-solution response at each + surface side to a unit source on edge poloidal mode k. Riccati fills it from the + vacuum-edge BVP, Galerkin from the `gal_rpec_flag` columns (transposed at pack time + from the (numpert_total × 2msing) block the Galerkin solve produces). Empty when + neither ran. + - `A::Union{Nothing,Matrix{ComplexF64}}` - PEST-3 interchange↔interchange block + (msing × msing). Galerkin only; `nothing` for Riccati, which persists only `raw` and + recovers the blocks on demand via `pest3_decompose`. + - `B::Union{Nothing,Matrix{ComplexF64}}` - PEST-3 interchange↔tearing block; see `A`. + - `Gamma::Union{Nothing,Matrix{ComplexF64}}` - PEST-3 tearing↔interchange block; see `A`. +""" +struct DeltaPrimeData + matrix::Matrix{ComplexF64} + raw::Matrix{ComplexF64} + coil::Matrix{ComplexF64} + A::Union{Nothing,Matrix{ComplexF64}} + B::Union{Nothing,Matrix{ComplexF64}} + Gamma::Union{Nothing,Matrix{ComplexF64}} +end diff --git a/src/ForceFreeStates/Matching/ResonantMatch.jl b/src/ForceFreeStates/Matching/ResonantMatch.jl new file mode 100644 index 000000000..4f3fd3056 --- /dev/null +++ b/src/ForceFreeStates/Matching/ResonantMatch.jl @@ -0,0 +1,80 @@ +# Outer<->inner resistive match, Wang et al. 2020 (PoP 27, 122509) Eq. 11: +# C = -(Δ_out - Δ_in(i2πf))^{-1} Δ_coil +# Raw STRIDE outer Δ' + raw coil drive matched to the GGJ inner layer (resist_eval -> solve_inner). +struct ResonantMatchResult + cout::Matrix{ComplexF64} # outer coeffs (2msing × ncoil) + cin::Matrix{ComplexF64} # inner coeffs (2msing × ncoil) + deltar::Matrix{ComplexF64} # inner-layer Δ per surface (msing × 2) + rpec_eig::Vector{ComplexF64} # forced eigenvalue γ_s = 2πi·n·f + reconnected_flux::Matrix{ComplexF64} # reconnected resonant flux (2msing × ncoil) + bpen::Matrix{ComplexF64} # area-weighted penetrated field (msing × ncoil); empty until Stage 2 + residual::Float64 +end + +function resonant_match_rpec(delta_out_raw::AbstractMatrix, delta_coil_raw::AbstractMatrix, + sings::Vector{SingType}, equil::Equilibrium.PlasmaEquilibrium, + intr::ForceFreeStatesInternal, ctrl::ForceFreeStatesControl) + + msing = size(delta_out_raw, 1) ÷ 2 + ncoil = size(delta_coil_raw, 2) + nn = intr.nlow + empty_bpen = Matrix{ComplexF64}(undef, 0, 0) + + size(delta_out_raw) == (2msing, 2msing) || error("delta_out_raw $(size(delta_out_raw)) != (2msing,2msing)") + length(sings) == msing || error("sings $(length(sings)) != msing $msing") + size(delta_coil_raw, 1) == 2msing || error("delta_coil_raw rows $(size(delta_coil_raw,1)) != 2msing") + + if ctrl.gal_ideal_flag # ideal limit: no inner layer, no reconnection + return ResonantMatchResult(zeros(ComplexF64,2msing,ncoil), zeros(ComplexF64,2msing,ncoil), + zeros(ComplexF64,msing,2), zeros(ComplexF64,msing), Matrix{ComplexF64}(delta_coil_raw), empty_bpen, 0.0) + end + for (nm,v) in (("gal_eta",ctrl.gal_eta),("gal_rho",ctrl.gal_rho),("gal_rotation",ctrl.gal_rotation)) + length(v) == msing || error("$nm length $(length(v)) != msing $msing") + end + + chi1 = 2π * equil.psio + deltar = zeros(ComplexF64, msing, 2) + rpec_eig = zeros(ComplexF64, msing) + # Layer-center (X=0) penetrated-field weights pen[i,k] = scale·Ψ_k(0)·rescale (match.f intotsol_b); + # solve_inner_profile returns the same Δ as solve_inner plus the inner-layer field needed for pen. + pen = zeros(ComplexF64, msing, 2) + for i in 1:msing + params = resist_eval(sings[i], equil, intr; eta=ctrl.gal_eta[i], rho=ctrl.gal_rho[i], gamma=ctrl.gal_gamma, ising=i) + γ = 2π*im*nn*ctrl.gal_rotation[i] + rpec_eig[i] = γ + inner = InnerLayer.solve_inner_profile(InnerLayer.GGJModel(; solver=:galerkin), params, γ; + xfac=ctrl.gal_inner_xfac, nx=ctrl.gal_inner_nx, nq=ctrl.gal_inner_nq, cutoff=ctrl.gal_inner_cutoff, kmax=ctrl.gal_inner_kmax) + deltar[i,1] = inner.Δ[1]; deltar[i,2] = inner.Δ[2] + scale = -2π * chi1 * im * nn * sings[i].q1 * inner.dψdx # b_m = −2πi·χ₁·n·q′·dψdx·rescale·Ψ (GalerkinMatch.jl) + pen[i,1] = scale * inner.Ψ[1,1] * inner.rescale # parity 1 (Ψ(0)≠0) + pen[i,2] = scale * inner.Ψ[1,2] * inner.rescale # parity 2 (Ψ(0)=0 ⇒ ~0) + end + + mat = zeros(ComplexF64, 4msing, 4msing) + rmat = zeros(ComplexF64, 4msing, ncoil) + @views mat[2msing+1:4msing, 1:2msing] .= transpose(delta_out_raw) + @views rmat[2msing+1:4msing, :] .= .-delta_coil_raw + for i in 1:msing + a=2i-1; b=2i; c=a+2msing; d=b+2msing + d1=deltar[i,1]; d2=deltar[i,2] + mat[a,a]=1; mat[b,b]=1 + mat[a,c]=-1; mat[a,d]=1 + mat[b,c]=-1; mat[b,d]=-1 + mat[c,c]=-d1; mat[c,d]=d2 + mat[d,c]=-d1; mat[d,d]=-d2 + end + + cof = mat \ rmat + residual = norm(mat*cof - rmat) / max(norm(rmat), 1e-300) + cout = cof[1:2msing, :] + cin = cof[2msing+1:4msing, :] + + reconnected_flux = delta_coil_raw .+ transpose(delta_out_raw)*cout + # Inner-layer penetrated (reconnected) resonant field per surface — ONE quantity per surface, read off + # the inner solution at the layer center (match.f intotsol_b; GalerkinMatch.jl): bpen[i,j] = pen₁(i)·cin[2i,j] + pen₂(i)·cin[2i-1,j]. + bpen = zeros(ComplexF64, msing, ncoil) + for i in 1:msing, j in 1:ncoil + bpen[i,j] = pen[i,1]*cin[2i,j] + pen[i,2]*cin[2i-1,j] + end + return ResonantMatchResult(cout, cin, deltar, rpec_eig, reconnected_flux, bpen, residual) +end diff --git a/src/ForceFreeStates/Riccati.jl b/src/ForceFreeStates/Riccati.jl deleted file mode 100644 index c1710cc12..000000000 --- a/src/ForceFreeStates/Riccati.jl +++ /dev/null @@ -1,1779 +0,0 @@ -""" - Riccati.jl - Dual Riccati reformulation of the Euler-Lagrange ODE - -Implements the dual Riccati matrix S = U₁ · U₂⁻¹ = P⁻¹, which satisfies a bounded -ODE even near singular surfaces where U₁, U₂ grow exponentially. This reduced stiffness -leads to fewer ODE integration steps and faster wall-clock time. - -Reference: Glasser (2018) Phys. Plasmas 25, 032507 — Eq. 19 (adapted for dual form S = P⁻¹) -where P = U₂ · U₁⁻¹ is the forward plasma response matrix. - -## Dual Riccati ODE - -Starting from the Euler-Lagrange system [Glasser 2016 eq. 24]: - dU₁/dψ = A·U₁ + B·U₂ A = -Q·F̄⁻¹·K̄, B = Q·F̄⁻¹·Q - dU₂/dψ = C·U₁ + D·U₂ C = Ḡ - K̄†·F̄⁻¹·K̄, D = K̄†·F̄⁻¹·Q - -with S = U₁·U₂⁻¹, differentiating gives the Riccati ODE: - dS/dψ = B + A·S - S·D - S·C·S - -Setting w = Q - K̄·S (shape N×N) and v = F̄⁻¹·w (Cholesky solve), this simplifies to: - dS/dψ = w†·v - S·Ḡ·S [Glasser 2018 eq. 19, dual form] - -## Integration Strategy - -### Why not integrate the Riccati ODE directly? - -`riccati_der!` evaluates the explicit Riccati RHS `dS/dψ = w†F̄⁻¹w − S·Ḡ·S` correctly, -but this ODE is **quadratic** in S. Near a rational surface, S grows large, so the quadratic -term `-SGS` dominates and the RHS grows as |S|². Explicit adaptive solvers (Vern9) use -*relative* error control: they accept a step when |Δu|/|u| < reltol. When |S| is large, -the absolute error |ΔS| can be enormous while the relative error stays within tolerance. -The solver takes large steps through what is effectively a near-blowup — no amount of -step-size adaptation saves it because the problem is the error *metric*, not the step size. -An implicit solver could handle this stiffness, but is deferred. - -### Actual implementation: EL ODE + renormalization - -Instead we integrate the standard EL ODE (`sing_der!`) in the (U₁, U₂) variables and -recover S = U₁·U₂⁻¹ by renormalization. This achieves the same Riccati trajectory with -**no accuracy loss**: - -- `sing_der!` evaluates the exact EL RHS — no approximation. -- Vern9 integrates (U₁, U₂) to **9th-order accuracy** with the adaptive step-size - controller enforcing the configured reltol at every accepted step. -- Renormalization `S = U₁·U₂⁻¹` is **exact** (a change of variables, not an approximation). -- The global error is the same as the standard EL path — controlled by the ODE solver - reltol, not by the renormalization frequency. - -This works because the EL ODE is **linear** in (U₁, U₂): the RHS does not grow with |S|, -so relative error control is faithful even when S is large. Renormalization triggered by -`renormalize_riccati_inplace!` in the callback (when max(|U₁|) or max(|U₂|) > ucrit) keeps -both matrices bounded, preventing overflow and maintaining a well-conditioned state for the -solver — exactly analogous to Gaussian reduction in the standard ODE. - -### Consistency with the Riccati ODE (local analysis) - -To verify the method is consistent with the Riccati ODE, consider a single step from (S, I): - - After one step: U₁_new = S + (A·S + B)·Δψ + O(Δψ²), U₂_new = I + (C·S + D)·Δψ + O(Δψ²) - Renorm: S_new = U₁_new · U₂_new⁻¹ = S + (B + A·S − S·D − S·C·S)·Δψ + O(Δψ²) ✓ - -The leading term matches the Riccati ODE exactly. This is a local consistency check only — -it does not imply the integration is first-order. In practice Vern9 captures all higher-order -terms through its internal stages, achieving 9th-order global accuracy at the configured reltol. - -## Storage Convention - -During chunk integration (with sing_der! as ODE RHS): - u[:,:,1] = U₁ (starts as S_prev, evolves toward new S) - u[:,:,2] = U₂ (starts as I, evolves with EL dynamics) - -After renormalization (at crossing or when norms exceed ucrit): - u[:,:,1] = S = U₁ · U₂⁻¹ - u[:,:,2] = I - -This is compatible with downstream code (which uses U₁/U₂ ratio): - - Free.jl: wp = u[:,:,2] / u[:,:,1] = I · S⁻¹ = P ✓ (post-renorm) - - FixedBoundaryStability.jl: crit = min_eigval(u[:,:,1] / u[:,:,2]) = min_eigval(S) ✓ - - Axis init: determined by `ctrl.fixed_axis`. When `true`, U₁=0, U₂=I → S(ψ₀)=0 (original - Glasser fixed-axis BC). When `false` (default), Frobenius eigenvalue init [Glasser 2016 Eq. 51] - sets U₂=I and U₁ to the regular Frobenius eigenvector per mode → S(ψ₀) = U₁_Frobenius is - nonzero in general. Riccati S-evolution remains well-defined either way. - -## Key Differences from Standard Integration - -1. `sing_der!` is used as the ODE RHS (same as standard, NOT `riccati_der!`) -2. `riccati_integrator_callback!` replaces `integrator_callback!`: uses - `renormalize_riccati_inplace!` instead of Gaussian reduction -3. `riccati_cross_ideal_singular_surf!` replaces `cross_ideal_singular_surf!`: skips Gaussian - reduction and uses ipert_res directly for column zeroing, then renormalizes to (S_new, I) -4. `transform_u!` is skipped — S is already the true solution -""" - -# Save-frequency thresholds for `riccati_integrator_callback!`. Near the right endpoint of -# a segment we save every step so that the crossing / chunk boundary captures fine detail; -# elsewhere we save every `ctrl.save_interval`-th step. The relative band catches normal- -# length chunks; the absolute floor catches short chunks where 5% of the span would be -# smaller than the typical ODE step. -const SAVE_NEAR_END_FRAC = 0.05 -const SAVE_NEAR_END_PSI = 1e-4 - -""" - assemble_fm_matrix(propagators, idx_range; condition=false) -> Matrix{ComplexF64} - -Assemble the 2N×2N fundamental matrix (propagator) by multiplying chunk propagators -in order for indices `idx_range`. Returns Φ_end * ... * Φ_start, so that the result -maps the IC at the start of `idx_range[1]` to the state at the end of `idx_range[end]`. - -Each `ChunkPropagator` stores the 2N columns of Φ split into two N×N×2 blocks: -``` - block_upper_ic[:,:,1:2] ↔ Φ[:,1:N] (result from IC=(I,0)) - block_lower_ic[:,:,1:2] ↔ Φ[:,N+1:2N] (result from IC=(0,I)) -``` - -When `condition=true`, applies Gaussian reduction (`condition_propagator!`) after each -multiplication step, following STRIDE's `ode_fixup` convention. This -prevents exponential growth of the accumulated product: without conditioning, products -of K chunk propagators can reach cond ~ (cond_per_chunk)^K, causing catastrophic -cancellation. With periodic conditioning, each step stays at O(cond_per_chunk) and -only the N well-conditioned U₂ columns (right half) survive. - -Use `condition=true` for the axis→first-surface segment, where the axis BC (U₁=0) -means only U₂ ICs are needed. Do NOT use for inter-surface segments where both U₁ -and U₂ components carry physical information. -""" -function assemble_fm_matrix(propagators::Vector{ChunkPropagator}, idx_range; - condition::Bool=false, - T_init::Union{Nothing,Matrix{ComplexF64}}=nothing) - # Determine matrix size from T_init if provided (lets us handle empty idx_range and even - # an empty propagators list, provided T_init carries the dimension). Otherwise fall back - # to the first propagator that actually exists in idx_range, with a final fallback to - # propagators[1] when both idx_range and T_init pin nothing down. - N = if T_init !== nothing - size(T_init, 1) ÷ 2 - elseif !isempty(idx_range) - size(propagators[first(idx_range)].block_upper_ic, 1) - else - @assert !isempty(propagators) "assemble_fm_matrix: cannot infer N from empty propagators with no T_init" - size(propagators[1].block_upper_ic, 1) - end - Phi = T_init !== nothing ? copy(T_init) : Matrix{ComplexF64}(I, 2N, 2N) - isempty(idx_range) && return Phi - for i in idx_range - p = propagators[i] - #! format: off - Phi_i = [p.block_upper_ic[:,:,1] p.block_lower_ic[:,:,1]; - p.block_upper_ic[:,:,2] p.block_lower_ic[:,:,2]] - #! format: on - Phi = Phi_i * Phi - if condition - condition_propagator!(Phi, N) - end - end - return Phi -end - -""" - condition_propagator!(Phi, N) - -Apply Gaussian reduction to the U₂-columns (columns N+1:2N) of a 2N×2N propagator -matrix in-place, following STRIDE's `ode_fixup` convention. Triangularizes the U₁ -(upper N rows) subblock by pivoted elimination, improving the condition number so -the propagator can be used in a BVP without losing numerical rank. - -After conditioning, only the U₂ columns carry meaningful information; the U₁ columns -(1:N) are zeroed. The BVP axis block uses `Phi[:, N+1:2N]` (the conditioned half). -""" -function condition_propagator!(Phi::Matrix{ComplexF64}, N::Int) - # Work on the right half: columns N+1:2N (U₂ initial conditions) - cols = view(Phi, :, N+1:2N) - - # Sort columns by norm of the U₁ (upper N) block — largest first - norms = [norm(view(cols, 1:N, k)) for k in 1:N] - order = sortperm(norms; rev=true) - - mask_col = trues(N) # which columns remain to process - mask_row = trues(N) # which pivot rows remain available - - for isol in 1:N - kcol = order[isol] - mask_col[kcol] = false - - # Find best pivot row (largest |element| among unmasked rows) - best_row = 0 - best_val = 0.0 - for r in 1:N - if mask_row[r] && abs(cols[r, kcol]) > best_val - best_val = abs(cols[r, kcol]) - best_row = r - end - end - if best_row == 0 || best_val == 0 - continue - end - mask_row[best_row] = false - - # Eliminate this pivot from all other unmasked columns - pivot = cols[best_row, kcol] - for jcol in 1:N - if mask_col[jcol] - factor = -cols[best_row, jcol] / pivot - @views cols[:, jcol] .+= factor .* cols[:, kcol] - cols[best_row, jcol] = 0 # exact zero - end - end - end - - # Zero the U₁ columns (left half) — they are no longer meaningful - Phi[:, 1:N] .= 0 - return Phi -end - -""" - compute_delta_prime_matrix!(intr, propagators, chunks; wv, psio, debug, ctrl, equil, ffit) - -Compute the inter-surface tearing stability matrix (msing × msing) using the -STRIDE global BVP formulation [Glasser 2018 Phys. Plasmas 25, 032501, Sec. III.B]. - -The BVP encodes the full plasma response with unknowns at each surface boundary: -``` - x_axis (N): free IC parameters at the axis (U₁ = 0 regular solutions) - x_left[j] (2N): state at left inner-layer boundary of surface j - x_right[j] (2N): state at right inner-layer boundary of surface j - x_edge (N): free IC parameters at the edge - Total unknowns: nMat = (2 + 4·msing)·N -``` - -## Edge boundary condition - -When `wv` is provided (the vacuum response matrix, singfac-scaled), the edge BC -follows the Fortran STRIDE convention: -``` - U₁ = c, U₂ = -wv·ψ₀²·c -``` -which is the free-boundary condition `wp + wv = 0` at the edge. -When `wv` is `nothing`, a conducting wall BC (`U₁ = 0`) is used. - -## Gaussian reduction (conditioning) - -Forward-propagated segment propagators (axis→surface, surface→surface) can be -extremely ill-conditioned (cond ~ 10²⁴) due to exponential growth of the big -solution. Following STRIDE's `ode_fixup`, Gaussian reduction is applied to each -assembled propagator's U₂ columns before inserting into the BVP matrix. This -keeps the BVP matrix full-rank and well-conditioned. - -## Output: PEST3-convention Δ' (deltap) - -The raw BVP solution is a 2·msing × 2·msing matrix `dp` with left/right -sub-indices at each surface. The PEST3-convention Δ' matrix is the linear -combination [Chance, PPPL-2527]: -``` - deltap(i,j) = dp(2i,2j) - dp(2i,2j-1) - dp(2i-1,2j) + dp(2i-1,2j-1) -``` -stored in `intr.delta_prime_matrix` (msing × msing). - -## Limitations - -This routine currently assumes exactly one resonant mode per singular surface -(the standard single-`n` case). When **any** surface carries more than one -resonant mode — i.e., a multi-`n` run where a single q value satisfies two -distinct `(m, n)` tuples (e.g. q = 2 with `(m=2, n=1)` AND `(m=4, n=2)`) — -the routine emits a warning and skips the inter-surface BVP rather than -crashing. Generalizing the BVP to multi-resonance surfaces is tracked as a -follow-up: the matrix shape becomes `n_res_total × n_res_total` with -`n_res_total = sum(length(intr.sing[j].m))` and a `(surface, mode, side)` -↔ BVP-row map; see PR discussion. - -Note: `intr.delta_prime_matrix` is the **only physically valid Δ'** produced -by this code. The per-surface ca-based stub `intr.sing[*].delta_prime` / -`delta_prime_col` (populated by `riccati_cross_ideal_singular_surf!`) is a -diagnostic placeholder for future intra-surface coupling work and is not -expected to agree with `delta_prime_matrix`. -""" -function compute_delta_prime_matrix!( - intr::ForceFreeStatesInternal, - propagators::Vector{ChunkPropagator}, - chunks::Vector{IntegrationChunk}; - wv::Union{Nothing,Matrix{ComplexF64}} = nothing, - psio::Float64 = 0.0, - debug::Bool = false, - S_at_surface_left::Union{Nothing,Vector{Matrix{ComplexF64}}} = nothing, - ctrl::Union{Nothing,ForceFreeStatesControl} = nothing, - equil::Union{Nothing,Equilibrium.PlasmaEquilibrium} = nothing, - ffit::Union{Nothing,FourFitVars} = nothing -) - intr.msing == 0 && return - _has_unsupported_multi_resonance(intr) && return - - sing, i_crossings, msing = _select_active_surfaces(intr, chunks) - msing == 0 && return - N = intr.numpert_total - - use_S_axis = S_at_surface_left !== nothing && length(S_at_surface_left) == msing - - # The FM-axis-BC fallback (use_S_axis=false) wires Phi_L_mats[j] as forward propagators - # in the BVP matrix. Crossing chunks with direction=-1 (bidirectional parallel FM) hold - # *backward* propagators, so applying them as forward would produce a silently wrong - # Δ' BVP. Forbid that combination explicitly — the parallel path always supplies - # S_at_surface_left (so use_S_axis=true) and any new caller hitting the FM-axis path - # needs forward crossing chunks. - if !use_S_axis - for ic in i_crossings - chunks[ic].direction == 1 || - error("compute_delta_prime_matrix!: FM-axis fallback (use_S_axis=false) requires forward crossing chunks; " * - "chunk $ic has direction=$(chunks[ic].direction). Either provide S_at_surface_left or use bidirectional=false.") - end - end - - Phi_L_mats, Phi_R_mats, Phi_R_halves = _assemble_segment_propagators( - propagators, chunks, i_crossings, msing, N, use_S_axis) - - ipert_all = [1 + sing[j].m[1] - intr.mlow + (sing[j].n[1] - intr.nlow) * intr.mpert for j in 1:msing] - has_ua = all(j -> !isempty(sing[j].ua_left), 1:msing) - T_left_mats, T_right_mats, T_left_inv, T_right_inv = - _build_asymptotic_basis_matrices(sing, has_ua, N, msing) - - debug && _log_bvp_setup(chunks, sing, S_at_surface_left, use_S_axis, has_ua, - Phi_L_mats, Phi_R_mats, Phi_R_halves, ipert_all, wv, psio, N, msing) - - if use_S_axis - uShootR, uShootL, uAxis = _build_S_axis_shooting_propagators( - propagators, chunks, i_crossings, sing, msing, N, - T_left_mats, T_right_mats, has_ua, ctrl, equil, ffit, intr, debug) - debug && _log_S_axis_shooting_propagators(uShootR, uShootL, uAxis, - S_at_surface_left, T_left_mats, - ipert_all, has_ua, msing, N) - M, nMat, col_edge = _assemble_bvp_S_axis( - uShootR, uShootL, uAxis, ipert_all, msing, N, wv, psio) - else - M, nMat, col_edge = _assemble_bvp_FM_axis( - Phi_L_mats, Phi_R_mats, ipert_all, msing, N, - T_left_inv, T_right_inv, has_ua, wv, psio) - end - - if debug - @info "Δ' BVP: nMat=$nMat, rank(M)=$(rank(M)), cond(M)=$(@sprintf("%.2e", cond(M)))" - end - - # rpec coil-response block: needs the S-axis row layout that `_solve_bvp_edge_coil` assumes - # for its edge rows, and a vacuum edge in the assembled matrix (col_edge in junc_rows). - if use_S_axis && wv !== nothing - intr.delta_coil = _solve_bvp_edge_coil(M, col_edge, msing, N, ipert_all) - end - - deltap, dp_raw_persisted = _solve_bvp_and_combine_pest3( - M, msing, N, nMat, use_S_axis, ipert_all, col_edge, ctrl, debug) - - # Persist both the PEST3 tearing projection (msing × msing) and the raw 2msing × 2msing - # D' matrix (side-major ordering, byte-compatible with Fortran rdcon/gal.f::gal_write_delta). - # The raw matrix is consumed by `pest3_decompose` to recover (A', B', Γ', Δ') for the full - # det(D' − D(γ)) = 0 eigenvalue problem; see ForceFreeStatesStructs.jl docstring. - intr.delta_prime_matrix = deltap - intr.delta_prime_raw = dp_raw_persisted -end - -# Column index helpers for the BVP matrix. j is the 1-based singular-surface index, -# N is numpert_total. Layout: c_axis(N), c_left[1](2N), c_right[1](2N), ..., c_edge(N). -_col_left(j::Int, N::Int) = (N + 4N*(j-1) + 1):(N + 4N*(j-1) + 2N) -_col_right(j::Int, N::Int) = (N + 4N*(j-1) + 2N + 1):(N + 4N*j) - -# Multi-resonance surfaces (one q value satisfying multiple (m,n) tuples in a multi-n run) -# are not yet handled by the inter-surface BVP. Returns true if any surface has >1 modes; -# emits a warning as a side effect. The stub per-surface delta_prime is unaffected. -function _has_unsupported_multi_resonance(intr::ForceFreeStatesInternal) - msing = intr.msing - n_res_per_surface = [length(intr.sing[j].m) for j in 1:msing] - any(>(1), n_res_per_surface) || return false - offenders = [(j, intr.sing[j].m, intr.sing[j].n) for j in 1:msing if n_res_per_surface[j] > 1] - @warn "compute_delta_prime_matrix!: skipping inter-surface Δ' BVP because some surfaces carry more than one resonant mode " * - "(multi-n collision; generalization tracked as follow-up). " * - "Per-surface Δ' is unaffected. Multi-resonance surfaces: $offenders" - return true -end - -# Map BVP surface index (1:msing_active) → intr.sing index using chunk.ising. Surfaces -# may be excluded at either end (below qlow or beyond psilim); each crossing chunk -# records its original surface index. Returns (sing alias, i_crossings, msing_active). -function _select_active_surfaces(intr::ForceFreeStatesInternal, chunks::Vector{IntegrationChunk}) - msing = intr.msing - i_crossings = findall(c -> c.needs_crossing, chunks) - sing_indices = [chunks[ic].ising for ic in i_crossings] - msing_active = length(i_crossings) - if msing_active < msing - excluded = setdiff(1:msing, sing_indices) - excluded_ms = [intr.sing[j].m for j in excluded] - @debug "compute_delta_prime_matrix!: $msing singular surfaces, $msing_active crossed (excluded: m=$excluded_ms)" - end - sing = [intr.sing[si] for si in sing_indices] - return sing, i_crossings, msing_active -end - -# Assemble all segment propagators: per-surface single-chunk FMs (Phi_L), inter-surface -# and edge multi-chunk FMs (Phi_R), and midpoint-split halves (Phi_R_halves) used by the -# diagnostic comparisons. Phi_R[1] is only built when use_S_axis=false (FM-axis fallback). -# Midpoint splitting halves each inter-surface span's condition number — STRIDE's trick: -# cond(full) = 10¹⁵ → cond(half) ≈ 10⁷·⁵, an 8-digit accuracy gain. -function _assemble_segment_propagators(propagators::Vector{ChunkPropagator}, - chunks::Vector{IntegrationChunk}, - i_crossings::Vector{Int}, msing::Int, N::Int, - use_S_axis::Bool) - Phi_L_mats = [assemble_fm_matrix(propagators, i_crossings[j]:i_crossings[j]) for j in 1:msing] - Phi_R_mats = Vector{Matrix{ComplexF64}}(undef, msing + 1) - if !use_S_axis - Phi_R_mats[1] = assemble_fm_matrix(propagators, 1:i_crossings[1]-1; condition=true) - end - for j in 2:msing - Phi_R_mats[j] = assemble_fm_matrix(propagators, i_crossings[j-1]+1:i_crossings[j]-1) - end - Phi_R_mats[msing+1] = assemble_fm_matrix(propagators, i_crossings[msing]+1:length(chunks)) - - Phi_R_halves = Vector{Tuple{Matrix{ComplexF64},Matrix{ComplexF64}}}(undef, msing - 1) - for j in 1:msing-1 - chunk_start = i_crossings[j] + 1 - chunk_end = i_crossings[j+1] - 1 - n_chunks = chunk_end - chunk_start + 1 - if n_chunks >= 2 - i_mid = chunk_start + div(n_chunks, 2) - 1 - Phi_left_half = assemble_fm_matrix(propagators, chunk_start:i_mid) - Phi_right_half = assemble_fm_matrix(propagators, i_mid+1:chunk_end) - Phi_R_halves[j] = (Phi_left_half, Phi_right_half) - else - Phi_R_halves[j] = (Matrix{ComplexF64}(I, 2N, 2N), Phi_R_mats[j+1]) - end - end - return Phi_L_mats, Phi_R_mats, Phi_R_halves -end - -# Asymptotic-basis transformation T = [ua[:,:,1]; ua[:,:,2]] maps (small/big) coefficients -# to raw (ξ,η) state. Column ordering of ua: 1:N = big solutions (z^{-α}, diverging), -# N+1:2N = small solutions (z^{+α}, bounded). Fortran STRIDE bakes T into the shooting -# propagators (uFM_sing_init); we multiply T into the BVP propagator blocks at each surface. -function _build_asymptotic_basis_matrices(sing::Vector{SingType}, has_ua::Bool, N::Int, msing::Int) - T_left_mats = Vector{Matrix{ComplexF64}}(undef, msing) - T_right_mats = Vector{Matrix{ComplexF64}}(undef, msing) - T_left_inv = Vector{Matrix{ComplexF64}}(undef, msing) - T_right_inv = Vector{Matrix{ComplexF64}}(undef, msing) - if has_ua - for j in 1:msing - sp = sing[j] - T_left_mats[j] = [sp.ua_left[:,:,1]; sp.ua_left[:,:,2]] - T_right_mats[j] = [sp.ua_right[:,:,1]; sp.ua_right[:,:,2]] - T_left_inv[j] = inv(T_left_mats[j]) - T_right_inv[j] = inv(T_right_mats[j]) - end - end - return T_left_mats, T_right_mats, T_left_inv, T_right_inv -end - -# Build the S-axis shooting propagators uShootR (forward from surface j right → midpoint) -# and uShootL (backward from surface j left → midpoint), and the conditioned axis -# propagator uAxis. uShootL[1] is built specially using the QR-conditioned axis path -# (Fortran ode_fixup) so that surface 1 inherits the well-conditioned S axis BC instead -# of going through a catastrophically ill-conditioned full axis FM. -function _build_S_axis_shooting_propagators( - propagators::Vector{ChunkPropagator}, chunks::Vector{IntegrationChunk}, - i_crossings::Vector{Int}, sing::Vector{SingType}, msing::Int, N::Int, - T_left_mats::Vector{Matrix{ComplexF64}}, T_right_mats::Vector{Matrix{ComplexF64}}, - has_ua::Bool, ctrl, equil, ffit, intr::ForceFreeStatesInternal, debug::Bool) - - can_reintegrate = has_ua && ctrl !== nothing && equil !== nothing && ffit !== nothing - uShootR = Vector{Matrix{ComplexF64}}(undef, msing) - uShootL = Vector{Matrix{ComplexF64}}(undef, msing) # uShootL[1] handled separately below - - for j in 1:msing - shoot_range_R = _midpoint_shoot_range(chunks, i_crossings, j, msing; side=:right) - if debug && !isempty(shoot_range_R) - psi_surf_R = chunks[first(shoot_range_R)].psi_start - psi_mid_R = chunks[last(shoot_range_R)].psi_end - psi_ua_R = sing[j].psi_ua_right - @info " uShootR[$j]: shoot_range=$(shoot_range_R), psi_chunk=$(@sprintf("%.6f", psi_surf_R)), psi_ua=$(@sprintf("%.6f", psi_ua_R)), psi_mid=$(@sprintf("%.6f", psi_mid_R)), Δψ_fix=$(@sprintf("%.6e", psi_ua_R - psi_surf_R))" - end - if can_reintegrate && !isempty(shoot_range_R) - uShootR[j] = integrate_fm_with_ua_ic(chunks, shoot_range_R, sing[j].ua_right, - ctrl, equil, ffit, intr; backward=false, psi_ua=sing[j].psi_ua_right) - else - T_init = has_ua ? T_right_mats[j] : nothing - uShootR[j] = assemble_fm_matrix(propagators, shoot_range_R; T_init=T_init) - end - - # uShootL[j>=2]: backward from surface j left to midpoint. uShootL[1] handled below. - j == 1 && continue - shoot_range_L = _midpoint_shoot_range(chunks, i_crossings, j, msing; side=:left) - if debug - psi_mid = chunks[first(shoot_range_L)].psi_start - psi_surf = chunks[last(shoot_range_L)].psi_end - psi_ua_L = sing[j].psi_ua_left - @info " uShootL[$j]: shoot_range=$(shoot_range_L), psi_mid=$(@sprintf("%.6f", psi_mid)), psi_chunk=$(@sprintf("%.6f", psi_surf)), psi_ua=$(@sprintf("%.6f", psi_ua_L)), Δψ_fix=$(@sprintf("%.6e", psi_ua_L - psi_surf))" - end - if can_reintegrate && !isempty(shoot_range_L) - uShootL[j] = integrate_fm_with_ua_ic(chunks, shoot_range_L, sing[j].ua_left, - ctrl, equil, ffit, intr; backward=true, psi_ua=sing[j].psi_ua_left) - else - T_init = has_ua ? T_left_mats[j] : nothing - uShootL[j] = assemble_fm_matrix(propagators, shoot_range_L; T_init=T_init) - end - end - - uAxis, i_axis_mid = _build_conditioned_axis_propagator(propagators, i_crossings, N) - uShootL[1] = _build_uShootL_first(propagators, chunks, i_crossings, sing, - T_left_mats, has_ua, can_reintegrate, i_axis_mid, - ctrl, equil, ffit, intr, N) - if debug - shoot_range_L1 = (i_axis_mid + 1):(i_crossings[1] - 1) - @info " Axis propagator: $(i_axis_mid) chunks, cond=$(@sprintf("%.2e", cond(uAxis)))" - @info " uShootL[1]: range=$(shoot_range_L1), cond=$(@sprintf("%.2e", cond(uShootL[1])))" - end - return uShootR, uShootL, uAxis -end - -# Locate the chunk midpoint between two singular surfaces (or surface↔edge) in ψ space. -# Side `:right` returns the range from chunk(i_crossings[j]+1) to the ψ-midpoint chunk -# (or to the last chunk for j==msing). Side `:left` returns the range from the midpoint -# chunk+1 to chunk(i_crossings[j]-1). The ψ midpoint is used (not the chunk-index midpoint) -# because chunks near singularities are packed tighter in ψ — Fortran convention. -function _midpoint_shoot_range(chunks::Vector{IntegrationChunk}, i_crossings::Vector{Int}, - j::Int, msing::Int; side::Symbol) - if side === :right - j == msing && return (i_crossings[msing] + 1):length(chunks) - chunk_start = i_crossings[j] + 1 - chunk_end = i_crossings[j+1] - 1 - else # :left, j >= 2 - chunk_start = i_crossings[j-1] + 1 - chunk_end = i_crossings[j] - 1 - end - psi_mid_target = (chunks[chunk_start].psi_start + chunks[chunk_end].psi_end) / 2 - i_mid_inter = chunk_start - for ic in chunk_start:chunk_end-1 - if chunks[ic].psi_end >= psi_mid_target - i_mid_inter = ic - break - end - i_mid_inter = ic - end - return side === :right ? (chunk_start:i_mid_inter) : ((i_mid_inter + 1):chunk_end) -end - -# Build a well-conditioned axis propagator by forward-propagating [0; I] through the -# pre-first-crossing chunks with QR fixup after each chunk (Fortran ode_fixup). The axis -# midpoint is placed one chunk before the first surface so that uShootL[1] covers only the -# last chunk, keeping it well-conditioned. -function _build_conditioned_axis_propagator(propagators::Vector{ChunkPropagator}, - i_crossings::Vector{Int}, N::Int) - n_pre_cross = i_crossings[1] - 1 - i_axis_mid = max(1, n_pre_cross - 1) - uAxis = zeros(ComplexF64, 2N, N) - for i in 1:N - uAxis[N+i, i] = 1 - end - for ic in 1:i_axis_mid - prop = propagators[ic] - upper_old = uAxis[1:N, :] - lower_old = uAxis[N+1:2N, :] - uAxis[1:N, :] .= prop.block_upper_ic[:,:,1] * upper_old .+ prop.block_lower_ic[:,:,1] * lower_old - uAxis[N+1:2N, :] .= prop.block_upper_ic[:,:,2] * upper_old .+ prop.block_lower_ic[:,:,2] * lower_old - Q, _ = qr(uAxis) - uAxis .= Matrix(Q)[:, 1:N] - end - for j in 1:N - uAxis[:, j] ./= norm(@view uAxis[:, j]) - end - return uAxis, i_axis_mid -end - -# Build uShootL[1]: backward propagator from surface 1 left boundary to the axis midpoint. -# Falls back to T_left_mats[1] (or identity if no ua) when there's only 1 chunk before the -# first crossing. -function _build_uShootL_first(propagators::Vector{ChunkPropagator}, - chunks::Vector{IntegrationChunk}, i_crossings::Vector{Int}, - sing::Vector{SingType}, T_left_mats::Vector{Matrix{ComplexF64}}, - has_ua::Bool, can_reintegrate::Bool, i_axis_mid::Int, - ctrl, equil, ffit, intr::ForceFreeStatesInternal, N::Int) - shoot_range_L1 = (i_axis_mid + 1):(i_crossings[1] - 1) - if can_reintegrate && !isempty(shoot_range_L1) - return integrate_fm_with_ua_ic(chunks, shoot_range_L1, sing[1].ua_left, - ctrl, equil, ffit, intr; - backward=true, psi_ua=sing[1].psi_ua_left) - elseif !isempty(shoot_range_L1) - return assemble_fm_matrix(propagators, shoot_range_L1; - T_init=has_ua ? T_left_mats[1] : nothing) - else - return has_ua ? T_left_mats[1] : Matrix{ComplexF64}(I, 2N, 2N) - end -end - -# Assemble the BVP matrix M with S-based axis BC. The Riccati S matrix at surface 1's left -# boundary encodes the axis BC (U₁ = S·U₂) in a well-conditioned form (cond ~ 10⁶), avoiding -# the catastrophically ill-conditioned axis FM. Fortran-matched structure with -# nMat = (2 + 4·msing)·N. Returns (M, nMat, col_edge). -function _assemble_bvp_S_axis(uShootR::Vector{Matrix{ComplexF64}}, - uShootL::Vector{Matrix{ComplexF64}}, - uAxis::Matrix{ComplexF64}, ipert_all::Vector{Int}, - msing::Int, N::Int, - wv::Union{Nothing,Matrix{ComplexF64}}, psio::Float64) - # STRIDE global BVP block structure [Glasser-Kolemen 2018 PoP 25, 032501 Eq. 37]. - nMat = (2 + 4 * msing) * N - col_axis = 1:N - col_edge = (nMat - N + 1):nMat - M = zeros(ComplexF64, nMat, nMat) - - # Axis matching: uShootL[1] · c_left[1] = uAxis · c_axis (2N equations) - M[1:2N, _col_left(1, N)] .= uShootL[1] - M[1:2N, col_axis] .= -uAxis - row_offset = 2N - - for j in 1:msing - ipert_j = ipert_all[j] - # Crossing: non-resonant modes continuity (asymptotic basis = identity) - for i in 1:2N - if i != ipert_j && i != ipert_j + N - row_offset += 1 - M[row_offset, _col_left(j, N)[i]] = 1 - M[row_offset, _col_right(j, N)[i]] = -1 - end - end - - junc_rows = (row_offset + 1):(row_offset + 2N) - if j < msing - # Midpoint matching between consecutive surfaces - M[junc_rows, _col_right(j, N)] .= -uShootR[j] - M[junc_rows, _col_left(j+1, N)] .= uShootL[j+1] - else - # Edge junction - M[junc_rows, _col_right(msing, N)] .= uShootR[msing] - if wv !== nothing - M[junc_rows[1:N], col_edge] .= -I(N) - M[junc_rows[N+1:end], col_edge] .= wv .* psio^2 - else - M[junc_rows[N+1:end], col_edge] .= -I(N) - end - end - row_offset = last(junc_rows) - end - - # Driving rows: set big-solution coefficient = 1 at each surface (asymptotic basis) - for j in 1:msing - ipert_j = ipert_all[j] - row_offset += 1 - M[row_offset, _col_left(j, N)[ipert_j]] = 1 - row_offset += 1 - M[row_offset, _col_right(j, N)[ipert_j]] = 1 - end - @assert row_offset == nMat "Row count mismatch: expected $nMat, got $row_offset" - return M, nMat, col_edge -end - -# Coil-response block for the Eq. (37) edge [Glasser-Kolemen 2018 PoP 25, 032501]: impose the rpec edge -# boundary condition — identity edge plus a unit source per poloidal mode, matching RDCON's -# `gal_set_boundary` rpec branch — then read the small-solution (+N slot) coefficient at every surface. -# The BC is the same for every edge mode, so the matrix is factorized once and all N modes are solved -# together as columns of one right-hand side. Returns delta_coil (2·msing × N), rows = surface side. -function _solve_bvp_edge_coil(M::Matrix{ComplexF64}, col_edge, msing::Int, N::Int, ipert_all::Vector{Int}) - nMat = size(M, 1) - bot = (nMat-2msing-N+1):(nMat-2msing) # Eq. (38) bottom rows, carrying the W_V block - Mc = copy(M) - Mc[bot, :] .= 0 - Mc[bot, col_edge] .= I(N) # Dirichlet edge: the edge coefficients equal the source - B = zeros(ComplexF64, nMat, N) - B[bot, :] .= I(N) # unit drive, one column per edge poloidal mode - X = lu(Mc) \ B - delta_coil = zeros(ComplexF64, 2msing, N) - for j in 1:msing - row_left = _col_left(j, N)[ipert_all[j]+N] - row_right = _col_right(j, N)[ipert_all[j]+N] - @views delta_coil[2j-1, :] .= X[row_left, :] - @views delta_coil[2j, :] .= X[row_right, :] - end - return delta_coil -end - -# Fallback BVP assembly with FM-based axis BC (used when no Riccati S matrices are available). -# Uses the conditioned axis propagator Phi_R[1][:,N+1:2N] in place of S-axis matching. -function _assemble_bvp_FM_axis(Phi_L_mats::Vector{Matrix{ComplexF64}}, - Phi_R_mats::Vector{Matrix{ComplexF64}}, ipert_all::Vector{Int}, - msing::Int, N::Int, - T_left_inv::Vector{Matrix{ComplexF64}}, - T_right_inv::Vector{Matrix{ComplexF64}}, has_ua::Bool, - wv::Union{Nothing,Matrix{ComplexF64}}, psio::Float64) - nMat = (2 + 4 * msing) * N - col_axis = 1:N - col_edge = (N + 4N*msing + 1):nMat - M = zeros(ComplexF64, nMat, nMat) - - M[1:2N, (N+1):(N+2N)] .= Phi_L_mats[1] - M[1:2N, col_axis] .= -view(Phi_R_mats[1], :, N+1:2N) - - row_drive_base = 2N + (4N-2)*msing - for j in 1:msing - ipert_j = ipert_all[j] - cl = _col_left(j, N) - cr = _col_right(j, N) - row_cont = 2N + (4N-2)*(j-1) - for i in 1:2N - if i != ipert_j && i != ipert_j + N - row_cont += 1 - M[row_cont, cl[i]] = 1 - M[row_cont, cr[i]] = -1 - end - end - junc_rows = (row_cont + 1):(2N + (4N-2)*j) - if j < msing - M[junc_rows, cr] .= Phi_R_mats[j+1] - M[junc_rows, _col_left(j+1, N)] .= -Phi_L_mats[j+1] - else - M[junc_rows, cr] .= Phi_R_mats[msing+1] - if wv !== nothing - M[junc_rows[1:N], col_edge] .= -I(N) - M[junc_rows[N+1:end], col_edge] .= wv .* psio^2 - else - M[junc_rows[N+1:end], col_edge] .= -I(N) - end - end - if has_ua - M[row_drive_base + 2j-1, cl] .= T_left_inv[j][ipert_j, :] - M[row_drive_base + 2j, cr] .= T_right_inv[j][ipert_j, :] - else - M[row_drive_base + 2j-1, cl[ipert_j]] = 1 - M[row_drive_base + 2j, cr[ipert_j]] = 1 - end - end - return M, nMat, col_edge -end - -# Solve the BVP for each driving configuration and apply the PEST3 four-term combination. -# Promotes to Complex{Double64} if ctrl.extended_precision_bvp (default true) — the PEST3 -# combination subtracts dp_raw entries up to ~3×10⁴ larger than the result, and Float64 -# precision lets the imaginary part drift 2–5× on DIIID-class equilibria. -function _solve_bvp_and_combine_pest3(M::Matrix{ComplexF64}, msing::Int, N::Int, nMat::Int, - use_S_axis::Bool, ipert_all::Vector{Int}, col_edge, - ctrl, debug::Bool) - s2 = 2 * msing - Tc = (ctrl === nothing || ctrl.extended_precision_bvp) ? Complex{Double64} : ComplexF64 - M_solve = Tc.(M) - - M_lu = lu(M_solve; check=false) - use_lu = issuccess(M_lu) - M_pinv = use_lu ? nothing : pinv(M_solve) - if !use_lu - @warn "Δ' BVP: LU factorization singular (rank $(rank(M))/$nMat), using pseudo-inverse fallback" - end - - dp_raw = zeros(Tc, s2, s2) - b = zeros(Tc, nMat) - for jsing in 1:msing, side in 1:2 - dRow = 2jsing - (2 - side) - fill!(b, 0) - drive_row = use_S_axis ? (nMat - s2 + dRow) : (2N + (4N-2)*msing + dRow) - b[drive_row] = 1 - x = use_lu ? (M_lu \ b) : (M_pinv * b) - - debug && _log_bvp_solve(x, b, M_solve, jsing, side, dRow, msing, N, - ipert_all, col_edge, use_S_axis) - - for ksing in 1:msing - ipert_k = ipert_all[ksing] - dp_raw[dRow, 2ksing-1] = x[_col_left(ksing, N)[ipert_k+N]] - dp_raw[dRow, 2ksing] = x[_col_right(ksing, N)[ipert_k+N]] - end - end - - # PEST3 four-term combination [Chance PPPL-2527; Glasser-Kolemen 2018 PoP 25, 032501 Eq. 31]. - # Δ'[i,j] = (NW − NE − SW + SE) on each 2×2 block of dp_raw, in extended precision. - deltap_ext = zeros(Tc, msing, msing) - for i in 1:msing, j in 1:msing - deltap_ext[i, j] = dp_raw[2i, 2j] - dp_raw[2i, 2j-1] - dp_raw[2i-1, 2j] + dp_raw[2i-1, 2j-1] - end - deltap = ComplexF64.(deltap_ext) - - debug && _log_bvp_pest3(dp_raw, deltap, s2, msing, Tc) - # Return the PEST3-combined matrix AND the raw 2msing×2msing D' matrix (ComplexF64 - # for compatibility with downstream pest3_decompose / HDF5 writer). - return deltap, ComplexF64.(dp_raw) -end - -# Logging helpers for `compute_delta_prime_matrix!`. Called only when debug=true. -function _log_bvp_setup(chunks, sing, S_at_surface_left, use_S_axis, has_ua, - Phi_L_mats, Phi_R_mats, Phi_R_halves, ipert_all, wv, psio, N, msing) - @info "Δ' BVP: $(length(chunks)) chunks, $msing surfaces, N=$N" - @info "Δ' BVP: Axis BC: $(use_S_axis ? "S-based (Riccati)" : "FM-based (conditioned)")" - @info "Δ' BVP: Asymptotic basis: $(has_ua ? "available" : "NOT available (raw basis driving)")" - if use_S_axis - for j in 1:msing - @info " S_left[$j]: max=$(@sprintf("%.2e", maximum(abs, S_at_surface_left[j]))), cond=$(@sprintf("%.2e", cond(S_at_surface_left[j])))" - end - end - if has_ua - for j in 1:msing - sp = sing[j] - T_l = [sp.ua_left[:,:,1]; sp.ua_left[:,:,2]] - T_r = [sp.ua_right[:,:,1]; sp.ua_right[:,:,2]] - @info " Surface $j: cond(T_left)=$(@sprintf("%.2e", cond(T_l))), cond(T_right)=$(@sprintf("%.2e", cond(T_r)))" - ipert_j = ipert_all[j] - @info " Surface $j ua_left (ipert=$ipert_j, psi_ua_left=$(@sprintf("%.8f", sp.psi_ua_left))):" - for i in 1:min(5, N) - @info " ua($i,$ipert_j,1)=$(@sprintf("%16.8e %16.8e", real(sp.ua_left[i,ipert_j,1]), imag(sp.ua_left[i,ipert_j,1]))) ua($i,$ipert_j,2)=$(@sprintf("%16.8e %16.8e", real(sp.ua_left[i,ipert_j,2]), imag(sp.ua_left[i,ipert_j,2])))" - end - @info " small: ua(1,$(ipert_j+N),1)=$(@sprintf("%16.8e %16.8e", real(sp.ua_left[1,ipert_j+N,1]), imag(sp.ua_left[1,ipert_j+N,1])))" - end - end - for j in 1:msing-1 - Phi_L_h, Phi_R_h = Phi_R_halves[j] - @info " Inter-surface $j→$(j+1): half_L cond=$(@sprintf("%.2e",cond(Phi_L_h))), half_R cond=$(@sprintf("%.2e",cond(Phi_R_h))), full cond=$(@sprintf("%.2e",cond(Phi_R_mats[j+1])))" - end - @info " Phi_R[$(msing+1)] (edge): cond=$(@sprintf("%.2e",cond(Phi_R_mats[msing+1])))" - for j in 1:msing - @info " Surface $j (m=$(sing[j].m[1])): ipert=$(ipert_all[j]), cond(Phi_L)=$(@sprintf("%.2e", cond(Phi_L_mats[j])))" - end - @info "Δ' BVP: Vacuum BC $(wv === nothing ? "off (conducting wall)" : "on (psio=$psio)")" - for j in 1:msing - if !isempty(sing[j].delta_prime) - @info " Surface $j ca-based Δ' = $(@sprintf("%.6f%+.6fi", real(sing[j].delta_prime[1]), imag(sing[j].delta_prime[1])))" - end - end -end - -function _log_S_axis_shooting_propagators(uShootR, uShootL, uAxis, S_at_surface_left, - T_left_mats, ipert_all, has_ua, msing, N) - @info " Shooting propagators (S-based axis BC, no axis unknowns):" - for j in 1:msing - shoot_R_str = @sprintf("%.2e", cond(uShootR[j])) - shoot_L_str = j >= 2 ? @sprintf("%.2e", cond(uShootL[j])) : "N/A (S axis BC)" - @info " uShootL[$j]: cond=$shoot_L_str, uShootR[$j]: cond=$shoot_R_str" - end - S1 = S_at_surface_left[1] - if has_ua - T1 = T_left_mats[1] - axis_BC = T1[1:N, :] - S1 * T1[N+1:2N, :] - @info " S-axis BC matrix: cond=$(@sprintf("%.2e", cond(axis_BC)))" - end - for j in 1:msing - ipert_j = ipert_all[j] - col_norms_R = [norm(view(uShootR[j], :, k)) for k in 1:2N] - @info " uShootR[$j] column norms: min=$(@sprintf("%.2e", minimum(col_norms_R))), max=$(@sprintf("%.2e", maximum(col_norms_R)))" - @info " uShootR[$j] col ipert=$ipert_j norm=$(@sprintf("%.2e", col_norms_R[ipert_j])), col ipert+N=$(ipert_j+N) norm=$(@sprintf("%.2e", col_norms_R[ipert_j+N]))" - if j >= 2 - col_norms_L = [norm(view(uShootL[j], :, k)) for k in 1:2N] - @info " uShootL[$j] column norms: min=$(@sprintf("%.2e", minimum(col_norms_L))), max=$(@sprintf("%.2e", maximum(col_norms_L)))" - @info " uShootL[$j] col ipert=$ipert_j norm=$(@sprintf("%.2e", col_norms_L[ipert_j])), col ipert+N=$(ipert_j+N) norm=$(@sprintf("%.2e", col_norms_L[ipert_j+N]))" - end - end - for j in 1:msing-1 - mid_block = hcat(uShootR[j], -uShootL[j+1]) - @info " Midpoint $j→$(j+1): cond([uShootR[$j] | -uShootL[$(j+1)]]) = $(@sprintf("%.2e", cond(mid_block)))" - col_norms_Ljp1 = [norm(view(uShootL[j+1], :, k)) for k in 1:2N] - @info " uShootL[$(j+1)] all col norms: $([(@sprintf("%.2e", c)) for c in col_norms_Ljp1])" - end -end - -function _log_bvp_solve(x, b, M_solve, jsing, side, dRow, msing, N, - ipert_all, col_edge, use_S_axis) - residual = norm(ComplexF64.(M_solve * x - b)) - side_str = side == 1 ? "left" : "right" - @info " BVP solve: jsing=$jsing side=$side_str (dRow=$dRow): ||Mx-b||=$(@sprintf("%.2e", residual)), ||x||=$(@sprintf("%.2e", Float64(norm(x))))" - for ks in 1:msing - ipert_ks = ipert_all[ks] - cl = _col_left(ks, N) - cr = _col_right(ks, N) - xl_big = ComplexF64(x[cl[ipert_ks]]) - xl_small = ComplexF64(x[cl[ipert_ks+N]]) - xr_big = ComplexF64(x[cr[ipert_ks]]) - xr_small = ComplexF64(x[cr[ipert_ks+N]]) - @info " surf $ks: x_left[big]=$(@sprintf("%+.4e%+.4ei", real(xl_big), imag(xl_big))), x_left[small]=$(@sprintf("%+.4e%+.4ei", real(xl_small), imag(xl_small)))" - @info " surf $ks: x_right[big]=$(@sprintf("%+.4e%+.4ei", real(xr_big), imag(xr_big))), x_right[small]=$(@sprintf("%+.4e%+.4ei", real(xr_small), imag(xr_small)))" - @info " surf $ks: ||x_left||=$(@sprintf("%.2e", Float64(norm(x[cl])))), ||x_right||=$(@sprintf("%.2e", Float64(norm(x[cr]))))" - end - if use_S_axis - @info " ||x_edge||=$(@sprintf("%.2e", Float64(norm(x[col_edge]))))" - end -end - -function _log_bvp_pest3(dp_raw, deltap, s2, msing, Tc) - @info "Δ' BVP: Full dp_raw matrix ($(s2)×$(s2)) [$(Tc)]:" - for i in 1:s2 - row_str = join([@sprintf("%+.6e", Float64(real(dp_raw[i,j]))) for j in 1:s2], " ") - @info " dp_raw[$i,:] = $row_str" - end - @info "Δ' BVP: Raw dp diagonal = $([@sprintf("%.4f%+.4fi", Float64(real(dp_raw[i,i])), Float64(imag(dp_raw[i,i]))) for i in 1:s2])" - @info "Δ' BVP: deltap diagonal = $([@sprintf("%.4f%+.4fi", real(deltap[i,i]), imag(deltap[i,i])) for i in 1:msing])" -end - -""" - pest3_decompose(dp_raw::AbstractMatrix) -> (A', B', Γ', Δ') - -Rotate the raw 2m×2m outer-region matching matrix `dp_raw` (side-major -ordering `[L_s1, R_s1, L_s2, R_s2, …]`) into the Pletzer–Dewar 1991 parity -blocks. Given rows and columns paired by surface (odd index = left, even -index = right), the Fortran RDCON parity combination is - -``` -A'(i,j) = RR + RL + LR + LL (even-i, even-j) — interchange↔interchange -B'(i,j) = RR − RL + LR − LL (even-i, odd-j) — interchange↔tearing -Γ'(i,j) = RR + RL − LR − LL (odd-i, even-j) — tearing↔interchange -Δ'(i,j) = RR − RL − LR + LL (odd-i, odd-j) — tearing↔tearing -``` - -where `RR = dp_raw[2i, 2j]`, `RL = dp_raw[2i, 2j−1]`, -`LR = dp_raw[2i−1, 2j]`, `LL = dp_raw[2i−1, 2j−1]`. Each block is m×m. - -Matches Fortran exactly — no ½ prefactor (Pletzer–Dewar multiply by ½, but -the Fortran RDCON code leaves it commented out and our Julia port follows -Fortran to keep the benchmark bit-identical; the prefactor cancels in -`det(D' − D(γ)) = 0`). - -The Δ' block returned here equals `intr.delta_prime_matrix` (the m×m PEST3 -tearing projection computed inside `compute_delta_prime_matrix!`). - -# Arguments - - - `dp_raw` — 2m×2m complex matrix (typically `intr.delta_prime_raw`). - -# Returns - -Named tuple `(A=A', B=B', Γ=Gp, Δ=Dp)` of four m×m complex matrices. In the -full `det(D' − D(γ)) = 0` eigenvalue problem, these fill the 2m×2m outer -matrix as `D' = [[A' B'] [Γ' Δ']]` with the interchange channel (Glasser -stabilization) in the upper-left block and the tearing channel in the -lower-right. -""" -function pest3_decompose(dp_raw::AbstractMatrix) - s2 = size(dp_raw, 1) - size(dp_raw, 2) == s2 || - throw(ArgumentError("pest3_decompose: dp_raw must be square, got $(size(dp_raw))")) - iseven(s2) || - throw(ArgumentError("pest3_decompose: dp_raw side must be 2m for integer m, got $s2")) - m = s2 ÷ 2 - Tc = eltype(dp_raw) - Ap = zeros(Tc, m, m) - Bp = zeros(Tc, m, m) - Gp = zeros(Tc, m, m) - Dp = zeros(Tc, m, m) - for i in 1:m, j in 1:m - LL = dp_raw[2i-1, 2j-1] - LR = dp_raw[2i-1, 2j] - RL = dp_raw[2i, 2j-1] - RR = dp_raw[2i, 2j] - Ap[i, j] = RR + RL + LR + LL - Bp[i, j] = RR - RL + LR - LL - Gp[i, j] = RR + RL - LR - LL - Dp[i, j] = RR - RL - LR + LL - end - return (A=Ap, B=Bp, Γ=Gp, Δ=Dp) -end - -""" - riccati_der!(du, u, params, psieval) - -Evaluate the explicit dual Riccati ODE right-hand side: - dS/dψ = w†·F̄⁻¹·w - S·Ḡ·S, w = Q - K̄·S - -where Q = diag(1/(m - n·q)) is the diagonal singular factor matrix. -The identity slice u[:,:,2] = I does not evolve (du[:,:,2] = 0). - -**REFERENCE IMPLEMENTATION — not called in production.** The explicit Riccati ODE is -numerically unstable for explicit solvers: the quadratic S·Ḡ·S term blows up when K̄·S ≫ Q. -The production path integrates `sing_der!` with periodic `renormalize_riccati_inplace!` -instead (see module docstring). Kept here for documentation of Eq. 19 in source form and -for future use with implicit solvers; exercised only by unit tests that verify the formula. - -See: Glasser (2018) Phys. Plasmas 25, 032507 — Eq. 19 (dual Riccati form) -""" -@with_pool pool function riccati_der!( - du::Array{ComplexF64,3}, - u::Array{ComplexF64,3}, - params::Tuple{ForceFreeStatesControl,Equilibrium.PlasmaEquilibrium, - FourFitVars,ForceFreeStatesInternal,OdeState,IntegrationChunk}, - psieval::Float64 -) - - _, equil, ffit, intr, odet, _ = params - - Npert = intr.numpert_total - S = @view u[:, :, 1] - dS = @view du[:, :, 1] - @view(du[:, :, 2]) .= 0 # identity does not evolve - - # Compute singfac = 1/(m - n·q) as column vector Q = diag(singfac_vec) - # [Glasser 2016 eq. 24] - singfac_vec = acquire!(pool, Float64, Npert) - singfac_mat = reshape(singfac_vec, intr.mpert, intr.npert) - odet.q = equil.profiles.q_spline(psieval; hint=odet.spline_hint) - singfac_mat .= 1.0 ./ ((intr.mlow:intr.mhigh) .- odet.q .* (intr.nlow:intr.nhigh)') - - # Allocate temporaries from pool - fmat_lower = acquire!(pool, ComplexF64, Npert, Npert) - kmat = similar!(pool, fmat_lower) - gmat = similar!(pool, fmat_lower) - w = similar!(pool, fmat_lower) # w = Q - K̄·S - v = similar!(pool, fmat_lower) # v = F̄⁻¹·w (then reused for S·Ḡ·S) - tmp = similar!(pool, fmat_lower) # scratch - - # Evaluate F̄ (Cholesky factor), K̄, Ḡ splines at current ψ - ffit.fmats_lower(vec(fmat_lower), psieval; hint=ffit._hint) - ffit.kmats(vec(kmat), psieval; hint=ffit._hint) - ffit.gmats(vec(gmat), psieval; hint=ffit._hint) - - # w = Q - K̄·S: w[i,j] = singfac_vec[i]·δ_ij - (K̄·S)[i,j] - # Q is DIAGONAL (singfac_vec[i] only on i==j), so we cannot broadcast singfac_vec - # over all columns — that would give the wrong off-diagonal values. - mul!(w, kmat, S) # w = K̄·S - @. w = -w # w = -K̄·S - for i in 1:Npert - @inbounds w[i, i] += singfac_vec[i] # add diagonal Q: w = Q - K̄·S - end - - # v = F̄⁻¹·w (in-place Cholesky solve with stored lower-triangular factor) - v .= w - ldiv!(LowerTriangular(fmat_lower), v) - ldiv!(UpperTriangular(fmat_lower'), v) - - # dS = w†·v - S·Ḡ·S [Glasser 2018 eq. 19, dual Riccati] - mul!(dS, adjoint(w), v) # dS = w†·v - - # Subtract S·Ḡ·S (reuse v and tmp to avoid extra allocation) - mul!(tmp, gmat, S) # tmp = Ḡ·S - mul!(v, S, tmp) # v = S·Ḡ·S - dS .-= v -end - -""" - riccati_integrator_callback!(integrator) - -Callback function for the Riccati ODE integrator. Handles tolerance updates, -renormalization, and storage at each step. - -Uses `sing_der!` as the ODE RHS: u[:,:,1] = U₁ (starts as S), u[:,:,2] = U₂ (starts as I). -When max(|U₁|) or max(|U₂|) exceeds `ctrl.ucrit`, applies `renormalize_riccati_inplace!` -to compute S = U₁·U₂⁻¹ and reset U₂ = I. This is the Riccati analogue of Gaussian -reduction in the standard `integrator_callback!`, and keeps the ODE inputs bounded. -""" -function riccati_integrator_callback!(integrator) - - ctrl, _, _, intr, odet, chunk = integrator.p - - odet.total_steps += 1 # count every accepted solver step (saved or not), as segment_callback! does - - # Use unified tolerance (matches integrate_el_region! on develop) - integrator.opts.reltol = ctrl.eulerlagrange_tolerance - - # Renormalize when norms exceed ucrit (analogous to Gaussian reduction in integrator_callback!) - # During sing_der! integration: u[:,:,1]=U₁ (grows), u[:,:,2]=U₂ (grows). - # Renorm computes S = U₁·U₂⁻¹ and resets U₂ = I, keeping inputs bounded. - if maximum(abs, @view(integrator.u[:, :, 1])) > ctrl.ucrit || - maximum(abs, @view(integrator.u[:, :, 2])) > ctrl.ucrit - renormalize_riccati_inplace!(integrator.u, intr.numpert_total) - end - - # Determine if we should save this step. Always save the first 1-2 steps of a segment - # and the last few steps near the right endpoint (relative band SAVE_NEAR_END_FRAC of the - # span, or absolute floor SAVE_NEAR_END_PSI for very short chunks); save every save_interval-th - # step in between. - psi_range = abs(integrator.sol.prob.tspan[2] - integrator.sol.prob.tspan[1]) - psi_remaining = abs(integrator.sol.prob.tspan[2] - integrator.t) - near_end = psi_remaining < SAVE_NEAR_END_FRAC * psi_range || psi_remaining < SAVE_NEAR_END_PSI - steps_in_segment = length(integrator.sol.t) - near_start = steps_in_segment <= 2 - should_save = near_start || near_end || (odet.step % ctrl.save_interval == 0) - - if should_save - store_ode_data!(odet, integrator.t, integrator.u) - end -end - -""" - riccati_integrate_chunk!(odet, ctrl, equil, ffit, intr, chunk) - -Integrate the dual Riccati ODE from `chunk.psi_start` to `chunk.psi_end`. - -Uses `sing_der!` as the ODE RHS with `riccati_integrator_callback!`, which applies -`renormalize_riccati_inplace!` (instead of Gaussian reduction) when norms exceed ucrit. -Starting state: u[:,:,1] = S_prev, u[:,:,2] = I (set by initialization or previous renorm). -Ending state: u[:,:,1] = U₁, u[:,:,2] = U₂ (ratio S = U₁·U₂⁻¹ is the updated Riccati matrix). -""" -function riccati_integrate_chunk!( - odet::OdeState, ctrl::ForceFreeStatesControl, equil::Equilibrium.PlasmaEquilibrium, - ffit::FourFitVars, intr::ForceFreeStatesInternal, chunk::IntegrationChunk -) - cb = DiscreteCallback((u, t, integrator) -> true, riccati_integrator_callback!) - rtol = ctrl.eulerlagrange_tolerance - prob = ODEProblem(sing_der!, odet.u, (chunk.psi_start, chunk.psi_end), - (ctrl, equil, ffit, intr, odet, chunk)) - sol = solve(prob, Vern9(); reltol=rtol, callback=cb, save_everystep=false, save_end=true) - odet.u .= sol.u[end] - odet.psifac = sol.t[end] - # Renormalize end state to (S, I) convention for the next chunk. - # When a crossing follows (needs_crossing=true), skip renorm so that ca_l is computed - # from the bounded (U₁, U₂) state in riccati_cross_ideal_singular_surf!: this gives - # consistent normalization with ca_r (also from pre-renorm state), enabling correct Δ'. - # The callback guarantees max(|U₁|), max(|U₂|) ≤ ucrit, so the state is bounded. - if !chunk.needs_crossing - renormalize_riccati_inplace!(odet.u, intr.numpert_total) - end -end - -""" - renormalize_riccati!(odet, intr) - -After a singular surface crossing, restore the canonical Riccati storage convention: - u[:,:,1] = S_new = U₁_new · U₂_new⁻¹ - u[:,:,2] = I - -`riccati_cross_ideal_singular_surf!` leaves u[:,:,1] = U₁_new and u[:,:,2] = U₂_new (not I), -so this step is required before continuing the Riccati integration. - -The u_store entry from the crossing correctly has U₁_new and U₂_new (stored before this call), -so `compute_smallest_eigenvalue` still computes U₁_new/U₂_new = S_new correctly. -""" -function renormalize_riccati!(odet::OdeState, intr::ForceFreeStatesInternal) - N = intr.numpert_total - # S_new = U₁_new · U₂_new⁻¹ (in-place to avoid allocation) - U2_copy = copy(@view odet.u[:, :, 2]) - rdiv!(@view(odet.u[:, :, 1]), lu!(U2_copy)) - # Reset U₂ = I - fill!(@view(odet.u[:, :, 2]), 0) - for i in 1:N - odet.u[i, i, 2] = 1 - end -end - -""" - renormalize_riccati_inplace!(u, N) - -In-place Riccati renormalization on an arbitrary N×N×2 array: - u[:,:,1] = U₁ · U₂⁻¹ (new S) - u[:,:,2] = I - -Used in `riccati_integrator_callback!` to renormalize the integrator's live state -when column norms grow beyond `ctrl.ucrit`, analogous to Gaussian reduction in the -standard ODE. This keeps the inputs to `sing_der!` bounded, preventing the same -exponential growth that occurs in the standard (non-Riccati) ODE without Gaussian reduction. -""" -function renormalize_riccati_inplace!(u::Array{ComplexF64,3}, N::Int) - U2_copy = copy(@view u[:, :, 2]) - rdiv!(@view(u[:, :, 1]), lu!(U2_copy)) - fill!(@view(u[:, :, 2]), 0) - for i in 1:N - u[i, i, 2] = 1 - end -end - -""" - riccati_cross_ideal_singular_surf!(odet, ctrl, equil, ffit, intr, ising) - -Cross a singular surface for the Riccati formulation. Replaces `cross_ideal_singular_surf!` -for the Riccati integration path with two key differences: - -1. **No Gaussian reduction**: `cross_ideal_singular_surf!` calls `compute_solution_norms!` - which applies Gaussian reduction to (S, I). This divides by pivot elements of S, which - can be near-zero (S = 0 at axis and grows slowly), producing NaN/Inf in U₂. For Riccati, - S is bounded so Gaussian reduction is unnecessary. - -2. **Direct column zeroing**: Instead of using the GR-sorted `odet.index` to identify the - column to zero, we use `ipert_res` directly (the resonant mode index). This is valid since - without GR there is no permutation applied to the columns of S. - -**Δ' normalization**: This function expects `odet.u` in the bounded (U₁, U₂) form produced by -`riccati_integrate_chunk!` with `needs_crossing=true` (final renorm skipped). ca_l is computed -from (U₁, U₂) before the crossing, and ca_r from (U₁_new, U₂_new) before `renormalize_riccati!`. -Since column `ipert_res` of [U₁_new; U₂_new] equals the introduced asymptotic solution exactly, -ca_r[ipert_res,ipert_res,2] = 1 regardless of other column normalizations. This gives a -physically meaningful Δ' = ca_r - ca_l with consistent left/right normalization. - -After the predictor step and asymptotic introduction, `renormalize_riccati!` is called -to restore the canonical (S_new, I) form before continuing integration. - -The u_store entry at the crossing step correctly stores (U₁_new, U₂_new) so that -`evaluate_stability_criterion!` can compute U₁_new / U₂_new = S_new correctly. -""" -function riccati_cross_ideal_singular_surf!( - odet::OdeState, ctrl::ForceFreeStatesControl, equil::Equilibrium.PlasmaEquilibrium, - ffit::FourFitVars, intr::ForceFreeStatesInternal, ising::Int -) - # Skip Gaussian reduction — S is bounded so no large-norm columns exist. - singp = intr.sing[ising] - dpsi = singp.psifac - odet.psifac # ψ_res - ψ_current (positive) - ipert_res = 1 .+ singp.m .- intr.mlow .+ (singp.n .- intr.nlow) .* intr.mpert - - sing_asymp_left, sing_asymp_right = _two_sided_singular_asymptotics(singp, ctrl, equil, ffit, intr) - _log_riccati_crossing_diagnostics(odet, intr, ising, singp, dpsi, sing_asymp_left, sing_asymp_right) - - _capture_left_crossing_data!(odet, singp, sing_asymp_left, dpsi, intr, ising) - _predict_across_singular_surface!(odet, ctrl, equil, ffit, intr, ising, ipert_res, dpsi, sing_asymp_right) - _capture_right_crossing_data!(odet, singp, sing_asymp_right, dpsi, intr, ising, ipert_res, ctrl) - - _stash_per_surface_delta_prime_stub!(odet, intr, ising, ipert_res, sing_asymp_right, equil, ctrl) - _store_crossing_step!(odet) - - # Restore canonical (S_new, I) form before continuing integration. - renormalize_riccati!(odet, intr) -end - -""" - _two_sided_singular_asymptotics(singp, ctrl, equil, ffit, intr) -> (left, right) - -Compute left- (`sig=-1`) and right- (`sig=+1`) side singular asymptotics matching -Fortran STRIDE's separate vmatl/vmatr (sing_vmat). Alpha is taken from the right -side and shared with the left. -""" -function _two_sided_singular_asymptotics(singp::SingType, ctrl::ForceFreeStatesControl, - equil::Equilibrium.PlasmaEquilibrium, ffit::FourFitVars, - intr::ForceFreeStatesInternal) - sing_asymp_right = compute_sing_asymptotics(singp, ctrl, equil, ffit, intr; sig=1.0) - sing_asymp_left = compute_sing_asymptotics(singp, ctrl, equil, ffit, intr; sig=-1.0, - alpha_override=sing_asymp_right.alpha) - return sing_asymp_left, sing_asymp_right -end - -# @debug-only per-crossing diagnostics. Enable via JULIA_DEBUG=GeneralizedPerturbedEquilibrium. -function _log_riccati_crossing_diagnostics(odet, intr, ising, singp, dpsi, sing_asymp_left, sing_asymp_right) - @debug begin - ipert_res_diag = 1 .+ singp.m .- intr.mlow .+ (singp.n .- intr.nlow) .* intr.mpert - msg = " ising=$ising: psi_sing=$(@sprintf("%.10f", singp.psifac)), psi_eval=$(@sprintf("%.10f", odet.psifac)), dpsi=$(@sprintf("%.10e", dpsi))\n" - msg *= " alpha_L = $(sing_asymp_left.alpha), alpha_R = $(sing_asymp_right.alpha)\n" - for ip in ipert_res_diag - msg *= " vmatL[0] big: vmat[$ip,$ip,1,1]=$(@sprintf("%.8e", real(sing_asymp_left.vmat[ip,ip,1,1]))), vmat[$ip,$ip,2,1]=$(@sprintf("%.8e", real(sing_asymp_left.vmat[ip,ip,2,1])))\n" - msg *= " vmatR[0] big: vmat[$ip,$ip,1,1]=$(@sprintf("%.8e", real(sing_asymp_right.vmat[ip,ip,1,1]))), vmat[$ip,$ip,2,1]=$(@sprintf("%.8e", real(sing_asymp_right.vmat[ip,ip,2,1])))\n" - end - msg - end -end - -# Capture left-side asymptotic data into odet.ca_l and singp.ua_left/psi_ua_left. -function _capture_left_crossing_data!(odet::OdeState, singp::SingType, sing_asymp_left, - dpsi::Float64, intr::ForceFreeStatesInternal, ising::Int) - ua = sing_get_ua(sing_asymp_left, dpsi) - singp.ua_left = copy(ua) - singp.psi_ua_left = odet.psifac - odet.ca_l[:, :, :, ising] .= sing_get_ca(odet.u, ua, intr) -end - -# Trapezoidal predictor across the singular surface: zero the resonant columns, -# evaluate sing_der! on both sides, advance odet by (du1 + du2)·dpsi, and jump -# odet.psifac to the right side. The zeroed columns stay zero through the predictor -# since du[:, ipert_res, :] = 0 when u[:, ipert_res, :] = 0. -function _predict_across_singular_surface!(odet::OdeState, ctrl::ForceFreeStatesControl, - equil::Equilibrium.PlasmaEquilibrium, ffit::FourFitVars, - intr::ForceFreeStatesInternal, ising::Int, - ipert_res, dpsi::Float64, sing_asymp_right) - if ctrl.kinetic_factor == 0 - for i in eachindex(sing_asymp_right.r1) - odet.u[:, ipert_res[i], :] .= 0 - end - end - params = (ctrl, equil, ffit, intr, odet, IntegrationChunk(0.0, 0.0, false, ising, 1)) - du1 = zeros(ComplexF64, intr.numpert_total, intr.numpert_total, 2) - du2 = zeros(ComplexF64, intr.numpert_total, intr.numpert_total, 2) - sing_der!(du1, odet.u, params, odet.psifac) - odet.psifac += 2 * dpsi # jump to other side of singular surface - sing_der!(du2, odet.u, params, odet.psifac) - odet.u .+= (du1 .+ du2) .* dpsi -end - -# Inject the right-side small asymptotic into the resonant columns of (U₁_new, U₂_new), -# capture odet.ca_r, and save singp.ua_right / psi_ua_right. -# Column ipert_res of [U₁_new; U₂_new] = ua[:, ipert_res+N, :] (the introduced small asymptotic), -# so ca_r[ipert_res, ipert_res, 2] = 1 regardless of other columns' normalization. -function _capture_right_crossing_data!(odet::OdeState, singp::SingType, sing_asymp_right, - dpsi::Float64, intr::ForceFreeStatesInternal, ising::Int, - ipert_res, ctrl::ForceFreeStatesControl) - ua = sing_get_ua(sing_asymp_right, dpsi) - singp.ua_right = copy(ua) - singp.psi_ua_right = odet.psifac - if ctrl.kinetic_factor == 0 - for i in eachindex(sing_asymp_right.r1) - odet.u[ipert_res[i], :, :] .= 0 - odet.u[:, ipert_res[i], :] .= ua[:, ipert_res[i]+intr.numpert_total, :] - end - end - odet.ca_r[:, :, :, ising] .= sing_get_ca(odet.u, ua, intr) -end - -# STUB: per-surface ca-based Δ' (not physically valid; see SingType.delta_prime docstring). -# The canonical Δ' is intr.delta_prime_matrix from compute_delta_prime_matrix!. -function _stash_per_surface_delta_prime_stub!(odet::OdeState, intr::ForceFreeStatesInternal, - ising::Int, ipert_res, sing_asymp_right, - equil::Equilibrium.PlasmaEquilibrium, - ctrl::ForceFreeStatesControl) - ctrl.kinetic_factor == 0 || return - denom = (2π)^2 * equil.psio - n_res = length(sing_asymp_right.r1) - N = intr.numpert_total - resize!(intr.sing[ising].delta_prime, n_res) - intr.sing[ising].delta_prime_col = zeros(ComplexF64, N, n_res) - for i in eachindex(sing_asymp_right.r1) - Δca_col = (odet.ca_r[:, ipert_res[i], 2, ising] - odet.ca_l[:, ipert_res[i], 2, ising]) / denom - intr.sing[ising].delta_prime_col[:, i] .= Δca_col - intr.sing[ising].delta_prime[i] = Δca_col[ipert_res[i]] - end -end - -# Store (U₁_new, U₂_new) into u_store before renormalization so that -# evaluate_stability_criterion! can recover S_new = U₁_new / U₂_new via compute_smallest_eigenvalue. -function _store_crossing_step!(odet::OdeState) - store_ode_data!(odet, odet.psifac, odet.u) -end - -""" - integrate_propagator_chunk!(prop, chunk, ctrl, equil, ffit, intr, odet_proxy) - -Compute the fundamental matrix (propagator) for one integration chunk by solving the -EL ODE twice from identity-block initial conditions. - -The first solve uses IC = (I_N, 0_N) (U₁=I, U₂=0) and stores the result in -`prop.block_upper_ic`. The second uses IC = (0_N, I_N) (U₁=0, U₂=I) and stores -the result in `prop.block_lower_ic`. - -`odet_proxy` is a per-thread lightweight `OdeState` used to provide thread-local -storage for `sing_der!` side effects (`q`, `ud`, `spline_hint`). Multiple threads -may call this function concurrently using distinct `odet_proxy` objects. - -No callback is used: the propagator integration proceeds without normalization or -storage steps, since the identity ICs ensure bounded solutions within each chunk. -""" -function integrate_propagator_chunk!( - prop::ChunkPropagator, - chunk::IntegrationChunk, - ctrl::ForceFreeStatesControl, - equil::Equilibrium.PlasmaEquilibrium, - ffit::FourFitVars, - intr::ForceFreeStatesInternal, - odet_proxy::OdeState -) - N = intr.numpert_total - # Reverse tspan for backward chunks (direction=-1): OrdinaryDiffEq handles negative tspan - # naturally. The resulting propagator maps state at psi_end → psi_start, which is - # well-conditioned because exponentially growing solutions (forward) decay backward. - tspan = chunk.direction == 1 ? - (chunk.psi_start, chunk.psi_end) : - (chunk.psi_end, chunk.psi_start) - rtol = ctrl.eulerlagrange_tolerance - params = (ctrl, equil, ffit, intr, odet_proxy, chunk) - - # Upper block IC: U₁ = I, U₂ = 0 - u_upper = zeros(ComplexF64, N, N, 2) - for i in 1:N - u_upper[i, i, 1] = 1 - end - odet_proxy.spline_hint[] = 1 - odet_proxy.ffit_hint[] = 1 - prob = ODEProblem(sing_der!, u_upper, tspan, params) - sol = solve(prob, Vern9(); reltol=rtol, save_everystep=false, save_end=true) - prop.block_upper_ic .= sol.u[end] - odet_proxy.total_steps += sol.stats.naccept # thread-local; summed into odet after the BVP barrier - - # Lower block IC: U₁ = 0, U₂ = I - u_lower = zeros(ComplexF64, N, N, 2) - for i in 1:N - u_lower[i, i, 2] = 1 - end - odet_proxy.spline_hint[] = 1 - odet_proxy.ffit_hint[] = 1 - prob = ODEProblem(sing_der!, u_lower, tspan, params) - sol = solve(prob, Vern9(); reltol=rtol, save_everystep=false, save_end=true) - prop.block_lower_ic .= sol.u[end] - odet_proxy.total_steps += sol.stats.naccept -end - -""" - integrate_fm_with_ua_ic(chunks, chunk_range, ua, ctrl, equil, ffit, intr; - backward=false) -> Matrix{ComplexF64} - -Re-integrate a span of chunks using ua (asymptotic solution) as initial conditions, matching -Fortran STRIDE's uFM_sing_init behavior. Returns a 2N×2N fundamental matrix -where column j is the ODE solution at the span endpoint with IC = column j of T = [ua[:,:,1]; ua[:,:,2]]. - -When `backward=false` (default): ua is the IC at psi_start, integrate forward to psi_end. -When `backward=true`: ua is the IC at psi_end, integrate backward to psi_start. The result -maps asymptotic coefficients at psi_end → state at psi_start. - -This provides numerically accurate propagators near singular surfaces because the ODE integrator -maintains per-column relative accuracy even when columns span a 10^8+ dynamic range (big/small -solutions). In contrast, post-multiplying a pre-computed identity-IC propagator by T loses the -small-solution information to roundoff. -""" -function integrate_fm_with_ua_ic( - chunks::Vector{IntegrationChunk}, - chunk_range::UnitRange{Int}, - ua::Array{ComplexF64,3}, - ctrl::ForceFreeStatesControl, - equil::Equilibrium.PlasmaEquilibrium, - ffit::FourFitVars, - intr::ForceFreeStatesInternal; - backward::Bool = false, - psi_ua::Float64 = NaN -) - N = intr.numpert_total - psi_start = chunks[first(chunk_range)].psi_start - psi_end = chunks[last(chunk_range)].psi_end - # Use stored ua ψ location if provided; otherwise fall back to chunk boundary. - # The ua is evaluated at the inner-layer boundary (exact ψ from singular crossing), - # which may differ slightly from the nearest chunk boundary. - if backward && !isnan(psi_ua) - psi_end = psi_ua # ua lives at psi_ua, not at chunk boundary - elseif !backward && !isnan(psi_ua) - psi_start = psi_ua # ua lives at psi_ua, not at chunk boundary - end - # For backward integration: start at psi_end (where ua lives), integrate to psi_start - tspan = backward ? (psi_end, psi_start) : (psi_start, psi_end) - rtol = ctrl.eulerlagrange_tolerance - - result = zeros(ComplexF64, 2N, 2N) - odet_proxy = OdeState(N, 1, 1, 0) - dummy_chunk = IntegrationChunk(psi_start, psi_end, false, 0, backward ? -1 : 1) - params = (ctrl, equil, ffit, intr, odet_proxy, dummy_chunk) - - # Batch 1: columns 1:N of T (big solutions) - u0 = zeros(ComplexF64, N, N, 2) - u0[:, :, 1] .= ua[:, 1:N, 1] - u0[:, :, 2] .= ua[:, 1:N, 2] - odet_proxy.spline_hint[] = 1 - odet_proxy.ffit_hint[] = 1 - prob = ODEProblem(sing_der!, u0, tspan, params) - sol = solve(prob, Vern9(); reltol=rtol, save_everystep=false, save_end=true) - result[1:N, 1:N] .= sol.u[end][:, :, 1] - result[N+1:2N, 1:N] .= sol.u[end][:, :, 2] - - # Batch 2: columns N+1:2N of T (small solutions) - u0[:, :, 1] .= ua[:, N+1:2N, 1] - u0[:, :, 2] .= ua[:, N+1:2N, 2] - odet_proxy.spline_hint[] = 1 - odet_proxy.ffit_hint[] = 1 - prob = ODEProblem(sing_der!, u0, tspan, params) - sol = solve(prob, Vern9(); reltol=rtol, save_everystep=false, save_end=true) - result[1:N, N+1:2N] .= sol.u[end][:, :, 1] - result[N+1:2N, N+1:2N] .= sol.u[end][:, :, 2] - - return result -end - -""" - apply_propagator!(odet, prop) - -Apply the chunk propagator `prop` to the current state `odet.u` in-place. - -The propagator acts as a linear map on the (U₁, U₂) pair: - - U₁_new = block_upper_ic[:,:,1] · U₁_prev + block_lower_ic[:,:,1] · U₂_prev - U₂_new = block_upper_ic[:,:,2] · U₁_prev + block_lower_ic[:,:,2] · U₂_prev - -This correctly propagates any state (not just the identity), including the -(S, I) form produced by Riccati-style crossings. - -Implements the subpropagator composition Φ(ψ₂, ψ₀) = Φ(ψ₂, ψ₁) · Φ(ψ₁, ψ₀) of -Glasser-Kolemen (2018) Phys. Plasmas 25, 032501 Eq. 29. -""" -function apply_propagator!(odet::OdeState, prop::ChunkPropagator) - U1_upper = @view prop.block_upper_ic[:, :, 1] - U2_upper = @view prop.block_upper_ic[:, :, 2] - U1_lower = @view prop.block_lower_ic[:, :, 1] - U2_lower = @view prop.block_lower_ic[:, :, 2] - - u1_prev = copy(@view odet.u[:, :, 1]) - u2_prev = copy(@view odet.u[:, :, 2]) - tmp = similar(u1_prev) - - # U₁_new = U1_upper · u1_prev + U1_lower · u2_prev - mul!(view(odet.u, :, :, 1), U1_upper, u1_prev) - mul!(tmp, U1_lower, u2_prev) - odet.u[:, :, 1] .+= tmp - - # U₂_new = U2_upper · u1_prev + U2_lower · u2_prev - mul!(view(odet.u, :, :, 2), U2_upper, u1_prev) - mul!(tmp, U2_lower, u2_prev) - odet.u[:, :, 2] .+= tmp -end - -""" - apply_propagator_inverse!(odet, prop) - -Apply the *inverse* of the chunk propagator `prop` to the current state `odet.u` in-place. - -Used for backward chunks (direction=-1): the stored propagator Φ_bwd maps state at -`psi_end` → state at `psi_start` (well-conditioned because solutions that grow -exponentially forward decay backward). To advance the Riccati state from `psi_start` -to `psi_end`, we solve Φ_bwd · x = u_old, which gives x = Φ_bwd⁻¹ · u_old = Φ_fwd · u_old. - -Since Φ_bwd is well-conditioned, the LU solve is accurate, giving the same result as -applying the (ill-conditioned) forward propagator Φ_fwd but with far better precision. - -Implements the inverse subpropagator identity Φ(ψ₂, ψ₁) = Φ(ψ₁, ψ₂)⁻¹ of -Glasser-Kolemen (2018) Phys. Plasmas 25, 032501 Eq. 33. -""" -function apply_propagator_inverse!(odet::OdeState, prop::ChunkPropagator) - N = size(odet.u, 1) - # Assemble 2N×2N backward FM Φ_bwd - #! format: off - Φ = [prop.block_upper_ic[:,:,1] prop.block_lower_ic[:,:,1]; - prop.block_upper_ic[:,:,2] prop.block_lower_ic[:,:,2]] - #! format: on - # Φ_bwd maps state at psi_end → psi_start (well-conditioned). - # We want Φ_fwd = Φ_bwd⁻¹ to advance state from psi_start → psi_end. - # Solving Φ_bwd · x = [U₁_old; U₂_old] gives x = Φ_bwd⁻¹ · [U₁_old; U₂_old]. - u_old = [odet.u[:,:,1]; odet.u[:,:,2]] # 2N × N - u_new = Φ \ u_old # LU solve, 2N × N - odet.u[:,:,1] .= u_new[1:N, :] - odet.u[:,:,2] .= u_new[N+1:2N, :] -end - -""" - riccati_eulerlagrange_integration(ctrl, equil, ffit, intr) -> (odet, propagators, chunks, S_left) - -The Riccati/STRIDE integrator: a chunked fundamental matrix (propagator) driver for the EL -integration. The trailing three return values feed the Δ' BVP in `compute_delta_prime_matrix!`; -this is the only branch that produces them. - -Solves the same system as [`forward_eulerlagrange_integration`](@ref), but integrates all bulk -chunks concurrently using `Threads.@threads`, then re-integrates the outer plasma serially: - -1. **Chunk generation**: calls `chunk_el_integration_bounds`, then `balance_integration_chunks` - to sub-divide chunks for load balancing. The chunk count depends only on `intr.msing` and - `ctrl.nchunks`, never on the thread count, so results are thread-independent. -2. **Propagator phase**: `integrate_propagator_chunk!` integrates each chunk independently - from identity initial conditions (no accumulated state, no normalization/callback). - Each thread uses a private `OdeState` proxy for `sing_der!` side effects. -3. **Serial assembly**: propagators are applied sequentially with `apply_propagator!`. - Rational surface crossings use `riccati_cross_ideal_singular_surf!` (no Gaussian - reduction). -4. **Outer plasma re-integration**: after the last rational surface crossing, the outer - plasma (from last ψ_s to psilim) is re-integrated using `riccati_integrate_chunk!`. - FM propagation in this region is prone to precision loss for high N (exponential growth - without renormalization); Riccati integration keeps matrices bounded and provides dense - checkpoints for `findmax_dW_edge!`. - -Select via `integrator = "riccati"` in `[ForceFreeStates]` of gpec.toml. Requires -`singfac_min != 0`. Uses whatever threads `julia -t` provides; `ctrl.nchunks` is the only -tunable. - -**Key differences from the forward integrator:** -- No Gaussian reduction in the propagator BVP phase (crossings use the - Riccati-style algorithm, `odet.ifix` stays 0) -- `transform_u!` is called on the odet but is a no-op (ifix=0) -- Outer plasma uses serial Riccati integration for numerical stability -- `odet.u_store` holds chunk-endpoint Riccati states, not dense Euler-Lagrange ξ, and - `odet.u_store_el_basis` stays `false`: this integrator never claims the EL basis, so - PerturbedEquilibrium and the HDF5 forward-integration ξ datasets require the forward path. - -**Bidirectional integration for large-N accuracy:** -The crossing chunk (nearest to each rational surface singL[j]) is integrated *backward* -(`direction=-1`, `tspan` reversed). Backward integration of a region where solutions grow -exponentially forward causes them to *decay*, so the resulting backward FM Φ_bwd is -well-conditioned. The accurate forward propagation is recovered as Φ_bwd⁻¹ via a stable -LU solve in `apply_propagator_inverse!`. This follows the same principle as STRIDE -(Glasser 2018 Phys. Plasmas 25, 032501). The all-forward path had ~10% energy error for -the DIIID-like example (N=26, n=1); bidirectional reduces this to within 2%. -""" -function riccati_eulerlagrange_integration( - ctrl::ForceFreeStatesControl, equil::Equilibrium.PlasmaEquilibrium, - ffit::FourFitVars, intr::ForceFreeStatesInternal -) - odet = _initialize_parallel_odet(ctrl, equil, ffit, intr) - chunks, propagators, odet_proxies = _setup_parallel_chunks_and_proxies(odet, ctrl, intr) - _log_parallel_start(ctrl, odet, equil, chunks) - - _run_parallel_bvp_phase!(propagators, chunks, ctrl, equil, ffit, intr, odet_proxies) - - # Harvest solver-step counts accumulated thread-locally in each proxy during the BVP phase. - # The outer re-integration below uses riccati_integrate_chunk!, which counts via its callback. - odet.total_steps += sum(p.total_steps for p in odet_proxies) - - S_at_surface_left, last_crossing_step = - _assemble_propagators_serially!(odet, propagators, chunks, ctrl, equil, ffit, intr) - - _reintegrate_outer_plasma!(odet, last_crossing_step, ctrl, equil, ffit, intr) - - chunks, propagators = _handle_edge_dW_scan!(odet, chunks, propagators, ctrl, equil, ffit, intr) - - # compute_delta_prime_matrix! is called from the main pipeline (after free_run) so - # that vacuum response wv is available for the edge BC. With self-consistent truncation, - # the propagators/chunks returned here match intr.psilim exactly, so Δ' is well-defined - # for both truncate_at_dW_peak=false (full domain) and =true (peak). - if ctrl.verbose - @info "Evaluating fixed-boundary stability criterion" - end - odet.nzero = evaluate_stability_criterion!(odet, equil.profiles) - transform_u!(odet, intr) # no-op when ifix=0 (no Gaussian reduction) - - return odet, propagators, chunks, S_at_surface_left -end - -# Build odet and initialize at the magnetic axis. Same path as serial eulerlagrange_integration. -function _initialize_parallel_odet(ctrl::ForceFreeStatesControl, - equil::Equilibrium.PlasmaEquilibrium, - ffit::FourFitVars, - intr::ForceFreeStatesInternal) - odet = OdeState(intr.numpert_total, ctrl.numsteps_init, ctrl.numunorms_init, intr.msing) - if ctrl.sing_start <= 0 - initialize_el_at_axis!(odet, ctrl, ffit, equil.profiles, intr) - elseif ctrl.sing_start <= intr.msing - error("sing_start > 0 not implemented yet!") - else - error("Invalid value for sing_start: $(ctrl.sing_start) > msing = $(intr.msing)") - end - # Prime odet.new = false (consistent with riccati path — no Gaussian reduction used). - odet.new = false - fill!(odet.unorm0, 1.0) - return odet -end - -# Build the (bidirectional) chunk list, allocate per-chunk propagators, and allocate -# per-thread proxy OdeStates sized by maxthreadid() (Julia 1.9+ may report threadid -# values above nthreads() due to the interactive thread pool). -function _setup_parallel_chunks_and_proxies(odet::OdeState, ctrl::ForceFreeStatesControl, - intr::ForceFreeStatesInternal) - # Bidirectional chunks: crossing chunks are assigned direction=-1 so they are - # integrated backward. The resulting Φ_bwd is well-conditioned because growing EL - # solutions decay backward; forward propagation is recovered via LU solve in - # apply_propagator_inverse! during serial assembly. - base_chunks = chunk_el_integration_bounds(odet, ctrl, intr; bidirectional=true) - chunks = balance_integration_chunks(base_chunks, ctrl, intr) - N = intr.numpert_total - propagators = [ChunkPropagator(N) for _ in chunks] - odet_proxies = [OdeState(N, 1, 1, 0) for _ in 1:Threads.maxthreadid()] - return chunks, propagators, odet_proxies -end - -function _log_parallel_start(ctrl::ForceFreeStatesControl, odet::OdeState, - equil::Equilibrium.PlasmaEquilibrium, - chunks::Vector{IntegrationChunk}) - ctrl.verbose || return - @info " ψ = $((@sprintf "%.3f" odet.psifac)), q = $((@sprintf "%.3f" equil.profiles.q_spline(odet.psifac)))" - @info " Riccati FM: $(length(chunks)) chunks over $(Threads.nthreads()) thread$(Threads.nthreads() == 1 ? "" : "s")" -end - -# Integrate each chunk's FM propagator from identity IC across whatever threads `julia -t` -# provides. The :static scheduler makes Threads.threadid() a stable index into odet_proxies. -# Each chunk is independent (identity IC, no accumulated state), so the result does not -# depend on how chunks are distributed across threads. -function _run_parallel_bvp_phase!(propagators::Vector{ChunkPropagator}, - chunks::Vector{IntegrationChunk}, - ctrl::ForceFreeStatesControl, - equil::Equilibrium.PlasmaEquilibrium, ffit::FourFitVars, - intr::ForceFreeStatesInternal, - odet_proxies::Vector{OdeState}) - Threads.@threads :static for i in eachindex(chunks) - integrate_propagator_chunk!(propagators[i], chunks[i], ctrl, equil, ffit, intr, - odet_proxies[Threads.threadid()]) - end -end - -# Apply per-chunk propagators serially to odet, renormalizing to (S, I) after each. -# This is the Julia equivalent of STRIDE's ode_fixup: products of K chunk FMs can have -# cond ~ (cond_per_chunk)^K causing catastrophic cancellation for large N (≥20); periodic -# renorm keeps each step at O(cond_per_chunk). Backward (direction=-1) crossing chunks are -# applied via apply_propagator_inverse! (Φ_bwd⁻¹ from LU solve). S_at_surface_left records -# the well-conditioned Riccati S at each surface's left boundary for use as the Δ' BVP -# axis BC. Returns (S_at_surface_left, last_crossing_step). -function _assemble_propagators_serially!(odet::OdeState, propagators::Vector{ChunkPropagator}, - chunks::Vector{IntegrationChunk}, - ctrl::ForceFreeStatesControl, - equil::Equilibrium.PlasmaEquilibrium, - ffit::FourFitVars, intr::ForceFreeStatesInternal) - N = intr.numpert_total - S_at_surface_left = Matrix{ComplexF64}[] - last_crossing_step = 1 - for (i, chunk) in enumerate(chunks) - if chunk.direction == -1 - apply_propagator_inverse!(odet, propagators[i]) - else - apply_propagator!(odet, propagators[i]) - end - renormalize_riccati_inplace!(odet.u, N) - odet.psifac = chunk.psi_end - odet.q = equil.profiles.q_spline(odet.psifac) - - if ctrl.verbose - @info " ψ = $((@sprintf "%.3f" odet.psifac)), q= $((@sprintf "%.3f" odet.q)), max(S) = $((@sprintf "%.2e" maximum(abs, odet.u[:,:,1]))), steps = $(odet.step-1)" - end - - if chunk.needs_crossing - ctrl.kinetic_factor > 0 && error("kinetic_factor > 0 not implemented yet in Riccati!") - # State is (S, I) from the renorm above — well-conditioned at the surface's left boundary. - push!(S_at_surface_left, copy(odet.u[:, :, 1])) - riccati_cross_ideal_singular_surf!(odet, ctrl, equil, ffit, intr, chunk.ising) - last_crossing_step = odet.step - 1 - else - # Save non-crossing end-of-chunk state. These columns are FM/Riccati chunk - # endpoints, not the Euler-Lagrange state, so the odet never claims the EL basis. - odet.u_store_el_basis = false - if odet.step >= size(odet.u_store, 4) - resize_storage!(odet) - end - odet.psi_store[odet.step] = odet.psifac - odet.q_store[odet.step] = odet.q - @views odet.u_store[:, :, :, odet.step] .= odet.u - odet.step += 1 - end - end - return S_at_surface_left, last_crossing_step -end - -# Re-integrate the outer plasma (last rational surface → psilim) with Riccati for numerical -# stability and dense checkpoint storage. FM propagation here is prone to precision loss at -# high N because the solution grows exponentially without renormalization; Riccati keeps -# matrices bounded. Dense checkpoints are also needed by findmax_dW_edge!. The u_store -# entry at last_crossing_step holds (U₁_new, U₂_new) from riccati_cross_ideal_singular_surf! -# before renormalization; we renorm here to (S_new, I) as the Riccati starting state. -function _reintegrate_outer_plasma!(odet::OdeState, last_crossing_step::Int, - ctrl::ForceFreeStatesControl, - equil::Equilibrium.PlasmaEquilibrium, ffit::FourFitVars, - intr::ForceFreeStatesInternal) - N = intr.numpert_total - odet.u .= odet.u_store[:, :, :, last_crossing_step] - odet.psifac = odet.psi_store[last_crossing_step] - odet.q = odet.q_store[last_crossing_step] - odet.step = last_crossing_step + 1 - renormalize_riccati_inplace!(odet.u, N) - outer_chunk = IntegrationChunk(; psi_start=odet.psifac, psi_end=intr.psilim * (1 - eps), - needs_crossing=false, ising=0) - riccati_integrate_chunk!(odet, ctrl, equil, ffit, intr, outer_chunk) - # Post: odet.u is in (S, I) form; odet.step points to next empty slot. -end - -# Edge-dW scan over [psiedge, psilim] — populates odet.edge_scan for HDF5. By default -# (truncate_at_dW_peak=false) it's diagnostic-only: integration domain is unchanged. -# When truncate_at_dW_peak=true, the dW peak becomes the new physical edge: intr.psilim, -# odet, propagators, and chunks are made self-consistent (straddling chunk rebuilt with -# shorter psi_end; chunks past the new boundary dropped). Without that rebuild, the Δ' BVP -# would apply the edge BC at the truncated psilim to a propagator still extending to the -# original psilim — silently shifting the outermost rational's Δ' by tens of percent. -# Returns the (possibly truncated) chunks and propagators arrays. -function _handle_edge_dW_scan!(odet::OdeState, chunks::Vector{IntegrationChunk}, - propagators::Vector{ChunkPropagator}, - ctrl::ForceFreeStatesControl, - equil::Equilibrium.PlasmaEquilibrium, ffit::FourFitVars, - intr::ForceFreeStatesInternal) - N = intr.numpert_total - odet.step -= 1 - trim_storage!(odet) - ctrl.psiedge < intr.psilim || return chunks, propagators - - saved_psifac, saved_u = odet.psifac, copy(odet.u) - peak_step = findmax_dW_edge!(odet, ctrl, equil, ffit, intr) - - if !ctrl.truncate_at_dW_peak - odet.psifac = saved_psifac - odet.u .= saved_u - if ctrl.verbose - @info "Edge-dW peak (diagnostic): ψ = $((@sprintf "%.2f" odet.psi_store[peak_step])), q = $((@sprintf "%.2f" odet.q_store[peak_step])); integration domain unchanged" - end - return chunks, propagators - end - - # Truncate to dW peak: relocate intr.psilim and rebuild Δ' BVP self-consistently. - n_chunks_before = length(chunks) - odet.step = peak_step - trim_storage!(odet) - intr.psilim = odet.psi_store[end] - intr.qlim = odet.q_store[end] - odet.u .= odet.u_store[:, :, :, end] - renormalize_riccati_inplace!(odet.u, N) # stored snapshot may be pre-renorm - - peak_psi = odet.psi_store[end] - last_chunk_idx = findlast(c -> c.psi_start < peak_psi, chunks) - if last_chunk_idx === nothing - error("truncate_at_dW_peak: peak ψ=$peak_psi lies before all chunk starts") - end - straddling = chunks[last_chunk_idx] - if straddling.psi_end > peak_psi - new_chunk = IntegrationChunk( - psi_start = straddling.psi_start, - psi_end = peak_psi, - needs_crossing = straddling.needs_crossing, - ising = straddling.ising, - direction = straddling.direction, - ) - chunks[last_chunk_idx] = new_chunk - odet_proxy = OdeState(N, 1, 1, 0) - integrate_propagator_chunk!(propagators[last_chunk_idx], new_chunk, - ctrl, equil, ffit, intr, odet_proxy) - end - n_dropped = 0 - if last_chunk_idx < length(chunks) - n_dropped = length(chunks) - last_chunk_idx - chunks = chunks[1:last_chunk_idx] - propagators = propagators[1:last_chunk_idx] - end - if ctrl.verbose - @info "Truncating integration at peak edge dW (self-consistent): ψ = $((@sprintf "%.4f" peak_psi)), q = $((@sprintf "%.3f" odet.q_store[end])). Rebuilt chunk $last_chunk_idx; dropped $n_dropped of $n_chunks_before outer chunks." - end - return chunks, propagators -end diff --git a/src/ForceFreeStates/Riccati/Crossings.jl b/src/ForceFreeStates/Riccati/Crossings.jl new file mode 100644 index 000000000..017792b1c --- /dev/null +++ b/src/ForceFreeStates/Riccati/Crossings.jl @@ -0,0 +1,157 @@ +# Singular-surface crossing algorithms for the Riccati/fundamental-matrix integration. + +""" + riccati_cross_ideal_singular_surf!(odet, ctrl, equil, ffit, intr, ising) + +Cross a singular surface for the Riccati formulation. Replaces `cross_ideal_singular_surf!` +for the Riccati integration path with two key differences: + +1. **No Gaussian reduction**: `cross_ideal_singular_surf!` calls `compute_solution_norms!` + which applies Gaussian reduction to (S, I). This divides by pivot elements of S, which + can be near-zero (S = 0 at axis and grows slowly), producing NaN/Inf in U₂. For Riccati, + S is bounded so Gaussian reduction is unnecessary. + +2. **Direct column zeroing**: Instead of using the GR-sorted `odet.index` to identify the + column to zero, we use `ipert_res` directly (the resonant mode index). This is valid since + without GR there is no permutation applied to the columns of S. + +**Δ' normalization**: This function expects `odet.u` in the bounded (U₁, U₂) form produced by +`riccati_integrate_chunk!` with `needs_crossing=true` (final renorm skipped). ca_l is computed +from (U₁, U₂) before the crossing, and ca_r from (U₁_new, U₂_new) before `renormalize_riccati!`. +Since column `ipert_res` of [U₁_new; U₂_new] equals the introduced asymptotic solution exactly, +ca_r[ipert_res,ipert_res,2] = 1 regardless of other column normalizations. This gives a +physically meaningful Δ' = ca_r - ca_l with consistent left/right normalization. + +After the predictor step and asymptotic introduction, `renormalize_riccati!` is called +to restore the canonical (S_new, I) form before continuing integration. + +The u_store entry at the crossing step correctly stores (U₁_new, U₂_new) so that +`evaluate_stability_criterion!` can compute U₁_new / U₂_new = S_new correctly. +""" +function riccati_cross_ideal_singular_surf!( + odet::OdeState, ctrl::ForceFreeStatesControl, equil::Equilibrium.PlasmaEquilibrium, + ffit::FourFitVars, intr::ForceFreeStatesInternal, ising::Int +) + # Skip Gaussian reduction — S is bounded so no large-norm columns exist. + singp = intr.sing[ising] + dpsi = singp.psifac - odet.psifac # ψ_res - ψ_current (positive) + ipert_res = 1 .+ singp.m .- intr.mlow .+ (singp.n .- intr.nlow) .* intr.mpert + + sing_asymp_left, sing_asymp_right = _two_sided_singular_asymptotics(singp, ctrl, equil, ffit, intr) + _log_riccati_crossing_diagnostics(odet, intr, ising, singp, dpsi, sing_asymp_left, sing_asymp_right) + + _capture_left_crossing_data!(odet, singp, sing_asymp_left, dpsi, intr, ising) + _predict_across_singular_surface!(odet, ctrl, equil, ffit, intr, ising, ipert_res, dpsi, sing_asymp_right) + _capture_right_crossing_data!(odet, singp, sing_asymp_right, dpsi, intr, ising, ipert_res, ctrl) + + _stash_per_surface_delta_prime_stub!(odet, intr, ising, ipert_res, sing_asymp_right, equil, ctrl) + _store_crossing_step!(odet) + + # Restore canonical (S_new, I) form before continuing integration. + renormalize_riccati!(odet, intr) +end + +""" + _two_sided_singular_asymptotics(singp, ctrl, equil, ffit, intr) -> (left, right) + +Compute left- (`sig=-1`) and right- (`sig=+1`) side singular asymptotics matching +Fortran STRIDE's separate vmatl/vmatr (sing_vmat). Alpha is taken from the right +side and shared with the left. +""" +function _two_sided_singular_asymptotics(singp::SingType, ctrl::ForceFreeStatesControl, + equil::Equilibrium.PlasmaEquilibrium, ffit::FourFitVars, + intr::ForceFreeStatesInternal) + sing_asymp_right = compute_sing_asymptotics(singp, ctrl, equil, ffit, intr; sig=1.0) + sing_asymp_left = compute_sing_asymptotics(singp, ctrl, equil, ffit, intr; sig=-1.0, + alpha_override=sing_asymp_right.alpha) + return sing_asymp_left, sing_asymp_right +end + +# @debug-only per-crossing diagnostics. Enable via JULIA_DEBUG=GeneralizedPerturbedEquilibrium. +function _log_riccati_crossing_diagnostics(odet, intr, ising, singp, dpsi, sing_asymp_left, sing_asymp_right) + @debug begin + ipert_res_diag = 1 .+ singp.m .- intr.mlow .+ (singp.n .- intr.nlow) .* intr.mpert + msg = " ising=$ising: psi_sing=$(@sprintf("%.10f", singp.psifac)), psi_eval=$(@sprintf("%.10f", odet.psifac)), dpsi=$(@sprintf("%.10e", dpsi))\n" + msg *= " alpha_L = $(sing_asymp_left.alpha), alpha_R = $(sing_asymp_right.alpha)\n" + for ip in ipert_res_diag + msg *= " vmatL[0] big: vmat[$ip,$ip,1,1]=$(@sprintf("%.8e", real(sing_asymp_left.vmat[ip,ip,1,1]))), vmat[$ip,$ip,2,1]=$(@sprintf("%.8e", real(sing_asymp_left.vmat[ip,ip,2,1])))\n" + msg *= " vmatR[0] big: vmat[$ip,$ip,1,1]=$(@sprintf("%.8e", real(sing_asymp_right.vmat[ip,ip,1,1]))), vmat[$ip,$ip,2,1]=$(@sprintf("%.8e", real(sing_asymp_right.vmat[ip,ip,2,1])))\n" + end + msg + end +end + +# Capture left-side asymptotic data into odet.ca_l and singp.ua_left/psi_ua_left. +function _capture_left_crossing_data!(odet::OdeState, singp::SingType, sing_asymp_left, + dpsi::Float64, intr::ForceFreeStatesInternal, ising::Int) + ua = sing_get_ua(sing_asymp_left, dpsi) + singp.ua_left = copy(ua) + singp.psi_ua_left = odet.psifac + odet.ca_l[:, :, :, ising] .= sing_get_ca(odet.u, ua, intr) +end + +# Trapezoidal predictor across the singular surface: zero the resonant columns, +# evaluate sing_der! on both sides, advance odet by (du1 + du2)·dpsi, and jump +# odet.psifac to the right side. The zeroed columns stay zero through the predictor +# since du[:, ipert_res, :] = 0 when u[:, ipert_res, :] = 0. +function _predict_across_singular_surface!(odet::OdeState, ctrl::ForceFreeStatesControl, + equil::Equilibrium.PlasmaEquilibrium, ffit::FourFitVars, + intr::ForceFreeStatesInternal, ising::Int, + ipert_res, dpsi::Float64, sing_asymp_right) + if ctrl.kinetic_factor == 0 + for i in eachindex(sing_asymp_right.r1) + odet.u[:, ipert_res[i], :] .= 0 + end + end + params = (ctrl, equil, ffit, intr, odet, IntegrationChunk(0.0, 0.0, false, ising, 1)) + du1 = zeros(ComplexF64, intr.numpert_total, intr.numpert_total, 2) + du2 = zeros(ComplexF64, intr.numpert_total, intr.numpert_total, 2) + sing_der!(du1, odet.u, params, odet.psifac) + odet.psifac += 2 * dpsi # jump to other side of singular surface + sing_der!(du2, odet.u, params, odet.psifac) + odet.u .+= (du1 .+ du2) .* dpsi +end + +# Inject the right-side small asymptotic into the resonant columns of (U₁_new, U₂_new), +# capture odet.ca_r, and save singp.ua_right / psi_ua_right. +# Column ipert_res of [U₁_new; U₂_new] = ua[:, ipert_res+N, :] (the introduced small asymptotic), +# so ca_r[ipert_res, ipert_res, 2] = 1 regardless of other columns' normalization. +function _capture_right_crossing_data!(odet::OdeState, singp::SingType, sing_asymp_right, + dpsi::Float64, intr::ForceFreeStatesInternal, ising::Int, + ipert_res, ctrl::ForceFreeStatesControl) + ua = sing_get_ua(sing_asymp_right, dpsi) + singp.ua_right = copy(ua) + singp.psi_ua_right = odet.psifac + if ctrl.kinetic_factor == 0 + for i in eachindex(sing_asymp_right.r1) + odet.u[ipert_res[i], :, :] .= 0 + odet.u[:, ipert_res[i], :] .= ua[:, ipert_res[i]+intr.numpert_total, :] + end + end + odet.ca_r[:, :, :, ising] .= sing_get_ca(odet.u, ua, intr) +end + +# STUB: per-surface ca-based Δ' (not physically valid; see SingType.delta_prime docstring). +# The canonical Δ' is intr.delta_prime_matrix from compute_delta_prime_matrix!. +function _stash_per_surface_delta_prime_stub!(odet::OdeState, intr::ForceFreeStatesInternal, + ising::Int, ipert_res, sing_asymp_right, + equil::Equilibrium.PlasmaEquilibrium, + ctrl::ForceFreeStatesControl) + ctrl.kinetic_factor == 0 || return + denom = (2π)^2 * equil.psio + n_res = length(sing_asymp_right.r1) + N = intr.numpert_total + resize!(intr.sing[ising].delta_prime, n_res) + intr.sing[ising].delta_prime_col = zeros(ComplexF64, N, n_res) + for i in eachindex(sing_asymp_right.r1) + Δca_col = (odet.ca_r[:, ipert_res[i], 2, ising] - odet.ca_l[:, ipert_res[i], 2, ising]) / denom + intr.sing[ising].delta_prime_col[:, i] .= Δca_col + intr.sing[ising].delta_prime[i] = Δca_col[ipert_res[i]] + end +end + +# Store (U₁_new, U₂_new) into u_store before renormalization so that +# evaluate_stability_criterion! can recover S_new = U₁_new / U₂_new via compute_smallest_eigenvalue. +function _store_crossing_step!(odet::OdeState) + store_ode_data!(odet, odet.psifac, odet.u) +end diff --git a/src/ForceFreeStates/Riccati/DeltaPrimeBVP.jl b/src/ForceFreeStates/Riccati/DeltaPrimeBVP.jl new file mode 100644 index 000000000..5cf75446d --- /dev/null +++ b/src/ForceFreeStates/Riccati/DeltaPrimeBVP.jl @@ -0,0 +1,731 @@ +# STRIDE global boundary-value problem: Delta-prime matrix assembly, solve, PEST-3 decomposition. + +""" + compute_delta_prime_matrix!(intr, propagators, chunks; wv, psio, debug, ctrl, equil, ffit) + +Compute the inter-surface tearing stability matrix (msing × msing) using the +STRIDE global BVP formulation [Glasser 2018 Phys. Plasmas 25, 032501, Sec. III.B]. + +The BVP encodes the full plasma response with unknowns at each surface boundary: +``` + x_axis (N): free IC parameters at the axis (U₁ = 0 regular solutions) + x_left[j] (2N): state at left inner-layer boundary of surface j + x_right[j] (2N): state at right inner-layer boundary of surface j + x_edge (N): free IC parameters at the edge + Total unknowns: nMat = (2 + 4·msing)·N +``` + +## Edge boundary condition + +When `wv` is provided (the vacuum response matrix, singfac-scaled), the edge BC +follows the Fortran STRIDE convention: +``` + U₁ = c, U₂ = -wv·ψ₀²·c +``` +which is the free-boundary condition `wp + wv = 0` at the edge. +When `wv` is `nothing`, a conducting wall BC (`U₁ = 0`) is used. + +## Gaussian reduction (conditioning) + +Forward-propagated segment propagators (axis→surface, surface→surface) can be +extremely ill-conditioned (cond ~ 10²⁴) due to exponential growth of the big +solution. Following STRIDE's `ode_fixup`, Gaussian reduction is applied to each +assembled propagator's U₂ columns before inserting into the BVP matrix. This +keeps the BVP matrix full-rank and well-conditioned. + +## Output: PEST3-convention Δ' (deltap) + +The raw BVP solution is a 2·msing × 2·msing matrix `dp` with left/right +sub-indices at each surface. The PEST3-convention Δ' matrix is the linear +combination [Chance, PPPL-2527]: +``` + deltap(i,j) = dp(2i,2j) - dp(2i,2j-1) - dp(2i-1,2j) + dp(2i-1,2j-1) +``` +stored in `intr.delta_prime_matrix` (msing × msing). + +## Limitations + +This routine currently assumes exactly one resonant mode per singular surface +(the standard single-`n` case). When **any** surface carries more than one +resonant mode — i.e., a multi-`n` run where a single q value satisfies two +distinct `(m, n)` tuples (e.g. q = 2 with `(m=2, n=1)` AND `(m=4, n=2)`) — +the routine emits a warning and skips the inter-surface BVP rather than +crashing. Generalizing the BVP to multi-resonance surfaces is tracked as a +follow-up: the matrix shape becomes `n_res_total × n_res_total` with +`n_res_total = sum(length(intr.sing[j].m))` and a `(surface, mode, side)` +↔ BVP-row map; see PR discussion. + +Note: `intr.delta_prime_matrix` is the **only physically valid Δ'** produced +by this code. The per-surface ca-based stub `intr.sing[*].delta_prime` / +`delta_prime_col` (populated by `riccati_cross_ideal_singular_surf!`) is a +diagnostic placeholder for future intra-surface coupling work and is not +expected to agree with `delta_prime_matrix`. +""" +function compute_delta_prime_matrix!( + intr::ForceFreeStatesInternal, + propagators::Vector{ChunkPropagator}, + chunks::Vector{IntegrationChunk}; + wv::Union{Nothing,Matrix{ComplexF64}} = nothing, + psio::Float64 = 0.0, + debug::Bool = false, + S_at_surface_left::Union{Nothing,Vector{Matrix{ComplexF64}}} = nothing, + ctrl::Union{Nothing,ForceFreeStatesControl} = nothing, + equil::Union{Nothing,Equilibrium.PlasmaEquilibrium} = nothing, + ffit::Union{Nothing,FourFitVars} = nothing +) + intr.msing == 0 && return + _has_unsupported_multi_resonance(intr) && return + + sing, i_crossings, msing = _select_active_surfaces(intr, chunks) + msing == 0 && return + N = intr.numpert_total + + use_S_axis = S_at_surface_left !== nothing && length(S_at_surface_left) == msing + + # The FM-axis-BC fallback (use_S_axis=false) wires Phi_L_mats[j] as forward propagators + # in the BVP matrix. Crossing chunks with direction=-1 (bidirectional parallel FM) hold + # *backward* propagators, so applying them as forward would produce a silently wrong + # Δ' BVP. Forbid that combination explicitly — the parallel path always supplies + # S_at_surface_left (so use_S_axis=true) and any new caller hitting the FM-axis path + # needs forward crossing chunks. + if !use_S_axis + for ic in i_crossings + chunks[ic].direction == 1 || + error("compute_delta_prime_matrix!: FM-axis fallback (use_S_axis=false) requires forward crossing chunks; " * + "chunk $ic has direction=$(chunks[ic].direction). Either provide S_at_surface_left or use bidirectional=false.") + end + end + + Phi_L_mats, Phi_R_mats, Phi_R_halves = _assemble_segment_propagators( + propagators, chunks, i_crossings, msing, N, use_S_axis) + + ipert_all = [1 + sing[j].m[1] - intr.mlow + (sing[j].n[1] - intr.nlow) * intr.mpert for j in 1:msing] + has_ua = all(j -> !isempty(sing[j].ua_left), 1:msing) + T_left_mats, T_right_mats, T_left_inv, T_right_inv = + _build_asymptotic_basis_matrices(sing, has_ua, N, msing) + + debug && _log_bvp_setup(chunks, sing, S_at_surface_left, use_S_axis, has_ua, + Phi_L_mats, Phi_R_mats, Phi_R_halves, ipert_all, wv, psio, N, msing) + + if use_S_axis + uShootR, uShootL, uAxis = _build_S_axis_shooting_propagators( + propagators, chunks, i_crossings, sing, msing, N, + T_left_mats, T_right_mats, has_ua, ctrl, equil, ffit, intr, debug) + debug && _log_S_axis_shooting_propagators(uShootR, uShootL, uAxis, + S_at_surface_left, T_left_mats, + ipert_all, has_ua, msing, N) + M, nMat, col_edge = _assemble_bvp_S_axis( + uShootR, uShootL, uAxis, ipert_all, msing, N, wv, psio) + else + M, nMat, col_edge = _assemble_bvp_FM_axis( + Phi_L_mats, Phi_R_mats, ipert_all, msing, N, + T_left_inv, T_right_inv, has_ua, wv, psio) + end + + if debug + @info "Δ' BVP: nMat=$nMat, rank(M)=$(rank(M)), cond(M)=$(@sprintf("%.2e", cond(M)))" + end + + # rpec coil-response block: needs the S-axis row layout that `_solve_bvp_edge_coil` assumes + # for its edge rows, and a vacuum edge in the assembled matrix (col_edge in junc_rows). + if use_S_axis && wv !== nothing + intr.delta_coil = _solve_bvp_edge_coil(M, col_edge, msing, N, ipert_all) + end + + deltap, dp_raw_persisted = _solve_bvp_and_combine_pest3( + M, msing, N, nMat, use_S_axis, ipert_all, col_edge, ctrl, debug) + + # Persist both the PEST3 tearing projection (msing × msing) and the raw 2msing × 2msing + # D' matrix (side-major ordering, byte-compatible with Fortran rdcon/gal.f::gal_write_delta). + # The raw matrix is consumed by `pest3_decompose` to recover (A', B', Γ', Δ') for the full + # det(D' − D(γ)) = 0 eigenvalue problem; see the `delta_prime_raw` docstring in CoreTypes.jl. + intr.delta_prime_matrix = deltap + intr.delta_prime_raw = dp_raw_persisted +end + +# Column index helpers for the BVP matrix. j is the 1-based singular-surface index, +# N is numpert_total. Layout: c_axis(N), c_left[1](2N), c_right[1](2N), ..., c_edge(N). +_col_left(j::Int, N::Int) = (N + 4N*(j-1) + 1):(N + 4N*(j-1) + 2N) +_col_right(j::Int, N::Int) = (N + 4N*(j-1) + 2N + 1):(N + 4N*j) + +# Multi-resonance surfaces (one q value satisfying multiple (m,n) tuples in a multi-n run) +# are not yet handled by the inter-surface BVP. Returns true if any surface has >1 modes; +# emits a warning as a side effect. The stub per-surface delta_prime is unaffected. +function _has_unsupported_multi_resonance(intr::ForceFreeStatesInternal) + msing = intr.msing + n_res_per_surface = [length(intr.sing[j].m) for j in 1:msing] + any(>(1), n_res_per_surface) || return false + offenders = [(j, intr.sing[j].m, intr.sing[j].n) for j in 1:msing if n_res_per_surface[j] > 1] + @warn "compute_delta_prime_matrix!: skipping inter-surface Δ' BVP because some surfaces carry more than one resonant mode " * + "(multi-n collision; generalization tracked as follow-up). " * + "Per-surface Δ' is unaffected. Multi-resonance surfaces: $offenders" + return true +end + +# Map BVP surface index (1:msing_active) → intr.sing index using chunk.ising. Surfaces +# may be excluded at either end (below qlow or beyond psilim); each crossing chunk +# records its original surface index. Returns (sing alias, i_crossings, msing_active). +function _select_active_surfaces(intr::ForceFreeStatesInternal, chunks::Vector{IntegrationChunk}) + msing = intr.msing + i_crossings = findall(c -> c.needs_crossing, chunks) + sing_indices = [chunks[ic].ising for ic in i_crossings] + msing_active = length(i_crossings) + if msing_active < msing + excluded = setdiff(1:msing, sing_indices) + excluded_ms = [intr.sing[j].m for j in excluded] + @debug "compute_delta_prime_matrix!: $msing singular surfaces, $msing_active crossed (excluded: m=$excluded_ms)" + end + sing = [intr.sing[si] for si in sing_indices] + return sing, i_crossings, msing_active +end + +# Assemble all segment propagators: per-surface single-chunk FMs (Phi_L), inter-surface +# and edge multi-chunk FMs (Phi_R), and midpoint-split halves (Phi_R_halves) used by the +# diagnostic comparisons. Phi_R[1] is only built when use_S_axis=false (FM-axis fallback). +# Midpoint splitting halves each inter-surface span's condition number — STRIDE's trick: +# cond(full) = 10¹⁵ → cond(half) ≈ 10⁷·⁵, an 8-digit accuracy gain. +function _assemble_segment_propagators(propagators::Vector{ChunkPropagator}, + chunks::Vector{IntegrationChunk}, + i_crossings::Vector{Int}, msing::Int, N::Int, + use_S_axis::Bool) + Phi_L_mats = [assemble_fm_matrix(propagators, i_crossings[j]:i_crossings[j]) for j in 1:msing] + Phi_R_mats = Vector{Matrix{ComplexF64}}(undef, msing + 1) + if !use_S_axis + Phi_R_mats[1] = assemble_fm_matrix(propagators, 1:i_crossings[1]-1; condition=true) + end + for j in 2:msing + Phi_R_mats[j] = assemble_fm_matrix(propagators, i_crossings[j-1]+1:i_crossings[j]-1) + end + Phi_R_mats[msing+1] = assemble_fm_matrix(propagators, i_crossings[msing]+1:length(chunks)) + + Phi_R_halves = Vector{Tuple{Matrix{ComplexF64},Matrix{ComplexF64}}}(undef, msing - 1) + for j in 1:msing-1 + chunk_start = i_crossings[j] + 1 + chunk_end = i_crossings[j+1] - 1 + n_chunks = chunk_end - chunk_start + 1 + if n_chunks >= 2 + i_mid = chunk_start + div(n_chunks, 2) - 1 + Phi_left_half = assemble_fm_matrix(propagators, chunk_start:i_mid) + Phi_right_half = assemble_fm_matrix(propagators, i_mid+1:chunk_end) + Phi_R_halves[j] = (Phi_left_half, Phi_right_half) + else + Phi_R_halves[j] = (Matrix{ComplexF64}(I, 2N, 2N), Phi_R_mats[j+1]) + end + end + return Phi_L_mats, Phi_R_mats, Phi_R_halves +end + +# Asymptotic-basis transformation T = [ua[:,:,1]; ua[:,:,2]] maps (small/big) coefficients +# to raw (ξ,η) state. Column ordering of ua: 1:N = big solutions (z^{-α}, diverging), +# N+1:2N = small solutions (z^{+α}, bounded). Fortran STRIDE bakes T into the shooting +# propagators (uFM_sing_init); we multiply T into the BVP propagator blocks at each surface. +function _build_asymptotic_basis_matrices(sing::Vector{SingType}, has_ua::Bool, N::Int, msing::Int) + T_left_mats = Vector{Matrix{ComplexF64}}(undef, msing) + T_right_mats = Vector{Matrix{ComplexF64}}(undef, msing) + T_left_inv = Vector{Matrix{ComplexF64}}(undef, msing) + T_right_inv = Vector{Matrix{ComplexF64}}(undef, msing) + if has_ua + for j in 1:msing + sp = sing[j] + T_left_mats[j] = [sp.ua_left[:,:,1]; sp.ua_left[:,:,2]] + T_right_mats[j] = [sp.ua_right[:,:,1]; sp.ua_right[:,:,2]] + T_left_inv[j] = inv(T_left_mats[j]) + T_right_inv[j] = inv(T_right_mats[j]) + end + end + return T_left_mats, T_right_mats, T_left_inv, T_right_inv +end + +# Build the S-axis shooting propagators uShootR (forward from surface j right → midpoint) +# and uShootL (backward from surface j left → midpoint), and the conditioned axis +# propagator uAxis. uShootL[1] is built specially using the QR-conditioned axis path +# (Fortran ode_fixup) so that surface 1 inherits the well-conditioned S axis BC instead +# of going through a catastrophically ill-conditioned full axis FM. +function _build_S_axis_shooting_propagators( + propagators::Vector{ChunkPropagator}, chunks::Vector{IntegrationChunk}, + i_crossings::Vector{Int}, sing::Vector{SingType}, msing::Int, N::Int, + T_left_mats::Vector{Matrix{ComplexF64}}, T_right_mats::Vector{Matrix{ComplexF64}}, + has_ua::Bool, ctrl, equil, ffit, intr::ForceFreeStatesInternal, debug::Bool) + + can_reintegrate = has_ua && ctrl !== nothing && equil !== nothing && ffit !== nothing + uShootR = Vector{Matrix{ComplexF64}}(undef, msing) + uShootL = Vector{Matrix{ComplexF64}}(undef, msing) # uShootL[1] handled separately below + + for j in 1:msing + shoot_range_R = _midpoint_shoot_range(chunks, i_crossings, j, msing; side=:right) + if debug && !isempty(shoot_range_R) + psi_surf_R = chunks[first(shoot_range_R)].psi_start + psi_mid_R = chunks[last(shoot_range_R)].psi_end + psi_ua_R = sing[j].psi_ua_right + @info " uShootR[$j]: shoot_range=$(shoot_range_R), psi_chunk=$(@sprintf("%.6f", psi_surf_R)), psi_ua=$(@sprintf("%.6f", psi_ua_R)), psi_mid=$(@sprintf("%.6f", psi_mid_R)), Δψ_fix=$(@sprintf("%.6e", psi_ua_R - psi_surf_R))" + end + if can_reintegrate && !isempty(shoot_range_R) + uShootR[j] = integrate_fm_with_ua_ic(chunks, shoot_range_R, sing[j].ua_right, + ctrl, equil, ffit, intr; backward=false, psi_ua=sing[j].psi_ua_right) + else + T_init = has_ua ? T_right_mats[j] : nothing + uShootR[j] = assemble_fm_matrix(propagators, shoot_range_R; T_init=T_init) + end + + # uShootL[j>=2]: backward from surface j left to midpoint. uShootL[1] handled below. + j == 1 && continue + shoot_range_L = _midpoint_shoot_range(chunks, i_crossings, j, msing; side=:left) + if debug + psi_mid = chunks[first(shoot_range_L)].psi_start + psi_surf = chunks[last(shoot_range_L)].psi_end + psi_ua_L = sing[j].psi_ua_left + @info " uShootL[$j]: shoot_range=$(shoot_range_L), psi_mid=$(@sprintf("%.6f", psi_mid)), psi_chunk=$(@sprintf("%.6f", psi_surf)), psi_ua=$(@sprintf("%.6f", psi_ua_L)), Δψ_fix=$(@sprintf("%.6e", psi_ua_L - psi_surf))" + end + if can_reintegrate && !isempty(shoot_range_L) + uShootL[j] = integrate_fm_with_ua_ic(chunks, shoot_range_L, sing[j].ua_left, + ctrl, equil, ffit, intr; backward=true, psi_ua=sing[j].psi_ua_left) + else + T_init = has_ua ? T_left_mats[j] : nothing + uShootL[j] = assemble_fm_matrix(propagators, shoot_range_L; T_init=T_init) + end + end + + uAxis, i_axis_mid = _build_conditioned_axis_propagator(propagators, i_crossings, N) + uShootL[1] = _build_uShootL_first(propagators, chunks, i_crossings, sing, + T_left_mats, has_ua, can_reintegrate, i_axis_mid, + ctrl, equil, ffit, intr, N) + if debug + shoot_range_L1 = (i_axis_mid + 1):(i_crossings[1] - 1) + @info " Axis propagator: $(i_axis_mid) chunks, cond=$(@sprintf("%.2e", cond(uAxis)))" + @info " uShootL[1]: range=$(shoot_range_L1), cond=$(@sprintf("%.2e", cond(uShootL[1])))" + end + return uShootR, uShootL, uAxis +end + +# Locate the chunk midpoint between two singular surfaces (or surface↔edge) in ψ space. +# Side `:right` returns the range from chunk(i_crossings[j]+1) to the ψ-midpoint chunk +# (or to the last chunk for j==msing). Side `:left` returns the range from the midpoint +# chunk+1 to chunk(i_crossings[j]-1). The ψ midpoint is used (not the chunk-index midpoint) +# because chunks near singularities are packed tighter in ψ — Fortran convention. +function _midpoint_shoot_range(chunks::Vector{IntegrationChunk}, i_crossings::Vector{Int}, + j::Int, msing::Int; side::Symbol) + if side === :right + j == msing && return (i_crossings[msing] + 1):length(chunks) + chunk_start = i_crossings[j] + 1 + chunk_end = i_crossings[j+1] - 1 + else # :left, j >= 2 + chunk_start = i_crossings[j-1] + 1 + chunk_end = i_crossings[j] - 1 + end + psi_mid_target = (chunks[chunk_start].psi_start + chunks[chunk_end].psi_end) / 2 + i_mid_inter = chunk_start + for ic in chunk_start:chunk_end-1 + if chunks[ic].psi_end >= psi_mid_target + i_mid_inter = ic + break + end + i_mid_inter = ic + end + return side === :right ? (chunk_start:i_mid_inter) : ((i_mid_inter + 1):chunk_end) +end + +# Build a well-conditioned axis propagator by forward-propagating [0; I] through the +# pre-first-crossing chunks with QR fixup after each chunk (Fortran ode_fixup). The axis +# midpoint is placed one chunk before the first surface so that uShootL[1] covers only the +# last chunk, keeping it well-conditioned. +function _build_conditioned_axis_propagator(propagators::Vector{ChunkPropagator}, + i_crossings::Vector{Int}, N::Int) + n_pre_cross = i_crossings[1] - 1 + i_axis_mid = max(1, n_pre_cross - 1) + uAxis = zeros(ComplexF64, 2N, N) + for i in 1:N + uAxis[N+i, i] = 1 + end + for ic in 1:i_axis_mid + prop = propagators[ic] + upper_old = uAxis[1:N, :] + lower_old = uAxis[N+1:2N, :] + uAxis[1:N, :] .= prop.block_upper_ic[:,:,1] * upper_old .+ prop.block_lower_ic[:,:,1] * lower_old + uAxis[N+1:2N, :] .= prop.block_upper_ic[:,:,2] * upper_old .+ prop.block_lower_ic[:,:,2] * lower_old + Q, _ = qr(uAxis) + uAxis .= Matrix(Q)[:, 1:N] + end + for j in 1:N + uAxis[:, j] ./= norm(@view uAxis[:, j]) + end + return uAxis, i_axis_mid +end + +# Build uShootL[1]: backward propagator from surface 1 left boundary to the axis midpoint. +# Falls back to T_left_mats[1] (or identity if no ua) when there's only 1 chunk before the +# first crossing. +function _build_uShootL_first(propagators::Vector{ChunkPropagator}, + chunks::Vector{IntegrationChunk}, i_crossings::Vector{Int}, + sing::Vector{SingType}, T_left_mats::Vector{Matrix{ComplexF64}}, + has_ua::Bool, can_reintegrate::Bool, i_axis_mid::Int, + ctrl, equil, ffit, intr::ForceFreeStatesInternal, N::Int) + shoot_range_L1 = (i_axis_mid + 1):(i_crossings[1] - 1) + if can_reintegrate && !isempty(shoot_range_L1) + return integrate_fm_with_ua_ic(chunks, shoot_range_L1, sing[1].ua_left, + ctrl, equil, ffit, intr; + backward=true, psi_ua=sing[1].psi_ua_left) + elseif !isempty(shoot_range_L1) + return assemble_fm_matrix(propagators, shoot_range_L1; + T_init=has_ua ? T_left_mats[1] : nothing) + else + return has_ua ? T_left_mats[1] : Matrix{ComplexF64}(I, 2N, 2N) + end +end + +# Assemble the BVP matrix M with S-based axis BC. The Riccati S matrix at surface 1's left +# boundary encodes the axis BC (U₁ = S·U₂) in a well-conditioned form (cond ~ 10⁶), avoiding +# the catastrophically ill-conditioned axis FM. Fortran-matched structure with +# nMat = (2 + 4·msing)·N. Returns (M, nMat, col_edge). +function _assemble_bvp_S_axis(uShootR::Vector{Matrix{ComplexF64}}, + uShootL::Vector{Matrix{ComplexF64}}, + uAxis::Matrix{ComplexF64}, ipert_all::Vector{Int}, + msing::Int, N::Int, + wv::Union{Nothing,Matrix{ComplexF64}}, psio::Float64) + # STRIDE global BVP block structure [Glasser-Kolemen 2018 PoP 25, 032501 Eq. 37]. + nMat = (2 + 4 * msing) * N + col_axis = 1:N + col_edge = (nMat - N + 1):nMat + M = zeros(ComplexF64, nMat, nMat) + + # Axis matching: uShootL[1] · c_left[1] = uAxis · c_axis (2N equations) + M[1:2N, _col_left(1, N)] .= uShootL[1] + M[1:2N, col_axis] .= -uAxis + row_offset = 2N + + for j in 1:msing + ipert_j = ipert_all[j] + # Crossing: non-resonant modes continuity (asymptotic basis = identity) + for i in 1:2N + if i != ipert_j && i != ipert_j + N + row_offset += 1 + M[row_offset, _col_left(j, N)[i]] = 1 + M[row_offset, _col_right(j, N)[i]] = -1 + end + end + + junc_rows = (row_offset + 1):(row_offset + 2N) + if j < msing + # Midpoint matching between consecutive surfaces + M[junc_rows, _col_right(j, N)] .= -uShootR[j] + M[junc_rows, _col_left(j+1, N)] .= uShootL[j+1] + else + # Edge junction + M[junc_rows, _col_right(msing, N)] .= uShootR[msing] + if wv !== nothing + M[junc_rows[1:N], col_edge] .= -I(N) + M[junc_rows[N+1:end], col_edge] .= wv .* psio^2 + else + M[junc_rows[N+1:end], col_edge] .= -I(N) + end + end + row_offset = last(junc_rows) + end + + # Driving rows: set big-solution coefficient = 1 at each surface (asymptotic basis) + for j in 1:msing + ipert_j = ipert_all[j] + row_offset += 1 + M[row_offset, _col_left(j, N)[ipert_j]] = 1 + row_offset += 1 + M[row_offset, _col_right(j, N)[ipert_j]] = 1 + end + @assert row_offset == nMat "Row count mismatch: expected $nMat, got $row_offset" + return M, nMat, col_edge +end + +# Coil-response block for the Eq. (37) edge [Glasser-Kolemen 2018 PoP 25, 032501]: impose the rpec edge +# boundary condition — identity edge plus a unit source per poloidal mode, matching RDCON's +# `gal_set_boundary` rpec branch — then read the small-solution (+N slot) coefficient at every surface. +# The BC is the same for every edge mode, so the matrix is factorized once and all N modes are solved +# together as columns of one right-hand side. Returns delta_coil (2·msing × N), rows = surface side. +function _solve_bvp_edge_coil(M::Matrix{ComplexF64}, col_edge, msing::Int, N::Int, ipert_all::Vector{Int}) + nMat = size(M, 1) + bot = (nMat-2msing-N+1):(nMat-2msing) # Eq. (38) bottom rows, carrying the W_V block + Mc = copy(M) + Mc[bot, :] .= 0 + Mc[bot, col_edge] .= I(N) # Dirichlet edge: the edge coefficients equal the source + B = zeros(ComplexF64, nMat, N) + B[bot, :] .= I(N) # unit drive, one column per edge poloidal mode + X = lu(Mc) \ B + delta_coil = zeros(ComplexF64, 2msing, N) + for j in 1:msing + row_left = _col_left(j, N)[ipert_all[j]+N] + row_right = _col_right(j, N)[ipert_all[j]+N] + @views delta_coil[2j-1, :] .= X[row_left, :] + @views delta_coil[2j, :] .= X[row_right, :] + end + return delta_coil +end + +# Fallback BVP assembly with FM-based axis BC (used when no Riccati S matrices are available). +# Uses the conditioned axis propagator Phi_R[1][:,N+1:2N] in place of S-axis matching. +function _assemble_bvp_FM_axis(Phi_L_mats::Vector{Matrix{ComplexF64}}, + Phi_R_mats::Vector{Matrix{ComplexF64}}, ipert_all::Vector{Int}, + msing::Int, N::Int, + T_left_inv::Vector{Matrix{ComplexF64}}, + T_right_inv::Vector{Matrix{ComplexF64}}, has_ua::Bool, + wv::Union{Nothing,Matrix{ComplexF64}}, psio::Float64) + nMat = (2 + 4 * msing) * N + col_axis = 1:N + col_edge = (N + 4N*msing + 1):nMat + M = zeros(ComplexF64, nMat, nMat) + + M[1:2N, (N+1):(N+2N)] .= Phi_L_mats[1] + M[1:2N, col_axis] .= -view(Phi_R_mats[1], :, N+1:2N) + + row_drive_base = 2N + (4N-2)*msing + for j in 1:msing + ipert_j = ipert_all[j] + cl = _col_left(j, N) + cr = _col_right(j, N) + row_cont = 2N + (4N-2)*(j-1) + for i in 1:2N + if i != ipert_j && i != ipert_j + N + row_cont += 1 + M[row_cont, cl[i]] = 1 + M[row_cont, cr[i]] = -1 + end + end + junc_rows = (row_cont + 1):(2N + (4N-2)*j) + if j < msing + M[junc_rows, cr] .= Phi_R_mats[j+1] + M[junc_rows, _col_left(j+1, N)] .= -Phi_L_mats[j+1] + else + M[junc_rows, cr] .= Phi_R_mats[msing+1] + if wv !== nothing + M[junc_rows[1:N], col_edge] .= -I(N) + M[junc_rows[N+1:end], col_edge] .= wv .* psio^2 + else + M[junc_rows[N+1:end], col_edge] .= -I(N) + end + end + if has_ua + M[row_drive_base + 2j-1, cl] .= T_left_inv[j][ipert_j, :] + M[row_drive_base + 2j, cr] .= T_right_inv[j][ipert_j, :] + else + M[row_drive_base + 2j-1, cl[ipert_j]] = 1 + M[row_drive_base + 2j, cr[ipert_j]] = 1 + end + end + return M, nMat, col_edge +end + +# Solve the BVP for each driving configuration and apply the PEST3 four-term combination. +# Promotes to Complex{Double64} if ctrl.extended_precision_bvp (default true) — the PEST3 +# combination subtracts dp_raw entries up to ~3×10⁴ larger than the result, and Float64 +# precision lets the imaginary part drift 2–5× on DIIID-class equilibria. +function _solve_bvp_and_combine_pest3(M::Matrix{ComplexF64}, msing::Int, N::Int, nMat::Int, + use_S_axis::Bool, ipert_all::Vector{Int}, col_edge, + ctrl, debug::Bool) + s2 = 2 * msing + Tc = (ctrl === nothing || ctrl.extended_precision_bvp) ? Complex{Double64} : ComplexF64 + M_solve = Tc.(M) + + M_lu = lu(M_solve; check=false) + use_lu = issuccess(M_lu) + M_pinv = use_lu ? nothing : pinv(M_solve) + if !use_lu + @warn "Δ' BVP: LU factorization singular (rank $(rank(M))/$nMat), using pseudo-inverse fallback" + end + + dp_raw = zeros(Tc, s2, s2) + b = zeros(Tc, nMat) + for jsing in 1:msing, side in 1:2 + dRow = 2jsing - (2 - side) + fill!(b, 0) + drive_row = use_S_axis ? (nMat - s2 + dRow) : (2N + (4N-2)*msing + dRow) + b[drive_row] = 1 + x = use_lu ? (M_lu \ b) : (M_pinv * b) + + debug && _log_bvp_solve(x, b, M_solve, jsing, side, dRow, msing, N, + ipert_all, col_edge, use_S_axis) + + for ksing in 1:msing + ipert_k = ipert_all[ksing] + dp_raw[dRow, 2ksing-1] = x[_col_left(ksing, N)[ipert_k+N]] + dp_raw[dRow, 2ksing] = x[_col_right(ksing, N)[ipert_k+N]] + end + end + + # PEST3 four-term combination [Chance PPPL-2527; Glasser-Kolemen 2018 PoP 25, 032501 Eq. 31]. + # Δ'[i,j] = (NW − NE − SW + SE) on each 2×2 block of dp_raw, in extended precision. + deltap_ext = zeros(Tc, msing, msing) + for i in 1:msing, j in 1:msing + deltap_ext[i, j] = dp_raw[2i, 2j] - dp_raw[2i, 2j-1] - dp_raw[2i-1, 2j] + dp_raw[2i-1, 2j-1] + end + deltap = ComplexF64.(deltap_ext) + + debug && _log_bvp_pest3(dp_raw, deltap, s2, msing, Tc) + # Return the PEST3-combined matrix AND the raw 2msing×2msing D' matrix (ComplexF64 + # for compatibility with downstream pest3_decompose / HDF5 writer). + return deltap, ComplexF64.(dp_raw) +end + +# Logging helpers for `compute_delta_prime_matrix!`. Called only when debug=true. +function _log_bvp_setup(chunks, sing, S_at_surface_left, use_S_axis, has_ua, + Phi_L_mats, Phi_R_mats, Phi_R_halves, ipert_all, wv, psio, N, msing) + @info "Δ' BVP: $(length(chunks)) chunks, $msing surfaces, N=$N" + @info "Δ' BVP: Axis BC: $(use_S_axis ? "S-based (Riccati)" : "FM-based (conditioned)")" + @info "Δ' BVP: Asymptotic basis: $(has_ua ? "available" : "NOT available (raw basis driving)")" + if use_S_axis + for j in 1:msing + @info " S_left[$j]: max=$(@sprintf("%.2e", maximum(abs, S_at_surface_left[j]))), cond=$(@sprintf("%.2e", cond(S_at_surface_left[j])))" + end + end + if has_ua + for j in 1:msing + sp = sing[j] + T_l = [sp.ua_left[:,:,1]; sp.ua_left[:,:,2]] + T_r = [sp.ua_right[:,:,1]; sp.ua_right[:,:,2]] + @info " Surface $j: cond(T_left)=$(@sprintf("%.2e", cond(T_l))), cond(T_right)=$(@sprintf("%.2e", cond(T_r)))" + ipert_j = ipert_all[j] + @info " Surface $j ua_left (ipert=$ipert_j, psi_ua_left=$(@sprintf("%.8f", sp.psi_ua_left))):" + for i in 1:min(5, N) + @info " ua($i,$ipert_j,1)=$(@sprintf("%16.8e %16.8e", real(sp.ua_left[i,ipert_j,1]), imag(sp.ua_left[i,ipert_j,1]))) ua($i,$ipert_j,2)=$(@sprintf("%16.8e %16.8e", real(sp.ua_left[i,ipert_j,2]), imag(sp.ua_left[i,ipert_j,2])))" + end + @info " small: ua(1,$(ipert_j+N),1)=$(@sprintf("%16.8e %16.8e", real(sp.ua_left[1,ipert_j+N,1]), imag(sp.ua_left[1,ipert_j+N,1])))" + end + end + for j in 1:msing-1 + Phi_L_h, Phi_R_h = Phi_R_halves[j] + @info " Inter-surface $j→$(j+1): half_L cond=$(@sprintf("%.2e",cond(Phi_L_h))), half_R cond=$(@sprintf("%.2e",cond(Phi_R_h))), full cond=$(@sprintf("%.2e",cond(Phi_R_mats[j+1])))" + end + @info " Phi_R[$(msing+1)] (edge): cond=$(@sprintf("%.2e",cond(Phi_R_mats[msing+1])))" + for j in 1:msing + @info " Surface $j (m=$(sing[j].m[1])): ipert=$(ipert_all[j]), cond(Phi_L)=$(@sprintf("%.2e", cond(Phi_L_mats[j])))" + end + @info "Δ' BVP: Vacuum BC $(wv === nothing ? "off (conducting wall)" : "on (psio=$psio)")" + for j in 1:msing + if !isempty(sing[j].delta_prime) + @info " Surface $j ca-based Δ' = $(@sprintf("%.6f%+.6fi", real(sing[j].delta_prime[1]), imag(sing[j].delta_prime[1])))" + end + end +end + +function _log_S_axis_shooting_propagators(uShootR, uShootL, uAxis, S_at_surface_left, + T_left_mats, ipert_all, has_ua, msing, N) + @info " Shooting propagators (S-based axis BC, no axis unknowns):" + for j in 1:msing + shoot_R_str = @sprintf("%.2e", cond(uShootR[j])) + shoot_L_str = j >= 2 ? @sprintf("%.2e", cond(uShootL[j])) : "N/A (S axis BC)" + @info " uShootL[$j]: cond=$shoot_L_str, uShootR[$j]: cond=$shoot_R_str" + end + S1 = S_at_surface_left[1] + if has_ua + T1 = T_left_mats[1] + axis_BC = T1[1:N, :] - S1 * T1[N+1:2N, :] + @info " S-axis BC matrix: cond=$(@sprintf("%.2e", cond(axis_BC)))" + end + for j in 1:msing + ipert_j = ipert_all[j] + col_norms_R = [norm(view(uShootR[j], :, k)) for k in 1:2N] + @info " uShootR[$j] column norms: min=$(@sprintf("%.2e", minimum(col_norms_R))), max=$(@sprintf("%.2e", maximum(col_norms_R)))" + @info " uShootR[$j] col ipert=$ipert_j norm=$(@sprintf("%.2e", col_norms_R[ipert_j])), col ipert+N=$(ipert_j+N) norm=$(@sprintf("%.2e", col_norms_R[ipert_j+N]))" + if j >= 2 + col_norms_L = [norm(view(uShootL[j], :, k)) for k in 1:2N] + @info " uShootL[$j] column norms: min=$(@sprintf("%.2e", minimum(col_norms_L))), max=$(@sprintf("%.2e", maximum(col_norms_L)))" + @info " uShootL[$j] col ipert=$ipert_j norm=$(@sprintf("%.2e", col_norms_L[ipert_j])), col ipert+N=$(ipert_j+N) norm=$(@sprintf("%.2e", col_norms_L[ipert_j+N]))" + end + end + for j in 1:msing-1 + mid_block = hcat(uShootR[j], -uShootL[j+1]) + @info " Midpoint $j→$(j+1): cond([uShootR[$j] | -uShootL[$(j+1)]]) = $(@sprintf("%.2e", cond(mid_block)))" + col_norms_Ljp1 = [norm(view(uShootL[j+1], :, k)) for k in 1:2N] + @info " uShootL[$(j+1)] all col norms: $([(@sprintf("%.2e", c)) for c in col_norms_Ljp1])" + end +end + +function _log_bvp_solve(x, b, M_solve, jsing, side, dRow, msing, N, + ipert_all, col_edge, use_S_axis) + residual = norm(ComplexF64.(M_solve * x - b)) + side_str = side == 1 ? "left" : "right" + @info " BVP solve: jsing=$jsing side=$side_str (dRow=$dRow): ||Mx-b||=$(@sprintf("%.2e", residual)), ||x||=$(@sprintf("%.2e", Float64(norm(x))))" + for ks in 1:msing + ipert_ks = ipert_all[ks] + cl = _col_left(ks, N) + cr = _col_right(ks, N) + xl_big = ComplexF64(x[cl[ipert_ks]]) + xl_small = ComplexF64(x[cl[ipert_ks+N]]) + xr_big = ComplexF64(x[cr[ipert_ks]]) + xr_small = ComplexF64(x[cr[ipert_ks+N]]) + @info " surf $ks: x_left[big]=$(@sprintf("%+.4e%+.4ei", real(xl_big), imag(xl_big))), x_left[small]=$(@sprintf("%+.4e%+.4ei", real(xl_small), imag(xl_small)))" + @info " surf $ks: x_right[big]=$(@sprintf("%+.4e%+.4ei", real(xr_big), imag(xr_big))), x_right[small]=$(@sprintf("%+.4e%+.4ei", real(xr_small), imag(xr_small)))" + @info " surf $ks: ||x_left||=$(@sprintf("%.2e", Float64(norm(x[cl])))), ||x_right||=$(@sprintf("%.2e", Float64(norm(x[cr]))))" + end + if use_S_axis + @info " ||x_edge||=$(@sprintf("%.2e", Float64(norm(x[col_edge]))))" + end +end + +function _log_bvp_pest3(dp_raw, deltap, s2, msing, Tc) + @info "Δ' BVP: Full dp_raw matrix ($(s2)×$(s2)) [$(Tc)]:" + for i in 1:s2 + row_str = join([@sprintf("%+.6e", Float64(real(dp_raw[i,j]))) for j in 1:s2], " ") + @info " dp_raw[$i,:] = $row_str" + end + @info "Δ' BVP: Raw dp diagonal = $([@sprintf("%.4f%+.4fi", Float64(real(dp_raw[i,i])), Float64(imag(dp_raw[i,i]))) for i in 1:s2])" + @info "Δ' BVP: deltap diagonal = $([@sprintf("%.4f%+.4fi", real(deltap[i,i]), imag(deltap[i,i])) for i in 1:msing])" +end + +""" + pest3_decompose(dp_raw::AbstractMatrix) -> (A', B', Γ', Δ') + +Rotate the raw 2m×2m outer-region matching matrix `dp_raw` (side-major +ordering `[L_s1, R_s1, L_s2, R_s2, …]`) into the Pletzer–Dewar 1991 parity +blocks. Given rows and columns paired by surface (odd index = left, even +index = right), the Fortran RDCON parity combination is + +``` +A'(i,j) = RR + RL + LR + LL (even-i, even-j) — interchange↔interchange +B'(i,j) = RR − RL + LR − LL (even-i, odd-j) — interchange↔tearing +Γ'(i,j) = RR + RL − LR − LL (odd-i, even-j) — tearing↔interchange +Δ'(i,j) = RR − RL − LR + LL (odd-i, odd-j) — tearing↔tearing +``` + +where `RR = dp_raw[2i, 2j]`, `RL = dp_raw[2i, 2j−1]`, +`LR = dp_raw[2i−1, 2j]`, `LL = dp_raw[2i−1, 2j−1]`. Each block is m×m. + +Matches Fortran exactly — no ½ prefactor (Pletzer–Dewar multiply by ½, but +the Fortran RDCON code leaves it commented out and our Julia port follows +Fortran to keep the benchmark bit-identical; the prefactor cancels in +`det(D' − D(γ)) = 0`). + +The Δ' block returned here equals `intr.delta_prime_matrix` (the m×m PEST3 +tearing projection computed inside `compute_delta_prime_matrix!`). + +# Arguments + + - `dp_raw` — 2m×2m complex matrix (typically `intr.delta_prime_raw`). + +# Returns + +Named tuple `(A=A', B=B', Γ=Gp, Δ=Dp)` of four m×m complex matrices. In the +full `det(D' − D(γ)) = 0` eigenvalue problem, these fill the 2m×2m outer +matrix as `D' = [[A' B'] [Γ' Δ']]` with the interchange channel (Glasser +stabilization) in the upper-left block and the tearing channel in the +lower-right. +""" +function pest3_decompose(dp_raw::AbstractMatrix) + s2 = size(dp_raw, 1) + size(dp_raw, 2) == s2 || + throw(ArgumentError("pest3_decompose: dp_raw must be square, got $(size(dp_raw))")) + iseven(s2) || + throw(ArgumentError("pest3_decompose: dp_raw side must be 2m for integer m, got $s2")) + m = s2 ÷ 2 + Tc = eltype(dp_raw) + Ap = zeros(Tc, m, m) + Bp = zeros(Tc, m, m) + Gp = zeros(Tc, m, m) + Dp = zeros(Tc, m, m) + for i in 1:m, j in 1:m + LL = dp_raw[2i-1, 2j-1] + LR = dp_raw[2i-1, 2j] + RL = dp_raw[2i, 2j-1] + RR = dp_raw[2i, 2j] + Ap[i, j] = RR + RL + LR + LL + Bp[i, j] = RR - RL + LR - LL + Gp[i, j] = RR + RL - LR - LL + Dp[i, j] = RR - RL - LR + LL + end + return (A=Ap, B=Bp, Γ=Gp, Δ=Dp) +end diff --git a/src/ForceFreeStates/Riccati/Driver.jl b/src/ForceFreeStates/Riccati/Driver.jl new file mode 100644 index 000000000..3defcdf24 --- /dev/null +++ b/src/ForceFreeStates/Riccati/Driver.jl @@ -0,0 +1,376 @@ +""" + Riccati/ - Dual Riccati reformulation of the Euler-Lagrange ODE + +Implements the dual Riccati matrix S = U₁ · U₂⁻¹ = P⁻¹, which satisfies a bounded +ODE even near singular surfaces where U₁, U₂ grow exponentially. This reduced stiffness +leads to fewer ODE integration steps and faster wall-clock time. + +Reference: Glasser (2018) Phys. Plasmas 25, 032507 — Eq. 19 (adapted for dual form S = P⁻¹) +where P = U₂ · U₁⁻¹ is the forward plasma response matrix. + +## Dual Riccati ODE + +Starting from the Euler-Lagrange system [Glasser 2016 eq. 24]: + dU₁/dψ = A·U₁ + B·U₂ A = -Q·F̄⁻¹·K̄, B = Q·F̄⁻¹·Q + dU₂/dψ = C·U₁ + D·U₂ C = Ḡ - K̄†·F̄⁻¹·K̄, D = K̄†·F̄⁻¹·Q + +with S = U₁·U₂⁻¹, differentiating gives the Riccati ODE: + dS/dψ = B + A·S - S·D - S·C·S + +Setting w = Q - K̄·S (shape N×N) and v = F̄⁻¹·w (Cholesky solve), this simplifies to: + dS/dψ = w†·v - S·Ḡ·S [Glasser 2018 eq. 19, dual form] + +## Integration Strategy + +### Why not integrate the Riccati ODE directly? + +`riccati_der!` evaluates the explicit Riccati RHS `dS/dψ = w†F̄⁻¹w − S·Ḡ·S` correctly, +but this ODE is **quadratic** in S. Near a rational surface, S grows large, so the quadratic +term `-SGS` dominates and the RHS grows as |S|². Explicit adaptive solvers (Vern9) use +*relative* error control: they accept a step when |Δu|/|u| < reltol. When |S| is large, +the absolute error |ΔS| can be enormous while the relative error stays within tolerance. +The solver takes large steps through what is effectively a near-blowup — no amount of +step-size adaptation saves it because the problem is the error *metric*, not the step size. +An implicit solver could handle this stiffness, but is deferred. + +### Actual implementation: EL ODE + renormalization + +Instead we integrate the standard EL ODE (`sing_der!`) in the (U₁, U₂) variables and +recover S = U₁·U₂⁻¹ by renormalization. This achieves the same Riccati trajectory with +**no accuracy loss**: + +- `sing_der!` evaluates the exact EL RHS — no approximation. +- Vern9 integrates (U₁, U₂) to **9th-order accuracy** with the adaptive step-size + controller enforcing the configured reltol at every accepted step. +- Renormalization `S = U₁·U₂⁻¹` is **exact** (a change of variables, not an approximation). +- The global error is the same as the standard EL path — controlled by the ODE solver + reltol, not by the renormalization frequency. + +This works because the EL ODE is **linear** in (U₁, U₂): the RHS does not grow with |S|, +so relative error control is faithful even when S is large. Renormalization triggered by +`renormalize_riccati_inplace!` in the callback (when max(|U₁|) or max(|U₂|) > ucrit) keeps +both matrices bounded, preventing overflow and maintaining a well-conditioned state for the +solver — exactly analogous to Gaussian reduction in the standard ODE. + +### Consistency with the Riccati ODE (local analysis) + +To verify the method is consistent with the Riccati ODE, consider a single step from (S, I): + + After one step: U₁_new = S + (A·S + B)·Δψ + O(Δψ²), U₂_new = I + (C·S + D)·Δψ + O(Δψ²) + Renorm: S_new = U₁_new · U₂_new⁻¹ = S + (B + A·S − S·D − S·C·S)·Δψ + O(Δψ²) ✓ + +The leading term matches the Riccati ODE exactly. This is a local consistency check only — +it does not imply the integration is first-order. In practice Vern9 captures all higher-order +terms through its internal stages, achieving 9th-order global accuracy at the configured reltol. + +## Storage Convention + +During chunk integration (with sing_der! as ODE RHS): + u[:,:,1] = U₁ (starts as S_prev, evolves toward new S) + u[:,:,2] = U₂ (starts as I, evolves with EL dynamics) + +After renormalization (at crossing or when norms exceed ucrit): + u[:,:,1] = S = U₁ · U₂⁻¹ + u[:,:,2] = I + +This is compatible with downstream code (which uses U₁/U₂ ratio): + - Free.jl: wp = u[:,:,2] / u[:,:,1] = I · S⁻¹ = P ✓ (post-renorm) + - FixedBoundaryStability.jl: crit = min_eigval(u[:,:,1] / u[:,:,2]) = min_eigval(S) ✓ + - Axis init: determined by `ctrl.fixed_axis`. When `true`, U₁=0, U₂=I → S(ψ₀)=0 (original + Glasser fixed-axis BC). When `false` (default), Frobenius eigenvalue init [Glasser 2016 Eq. 51] + sets U₂=I and U₁ to the regular Frobenius eigenvector per mode → S(ψ₀) = U₁_Frobenius is + nonzero in general. Riccati S-evolution remains well-defined either way. + +## Key Differences from Standard Integration + +1. `sing_der!` is used as the ODE RHS (same as standard, NOT `riccati_der!`) +2. `riccati_integrator_callback!` replaces `integrator_callback!`: uses + `renormalize_riccati_inplace!` instead of Gaussian reduction +3. `riccati_cross_ideal_singular_surf!` replaces `cross_ideal_singular_surf!`: skips Gaussian + reduction and uses ipert_res directly for column zeroing, then renormalizes to (S_new, I) +4. `transform_u!` is skipped — S is already the true solution +""" + +""" + riccati_eulerlagrange_integration(ctrl, equil, ffit, intr) -> (odet, propagators, chunks, S_left) + +The Riccati/STRIDE integrator: a chunked fundamental matrix (propagator) driver for the EL +integration. The trailing three return values feed the Δ' BVP in `compute_delta_prime_matrix!`; +this is the only branch that produces them. + +Solves the same system as [`forward_eulerlagrange_integration`](@ref), but integrates all bulk +chunks concurrently using `Threads.@threads`, then re-integrates the outer plasma serially: + +1. **Chunk generation**: calls `chunk_el_integration_bounds`, then `balance_integration_chunks` + to sub-divide chunks for load balancing. The chunk count depends only on `intr.msing` and + `ctrl.nchunks`, never on the thread count, so results are thread-independent. +2. **Propagator phase**: `integrate_propagator_chunk!` integrates each chunk independently + from identity initial conditions (no accumulated state, no normalization/callback). + Each thread uses a private `OdeState` proxy for `sing_der!` side effects. +3. **Serial assembly**: propagators are applied sequentially with `apply_propagator!`. + Rational surface crossings use `riccati_cross_ideal_singular_surf!` (no Gaussian + reduction). +4. **Outer plasma re-integration**: after the last rational surface crossing, the outer + plasma (from last ψ_s to psilim) is re-integrated using `riccati_integrate_chunk!`. + FM propagation in this region is prone to precision loss for high N (exponential growth + without renormalization); Riccati integration keeps matrices bounded and provides dense + checkpoints for `findmax_dW_edge!`. + +Select via `integrator = "riccati"` in `[ForceFreeStates]` of gpec.toml. Requires +`singfac_min != 0`. Uses whatever threads `julia -t` provides; `ctrl.nchunks` is the only +tunable. + +**Key differences from the forward integrator:** +- No Gaussian reduction in the propagator BVP phase (crossings use the + Riccati-style algorithm, `odet.ifix` stays 0) +- `transform_u!` is called on the odet but is a no-op (ifix=0) +- Outer plasma uses serial Riccati integration for numerical stability +- `odet.u_store` holds chunk-endpoint Riccati states, not dense Euler-Lagrange ξ, and + `odet.u_store_el_basis` stays `false`: this integrator never claims the EL basis, so + PerturbedEquilibrium and the HDF5 forward-integration ξ datasets require the forward path. + +**Bidirectional integration for large-N accuracy:** +The crossing chunk (nearest to each rational surface singL[j]) is integrated *backward* +(`direction=-1`, `tspan` reversed). Backward integration of a region where solutions grow +exponentially forward causes them to *decay*, so the resulting backward FM Φ_bwd is +well-conditioned. The accurate forward propagation is recovered as Φ_bwd⁻¹ via a stable +LU solve in `apply_propagator_inverse!`. This follows the same principle as STRIDE +(Glasser 2018 Phys. Plasmas 25, 032501). The all-forward path had ~10% energy error for +the DIIID-like example (N=26, n=1); bidirectional reduces this to within 2%. +""" +function riccati_eulerlagrange_integration( + ctrl::ForceFreeStatesControl, equil::Equilibrium.PlasmaEquilibrium, + ffit::FourFitVars, intr::ForceFreeStatesInternal +) + odet = _initialize_parallel_odet(ctrl, equil, ffit, intr) + chunks, propagators, odet_proxies = _setup_parallel_chunks_and_proxies(odet, ctrl, intr) + _log_parallel_start(ctrl, odet, equil, chunks) + + _run_parallel_bvp_phase!(propagators, chunks, ctrl, equil, ffit, intr, odet_proxies) + + # Harvest solver-step counts accumulated thread-locally in each proxy during the BVP phase. + # The outer re-integration below uses riccati_integrate_chunk!, which counts via its callback. + odet.total_steps += sum(p.total_steps for p in odet_proxies) + + S_at_surface_left, last_crossing_step = + _assemble_propagators_serially!(odet, propagators, chunks, ctrl, equil, ffit, intr) + + _reintegrate_outer_plasma!(odet, last_crossing_step, ctrl, equil, ffit, intr) + + chunks, propagators = _handle_edge_dW_scan!(odet, chunks, propagators, ctrl, equil, ffit, intr) + + # compute_delta_prime_matrix! is called from the main pipeline (after free_run) so + # that vacuum response wv is available for the edge BC. With self-consistent truncation, + # the propagators/chunks returned here match intr.psilim exactly, so Δ' is well-defined + # for both truncate_at_dW_peak=false (full domain) and =true (peak). + if ctrl.verbose + @info "Evaluating fixed-boundary stability criterion" + end + odet.nzero = evaluate_stability_criterion!(odet, equil.profiles) + transform_u!(odet, intr) # no-op when ifix=0 (no Gaussian reduction) + + return odet, propagators, chunks, S_at_surface_left +end + +# Build odet and initialize at the magnetic axis. Same path as serial eulerlagrange_integration. +function _initialize_parallel_odet(ctrl::ForceFreeStatesControl, + equil::Equilibrium.PlasmaEquilibrium, + ffit::FourFitVars, + intr::ForceFreeStatesInternal) + odet = OdeState(intr.numpert_total, ctrl.numsteps_init, ctrl.numunorms_init, intr.msing) + if ctrl.sing_start <= 0 + initialize_el_at_axis!(odet, ctrl, ffit, equil.profiles, intr) + elseif ctrl.sing_start <= intr.msing + error("sing_start > 0 not implemented yet!") + else + error("Invalid value for sing_start: $(ctrl.sing_start) > msing = $(intr.msing)") + end + # Prime odet.new = false (consistent with riccati path — no Gaussian reduction used). + odet.new = false + fill!(odet.unorm0, 1.0) + return odet +end + +# Build the (bidirectional) chunk list, allocate per-chunk propagators, and allocate +# per-thread proxy OdeStates sized by maxthreadid() (Julia 1.9+ may report threadid +# values above nthreads() due to the interactive thread pool). +function _setup_parallel_chunks_and_proxies(odet::OdeState, ctrl::ForceFreeStatesControl, + intr::ForceFreeStatesInternal) + # Bidirectional chunks: crossing chunks are assigned direction=-1 so they are + # integrated backward. The resulting Φ_bwd is well-conditioned because growing EL + # solutions decay backward; forward propagation is recovered via LU solve in + # apply_propagator_inverse! during serial assembly. + base_chunks = chunk_el_integration_bounds(odet, ctrl, intr; bidirectional=true) + chunks = balance_integration_chunks(base_chunks, ctrl, intr) + N = intr.numpert_total + propagators = [ChunkPropagator(N) for _ in chunks] + odet_proxies = [OdeState(N, 1, 1, 0) for _ in 1:Threads.maxthreadid()] + return chunks, propagators, odet_proxies +end + +function _log_parallel_start(ctrl::ForceFreeStatesControl, odet::OdeState, + equil::Equilibrium.PlasmaEquilibrium, + chunks::Vector{IntegrationChunk}) + ctrl.verbose || return + @info " ψ = $((@sprintf "%.3f" odet.psifac)), q = $((@sprintf "%.3f" equil.profiles.q_spline(odet.psifac)))" + @info " Riccati FM: $(length(chunks)) chunks over $(Threads.nthreads()) thread$(Threads.nthreads() == 1 ? "" : "s")" +end + +# Integrate each chunk's FM propagator from identity IC across whatever threads `julia -t` +# provides. The :static scheduler makes Threads.threadid() a stable index into odet_proxies. +# Each chunk is independent (identity IC, no accumulated state), so the result does not +# depend on how chunks are distributed across threads. +function _run_parallel_bvp_phase!(propagators::Vector{ChunkPropagator}, + chunks::Vector{IntegrationChunk}, + ctrl::ForceFreeStatesControl, + equil::Equilibrium.PlasmaEquilibrium, ffit::FourFitVars, + intr::ForceFreeStatesInternal, + odet_proxies::Vector{OdeState}) + Threads.@threads :static for i in eachindex(chunks) + integrate_propagator_chunk!(propagators[i], chunks[i], ctrl, equil, ffit, intr, + odet_proxies[Threads.threadid()]) + end +end + +# Apply per-chunk propagators serially to odet, renormalizing to (S, I) after each. +# This is the Julia equivalent of STRIDE's ode_fixup: products of K chunk FMs can have +# cond ~ (cond_per_chunk)^K causing catastrophic cancellation for large N (≥20); periodic +# renorm keeps each step at O(cond_per_chunk). Backward (direction=-1) crossing chunks are +# applied via apply_propagator_inverse! (Φ_bwd⁻¹ from LU solve). S_at_surface_left records +# the well-conditioned Riccati S at each surface's left boundary for use as the Δ' BVP +# axis BC. Returns (S_at_surface_left, last_crossing_step). +function _assemble_propagators_serially!(odet::OdeState, propagators::Vector{ChunkPropagator}, + chunks::Vector{IntegrationChunk}, + ctrl::ForceFreeStatesControl, + equil::Equilibrium.PlasmaEquilibrium, + ffit::FourFitVars, intr::ForceFreeStatesInternal) + N = intr.numpert_total + S_at_surface_left = Matrix{ComplexF64}[] + last_crossing_step = 1 + for (i, chunk) in enumerate(chunks) + if chunk.direction == -1 + apply_propagator_inverse!(odet, propagators[i]) + else + apply_propagator!(odet, propagators[i]) + end + renormalize_riccati_inplace!(odet.u, N) + odet.psifac = chunk.psi_end + odet.q = equil.profiles.q_spline(odet.psifac) + + if ctrl.verbose + @info " ψ = $((@sprintf "%.3f" odet.psifac)), q= $((@sprintf "%.3f" odet.q)), max(S) = $((@sprintf "%.2e" maximum(abs, odet.u[:,:,1]))), steps = $(odet.step-1)" + end + + if chunk.needs_crossing + ctrl.kinetic_factor > 0 && error("kinetic_factor > 0 not implemented yet in Riccati!") + # State is (S, I) from the renorm above — well-conditioned at the surface's left boundary. + push!(S_at_surface_left, copy(odet.u[:, :, 1])) + riccati_cross_ideal_singular_surf!(odet, ctrl, equil, ffit, intr, chunk.ising) + last_crossing_step = odet.step - 1 + else + # Save non-crossing end-of-chunk state. These columns are FM/Riccati chunk + # endpoints, not the Euler-Lagrange state, so the odet never claims the EL basis. + odet.u_store_el_basis = false + if odet.step >= size(odet.u_store, 4) + resize_storage!(odet) + end + odet.psi_store[odet.step] = odet.psifac + odet.q_store[odet.step] = odet.q + @views odet.u_store[:, :, :, odet.step] .= odet.u + odet.step += 1 + end + end + return S_at_surface_left, last_crossing_step +end + +# Re-integrate the outer plasma (last rational surface → psilim) with Riccati for numerical +# stability and dense checkpoint storage. FM propagation here is prone to precision loss at +# high N because the solution grows exponentially without renormalization; Riccati keeps +# matrices bounded. Dense checkpoints are also needed by findmax_dW_edge!. The u_store +# entry at last_crossing_step holds (U₁_new, U₂_new) from riccati_cross_ideal_singular_surf! +# before renormalization; we renorm here to (S_new, I) as the Riccati starting state. +function _reintegrate_outer_plasma!(odet::OdeState, last_crossing_step::Int, + ctrl::ForceFreeStatesControl, + equil::Equilibrium.PlasmaEquilibrium, ffit::FourFitVars, + intr::ForceFreeStatesInternal) + N = intr.numpert_total + odet.u .= odet.u_store[:, :, :, last_crossing_step] + odet.psifac = odet.psi_store[last_crossing_step] + odet.q = odet.q_store[last_crossing_step] + odet.step = last_crossing_step + 1 + renormalize_riccati_inplace!(odet.u, N) + outer_chunk = IntegrationChunk(; psi_start=odet.psifac, psi_end=intr.psilim * (1 - eps), + needs_crossing=false, ising=0) + riccati_integrate_chunk!(odet, ctrl, equil, ffit, intr, outer_chunk) + # Post: odet.u is in (S, I) form; odet.step points to next empty slot. +end + +# Edge-dW scan over [psiedge, psilim] — populates odet.edge_scan for HDF5. By default +# (truncate_at_dW_peak=false) it's diagnostic-only: integration domain is unchanged. +# When truncate_at_dW_peak=true, the dW peak becomes the new physical edge: intr.psilim, +# odet, propagators, and chunks are made self-consistent (straddling chunk rebuilt with +# shorter psi_end; chunks past the new boundary dropped). Without that rebuild, the Δ' BVP +# would apply the edge BC at the truncated psilim to a propagator still extending to the +# original psilim — silently shifting the outermost rational's Δ' by tens of percent. +# Returns the (possibly truncated) chunks and propagators arrays. +function _handle_edge_dW_scan!(odet::OdeState, chunks::Vector{IntegrationChunk}, + propagators::Vector{ChunkPropagator}, + ctrl::ForceFreeStatesControl, + equil::Equilibrium.PlasmaEquilibrium, ffit::FourFitVars, + intr::ForceFreeStatesInternal) + N = intr.numpert_total + odet.step -= 1 + trim_storage!(odet) + ctrl.psiedge < intr.psilim || return chunks, propagators + + saved_psifac, saved_u = odet.psifac, copy(odet.u) + peak_step = findmax_dW_edge!(odet, ctrl, equil, ffit, intr) + + if !ctrl.truncate_at_dW_peak + odet.psifac = saved_psifac + odet.u .= saved_u + if ctrl.verbose + @info "Edge-dW peak (diagnostic): ψ = $((@sprintf "%.2f" odet.psi_store[peak_step])), q = $((@sprintf "%.2f" odet.q_store[peak_step])); integration domain unchanged" + end + return chunks, propagators + end + + # Truncate to dW peak: relocate intr.psilim and rebuild Δ' BVP self-consistently. + n_chunks_before = length(chunks) + odet.step = peak_step + trim_storage!(odet) + intr.psilim = odet.psi_store[end] + intr.qlim = odet.q_store[end] + odet.u .= odet.u_store[:, :, :, end] + renormalize_riccati_inplace!(odet.u, N) # stored snapshot may be pre-renorm + + peak_psi = odet.psi_store[end] + last_chunk_idx = findlast(c -> c.psi_start < peak_psi, chunks) + if last_chunk_idx === nothing + error("truncate_at_dW_peak: peak ψ=$peak_psi lies before all chunk starts") + end + straddling = chunks[last_chunk_idx] + if straddling.psi_end > peak_psi + new_chunk = IntegrationChunk( + psi_start = straddling.psi_start, + psi_end = peak_psi, + needs_crossing = straddling.needs_crossing, + ising = straddling.ising, + direction = straddling.direction, + ) + chunks[last_chunk_idx] = new_chunk + odet_proxy = OdeState(N, 1, 1, 0) + integrate_propagator_chunk!(propagators[last_chunk_idx], new_chunk, + ctrl, equil, ffit, intr, odet_proxy) + end + n_dropped = 0 + if last_chunk_idx < length(chunks) + n_dropped = length(chunks) - last_chunk_idx + chunks = chunks[1:last_chunk_idx] + propagators = propagators[1:last_chunk_idx] + end + if ctrl.verbose + @info "Truncating integration at peak edge dW (self-consistent): ψ = $((@sprintf "%.4f" peak_psi)), q = $((@sprintf "%.3f" odet.q_store[end])). Rebuilt chunk $last_chunk_idx; dropped $n_dropped of $n_chunks_before outer chunks." + end + return chunks, propagators +end diff --git a/src/ForceFreeStates/Riccati/Propagators.jl b/src/ForceFreeStates/Riccati/Propagators.jl new file mode 100644 index 000000000..5b0abefe4 --- /dev/null +++ b/src/ForceFreeStates/Riccati/Propagators.jl @@ -0,0 +1,518 @@ +# Chunk-propagator integration: EL-ODE fundamental matrices, Riccati renormalization, assembly. + +# Save-frequency thresholds for `riccati_integrator_callback!`. Near the right endpoint of +# a segment we save every step so that the crossing / chunk boundary captures fine detail; +# elsewhere we save every `ctrl.save_interval`-th step. The relative band catches normal- +# length chunks; the absolute floor catches short chunks where 5% of the span would be +# smaller than the typical ODE step. +const SAVE_NEAR_END_FRAC = 0.05 +const SAVE_NEAR_END_PSI = 1e-4 + +""" + assemble_fm_matrix(propagators, idx_range; condition=false) -> Matrix{ComplexF64} + +Assemble the 2N×2N fundamental matrix (propagator) by multiplying chunk propagators +in order for indices `idx_range`. Returns Φ_end * ... * Φ_start, so that the result +maps the IC at the start of `idx_range[1]` to the state at the end of `idx_range[end]`. + +Each `ChunkPropagator` stores the 2N columns of Φ split into two N×N×2 blocks: +``` + block_upper_ic[:,:,1:2] ↔ Φ[:,1:N] (result from IC=(I,0)) + block_lower_ic[:,:,1:2] ↔ Φ[:,N+1:2N] (result from IC=(0,I)) +``` + +When `condition=true`, applies Gaussian reduction (`condition_propagator!`) after each +multiplication step, following STRIDE's `ode_fixup` convention. This +prevents exponential growth of the accumulated product: without conditioning, products +of K chunk propagators can reach cond ~ (cond_per_chunk)^K, causing catastrophic +cancellation. With periodic conditioning, each step stays at O(cond_per_chunk) and +only the N well-conditioned U₂ columns (right half) survive. + +Use `condition=true` for the axis→first-surface segment, where the axis BC (U₁=0) +means only U₂ ICs are needed. Do NOT use for inter-surface segments where both U₁ +and U₂ components carry physical information. +""" +function assemble_fm_matrix(propagators::Vector{ChunkPropagator}, idx_range; + condition::Bool=false, + T_init::Union{Nothing,Matrix{ComplexF64}}=nothing) + # Determine matrix size from T_init if provided (lets us handle empty idx_range and even + # an empty propagators list, provided T_init carries the dimension). Otherwise fall back + # to the first propagator that actually exists in idx_range, with a final fallback to + # propagators[1] when both idx_range and T_init pin nothing down. + N = if T_init !== nothing + size(T_init, 1) ÷ 2 + elseif !isempty(idx_range) + size(propagators[first(idx_range)].block_upper_ic, 1) + else + @assert !isempty(propagators) "assemble_fm_matrix: cannot infer N from empty propagators with no T_init" + size(propagators[1].block_upper_ic, 1) + end + Phi = T_init !== nothing ? copy(T_init) : Matrix{ComplexF64}(I, 2N, 2N) + isempty(idx_range) && return Phi + for i in idx_range + p = propagators[i] + #! format: off + Phi_i = [p.block_upper_ic[:,:,1] p.block_lower_ic[:,:,1]; + p.block_upper_ic[:,:,2] p.block_lower_ic[:,:,2]] + #! format: on + Phi = Phi_i * Phi + if condition + condition_propagator!(Phi, N) + end + end + return Phi +end + +""" + condition_propagator!(Phi, N) + +Apply Gaussian reduction to the U₂-columns (columns N+1:2N) of a 2N×2N propagator +matrix in-place, following STRIDE's `ode_fixup` convention. Triangularizes the U₁ +(upper N rows) subblock by pivoted elimination, improving the condition number so +the propagator can be used in a BVP without losing numerical rank. + +After conditioning, only the U₂ columns carry meaningful information; the U₁ columns +(1:N) are zeroed. The BVP axis block uses `Phi[:, N+1:2N]` (the conditioned half). +""" +function condition_propagator!(Phi::Matrix{ComplexF64}, N::Int) + # Work on the right half: columns N+1:2N (U₂ initial conditions) + cols = view(Phi, :, N+1:2N) + + # Sort columns by norm of the U₁ (upper N) block — largest first + norms = [norm(view(cols, 1:N, k)) for k in 1:N] + order = sortperm(norms; rev=true) + + mask_col = trues(N) # which columns remain to process + mask_row = trues(N) # which pivot rows remain available + + for isol in 1:N + kcol = order[isol] + mask_col[kcol] = false + + # Find best pivot row (largest |element| among unmasked rows) + best_row = 0 + best_val = 0.0 + for r in 1:N + if mask_row[r] && abs(cols[r, kcol]) > best_val + best_val = abs(cols[r, kcol]) + best_row = r + end + end + if best_row == 0 || best_val == 0 + continue + end + mask_row[best_row] = false + + # Eliminate this pivot from all other unmasked columns + pivot = cols[best_row, kcol] + for jcol in 1:N + if mask_col[jcol] + factor = -cols[best_row, jcol] / pivot + @views cols[:, jcol] .+= factor .* cols[:, kcol] + cols[best_row, jcol] = 0 # exact zero + end + end + end + + # Zero the U₁ columns (left half) — they are no longer meaningful + Phi[:, 1:N] .= 0 + return Phi +end + +""" + riccati_der!(du, u, params, psieval) + +Evaluate the explicit dual Riccati ODE right-hand side: + dS/dψ = w†·F̄⁻¹·w - S·Ḡ·S, w = Q - K̄·S + +where Q = diag(1/(m - n·q)) is the diagonal singular factor matrix. +The identity slice u[:,:,2] = I does not evolve (du[:,:,2] = 0). + +**REFERENCE IMPLEMENTATION — not called in production.** The explicit Riccati ODE is +numerically unstable for explicit solvers: the quadratic S·Ḡ·S term blows up when K̄·S ≫ Q. +The production path integrates `sing_der!` with periodic `renormalize_riccati_inplace!` +instead (see module docstring). Kept here for documentation of Eq. 19 in source form and +for future use with implicit solvers; exercised only by unit tests that verify the formula. + +See: Glasser (2018) Phys. Plasmas 25, 032507 — Eq. 19 (dual Riccati form) +""" +@with_pool pool function riccati_der!( + du::Array{ComplexF64,3}, + u::Array{ComplexF64,3}, + params::Tuple{ForceFreeStatesControl,Equilibrium.PlasmaEquilibrium, + FourFitVars,ForceFreeStatesInternal,OdeState,IntegrationChunk}, + psieval::Float64 +) + + _, equil, ffit, intr, odet, _ = params + + Npert = intr.numpert_total + S = @view u[:, :, 1] + dS = @view du[:, :, 1] + @view(du[:, :, 2]) .= 0 # identity does not evolve + + # Compute singfac = 1/(m - n·q) as column vector Q = diag(singfac_vec) + # [Glasser 2016 eq. 24] + singfac_vec = acquire!(pool, Float64, Npert) + singfac_mat = reshape(singfac_vec, intr.mpert, intr.npert) + odet.q = equil.profiles.q_spline(psieval; hint=odet.spline_hint) + singfac_mat .= 1.0 ./ ((intr.mlow:intr.mhigh) .- odet.q .* (intr.nlow:intr.nhigh)') + + # Allocate temporaries from pool + fmat_lower = acquire!(pool, ComplexF64, Npert, Npert) + kmat = similar!(pool, fmat_lower) + gmat = similar!(pool, fmat_lower) + w = similar!(pool, fmat_lower) # w = Q - K̄·S + v = similar!(pool, fmat_lower) # v = F̄⁻¹·w (then reused for S·Ḡ·S) + tmp = similar!(pool, fmat_lower) # scratch + + # Evaluate F̄ (Cholesky factor), K̄, Ḡ splines at current ψ + ffit.fmats_lower(vec(fmat_lower), psieval; hint=ffit._hint) + ffit.kmats(vec(kmat), psieval; hint=ffit._hint) + ffit.gmats(vec(gmat), psieval; hint=ffit._hint) + + # w = Q - K̄·S: w[i,j] = singfac_vec[i]·δ_ij - (K̄·S)[i,j] + # Q is DIAGONAL (singfac_vec[i] only on i==j), so we cannot broadcast singfac_vec + # over all columns — that would give the wrong off-diagonal values. + mul!(w, kmat, S) # w = K̄·S + @. w = -w # w = -K̄·S + for i in 1:Npert + @inbounds w[i, i] += singfac_vec[i] # add diagonal Q: w = Q - K̄·S + end + + # v = F̄⁻¹·w (in-place Cholesky solve with stored lower-triangular factor) + v .= w + ldiv!(LowerTriangular(fmat_lower), v) + ldiv!(UpperTriangular(fmat_lower'), v) + + # dS = w†·v - S·Ḡ·S [Glasser 2018 eq. 19, dual Riccati] + mul!(dS, adjoint(w), v) # dS = w†·v + + # Subtract S·Ḡ·S (reuse v and tmp to avoid extra allocation) + mul!(tmp, gmat, S) # tmp = Ḡ·S + mul!(v, S, tmp) # v = S·Ḡ·S + dS .-= v +end + +""" + riccati_integrator_callback!(integrator) + +Callback function for the Riccati ODE integrator. Handles tolerance updates, +renormalization, and storage at each step. + +Uses `sing_der!` as the ODE RHS: u[:,:,1] = U₁ (starts as S), u[:,:,2] = U₂ (starts as I). +When max(|U₁|) or max(|U₂|) exceeds `ctrl.ucrit`, applies `renormalize_riccati_inplace!` +to compute S = U₁·U₂⁻¹ and reset U₂ = I. This is the Riccati analogue of Gaussian +reduction in the standard `integrator_callback!`, and keeps the ODE inputs bounded. +""" +function riccati_integrator_callback!(integrator) + + ctrl, _, _, intr, odet, chunk = integrator.p + + odet.total_steps += 1 # count every accepted solver step (saved or not), as segment_callback! does + + # Use unified tolerance (matches integrate_el_region! on develop) + integrator.opts.reltol = ctrl.eulerlagrange_tolerance + + # Renormalize when norms exceed ucrit (analogous to Gaussian reduction in integrator_callback!) + # During sing_der! integration: u[:,:,1]=U₁ (grows), u[:,:,2]=U₂ (grows). + # Renorm computes S = U₁·U₂⁻¹ and resets U₂ = I, keeping inputs bounded. + if maximum(abs, @view(integrator.u[:, :, 1])) > ctrl.ucrit || + maximum(abs, @view(integrator.u[:, :, 2])) > ctrl.ucrit + renormalize_riccati_inplace!(integrator.u, intr.numpert_total) + end + + # Determine if we should save this step. Always save the first 1-2 steps of a segment + # and the last few steps near the right endpoint (relative band SAVE_NEAR_END_FRAC of the + # span, or absolute floor SAVE_NEAR_END_PSI for very short chunks); save every save_interval-th + # step in between. + psi_range = abs(integrator.sol.prob.tspan[2] - integrator.sol.prob.tspan[1]) + psi_remaining = abs(integrator.sol.prob.tspan[2] - integrator.t) + near_end = psi_remaining < SAVE_NEAR_END_FRAC * psi_range || psi_remaining < SAVE_NEAR_END_PSI + steps_in_segment = length(integrator.sol.t) + near_start = steps_in_segment <= 2 + should_save = near_start || near_end || (odet.step % ctrl.save_interval == 0) + + if should_save + store_ode_data!(odet, integrator.t, integrator.u) + end +end + +""" + riccati_integrate_chunk!(odet, ctrl, equil, ffit, intr, chunk) + +Integrate the dual Riccati ODE from `chunk.psi_start` to `chunk.psi_end`. + +Uses `sing_der!` as the ODE RHS with `riccati_integrator_callback!`, which applies +`renormalize_riccati_inplace!` (instead of Gaussian reduction) when norms exceed ucrit. +Starting state: u[:,:,1] = S_prev, u[:,:,2] = I (set by initialization or previous renorm). +Ending state: u[:,:,1] = U₁, u[:,:,2] = U₂ (ratio S = U₁·U₂⁻¹ is the updated Riccati matrix). +""" +function riccati_integrate_chunk!( + odet::OdeState, ctrl::ForceFreeStatesControl, equil::Equilibrium.PlasmaEquilibrium, + ffit::FourFitVars, intr::ForceFreeStatesInternal, chunk::IntegrationChunk +) + cb = DiscreteCallback((u, t, integrator) -> true, riccati_integrator_callback!) + rtol = ctrl.eulerlagrange_tolerance + prob = ODEProblem(sing_der!, odet.u, (chunk.psi_start, chunk.psi_end), + (ctrl, equil, ffit, intr, odet, chunk)) + sol = solve(prob, Vern9(); reltol=rtol, callback=cb, save_everystep=false, save_end=true) + odet.u .= sol.u[end] + odet.psifac = sol.t[end] + # Renormalize end state to (S, I) convention for the next chunk. + # When a crossing follows (needs_crossing=true), skip renorm so that ca_l is computed + # from the bounded (U₁, U₂) state in riccati_cross_ideal_singular_surf!: this gives + # consistent normalization with ca_r (also from pre-renorm state), enabling correct Δ'. + # The callback guarantees max(|U₁|), max(|U₂|) ≤ ucrit, so the state is bounded. + if !chunk.needs_crossing + renormalize_riccati_inplace!(odet.u, intr.numpert_total) + end +end + +""" + renormalize_riccati!(odet, intr) + +After a singular surface crossing, restore the canonical Riccati storage convention: + u[:,:,1] = S_new = U₁_new · U₂_new⁻¹ + u[:,:,2] = I + +`riccati_cross_ideal_singular_surf!` leaves u[:,:,1] = U₁_new and u[:,:,2] = U₂_new (not I), +so this step is required before continuing the Riccati integration. + +The u_store entry from the crossing correctly has U₁_new and U₂_new (stored before this call), +so `compute_smallest_eigenvalue` still computes U₁_new/U₂_new = S_new correctly. +""" +function renormalize_riccati!(odet::OdeState, intr::ForceFreeStatesInternal) + N = intr.numpert_total + # S_new = U₁_new · U₂_new⁻¹ (in-place to avoid allocation) + U2_copy = copy(@view odet.u[:, :, 2]) + rdiv!(@view(odet.u[:, :, 1]), lu!(U2_copy)) + # Reset U₂ = I + fill!(@view(odet.u[:, :, 2]), 0) + for i in 1:N + odet.u[i, i, 2] = 1 + end +end + +""" + renormalize_riccati_inplace!(u, N) + +In-place Riccati renormalization on an arbitrary N×N×2 array: + u[:,:,1] = U₁ · U₂⁻¹ (new S) + u[:,:,2] = I + +Used in `riccati_integrator_callback!` to renormalize the integrator's live state +when column norms grow beyond `ctrl.ucrit`, analogous to Gaussian reduction in the +standard ODE. This keeps the inputs to `sing_der!` bounded, preventing the same +exponential growth that occurs in the standard (non-Riccati) ODE without Gaussian reduction. +""" +function renormalize_riccati_inplace!(u::Array{ComplexF64,3}, N::Int) + U2_copy = copy(@view u[:, :, 2]) + rdiv!(@view(u[:, :, 1]), lu!(U2_copy)) + fill!(@view(u[:, :, 2]), 0) + for i in 1:N + u[i, i, 2] = 1 + end +end + +""" + integrate_propagator_chunk!(prop, chunk, ctrl, equil, ffit, intr, odet_proxy) + +Compute the fundamental matrix (propagator) for one integration chunk by solving the +EL ODE twice from identity-block initial conditions. + +The first solve uses IC = (I_N, 0_N) (U₁=I, U₂=0) and stores the result in +`prop.block_upper_ic`. The second uses IC = (0_N, I_N) (U₁=0, U₂=I) and stores +the result in `prop.block_lower_ic`. + +`odet_proxy` is a per-thread lightweight `OdeState` used to provide thread-local +storage for `sing_der!` side effects (`q`, `ud`, `spline_hint`). Multiple threads +may call this function concurrently using distinct `odet_proxy` objects. + +No callback is used: the propagator integration proceeds without normalization or +storage steps, since the identity ICs ensure bounded solutions within each chunk. +""" +function integrate_propagator_chunk!( + prop::ChunkPropagator, + chunk::IntegrationChunk, + ctrl::ForceFreeStatesControl, + equil::Equilibrium.PlasmaEquilibrium, + ffit::FourFitVars, + intr::ForceFreeStatesInternal, + odet_proxy::OdeState +) + N = intr.numpert_total + # Reverse tspan for backward chunks (direction=-1): OrdinaryDiffEq handles negative tspan + # naturally. The resulting propagator maps state at psi_end → psi_start, which is + # well-conditioned because exponentially growing solutions (forward) decay backward. + tspan = chunk.direction == 1 ? + (chunk.psi_start, chunk.psi_end) : + (chunk.psi_end, chunk.psi_start) + rtol = ctrl.eulerlagrange_tolerance + params = (ctrl, equil, ffit, intr, odet_proxy, chunk) + + # Upper block IC: U₁ = I, U₂ = 0 + u_upper = zeros(ComplexF64, N, N, 2) + for i in 1:N + u_upper[i, i, 1] = 1 + end + odet_proxy.spline_hint[] = 1 + odet_proxy.ffit_hint[] = 1 + prob = ODEProblem(sing_der!, u_upper, tspan, params) + sol = solve(prob, Vern9(); reltol=rtol, save_everystep=false, save_end=true) + prop.block_upper_ic .= sol.u[end] + odet_proxy.total_steps += sol.stats.naccept # thread-local; summed into odet after the BVP barrier + + # Lower block IC: U₁ = 0, U₂ = I + u_lower = zeros(ComplexF64, N, N, 2) + for i in 1:N + u_lower[i, i, 2] = 1 + end + odet_proxy.spline_hint[] = 1 + odet_proxy.ffit_hint[] = 1 + prob = ODEProblem(sing_der!, u_lower, tspan, params) + sol = solve(prob, Vern9(); reltol=rtol, save_everystep=false, save_end=true) + prop.block_lower_ic .= sol.u[end] + odet_proxy.total_steps += sol.stats.naccept +end + +""" + integrate_fm_with_ua_ic(chunks, chunk_range, ua, ctrl, equil, ffit, intr; + backward=false) -> Matrix{ComplexF64} + +Re-integrate a span of chunks using ua (asymptotic solution) as initial conditions, matching +Fortran STRIDE's uFM_sing_init behavior. Returns a 2N×2N fundamental matrix +where column j is the ODE solution at the span endpoint with IC = column j of T = [ua[:,:,1]; ua[:,:,2]]. + +When `backward=false` (default): ua is the IC at psi_start, integrate forward to psi_end. +When `backward=true`: ua is the IC at psi_end, integrate backward to psi_start. The result +maps asymptotic coefficients at psi_end → state at psi_start. + +This provides numerically accurate propagators near singular surfaces because the ODE integrator +maintains per-column relative accuracy even when columns span a 10^8+ dynamic range (big/small +solutions). In contrast, post-multiplying a pre-computed identity-IC propagator by T loses the +small-solution information to roundoff. +""" +function integrate_fm_with_ua_ic( + chunks::Vector{IntegrationChunk}, + chunk_range::UnitRange{Int}, + ua::Array{ComplexF64,3}, + ctrl::ForceFreeStatesControl, + equil::Equilibrium.PlasmaEquilibrium, + ffit::FourFitVars, + intr::ForceFreeStatesInternal; + backward::Bool = false, + psi_ua::Float64 = NaN +) + N = intr.numpert_total + psi_start = chunks[first(chunk_range)].psi_start + psi_end = chunks[last(chunk_range)].psi_end + # Use stored ua ψ location if provided; otherwise fall back to chunk boundary. + # The ua is evaluated at the inner-layer boundary (exact ψ from singular crossing), + # which may differ slightly from the nearest chunk boundary. + if backward && !isnan(psi_ua) + psi_end = psi_ua # ua lives at psi_ua, not at chunk boundary + elseif !backward && !isnan(psi_ua) + psi_start = psi_ua # ua lives at psi_ua, not at chunk boundary + end + # For backward integration: start at psi_end (where ua lives), integrate to psi_start + tspan = backward ? (psi_end, psi_start) : (psi_start, psi_end) + rtol = ctrl.eulerlagrange_tolerance + + result = zeros(ComplexF64, 2N, 2N) + odet_proxy = OdeState(N, 1, 1, 0) + dummy_chunk = IntegrationChunk(psi_start, psi_end, false, 0, backward ? -1 : 1) + params = (ctrl, equil, ffit, intr, odet_proxy, dummy_chunk) + + # Batch 1: columns 1:N of T (big solutions) + u0 = zeros(ComplexF64, N, N, 2) + u0[:, :, 1] .= ua[:, 1:N, 1] + u0[:, :, 2] .= ua[:, 1:N, 2] + odet_proxy.spline_hint[] = 1 + odet_proxy.ffit_hint[] = 1 + prob = ODEProblem(sing_der!, u0, tspan, params) + sol = solve(prob, Vern9(); reltol=rtol, save_everystep=false, save_end=true) + result[1:N, 1:N] .= sol.u[end][:, :, 1] + result[N+1:2N, 1:N] .= sol.u[end][:, :, 2] + + # Batch 2: columns N+1:2N of T (small solutions) + u0[:, :, 1] .= ua[:, N+1:2N, 1] + u0[:, :, 2] .= ua[:, N+1:2N, 2] + odet_proxy.spline_hint[] = 1 + odet_proxy.ffit_hint[] = 1 + prob = ODEProblem(sing_der!, u0, tspan, params) + sol = solve(prob, Vern9(); reltol=rtol, save_everystep=false, save_end=true) + result[1:N, N+1:2N] .= sol.u[end][:, :, 1] + result[N+1:2N, N+1:2N] .= sol.u[end][:, :, 2] + + return result +end + +""" + apply_propagator!(odet, prop) + +Apply the chunk propagator `prop` to the current state `odet.u` in-place. + +The propagator acts as a linear map on the (U₁, U₂) pair: + + U₁_new = block_upper_ic[:,:,1] · U₁_prev + block_lower_ic[:,:,1] · U₂_prev + U₂_new = block_upper_ic[:,:,2] · U₁_prev + block_lower_ic[:,:,2] · U₂_prev + +This correctly propagates any state (not just the identity), including the +(S, I) form produced by Riccati-style crossings. + +Implements the subpropagator composition Φ(ψ₂, ψ₀) = Φ(ψ₂, ψ₁) · Φ(ψ₁, ψ₀) of +Glasser-Kolemen (2018) Phys. Plasmas 25, 032501 Eq. 29. +""" +function apply_propagator!(odet::OdeState, prop::ChunkPropagator) + U1_upper = @view prop.block_upper_ic[:, :, 1] + U2_upper = @view prop.block_upper_ic[:, :, 2] + U1_lower = @view prop.block_lower_ic[:, :, 1] + U2_lower = @view prop.block_lower_ic[:, :, 2] + + u1_prev = copy(@view odet.u[:, :, 1]) + u2_prev = copy(@view odet.u[:, :, 2]) + tmp = similar(u1_prev) + + # U₁_new = U1_upper · u1_prev + U1_lower · u2_prev + mul!(view(odet.u, :, :, 1), U1_upper, u1_prev) + mul!(tmp, U1_lower, u2_prev) + odet.u[:, :, 1] .+= tmp + + # U₂_new = U2_upper · u1_prev + U2_lower · u2_prev + mul!(view(odet.u, :, :, 2), U2_upper, u1_prev) + mul!(tmp, U2_lower, u2_prev) + odet.u[:, :, 2] .+= tmp +end + +""" + apply_propagator_inverse!(odet, prop) + +Apply the *inverse* of the chunk propagator `prop` to the current state `odet.u` in-place. + +Used for backward chunks (direction=-1): the stored propagator Φ_bwd maps state at +`psi_end` → state at `psi_start` (well-conditioned because solutions that grow +exponentially forward decay backward). To advance the Riccati state from `psi_start` +to `psi_end`, we solve Φ_bwd · x = u_old, which gives x = Φ_bwd⁻¹ · u_old = Φ_fwd · u_old. + +Since Φ_bwd is well-conditioned, the LU solve is accurate, giving the same result as +applying the (ill-conditioned) forward propagator Φ_fwd but with far better precision. + +Implements the inverse subpropagator identity Φ(ψ₂, ψ₁) = Φ(ψ₁, ψ₂)⁻¹ of +Glasser-Kolemen (2018) Phys. Plasmas 25, 032501 Eq. 33. +""" +function apply_propagator_inverse!(odet::OdeState, prop::ChunkPropagator) + N = size(odet.u, 1) + # Assemble 2N×2N backward FM Φ_bwd + #! format: off + Φ = [prop.block_upper_ic[:,:,1] prop.block_lower_ic[:,:,1]; + prop.block_upper_ic[:,:,2] prop.block_lower_ic[:,:,2]] + #! format: on + # Φ_bwd maps state at psi_end → psi_start (well-conditioned). + # We want Φ_fwd = Φ_bwd⁻¹ to advance state from psi_start → psi_end. + # Solving Φ_bwd · x = [U₁_old; U₂_old] gives x = Φ_bwd⁻¹ · [U₁_old; U₂_old]. + u_old = [odet.u[:,:,1]; odet.u[:,:,2]] # 2N × N + u_new = Φ \ u_old # LU solve, 2N × N + odet.u[:,:,1] .= u_new[1:N, :] + odet.u[:,:,2] .= u_new[N+1:2N, :] +end diff --git a/src/ForceFreeStates/Riccati/Types.jl b/src/ForceFreeStates/Riccati/Types.jl new file mode 100644 index 000000000..f876614f6 --- /dev/null +++ b/src/ForceFreeStates/Riccati/Types.jl @@ -0,0 +1,52 @@ +# Integration-chunk and chunk-propagator types for the fundamental-matrix (Riccati/STRIDE) driver. + +""" + IntegrationChunk + +A struct representing a region of integration in the Euler-Lagrange solver. + +## Fields + + - `psi_start::Float64` - Starting ψ coordinate for this integration region + - `psi_end::Float64` - Ending ψ coordinate for this integration region + - `needs_crossing::Bool` - Whether a rational surface crossing is needed after this chunk + - `ising::Int` - Index of the singular surface associated with this chunk (0 if none) + - `direction::Int` - Integration direction: +1 forward (axis→edge), -1 backward (edge→axis). + For `direction=-1` chunks, `psi_start` < `psi_end` but integration proceeds from `psi_end` + toward `psi_start`. The resulting propagator maps state at `psi_end` → state at `psi_start`. + Used in bidirectional parallel FM to produce well-conditioned crossing-chunk propagators: + solutions that grow exponentially forward (toward a singularity) decay when integrated + backward, so the backward propagator is well-conditioned. +""" +@kwdef struct IntegrationChunk + psi_start::Float64 + psi_end::Float64 + needs_crossing::Bool + ising::Int = 0 + direction::Int = 1 # +1 forward, -1 backward +end + +""" + ChunkPropagator + +Fundamental matrix for one integration chunk, stored as two N×N×2 solution blocks. +Represents the propagator Φ(ψ₂,ψ₁) computed by integrating the EL ODE from two +identity-block initial conditions: + + - `block_upper_ic`: result of integrating with IC = (I_N, 0_N) (U₁ = I, U₂ = 0) + - `block_lower_ic`: result of integrating with IC = (0_N, I_N) (U₁ = 0, U₂ = I) + +Applying the propagator to the current state `u_prev`: + +u₁_new = block_upper_ic[:,:,1] · u₁_prev + block_lower_ic[:,:,1] · u₂_prev +u₂_new = block_upper_ic[:,:,2] · u₁_prev + block_lower_ic[:,:,2] · u₂_prev + +Since each chunk starts from a bounded identity IC (rather than the accumulated state), +exponential growth within a chunk does not affect the conditioning of the overall +assembly. This enables `Threads.@threads` parallel integration across all chunks. +""" +struct ChunkPropagator + block_upper_ic::Array{ComplexF64,3} # shape (N, N, 2) — result from IC = (I, 0) + block_lower_ic::Array{ComplexF64,3} # shape (N, N, 2) — result from IC = (0, I) +end +ChunkPropagator(N::Int) = ChunkPropagator(zeros(ComplexF64, N, N, 2), zeros(ComplexF64, N, N, 2)) diff --git a/src/ForceFreeStates/Sing.jl b/src/ForceFreeStates/Surfaces/Asymptotics.jl similarity index 62% rename from src/ForceFreeStates/Sing.jl rename to src/ForceFreeStates/Surfaces/Asymptotics.jl index 960530f6a..f9b37f19f 100644 --- a/src/ForceFreeStates/Sing.jl +++ b/src/ForceFreeStates/Surfaces/Asymptotics.jl @@ -1,196 +1,4 @@ -""" - _find_rational_surfaces(equil::Equilibrium.PlasmaEquilibrium, nlow::Int, nhigh::Int) - -Locate all rational q-surfaces q = m/n for n in `nlow:nhigh` by Brent bisection between -consecutive extrema of the q-profile (reverse shear gives one root per monotone segment). -Returns a vector of `(m, n, psifac)` named tuples in discovery order (n outer, ψ-interval -inner). Requires `equilibrium_qfind!` to have populated `equil.params.qextrema_*`. -""" -function _find_rational_surfaces(equil::Equilibrium.PlasmaEquilibrium, nlow::Int, nhigh::Int) - profiles = equil.profiles - surfaces = @NamedTuple{m::Int, n::Int, psifac::Float64}[] - - # Loop over all toroidal mode numbers - for n in nlow:nhigh - hint = Ref(1) - # Loop over extrema of q, find all rational values in between - for iex in 2:equil.params.mextrema - dq = equil.params.qextrema_q[iex] - equil.params.qextrema_q[iex-1] - m = trunc(Int, n * equil.params.qextrema_q[iex-1]) - if dq > 0 - m += 1 - end - dm = Int(sign(dq * n)) - - # Loop over possible m's in interval - while (m - n * equil.params.qextrema_q[iex-1]) * (m - n * equil.params.qextrema_q[iex]) <= 0 - psi0 = equil.params.qextrema_psi[iex-1] - psi1 = equil.params.qextrema_psi[iex] - - psifac = find_zero(psi -> m - n * profiles.q_spline(psi; hint=hint), (psi0, psi1), Roots.Brent()) - push!(surfaces, (m=m, n=n, psifac=psifac)) - m += dm - end - end - end - return surfaces -end - -""" - rational_psi_nodes(equil::Equilibrium.PlasmaEquilibrium; nlow::Int, nhigh::Int=nlow) - -Unique ψ_N locations of all rational surfaces q = m/n for n in `nlow:nhigh`, sorted -increasing. Used as mandatory knots for the two-pass equilibrium grid refinement (the -same physical surface reached through several (m, n) pairs is deduplicated by q value). -""" -function rational_psi_nodes(equil::Equilibrium.PlasmaEquilibrium; nlow::Int, nhigh::Int=nlow) - surfaces = _find_rational_surfaces(equil, nlow, nhigh) - nodes = Float64[] - qs = Float64[] - for s in surfaces - any(q -> isapprox(q, s.m / s.n; atol=1e-8), qs) && continue - push!(qs, s.m / s.n) - push!(nodes, s.psifac) - end - return sort!(nodes) -end - -""" - sing_find!(intr::ForceFreeStatesInternal, equil::Equilibrium.PlasmaEquilibrium) - -Locate singular rational q-surfaces (q = m/nn) using a bisection method -between extrema of the q-profile, and store their properties in `intr.sing`. -Performs the same function as `sing_find` in the Fortran code. -""" -function sing_find!(intr::ForceFreeStatesInternal, equil::Equilibrium.PlasmaEquilibrium) - profiles = equil.profiles - hint = Ref(1) - - for s in _find_rational_surfaces(equil, intr.nlow, intr.nhigh) - m, n, psifac = s.m, s.n, s.psifac - if any(sg -> isapprox(sg.q, m / n; atol=1e-8), intr.sing) - # Rational surface with multiplicity > 1, add this m,n to the resonant mode numbers - # Technically only need m or n, but simplifies some later code and cheap to store both - idx = findfirst(sg -> isapprox(sg.q, m / n; atol=1e-8), intr.sing) - push!(intr.sing[idx].m, m) - push!(intr.sing[idx].n, n) - else - push!(intr.sing, SingType(; - m=[m], - n=[n], - psifac=psifac, - rho=sqrt(psifac), - q=m / n, - q1=profiles.q_deriv(psifac; hint=hint) - )) - intr.msing += 1 - end - end - # Sort singular surfaces by increasing ψ - intr.sing = sort(intr.sing; by=s -> s.psifac) -end - -""" - sing_lim!(intr::ForceFreeStatesInternal, ctrl::ForceFreeStatesControl, equil::Equilibrium.PlasmaEquilibrium) - -Compute and set integration ψ, q, and q' limits by handling cases where user truncates -before the last singular surface. Performs a similar function to `sing_lim` -in the Fortran code. Main differences include renaming of sas_flag -> set_psilim_via_dmlim, -removing dW edge storage variables since we now store all integration terms in memory, and -simplification of the logic. - -The target value `qlim` is first determined from user-specified control parameters -(`ctrl.qhigh` or `ctrl.dmlim`), subject to the constraint that it does not exceed -`equil.params.qmax`. If `set_psilim_via_dmlim` is true, `qlim` is adjusted to the largest -rational surface such that `nq + dmlim < qmax`. If `qlim < qmax`, a Newton iteration is -performed to find the corresponding `psilim` to integrate to. - -Note that the Newton iteration will be triggered if either `set_psilim_via_dmlim` is true -or `ctrl.qhigh < equil.params.qmax`. Otherwise, the equilibrium edge values are used. -""" -function sing_lim!(intr::ForceFreeStatesInternal, ctrl::ForceFreeStatesControl, equil::Equilibrium.PlasmaEquilibrium) - - profiles = equil.profiles - - # Initial guesses based on equilibrium - intr.qlim = min(equil.params.qmax, ctrl.qhigh) # equilibrium solve only goes up to qmax, so we're capped there - intr.q1lim = profiles.q_deriv(profiles.xs[end]; hint=Ref(profiles.npts_minus_1)) - intr.psilim = equil.params.psihigh_resolved - - # Optionally override qlim based on dmlim (Fortran sas_flag=t equivalent). The cutoff reads - # the *resolved* toroidal range on `intr`, so callers must assign intr.nlow / intr.nhigh - # before calling; an unresolved range is an error rather than a silent change of truncation - # strategy. Multi-n runs are not supported — the "outermost rational + dmlim/n" cutoff depends - # on which n is used — and fall back to qhigh / psihigh truncation with a warning. - if ctrl.set_psilim_via_dmlim && intr.nlow <= 0 - error("sing_lim!: set_psilim_via_dmlim = true requires a resolved toroidal range, but got intr.nlow=$(intr.nlow). " * - "Assign intr.nlow / intr.nhigh (from ctrl.nn_low / ctrl.nn_high) before calling sing_lim!, " * - "or set set_psilim_via_dmlim = false to truncate via qhigh / psihigh instead.") - elseif ctrl.set_psilim_via_dmlim && intr.nlow != intr.nhigh - @warn "set_psilim_via_dmlim = true is ignored for multi-n runs (nn_low=$(intr.nlow), nn_high=$(intr.nhigh)); falling back to qhigh / psihigh truncation." - elseif ctrl.set_psilim_via_dmlim - @info "Setting psilim via dmlim: initial qlim = $(@sprintf("%.3f", intr.qlim)), dmlim = $(@sprintf("%.3f", ctrl.dmlim))" - # Normalize dmlim ∈ [0,1) - dmlim = mod(ctrl.dmlim, 1.0) - intr.qlim = (trunc(Int, intr.nlow * intr.qlim) + dmlim) / intr.nlow - - # Reduce qlim if above qmax - while intr.qlim > equil.params.qmax - intr.qlim -= 1.0 / intr.nlow - end - end - - # If set_psilim_via_dmlim decreased qlim or qhigh < qmax, we need to find the precise psilim via newton iteration - if intr.qlim < equil.params.qmax - # Find nearest ψ index where q ≈ qlim - _, jpsi = findmin(abs.(profiles.q_spline.y .- intr.qlim)) - jpsi = min(jpsi, length(profiles.xs) - 1) - - hint = Ref(jpsi) - intr.psilim = find_zero( - (psi -> profiles.q_spline(psi; hint=hint) - intr.qlim, - psi -> profiles.q_deriv(psi; hint=hint)), - profiles.xs[jpsi], Roots.Newton() - ) - intr.q1lim = profiles.q_deriv(intr.psilim) - end -end - -""" - sing_min!(intr::ForceFreeStatesInternal, ctrl::ForceFreeStatesControl, equil::Equilibrium.PlasmaEquilibrium) - -Set the lower integration bound `intr.psilow`. Port of Fortran RDCON `sing_min` (sing.f): -when `qlow > qmin`, the q < qlow core (including any q ≤ 1 sawtooth/internal-kink surfaces) must be -excluded from the outer-region Galerkin domain — otherwise the Hermite FEM integrates through those -ideal singularities without imposing the ideal constraint, contaminating Δ′ at the innermost kept -surface. A Newton iteration locates ψ where q = qlow; scanning starts from the edge inward for -robustness in reverse-shear cores. When `qlow ≤ qmin` the axis value is kept. -""" -function sing_min!(intr::ForceFreeStatesInternal, ctrl::ForceFreeStatesControl, equil::Equilibrium.PlasmaEquilibrium) - profiles = equil.profiles - intr.psilow = profiles.xs[1] # default: equilibrium axis-side bound - ctrl.qlow > equil.params.qmin || return intr.psilow - - # Scan from the edge inward for the first node with q < qlow (robust for reverse-shear q). - qy = profiles.q_spline.y - jpsi = 1 - for j in (length(profiles.xs)-1):-1:1 - if qy[j] < ctrl.qlow - jpsi = j - break - end - end - - hint = Ref(jpsi) - intr.psilow = find_zero( - (psi -> profiles.q_spline(psi; hint=hint) - ctrl.qlow, - psi -> profiles.q_deriv(psi; hint=hint)), - profiles.xs[jpsi], Roots.Newton() - ) - @info "sing_min: qlow=$(@sprintf("%.3f", ctrl.qlow)) > qmin=$(@sprintf("%.3f", equil.params.qmin)); " * - "raising psilow from $(@sprintf("%.5f", profiles.xs[1])) to $(@sprintf("%.5f", intr.psilow)) (excludes q 0, equil, ffit, intr, psieval, odet.spline_hint, odet.ffit_hint) - return nothing -end - -""" - el_derivatives!(du, u, kinetic, equil, ffit, intr, psieval, spline_hint, ffit_hint) -> q - -Euler-Lagrange (or, when `kinetic` is true, FKG) derivative kernel: writes du₁/dψ and du₂/dψ at -`psieval` into `du` and returns q there. Holds no state of its own — the two hints are the -caller's interval-search accelerators, so concurrent callers just pass their own. -""" -@with_pool pool function el_derivatives!(du::Array{ComplexF64,3}, u::Array{ComplexF64,3}, - kinetic::Bool, equil::Equilibrium.PlasmaEquilibrium, ffit::FourFitVars, - intr::ModeSpace, psieval::Float64, spline_hint::Base.RefValue{Int}, ffit_hint::Base.RefValue{Int}) - - # Allocate temporary arrays from the pool - Npert = intr.numpert_total - - singfac_vec = acquire!(pool, Float64, Npert) - singfac_mat = reshape(singfac_vec, intr.mpert, intr.npert) - - fmat_lower = acquire!(pool, ComplexF64, Npert, Npert) - kmat = similar!(pool, fmat_lower) - gmat = similar!(pool, fmat_lower) - tmp_mat = similar!(pool, fmat_lower) - - fill!(tmp_mat, zero(ComplexF64)) - u1 = @view(u[:, :, 1]) - u2 = @view(u[:, :, 2]) - du1 = @view(du[:, :, 1]) - du2 = @view(du[:, :, 2]) - - # Compute singfac = 1 / (m - nq) - # Use caller-supplied hint for O(1) interval lookup during sequential ODE integration - q = equil.profiles.q_spline(psieval; hint=spline_hint) - singfac_mat .= 1.0 ./ ((intr.mlow:intr.mhigh) .- q .* (intr.nlow:intr.nhigh)') - - if kinetic - # ---- Kinetic path with pre-computed FKG matrices ---- - # Use the caller's hint, not ffit._hint (shared, racy in the parallel BVP) - # Load FKG sub-matrices (note: reusing fmat_lower/kmat/gmat as workspace) - f0mat = similar!(pool, fmat_lower) - pmat_kin = similar!(pool, fmat_lower) - paat_kin = similar!(pool, fmat_lower) - kkmat_kin = similar!(pool, fmat_lower) - kkaat_kin = similar!(pool, fmat_lower) - r1mat_kin = similar!(pool, fmat_lower) - r2mat_kin = similar!(pool, fmat_lower) - r3mat_kin = similar!(pool, fmat_lower) - gaat_kin = similar!(pool, fmat_lower) - - ffit.f0mats(vec(f0mat), psieval; hint=ffit_hint) - ffit.pmats(vec(pmat_kin), psieval; hint=ffit_hint) - ffit.paats(vec(paat_kin), psieval; hint=ffit_hint) - ffit.kkmats(vec(kkmat_kin), psieval; hint=ffit_hint) - ffit.kkaats(vec(kkaat_kin), psieval; hint=ffit_hint) - ffit.r1mats(vec(r1mat_kin), psieval; hint=ffit_hint) - ffit.r2mats(vec(r2mat_kin), psieval; hint=ffit_hint) - ffit.r3mats(vec(r3mat_kin), psieval; hint=ffit_hint) - ffit.gaats(vec(gaat_kin), psieval; hint=ffit_hint) - - # Build singfac-dependent F̄, K̄, K̄†, Ḡ† matrices (Logan 2015 Appendix C, Eqs C.5-C.11): - # F̄(i,j) = q1*f0*q2 - q1*P - P†'*q2 + R1 - # K̄(i,j) = q1*KK + R2 - # K̄†(i,j) = KK†*q2 + R3 - # where q1 = (m₁ - n*q), q2 = (m₂ - n*q) — direct singfac, NOT 1/(m-nq) as in ideal path - singfac_direct = acquire!(pool, Float64, Npert) - singfac_direct_mat = reshape(singfac_direct, intr.mpert, intr.npert) - singfac_direct_mat .= (intr.mlow:intr.mhigh) .- q .* (intr.nlow:intr.nhigh)' - - # Build F, K, K† with singfac (using fmat_lower, kmat, gmat as workspace for F, K, K†) - kaat_kin = similar!(pool, fmat_lower) # K† matrix - for j in 1:Npert - q2 = singfac_direct[j] - for i in 1:Npert - q1 = singfac_direct[i] - fmat_lower[i, j] = q1 * f0mat[i, j] * q2 - q1 * pmat_kin[i, j] - - conj(paat_kin[j, i]) * q2 + r1mat_kin[i, j] - kmat[i, j] = q1 * kkmat_kin[i, j] + r2mat_kin[i, j] - kaat_kin[i, j] = kkaat_kin[i, j] * q2 + r3mat_kin[i, j] - end - end - # gmat = gaat (already loaded) - gmat .= gaat_kin - - # Kinetic ODE (Logan 2015 Eq 7.46): singfac absorbed into F̄/K̄/K̄†, no explicit Q⁻¹ - # du₁ = F̄⁻¹(u₂ - K̄·u₁) - du1 .= u2 - mul!(tmp_mat, kmat, u1) - du1 .-= tmp_mat - # LU factorize F (non-Hermitian, non-symmetric); direct LAPACK for the same hot-loop reason - _, ipiv2, _ = LAPACK.getrf!(fmat_lower) - LAPACK.getrs!('N', fmat_lower, ipiv2, du1) - - # du₂ = Ḡ†·u₁ + K̄†·du₁ (Logan 2015 Eq C.10-C.11) - mul!(tmp_mat, gmat, u1) - du2 .= tmp_mat - mul!(tmp_mat, kaat_kin, du1) - du2 .+= tmp_mat - - else - # ---- Ideal path ---- - # Evaluate matrix splines at the current psi (hint is the caller's, never shared) - ffit.fmats_lower(vec(fmat_lower), psieval; hint=ffit_hint) - ffit.kmats(vec(kmat), psieval; hint=ffit_hint) - ffit.gmats(vec(gmat), psieval; hint=ffit_hint) - - # See equations 22-24 in Glasser 2016 DCON paper for derivation - # du[1] = - F̄⁻¹ * K̄ * u[1] + F̄⁻¹ * Q⁻¹ * u[2] - du1 .= u2 .* singfac_vec - mul!(tmp_mat, kmat, u1) - du1 .-= tmp_mat - ldiv!(LowerTriangular(fmat_lower), du1) - ldiv!(UpperTriangular(fmat_lower'), du1) - # du[2] = G * u[1] + K̄^† * du[1] = G * u[1] - K̄^† * F̄⁻¹ * K̄ * u[1] + K̄^† * F̄⁻¹ * Q⁻¹ * u[2] - mul!(tmp_mat, gmat, u1) - du2 .= tmp_mat - mul!(tmp_mat, adjoint(kmat), du1) - du2 .+= tmp_mat - # du[1] = - Q⁻¹ * F̄⁻¹ * K̄ * u[1] + Q⁻¹ * F̄⁻¹ * Q⁻¹ * u[2] - du1 .*= singfac_vec - end - return q -end - -""" - compute_node_xi_s!(xi_s, du1, u1, ffit, psieval; kinetic=false, hint=Ref(1)) - -Evaluate Ξ_s = -A⁻¹(B·Ξ′_ψ + C·Ξ_ψ) [Glasser Phys. Plasmas 2016 112506 eq. 18] at `psieval`, -writing into `xi_s`. `du1` and `u1` are the Ξ′_ψ and Ξ_ψ blocks at the same ψ, i.e. slices of a -`el_derivatives!` result and its input state. - -Split out of the derivative kernel because Ξ_s is needed only at saved nodes, not at every -Runge-Kutta stage. Ideal runs factor the Hermitian A by Cholesky; with `kinetic=true` A picks up -non-Hermitian contributions and needs an LU. -""" -@with_pool pool function compute_node_xi_s!(xi_s::AbstractMatrix{ComplexF64}, du1::AbstractMatrix{ComplexF64}, - u1::AbstractMatrix{ComplexF64}, ffit::FourFitVars, psieval::Float64; kinetic::Bool=false, hint::Base.RefValue{Int}=Ref(1)) - - Npert = size(u1, 1) - amat = acquire!(pool, ComplexF64, Npert, Npert) - bmat = similar!(pool, amat) - cmat = similar!(pool, amat) - tmp_mat = similar!(pool, amat) - - ffit.amats(vec(amat), psieval; hint=hint) - ffit.bmats(vec(bmat), psieval; hint=hint) - ffit.cmats(vec(cmat), psieval; hint=hint) - - # Solve bmat = A⁻¹ * bmat, cmat = A⁻¹ * cmat in-place - if kinetic - _, ipiv, _ = LAPACK.getrf!(amat) - LAPACK.getrs!('N', amat, ipiv, bmat) - LAPACK.getrs!('N', amat, ipiv, cmat) - else - LAPACK.potrf!('U', amat) - LAPACK.potrs!('U', amat, bmat) - LAPACK.potrs!('U', amat, cmat) - end - - mul!(tmp_mat, bmat, du1) - xi_s .= .-tmp_mat - mul!(tmp_mat, cmat, u1) - xi_s .-= tmp_mat - return xi_s -end - -""" - evaluate_fbar_condition(psi, ffit, equil, intr; hint=Ref(1)) - -Evaluate the condition number of the kinetic F̄ matrix at a given ψ. Uses cond(F̄) -as a scale-invariant measure of near-singularity. Mirrors the intent of Fortran -`sing_get_f_det` (`sing.f:1298-1481`) which computes det(F̄). - -F̄(i,j) = q₁·f0(i,j)·q₂ - q₁·P(i,j) - conj(P†(j,i))·q₂ + R1(i,j) - -where q₁ = m₁ - n·q(ψ), q₂ = m₂ - n·q(ψ) are the direct singularity factors. -""" -function evaluate_fbar_condition(psi::Float64, ffit::FourFitVars, equil::Equilibrium.PlasmaEquilibrium, intr::ForceFreeStatesInternal; hint=Ref(1)) - np = intr.numpert_total - - # Evaluate q(ψ) and compute singfac = m - n*q - q = equil.profiles.q_spline(psi; hint=hint) - singfac = Float64[(m - q * n) for m in intr.mlow:intr.mhigh for n in intr.nlow:intr.nhigh] - - # Evaluate FKG sub-matrices from splines - f0_vec = zeros(ComplexF64, np * np) - p_vec = zeros(ComplexF64, np * np) - pa_vec = zeros(ComplexF64, np * np) - r1_vec = zeros(ComplexF64, np * np) - ffit.f0mats(f0_vec, psi; hint=hint) - ffit.pmats(p_vec, psi; hint=hint) - ffit.paats(pa_vec, psi; hint=hint) - ffit.r1mats(r1_vec, psi; hint=hint) - f0mat = reshape(f0_vec, np, np) - pmat = reshape(p_vec, np, np) - paat = reshape(pa_vec, np, np) - r1mat = reshape(r1_vec, np, np) - - # Assemble F̄ [Fortran sing.f lines 1412-1423, sing_get_f_det with fkg_kmats_flag=true] - fbar = zeros(ComplexF64, np, np) - for j in 1:np - q2 = singfac[j] - for i in 1:np - q1 = singfac[i] - fbar[i, j] = q1 * f0mat[i, j] * q2 - q1 * pmat[i, j] - conj(paat[j, i]) * q2 + r1mat[i, j] - end - end - - return cond(fbar) -end - -""" - find_kinetic_singular_surfaces!(ffit, equil, intr; ngrid=2000, cond_threshold=1e8) - -Find kinetically-displaced singular surfaces — locations where cond(F̄) peaks, -indicating near-singularity of the kinetic F̄ matrix in the ODE RHS. Populates -`intr.kinsing` and `intr.kmsing`. - -Mirrors the intent of Fortran `ksing_find` (`sing.f:1486-1616`) which finds zeros of -det(F̄) via adaptive bisection. Here we use condition number peaks instead of -determinant zeros for better numerical robustness and scale invariance. - -Algorithm: - - 1. Evaluate cond(F̄) on a dense ψ grid - 2. Find local maxima (peaks where gradient changes from + to -) - 3. Refine each peak with golden-section minimization of -cond - 4. Filter by threshold and resonance condition -""" -function find_kinetic_singular_surfaces!(ffit::FourFitVars, equil::Equilibrium.PlasmaEquilibrium, intr::ForceFreeStatesInternal; ngrid::Int=2000, cond_threshold::Float64=1e8) - psilow = equil.profiles.xs[1] - psihigh = intr.psilim - - # Evaluate cond(F̄) on a dense grid - psi_grid = collect(range(psilow, psihigh; length=ngrid)) - cond_vals = zeros(ngrid) - hint = Ref(1) - for i in 1:ngrid - try - cond_vals[i] = evaluate_fbar_condition(psi_grid[i], ffit, equil, intr; hint=hint) - catch - cond_vals[i] = Inf # singular matrix — definitely a kinsing surface - end - end - - # Persist the scan so callers/HDF5 output can plot cond(F̄) vs ψ and show why peaks - # were (or weren't) accepted as kinetic singular surfaces. - intr.kinsing_scan_psi = psi_grid - intr.kinsing_scan_cond = cond_vals - intr.kinsing_scan_threshold = cond_threshold - - # Find local maxima of cond(F̄): points where cond increases then decreases - peak_indices = Int[] - for i in 2:(ngrid-1) - if cond_vals[i] > cond_vals[i-1] && cond_vals[i] > cond_vals[i+1] && cond_vals[i] > cond_threshold - push!(peak_indices, i) - end - end - - # Refine each peak to find the precise ψ location - kinsing_surfaces = SingType[] - for idx in peak_indices - psi_lo = psi_grid[max(idx - 1, 1)] - psi_hi = psi_grid[min(idx + 1, ngrid)] - - # Golden-section search to maximize cond (minimize -cond) - psi_refined = _golden_section_max(psi_lo, psi_hi, psi -> evaluate_fbar_condition(psi, ffit, equil, intr)) - - # Evaluate q and q' at refined location - hint_ref = Ref(1) - q_val = equil.profiles.q_spline(psi_refined; hint=hint_ref) - q1_val = equil.profiles.q_deriv(psi_refined; hint=hint_ref) - - # Check resonance: at least one mode m satisfies mlow ≤ n*q ≤ mhigh - has_resonant = false - for n in intr.nlow:intr.nhigh - nq = n * q_val - if intr.mlow <= nq && nq <= intr.mhigh - has_resonant = true - break - end - end - if !has_resonant - continue - end - - push!( - kinsing_surfaces, - SingType(; - psifac=psi_refined, - rho=sqrt(psi_refined), - m=[round(Int, n * q_val) for n in intr.nlow:intr.nhigh], - n=collect(intr.nlow:intr.nhigh), - q=q_val, - q1=q1_val - ) - ) - end - - # Sort by ψ location - sort!(kinsing_surfaces; by=s -> s.psifac) - - intr.kinsing = kinsing_surfaces - intr.kmsing = length(kinsing_surfaces) - - if intr.kmsing > 0 - @info "Found $(intr.kmsing) kinetic singular surface(s):" - for (i, ks) in enumerate(intr.kinsing) - @info @sprintf(" kinsing[%d]: ψ = %.6f, q = %.4f", i, ks.psifac, ks.q) - end - else - @info "No kinetic singular surfaces found (cond threshold = $(cond_threshold))" - end - - return nothing -end - -""" -Golden-section search to find the ψ that maximizes f(ψ) on [a, b]. -""" -function _golden_section_max(a::Float64, b::Float64, f::Function; tol::Float64=1e-10) - gr = (sqrt(5) + 1) / 2 - c = b - (b - a) / gr - d = a + (b - a) / gr - for _ in 1:100 - if abs(b - a) < tol - break - end - if f(c) > f(d) - b = d - else - a = c - end - c = b - (b - a) / gr - d = a + (b - a) / gr - end - return (a + b) / 2 -end diff --git a/src/ForceFreeStates/Surfaces/Finding.jl b/src/ForceFreeStates/Surfaces/Finding.jl new file mode 100644 index 000000000..1d2d7d3a9 --- /dev/null +++ b/src/ForceFreeStates/Surfaces/Finding.jl @@ -0,0 +1,368 @@ +# Rational-surface finding: ideal (q = m/n) and kinetic (cond(F) peak) singular surfaces. + +""" + _find_rational_surfaces(equil::Equilibrium.PlasmaEquilibrium, nlow::Int, nhigh::Int) + +Locate all rational q-surfaces q = m/n for n in `nlow:nhigh` by Brent bisection between +consecutive extrema of the q-profile (reverse shear gives one root per monotone segment). +Returns a vector of `(m, n, psifac)` named tuples in discovery order (n outer, ψ-interval +inner). Requires `equilibrium_qfind!` to have populated `equil.params.qextrema_*`. +""" +function _find_rational_surfaces(equil::Equilibrium.PlasmaEquilibrium, nlow::Int, nhigh::Int) + profiles = equil.profiles + surfaces = @NamedTuple{m::Int, n::Int, psifac::Float64}[] + + # Loop over all toroidal mode numbers + for n in nlow:nhigh + hint = Ref(1) + # Loop over extrema of q, find all rational values in between + for iex in 2:equil.params.mextrema + dq = equil.params.qextrema_q[iex] - equil.params.qextrema_q[iex-1] + m = trunc(Int, n * equil.params.qextrema_q[iex-1]) + if dq > 0 + m += 1 + end + dm = Int(sign(dq * n)) + + # Loop over possible m's in interval + while (m - n * equil.params.qextrema_q[iex-1]) * (m - n * equil.params.qextrema_q[iex]) <= 0 + psi0 = equil.params.qextrema_psi[iex-1] + psi1 = equil.params.qextrema_psi[iex] + + psifac = find_zero(psi -> m - n * profiles.q_spline(psi; hint=hint), (psi0, psi1), Roots.Brent()) + push!(surfaces, (m=m, n=n, psifac=psifac)) + m += dm + end + end + end + return surfaces +end + +""" + rational_psi_nodes(equil::Equilibrium.PlasmaEquilibrium; nlow::Int, nhigh::Int=nlow) + +Unique ψ_N locations of all rational surfaces q = m/n for n in `nlow:nhigh`, sorted +increasing. Used as mandatory knots for the two-pass equilibrium grid refinement (the +same physical surface reached through several (m, n) pairs is deduplicated by q value). +""" +function rational_psi_nodes(equil::Equilibrium.PlasmaEquilibrium; nlow::Int, nhigh::Int=nlow) + surfaces = _find_rational_surfaces(equil, nlow, nhigh) + nodes = Float64[] + qs = Float64[] + for s in surfaces + any(q -> isapprox(q, s.m / s.n; atol=1e-8), qs) && continue + push!(qs, s.m / s.n) + push!(nodes, s.psifac) + end + return sort!(nodes) +end + +""" + sing_find!(intr::ForceFreeStatesInternal, equil::Equilibrium.PlasmaEquilibrium) + +Locate singular rational q-surfaces (q = m/nn) using a bisection method +between extrema of the q-profile, and store their properties in `intr.sing`. +Performs the same function as `sing_find` in the Fortran code. +""" +function sing_find!(intr::ForceFreeStatesInternal, equil::Equilibrium.PlasmaEquilibrium) + profiles = equil.profiles + hint = Ref(1) + + for s in _find_rational_surfaces(equil, intr.nlow, intr.nhigh) + m, n, psifac = s.m, s.n, s.psifac + if any(sg -> isapprox(sg.q, m / n; atol=1e-8), intr.sing) + # Rational surface with multiplicity > 1, add this m,n to the resonant mode numbers + # Technically only need m or n, but simplifies some later code and cheap to store both + idx = findfirst(sg -> isapprox(sg.q, m / n; atol=1e-8), intr.sing) + push!(intr.sing[idx].m, m) + push!(intr.sing[idx].n, n) + else + push!(intr.sing, SingType(; + m=[m], + n=[n], + psifac=psifac, + rho=sqrt(psifac), + q=m / n, + q1=profiles.q_deriv(psifac; hint=hint) + )) + intr.msing += 1 + end + end + # Sort singular surfaces by increasing ψ + intr.sing = sort(intr.sing; by=s -> s.psifac) +end + +""" + sing_lim!(intr::ForceFreeStatesInternal, ctrl::ForceFreeStatesControl, equil::Equilibrium.PlasmaEquilibrium) + +Compute and set integration ψ, q, and q' limits by handling cases where user truncates +before the last singular surface. Performs a similar function to `sing_lim` +in the Fortran code. Main differences include renaming of sas_flag -> set_psilim_via_dmlim, +removing dW edge storage variables since we now store all integration terms in memory, and +simplification of the logic. + +The target value `qlim` is first determined from user-specified control parameters +(`ctrl.qhigh` or `ctrl.dmlim`), subject to the constraint that it does not exceed +`equil.params.qmax`. If `set_psilim_via_dmlim` is true, `qlim` is adjusted to the largest +rational surface such that `nq + dmlim < qmax`. If `qlim < qmax`, a Newton iteration is +performed to find the corresponding `psilim` to integrate to. + +Note that the Newton iteration will be triggered if either `set_psilim_via_dmlim` is true +or `ctrl.qhigh < equil.params.qmax`. Otherwise, the equilibrium edge values are used. +""" +function sing_lim!(intr::ForceFreeStatesInternal, ctrl::ForceFreeStatesControl, equil::Equilibrium.PlasmaEquilibrium) + + profiles = equil.profiles + + # Initial guesses based on equilibrium + intr.qlim = min(equil.params.qmax, ctrl.qhigh) # equilibrium solve only goes up to qmax, so we're capped there + intr.q1lim = profiles.q_deriv(profiles.xs[end]; hint=Ref(profiles.npts_minus_1)) + intr.psilim = equil.params.psihigh_resolved + + # Optionally override qlim based on dmlim (Fortran sas_flag=t equivalent). The cutoff reads + # the *resolved* toroidal range on `intr`, so callers must assign intr.nlow / intr.nhigh + # before calling; an unresolved range is an error rather than a silent change of truncation + # strategy. Multi-n runs are not supported — the "outermost rational + dmlim/n" cutoff depends + # on which n is used — and fall back to qhigh / psihigh truncation with a warning. + if ctrl.set_psilim_via_dmlim && intr.nlow <= 0 + error("sing_lim!: set_psilim_via_dmlim = true requires a resolved toroidal range, but got intr.nlow=$(intr.nlow). " * + "Assign intr.nlow / intr.nhigh (from ctrl.nn_low / ctrl.nn_high) before calling sing_lim!, " * + "or set set_psilim_via_dmlim = false to truncate via qhigh / psihigh instead.") + elseif ctrl.set_psilim_via_dmlim && intr.nlow != intr.nhigh + @warn "set_psilim_via_dmlim = true is ignored for multi-n runs (nn_low=$(intr.nlow), nn_high=$(intr.nhigh)); falling back to qhigh / psihigh truncation." + elseif ctrl.set_psilim_via_dmlim + @info "Setting psilim via dmlim: initial qlim = $(@sprintf("%.3f", intr.qlim)), dmlim = $(@sprintf("%.3f", ctrl.dmlim))" + # Normalize dmlim ∈ [0,1) + dmlim = mod(ctrl.dmlim, 1.0) + intr.qlim = (trunc(Int, intr.nlow * intr.qlim) + dmlim) / intr.nlow + + # Reduce qlim if above qmax + while intr.qlim > equil.params.qmax + intr.qlim -= 1.0 / intr.nlow + end + end + + # If set_psilim_via_dmlim decreased qlim or qhigh < qmax, we need to find the precise psilim via newton iteration + if intr.qlim < equil.params.qmax + # Find nearest ψ index where q ≈ qlim + _, jpsi = findmin(abs.(profiles.q_spline.y .- intr.qlim)) + jpsi = min(jpsi, length(profiles.xs) - 1) + + hint = Ref(jpsi) + intr.psilim = find_zero( + (psi -> profiles.q_spline(psi; hint=hint) - intr.qlim, + psi -> profiles.q_deriv(psi; hint=hint)), + profiles.xs[jpsi], Roots.Newton() + ) + intr.q1lim = profiles.q_deriv(intr.psilim) + end +end + +""" + sing_min!(intr::ForceFreeStatesInternal, ctrl::ForceFreeStatesControl, equil::Equilibrium.PlasmaEquilibrium) + +Set the lower integration bound `intr.psilow`. Port of Fortran RDCON `sing_min` (sing.f): +when `qlow > qmin`, the q < qlow core (including any q ≤ 1 sawtooth/internal-kink surfaces) must be +excluded from the outer-region Galerkin domain — otherwise the Hermite FEM integrates through those +ideal singularities without imposing the ideal constraint, contaminating Δ′ at the innermost kept +surface. A Newton iteration locates ψ where q = qlow; scanning starts from the edge inward for +robustness in reverse-shear cores. When `qlow ≤ qmin` the axis value is kept. +""" +function sing_min!(intr::ForceFreeStatesInternal, ctrl::ForceFreeStatesControl, equil::Equilibrium.PlasmaEquilibrium) + profiles = equil.profiles + intr.psilow = profiles.xs[1] # default: equilibrium axis-side bound + ctrl.qlow > equil.params.qmin || return intr.psilow + + # Scan from the edge inward for the first node with q < qlow (robust for reverse-shear q). + qy = profiles.q_spline.y + jpsi = 1 + for j in (length(profiles.xs)-1):-1:1 + if qy[j] < ctrl.qlow + jpsi = j + break + end + end + + hint = Ref(jpsi) + intr.psilow = find_zero( + (psi -> profiles.q_spline(psi; hint=hint) - ctrl.qlow, + psi -> profiles.q_deriv(psi; hint=hint)), + profiles.xs[jpsi], Roots.Newton() + ) + @info "sing_min: qlow=$(@sprintf("%.3f", ctrl.qlow)) > qmin=$(@sprintf("%.3f", equil.params.qmin)); " * + "raising psilow from $(@sprintf("%.5f", profiles.xs[1])) to $(@sprintf("%.5f", intr.psilow)) (excludes q cond_vals[i-1] && cond_vals[i] > cond_vals[i+1] && cond_vals[i] > cond_threshold + push!(peak_indices, i) + end + end + + # Refine each peak to find the precise ψ location + kinsing_surfaces = SingType[] + for idx in peak_indices + psi_lo = psi_grid[max(idx - 1, 1)] + psi_hi = psi_grid[min(idx + 1, ngrid)] + + # Golden-section search to maximize cond (minimize -cond) + psi_refined = _golden_section_max(psi_lo, psi_hi, psi -> evaluate_fbar_condition(psi, ffit, equil, intr)) + + # Evaluate q and q' at refined location + hint_ref = Ref(1) + q_val = equil.profiles.q_spline(psi_refined; hint=hint_ref) + q1_val = equil.profiles.q_deriv(psi_refined; hint=hint_ref) + + # Check resonance: at least one mode m satisfies mlow ≤ n*q ≤ mhigh + has_resonant = false + for n in intr.nlow:intr.nhigh + nq = n * q_val + if intr.mlow <= nq && nq <= intr.mhigh + has_resonant = true + break + end + end + if !has_resonant + continue + end + + push!( + kinsing_surfaces, + SingType(; + psifac=psi_refined, + rho=sqrt(psi_refined), + m=[round(Int, n * q_val) for n in intr.nlow:intr.nhigh], + n=collect(intr.nlow:intr.nhigh), + q=q_val, + q1=q1_val + ) + ) + end + + # Sort by ψ location + sort!(kinsing_surfaces; by=s -> s.psifac) + + intr.kinsing = kinsing_surfaces + intr.kmsing = length(kinsing_surfaces) + + if intr.kmsing > 0 + @info "Found $(intr.kmsing) kinetic singular surface(s):" + for (i, ks) in enumerate(intr.kinsing) + @info @sprintf(" kinsing[%d]: ψ = %.6f, q = %.4f", i, ks.psifac, ks.q) + end + else + @info "No kinetic singular surfaces found (cond threshold = $(cond_threshold))" + end + + return nothing +end + +""" +Golden-section search to find the ψ that maximizes f(ψ) on [a, b]. +""" +function _golden_section_max(a::Float64, b::Float64, f::Function; tol::Float64=1e-10) + gr = (sqrt(5) + 1) / 2 + c = b - (b - a) / gr + d = a + (b - a) / gr + for _ in 1:100 + if abs(b - a) < tol + break + end + if f(c) > f(d) + b = d + else + a = c + end + c = b - (b - a) / gr + d = a + (b - a) / gr + end + return (a + b) / 2 +end diff --git a/src/ForceFreeStates/Resist.jl b/src/ForceFreeStates/Surfaces/Resist.jl similarity index 58% rename from src/ForceFreeStates/Resist.jl rename to src/ForceFreeStates/Surfaces/Resist.jl index 3c436daa2..ecfbcd0f8 100644 --- a/src/ForceFreeStates/Resist.jl +++ b/src/ForceFreeStates/Surfaces/Resist.jl @@ -122,84 +122,3 @@ function resist_eval(sing::SingType, equil::Equilibrium.PlasmaEquilibrium, return InnerLayer.GGJParameters(; E=E, F=F, G=G, H=H, K=K, M=M, taua=taua, taur=taur, v1=v1norm, ising=ising) end - -# Outer<->inner resistive match, Wang et al. 2020 (PoP 27, 122509) Eq. 11: -# C = -(Δ_out - Δ_in(i2πf))^{-1} Δ_coil -# Raw STRIDE outer Δ' + raw coil drive matched to the GGJ inner layer (resist_eval -> solve_inner). -struct ResonantMatchResult - cout::Matrix{ComplexF64} # outer coeffs (2msing × ncoil) - cin::Matrix{ComplexF64} # inner coeffs (2msing × ncoil) - deltar::Matrix{ComplexF64} # inner-layer Δ per surface (msing × 2) - rpec_eig::Vector{ComplexF64} # forced eigenvalue γ_s = 2πi·n·f - reconnected_flux::Matrix{ComplexF64} # reconnected resonant flux (2msing × ncoil) - bpen::Matrix{ComplexF64} # area-weighted penetrated field (msing × ncoil); empty until Stage 2 - residual::Float64 -end - -function resonant_match_rpec(delta_out_raw::AbstractMatrix, delta_coil_raw::AbstractMatrix, - sings::Vector{SingType}, equil::Equilibrium.PlasmaEquilibrium, - intr::ForceFreeStatesInternal, ctrl::ForceFreeStatesControl) - - msing = size(delta_out_raw, 1) ÷ 2 - ncoil = size(delta_coil_raw, 2) - nn = intr.nlow - empty_bpen = Matrix{ComplexF64}(undef, 0, 0) - - size(delta_out_raw) == (2msing, 2msing) || error("delta_out_raw $(size(delta_out_raw)) != (2msing,2msing)") - length(sings) == msing || error("sings $(length(sings)) != msing $msing") - size(delta_coil_raw, 1) == 2msing || error("delta_coil_raw rows $(size(delta_coil_raw,1)) != 2msing") - - if ctrl.gal_ideal_flag # ideal limit: no inner layer, no reconnection - return ResonantMatchResult(zeros(ComplexF64,2msing,ncoil), zeros(ComplexF64,2msing,ncoil), - zeros(ComplexF64,msing,2), zeros(ComplexF64,msing), Matrix{ComplexF64}(delta_coil_raw), empty_bpen, 0.0) - end - for (nm,v) in (("gal_eta",ctrl.gal_eta),("gal_rho",ctrl.gal_rho),("gal_rotation",ctrl.gal_rotation)) - length(v) == msing || error("$nm length $(length(v)) != msing $msing") - end - - chi1 = 2π * equil.psio - deltar = zeros(ComplexF64, msing, 2) - rpec_eig = zeros(ComplexF64, msing) - # Layer-center (X=0) penetrated-field weights pen[i,k] = scale·Ψ_k(0)·rescale (match.f intotsol_b); - # solve_inner_profile returns the same Δ as solve_inner plus the inner-layer field needed for pen. - pen = zeros(ComplexF64, msing, 2) - for i in 1:msing - params = resist_eval(sings[i], equil, intr; eta=ctrl.gal_eta[i], rho=ctrl.gal_rho[i], gamma=ctrl.gal_gamma, ising=i) - γ = 2π*im*nn*ctrl.gal_rotation[i] - rpec_eig[i] = γ - inner = InnerLayer.solve_inner_profile(InnerLayer.GGJModel(; solver=:galerkin), params, γ; - xfac=ctrl.gal_inner_xfac, nx=ctrl.gal_inner_nx, nq=ctrl.gal_inner_nq, cutoff=ctrl.gal_inner_cutoff, kmax=ctrl.gal_inner_kmax) - deltar[i,1] = inner.Δ[1]; deltar[i,2] = inner.Δ[2] - scale = -2π * chi1 * im * nn * sings[i].q1 * inner.dψdx # b_m = −2πi·χ₁·n·q′·dψdx·rescale·Ψ (GalerkinMatch.jl) - pen[i,1] = scale * inner.Ψ[1,1] * inner.rescale # parity 1 (Ψ(0)≠0) - pen[i,2] = scale * inner.Ψ[1,2] * inner.rescale # parity 2 (Ψ(0)=0 ⇒ ~0) - end - - mat = zeros(ComplexF64, 4msing, 4msing) - rmat = zeros(ComplexF64, 4msing, ncoil) - @views mat[2msing+1:4msing, 1:2msing] .= transpose(delta_out_raw) - @views rmat[2msing+1:4msing, :] .= .-delta_coil_raw - for i in 1:msing - a=2i-1; b=2i; c=a+2msing; d=b+2msing - d1=deltar[i,1]; d2=deltar[i,2] - mat[a,a]=1; mat[b,b]=1 - mat[a,c]=-1; mat[a,d]=1 - mat[b,c]=-1; mat[b,d]=-1 - mat[c,c]=-d1; mat[c,d]=d2 - mat[d,c]=-d1; mat[d,d]=-d2 - end - - cof = mat \ rmat - residual = norm(mat*cof - rmat) / max(norm(rmat), 1e-300) - cout = cof[1:2msing, :] - cin = cof[2msing+1:4msing, :] - - reconnected_flux = delta_coil_raw .+ transpose(delta_out_raw)*cout - # Inner-layer penetrated (reconnected) resonant field per surface — ONE quantity per surface, read off - # the inner solution at the layer center (match.f intotsol_b; GalerkinMatch.jl): bpen[i,j] = pen₁(i)·cin[2i,j] + pen₂(i)·cin[2i-1,j]. - bpen = zeros(ComplexF64, msing, ncoil) - for i in 1:msing, j in 1:ncoil - bpen[i,j] = pen[i,1]*cin[2i,j] + pen[i,2]*cin[2i-1,j] - end - return ResonantMatchResult(cout, cin, deltar, rpec_eig, reconnected_flux, bpen, residual) -end \ No newline at end of file diff --git a/src/ForceFreeStates/ResistEval.jl b/src/ForceFreeStates/Surfaces/ResistEval.jl similarity index 100% rename from src/ForceFreeStates/ResistEval.jl rename to src/ForceFreeStates/Surfaces/ResistEval.jl diff --git a/src/ForceFreeStates/Surfaces/Types.jl b/src/ForceFreeStates/Surfaces/Types.jl new file mode 100644 index 000000000..5f995d2a0 --- /dev/null +++ b/src/ForceFreeStates/Surfaces/Types.jl @@ -0,0 +1,64 @@ +# Singular-surface data types shared across subsystems. + +""" + SingType + +A mutable struct holding data related to the singular surfaces in the equilibrium. + +## Fields + + - `psifac::Float64` - Normalized flux coordinate at the singular surface + - `rho::Float64` - Radial coordinate (√ψ) + - `m::Vector{Int}` - Poloidal mode number(s) + - `n::Vector{Int}` - Toroidal mode number(s) + - `q::Float64` - Safety factor (= m/n) + - `q1::Float64` - Derivative of safety factor with respect to ψ + - `delta_prime::Vector{ComplexF64}` - **STUB (not physically valid)**. Per-surface ca-based Δ' estimate retained for future work / debugging only. The physically valid Δ' is `ForceFreeStatesInternal.delta_prime_matrix`, computed via the STRIDE global BVP (Glasser 2018 PoP 25, 032501). Do not use this field for tearing-stability analysis; do not expect agreement with `delta_prime_matrix`. + - `delta_prime_col::Matrix{ComplexF64}` - **STUB (not physically valid)**. Per-surface ca-based Δ' column retained for future work / debugging only. Shape (numpert_total × n_res_modes); `delta_prime_col[j, i] = (ca_r[j,ipert_res_i,2] - ca_l[j,ipert_res_i,2]) / (4π²·psio)`. The diagonal element matches the (also stubbed) `delta_prime[i]`. Only populated for the Riccati/parallel FM paths. The physically valid Δ' is `ForceFreeStatesInternal.delta_prime_matrix`; this field exists for future development on intra-surface coupling diagnostics, not for production use. +""" +@kwdef mutable struct SingType + psifac::Float64 = 0.0 + rho::Float64 = 0.0 + m::Vector{Int} = Int[] + n::Vector{Int} = Int[] + q::Float64 = 0.0 + q1::Float64 = 0.0 + delta_prime::Vector{ComplexF64} = ComplexF64[] + delta_prime_col::Matrix{ComplexF64} = Matrix{ComplexF64}(undef, 0, 0) + ua_left::Array{ComplexF64,3} = Array{ComplexF64}(undef, 0, 0, 0) # asymptotic basis at left inner-layer boundary + ua_right::Array{ComplexF64,3} = Array{ComplexF64}(undef, 0, 0, 0) # asymptotic basis at right inner-layer boundary + psi_ua_left::Float64 = 0.0 # ψ where ua_left was evaluated (left inner-layer boundary) + psi_ua_right::Float64 = 0.0 # ψ where ua_right was evaluated (right inner-layer boundary) + restype::Any = nothing # ResistGeometry from ResistEval.jl (populated by resist_eval_all!); typed `Any` to avoid a cross-file type reference +end + +""" + SingAsymptotics + +A struct containing asymptotic expansion data for ideal ForceFreeStates calculations at a singular surface. +This data is computed on-demand during singular surface crossings in `cross_ideal_singular_surf!`. + +## Fields + + - `alpha::Vector{ComplexF64}` - Resonant matrix eigenvalues + - `r1::Vector{Int}` - Resonant indices along first index + - `r2::Vector{Int}` - Resonant indices along second index + - `n1::Vector{Int}` - Nonresonant indices along first index + - `n2::Vector{Int}` - Nonresonant indices along second index + - `power::Vector{ComplexF64}` - Power series coefficients + - `vmat::Array{ComplexF64,4}` - Power series of V matrix for asymptotic analysis + - `mmat::Array{ComplexF64,4}` - Power series of M matrix for asymptotic analysis + - `m0mat::Matrix{ComplexF64}` - Zeroth order M matrix projected onto resonant subspace +""" +struct SingAsymptotics + sing_order::Int + alpha::Vector{ComplexF64} + r1::Vector{Int} + r2::Vector{Int} + n1::Vector{Int} + n2::Vector{Int} + power::Vector{ComplexF64} + vmat::Array{ComplexF64,4} + mmat::Array{ComplexF64,4} + m0mat::Matrix{ComplexF64} +end