Symbolics 7 compatibility breaks extension loading and OUQ variable discovery

[ChrisRackauckas-Claude]

Summary

main at 246141f8f176de2ad712709486533c84f03b3e5a is not compatible with the Symbolics 7 / SymbolicUtils 4 versions admitted by #40. There are two independent stages:

1. CanonicalMoments.SymbolicsExt cannot precompile because it imports the removed, non-public Symbolics.array_term, and its @register_array_symbolic signature uses the AbstractVecOrMat union that Symbolics 7's wrapper expansion cannot process.
2. After a public-API-only prototype clears precompilation and the OUQ operator promotion ambiguity, SymbolicUtils 4's documented operator-boundary behavior makes get_variables(𝔼(Q)) return 𝔼(Q) instead of Q. OUQBase then cannot associate constraints with random-variable groups.

This therefore needs a broader Symbolics 7 migration, including a deliberate variable-discovery design for OUQ operators, rather than only a one-line compatibility fix.

CI and dependency boundary

The failures began at merge commit 246141f8, PR #40, which added Symbolics 7.30 and SymbolicUtils 4.38 compatibility. The initial push failures are visible in:

• umbrella CI, Julia 1
• OUQBase, Julia 1
• OUQBase, Julia 1.10 LTS
• OUQBase, Julia 1.13.0-rc1

The Julia 1 job resolved Julia 1.12.6, Symbolics 7.31.0, and SymbolicUtils 4.39.0. A clean local reproduction on current compatible releases resolved Symbolics 7.32.0 / SymbolicUtils 4.40.0 and failed identically, so this is still current.

Symbolics.array_term existed in Symbolics 6.39.1 as an unexported/non-public helper. It was removed by Symbolics commit 7b59796, with v7.0.0 the first tag containing the removal. Thus the source boundary is Symbolics 6 to 7, not specifically 7.31.

Stage 1: extension and operator construction failures

On clean main:

timeout 3600 env JULIA_DEPOT_PATH="$PWD/.julia-depot" GROUP=Core \
  julia +1.12 --project=. -e '
    import Pkg
    Pkg.test(; coverage=false,
      julia_args=["--check-bounds=auto", "--compiled-modules=yes", "--depwarn=yes"],
      force_latest_compatible_version=false,
      allow_reresolve=true)
  '

The extension reports:

WARNING: Imported binding Symbolics.array_term was undeclared at import time
ERROR: FieldError: type Union has no field `parameters`
...
Symbolics/src/wrapper-types.jl:160
Symbolics/src/register.jl:149
lib/CanonicalMoments/ext/SymbolicsExt.jl:1

The Union is the AbstractVecOrMat registration. Splitting it into vector and matrix registrations is not sufficient: @register_array_symbolic generates overlapping scalar-symbolic wrapper methods for the two registrations, which is forbidden during precompilation.

OUQBase also defines broad downstream methods such as:

SymbolicUtils.promote_symtype(::𝔼_, x) = x

These are ambiguous with SymbolicUtils 4's promote_symtype(::Operator, ::Type{T}) method. The LTS/pre jobs report:

MethodError: promote_symtype(::OUQBase.𝔼_, ::Type{Real}) is ambiguous

Public-API prototype that cleared stage 1

A local prototype used only bindings verified public through their owning modules:

• Symbolics.wrap
• SymbolicUtils.unwrap, SymbolicUtils.term, and SymbolicUtils.symtype

SymbolicUtils.shape is not public through its owning module and was not used. The prototype:

• replaced array_term with SymbolicUtils.term(...; type=AbstractMatrix{Real}, shape=axes(companion)), followed by Symbolics.wrap;
• registered the two-argument eigvec operation once on AbstractArray{<:Real} with a symtype-based 1-D/2-D guard, preserving vector and matrix behavior and rejecting a 3-D symbolic array;
• declared SymbolicUtils as a weak dependency/extension trigger;
• constructed 𝔼 and ℙ with exported SymbolicUtils.term(...; type=symtype(x)) and removed the ambiguous downstream promote_symtype methods.

Focused tests covered concrete symbolic shapes, numeric build_function equivalence for the companion matrix, eigvals, eigvecs with a vector of eigenvalues, generalized eigvecs with a matrix metric, and 3-D rejection. The actual grouped harness passed:

Symbolics extension | 11  11  35.3s
Testing CanonicalMoments tests passed

This proves a public construction route exists, but it exposes stage 2.

Stage 2: OUQ variable discovery stops at operators

With the public prototype applied, this Julia 1.10 command cleared extension precompilation and the previous promotion ambiguity:

timeout 3600 env JULIA_DEPOT_PATH="$PWD/.julia-depot" GROUP=OUQBase \
  julia +1.10 --project=. -e '
    import Pkg
    Pkg.develop([Pkg.PackageSpec(path=joinpath("lib", name))
      for name in readdir("lib") if isdir(joinpath("lib", name))])
    Pkg.test(; coverage=false,
      julia_args=["--check-bounds=auto", "--compiled-modules=yes", "--depwarn=yes"],
      force_latest_compatible_version=false,
      allow_reresolve=true)
  '

The focused operator test passed 2/2. The existing Flood Problem (Q only) test then failed:

ArgumentError: A random variable can only belong to one random variable group
  get_group_name(...)
  create_constraints_map(...)

The immediate behavior is:

constraints = [𝔼(Q) ~ 1320.42]
Symbolics.get_variables(only(constraints))
# Set(Any[𝔼_()(Q)])

The admissible-set map contains Q, so 𝔼(Q) cannot match it. SymbolicUtils 4's default_is_atomic intentionally treats Terms whose operation is an Operator as atomic. OUQBase variable discovery therefore needs a package-specific, public-API traversal or preprocessing step that deliberately crosses 𝔼/ℙ boundaries while preserving the intended behavior for other symbolic operators.

Scope/status

No test was skipped, silenced, weakened, or loosened during this investigation. The public prototype was reverted after the full OUQBase group exposed stage 2; no patch or dependency pin was retained.

[avinashresearch1]

This is addressed in the internal OUQ-dev.jl. This public package is only updated a couple of times a year.

[ChrisRackauckas-Claude]

Fresh main reproduction, bisect, and discriminating regression evidence (2026-08-11)

I reproduced the extension-loading failure from a freshly fetched clean worktree at origin/main (246141f8f176de2ad712709486533c84f03b3e5a) with the repository's actual umbrella QA entrypoint:

GROUP=QA /home/crackauc/.juliaup/bin/julia +1.12.4 --project=. -e 'using Pkg; Pkg.test()'

The resolver selected SciMLTesting 1.8.0, Symbolics 7.36.0, and SymbolicUtils 4.45.0. The command exited 1 while precompiling CanonicalMoments -> SymbolicsExt, then failed OUQBase and the umbrella package transitively. The root exception was:

WARNING: Imported binding Symbolics.array_term was undeclared at import time during import to SymbolicsExt.
ERROR: LoadError: FieldError: type Union has no field `parameters`, available fields: `a`, `b`
Stacktrace:
  [1] getproperty(x::Type, f::Symbol)
    @ Base ./Base_compiler.jl:48
  [2] (::Symbolics.var"#is_wrapped_array_eltype#wrap_func_expr##9")(T::Type)
    @ Symbolics ~/.julia/packages/Symbolics/HQrFy/src/wrapper-types.jl:220
  [3] (::Symbolics.var"#wrap_func_expr##14#wrap_func_expr##15"{Tuple{DataType, UnionAll}, Symbolics.var"#is_wrapped_array_eltype#wrap_func_expr##9"})(::Tuple{Int64, Symbol})
    @ Symbolics ~/.julia/packages/Symbolics/HQrFy/src/wrapper-types.jl:238
  [4] iterate
    @ ./generator.jl:48 [inlined]
  [5] collect_to!(dest::Vector{Symbol}, itr::Base.Generator{Base.Iterators.Enumerate{Vector{Any}}, Symbolics.var"#wrap_func_expr##14#wrap_func_expr##15"{Tuple{DataType, UnionAll}, Symbolics.var"#is_wrapped_array_eltype#wrap_func_expr##9"}}, offs::Int64, st::Tuple{Int64, Int64})
    @ Base ./array.jl:848
  [6] collect_to_with_first!(dest::Vector{Symbol}, v1::Symbol, itr::Base.Generator{Base.Iterators.Enumerate{Vector{Any}}, Symbolics.var"#wrap_func_expr##14#wrap_func_expr##15"{Tuple{DataType, UnionAll}, Symbolics.var"#is_wrapped_array_eltype#wrap_func_expr##9"}}, st::Tuple{Int64, Int64})
    @ Base ./array.jl:826
  [7] collect(itr::Base.Generator{Base.Iterators.Enumerate{Vector{Any}}, Symbolics.var"#wrap_func_expr##14#wrap_func_expr##15"{Tuple{DataType, UnionAll}, Symbolics.var"#is_wrapped_array_eltype#wrap_func_expr##9"}})
    @ Base ./array.jl:800
  [8] map
    @ ./abstractarray.jl:3402 [inlined]
  [9] (::Symbolics.var"#wrap_func_expr##10#wrap_func_expr##11"{Symbolics.var"#is_wrapped_array_eltype#wrap_func_expr##9", Vector{Tuple{UnionAll, DataType, UnionAll, UnionAll, UnionAll}}, Vector{Any}, Symbol, Vector{Any}, Expr})(Ts::Tuple{DataType, UnionAll})
    @ Symbolics ~/.julia/packages/Symbolics/HQrFy/src/wrapper-types.jl:234
 [10] iterate
    @ ./generator.jl:48 [inlined]
 [11] collect(itr::Base.Generator{Base.Iterators.Drop{Base.Iterators.ProductIterator{Tuple{Tuple{UnionAll, DataType, UnionAll, UnionAll, UnionAll}, Tuple{UnionAll, DataType, UnionAll, UnionAll, UnionAll}}}}, Symbolics.var"#wrap_func_expr##10#wrap_func_expr##11"{Symbolics.var"#is_wrapped_array_eltype#wrap_func_expr##9", Vector{Tuple{UnionAll, DataType, UnionAll, UnionAll, UnionAll}}, Vector{Any}, Symbol, Vector{Any}, Expr}})
    @ Base ./array.jl:790
 [12] map(f::Function, A::Base.Iterators.Drop{Base.Iterators.ProductIterator{Tuple{Tuple{UnionAll, DataType, UnionAll, UnionAll, UnionAll}, Tuple{UnionAll, DataType, UnionAll, UnionAll, UnionAll}}}})
    @ Base ./abstractarray.jl:3402
 [13] wrap_func_expr(mod::Module, expr::Expr, wrap_arrays::Bool)
    @ Symbolics ~/.julia/packages/Symbolics/HQrFy/src/wrapper-types.jl:228
 [14] var"@wrapped"(__source__::LineNumberNode, __module__::Module, expr::Any, wrap_arrays::Any)
    @ Symbolics ~/.julia/packages/Symbolics/HQrFy/src/wrapper-types.jl:337
in expression starting at lib/CanonicalMoments/ext/SymbolicsExt.jl:1

The environment was:

Julia Version 1.12.4
Commit 01a2eadb047 (2026-01-06 16:56 UTC)
Linux x86_64-linux-gnu
128 x AMD EPYC 7502 32-Core Processor
LLVM 18.1.7
Threads: 1 default, 1 interactive, 1 GC

Bisect

After unshallowing the repository history, git bisect identified 246141f8f176de2ad712709486533c84f03b3e5a as the first bad commit. Its parent, aaf3631b25a6db31bbc43c5aab37c6fdecc2966a, resolved Symbolics 6.58.0 and passed the exact full QA command above:

Test Summary: | Pass  Total     Time
QA            |   18     18  2m24.3s
Testing OptimalUncertaintyQuantification tests passed

This confirms that the Symbolics 7 / SymbolicUtils 4 compat expansion in PR 40 introduced the default-branch QA failure.

Failing-before / passing-after extension regression

I added a local regression test that constructs a public StieltjesTransform from symbolic B/C vectors, then calls the public support and weights APIs with EigvalSupportAlg / EigvecWeightAlg, checking the (2,) symbolic outputs. Symbolics was added only to CanonicalMoments' test extras/target.

With the regression test present but the extension fix absent, this exact command exited 1 with the root exception above:

GROUP=CanonicalMoments_Core /home/crackauc/.juliaup/bin/julia +1.12.4 --project=. -e 'using Pkg; Pkg.test()'

An extension-only prototype then:

• changed the AbstractVecOrMat registration to a single non-Union AbstractArray registration;
• registered LinearAlgebra.SymTridiagonal through the public @register_array_symbolic API; and
• removed the special _companion_matrix(::Vector{Num}, ::Vector{Num}) method and its removed/non-public array_term dependency.

The identical command exited 0:

Test Summary:               | Pass  Total  Time
Orthogonal Polynomial Roots |   11     11  3.3s
Test Summary:   | Pass  Total   Time
Moment Sequence |  264    264  22.0s
Testing CanonicalMoments tests passed
Testing OptimalUncertaintyQuantification tests passed

Runic 1.7.0 also reported no formatting changes across all tracked Julia files.

This validates an extension-loading/public-construction direction only. It does not resolve or retest the stage-2 OUQ variable-discovery problem already documented in this issue, and the prototype has not been pushed as a PR.