Skip to content

Add geometric and projected Armijo line searches - #106

Merged
ChrisRackauckas merged 3 commits into
SciML:mainfrom
ChrisRackauckas-Claude:feature/projected-backtracking
Sep 13, 2026
Merged

ChrisRackauckas merged 3 commits into
SciML:mainfrom
ChrisRackauckas-Claude:feature/projected-backtracking

Conversation

@ChrisRackauckas-Claude

@ChrisRackauckas-Claude ChrisRackauckas-Claude commented Sep 13, 2026 •

Copy link
Copy Markdown
Member

Add ArmijoLineSearch (fixed-factor backtracking) and ProjectedBackTracking (Armijo along a box-projection arc). They share one search loop and the existing residual/objective merit evaluator; get_trial exposes the cached projected point and residual so outer solvers can accept a trial without another function evaluation. Existing algorithm defaults are unchanged.

The ordinary method follows Armijo (1966); the projected rule follows Bertsekas (1976). The projected condition uses dot(gradient, trial - u), including after multiple bound hits. It requires the caller's merit gradient, avoiding an extra Jacobian or derivative operator. Both names and get_trial are documented in the rendered API. These compatible additions ship as 0.1.18; no dependencies were added.

Verification (all commands exited 0):

GROUP=Core /home/crackauc/.juliaup/bin/julia +1.13.0 --project -e 'using Pkg; Pkg.test()'
Test Summary:        | Pass  Total   Time
Core/armijo_tests.jl |  191    191  33.4s
Test Summary:                  | Pass  Total   Time
Core/custom_optimizer_tests.jl |  168    168  53.1s
Test Summary:            | Pass  Total   Time
Core/gpu_arrays_tests.jl |  151    151  16.1s
Test Summary:                 | Pass  Total   Time
Core/objective_merit_tests.jl |  106    106  15.9s
Test Summary:              | Pass  Total  Time
Core/optimization_tests.jl |   13     13  1.0s
Test Summary:              | Pass  Total   Time
Core/root_finding_tests.jl |  107    107  19.5s
     Testing LineSearch tests passed 

GROUP=Core /home/crackauc/.juliaup/bin/julia +1.10.12 --project=../scratch/armijo-lts -e 'using Pkg; Pkg.develop(path="."); Pkg.test("LineSearch")'
Test Summary:        | Pass  Total   Time
Core/armijo_tests.jl |  191    191  51.3s
Test Summary:                  | Pass  Total   Time
Core/custom_optimizer_tests.jl |  168    168  42.3s
Test Summary:            | Pass  Total   Time
Core/gpu_arrays_tests.jl |  148    148  11.7s
Test Summary:                 | Pass  Total   Time
Core/objective_merit_tests.jl |  106    106  11.8s
Test Summary:              | Pass  Total  Time
Core/optimization_tests.jl |   13     13  0.9s
Test Summary:              | Pass  Total   Time
Core/root_finding_tests.jl |  107    107  22.1s
     Testing LineSearch tests passed 

GROUP=QA /home/crackauc/.juliaup/bin/julia +1.12 --project -e 'using Pkg; Pkg.test()'
Test Summary:          | Pass  Total   Time
QA/allocation_tests.jl |   30     30  37.1s
Test Summary: | Pass  Total   Time
QA/qa.jl      |   21     21  55.5s
     Testing LineSearch tests passed 

/home/crackauc/.juliaup/bin/julia +1.10.12 --project=test/enzyme test/enzyme/runtests.jl
Test Summary:                            | Pass  Total   Time
Enzyme: LineSearches.jl Custom Optimizer |   24     24  11.4s
Test Summary:                               | Pass  Total  Time
Enzyme: Native Line Search Custom Optimizer |   12     12  1.5s
Test Summary:                          | Pass  Total  Time
Enzyme: LineSearches.jl Newton Raphson |   20     20  9.7s
Test Summary:                             | Pass  Total  Time
Enzyme: Native Line Search Newton Raphson |    8      8  2.2s

/home/crackauc/.juliaup/bin/julia +1.13.0 --project=docs docs/make.jl
RenderDocument: rendering document.
HTMLWriter: rendering HTML pages.

Runic, typos over changed files, and git diff --check passed. Tests cover scalar, static, matrix and in-place states; JLArrays without scalar indexing; supplied/analytic/AD derivatives; finite evaluation limits; rejection of nonfinite merit; projection-specific sufficient decrease; fixed/one-sided bounds; and parameter reinitialization. A first docs attempt received HTTP 429 from GitHub's ColPrac link; an unchanged retry passed with link checking enabled. The LTS run used a separate local environment to avoid reusing a Julia 1.13 stdlib manifest.

Actual CUDA/ROCm hardware and external downstream packages were not tested locally. These are sufficient-decrease methods, not Wolfe-curvature searches; convergence of a projected search with a general supplied direction depends on the outer method.

This supplies the core API needed to remove NonlinearSolve's local bounded Armijo loop, following SciML/NonlinearSolve.jl#1254 and SciML/NonlinearSolve.jl#1263.

The mixed-precision regression was also checked before and after the inward-rounding fix. With 6823a44 and the new test in place, the same GROUP=Core command exited 1:

Mixed-precision bounds remain feasible: Error During Test
AssertionError: lb <= only(u) <= ub

A Float32 trial at a Float64 upper bound of 0.1 had returned Success despite trial > ub; the symmetric lower-bound case also failed. Both tests keep domain assertions inside the residual. With the fix, the full Core runs above exit 0, including all 191 Armijo assertions. Bounds are rounded inward once during cache initialization, preserving state types and feasibility.

Ignore this draft until reviewed by @ChrisRackauckas.

🤖 Generated with Codex CLI 0.154.0 (model: gpt-6-astra). Session: local transcript /home/crackauc/.codex/sessions/2026/09/12/rollout-2026-09-12T04-30-15-01a094bd-179a-7972-97ec-36badb9448cd.jsonl.

ChrisRackauckas and others added 2 commits September 13, 2026 03:35
Share merit evaluation, trial buffers, contraction, and sufficient-decrease handling between ordinary Armijo and box-projected searches. Expose cached trial results for solver reuse and document both published rules.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Codex <noreply@openai.com>
Agent-Harness: Codex CLI 0.154.0
Agent-Model: gpt-6-astra
Agent-Session: local transcript /home/crackauc/.codex/sessions/2026/09/12/rollout-2026-09-12T04-30-15-01a094bd-179a-7972-97ec-36badb9448cd.jsonl
Round cached bounds inward to the state element type. Float32 trials must not round outside a Float64 bound before residual evaluation.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Codex <noreply@openai.com>
Agent-Harness: Codex CLI 0.154.0
Agent-Model: gpt-6-astra
Agent-Session: local transcript /home/crackauc/.codex/sessions/2026/09/12/rollout-2026-09-12T04-30-15-01a094bd-179a-7972-97ec-36badb9448cd.jsonl
Comment thread Project.toml Outdated
name = "LineSearch"
uuid = "87fe0de2-c867-4266-b59a-2f0a94fc965b"
version = "0.1.17"
version = "0.2.0"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this breaking?

@ChrisRackauckas
ChrisRackauckas marked this pull request as ready for review September 13, 2026 09:33
@ChrisRackauckas
ChrisRackauckas merged commit 9b7474a into SciML:main Sep 13, 2026
7 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants