Skip to content

fix(llama-cpp): constrain strict LFM arguments - #12799

Draft
mudler-agent wants to merge 2 commits into
masterfrom
fix/native-lfm-schema
Draft

mudler-agent wants to merge 2 commits into
masterfrom
fix/native-lfm-schema

Conversation

@mudler-agent

Copy link
Copy Markdown
Collaborator

Description

Carry a LocalAI-only llama.cpp patch for strict native LFM2/LFM2.5 tool arguments. The pinned Python-style builder ignores required fields and other schema constraints. Offline reproduction accepts 30 invalid cases in the original 44-case matrix.

  • Preserve function.strict through native tool parsing and serialization.
  • Enforce the supported Sound schema subset: required and unique arguments, types, enums, string lengths, numeric bounds and the musical-meter pattern. Reject unsupported schema constructs explicitly.
  • Keep native Python-style calls. Generate canonical arguments while recognizing valid reordered arguments, escaped enum equivalents and in-range exponents.
  • Reject incomplete FINAL LFM tool envelopes before publishing calls, including non-strict calls. Leave partial streaming unchanged.
  • Add CPU-only tests using actual generated grammar, complete PEG parsing and exact mapped tool names, counts and argument values.

Related: #12772, #12783, #12787. No llama.cpp upstream submission is part of this PR.

Notes for Reviewers

This is a draft, not a complete or deployed repair. The patch targets llama.cpp c0fd78626405c6d9b934efef0ec7291fcc20e3d6.

Remaining before ready for review/merge:

  • Idempotent, fail-closed patch preparation and build dependencies; fresh/repeated/copied/partial/conflicting source tests.
  • Go duplicate-key rejection for Sound arguments and normal/auth regressions.
  • CPU CI integration and full backend build verification.
  • Controlled unpatched/patched construction and recognition performance comparison.
  • Direct partial-streaming regression coverage, final cumulative acceptance review and user-facing documentation.

Independent draft-publication review passes fresh patch provenance, all three native tests (13.24 seconds), matrix and edge checks. Partial streaming is unchanged by inspection of the guard, but lacks direct regression coverage in these new harnesses.

No live inference, model loading, GPU access or deployment was performed. These tests establish offline native parser behavior, not conformance of the installed backend or model sampling.

Verification

From the LocalAI root, with an isolated exact-pin source tree patched as described in backend/cpp/llama-cpp/tests/README-native-strict.md:

backend/cpp/llama-cpp/tests/run-native-strict.sh /absolute/pinned-source /absolute/scratch-build

Latest local run:

  • All three CTest executables pass (18.22 seconds).
  • 1,092 schema-matrix rows: zero mismatches; invalid cases publish zero calls.
  • 639 semantic/mode/FINAL-truncation checks: zero failures, plus six JSON-comparison controls.
  • Fresh archive patch forward/reverse checks and byte comparison of all seven resulting native files pass.

Full make build, tokenized sampling and live model checks are not run.

Signed commits

  • Yes, I signed my commits (human maintainer DCO; Assisted-by: nib:gpt-6.1-sol).
  • Documentation updated (docs/content/) for user-facing changes, or not applicable — pending before this draft is ready.

Carry a local native patch that preserves strict through tool parsing
and serialization. Keep false and absent values compatible with legacy
callers. Add a CPU-only native round-trip regression test.

This prerequisite does not enforce strict argument schemas yet.

Assisted-by: nib:gpt-6.1-sol
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
The native LFM builder accepts arguments that violate strict schemas.
Carry a local schema-aware patch and validate decoded mapped values.
Require complete final tool envelopes before publishing calls.

Add offline native grammar, parser and exact-mapping regressions.
Build integration and full acceptance remain pending in the draft PR.

Assisted-by: nib:gpt-6.1-sol
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants