Skip to content

Detect aLoRA adapters and prevent incorrect eager activation #321

Description

@leehack

Problem

llamadart currently applies every native LoRA adapter globally from the start of inference. aLoRA adapters require activation only after their invocation-token sequence. Treating them as ordinary LoRA adapters can silently produce incorrect output.

llama.cpp added aLoRA support in ggml-org/llama.cpp#15327, and the pinned b10333 bindings expose the invocation-token inspection functions.

Immediate scope

  • Inspect loaded adapters for aLoRA invocation tokens.
  • Reject aLoRA through a typed, actionable unsupported exception until invocation-aware execution is implemented.
  • Document the current limitation so native LoRA support is not interpreted as aLoRA support.
  • Add ordinary-LoRA and aLoRA/version-skew regression tests.

Follow-up/full support

  • Detect the invocation-token boundary in the prompt.
  • Decode the prefix with the adapter disabled and enable it at the boundary.
  • Define prompt-cache behavior and prevent unsafe prefix reuse.
  • Reject unsupported multiple-aLoRA combinations.
  • Validate output against upstream llama-server with a real aLoRA adapter.

The safety guard should land independently of full aLoRA support.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions