This might be related to #73.
The stock LFM2.5-8B-A1B chat_template.jinja (https://huggingface.co/LiquidAI/LFM2.5-8B-A1B/blob/main/chat_template.jinja) never tells the model what format to emit a tool call in. The system block it builds is only:
Today's date: ... \n\n List of tools: <JSON>
It never mentions <|tool_call_start|> / <|tool_call_end|> nor shows the pythonic list. On a fresh session (no prior assistant tool-call turn to
imitate), the model sometimes drifts to an OpenAI-style JSON object instead of the pythonic call, so the tool call is silently lost or the turn ends
with no structured output.
Repro: chat request with tools + "explore the repo, then call a tool", temperature ~0.8, empty history. The model sometimes emits something like
...reasoning... {"TaskCreate": {...}, "activeForm": ...}
instead of:
<|tool_call_start|>[TaskCreate(subject='...', description='...')]<|tool_call_end|>
Observed from vLLM and tabbyAPI (both load this same template). Severity is lowest on vLLM, the reference path where the sentinels are registered as special tokens; highest on providers that match the sentinels as plain text.
Second, independent bug: format_arg_value wraps string args in single quotes without escaping, so a value like "It's" renders the malformed
[send_message(text='It's fine')]
into the assistant tool-call history.
Suggested fix (applies cleanly; both changes validated with jinja2):
--- a/chat_template.jinja
+++ b/chat_template.jinja
@@ format_arg_value (escape quotes/backslashes before wrapping):
- {{- "'" + arg_value + "'" -}}
+ {{- "'" + (arg_value | replace("\\", "\\\\") | replace("'", "\\'")) + "'" -}}
@@ tools system block (append a format directive after the tool list):
- " + (tools | tojson) -%}
+ " + (tools | tojson)
+ + "\n\nTo call a tool, output EXACTLY one line wrapped in "
+ "<|tool_call_start|> and <|tool_call_end|>: "
+ "[function_name(arg='value', arg2=123)]. Never emit JSON. "
+ "If no tool is needed, answer in prose." -%}
I validated this change with LFM2.5-8B in hermes and pi. With both harnesses the model generated long chains of tool calls, thinking blocks and responses. It's working great!
This might be related to #73.
The stock LFM2.5-8B-A1B chat_template.jinja (https://huggingface.co/LiquidAI/LFM2.5-8B-A1B/blob/main/chat_template.jinja) never tells the model what format to emit a tool call in. The system block it builds is only:
It never mentions <|tool_call_start|> / <|tool_call_end|> nor shows the pythonic list. On a fresh session (no prior assistant tool-call turn to
imitate), the model sometimes drifts to an OpenAI-style JSON object instead of the pythonic call, so the tool call is silently lost or the turn ends
with no structured output.
Repro: chat request with tools + "explore the repo, then call a tool", temperature ~0.8, empty history. The model sometimes emits something like
instead of:
Observed from vLLM and tabbyAPI (both load this same template). Severity is lowest on vLLM, the reference path where the sentinels are registered as special tokens; highest on providers that match the sentinels as plain text.
Second, independent bug: format_arg_value wraps string args in single quotes without escaping, so a value like "It's" renders the malformed
into the assistant tool-call history.
Suggested fix (applies cleanly; both changes validated with jinja2):
I validated this change with LFM2.5-8B in hermes and pi. With both harnesses the model generated long chains of tool calls, thinking blocks and responses. It's working great!