fix(memos-local-plugin): add maxTokens to LlmSchema & SkillEvolverSchema - #1896
fix(memos-local-plugin): add maxTokens to LlmSchema & SkillEvolverSchema#1896chouti wants to merge 3 commits into
Conversation
Reasoning models (deepseek-reasoner, o1*, gpt-5-thinking) consume hundreds of tokens on chain-of-thought. The hard-coded 1024 cap on LLM JSON reflection caused 73+ 'llm.json malformed' errors per episode, blocking episode closure and disconnecting the bridge. - Add maxTokens: NumberInRange(4000, 1024, 32768) to LlmSchema - Add maxTokens: NumberInRange(4000, 1024, 32768) to SkillEvolverSchema - Add corresponding defaults in defaults.ts (backward compat) TypeBox's Value.Default preserves user-supplied fields, so old configs without maxTokens get the 4000 default; new configs can override.
|
Automated Test Results: FAILED Cloud test-engine rerun against
Failed cases:
Do not merge until this is fixed and cloud tests pass. |
🤖 Open Code ReviewTarget: PR #1896 ✅ OpenCodeReview: No comments generated. Looks good to me. Generated by cloud-assistant via Open Code Review. |
✅ Automated Test Results: PASSEDAll tests passed (103/103 executed). memos_local_plugin/unit: 103/103. Duration: 13s [advisory, non-gating] AI-generated tests on branch test/auto-gen-c3f809a70b09c2e6-20260806195541: 77/78 passed, 1 failed — these do NOT affect the PR verdict; review the branch manually. Branch: |
There was a problem hiding this comment.
Thanks for addressing this issue. The truncation problem is still present on main, and the added runtime wiring tests for both shared and dedicated LLM clients are valuable.
Before merging, please reconsider the global 1024–32768 validation range. This is a provider-agnostic configuration, so a minimum of 1024 prevents legitimate smaller budgets, while the 32768 upper bound rejects models that support larger outputs. Please use a more permissive provider-neutral range, or only validate that the value is a positive integer.
Once the range and corresponding tests are updated, this should be ready for another review.
Problem
Reasoning models (e.g.
deepseek-reasoner,o1*,gpt-5-thinking) consume hundreds of tokens on chain-of-thought before emitting JSON content. Whenllm.jsonreflection /skillEvolver.crystallizeis called with the legacy hard-codedmax_tokens: 1024, the response is truncated mid-JSON and the bridge logsllm.json malformed73+ times in a single reflection cycle.Result: episodes never close,
recoveryReason: "dirty_reward_rescore"piles up, and the MemOS viewer reportsbridge.status: disconnectedeven though the daemon is alive.Reproduction
Start bridge with no override of
max_tokens. After ~1 episode:Direct curl confirms root cause: API returns
finish_reason: "length"with truncated content.Fix
Add
maxTokens(default 4_000) toLlmSchemaandSkillEvolverSchema. Reasonable budget for reasoning models while still clamping below OpenAI's 32k ceiling.LlmSchema.maxTokens: NumberInRange(4000, 1024, 32768)SkillEvolverSchema.maxTokens: NumberInRange(4000, 1024, 32768)defaults.tsfor old configsTypeBox's
Value.Defaultalready preserves user-supplied fields, so the change is backward compatible — old configs withoutmaxTokensget the 4_000 default; new configs can override.Verification
Before patch:
bridge.status: disconnectedAfter patch (config unchanged, schema updated):
bridge.status: connectedFiles changed
apps/memos-local-plugin/core/config/schema.ts(+15 lines, 2 field defs)apps/memos-local-plugin/core/config/defaults.ts(+2 lines, 2 default values)