feat(agent-eval):添加可选轨迹摘要

[cx-hub666]

变更描述

Problem

Agent eval run 当前会返回最终结果和 langfuse_trace_id，但调用方无法在 API 响应中快速看到本次评估的工具调用概况，例如调用了多少工具、哪些工具出错、是否发生人工中断。维护者建议先通过 Langfuse 查看详细链路，因此本 PR 不替代 Langfuse，只补充一个轻量摘要入口。

Solution

在 agent eval 创建接口中新增可选参数 include_trajectory_summary，默认保持 false。当调用方显式开启时，服务从现有 run stream events 中读取最多 500 条事件，生成轻量 trajectory_summary，并在可用时关联已有 langfuse_trace_id，方便 API 调用方快速定位工具调用概况，详细轨迹仍可继续在 Langfuse 中查看。

变更类型

• 新功能
• Bug 修复
• 文档更新
• 其他

Changes

• AgentEvalRunCreate 新增 include_trajectory_summary 可选字段，默认不改变现有返回。
• run_agent_eval 支持按需附加 trajectory_summary。
• 新增从 run events 构建工具调用摘要的逻辑，包含：
  • event 数量和 seq 范围
  • tool call 总数
  • tool error 总数
  • interrupt 总数
  • 按工具名聚合的调用次数和错误次数
  • 可用时附加 langfuse_trace_id
• 增加 agent eval service 单元测试，覆盖正常路径、空事件、读取上限和默认兼容性。

测试

• 相关功能正常工作

相关日志或者截图

uv run python -m pytest test/unit/services/test_agent_eval_run_service.py test/unit/services/test_agent_run_service.py::test_get_agent_run_result_uses_output_message_id -q
# 7 passed, 2 warnings

uv run ruff check server/routers/agent_router.py package/yuxi/services/agent_eval_run_service.py test/unit/services/test_agent_eval_run_service.py
# All checks passed!

uv run python -m py_compile server/routers/agent_router.py package/yuxi/services/agent_eval_run_service.py test/unit/services/test_agent_eval_run_service.py
# passed

git diff --check
# no whitespace errors

Notes for Reviewer

• include_trajectory_summary 默认为 false，用于避免改变现有 agent eval API 的默认响应结构。
• 摘要只读取现有 Redis run events，不新增持久化表，也不保存完整 trajectory。
• TRAJECTORY_SUMMARY_EVENT_LIMIT 当前设为 500，用于控制响应构建成本；详细 trace 仍建议通过 Langfuse 查看。

说明

Related discussion: #796

[gemini-code-assist]

Code Review

This pull request introduces a lightweight tool-call trajectory summary feature for agent evaluation runs, including schema updates, helper functions to parse stream events, and comprehensive unit tests. The review feedback highlights a logical bug in generating fallback keys for tool calls without IDs, a potential string conversion issue where missing sequence IDs could output "None" instead of null, and a fault tolerance concern where a failure in loading the optional trajectory summary could crash the entire evaluation API.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[xerrors]

这个计数重复的问题，你看一下是否存在：

---

interrupt_count is double-counted for every human interrupt** — agent_invocation_service.py (_build_trajectory_summary, the two summary["interrupt_count"] += 1 sites)

Every real interrupt (ask_user_question_required / human_approval_required / interrupted) causes the worker to append two stream events (see run_worker.py:410-456):

• an event_type="interrupt" event (via _map_chunk_to_run_event, run_worker.py:226-228)
• an event_type="end" event whose payload chunk retains the original status (ask_user_question_required, etc.) — e.g. run_worker.py:456 _append_end_event(..., payload={"chunk": chunk}).

The parser increments once on the first event (the event_type == "interrupt" branch), then increments again on the end event via the chunk-status branch — which is guarded only by event_type != "interrupt", a check the "end" event passes. Failure scenario: a run with one human-approval interrupt reports interrupt_count: 2; a run interrupted twice reports 4. The headline metric is consistently ~2× reality. The two tests that assert interrupt_count (interrupt event_type + a bare payload.reason) never exercise the real ask/approval status-carrying chunk, so they pass while the production path is wrong.

by claude code

---

然后是函数出现的顺序，修改为符合从上到下的顺序，而不是先声明后使用的倒序，方便阅读。代码可以做进一步的简化，并添加 docstring（中文）

[xerrors]

测试一下 @codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Can't wait for the next one!

Reviewed commit: 78454bbb53

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[cx-hub666]

已修改，请查看