Skip to content

Commit 57fa8b1

Browse files
committed
chore(agent-context): drop the parse-only trigger-case test; the evaluation is manual-only
A test that only parses the case files proves nothing about trigger behaviour, so the cases and the documented run command are the evidence. README no longer claims a CI check.
1 parent ff96332 commit 57fa8b1

2 files changed

Lines changed: 1 addition & 71 deletions

File tree

‎agent-context/evals/README.md‎

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -20,9 +20,7 @@ agent-context/evals/run-skill-triggers.sh <out-dir>
2020

2121
It starts real Claude sessions on your own login (25 cases x 3 runs = 75 sessions, a few
2222
minutes with `-j 4`) and writes `<out-dir>/result.json`. Use `--case '<glob>'` to run a
23-
subset. It is not part of CI because model behaviour is not deterministic; CI only checks
24-
that the cases parse and name existing skills
25-
(`server/typescript/packages/sdk/test/agent-context/skill-trigger-evals.test.ts`).
23+
subset. It is manual-only and not part of CI, because model behaviour is not deterministic.
2624

2725
Re-run it after editing either description. A `Reached maximum number of turns` note on a
2826
case is expected: the runs are capped at 3 turns because only the first tool call matters.

‎server/typescript/packages/sdk/test/agent-context/skill-trigger-evals.test.ts‎

Lines changed: 0 additions & 68 deletions
This file was deleted.

0 commit comments

Comments
 (0)