feat(cli): add intent-based command search

[samzong]

Summary

• add difyctl search "<intent>" over live command help metadata
• update the Agent Skill to discover candidates with search and verify them with per-command help
• cover ranking, generated registry wiring, Skill guidance, and agent workflow behavior

Issue: none; this draft is opened for maintainer discussion.

Screenshots

Not applicable — CLI-only change.

Checklist

• Documentation is updated in cli/README.md.
• I understand this PR may be closed without prior discussion or an assigned issue.
• Tests cover each introduced behavior and the change is atomic.
• pnpm tree:check, pnpm test, local agent-workflow E2E, pnpm build, and staged-path vp check passed.

From Cursor

[samzong]

Four-model agent workflow A/B report

Recommendation

Keep the search command and the final search-first Agent Skill workflow.

The evaluation covered four models, six command-discovery tasks, and two workflows for 48 isolated Cursor Agent runs. Both workflows selected the expected command path in every run. The measurable benefit is lower context and discovery overhead rather than higher accuracy on this already-saturated task set.

Aggregate results

Metric	Help-only baseline	Search loop	Change
Correct command path	24/24	24/24	unchanged
Runs loading full help -o json	24/24	0/24	eliminated
Runs loading per-command help	23/24	24/24	complete verification
Average CLI output	45,510 B	4,344 B	-90.5%
Average input footprint	188,893 tokens	137,959 tokens	-27.0%
Median input footprint	162,870 tokens	119,242 tokens	-26.8%
Average duration	29.5 s	24.8 s	-15.9%
Median duration	29.1 s	23.4 s	-19.4%
Average model output	1,208 tokens	833 tokens	-31.0%
Average tool calls	5.54	4.00	-27.8%
Average CLI calls	2.38	2.38	unchanged

Input footprint is inputTokens + cacheReadTokens + cacheWriteTokens from Cursor Agent usage. It is useful for within-evaluation comparisons and is not a billing-token measurement.

Per-model results

Model	Accuracy	Input footprint	CLI output	Average duration
Claude Sonnet 5 High	6/6 → 6/6	-23.6%	-87.7%	-10.6%
GPT-5.6 Sol High Fast	6/6 → 6/6	-24.1%	-90.7%	-25.0%
Cursor Grok 4.6 High Fast	6/6 → 6/6	-32.9%	-91.3%	-26.6%
GPT-5.6 Luna High Fast	6/6 → 6/6	-29.8%	-91.2%	+3.1%

Luna's median duration improved by 6.9% despite its average increasing by 3.1%. With one sample per model/task/workflow combination, latency is not statistically significant; CLI bytes and token footprint are the stronger signals.

Agent-workflow test cases

Each case was run once per model against both the help-only baseline and the search loop. Agents were required to return the command path, effect, required arguments, and relevant flags without reading source code or executing the business command.

Case	Prompt	Expected path
Export definition	Export the editable Dify Studio app definition as a YAML backup.	export studio-app
Import definition	Create or update an editable Studio app from a Dify DSL YAML file.	import studio-app
Inspect input schema	Obtain a published app's mode and complete input schema before running it.	describe app
Resume human input	Continue a workflow app paused for a human-input form submission.	resume app
Revoke devices	Revoke one session device, with an option to revoke all session devices.	auth devices revoke
Switch account	Switch the active account when several Dify accounts are authenticated locally.	use account

The final search-loop matrix used search in 24/24 runs, exact per-command help in 24/24, full sitemap help in 0/24, help agent in 0/24, and business-command execution in 0/24.

Automated test cases and verification

cli/src/commands/search/search.test.ts covers:

• path matches ranking above description matches, including simple plural folding;
• flag and agent-guide token matches without repeated-word inflation;
• deterministic path ordering for equal scores;
• empty results for intents with no searchable tokens.

cli/test/e2e/suites/agent/agent-skill-workflow.e2e.ts covers:

• the installed Skill's search → per-command-help workflow;
• zero command-tree enumeration in the Skill;
• search ranking export studio-app first for export app;
• candidate output containing path, description, effect, and score while excluding flags and agent guides;
• machine-readable help and effect metadata used by the confirmation gate.

Fresh verification on the committed diff:

• pnpm tree:check — passed;
• pnpm test — 1,252 passed, 26 skipped;
• local agent-workflow E2E — 62 passed, 29 credential-dependent cases skipped;
• pnpm build — passed;
• vp check on all staged CLI paths — passed with no formatting, lint, or type errors.

Interpretation limits

These tasks are correctly solvable from the existing sitemap, so the evaluation demonstrates that search preserves command-selection accuracy while reducing context cost. It does not establish an accuracy improvement for ambiguous, synonym-heavy, no-match, or noun-collision intents. Those require a larger blind ranking set with top-1, top-3, and MRR measurements.