docs: add CLI / tenant-config eval authoring guide

[sanchitmehtagit]

✏️ Changes

Adds docs/CLI_CONFIG_EVALS.md, a concise how-to for authoring CLI / tenant-config evals: evals that grade the agent's command trace against a live, provisioned throwaway tenant rather than files on disk.

ADDING_EVALS.md already mentions provision and the event graders in passing, but no single page walks through authoring one of these evals end to end. This guide fills that gap so the pattern is repeatable.

The guide covers how CLI evals differ from standard file-based evals (artifact, scaffold, graders, judge, environment), a goal-only PROMPT.md with provision: auth0-tenant and what that field does, and a graders.ts using the event primitives (ranCommand / ranCommandOneOf / ranCommandsInOrder) plus a trace-aware holistic judge. It points at the mfa_tenant_cli reference eval, and adds a cross-link from ADDING_EVALS.md.

Docs-only; no code changes.

• I described the changes on this PR.

🔮 Type of Change

• Standard
• Emergency
• Significant

🔗 References

• Reference eval: apps/auth0-evals/src/evals/mfa/tenant-cli/ (mfa_tenant_cli)

• I added at least one link (task, Slack thread, etc.) to explain why this change is needed.

📖 Documentation

This PR is the documentation change: it adds docs/CLI_CONFIG_EVALS.md and cross-links it from docs/ADDING_EVALS.md.

• I reflected this change in the (internal and/or user-facing) documentation, or explained why no update is needed.

🎯 Testing

No automated tests apply; this is a docs-only change. Rendering verified locally.

• I described how I tested these changes, or explained why I did not.
• This change has integration, unit, or performance test coverage, or I explained why it does not.

]

[coderabbitai]

📝 Walkthrough

Walkthrough

The changes add a dedicated guide for CLI and tenant-configuration evaluations. The existing evaluation guide now directs authors to it and limits its scope to standard file-based evaluations.

Changes

CLI evaluation documentation

Layer / File(s)	Summary
CLI evaluation authoring guidance docs/ADDING_EVALS.md, docs/CLI_CONFIG_EVALS.md	The existing guide links to the new guide. The new guide documents tenant provisioning, goal-only prompts, trace-based graders, MFA checks, verification commands, and the reference evaluation.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

• auth0/auth0-evals#182: Documents the CLI tenant MFA evaluation and injected CLI context described by this guide.

Suggested reviewers: frederikprijck

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: adding documentation for CLI and tenant-configuration evaluation authoring.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch docs/cli-config-evals

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/CLI_CONFIG_EVALS.md`:
- Around line 93-95: Update the verification command in CLI_CONFIG_EVALS.md to
include the required npm run format check alongside npm run lint and npm test,
preserving the existing build step and command sequence.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: c379eb57-7be6-4549-af6c-85af8928df5a

📥 Commits

Reviewing files that changed from the base of the PR and between 52b45da and 6b06b24.

📒 Files selected for processing (2)

• docs/ADDING_EVALS.md
• docs/CLI_CONFIG_EVALS.md