Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/users-create-confirmation-prompt.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"clerk": patch
---

Make `clerk users create` confirm before writing. The command registers `--yes` as "Skip confirmation prompt" and its examples recommend passing it, but no prompt existed, so the flag did nothing and the user was created immediately. Human mode now previews the redacted request body and asks before the POST. Agent mode is unchanged: it never prompts, so `--dry-run` remains the safety net there.
32 changes: 20 additions & 12 deletions .claude/skills/audit-clerk-skill/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: audit-clerk-skill
description: Audits the Clerk CLI source tree and proposes updates to the bundled `clerk-cli` skill so it stays in sync with the binary. Use when the user says "audit the clerk-cli skill", "update the clerk-cli skill", "check the skill against the code", "resync clerk-cli skill", or after adding/renaming/removing CLI commands, flags, or agent-mode behavior.
description: Audits the Clerk CLI source tree and proposes updates to the `clerk-cli` skill in clerk/skills so it stays in sync with the binary. Use when the user says "audit the clerk-cli skill", "update the clerk-cli skill", "check the skill against the code", "resync clerk-cli skill", or after adding/renaming/removing CLI commands, flags, or agent-mode behavior.
effort: high
user-invocable: true
disable-model-invocation: true
Expand All @@ -11,15 +11,23 @@ metadata:

# Audit the clerk-cli Skill

Cross-check `skills/clerk-cli/` against the actual CLI source in `packages/cli-core/` and propose precise edits wherever they have drifted. The binary is the source of truth; the skill is documentation that must track it.
Cross-check the `clerk-cli` skill against the actual CLI source in `packages/cli-core/` and propose precise edits wherever they have drifted. The binary is the source of truth; the skill is documentation that must track it.

**The skill lives in another repo.** #315 removed the bundled copy, so it is now `skills/core/clerk-cli/` in [clerk/skills](https://github.com/clerk/skills), versioned independently of the CLI. This audit reads the CLI here and proposes edits there; they land as a PR against clerk/skills. Set `$SKILL_ROOT` to a local clone before starting, and stop if there is not one:

```sh
SKILL_ROOT=../skills/skills/core/clerk-cli # adjust to the checkout
ls "$SKILL_ROOT/SKILL.md" || echo "clone clerk/skills first"
```
Comment on lines +18 to +21

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Make the missing skill checkout fail closed.

ls ... || echo ... prints an error but returns success, so the audit can continue despite the instruction to stop when $SKILL_ROOT/SKILL.md is missing. Use a non-zero exit path instead.

Proposed fix
 SKILL_ROOT=../skills/skills/core/clerk-cli   # adjust to the checkout
-ls "$SKILL_ROOT/SKILL.md" || echo "clone clerk/skills first"
+test -f "$SKILL_ROOT/SKILL.md" || {
+  echo "clone clerk/skills first" >&2
+  exit 1
+}
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
```sh
SKILL_ROOT=../skills/skills/core/clerk-cli # adjust to the checkout
ls "$SKILL_ROOT/SKILL.md" || echo "clone clerk/skills first"
```
SKILL_ROOT=../skills/skills/core/clerk-cli # adjust to the checkout
test -f "$SKILL_ROOT/SKILL.md" || {
echo "clone clerk/skills first" >&2
exit 1
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.claude/skills/audit-clerk-skill/SKILL.md around lines 18 - 21, Update the
SKILL_ROOT checkout validation in the audit instructions so a missing SKILL.md
exits through a non-zero failure path instead of only printing a message.
Preserve the existing success path when the file exists and ensure the audit
stops when the required checkout is absent.


Every skill path below is relative to `$SKILL_ROOT`.

**ultrathink** on this task. It requires building a full command tree, comparing two representations of it, and making judgment calls about what belongs in the skill vs. in `references/*.md` vs. in `--help`. Shallow passes miss drift.

## Inputs

- **Source of truth**: `packages/cli-core/src/commands/**` (one directory per top-level command), plus `packages/cli-core/src/cli.ts`, `cli-program.ts`, `mode.ts`, and anything in `packages/cli-core/src/lib/` referenced by commands (runner preference, agent mode, doctor checks, key resolution).
- **Target**: `skills/clerk-cli/SKILL.md` and `skills/clerk-cli/references/*.md`.
- **Template markers**: the skill uses `{{CLI_VERSION}}` placeholders substituted at install time by `clerk skill install`. Preserve them; do not expand.
- **Target**: `$SKILL_ROOT/SKILL.md` and `$SKILL_ROOT/references/*.md`.

## Workflow

Expand All @@ -39,7 +47,7 @@ Read the per-command `README.md` (`packages/cli-core/src/commands/<name>/README.
- Flag commands or sub-paths marked mocked/stubbed (blockquote at top of the README). The skill should not document these as production-ready.
- Cross-check Clerk API endpoint claims in `references/recipes.md`.

Do **not** propose bundling the READMEs into `skills/clerk-cli/references/` (symlinks or text imports). They ship internal detail agents do not need, and inflate the compiled binary. They are a reference for the audit, not for the skill.
Do **not** propose bundling the READMEs into `$SKILL_ROOT/references/` (symlinks or text imports). They ship internal detail agents do not need, and inflate the compiled binary. They are a reference for the audit, not for the skill.

Also capture cross-cutting behavior:

Expand All @@ -52,7 +60,7 @@ Don't memorize output. Prefer reading the source directly over running the binar

### 2. Extract the skill's current claims

Read `skills/clerk-cli/SKILL.md` and each file under `skills/clerk-cli/references/`. Extract every concrete claim:
Read `$SKILL_ROOT/SKILL.md` and each file under `$SKILL_ROOT/references/`. Extract every concrete claim:

- Every command mentioned in the "Core commands at a glance" table and the Invoking-the-CLI table.
- Every flag called out by name.
Expand Down Expand Up @@ -94,25 +102,25 @@ If a new reference file is warranted (e.g. a `references/commands.md` table), pr

Emit a review-ready proposal. For each change:

- Path (`skills/clerk-cli/SKILL.md` or `skills/clerk-cli/references/<file>.md`).
- Path (`$SKILL_ROOT/SKILL.md` or `$SKILL_ROOT/references/<file>.md`).
- Why (source citation: `packages/cli-core/src/commands/<cmd>/<file>.ts:<line>`).
- Either a unified diff (preferred) or a before/after block for prose sections.
- Severity: `drift` (factually wrong today), `gap` (missing coverage), `polish` (clearer wording, better placement, or cuts that route the agent to `--help` instead of duplicating it).

Group the proposal by file. Do **not** touch `{{CLI_VERSION}}` markers. Do **not** rewrite sections that are still accurate just because they are near a change.
Group the proposal by file. Do **not** rewrite sections that are still accurate just because they are near a change.

### 6. Apply or hand back

Default: present the proposal and stop. The user reviews, then says apply.

If invoked as `/audit-clerk-skill --apply`, apply `drift` and `gap` edits directly but still list `polish` suggestions for review. Run `bun run format` after applying so markdown tables and frontmatter match repo style.
If invoked as `/audit-clerk-skill --apply`, apply `drift` and `gap` edits directly to `$SKILL_ROOT` but still list `polish` suggestions for review. Edits land in the clerk/skills clone, not this repo, so commit them on a branch there and open a PR against clerk/skills. Nothing in this repo changes.

## Guardrails

- **Never invent flags.** If a flag appears in a test but not in the command's argument parser, treat it as test-only and flag it for human review.
- **Preserve voice.** The existing skill is terse and third-person; match it. No first- or second-person drift.
- **No em-dashes** anywhere in proposals (repo style rule).
- **Respect the template.** `{{CLI_VERSION}}` stays verbatim. The Invoking-the-CLI runner table is generated from `preferredRunner` logic; if that logic changes, update the table, otherwise leave it alone.
- **Respect the template.** The Invoking-the-CLI runner table is generated from `preferredRunner` logic; if that logic changes, update the table, otherwise leave it alone.
- **Stay within the 500-line guidance** for `SKILL.md`. When the budget is tight, move content to `references/` rather than deleting it outright.

## Output shape
Expand All @@ -125,13 +133,13 @@ Return the proposal as:
## Summary
<counts per bucket, one-line headline of the biggest drift>

## skills/clerk-cli/SKILL.md
## $SKILL_ROOT/SKILL.md
### <section name>
- [drift|gap|polish] <one-line description>
- source: packages/cli-core/src/commands/<...>:<line>
- <diff or before/after>

## skills/clerk-cli/references/<file>.md
## $SKILL_ROOT/references/<file>.md
...

## New files (if any)
Expand Down
69 changes: 68 additions & 1 deletion packages/cli-core/src/commands/users/create.test.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { test, expect, describe, beforeEach, afterEach, mock, spyOn } from "bun:test";
import { useCaptureLog } from "../../test/lib/stubs.ts";
import { BapiError, CliError, ERROR_CODE, EXIT_CODE } from "../../lib/errors.ts";
import { BapiError, CliError, ERROR_CODE, EXIT_CODE, UserAbortError } from "../../lib/errors.ts";

const mockResolveBapiSecretKey = mock();
const mockHandleBapiError = mock((_error: unknown) => false);
Expand All @@ -27,6 +27,11 @@ mock.module("./create-wizard.ts", () => ({
runCreateWizard: (...args: unknown[]) => mockRunCreateWizard(...args),
}));

const mockConfirm = mock();
mock.module("../../lib/prompts.ts", () => ({
confirm: (...args: unknown[]) => mockConfirm(...args),
}));

mock.module("../../lib/spinner.ts", () => ({
intro: () => {},
outro: () => {},
Expand All @@ -46,6 +51,7 @@ describe("users create", () => {
mockIsAgent.mockReturnValue(false);
mockResolveBapiSecretKey.mockResolvedValue("sk_test_123");
mockRunCreateWizard.mockResolvedValue({ fields: {}, targeting: {} });
mockConfirm.mockResolvedValue(true);
mockBapiRequest.mockResolvedValue({
status: 200,
headers: new Headers(),
Expand All @@ -64,6 +70,7 @@ describe("users create", () => {
mockBapiRequest.mockReset();
mockIsAgent.mockReset();
mockRunCreateWizard.mockReset();
mockConfirm.mockReset();
logSpy.mockRestore();
errorSpy.mockRestore();
});
Expand Down Expand Up @@ -260,4 +267,64 @@ describe("users create", () => {
expect(mockResolveBapiSecretKey).not.toHaveBeenCalled();
expect(mockBapiRequest).not.toHaveBeenCalled();
});

test("confirms before creating in human mode and redacts sensitive preview fields", async () => {
await runCreate({
app: "app_123",
email: "alice@example.com",
password: "Password123!",
});

expect(mockConfirm).toHaveBeenCalledWith({ message: "Proceed?" });
expect(captured.err).toContain("About to POST /v1/users");
expect(captured.err).toContain("[REDACTED]");
expect(captured.err).not.toContain("Password123!");
expect(mockBapiRequest).toHaveBeenCalled();
});

test("aborts without calling BAPI when the confirmation is declined", async () => {
mockConfirm.mockResolvedValue(false);

const error = await runCreate({
app: "app_123",
email: "alice@example.com",
}).catch((caught) => caught);

expect(error).toBeInstanceOf(UserAbortError);
expect(mockBapiRequest).not.toHaveBeenCalled();
});

test("skips the confirmation when --yes is passed", async () => {
await runCreate({
app: "app_123",
email: "alice@example.com",
yes: true,
});

expect(mockConfirm).not.toHaveBeenCalled();
expect(mockBapiRequest).toHaveBeenCalled();
});

test("never confirms in agent mode, with or without --yes", async () => {
mockIsAgent.mockReturnValue(true);

await runCreate({
app: "app_123",
email: "alice@example.com",
});

expect(mockConfirm).not.toHaveBeenCalled();
expect(mockBapiRequest).toHaveBeenCalled();
});
Comment on lines +308 to +318

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Exercise the --yes agent-mode case.

The test title promises coverage “with or without --yes,” but only invokes create without yes: true. Add a second invocation with yes: true, or narrow the title.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/cli-core/src/commands/users/create.test.ts` around lines 308 - 318,
Update the test “never confirms in agent mode, with or without --yes” to
explicitly invoke runCreate with yes: true in addition to the existing
invocation, and assert that agent mode still avoids confirmation while
performing the request.


test("does not confirm on --dry-run", async () => {
await runCreate({
app: "app_123",
email: "alice@example.com",
dryRun: true,
});

expect(mockConfirm).not.toHaveBeenCalled();
expect(mockBapiRequest).not.toHaveBeenCalled();
});
});
18 changes: 17 additions & 1 deletion packages/cli-core/src/commands/users/create.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
import { handleBapiError, resolveBapiSecretKey } from "../../lib/bapi-command.ts";
import { UserAbortError, isPromptExitError, throwUsageError } from "../../lib/errors.ts";
import {
UserAbortError,
isPromptExitError,
throwUsageError,
throwUserAbort,
} from "../../lib/errors.ts";
import { isInsideGutter, log } from "../../lib/log.ts";
import {
buildCreateUserPayload,
Expand All @@ -9,6 +14,7 @@ import {
redactUsersDisplayPayload,
} from "../../lib/users.ts";
import { isAgent, isHuman } from "../../mode.ts";
import { confirm } from "../../lib/prompts.ts";
import { bapiRequest } from "../../lib/bapi.ts";
import { withSpinner, intro, outro, pausedOutro } from "../../lib/spinner.ts";
import { handleUsersBapiError, printUsersMutationResult } from "./output.ts";
Expand Down Expand Up @@ -62,6 +68,16 @@ export async function create(options: CreateUserOptions): Promise<void> {
if (shouldWrap) intro("Creating user");

try {
if (isHuman() && !resolved.yes) {
log.info("\nAbout to POST /v1/users");
log.blank();
log.info(JSON.stringify(redactUsersDisplayPayload(payload), null, 2));
const ok = await confirm({ message: "Proceed?" });
if (!ok) {
throwUserAbort();
}
}

const response = await withSpinner("Creating user...", () =>
bapiRequest({
method: "POST",
Expand Down