Skip to content

Commit 2a153e4

Browse files
committed
feat(rules): generate and improve rules through the v2 API, verifying every served file
1 parent cb18b3a commit 2a153e4

21 files changed

Lines changed: 1207 additions & 852 deletions

‎openspec/changes/cli-v2-rule-api/tasks.md‎

Lines changed: 17 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -49,28 +49,29 @@ upgradeUrl }`, strip C0/C1 control characters except newline from
4949

5050
## 3. Generation on v2 (slice 2)
5151

52-
- [ ] 3.1 Add `rules/verify-delivery.ts`: verify a served file set against its
52+
- [x] 3.1 Add `rules/verify-delivery.ts`: verify a served file set against its
5353
`signatures` (every signature names a file, every non-`.tests/` file has
5454
one, each hash matches, runtime `signature` equals the `check.ts` entry)
5555
and that `rules` holds exactly one set whose `id` is the requested id.
5656
Unit tests for each refusal.
57-
- [ ] 3.2 Make `writeDeliveredFileSet` the only write path for a served rule and
58-
make it replace the directory (purge files the set lacks, `.tests/`
59-
included; create each file's parent directories).
60-
Drop the legacy single-`content` branch from `deliver.ts` and
61-
`files.ts`. `deliver.test.ts` covers a local extra capture being removed.
62-
- [ ] 3.3 Move `rule create` to v2: submit, poll, fetch each produced
57+
- [x] 3.2 Make `writeDeliveredFileSet` the only write path for a served rule
58+
(`writeServedRule`) and make it replace the directory (purge files the
59+
set lacks, `.tests/` included; create each file's parent directories).
60+
`deliver.test.ts` covers a stale fixture and a local extra capture being
61+
removed. The legacy single-`content` branch still has a caller in the v1
62+
repair path until 5.3, so it is dropped in 8.1.
63+
- [x] 3.3 Move `rule create` to v2: submit, poll, fetch each produced
6364
`{ ruleId, revisionId }` head in parallel without `revision`, confirm
6465
`revisionId`, verify, write. Print `error` verbatim (sanitized) on
6566
`failed` / `unsupported`. `--json` prints `requestId` and `rules`, no
6667
`ruleId`; update `schemas/rules-create.ts`. Tests use a stubbed v2 server.
67-
- [ ] 3.4 Move `rule improve` to `POST v2/rule/{ruleId}/iterate`, with the input
68+
- [x] 3.4 Move `rule improve` to `POST v2/rule/{ruleId}/iterate`, with the input
6869
`ruleId` meaning the directory name; `404 rule_not_found` →
6970
`RULE_NOT_FOUND`. Tests cover success and the not-found code.
70-
- [ ] 3.5 Keep the write-time entitlement warning for runtime sets served with
71+
- [x] 3.5 Keep the write-time entitlement warning for runtime sets served with
7172
`runtimeSignatures: false`; `rule-create-entitlement.test.ts` passes
7273
against v2 fixtures.
73-
- [ ] 3.6 Update the `create-remote-rule`, `improve-rule`, and `rule-meta`
74+
- [x] 3.6 Update the `create-remote-rule`, `improve-rule`, and `rule-meta`
7475
recipes: record the rule ids from `rules`, pass a directory name to
7576
`improve`, never the request id. `recipe-cross-references.test.ts` passes.
7677

@@ -156,8 +157,12 @@ upgradeUrl }`, strip C0/C1 control characters except newline from
156157

157158
## 8. Retire v1 (slice 5)
158159

159-
- [ ] 8.1 Delete `api/rules.ts`, `api/reconcile.ts`, `api/restore.ts`, the
160-
frozen v1 `api.schema.json` / `api.d.ts`, and every v1 type use; move `auth/whoami.ts` and `auth/org.ts` to v2 whoami. Verify
160+
- [ ] 8.1 Delete `api/rules.ts` (its v1 request, poll, and iterate calls went
161+
in slice 2), `api/reconcile.ts`, `api/restore.ts`, the frozen v1
162+
`api.schema.json` / `api.d.ts`, the legacy single-`content` path in
163+
`files.ts` / `deliver.ts` (`writeRuleFile`, `writeRuleTestFile`,
164+
`writeRuleMetaFiles`, `deliveredFiles`, `resolveIngestEngine`), and
165+
every v1 type use; move `auth/whoami.ts` and `auth/org.ts` to v2 whoami. Verify
161166
`grep -rn "/cli/api/" packages/cli/src` finds only `/cli/api/v2/` paths.
162167
- [ ] 8.2 Add a vite build check (per the code style guide, not a test that
163168
scans output) that fails the build if the bundle contains a `/cli/api/`

‎packages/cli/src/agent/create-remote-rule.md‎

Lines changed: 15 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Topic: create-remote-rule (CLI v%(CLI_VERSION)s / topic v3)
1+
# Topic: create-remote-rule (CLI v%(CLI_VERSION)s / topic v4)
22

33
## You are here
44
This is `create-remote-rule`. It helps you have the Taskless service
@@ -125,17 +125,20 @@ Two ways to legitimately be here:
125125
8. **Clean up.** Delete `.taskless/.tmp-rule-request.json` whether the
126126
call succeeded or failed.
127127

128-
9. **Report.** The service writes the rule to
129-
`.taskless/rules/sg/<id>/<id>.yml` and its tests to
130-
`.taskless/rules/sg/<id>/.tests/<id>-YYYYMMDD-test.yml`. These are
131-
the same paths and the same shape a locally authored rule uses, so
132-
`check`, `improve-rule`, `verify`, and `test` treat them
133-
identically. Nothing is written under `.taskless/rule-metadata/`.
134-
Show the user the paths and suggest `%(TASKLESS_CLI)s agent check`.
135-
136-
**Record the `ruleId` from the `--json` output.** It is the ticket
137-
id the iterate endpoint is addressed by, `improve-rule` asks for it,
138-
and no file on disk carries it.
128+
9. **Report.** The CLI writes each generated rule, fixtures included,
129+
to `.taskless/rules/<engine>/<id>/`, replacing anything already in
130+
that directory. These are the same paths and the same shape a
131+
locally authored rule uses, so `check`, `improve-rule`, `verify`, and
132+
`test` treat them identically. Nothing is written under
133+
`.taskless/rule-metadata/`. Show the user the paths and suggest
134+
`%(TASKLESS_CLI)s agent check`.
135+
136+
**`rules` in the `--json` output lists each written rule's id**, and
137+
the id is the rule's directory name (for example
138+
`no-eval-3fa9c21b`). It is what `rule improve`, `rule restore`, and
139+
`rule rollback` take, and it is on disk, so nothing needs recording.
140+
`requestId` names the generation request only; no command takes it
141+
back, and passing it to `rule improve` fails with `RULE_NOT_FOUND`.
139142

140143
**Read `notices` if it is present.** It is an optional array of
141144
advisory messages about a delivery that was written anyway. A rule

‎packages/cli/src/agent/improve-rule.md‎

Lines changed: 20 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
1-
# Topic: improve-rule (CLI v%(CLI_VERSION)s / topic v5)
1+
# Topic: improve-rule (CLI v%(CLI_VERSION)s / topic v6)
22

33
## Goal
44
Iterate on an existing Taskless rule. The CLI submits the user's
55
guidance to the Taskless API iterate endpoint, which returns an
66
updated rule that overwrites the original on disk. The agent's job
7-
is to gather the right ruleId + guidance + supporting references and
7+
is to gather the right rule id + guidance + supporting references and
88
to report the result.
99

1010
If the user wants the local-only flow (no API call), fetch
@@ -18,33 +18,28 @@ If the user wants the local-only flow (no API call), fetch
1818
`[unknown]`, stop and say the tier is unavailable rather than
1919
submitting. `auth login` does not fix it, no GitHub owner is a
2020
property of the project, not the session.
21-
- The target rule exists at `.taskless/rules/sg/<id>/<id>.yml`.
22-
- You have the rule's **ticket id**: the value `%(TASKLESS_CLI)s rule
23-
create --json` printed as `ruleId` when the rule was generated. The
24-
iterate endpoint is addressed by that id. Nothing on disk holds it,
25-
so it comes from the create output or from the user. Without it,
26-
fetch the anonymous variant instead.
21+
- The target rule exists at `.taskless/rules/<engine>/<id>/`, and the
22+
Taskless service issued it. Its **rule id is its directory name**
23+
(for example `no-eval-3fa9c21b`), which is what the iterate endpoint
24+
is addressed by. A rule you wrote locally, or one generated before
25+
CLI 0.12.0, is not known to the service: improving it fails with
26+
`RULE_NOT_FOUND`, so fetch the anonymous variant instead.
2727

2828
## Steps
2929

3030
1. **Confirm auth.** Run `%(TASKLESS_CLI)s info --json` and check
3131
`loggedIn`. If false, fetch `%(TASKLESS_CLI)s agent auth`.
3232

3333
2. **Identify the rule to improve.** If the user named one, use it.
34-
Otherwise, list rules in `.taskless/rules/sg/` and ask which one.
35-
Read the existing rule file so you can summarize what it does.
34+
Otherwise, list the rule directories under `.taskless/rules/<engine>/`
35+
and ask which one. Read the existing rule files so you can summarize
36+
what the rule does.
3637

37-
3. **Get the ticket id.** This is the id the iterate endpoint is
38-
addressed by, and it is the `ruleId` field from that rule's
39-
`%(TASKLESS_CLI)s rule create --json` output. Take it from the
40-
session that created the rule, or ask the user for it.
41-
42-
**Do not run `%(TASKLESS_CLI)s rule meta <id>` to get it.** That
43-
command reads `.taskless/rule-metadata/<id>.yml`, a sidecar this CLI
44-
never writes, so it exits 1 with `RULE_META_UNAVAILABLE` for every
45-
rule. If no one has the ticket id, fetch
46-
`%(TASKLESS_CLI)s agent improve-rule --anonymous` and iterate
47-
locally.
38+
3. **Take the rule id from the directory name.** It is the `<id>` in
39+
`.taskless/rules/<engine>/<id>/`, and the same value
40+
`%(TASKLESS_CLI)s rule create --json` listed in `rules`. It is never
41+
the `requestId` that command printed. You do not need
42+
`%(TASKLESS_CLI)s rule meta` for it; that command has nothing to read.
4843

4944
4. **Gather improvement guidance.** Ask the user what should change:
5045
- Are there false positives we need to exclude?
@@ -116,9 +111,9 @@ The `--from` JSON file conforms to:
116111
%(INPUT_SCHEMA)s
117112
```
118113

119-
`ruleId` is the original rule's ticket ID, printed as `ruleId` by
120-
`%(TASKLESS_CLI)s rule create --json`. It is not the YAML file name,
121-
and it is not readable from anything under `.taskless/`.
114+
`ruleId` is the rule's directory name under `.taskless/rules/<engine>/`,
115+
as `%(TASKLESS_CLI)s rule create --json` lists it in `rules`. It is not
116+
the `requestId` from that output.
122117

123118
## Errors
124119

@@ -132,7 +127,7 @@ When `--json` is set, failures emit `{ ok: false, code, message }`:
132127
| `NO_ORIGIN_REMOTE` | git repository, no `origin` | tell the user; `auth login` cannot fix it |
133128
| `UNSUPPORTED_REMOTE_HOST`| `origin` is not GitHub | tell the user; `auth login` cannot fix it |
134129
| `INVALID_INPUT` | `--from` JSON failed validation | re-read input schema, fix, retry |
135-
| `RULE_NOT_FOUND` | the service has no such ticket id | re-check the id from `rule create --json` |
130+
| `RULE_NOT_FOUND` | the service did not issue this rule | re-check the directory name; a local or pre-0.12.0 rule needs the anonymous flow |
136131
| `NETWORK_ERROR` | API submit/poll failed | report and suggest retry |
137132
| `RULE_GENERATION_FAILED` | API returned a generation failure | report; suggest enriching guidance/references |
138133
| `RULE_UNSUPPORTED` | plan lacks this generation type | tell the user to enable it; do not retry |

‎packages/cli/src/agent/rule-meta.md‎

Lines changed: 8 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Topic: rule-meta (CLI v%(CLI_VERSION)s / topic v3)
1+
# Topic: rule-meta (CLI v%(CLI_VERSION)s / topic v4)
22

33
## Goal
44
Report what `%(TASKLESS_CLI)s rule meta` does today, so no recipe and no
@@ -18,22 +18,19 @@ retry with a different id; there is no id that works.
1818

1919
## What to do instead
2020

21-
`rule improve` needs the ticket id, and that id comes from the machine
22-
that created the rule, not from disk:
23-
24-
- `%(TASKLESS_CLI)s rule create --json` prints it as `ruleId` on
25-
success. Record it when you create a rule you expect to iterate on.
26-
- If the id was not recorded, ask the user for it.
27-
- If nobody has it, iterate locally: fetch
28-
`%(TASKLESS_CLI)s agent improve-rule --anonymous`.
21+
Nothing you would have used it for needs it. `rule improve`,
22+
`rule restore`, and `rule rollback` take a rule's id, and the id is the
23+
rule's directory name under `.taskless/rules/<engine>/`, which is on
24+
disk. `%(TASKLESS_CLI)s rule create --json` lists the same ids in
25+
`rules`.
2926

3027
## Errors
3128

3229
| code | meaning | fix |
3330
|-------------------------|------------------------------------------|---------------------------------------|
34-
| `RULE_META_UNAVAILABLE` | no sidecar exists, and none is written | Use the ticket id from `rule create` |
31+
| `RULE_META_UNAVAILABLE` | no sidecar exists, and none is written | Use the rule's directory name |
3532
| `INVALID_INPUT` | a sidecar exists and is malformed | Delete it; nothing here depends on it |
3633

3734
## See Also
3835

39-
- `%(TASKLESS_CLI)s agent improve-rule`: how the ticket id is actually sourced
36+
- `%(TASKLESS_CLI)s agent improve-rule`: iterating a rule by its id

‎packages/cli/src/agent/rule.md‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Topic: rule (CLI v%(CLI_VERSION)s / topic v2)
1+
# Topic: rule (CLI v%(CLI_VERSION)s / topic v3)
22

33
## Goal
44
Umbrella for rule operations. Fetch the topic for the action you want.
@@ -15,7 +15,8 @@ Umbrella for rule operations. Fetch the topic for the action you want.
1515

1616
`rule meta` reads a sidecar this CLI never writes, so it fails for every
1717
rule. Fetch `rule-meta` only to learn what to do instead. `improve-rule`
18-
takes the ticket id from `rule create --json`.
18+
takes the rule's id, which is its directory name under
19+
`.taskless/rules/<engine>/`.
1920

2021
`route` is the entry point for authoring: it reads the request and
2122
names the `create-*-rule` topic that fits, so you do not pick an engine

‎packages/cli/src/api/rules.ts‎

Lines changed: 0 additions & 160 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,4 @@
11
import type { paths } from "../generated/api";
2-
import { createApiClient } from "./client";
3-
import { CLIError } from "../util/cli-error";
4-
import { getCliPrefix } from "../util/package-manager";
52

63
// --- Types extracted from the generated schema ---
74

@@ -48,160 +45,3 @@ export function isSingleContentRule(
4845
): rule is SingleContentRule {
4946
return !isFileSetRule(rule);
5047
}
51-
52-
// --- Helpers ---
53-
54-
/** Extract error details from an untyped error response body */
55-
function parseErrorBody(rawError: unknown): Record<string, unknown> {
56-
if (rawError && typeof rawError === "object") {
57-
return rawError as Record<string, unknown>;
58-
}
59-
return {};
60-
}
61-
62-
/**
63-
* The server returns the same 404 `organization_not_found` whether the org
64-
* isn't yours or its GitHub App installation doesn't cover this repository (it
65-
* deliberately doesn't distinguish, to avoid leaking org existence), so the
66-
* message names both causes — coverage first, since a resolved org subject
67-
* makes membership the less likely one.
68-
*/
69-
function orgNotFoundMessage(): string {
70-
return [
71-
"Taskless could not act on this repository for your organization.",
72-
"",
73-
"Most often the organization's Taskless GitHub App installation does not cover this repository. It can also mean your login no longer has access to the organization.",
74-
"",
75-
"- Confirm the Taskless app is installed on this repository's owner and includes this repository.",
76-
`- If access recently changed, re-authenticate with \`${getCliPrefix()} auth login\`.`,
77-
].join("\n");
78-
}
79-
80-
// --- API functions ---
81-
82-
/** Submit a new rule generation request */
83-
export async function submitRule(
84-
token: string,
85-
request: {
86-
/** Org subject: Taskless UUID (preferred) or numeric GitHub org id. */
87-
orgId: string | number;
88-
repositoryUrl: string;
89-
prompt: string;
90-
successCases?: string[];
91-
failureCases?: string[];
92-
}
93-
) {
94-
const client = createApiClient(token);
95-
const { data, error, response } = await client.POST("/cli/api/request", {
96-
body: request,
97-
});
98-
99-
if (!data) {
100-
const errorData = parseErrorBody(error);
101-
if (response.status === 400 && errorData.error === "validation_error") {
102-
const details = (errorData.details as string[]) ?? [];
103-
throw new Error(`Validation error: ${details.join(", ")}`);
104-
}
105-
if (
106-
response.status === 403 &&
107-
errorData.error === "repository_not_accessible"
108-
) {
109-
throw new Error(
110-
[
111-
"Repository is not accessible to this organization.",
112-
"",
113-
"- Verify that your local `origin` remote points to the intended GitHub repository.",
114-
"- Confirm that your GitHub user/organization has access to that repository.",
115-
`- If you recently changed access or remotes, try re-authenticating with \`${getCliPrefix()} auth login\`.`,
116-
].join("\n")
117-
);
118-
}
119-
if (
120-
response.status === 404 &&
121-
errorData.error === "organization_not_found"
122-
) {
123-
throw new Error(orgNotFoundMessage());
124-
}
125-
throw new Error(
126-
`Request submission failed (HTTP ${String(response.status)})`
127-
);
128-
}
129-
130-
return data;
131-
}
132-
133-
/** Poll for rule generation status */
134-
export async function pollRuleStatus(token: string, requestId: string) {
135-
const client = createApiClient(token);
136-
const { data, error, response } = await client.GET(
137-
"/cli/api/request/{requestId}",
138-
{
139-
params: { path: { requestId } },
140-
}
141-
);
142-
143-
if (!data) {
144-
const errorData = parseErrorBody(error);
145-
if (response.status === 403 && errorData.error === "access_denied") {
146-
throw new Error("Access denied to this request.");
147-
}
148-
if (response.status === 404 && errorData.error === "request_not_found") {
149-
throw new Error("Request not found. It may have expired.");
150-
}
151-
throw new Error(`Status polling failed (HTTP ${String(response.status)})`);
152-
}
153-
154-
return data;
155-
}
156-
157-
/** Submit an improve/iterate request for an existing rule */
158-
export async function iterateRule(
159-
token: string,
160-
requestId: string,
161-
request: {
162-
/** Org subject: Taskless UUID (preferred) or numeric GitHub org id. */
163-
orgId: string | number;
164-
guidance: string;
165-
references?: Array<{ filename: string; content: string }>;
166-
}
167-
) {
168-
const client = createApiClient(token);
169-
const { data, error, response } = await client.POST(
170-
"/cli/api/request/{requestId}/iterate",
171-
{
172-
params: { path: { requestId } },
173-
body: request,
174-
}
175-
);
176-
177-
if (!data) {
178-
const errorData = parseErrorBody(error);
179-
if (response.status === 400 && errorData.error === "validation_error") {
180-
const details = (errorData.details as string[]) ?? [];
181-
throw new Error(`Validation error: ${details.join(", ")}`);
182-
}
183-
if (response.status === 403 && errorData.error === "access_denied") {
184-
throw new Error("Access denied to this request.");
185-
}
186-
if (response.status === 404 && errorData.error === "request_not_found") {
187-
// The caller supplied this ticket id (from `rule create --json`, or from
188-
// the user), so "no such id" is a state they can act on: re-check the id.
189-
// The code travels on the error so `improveCommand` reports
190-
// RULE_NOT_FOUND rather than folding it into NETWORK_ERROR, which would
191-
// tell an agent to retry an id that will never resolve.
192-
throw new CLIError(
193-
"Rule not found. It may have expired.",
194-
"RULE_NOT_FOUND"
195-
);
196-
}
197-
if (
198-
response.status === 404 &&
199-
errorData.error === "organization_not_found"
200-
) {
201-
throw new Error(orgNotFoundMessage());
202-
}
203-
throw new Error(`Iterate request failed (HTTP ${String(response.status)})`);
204-
}
205-
206-
return data;
207-
}

0 commit comments

Comments
 (0)