Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion docs/specs/agent-plugins.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,14 @@ Last verified: 2026-08-07 against [Agent Plugins v1.0.0](https://agent-plugins.o

## What this repo does

Root `plugin.json` targets the Agent Plugins 1.0.0 manifest schema:
Root `plugin.json` follows the Agent Plugins 1.0.0 manifest authoring rules (field set and shapes) but **currently omits the `$schema` field**:

```text
https://agent-plugins.org/schemas/1.0.0/plugin.schema.json
```

**Why `$schema` is withheld (#1412):** Codex >= 0.147 ([openai/codex#37027](https://github.com/openai/codex/pull/37027)) treats a root `plugin.json` whose `$schema` starts with `https://agent-plugins.org/schemas/` as an Agent Plugin, and for Agent Plugin skills injects only the first `MAX_SKILL_PROMPT_BYTES` (8000) of each `SKILL.md` into the model-visible prompt, silently dropping the rest. Legacy manifests (`.codex-plugin/plugin.json`) are exempt. Most bundled skills exceed 8000 bytes, so shipping the `$schema` truncates them on Codex. `tests/codex-skill-prompt-budget.test.ts` pins this: it forbids the `$schema` while any skill is over budget, and holds a shrink-only allowlist of over-budget skills (CRLF-adjusted, since Windows checkouts inflate the byte count). Restore the `$schema` only once that allowlist is empty.

Layout already matches the portable package shape: root manifest + `skills/<name>/SKILL.md`. No `mcp.json` (valid — MCP is optional).

CI pins authoring rules in `tests/release-metadata.test.ts` (schema const, name pattern, closed field set, field shapes). Rules are pinned locally; tests never fetch the schema at runtime.
Expand Down Expand Up @@ -44,6 +46,8 @@ Agent Plugins discovers skills via the [Agent Skills](https://agentskills.io/spe

## Re-verify when

- Every `SKILL.md` fits Codex's 8000-byte prompt bound (then restore `$schema`)
- Codex changes `MAX_SKILL_PROMPT_BYTES` or applies it to legacy/host skills ([openai/codex#37463](https://github.com/openai/codex/issues/37463))
- Agent Plugins leaves Working Draft / publishes a new schema version
- Adding top-level fields to root `plugin.json`
- A concrete Agent Plugins client is observed to skip or reject skills with Claude-only frontmatter
Expand Down
1 change: 0 additions & 1 deletion plugin.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
{
"$schema": "https://agent-plugins.org/schemas/1.0.0/plugin.schema.json",
"name": "compound-engineering",
"version": "3.22.2",
"description": "Brainstorm, plan, debug, review, and compound learnings with AI agents",
Expand Down
107 changes: 107 additions & 0 deletions tests/codex-skill-prompt-budget.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
import { describe, expect, test } from "bun:test"
import { readdirSync, readFileSync, statSync } from "node:fs"
import path from "node:path"

/**
* Codex >= 0.147 (openai/codex#37027) classifies a plugin as an Agent Plugin when the
* root `plugin.json` carries an `https://agent-plugins.org/schemas/...` `$schema`, and
* then injects only the first MAX_SKILL_PROMPT_BYTES (8000) of each SKILL.md into the
* model-visible prompt (#1412). Legacy manifests (`.codex-plugin/plugin.json`) are exempt.
*
* Until every skill entrypoint fits, the root manifest must not carry that `$schema`,
* and no skill may newly cross the bound. Shrink OVER_BUDGET as skills are restructured;
* when it is empty, the `$schema` may return.
*/
const CODEX_MAX_SKILL_PROMPT_BYTES = 8_000
const AGENT_PLUGINS_SCHEMA_PREFIX = "https://agent-plugins.org/schemas/"

/**
* Skills known to exceed the bound. Membership is a set on purpose: an over-budget skill is
* already truncated on Codex, so its exact size is not pinned and ordinary edits do not churn
* this list. Remove a name once its SKILL.md fits; never add one for a new skill.
*/
const OVER_BUDGET = new Set([
"ce-babysit-pr",
"ce-brainstorm",
"ce-code-review",
"ce-commit-push-pr",
"ce-compound",
"ce-compound-refresh",
"ce-debug",
"ce-doc-review",
"ce-dogfood",
"ce-explain",
"ce-handoff",
"ce-ideate",
"ce-optimize",
"ce-plan",
"ce-pov",
"ce-product-pulse",
"ce-proof",
"ce-prototype",
"ce-resolve-pr-feedback",
"ce-retune",
"ce-setup",
"ce-strategy",
"ce-sweep",
"ce-test-browser",
"ce-work",
"lfg",
])

const repoRoot = path.join(import.meta.dir, "..")
const skillsDir = path.join(repoRoot, "skills")

/** Byte size as a Windows checkout with CRLF line endings would inject it. */
function crlfByteSize(contents: string): number {
const lf = contents.replace(/\r\n/g, "\n")
return Buffer.byteLength(lf, "utf8") + (lf.match(/\n/g)?.length ?? 0)
}

function skillSizes(): Map<string, number> {
const sizes = new Map<string, number>()
for (const name of readdirSync(skillsDir)) {
const file = path.join(skillsDir, name, "SKILL.md")
if (!statSync(path.join(skillsDir, name)).isDirectory()) continue
try {
sizes.set(name, crlfByteSize(readFileSync(file, "utf8")))
} catch {
// no SKILL.md; other tests own that invariant
}
}
return sizes
}

describe("Codex skill prompt budget (#1412)", () => {
const sizes = skillSizes()

test("no skill newly exceeds Codex's 8000-byte prompt bound (CRLF-adjusted)", () => {
const violations: string[] = []
for (const [name, size] of sizes) {
if (size > CODEX_MAX_SKILL_PROMPT_BYTES && !OVER_BUDGET.has(name)) {
violations.push(`${name}: ${size} bytes > ${CODEX_MAX_SKILL_PROMPT_BYTES}`)
}
}
expect(violations).toEqual([])
})

test("OVER_BUDGET only lists skills that still exceed the bound (ratchet down)", () => {
const stale = [...OVER_BUDGET].filter(
(name) => (sizes.get(name) ?? 0) <= CODEX_MAX_SKILL_PROMPT_BYTES,
)
expect(stale).toEqual([])
})

test("root plugin.json omits the Agent Plugins $schema while any skill is over budget", () => {
const manifest = JSON.parse(
readFileSync(path.join(repoRoot, "plugin.json"), "utf8"),
) as Record<string, unknown>
const schema = typeof manifest.$schema === "string" ? manifest.$schema : ""
const anyOverBudget = [...sizes.values()].some(
(size) => size > CODEX_MAX_SKILL_PROMPT_BYTES,
)
if (anyOverBudget) {
expect(schema.startsWith(AGENT_PLUGINS_SCHEMA_PREFIX)).toBe(false)
}
})
})
6 changes: 4 additions & 2 deletions tests/release-metadata.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -937,8 +937,10 @@ const AGENT_PLUGINS_STRING_FIELDS = [
function agentPluginsManifestErrors(manifest: Record<string, unknown>): string[] {
const errors: string[] = []

if (manifest.$schema !== AGENT_PLUGINS_SCHEMA) {
errors.push(`$schema must be ${AGENT_PLUGINS_SCHEMA}`)
// $schema is deliberately absent while any SKILL.md exceeds Codex's 8000-byte
// Agent Plugin prompt bound (see tests/codex-skill-prompt-budget.test.ts, #1412).
if (manifest.$schema !== undefined && manifest.$schema !== AGENT_PLUGINS_SCHEMA) {
errors.push(`$schema must be ${AGENT_PLUGINS_SCHEMA} when present`)
}

if (typeof manifest.name !== "string") {
Expand Down