Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,10 @@ opensrc/
CHANGES.md
**/CHANGES.md

# Private-term overlay for the public skill sync - never commit.
# See public-manifest.local.json.example.
public-manifest.local.json

# Personal terms list for gitleaks — never commit. See .gitleaks.local.toml.example.
.gitleaks.local.toml

Expand Down
8 changes: 7 additions & 1 deletion .gitleaks.toml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,10 @@ tags = ["pii", "path"]
[[rules]]
id = "rfc1918-ipv4"
description = "Private IPv4 address (homelab/LAN). Use a generic example or env var."
regex = '''\b(?:10|172\.(?:1[6-9]|2\d|3[01])|192\.168)\.\d{1,3}\.\d{1,3}\b'''
# All four octets are required in every branch. The earlier `10|...` alternation
# made the 10/8 branch three-part, so it matched semver strings ("10.0.2" in any
# lockfile) far more often than it matched an address.
regex = '''\b(?:10(?:\.\d{1,3}){3}|172\.(?:1[6-9]|2\d|3[01])(?:\.\d{1,3}){2}|192\.168(?:\.\d{1,3}){2})\b'''
tags = ["pii", "network"]

[[rules]]
Expand All @@ -53,6 +56,9 @@ paths = [
'''(^|/)node_modules/''',
'''(^|/)\.gitleaks\.toml$''', # This file (regex literals would self-match)
'''(^|/)\.gitleaks\.local\.toml(\.example)?$''',
# Same reason: this file's job is to hold example leaks and assert they are
# detected. Its fixtures are invented, never copied from a real machine.
'''(^|/)scripts/sync-public\.test\.mjs$''',
'''(^|/)\.github/workflows/[^/]+\.ya?ml$''', # CI configs may reference 192.168.x.x in matrix
]

Expand Down
2 changes: 2 additions & 0 deletions biome.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@
"rules/**",
"scripts/**",
"skills/**",
"!skills/*/assets",
"!skills/*/templates",
"!skills/art/Tools",
"!skills/art/Lib",
"themes/**",
Expand Down
119 changes: 119 additions & 0 deletions docs/public-sync.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
# Public skill sync

Most skills under `skills/` are **generated** from a private skill store. You edit
the skill once, in the store, and this repo is rebuilt from it with private detail
removed. `public-manifest.json` is the contract; `scripts/sync-public.mjs` executes it.

Skills the manifest marks `public-owned` or `forked` are authored here and are never
touched by the sync.

## Two manifests, on purpose

`public-manifest.json` is tracked and holds only **structural** patterns: shapes that are
private regardless of who maintains the repo, like a macOS home path or an RFC1918 address.

`public-manifest.local.json` is **gitignored** and holds every pattern or replacement that
**names** something: your machines, private repos, email domains, internal framework paths.
An enumeration of those names is itself metadata about your private ecosystem, so shipping
the list publicly would leak the very thing the list exists to protect. This mirrors the
`.gitleaks.toml` / `.gitleaks.local.toml` split the repo already uses.

The sync merges them (overlay patterns appended, overlay replacements last) and **exits 1
if the overlay is missing**. A scrub that quietly covers less than intended is worse than
one that refuses to run. Copy `public-manifest.local.json.example` to get started.

## Commands

```bash
bun scripts/sync-public.mjs --check # report drift, write nothing (exit 1 on drift)
bun scripts/sync-public.mjs # write
bun scripts/sync-public.mjs --prune # also delete orphaned generated files
bun scripts/sync-public.mjs --skill find-docs
```

The store defaults to `~/.agents/skills`. Point `AGENT_SKILLS_STORE` elsewhere to
override it. When the store is absent, `--check` exits 0 with a notice saying drift
was **not** verified, so contributors without the store are not blocked. Only the
golden tests run everywhere.

## Modes

| Mode | Who is canonical | What sync does |
|---|---|---|
| `mirror` | the store | Copies and transforms store files into `skills/<name>/`. |
| `public-owned` | this repo | Skipped entirely. For skills authored here with no private counterpart. |
| `forked` | both, deliberately | Skipped, with a required `reason` recording why they diverged. |

`mirror` entries may declare:

- `source`: the store directory name, when it differs from the public one
(`git-workflow` in the store, `gitworkflow` here).
- `publicOwned`: globs the sync must never write **or** delete. This is what lets a
public skill ship tooling (`scripts/`, `fixtures/`, `agents/openai.yaml`) that the
store does not carry.
- `exclude`: globs dropped from the published output entirely. For files that are
private by nature rather than by wording.
- `replace` / `dropLines`: per-skill text rules, applied after the global ones.
`dropLines` removes whole lines matching a regex, which is how a table row pointing
at an excluded file gets removed.

An entry that is not `mirror` must record a `reason`. A test enforces that, so the
reasons stay readable as a record of why two copies diverged.

## The four gates

Being "verifiable" here means a transform gap fails a run rather than shipping.

1. **Leak assertion** (`scripts/sync-public.mjs`). Every transformed text file is
scanned against `leakPatterns` *after* transforms. Any hit aborts that skill with a
`file:line [pattern-id] excerpt` report and writes nothing. This is the backstop for
a replacement rule that was never written, which is the failure mode a find-and-replace
pipeline otherwise hides. Note that only extensions in `textExtensions` are scanned:
anything else is copied byte-for-byte, so add an extension before publishing a new
file type.

2. **Excluded-reference check.** Excluding a private file is fine. Leaving the published
SKILL.md pointing at it ships a broken skill, so any published text that still names
an excluded path aborts the run. Resolve it by publishing the target, or by dropping
the pointer with `dropLines` / `replace`.

3. **Golden tests** (`scripts/sync-public.test.mjs`, part of `bun run test`). The
transform chain and every leak pattern are tested against fixtures, so the scrub
itself is proven rather than trusted. These run without the store.

4. **gitleaks pre-commit**. `.gitleaks.toml` (tracked, structural PII) plus
`.gitleaks.local.toml` (gitignored, per-contributor terms). Independent of the sync,
so it also covers hand-edited files.

Run `--check` on a schedule to catch staleness. Nothing in `bun run check` depends on
the store being present.

## Formatting owns nothing generated

`biome.json` excludes `skills/*/assets/**` and `skills/*/templates/**`. Those are payloads
a skill ships, and their formatting belongs to whoever wrote them. Without the exclusion,
`lint:fix` rewrites generated files and the very next `--check` reports drift that no one
introduced.

## Circular sources

A store entry that is a symlink **into this repo** makes source and destination the
same bytes, so "syncing" it means no scrub ever ran. The script detects this and
refuses:

```
diataxis-docs-site: store entry resolves inside this repo (…/skills/diataxis-docs-site).
It is still a symlink into the public checkout, so no scrub can run.
Materialize it in the store first, then re-run.
```

Fix it by replacing the store symlink with a real directory holding the content, then
repointing any harness lane symlinks at the store rather than at this repo.

## Adding a skill

1. Add an entry to `public-manifest.json` with a mode. Non-`mirror` modes require a `reason`.
2. `bun scripts/sync-public.mjs --skill <name> --check` and read the plan.
3. Resolve any leak findings by fixing the source, adding a transform, or excluding the file.
4. Sync, then run `npm test`. A skill with content tests will tell you if the store
version and the published tooling have drifted apart.
6 changes: 4 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -54,10 +54,12 @@
"lint": "biome lint .",
"lint:fix": "biome check --write .",
"prepare": "husky",
"test": "node extensions/autopilot/v2-smoke-test.mjs && node extensions/question/question-smoke-test.mjs && node extensions/conditional-hooks/smoke-test.mjs && node scripts/lib/frontmatter-test.mjs && node scripts/lib/bundle-refs-test.mjs && node scripts/cli-entrypoint-test.mjs && python3 skills/diataxis-docs-site/tests/test_create_site.py && bun test scripts/lib/package-links.test.mjs scripts/validate-lockfile.test.mjs && bun test skills/pr-review-queue/ && bun run test:herdr-fleet",
"test": "node extensions/autopilot/v2-smoke-test.mjs && node extensions/question/question-smoke-test.mjs && node extensions/conditional-hooks/smoke-test.mjs && node scripts/lib/frontmatter-test.mjs && node scripts/lib/bundle-refs-test.mjs && node scripts/cli-entrypoint-test.mjs && python3 skills/diataxis-docs-site/tests/test_create_site.py && bun test scripts/lib/package-links.test.mjs scripts/validate-lockfile.test.mjs scripts/sync-public.test.mjs && bun test skills/pr-review-queue/ && bun run test:herdr-fleet",
"test:herdr-fleet": "bun skills/herdr-fleet/scripts/resolve-project-key.mjs --self-test && bun skills/herdr-fleet/scripts/watch-fleet.mjs --self-test && bun skills/herdr-fleet/scripts/consume-events.mjs --self-test && bun test skills/herdr-fleet/skill-content.test.mjs skills/herdr-fleet/scripts/fleet-state.test.mjs skills/herdr-fleet/scripts/review-thread-gate.test.mjs",
"typecheck": "tsc --noEmit",
"validate:skills": "bun scripts/validate-agent-skills.mjs"
"validate:skills": "bun scripts/validate-agent-skills.mjs",
"sync:public": "bun scripts/sync-public.mjs",
"sync:public:check": "bun scripts/sync-public.mjs --check"
},
"peerDependencies": {
"@mariozechner/pi-ai": "*",
Expand Down
153 changes: 153 additions & 0 deletions public-manifest.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
{
"$comment": "Drives scripts/sync-public.mjs. The private skill store is canonical for mirror skills; this repo is generated from it. Patterns and replacements that NAME private things live in the gitignored localOverlay, never here: the list itself is metadata about a private ecosystem. See docs/public-sync.md.",
"storeRoot": "~/.agents/skills",
"storeRootEnv": "AGENT_SKILLS_STORE",
"publicSkillsDir": "skills",
"textExtensions": [
".md",
".mjs",
".js",
".ts",
".json",
".yaml",
".yml",
".hbs",
".py",
".rb",
".sh",
".toml",
".txt",
".cfg",
".ini",
".env",
".example"
],
"transforms": {
"$comment": "Applied in order to every text file: frontmatter keys dropped, then the skill name rewritten, then literal replacements.",
"dropFrontmatterKeys": [
"metadata.machines",
"metadata.requires"
],
"replace": [
{
"find": "~/.claude",
"replaceWith": "$AGENT_HOME"
},
{
"find": "Ossie",
"replaceWith": "the user"
}
]
},
"leakPatterns": [
{
"id": "macos-home-path",
"description": "Contributor home directory path",
"regex": "/Users/(?!(?:you|me|user|username|name)/)[a-z][a-z0-9._-]*/"
},
{
"id": "rfc1918-ipv4",
"description": "Private LAN address. All four octets are required; a three-part `10.0.2` is a semver, not an address.",
"regex": "\\b(?:10(?:\\.\\d{1,3}){3}|172\\.(?:1[6-9]|2\\d|3[01])(?:\\.\\d{1,3}){2}|192\\.168(?:\\.\\d{1,3}){2})\\b"
}
],
"skills": {
"adversarial-review": {
"mode": "mirror",
"publicOwned": [
"agents/**"
]
},
"deep-dive": {
"mode": "mirror",
"publicOwned": [
"agents/**"
]
},
"find-docs": {
"mode": "mirror",
"publicOwned": [
"agents/**"
]
},
"diataxis-docs-site": {
"mode": "mirror",
"$comment": "Was a store symlink into this repo until 2026-08-18; store is now canonical."
},
"github-wiki": {
"mode": "mirror",
"$comment": "Was a store symlink into this repo until 2026-08-18; store is now canonical."
},
"gitworkflow": {
"mode": "mirror",
"source": "git-workflow",
"publicOwned": [
"agents/**",
"AGENT.md",
"templates/issue-labeler.yml"
],
"exclude": [
"workflows/SetIdentity.md",
"tools/**"
],
"$comment": "SetIdentity hardcodes real names and email addresses, including a third party's. tools/changelog is a local copy of this repo's own root changelog/ package, so published text points at the package instead.",
"dropLines": [
"\\|\\s*\\*\\*SetIdentity\\*\\*\\s*\\|"
],
"replace": [
{
"find": "Wrapper if not on PATH: `$AGENT_HOME/skills/git-workflow/tools/changelog/changelog` (bundled source in `tools/changelog/`, needs `uv`). Canonical home of the installed CLI: public",
"replaceWith": "Install the CLI from"
},
{
"find": "# or $AGENT_HOME/skills/GitWorkflow/tools/changelog/changelog",
"replaceWith": "# install with: uv tool install --from git+https://github.com/AojdevStudio/agentic-utilities#subdirectory=changelog changelog"
}
]
},
"harness-audit": {
"mode": "mirror",
"exclude": [
"references/sanity-check.md"
],
"$comment": "sanity-check.md cites two private-vault pages as its source of truth, so it is useless outside that vault.",
"publicOwned": [
"agents/**",
"references/evidence-protocol.md",
"references/smoke-ticket-eval.md",
"references/stack-go.md",
"references/symphony-readiness.md",
"references/workflow-template.md"
]
},
"herdr-fleet": {
"mode": "forked",
"reason": "Public version ships 18 files (protocols.md, launch-fleet.md, tested scripts/) that its SKILL.md documents. The private counterpart is a lean rewrite composing native herdr commands directly and references none of them, so syncing it orphans the tooling."
},
"pr-review-queue": {
"mode": "forked",
"reason": "Public version's SKILL.md carries a documented untrusted-data boundary and an explicit-assignment requirement that skills/pr-review-queue/skill-content.test.mjs enforces. The private counterpart dropped both and does not document the shipped scripts/ helpers; syncing it fails 5 content assertions."
},
"art": {
"mode": "forked",
"reason": "Private counterpart is a superset coupled to a private agent framework (extra Tools, 100+ private-path references). Reconciling it is a rewrite, not a transform."
},
"awesome-readme": {
"mode": "forked",
"reason": "Private counterpart was restructured into the nested TitleCase Workflows/References layout; public keeps the flat kebab-case shape."
},
"bambu-slicer": {
"mode": "public-owned",
"reason": "Authored here. No private counterpart."
},
"scaffold-notes": {
"mode": "public-owned",
"reason": "Maintenance helper for this repo only."
},
"harness-worktrees": {
"mode": "public-owned",
"reason": "Private counterpart was archived and superseded by a differently-shaped skill."
}
},
"localOverlay": "public-manifest.local.json"
}
27 changes: 27 additions & 0 deletions public-manifest.local.json.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
{
"$comment": "Copy to public-manifest.local.json (gitignored) and fill in your own terms. Merged into public-manifest.json by scripts/sync-public.mjs: leakPatterns are appended, transforms.replace runs after the tracked rules.",

"$why": "The tracked manifest holds only structural patterns, the shapes that are private regardless of who maintains this repo (home paths, RFC1918 addresses). Anything that NAMES your machines, repos, domains, or internal frameworks belongs here instead. An enumeration of those names is itself metadata about your private ecosystem, so publishing the list would leak the thing the list exists to protect. Same reasoning as .gitleaks.local.toml.",

"leakPatterns": [
{
"id": "private-email",
"description": "Your personal or business email domains",
"regex": "[A-Za-z0-9._%+-]+@(?:your-domain)\\.[A-Za-z]{2,}"
},
{
"id": "private-machine-names",
"description": "Your machine and host names",
"regex": "\\b(?:your-laptop|your-server)\\b"
},
{
"id": "private-repo-names",
"description": "Private repos a public skill must not cite",
"regex": "\\b(?:your-private-repo|your-vault)\\b"
}
],

"transforms": {
"replace": [{ "find": "your-private-repo", "replaceWith": "a private repo" }]
}
}
Loading
Loading