Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# CI (including the sanitization gate's caller), the gate's per-repo config, and
# the gate itself need the owner's review. Advisory today: no branch protection
# requires code-owner review, so this blocks nothing.
/.github/ @juniperbevensee
/sanitize.config.json @juniperbevensee
/scripts/check_sanitization.py @juniperbevensee
67 changes: 30 additions & 37 deletions .github/workflows/sanitization.yml
Original file line number Diff line number Diff line change
@@ -1,46 +1,39 @@
name: sanitization

# The commons gate. Runs on every PR. Scans ONLY the diff (fast, never
# flaky-blocks a contributor on unrelated pre-existing content) — the full-tree
# audit belongs in a scheduled/gardener lane, not per-PR.
# The sanitization gate (the Alembic gate) is CANONICAL: one script, one reusable
# workflow, both in cyborg-garden/swarm-map. This repo keeps only what is its own:
# sanitize.config.json (what a "particular" means here, which paths carry
# content) and this caller. Do not vendor the script back in — the per-repo
# copies drifted apart, each missing a protection another had.
#
# SECURITY NOTE for when you open this to untrusted forks: switch the trigger to
# `pull_request_target` and check out the PR head explicitly, so the build-time
# ANTHROPIC_API_KEY is never exposed to injected PR code. Until then, keep
# `pull_request` (key only runs on trusted branches).
# Runs on every PR, scanning ONLY the diff (the full PR range, three-dot). The
# full-tree audit is a separate lane (run the canonical gate with --all locally, or add a caller with mode: full-tree).
#
# The reusable workflow runs, as separate steps so a red check names its cause:
# the gate self-tests, a config check, the deterministic secrets/PII layer
# (always, including fork PRs), and the semantic particulars layer with
# --require-semantic (fails closed on a missing or broken OPENROUTER_API_KEY).
# Fork PRs get no secrets, so they get the deterministic layer plus a loud
# "partial coverage" warning: a maintainer reviews particulars by hand. Making
# the semantic layer run on forks needs pull_request_target, which is a
# deliberate maintainer decision and intentionally NOT made here.
#
# Pinned to a release tag, never @main. Bump this pin and GATE_TAG in
# scripts/check_sanitization.py together (tests/test_sanitization_pin.py).

on:
pull_request:

permissions:
contents: read

jobs:
sanitize:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0

- uses: actions/setup-python@v5
with:
python-version: "3.12"

- name: Install deps (semantic layer + self-tests)
run: pip install anthropic pytest

- name: Scanner self-tests (deterministic, no key needed)
run: python -m pytest tests/test_check_sanitization.py -q

- name: Sanitization gate (diff)
# The semantic (LLM) layer runs automatically when ANTHROPIC_API_KEY is set
# as a repo Actions secret — strongly recommended, since it's the layer that
# catches engagement particulars the deterministic regexes can't. Without the
# key it skips with a loud warning (deterministic layer still hard-fails on
# secrets/PII). To make the semantic layer MANDATORY once your key is wired,
# append `--require-semantic` below: the gate then fails closed if the key is
# ever missing instead of silently passing deterministic-only.
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
run: |
BASE="origin/${{ github.base_ref }}"
git diff --name-only "$BASE...HEAD" \
| xargs -r python scripts/check_sanitization.py
# The pinned tag MUST exist in cyborg-garden/swarm-map. If it does not, this
# job fails at 0s with no check shown on the PR, which can look green. Keep
# this repo's sanitization check REQUIRED on main so that case blocks merges.
uses: cyborg-garden/swarm-map/.github/workflows/sanitization-gate.yml@sanitization-gate-v1.0.0
with:
mode: pr
secrets:
OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
16 changes: 8 additions & 8 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,14 +60,14 @@ flag is **not** an automatic rejection — it routes the PR to a human maintaine
makes the call. If your PR is flagged and you believe it's clean, say so in the PR
description; a maintainer reviews the flagged content and decides.

The semantic layer needs `ANTHROPIC_API_KEY`. Set it as a repo Actions secret (this is
on the onboarding checklist) so the layer runs in CI — it's the part that catches
engagement particulars the regexes can't. Without the key it skips with a loud warning
(the deterministic layer still hard-fails on secrets/PII), so local/offline runs work.
**Once your key is wired, make the semantic layer mandatory** by appending
`--require-semantic` to the gate step in `.github/workflows/sanitization.yml`: the gate
then fails closed if the key is ever missing, instead of silently passing
deterministic-only.
The semantic layer needs `OPENROUTER_API_KEY`, set as a repo Actions secret (this is
on the onboarding checklist) — it's the part that catches engagement particulars the
regexes can't. In CI the layer is **mandatory**: the canonical gate (a reusable workflow
in `cyborg-garden/swarm-map`, called from `.github/workflows/sanitization.yml`) runs it
with `--require-semantic`, so a missing or broken key fails closed instead of silently
passing deterministic-only. Fork PRs get no secrets, so they get the deterministic layer
plus a loud partial-coverage warning. Locally, without the key, the semantic layer skips
with a loud warning so offline runs still work.

To tune what counts as a particular for this domain, edit `sanitize.config.json` —
that is the single configuration surface for the gate. See the inline comments in that
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ but the scaffolding — base image, instance bootstrap, HSM consumption — assu

Click **"Use this template"** on GitHub to create your private repo, then follow the instantiation checklist in [SETUP.md](SETUP.md).

The checklist covers: renaming placeholders, filling `sanitize.config.json`, wiring `ANTHROPIC_API_KEY` in CI, and verifying the gate runs clean before your first real commit.
The checklist covers: renaming placeholders, filling `sanitize.config.json`, wiring `OPENROUTER_API_KEY` in CI, and verifying the gate runs clean before your first real commit.

---

Expand Down Expand Up @@ -64,7 +64,7 @@ You build under real conditions in your private instance. When a capability prov
| `docker/Dockerfile.{{SLUG}}` | System-level deps (binaries, libraries) for the agent image | Only if you need system deps |
| `engagements/` | Per-engagement working data — gitignored, stays local | N/A — ignored by construction |
| `instance/` `.overlay/` | Operator-local overrides, instance-specific config | N/A — ignored by construction |
| `scripts/check_sanitization.py` | The sanitization gate — runs in CI and via gardener | No — leave alone |
| `scripts/check_sanitization.py` | Local launcher for the canonical sanitization gate (in `cyborg-garden/swarm-map`; CI calls it as a reusable workflow) | No — leave alone |
| `.github/workflows/sanitization.yml` | CI gate | No — leave alone |
| `.gitignore` | Instance-overlay privacy model | No — leave alone |
| `sanitize.config.json` | **The one tuning surface** — teach the gate your domain | Yes — required on setup |
Expand Down
38 changes: 17 additions & 21 deletions SETUP.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,8 +82,8 @@ contains a `$comment`.
"user@example",
"127.0.0.1",
"0.0.0.0"
// add any known-safe strings that the regex layer would otherwise hit —
// exact line substrings, e.g. a placeholder email in a README code block
// add any known-safe values that the regex layer would otherwise hit —
// matched against the DETECTED VALUE, not the whole line
]
}
}
Expand All @@ -97,24 +97,19 @@ conservative and fail closed. Only `flag_examples`, `do_not_flag_examples`, and

## 3. Add the CI secret

The semantic layer in CI requires an Anthropic API key. Without it the workflow
logs a loud warning and skips the semantic check.
The semantic layer in CI requires an OpenRouter API key, and it is **mandatory**:
`.github/workflows/sanitization.yml` calls the canonical gate (a reusable workflow
in `cyborg-garden/swarm-map`), which runs `--require-semantic`. Until the secret
exists, every same-repo PR's `sanitization` check fails closed. Fork PRs never get
secrets, so they run the deterministic layer plus a loud partial-coverage warning.

1. Go to **Settings → Secrets and variables → Actions → New repository secret**.
2. Name: `ANTHROPIC_API_KEY`, value: your key.
2. Name: `OPENROUTER_API_KEY`, value: your key.
3. Use a **dedicated, capped key** (a separate key per package is easy to rotate
and lets you monitor usage per package in the Anthropic console).
and lets you monitor usage per package).

If you want CI to hard-fail when the key is missing (e.g. to catch misconfigured
forks), edit `.github/workflows/sanitization.yml` and add `--require-semantic` to
the gate step:

```yaml
run: |
BASE="origin/${{ github.base_ref }}"
git diff --name-only "$BASE...HEAD" \
| xargs -r python scripts/check_sanitization.py --require-semantic
```
Do not copy the gate script into this repo. `scripts/check_sanitization.py` is a
launcher that fetches the canonical script at the tag pinned in the workflow.

---

Expand Down Expand Up @@ -174,20 +169,21 @@ Read `docs/privacy-and-visibility.md` before flipping to public. The short versi

## 7. Verify the gate locally

Install the test dep and run the deterministic layer offline (no API key needed):
Run the deterministic layer locally (no API key needed; the launcher fetches the
pinned canonical gate once, or set `SANITIZE_GATE_PATH` to a local copy):

```bash
pip install anthropic # for the semantic layer
pip install pytest # if not already installed

# Run the scanner self-tests (deterministic, no key needed):
# Check the gate pin and config (no key needed):
python -m pytest tests/ -q
python scripts/check_sanitization.py --check-config

# Full-tree audit, deterministic only (no key):
python scripts/check_sanitization.py --full-tree

# Full-tree audit with the semantic layer:
export ANTHROPIC_API_KEY=sk-ant-...
export OPENROUTER_API_KEY=...
python scripts/check_sanitization.py --full-tree
```

Expand All @@ -197,7 +193,7 @@ Expected output when clean:
ok hermes-skill/SKILL.md
ok hermes-plugin/your_tool.py
...
sanitization: clean.
sanitization: clean (both layers ran).
```

A `SECRET/PII` line is a hard fail — remove the content before any commit.
Expand Down
11 changes: 6 additions & 5 deletions docs/promotion-and-upstream.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,15 +50,16 @@ if the technique only worked once, it hasn't earned generalization yet.
`main`.

3. **Let the gate run.** The sanitization gate
(`scripts/check_sanitization.py`, wired into
`.github/workflows/sanitization.yml`) runs on the diff:
(canonical in `cyborg-garden/swarm-map`, called from
`.github/workflows/sanitization.yml`; `scripts/check_sanitization.py` is a
local launcher for it) runs on the diff:
- **Layer 1 — deterministic** (secrets/PII: API keys, private keys, emails,
phones, IPs). No dependencies, fails closed, **hard fail on any hit.** A red
deterministic check is a stop, not a discussion.
- **Layer 2 — semantic** (use-case particulars, per your
`sanitize.config.json`). Advisory → routes to human review. Skipped with a
warning if no `ANTHROPIC_API_KEY` is present; CI can force it with
`--require-semantic`.
`sanitize.config.json`). Advisory → routes to human review. Needs
`OPENROUTER_API_KEY`; CI runs it with `--require-semantic`, so a missing
key fails closed (locally it skips with a warning).

4. **Review and merge.** A maintainer review is the discipline. If you're a solo
operator, that's a self-review — and that's fine, because the deterministic
Expand Down
4 changes: 2 additions & 2 deletions sanitize.config.json
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,11 @@
"tool, API, library, or product names",
"well-known public reference material used illustratively"
],
"model": "claude-opus-4-8"
"model": "moonshotai/kimi-k3"
},

"deterministic": {
"$comment": "The credential/PII layer runs with NO external dependencies and FAILS CLOSED. These patterns are conservative defaults; add your own. Allowlist false-positives by exact line substring in 'allow_substrings'.",
"$comment": "The credential/PII layer runs with NO external dependencies and FAILS CLOSED. These patterns are conservative defaults; add your own. Allowlist a false positive by putting the EXACT detected value in 'allow_substrings' (despite the key's name, entries are never substrings). The only wider forms: an email domain such as 'example.com' excuses addresses at exactly that domain, and a three-octet prefix such as '198.51.100.' excuses that /24. Layer 1 cannot be switched off, and skip_paths may name only test-fixture paths.",
"enabled": true,
"allow_substrings": [
"noreply@anthropic.com",
Expand Down
Loading
Loading