Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,3 +20,7 @@ PR titles follow Conventional Commits (feat:, fix:, refactor:, chore:, docs:, ..
`cargo clippy --workspace --all-targets -- -D warnings`
- [ ] JS gates pass (if touched): `npm run build --workspaces && npm test --workspaces`
- [ ] No new compiler/linter warnings
- [ ] Source hygiene: `node scripts/verify-no-control-bytes.mjs` exits 0
- [ ] **Before any `--admin` merge**: run `node scripts/verify-pr-checks.mjs <pr-number>`
and use the `gh pr merge --squash --match-head-commit <sha>` command it emits
(PF-017: a cancelled run reads as green without this check)
23 changes: 23 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -306,3 +306,26 @@ jobs:
- name: pytest against the installed wheel (perf, advisory)
continue-on-error: true
run: pytest crates/mds-python/tests -q -m perf

# -------------------------------------------------------------------------
# #288: Source-hygiene gate — rejects hazardous codepoints (control bytes,
# bidi overrides, BOM) from tracked source. Scans the full tracked tree via
# `git ls-files`, reads content at codepoint level (pure Node; no grep -P
# which BSD grep lacks). Positive-control suite proves the check is live.
#
# D-CB7: BSD grep lacks -P and exits 2 with empty output, making the absence
# of hazard bytes indistinguishable from a broken invocation (avoids PF-013).
# D-CB5: Zero-files-scanned is exit 1, not exit 0 (avoids PF-016).
# -------------------------------------------------------------------------
source-hygiene:
name: Source hygiene
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/setup-node@v6
with:
node-version: 22
- name: Scan tracked source for hazardous codepoints
run: node scripts/verify-no-control-bytes.mjs
- name: Run positive-control and class-completeness suite
run: node --test scripts/__test__/*.spec.mjs
10 changes: 10 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,16 @@ jobs:
with: { node-version: 22 }
- name: "Assert synchronized versions, no file: refs"
run: node scripts/verify-versions.mjs
# #288: Source-hygiene gate — also runs on tag pushes via this job.
# ci.yml does not run on tag pushes, so these two steps ensure the full
# gate (scanner + positive-control suite) is enforced at release time.
# The positive-control suite pins HAZARD_RANGES (D-CB1a) so a silently-
# narrowed hazard class cannot exit 0 on the release path (ADR-009/PF-013).
# Uses the same Node 22 install above.
- name: "Assert no hazardous codepoints in tracked source"
run: node scripts/verify-no-control-bytes.mjs
- name: "Run positive-control and class-completeness suite"
run: node --test scripts/__test__/*.spec.mjs

# ---------------------------------------------------------------------------
# A6 — cross-compile the native addon for all 7 targets.
Expand Down
26 changes: 26 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -624,6 +624,32 @@ diagnostic messages must update to check for the `\uXXXX` literal form instead.
(span attribution machinery, `end_offset` fields, `FixLineSpan` planner) pushed
the optimized WASM binary to ~808 KB. The guard in `ci.yml` was raised accordingly.

- **Code of Conduct** (#38): `CODE_OF_CONDUCT.md` at the repository root, using
Contributor Covenant 2.1 with `deanshrn@gmail.com` as the enforcement contact.
Linked from `CONTRIBUTING.md` and `README.md`.

- **Source-hygiene CI gate** (#288): `scripts/verify-no-control-bytes.mjs` scans
every tracked file for hazardous codepoints — C0 control characters (excluding
TAB and LF), DEL, C1 (at codepoint level, catching UTF-8-encoded NEL U+0085),
the twelve `Bidi_Control=Yes` characters (Trojan Source / CVE-2021-42574), the
JavaScript line/paragraph separators U+2028 and U+2029, and U+FEFF (BOM).
Runs in CI on every pull_request and on tag pushes (release.yml). An opt-in
pre-commit hook (`scripts/hooks/pre-commit`) is provided; it reads the staged
blob via `git cat-file`, not the working tree. Also remediates seven live
U+0085 bytes that had been injected into tracked source by the edit tooling
(PF-018).

- **Pre-merge check verifier** (#289): `scripts/verify-pr-checks.mjs` guards
against PF-017 (a cancelled CI run reads as "not failing" to `gh pr merge
--admin`). It evaluates three tiers: Tier A asserts every required
branch-protection context is `completed+success`; Tier B fails on any
non-required check-run that concluded
`failure/cancelled/timed_out/action_required/stale`; Tier C (legacy commit
statuses) is advisory. It emits a `gh pr merge --squash --match-head-commit
<sha>` command pinned to the verified SHA. Exit 0: Tier A and Tier B pass;
exit 1: any Tier A/B failure or zero check-runs found; exit 2:
tool/permission errors.

### Changed

- **napi and Python `compileFile` / `compile_file` now emit root-relative
Expand Down
2 changes: 2 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,3 +48,5 @@ See @RELEASING.md for the full runbook.
- `crates/mds-python/build.rs` emits a cdylib-scoped `-undefined dynamic_lookup` so bare `cargo build` links the extension on macOS (Linux allows undefined cdylib symbols; maturin passes the flag itself when it builds the wheel)
- Local Python dev: `maturin develop` needs an active **virtualenv** + `python3` on PATH; CI has no venv so it uses `pip install ./crates/mds-python` (the maturin PEP 517 backend). Wheels are `cp311-abi3` (one per platform)
- `crates/mds-python` is free-threading ready (frozen result classes, `#[pymodule(gil_used = false)]`, GIL released around each compile); the `cp314t` free-threaded wheel is a separate ABI and is deferred with the wheel matrix + PyPI publishing (follow-up to #132)
- **Source hygiene gate** (#288): `node scripts/verify-no-control-bytes.mjs` scans tracked source for hazardous codepoints (C0, C1, bidi, BOM). BSD grep has no `-P` (exits 2, empty output reads as clean) — never use grep to verify absence of control bytes; the gate uses pure Node codepoint iteration. When writing codepoints in source or docs, use numeric notation (U+202E, 0x202e) rather than `\uXXXX` escapes — the edit tooling decodes 4-hex `\uXXXX` to live bytes (PF-018).
- **Pre-merge check verifier** (#289, PF-017): a CANCELLED GitHub Actions run reads as "not failing" to `gh pr merge --admin`, which can merge an unverified head. Before any `--admin` merge, run `node scripts/verify-pr-checks.mjs <pr-number>` and use the `gh pr merge --squash --match-head-commit <sha>` command it emits. This verifies all required contexts are `completed+success` and pins the SHA.
83 changes: 83 additions & 0 deletions CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# Contributor Covenant Code of Conduct

## Our Pledge

We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, caste, color, religion, or sexual identity and orientation.

We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community.

## Our Standards

Examples of behavior that contributes to a positive environment for our community include:

* Demonstrating empathy and kindness toward other people
* Being respectful of differing opinions, viewpoints, and experiences
* Giving and gracefully accepting constructive feedback
* Accepting responsibility and apologizing to those affected by our mistakes, and learning from the experience
* Focusing on what is best not just for us as individuals, but for the overall community

Examples of unacceptable behavior include:

* The use of sexualized language or imagery, and sexual attention or advances of any kind
* Trolling, insulting or derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or email address, without their explicit permission
* Other conduct which could reasonably be considered inappropriate in a professional setting

## Enforcement Responsibilities

Community leaders are responsible for clarifying and enforcing our standards of acceptable behavior and will take appropriate and fair corrective action in response to any behavior that they deem inappropriate, threatening, offensive, or harmful.

Community leaders have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, and will communicate reasons for moderation decisions when appropriate.

## Scope

This Code of Conduct applies within all community spaces, and also applies when an individual is officially representing the community in public spaces. Examples of representing our community include using an official e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event.

## Enforcement

Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the community leaders responsible for enforcement at deanshrn@gmail.com. All complaints will be reviewed and investigated promptly and fairly.

All community leaders are obligated to respect the privacy and security of the reporter of any incident.

## Enforcement Guidelines

Community leaders will follow these Community Impact Guidelines in determining the consequences for any action they deem in violation of this Code of Conduct:

### 1. Correction

**Community Impact**: Use of inappropriate language or other behavior deemed unprofessional or unwelcome in the community.

**Consequence**: A private, written warning from community leaders, providing clarity around the nature of the violation and an explanation of why the behavior was inappropriate. A public apology may be requested.

### 2. Warning

**Community Impact**: A violation through a single incident or series of actions.

**Consequence**: A warning with consequences for continued behavior. No interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, for a specified period of time. This includes avoiding interactions in community spaces as well as external channels like social media. Violating these terms may lead to a temporary or permanent ban.

### 3. Temporary Ban

**Community Impact**: A serious violation of community standards, including sustained inappropriate behavior.

**Consequence**: A temporary ban from any sort of interaction or public communication with the community for a specified period of time. No public or private interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, is allowed during this period. Violating these terms may lead to a permanent ban.

### 4. Permanent Ban

**Community Impact**: Demonstrating a pattern of violation of community standards, including sustained inappropriate behavior, harassment of an individual, or aggression toward or disparagement of classes of individuals.

**Consequence**: A permanent ban from any sort of public interaction within the community.

## Attribution

This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 2.1, available at [https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].

Community Impact Guidelines were inspired by [Mozilla's code of conduct enforcement ladder][Mozilla CoC].

For answers to common questions about this code of conduct, see the FAQ at [https://www.contributor-covenant.org/faq][FAQ]. Translations are available at [https://www.contributor-covenant.org/translations][translations].

[homepage]: https://www.contributor-covenant.org
[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html
[Mozilla CoC]: https://github.com/mozilla/diversity
[FAQ]: https://www.contributor-covenant.org/faq
[translations]: https://www.contributor-covenant.org/translations
87 changes: 87 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,44 @@ MDS_BACKEND=native npm test -w @mdscript/mds
MDS_BACKEND=wasm npm test -w @mdscript/mds
```

### Source hygiene

All tracked source must be free of hazardous codepoints. The gate runs
automatically in CI (`source-hygiene` job) and can be run locally:

```bash
node scripts/verify-no-control-bytes.mjs # full tracked-tree scan
node scripts/verify-no-control-bytes.mjs --staged # staged-only (pre-commit)
npm run test:gates # positive-control spec suite
```

Exit codes are a contract: `0` no hazards found (prints file and byte counts
for non-vacuity), `1` hazard found or scan failed closed (zero files scanned,
unreadable path, stale allowlist entry, git not on PATH), `2` indeterminate
(a git subcommand failed unexpectedly — never treat `2` as clean).

**Opt-in pre-commit hook** (replaces `.git/hooks` wholesale — document your
existing local hooks before enabling):

```bash
git config core.hooksPath scripts/hooks
```

**Hazard class**: C0 (0x00-0x1F) excluding TAB and LF, DEL (0x7F), C1
(0x80-0x9F at codepoint level — catches UTF-8-encoded NEL 0xC2 0x85), the
twelve Unicode `Bidi_Control=Yes` codepoints including U+061C (Trojan Source,
CVE-2021-42574), plus U+2028 (LS), U+2029 (PS), and U+FEFF (BOM). CR (U+000D)
is permitted only as the first byte of CRLF.

**BSD grep trap**: macOS ships BSD grep, which has no `-P` flag and exits 2
with empty output. That empty output is indistinguishable from a clean scan.
The gate uses pure Node codepoint iteration — never grep.

**Authoring rule**: when writing code or documentation that mentions hazardous
codepoints, use numeric notation (`U+202E`, `0x202e`, or `String.fromCodePoint(0x202e)`)
rather than backslash-u escapes. The edit tooling decodes the 4-hex-digit form
`\uXXXX` to live bytes, injecting the hazard into the very file that warns about it.

## Pull requests

- **Conventional Commits**: PR titles and commits follow
Expand All @@ -73,8 +111,57 @@ MDS_BACKEND=wasm npm test -w @mdscript/mds
implementation details.
- **No regressions**: every existing test must still pass.

## Merging

**Admin merges require the pre-merge check verifier.** GitHub's `--admin`
flag bypasses required-status enforcement; a cancelled CI run reads as
"not failing" rather than as failing (PF-017). Run the verifier before any
`gh pr merge --admin`:

```bash
node scripts/verify-pr-checks.mjs <pr-number>
```

The verifier reads required contexts from live branch protection, checks that
every context is `status=completed` AND `conclusion=success`, and on pass
emits a `gh pr merge --squash --match-head-commit <sha>` command pinned to
the verified SHA (closes the TOCTOU window).

Exit codes are a contract: `0` all Tier A and Tier B checks passed, `1` any
Tier A failure (required context missing or non-success), any Tier B failure
(non-required check-run concluded failure/cancelled/timed_out/action_required/
stale), or zero check-runs found, `2` the tool could not tell (protection
unreadable, no required contexts configured, `gh` older than 2.31, incomplete
pagination). **Only `0` means verified** — never read `2` as a pass.

Tier B is load-bearing: `source-hygiene` is not among `main`'s required
branch-protection contexts, so Tier B is the sole mechanism that makes a
failing `source-hygiene` run block an `--admin` merge.

Scope, stated so it is not assumed: the verifier checks the checks *on one
commit*. It does **not** assert that the head is up to date with the base
branch, so a stale-but-green head can still be merged under `--admin` even
after the verifier passes. Keep the branch rebased. It does **not** assert that
`source-hygiene` is a required context — `--admin` bypasses required-status
enforcement outright for non-required checks, and Tier B is the binding
mechanism. Tier B skips non-required check-runs still `queued` or `in_progress`:
a verifier pass issued while `source-hygiene` is still running has verified
nothing about source hygiene. Ensure all jobs have completed before running the
verifier.

If the base branch is unprotected (e.g. a wave branch), supply `--required-from`:

```bash
node scripts/verify-pr-checks.mjs <pr-number> --required-from main
```

## Security

Please report vulnerabilities privately. See [SECURITY.md](./SECURITY.md). Do not
open public issues for security problems.

## Code of Conduct

This project follows the [Contributor Covenant 2.1](CODE_OF_CONDUCT.md). By
participating, you agree to abide by its terms.

3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -326,7 +326,8 @@ See [spec.md](spec.md) for the full MDS v0.4.0 language specification.
## Contributing

Contributions are welcome! See [CONTRIBUTING.md](CONTRIBUTING.md) for the local
workflow and quality gates.
workflow and quality gates. By participating you agree to the
[Contributor Covenant 2.1](CODE_OF_CONDUCT.md).

## Security

Expand Down
23 changes: 19 additions & 4 deletions RELEASING.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,6 @@ These are **not** automated and must be done before the first release:
`mds-core` and `mds-cli` on crates.io.
4. **Enable GitHub private vulnerability reporting** (Settings → Code security →
Private vulnerability reporting) so the SECURITY.md flow works.
5. Add **`CODE_OF_CONDUCT.md`** (tracked in #38) if not already present.

## Pre-flight (before tagging)

Expand All @@ -58,6 +57,12 @@ npm run build --workspaces --if-present
npm test --workspaces --if-present
node scripts/verify-versions.mjs

# Source hygiene and pre-merge check gates
node scripts/verify-no-control-bytes.mjs
npm run test:gates # positive-control spec suite
# Before any --admin merge (PF-017 guard — cancelled runs read as green):
node scripts/verify-pr-checks.mjs <pr-number>

# Packaging spot-check (inspect tarball contents)
npm pack -w @mdscript/mds --dry-run
npm pack -w @mdscript/mds-wasm --dry-run
Expand Down Expand Up @@ -88,9 +93,19 @@ The release is driven by pushing a `vX.Y.Z` tag. This is how all versions have s

1. **Bump versions:** `node scripts/bump-version.mjs X.Y.Z` (updates all
manifests and stamps the CHANGELOG, opening a fresh `[Unreleased]`).
2. **Land the bump on `main`:** open a PR (CI-gated). `main` is protected and the
sole code-owner can't self-approve, so the merge needs an admin override
(`enforce_admins=false` permits it). Squash-merge to keep linear history.
2. **Land the bump on `main`:** open a PR (CI-gated). Once CI is green, run the
pre-merge check verifier before merging — a cancelled run reads as green under
`--admin` (PF-017):
```bash
node scripts/verify-pr-checks.mjs <pr-number>
```
On exit 0 the script prints the exact merge command — copy and run it verbatim:
```bash
gh pr merge --squash --match-head-commit <headSha>
```
(`main` is protected; the sole code-owner can't self-approve so `--admin` is
required. `--match-head-commit` closes the TOCTOU window between verification
and merge.)
3. **Tag the merged commit and push:**
```bash
git tag -a vX.Y.Z -m vX.Y.Z
Expand Down
Loading
Loading