Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,5 @@ coverage/
.internal/
.vercel
.roast/

.worktrees/
11 changes: 8 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

- **`--exit-code`** flag: exit `3` if any workflow is invalid, `4` if nothing fires — for using actwhy as a CI gate. Default behavior is unchanged (always exit `0`).
- Workflow `name:` is now read and exposed on each verdict (`--json`).
- Commit-message skip directives are evaluated for `push` and `pull_request`, including the `skip-checks: true` trailer.
- GitHub object-filter expressions such as `pull_request.labels.*.name` are evaluated when payload values are known.

### Changed

- A static matrix exceeding GitHub's 256-job cap now emits a `matrix-over-limit` warning instead of reporting the job as firing.
- A workflow with both `paths` and `paths-ignore` on one event now warns (`paths-and-paths-ignore`) and follows GitHub (uses `paths`).
- Slash-delimited globstars now match zero directories, so `docs/**/*.md` includes `docs/README.md`.
- Tag pushes ignore path filters, matching GitHub, and known-empty diffs skip path-filtered workflows.
- A static matrix exceeding GitHub's 256-job cap is now a `matrix-over-limit` job error and contributes zero firing variants.
- Mutually exclusive include/ignore filter pairs are reported as invalid instead of receiving invented precedence.
- An in-sync branch now reports zero outgoing files instead of silently substituting the last commit.
- `--event` payloads are validated (must be a JSON object, size-capped) and `__proto__`/`constructor` keys are ignored.
- The CLI errors clearly on Node < 20; CI actions are pinned by commit SHA.

Expand Down Expand Up @@ -48,7 +53,7 @@ Initial release.

- `schedule`, `merge_group`, `workflow_run`, and `workflow_call` are classified but not evaluated.
- Concurrency is not modeled; step-level `env:` is not evaluated; verdicts assume runs succeed.
- The >1,000-changed-files paths-filter skip that GitHub performs is not modeled.
- GitHub's >1,000-commit/diff-timeout fallback and 3,000-file path-filter window are not modeled.

See [docs/limitations.md](docs/limitations.md) for the full list and what actwhy reports in each case.

Expand Down
16 changes: 7 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

`act` executes your workflows (heavy, needs Docker, breaks on runner mismatches). `actionlint` lints their syntax. GitHub's own UI only explains what fired *after* you push. **actwhy** answers a different question — *for this exact push or PR, which workflows fire, which are skipped and by which filter, and which can't be decided offline* — statically, instantly, and with zero network calls.

It never guesses. Every workflow gets one of three verdicts: **FIRES**, **SKIPPED** (with the failing filter quoted in plain English), or **UNKNOWN** (naming the exact runtime-only value it would need, such as `secrets.*` or `needs.*.outputs.*`, and how to supply it).
It never guesses. Every workflow gets one of four verdicts: **FIRES**, **SKIPPED** (with the failing filter quoted in plain English), **UNKNOWN** (naming the exact runtime-only value it would need, such as `secrets.*` or `needs.*.outputs.*`, and how to supply it), or **ERROR** for a configuration GitHub would reject.

![actwhy demo](docs/assets/demo.png)

Expand Down Expand Up @@ -38,10 +38,6 @@ npm install -g actwhy
actwhy --help
```

> **Not on npm yet?** actwhy publishes to npm immediately after launch. Until then, install
> from source: `git clone https://github.com/Co-Messi/actwhy && cd actwhy && npm install && npm run build`,
> then run `node dist/actwhy.js` (or `npm link` to expose the `actwhy` command).

Requires **Node.js ≥ 20**.

## What it looks like
Expand Down Expand Up @@ -73,7 +69,8 @@ And the case above, where the closest miss is surfaced so you know which filter
- **It never guesses.** A value it cannot know offline (a secret, a needed job's output) becomes an honest `UNKNOWN` naming that value — not a fabricated pass or fail.
- **It quotes the exact filter that decided the outcome.** Not "skipped" — *"branches filter `["main"]` does not match `feat/login`"*.
- **It uses GitHub's own parsing and expression semantics.** Workflow parsing and expression coercion run on GitHub's MIT-licensed [`@actions/workflow-parser`](https://github.com/actions/languageservices) and [`@actions/expressions`](https://github.com/actions/languageservices) — the libraries behind the official Actions language services, not a reimplementation.
- **It gets filter patterns right.** GitHub's `?` and `+` are regex-style quantifiers on the *preceding character*, not glob wildcards — a distinction most third-party matchers get wrong. actwhy implements GitHub's exact semantics, including `!` negation ordering.
- **It gets filter patterns right.** GitHub's `?` and `+` are regex-style quantifiers on the *preceding character*, not glob wildcards — a distinction most third-party matchers get wrong. actwhy implements GitHub's semantics, including `!` negation ordering and slash-delimited globstars such as `docs/**/*.md` matching files directly under `docs/`.
- **It models GitHub's quiet skip rules.** Tag pushes ignore path filters, empty diffs do not start path-filtered workflows, and commit messages containing `[skip ci]`-style directives suppress `push` and `pull_request` runs.
- **It catches the classic always-true footgun.** `if: ${{ github.ref }} == 'refs/heads/main'` renders to a non-empty string and is *always* truthy — actwhy warns instead of letting it silently pass.
- **It runs fully local.** Zero network calls, zero telemetry. It reads your workflow files and git metadata, nothing else.

Expand All @@ -99,7 +96,7 @@ actwhy has two subcommands. `push` is the default when you run `actwhy` with no

### `actwhy push`

Simulate a push. With no flags it infers the current branch and the outgoing changed files from git (your branch vs its upstream).
Simulate a push. With no flags it infers the current branch and the outgoing changed files from git (your branch vs its upstream). If the branch is already in sync, actwhy reports an explicit empty outgoing set; it never substitutes an already-pushed commit.

| Flag | Description |
|---|---|
Expand Down Expand Up @@ -134,9 +131,10 @@ Simulate a pull request against a base branch.

## How it works

- **Three-valued verdicts.** Each workflow and job resolves to `FIRES`, `SKIPPED`, or `UNKNOWN`. `SKIPPED` always carries the exact failing filter or subexpression; `UNKNOWN` always names the runtime-only value it lacks and how to supply it (`--event payload.json`). actwhy never fabricates a pass or fail.
- **Honest verdicts.** Each workflow and job resolves to `FIRES`, `SKIPPED`, `UNKNOWN`, or `ERROR`. `SKIPPED` carries the exact failing filter or subexpression; `UNKNOWN` names the runtime-only value it lacks and how to supply it (`--event payload.json`); `ERROR` identifies invalid configuration such as mutually exclusive include/ignore filters or a matrix above 256 jobs.
- **GitHub's own libraries.** Parsing and expression coercion run on GitHub's MIT-licensed `@actions/workflow-parser` and `@actions/expressions`, so the grammar, type coercion, and function semantics match what GitHub actually does — this is GitHub's code, not a reimplementation.
- **An exact filter-pattern engine.** `*` and `**`, character classes `[…]`, `!` negation (order-sensitive), and — critically — the regex-style quantifiers `?` (zero or one of the *preceding* character) and `+` (one or more of the *preceding* character), which standard glob libraries misinterpret as wildcards.
- **An exact filter-pattern engine.** `*` and `**` (including zero-directory slash-delimited globstars), character classes `[…]`, `!` negation (order-sensitive), and — critically — the regex-style quantifiers `?` (zero or one of the *preceding* character) and `+` (one or more of the *preceding* character), which standard glob libraries misinterpret as wildcards.
- **GitHub object filters.** Expressions such as `github.event.pull_request.labels.*.name` are projected and evaluated when the payload values are known.
- **Kleene (three-valued) logic.** Unknowns propagate only when they change the outcome. `<unknown> && false` is still a decisive `SKIPPED`; `<unknown> || true` is still `FIRES`. A verdict becomes `UNKNOWN` only when the unknown genuinely decides it.

For the precise boundaries of v0.1 — which events are evaluated versus classified, and what actwhy reports instead of guessing — see **[docs/limitations.md](docs/limitations.md)**.
Expand Down
10 changes: 5 additions & 5 deletions REVIEW.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ was fixed, and what remains — honestly.
|---|---|---|
| 1 | A single malformed filter pattern (e.g. `paths: ['src/[']`) threw out of the evaluator and crashed the entire run — every other workflow lost its verdict | Pattern-compilation failures now degrade to a per-workflow `invalid workflow` verdict quoting the error; the rest of the run is unaffected |
| 2 | The playground's install section claimed the CLI could read "YAML on stdin" — no stdin handling exists | Claim removed (stdin support is tracked as a starter issue) |
| 3 | The most common stranger flow — running `actwhy` on a branch in sync with its upstream — produced a false "NOTHING fires" from an empty changed-file set | In-sync branches now simulate the last commit's push, with an explicit header label saying so |
| 3 | The most common stranger flow — running `actwhy` on a branch in sync with its upstream — was ambiguous | In-sync branches now report a truthful empty outgoing set with an explicit header label; already-pushed files are never substituted |
| 4 | Site footer and JSON-LD pointed at an npm package page that 404s pre-publish | Repointed to the GitHub repository until the npm release is live |
| 5 | The social-preview image had overlapping, unreadable text (font-metric assumptions) | Layout rebuilt with fixed columns and end-anchored badges; re-rendered and visually verified |
| 6 | A launch-copy draft overstated provenance ("filter semantics are GitHub's code" — the filter-pattern engine is actwhy's own implementation of GitHub's documented grammar) | Corrected; the precise split (GitHub's libraries for parsing/expressions, actwhy's verified engine for filter patterns) is stated everywhere |
Expand Down Expand Up @@ -50,8 +50,8 @@ was fixed, and what remains — honestly.
stay `UNKNOWN` by design.
- Step-level `env:` is not evaluated; matrix-dependent step conditions
report `UNKNOWN`.
- GitHub skips paths filters on pushes touching >1000 files; actwhy does not
model that edge (documented).
- GitHub's >1,000-commit/diff-timeout fallback and 3,000-file path-filter window
are not modeled (documented).
- The web playground share-link format has no versioning guarantee yet.

Fidelity disputes are P1 bugs — file a
Expand All @@ -67,8 +67,8 @@ plus several improvements. All were fixed and covered by regression tests.
|---|---|---|
| **Critical** | The earlier ReDoS fix only special-cased star atoms; `branches: ['a++']` still compiled to a nested-quantifier `RegExp` and hung the process for minutes on a ~30-char value (uncatchable — a hang, not a throw). | Replaced the RegExp compiler with a **linear-time Thompson-NFA matcher**. Backtracking is impossible by construction; the reviewer's 120 s hang is now ~2 ms. |
| **High** | Attacker-controlled workflow text (filter patterns, branch names, commit messages, parser errors) was written to the terminal verbatim — a crafted workflow could inject ANSI escapes to forge a green `FIRES` line. | `render.ts` now strips all C0/C1 control chars and `ESC` from every attacker-derived string before output. Verified: zero `0x1b` bytes reach stdout for a malicious workflow. |
| Medium | A static matrix of 257–4096 legs was reported as firing; GitHub caps matrices at 256 jobs (and fails the run). | Emit a `matrix-over-limit` warning at >256. |
| Medium | `paths` + `paths-ignore` on one event silently dropped `paths-ignore`. | Follow GitHub (use `paths`) and warn (`paths-and-paths-ignore`). |
| Medium | A static matrix of 257–4096 legs was reported as firing; GitHub caps matrices at 256 jobs (and fails the run). | Report the job as a `matrix-over-limit` error and count zero firing variants. |
| Medium | `paths` + `paths-ignore` on one event silently dropped `paths-ignore`. | Report mutually exclusive include/ignore pairs as invalid instead of inventing precedence. |
| Medium | The CLI always exited `0`, so it couldn't gate CI. | Added `--exit-code` (3 = invalid workflow, 4 = nothing fires). |
| Low | `--event` JSON was unvalidated/unbounded; `__proto__` keys merged; Node < 20 crashed cryptically; git base ref could be a flag; `workflowName()` was a dead stub; CI actions pinned by tag. | Validated/size-capped `--event` + forbidden-key filter; clear Node-version guard; `--end-of-options` on git diffs; wired `name:`; SHA-pinned CI actions. |

Expand Down
24 changes: 16 additions & 8 deletions docs/limitations.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Limitations (v0.1)
# Limitations (v0.1.x)

actwhy's value comes from being *honest*: where it cannot decide something offline, it says so rather than guessing. This page lists the boundaries of the v0.1 release and, for each, exactly what actwhy reports instead of inventing an answer.

Expand Down Expand Up @@ -61,29 +61,37 @@ Step-level `env:` is not evaluated. Conditions that read a variable set by an ea

Static matrices are expanded and counted. A matrix built from `fromJSON(...)` or another dynamic source is not enumerated.

**What actwhy says instead of guessing:** the job's matrix count is reported as `unknown` with a note, rather than a fabricated variant count. If a static matrix would produce **more than 256 jobs** — GitHub's hard cap, which fails the run — actwhy emits a `matrix-over-limit` warning instead of quietly reporting the job as firing.
**What actwhy says instead of guessing:** the job's matrix count is reported as `unknown` with a note, rather than a fabricated variant count. If a static matrix would produce **more than 256 jobs** — GitHub's hard cap — actwhy reports that job as an `ERROR` with `matrix-over-limit` and does not count any of its variants as firing.

## Concurrency

`concurrency:` groups and cancel-in-progress behavior are not modeled. actwhy answers *"would this workflow be triggered?"*, not *"would a previous run cancel it?"*.

**What actwhy says instead of guessing:** concurrency is simply not part of the verdict. A workflow that would be triggered is reported as `FIRES` regardless of whether a concurrency rule might later cancel it.

### `paths` and `paths-ignore` together
### Mutually exclusive include/ignore filters

GitHub documents `paths` and `paths-ignore` as mutually exclusive on the same event; when both are present it applies `paths` and ignores `paths-ignore`.
GitHub does not allow `paths` with `paths-ignore`, `branches` with `branches-ignore`, or `tags` with `tags-ignore` on the same event.

**What actwhy says instead of guessing:** it follows GitHub (evaluates `paths`, drops `paths-ignore`) and emits a `paths-and-paths-ignore` warning so you know half the author's intent is being discarded.
**What actwhy says instead of guessing:** the workflow is reported as an `ERROR` with `invalid-filter-combination`. actwhy does not invent precedence for an invalid trigger.

### Commit-message skip directives

GitHub suppresses `push` and `pull_request` workflows for the documented bracketed directives (`[skip ci]`, `[ci skip]`, `[no ci]`, `[skip actions]`, and `[actions skip]`) and a final `skip-checks: true` trailer. These directives do not apply to `pull_request_target`.

actwhy evaluates these directives when the commit message is supplied or inferred from `HEAD`.

## Exit codes

By default actwhy always exits `0` — a skipped or nothing-fires result is information, not a failure. Pass `--exit-code` to use it as a CI gate: it then exits `3` if any workflow is invalid and `4` if nothing fires. Usage errors (`2`) and a missing workflows directory (`1`) are always non-zero.

## Paths filters on large pushes
## Paths filters on large diffs

GitHub always runs a workflow when it cannot generate the diff because the push contains **more than 1,000 commits** or diff generation times out. Separately, GitHub evaluates path filters against only the first **3,000 changed files** returned by the diff.

GitHub stops applying `paths` / `paths-ignore` filters on a push that changes **more than 1,000 files**, running the workflow regardless. actwhy does not model this edge case; it applies paths filters to whatever changed-file list it is given.
actwhy does not know whether GitHub's diff timed out and does not truncate an explicitly supplied file list to 3,000 entries.

**What actwhy says instead of guessing:** for pushes near or above that threshold, treat a `paths`-based `SKIPPED` with suspicion — GitHub may run the workflow anyway. This is a known modeling gap, noted here rather than silently mis-reported.
**What actwhy says instead of guessing:** actwhy evaluates the complete changed-file list it receives. For pushes above 1,000 commits, timed-out diffs, or cases where a relevant path may fall beyond GitHub's 3,000-file window, compare the verdict with these documented GitHub limits.

## Fidelity

Expand Down
Loading