Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
5cf44af
test(noalias): gate proves VERIFY_NO_ALIAS is a release optimizer fac…
joyful-ii-V-I Sep 12, 2026
5a48be4
diag(noalias): VERIFY_NO_ALIAS becomes an optimizer fact in release; …
joyful-ii-V-I Sep 12, 2026
225672a
test(noalias): pin noaliascheck as binary-independent in binoverridec…
joyful-ii-V-I Sep 12, 2026
95627a0
docs(changelog): VERIFY_NO_ALIAS as a release optimizer fact, VERIFY_…
joyful-ii-V-I Sep 12, 2026
b62c1dc
build(noalias): LLVM 17 keeps BasicAA's separate_storage reader off; …
joyful-ii-V-I Sep 12, 2026
2b2b5d2
test(noalias): the capability probe is the real slice, classified thr…
joyful-ii-V-I Sep 12, 2026
5156d66
diag(noalias): §6 states the LLVM 17 switch and the CMake option
joyful-ii-V-I Sep 12, 2026
eced7a5
merge(noalias): origin/main into lane/noalias-gate
joyful-ii-V-I Sep 12, 2026
8efab9f
build(noalias): cite why LLVM 17 left separate-storage off, and why f…
joyful-ii-V-I Sep 12, 2026
79d35ad
test(noalias): classify the LOOP path on its own real slice — LLVM 17…
joyful-ii-V-I Sep 12, 2026
cd18967
test(noalias): guard the direct-builtin loop probe — arm 7 compiles t…
joyful-ii-V-I Sep 12, 2026
912b897
test(noalias): answer the two review findings — compiler identity bef…
joyful-ii-V-I Sep 12, 2026
e9aba99
diag(noalias): §6 names the one null-free form that keeps the effect …
joyful-ii-V-I Sep 12, 2026
49d52df
diag(noalias): §6 — the early return is where it already exists, not …
joyful-ii-V-I Sep 12, 2026
55c7b50
Merge remote-tracking branch 'origin/main' into lane/noalias-gate
joyful-ii-V-I Sep 12, 2026
5856a61
diag(noalias): VERIFY_NO_ALIAS_BUF refuses views at compile time; a C…
joyful-ii-V-I Sep 12, 2026
553f91a
Merge remote-tracking branch 'origin/main' into lane/noalias-gate
joyful-ii-V-I Sep 12, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,21 @@ out-of-tree row (12 → 13). `kParserVer` 92 → 93 with the mirror (the branch
re-pins with reasons in-file: `test/qschemetrip.hash`, `test/printf_parity.manifest` (the `--impact` help and
legend name the two new closure kinds; the `--deps` legend's lazy definition gains the rescue class). `docs/COMMANDS.md`
regenerated (2026-09-11).
### Changed — `VERIFY_NO_ALIAS` is a release optimizer fact, on LLVM 17 as well

- **`VERIFY_NO_ALIAS` is now an optimizer fact in release, not an inert assume.** `src/infra/Diagnostics.h` §6 adds
`__builtin_assume_separate_storage` (clang 17+, `__has_builtin`-guarded, `( (void)0 )` elsewhere) beside the debug
check, so codegen matches `__restrict__` on the parameters (`out=a; out+=b; out+=a;` arm64 10 → 6 instructions);
`VERIFY_NO_ALIAS_BUF` is the form for two OWNING containers (the object form is inert for their loops; views — `std::span`, `std::string_view` — can share one allocation and are refused at compile time); the comment carries
the complete-object contract and the macOS `<sys/cdefs.h>` trap that deletes bare `__restrict` in C++ —
`__restrict__` is the only spelling allowed in `src/`. `test/noaliascheck.sh` (eight arms, red against the old
definition) proves it. The optimizer half is a separate switch: BasicAA reads the bundle only when
`basic-aa-separate-storage` is on — `cl::init(false)` in LLVM 17 (AppleClang 16 / Xcode 16.2: the macos-14 CI
runners and the macos-arm64 release leg), `true` from LLVM 18 — so CMake now probes and passes
`-mllvm -basic-aa-separate-storage` to our targets (and to the ld64 link under LTO), and the gate classifies the
compiler by compiling the real slice three ways, with a `=false` negative control and a cross-check against the
cached CMake probe.

## [0.6.0] — 2026-09-11

**Languages and integrations from outside the project, much faster on the largest trees, and answers that say where
Expand Down
85 changes: 85 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -914,6 +914,91 @@ if(RIPWIRE_TSAN)
endforeach()
endif()

# ---- separate-storage alias analysis: -mllvm -basic-aa-separate-storage (LLVM 17 keeps it OFF) ----
# src/infra/Diagnostics.h §6's VERIFY_NO_ALIAS lowers, in release, to `llvm.assume [ "separate_storage"(a, b) ]`
# through __builtin_assume_separate_storage. The FRONT END has emitted that bundle since clang 17; whether BasicAA
# READS it is a separate switch inside the optimizer, llvm/lib/Analysis/BasicAliasAnalysis.cpp:
# static cl::opt<bool> EnableSeparateStorageAnalysis("basic-aa-separate-storage", cl::Hidden, cl::init(false))
# in LLVM 17, and cl::init(true) from LLVM 18 on. So on an LLVM-17 toolchain — AppleClang 16.0.0.16000026 (Xcode
# 16.2), the macos-14 CI runners and release.yml's macos-arm64 leg — the header compiles, the IR carries the bundle,
# and codegen is byte-identical to no promise at all. Found by CI job "release (macos-14, plain, appleclang, shard
# 4/4)" on PR #200: test/noaliascheck.sh arms 2, 2/3 and 6 red ("accNew still reloads a after the store", "NO
# CONTRAST", "not at least 2 below plain"), arm 7 and the IR-bundle row green — the front-end half worked, the
# optimizer half was off. Reproduced on Apple clang 21: the arm-2 probe is 5 instructions by default, 9 with
# `-mllvm -basic-aa-separate-storage=false`, 5 with `=true`.
#
# Why LLVM 17 left it off, and why turning it on there is safe: llvm/llvm-project#76864 "[BasicAA] Enable separate
# storage hints by default" (merged 2024-01-03) flipped the default because "a few months of experimentation in a
# large codebase did not reveal any significant build speed regressions" — the LLVM-17 default was a compile-time
# hedge, not a soundness one; #76770 the same week only makes the hint fire in more contexts. LLVM 17 with the option
# on runs the analysis LLVM 18+ runs unconditionally.
#
# The option's name is the same from LLVM 17 through 21 and clang takes it as a compile flag, so it is passed
# wherever the compiler accepts it: on LLVM 17 it turns the analysis on, on 18+ it restates the default (a no-op).
# What the option buys on LLVM 17 is the SCALAR half only: the reload after a store through a possibly-aliasing
# pointer goes away, but the loop vectorizer's overlap checks stay, because LLVM 17 consults the hint only at the
# assume's own context and LoopAccessAnalysis never supplies one (llvm/llvm-project#64666, fixed in LLVM 18 by
# #76770). CI showed exactly that split on PR #200 head 5156d668: noaliascheck arm 2 green, arm 6 at plain.
# Probed, never assumed — GCC rejects -mllvm outright and a clang whose LLVM had dropped the name would reject the
# option; either way the check says no, the build keeps the compiler default, and the header's debug check still
# runs. The two-token option rides in CMAKE_REQUIRED_FLAGS (a whitespace-split string, the way the libFuzzer probe
# above passes its flags). Verified 2026-09-12 in a scratch configure: Apple clang 21 says yes, and the same probe
# over `-mllvm -bogus-option-xyz` says no (clang exits 1 on an unknown -mllvm name). test/noaliascheck.sh reads the
# cached answer back out of build/CMakeCache.txt and fails if its own probe of the same compiler disagrees.
#
# OUR TARGETS ONLY, via target_compile_options — never add_compile_options: the tree-sitter core and the grammar
# objects are C that never spells the builtin, and per-target attachment is the rule this file already follows for
# the remarks and sanitizer flags. RIPWIRE_OWNED_CXX_TARGETS is ripwire, ripwire_probe and the test executables, in
# every flavour (plain, Release, asan, tsan): the list is flavour-independent, so nothing here is.
#
# LTO. With -flto the compile step still runs the PRE-link pipeline (which already removes the arm-2 reload under
# this option — measured: compile default + link `=false` gives 5), and ld64's libLLVMLTO runs the POST-link one,
# whose BasicAA reads the same cl::opt. So under LTO the option must reach the link as well, or every inlining
# opportunity the link step creates on an LLVM-17 toolchain is analysed with the promise off. Apple ld64 takes it as
# `-Wl,-mllvm,-basic-aa-separate-storage` — verified: compile `=false` + link `=false` gives 9 instructions, compile
# `=false` + link `=true` gives 5, and a bogus name fails the link ("libLLVMLTO: Unknown command line argument") —
# and it is probed below with a real -flto link before it is attached. The ELF spelling
# (`-Wl,-plugin-opt=-basic-aa-separate-storage` for lld / gold) is deliberately NOT added: it could not be verified
# on this machine, and no CI or release leg pairs LTO with an LLVM-17 ELF toolchain (ubuntu-24.04's clang is 18,
# the Linux release leg is gcc, which never sees the builtin). An LTO build on such a toolchain keeps the compile-time
# option and the linker's default; that limit is stated here rather than papered over with an unverified flag.
include(CheckCXXCompilerFlag)
set(_ripwire_required_flags_save "${CMAKE_REQUIRED_FLAGS}")
set(CMAKE_REQUIRED_FLAGS "-mllvm -basic-aa-separate-storage")
check_cxx_compiler_flag("" RIPWIRE_CXX_HAS_BASIC_AA_SEPARATE_STORAGE)
set(CMAKE_REQUIRED_FLAGS "${_ripwire_required_flags_save}")
if(RIPWIRE_CXX_HAS_BASIC_AA_SEPARATE_STORAGE)
foreach(_t IN LISTS RIPWIRE_OWNED_CXX_TARGETS)
target_compile_options(${_t} PRIVATE -mllvm -basic-aa-separate-storage)
endforeach()
set(_ripwire_separate_storage_link "no LTO, so nothing runs at link")
if(RIPWIRE_LTO)
if(APPLE)
include(CheckCXXSourceCompiles)
set(_ripwire_required_flags_save "${CMAKE_REQUIRED_FLAGS}")
set(_ripwire_required_link_options_save "${CMAKE_REQUIRED_LINK_OPTIONS}")
set(CMAKE_REQUIRED_FLAGS "-flto")
set(CMAKE_REQUIRED_LINK_OPTIONS "-flto;-Wl,-mllvm,-basic-aa-separate-storage")
check_cxx_source_compiles("int main() { return 0; }" RIPWIRE_LD64_HAS_BASIC_AA_SEPARATE_STORAGE)
set(CMAKE_REQUIRED_FLAGS "${_ripwire_required_flags_save}")
set(CMAKE_REQUIRED_LINK_OPTIONS "${_ripwire_required_link_options_save}")
if(RIPWIRE_LD64_HAS_BASIC_AA_SEPARATE_STORAGE)
foreach(_t IN LISTS RIPWIRE_OWNED_CXX_TARGETS)
target_link_options(${_t} PRIVATE "-Wl,-mllvm,-basic-aa-separate-storage")
endforeach()
set(_ripwire_separate_storage_link "forced on at the LTO link too (-Wl,-mllvm,-basic-aa-separate-storage)")
else()
set(_ripwire_separate_storage_link "LTO link keeps the linker default (ld64 refused -Wl,-mllvm,-basic-aa-separate-storage)")
endif()
else()
set(_ripwire_separate_storage_link "LTO link keeps the linker default (the ELF -plugin-opt spelling is unverified — see the comment above)")
endif()
endif()
message(STATUS "separate-storage alias analysis: forced on (-mllvm -basic-aa-separate-storage — turns BasicAA's separate_storage reader on for LLVM 17 / AppleClang 16, a no-op on LLVM 18+); ${_ripwire_separate_storage_link}")
else()
message(STATUS "separate-storage alias analysis: compiler default (${CMAKE_CXX_COMPILER_ID} ${CMAKE_CXX_COMPILER_VERSION} does not accept -mllvm -basic-aa-separate-storage; VERIFY_NO_ALIAS keeps its debug check, the release promise is whatever this optimizer does by default)")
endif()

# ---- opt-in ingestion fuzzing: isolated coverage-instrumented parser objects ----
# Keep these copies separate from production objects: fuzzer-no-link coverage callbacks must never leak
# into a normal executable. RIPWIRE_FUZZ is OFF by default and every target is EXCLUDE_FROM_ALL.
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1793,9 +1793,9 @@ wrong, and it has. These are the results that say so, all in-tree, all published
### In the tests

<details>
<summary><b>608 gate scripts</b>, five contracts no unit test can hold, and the house rule: write the gate before the code it measures</summary> <!-- gatecount -->
<summary><b>609 gate scripts</b>, five contracts no unit test can hold, and the house rule: write the gate before the code it measures</summary> <!-- gatecount -->

`test/regression.sh` names **608 gate scripts** and is the authoritative list; <!-- gatecount -->
`test/regression.sh` names **609 gate scripts** and is the authoritative list; <!-- gatecount -->
`python3 test/pargates.py . ./build/ripwire -j 6` runs the same set in parallel. On top of them sit the
contracts that do not fit a unit test: two runs byte-identical, warm output identical to cold, output
that pipes clean through `xmllint --noout`, a sanitizer build with `-fno-sanitize-recover=all`, and a
Expand Down
6 changes: 3 additions & 3 deletions docs/EVALS.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ section, and it is not an afterthought.
| **Co-change / known-item evals** | `--eval`, `--eval-retrieval` (see `bench/ANSWERQUALITY.md`) | Whether the tool surfaces the other files a real historical commit touched; and known-item retrieval across four rankers. |
| **Ensemble calibration harness** | `bench/ensemblecal/` | Whether `--ensemble`'s four evidence families are actually orthogonal, how often each fires, how stable each is across commits — and the preset ladder derived from that (§9). |
| **Differential argv harness** | `test/argvdiffcheck.sh` | That a refactor changed *nothing observable*: two binaries, every argv vector, stdout + stderr + exit code byte-identical. |
| **The gate suite** | `test/regression.sh`, `test/pargates.py` | 608 gate scripts plus the determinism, cache-transparency and golden contracts. <!-- gatecount --> |
| **The gate suite** | `test/regression.sh`, `test/pargates.py` | 609 gate scripts plus the determinism, cache-transparency and golden contracts. <!-- gatecount --> |
| **`--quality-delta`** | `src/quality.h` | Ten measured code-quality failure modes, reported only where a change made them worse. |

### The labeling protocol (why the held-out eval is allowed to disagree with the ranker)
Expand Down Expand Up @@ -5834,7 +5834,7 @@ copy here would be exactly the dialect divergence that gate exists to catch. Com
tags, wrap, stable-order defaults), seven individually invoked standalone gates (`g1freshcheck`,
`skillscan`, `htmlexport`, `compresscheck`, `handoffcheck`, `releaseinstallcheck`,
`taskroutecheck`), and a single loop
naming **608 gate scripts**, all of which exist on disk. <!-- gatecount -->
naming **609 gate scripts**, all of which exist on disk. <!-- gatecount -->

`python3 test/pargates.py . ./build/ripwire -j 6` runs the same scripts in parallel so a full
verification fits in one sitting. It does not modify `regression.sh`.
Expand Down Expand Up @@ -6846,7 +6846,7 @@ Listed because the reason is more useful than the silence.
shipped**. See `bench/locbench/anchorhop_calib.json`. The mention anchor's reproducible numbers are
the ablations in §4.
- **A single round gate-count.** Two in-tree numbers disagree (`test/pargates.py`'s docstring says
~210; `test/argvdiffcheck.sh` says 200+), while the loop in `test/regression.sh` names 608. The <!-- gatecount -->
~210; `test/argvdiffcheck.sh` says 200+), while the loop in `test/regression.sh` names 609. The <!-- gatecount -->
loop is the authority; the stale docstrings are a known drift. Since 2026-09-10 the number is not
written by hand anywhere: `docs/gatecount_build.py` derives it from the loop and rewrites every
published site, `test/gatecountcheck.sh` fails if any of them drifts, and `test/manifestcheck.sh`
Expand Down
6 changes: 3 additions & 3 deletions present/deck5_ripwire_build.js
Original file line number Diff line number Diff line change
Expand Up @@ -1068,7 +1068,7 @@ function storyCards(s, { kick, head, stories, footText }){
kicker(s, "// how it stays true", AMBER);
title(s, "Proven, not promised");
const cards = [
["608 gate scripts", "the suite runs on every push — plus determinism, cache-transparency and golden contracts; the gate count itself is gated against the runner's own loop"], // gatecount
["609 gate scripts", "the suite runs on every push — plus determinism, cache-transparency and golden contracts; the gate count itself is gated against the runner's own loop"], // gatecount
["byte-identical, always", "two runs over the same tree produce the same bytes; warm equals cold. Enforced in CI, twice — Release AND a plain flavour, because NDEBUG once blinded a whole class of checks"],
["differential refactoring", "a refactor must prove it changed nothing observable: two binaries, hundreds of argv vectors, stdout + stderr + exit codes byte-identical"],
["held-out labels, authored blind", "eval labels were written by reading source before the ranker ever ran on them — so the eval is allowed to say the ranker is wrong. It has."],
Expand All @@ -1092,7 +1092,7 @@ function storyCards(s, { kick, head, stories, footText }){
title(s, "Claims you can trust, because we publish what failed", { size: 32 });

card(s, MX, 1.72, 3.86, 1.72);
stat(s, "608", "gate scripts named by test/regression.sh — and the COUNT itself is gated against the runner's own loop, so it cannot go stale quietly", // gatecount
stat(s, "609", "gate scripts named by test/regression.sh — and the COUNT itself is gated against the runner's own loop, so it cannot go stale quietly", // gatecount
MX+0.15, 1.86, 3.56, CYAN, { bsize: 42, bh: 0.66, lsize: 9.5 });
card(s, 4.68, 1.72, 3.86, 1.72, CARD2);
stat(s, "8", "registered NEGATIVES — changes built, gated green, measured against a band written before the code, and reverted rather than tuned",
Expand Down Expand Up @@ -1342,7 +1342,7 @@ function storyCards(s, { kick, head, stories, footText }){
["179 long flags · 33 slides", "bash test/deckclaimcheck.sh"],
["every --flag named here exists", "bash test/deckcheck.sh"],
["74.7% fewer element bytes", "bash test/showcasecapturecheck.sh"],
["608 gate scripts", "bash test/manifestcheck.sh"], // gatecount
["609 gate scripts", "bash test/manifestcheck.sh"], // gatecount
["49 repos · 70 papers · 237 surveyed","bash test/readmedriftcheck.sh"],
["the ten moments, any row", "ripwire . --callers=SYM | wc -c"],
["the head-to-head table", "bench/headtohead/r4-2026-08-06/"],
Expand Down
Loading
Loading