From 2a30c346bd94befc9a5cd15d6a8d7577f919b2e1 Mon Sep 17 00:00:00 2001 From: joyful-ii-V-I Date: Wed, 9 Sep 2026 09:51:35 -0400 Subject: [PATCH 1/3] =?UTF-8?q?fix(crawl):=20the=20unindexed=20grep=20scan?= =?UTF-8?q?=20no=20longer=20serves=20gitignored=20files=20=E2=80=94=20the?= =?UTF-8?q?=20unsupported-ext=20class=20consults=20the=20ignore=20verdict?= =?UTF-8?q?=20before=20it=20records=20a=20row?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --grep/--regex additionally scan the crawl's unsupported-ext, text-looking population (CrawlSkips::unsupported) and print those hits in the trailing block. That population was recorded before the ignore set was consulted: collectSources classified the extension first (recordPreSizeDrop) and asked git only about files that survived, so a file that was BOTH gitignored AND of an unindexed extension was rowed as unsupported-ext and grep read it. Measured 2026-09-09: --regex='^#include' --grep-in=any served four hits from a .cpp.bak the repository's own .gitignore names (rg does not open it), and unindexed_files_scanned= / unsupported_ext= both counted it. The fix keeps the ordering contract (the ignore lookup stringifies the path, so it is still not paid for a binary asset or an --exclude'd file) and adds ONE consult for ONE class: the unsupported-ext branch asks a lazy `ignored` predicate — the same predicate the indexable-file test already uses, now defined once — before it records a row, and records nothing when the answer is yes. The file is then in NO class, exactly as an --exclude'd unsupported-ext file already was: not unsupported-ext (grep serves that list), and not ignored= (that counter describes only what would otherwise have been indexed, the number the header's accounting invariant carries). --no-ignore makes the predicate false, so the escape hatch restores the row and both counts with it. Gate first: test/grepignorecheck.sh (written RED — arms A B C D G failed on the pre-fix binary, every mutation control fired) pins the served set against an independent oracle (`git ls-files -co --exclude-standard` piped to grep -l, plus rg where present), on both --grep and the report's own --regex='^#include'; that unindexed_files_scanned= and the skipped verb's unsupported_ext= describe what was actually scanned; that a tracked file matching the pattern and an untracked-but-unignored file are still served (git's rule, not rg's textual one); the --no-ignore and non-git escape hatches; the MCP grep twin; determinism; xmllint. Absorbed into regression.sh's loop; the eight stated gate counts move 563 -> 564. Also: the skipped legend's definition of unsupported_ext= now names both exclusions; ARCHITECTURE.md still said ".gitignore is not consulted", which has been false since the N6-C lane — corrected. Replayed on the reporting corpus (read-only, --no-cache): default run 0 hits in the .bak, unindexed_files_scanned 409 -> 52 (+140 skipped = unsupported_ext 192, reconciled); --no-ignore still serves it. Co-Authored-By: Claude Fable 5.1 --- docs/ARCHITECTURE.md | 15 ++- src/ingest_crawl.h | 47 +++++-- src/model.h | 5 +- src/verbs_report.h | 4 +- test/grepignorecheck.sh | 272 ++++++++++++++++++++++++++++++++++++++++ test/regression.sh | 2 +- 6 files changed, 328 insertions(+), 17 deletions(-) create mode 100755 test/grepignorecheck.sh diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 1ec6d7be..26985679 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -46,12 +46,15 @@ runs. The parse itself runs one tree-sitter parser per worker thread and merges lists afterwards, which is safe precisely because the definitions and references are re-sorted before they are used — collection order never reaches the output. -**`.gitignore` is not consulted.** Skipping is a fixed, committed denylist (`kCrawlSkipDirs[]` in -`src/ingest.h`, shared with the CMake walk in `darkflags.h` so the two crawlers cannot disagree about -what counts as source), not a per-repository ignore file. That is a real difference from a -`.gitignore`-aware tool in both directions: a build directory this repository happens not to ignore is -still pruned, and a directory a project ignores but that is not on the list is still indexed. What is -skipped: +**`.gitignore` is consulted, after the denylist.** Skipping starts from a fixed, committed denylist +(`kCrawlSkipDirs[]` in `src/ingest.h`, shared with the CMake walk in `darkflags.h` so the two crawlers +cannot disagree about what counts as source). In a git work tree the crawl then also honours git's own +ignore verdict — one `git ls-files --others --ignored --exclude-standard --directory` fork per root, so +the answer is git's and never a re-implemented matcher — and `--no-ignore` turns that half off. The +denylist still prunes a build directory the repository happens not to ignore; what the repository +ignores leaves the map and is disclosed as `ignored_files=` / `ignored_dirs=`, and the `--grep` +unindexed scan reads none of it either (a gitignored file of an unindexed extension is in no class at +all, the same treatment an `--exclude`'d one gets). What the denylist skips: - **directories by NAME:** `.git`, `.claude`, `.hg`, `.svn`, `node_modules`, `vendor`, `third_party`, `.cache`, `build`, `dist`, `out`, `target`, `.venv`, `venv`, `__pycache__`, `.idea`, `.vscode`, diff --git a/src/ingest_crawl.h b/src/ingest_crawl.h index aef44b39..ae5cca35 100644 --- a/src/ingest_crawl.h +++ b/src/ingest_crawl.h @@ -828,16 +828,34 @@ void recordCrawlDrop( std::vector& rows, std::uint64_t& exactCount, // the user did NOT ask to hide (an --exclude'd .ml is requested absence, not a language this build cannot // read). Swap the two and both classes start lying. // +// THE THIRD TEST, FOR ONE CLASS ONLY (§N6-C, closed 2026-09-09). The unsupported-ext population is not +// merely reported — grep's aux scan (search.h grepCollectAux) READS it and SERVES its hits — so it must +// hold only files the REPOSITORY did not ask to hide either, on exactly the rule it already applies to +// --exclude. Before this test existed the crawl asked the ignore set only about files that survived +// here, so a file that was both gitignored and of an unindexed extension was rowed as unsupported-ext: +// measured, --regex='^#include' served four hits from a `.cpp.bak` beside its source that the +// repository's own .gitignore names (rg does not open it), while unindexed_files_scanned= and +// unsupported_ext= both counted it, so nothing disclosed that an ignored file had been read. +// test/grepignorecheck.sh pins the fix. `ignored` is a LAZY predicate for the same reason fullPath is: +// the lookup stringifies the path, so it is paid only once the two cheaper tests have already admitted +// the file to this class — never for a binary asset or an --exclude'd file, and never for an indexable +// file, which takes the crawl's own ignore test after this returns false. A file dropped here is in NO +// class — neither this one nor ignored=, exactly as an --exclude'd unsupported-ext file is in neither +// this one nor excluded=: ignored= describes only what would OTHERWISE HAVE BEEN INDEXED (the number the +// map header's accounting invariant carries), and a language this build cannot read that the repository +// hid is not a disclosure the reader is owed. --no-ignore makes the predicate false, so the escape hatch +// restores the row and both counts with it. +// // `fullPath` is the caller's LAZY path materializer, taken as a template parameter rather than a // std::string: a monorepo crawl walks far more non-source files than source ones, and stringifying every // one of them to record the handful that are reportable would be a real per-file cost for nothing. -template< typename PathFn > +template< typename PathFn, typename IgnoredFn > bool recordPreSizeDrop( CrawlSkips& skips, HashMap& extTally, - const std::string& ext, bool excluded, const fs::directory_entry& entry, PathFn&& fullPath ) + const std::string& ext, bool excluded, const fs::directory_entry& entry, PathFn&& fullPath, IgnoredFn&& ignored ) { if( lookupLang( ext ) == nullptr && !docparse::isDocExtension( ext ) ) { - if( !excluded && !isNonTextExtension( ext ) ) + if( !excluded && !isNonTextExtension( ext ) && !ignored() ) { ++extTally[ ext ]; recordCrawlDrop( skips.unsupported, skips.unsupportedFiles, fullPath(), ext, entry ); @@ -1002,8 +1020,11 @@ GitIgnoreSet collectGitIgnored( const char* rootDir ) // extension classification and the --exclude match (the same reason recordPreSizeDrop's header gives for // its own two): `ignored` then only ever describes a file that would OTHERWISE HAVE BEEN INDEXED, which is // what lets the header's accounting invariant carry it — indexed= + oversize= + excluded= + ignored= = the -// population the crawl enumerated — and keeps unsupported_ext=/unindexed= meaning exactly what they meant -// before this lane. The DIRECTORY test runs after the built-in denylist for the mirror reason: ignoredDirs= +// population the crawl enumerated. The ONE earlier consult is recordPreSizeDrop's unsupported-ext branch, +// which asks the same predicate before it records a row and records NOTHING when the answer is yes: that +// class is served by grep's aux scan, so unsupported_ext=/unindexed= describe the population grep actually +// reads, and an ignored file of an unindexed extension is counted in neither class (its header has the +// measured leak). The DIRECTORY test runs after the built-in denylist for the mirror reason: ignoredDirs= // then counts only the subtrees no rule this build already carried had pruned. bool pathInIgnoreSet( const std::vector& sorted, std::string_view rel ) noexcept { @@ -1220,16 +1241,26 @@ CrawlResult collectSources( const char* rootDir, const std::vector& // doc post-pass instead). Use the filename here so rejected regular files do not pay to stringify the // full path; materialize the full path only after the extension survives. // + // §N6-C: the repository's own verdict on this file, ONE lazy predicate shared by the two sites that + // ask it — the lookup stringifies the path (relForHash over fullPath), so it is evaluated only where + // a class actually consults it, never for a binary asset or an --exclude'd file. False under + // --no-ignore, on a non-git root, and when git could not answer (ignoreSet.available). + const auto ignored = [ & ]() -> bool + { + return ignoreSet.available && pathInIgnoreSet( ignoreSet.files, relForHash( fullPath(), rootDir ) ); + }; + // §L1: the two NON-SIZE drops are classified and recorded together (recordPreSizeDrop) — see its - // header for why the two tests must run in that order, and why they are not written inline here. + // header for why the tests must run in that order, why the unsupported-ext class alone consults the + // ignore verdict BEFORE it records a row, and why none of it is written inline here. const std::string ext = lowerExtensionOf( name ); - if( recordPreSizeDrop( skips, extTally, ext, excluded, *it, fullPath ) ) + if( recordPreSizeDrop( skips, extTally, ext, excluded, *it, fullPath, ignored ) ) { continue; } // §N6-C: AFTER the extension and the --exclude match — see pathInIgnoreSet's header for the ordering. - if( ignoreSet.available && pathInIgnoreSet( ignoreSet.files, relForHash( fullPath(), rootDir ) ) ) + if( ignored() ) { recordCrawlDrop( skips.ignored, skips.ignoredFiles, fullPath(), ext, *it ); continue; diff --git a/src/model.h b/src/model.h index e26317aa..93ebaa08 100644 --- a/src/model.h +++ b/src/model.h @@ -821,7 +821,10 @@ struct CrawlSkips // extension classification, the --exclude match and the built-in denylist, so every existing counter // keeps exactly the meaning it had: ignoredFiles counts files that would OTHERWISE HAVE BEEN INDEXED // (which is what makes it the number the header's accounting invariant can carry), and ignoredDirs - // counts only the subtrees no other rule had already pruned. + // counts only the subtrees no other rule had already pruned. The one class that consults the verdict + // EARLIER is `unsupported` above: grep serves that population, so a gitignored file of an unindexed + // extension is not rowed there either — it is in no class at all, exactly as an --exclude'd one + // already was (ingest_crawl.h recordPreSizeDrop's header). std::vector ignored; // capped rows, path-sorted — the individual ignored files std::vector ignoredDirRows; // capped rows, path-sorted — the pruned subtrees (bytes 0, ext "") std::uint64_t ignoredFiles = 0; // EXACT count (rows may be fewer) diff --git a/src/verbs_report.h b/src/verbs_report.h index 8e016fd3..fc63d8b8 100644 --- a/src/verbs_report.h +++ b/src/verbs_report.h @@ -1652,7 +1652,9 @@ constexpr const char* kSkippedLegend = " HEADER: indexed= is files= on the map; the ACCOUNTING INVARIANT is indexed= + oversize= + excluded= = the candidate" " population the crawl ENUMERATED, at every ceiling and exclude setting. unsupported_ext= counts source/text-looking files" " outside that population (binary/asset extensions are deliberately not counted — an unindexed .png is a picture, not a" - " language this build failed to read); its per-extension breakdown is the rows, which the map header rolls" + " language this build failed to read — and neither is a file an exclude or the repository's own ignore rules hid:" + " requested absence, not an unread language; the grep verb's unindexed scan reads exactly this class, nothing hidden);" + " its per-extension breakdown is the rows, which the map header rolls" " up as unindexed= — a TOP-6 list, and the map's unindexed_exts= beside it names how many DISTINCT such" " extensions exist, present exactly when that list was cut and absent when it is complete." " excluded_dirs= counts SUBTREES an exclude pruned: the walk stopped at the directory, so how many files" diff --git a/test/grepignorecheck.sh b/test/grepignorecheck.sh new file mode 100755 index 00000000..23fc2446 --- /dev/null +++ b/test/grepignorecheck.sh @@ -0,0 +1,272 @@ +#!/usr/bin/env bash +# grepignorecheck.sh — grep's UNINDEXED aux scan honours .gitignore, exactly as the crawl already does. +# +# THE DEFECT THIS PINS. ripwire's documented default is ripgrep's: in a git work tree the crawl honours +# git's ignore rules. It did — for every file it could have INDEXED. But --grep/--regex additionally scan +# the crawl's "unsupported-ext, text-looking" population (CrawlSkips::unsupported, search.h grepCollectAux) +# and print those hits in the trailing block, and THAT population was recorded before the +# ignore verdict was ever consulted: collectSources classified the extension first (recordPreSizeDrop, which +# rowed any grammar-less, non-asset extension as unsupported-ext) and tested the ignore set only on what +# survived. A file that was BOTH gitignored AND of an unindexed extension was therefore never asked, and +# grep read it and served it. Measured 2026-09-09 on the canyonraid48 corpus: +# ripwire --regex='^#include' --grep-in=any → four hits inside canyon/personality.cpp.bak +# git check-ignore -v canyon/personality.cpp.bak → .gitignore:78:canyon/*.bak +# rg '^#include' → does not open it +# The header's own counters agreed with the leak: unindexed_files_scanned= counted the file, and the +# skipped verb's unsupported_ext= counted it too, so nothing disclosed that an ignored file had been read. +# +# THE CONTRACT. The unsupported-ext class is not merely REPORTED, it is SERVED — so it may hold only files +# the repository did not ask to hide, on the same rule the class already applies to --exclude (an +# --exclude'd .ml is requested absence, not a language this build cannot read). A file dropped that way is +# in NO class: not unsupported-ext, and not ignored= either, because ignored= describes only what would +# OTHERWISE HAVE BEEN INDEXED — the number the map header's accounting invariant carries. And the two +# counts that describe this population — the grep root's unindexed_files_scanned= and the skipped verb's +# unsupported_ext= — must describe what was ACTUALLY scanned: a file dropped here is counted nowhere. +# +# Assertions (RED-FIRST: recorded against the pre-fix binary — A B C D G fail, the rest already pass): +# 0 presence guards — git really ignores the hidden file, really does NOT ignore the tracked-anyway +# one, and (when rg is on PATH) the independent tool agrees about the hidden file +# A --grep: the set of files served (indexed block AND unindexed block) is EXACTLY the set git does +# not ignore that contains the needle — an independent oracle, `git ls-files -co --exclude-standard` +# piped to grep -l; and the gitignored file appears NOWHERE in the answer +# B --regex: the same equality for the pattern the report was filed with, '^#include' +# C unindexed_files_scanned= counts only what was served (it is the number of aux files git does not +# ignore, derived from git check-ignore, never from ripwire) +# D --skipped agrees: unsupported_ext= equals that same number, equals unindexed_files_scanned=, the +# hidden file has no row, the histogram counts it out, and ignored= is untouched +# (the file is in neither class — the accounting invariant test/gitignorecheck.sh arm 4 pins) +# E --no-ignore is a real escape hatch: the hidden file is served again and both counts grow by one +# F a NON-GIT root is unchanged: every text file is served (the feature may not shrink a corpus it +# cannot explain) +# G the MCP grep verb serves the same population (it reuses grepCollectAux over the same rows) +# H determinism: two default runs byte-identical +# I MUTATION self-tests — each assertion can see its own regression +# J G4: xmllint --noout clean +# +# Usage: test/grepignorecheck.sh # uses build/ripwire +# RIPWIRE_BIN=asan/ripwire test/grepignorecheck.sh +# Exits non-zero on any failure. Does NOT edit regression.sh. + +set -u +ROOT="$( cd "$( dirname "$0" )/.." && pwd )" +BIN="${1:-${RIPWIRE_BIN:-$ROOT/build/ripwire}}" +[ "${BIN#/}" = "$BIN" ] && BIN="$ROOT/$BIN" # allow a repo-relative RIPWIRE_BIN +TMP="$( mktemp -d )"; trap 'rm -rf "$TMP"' EXIT +fail=0 +ok(){ printf ' PASS %s\n' "$*"; } +no(){ printf ' FAIL %s\n' "$*"; fail=1; } + +[ -x "$BIN" ] || { echo "no ripwire binary at $BIN — build first (cmake --build build -j)"; exit 2; } +command -v python3 >/dev/null 2>&1 || { echo "python3 required"; exit 2; } +command -v git >/dev/null 2>&1 || { echo "grepignorecheck: no git on PATH — the feature's whole premise; SKIP"; exit 0; } +cd "$ROOT" +echo "grepignorecheck: BIN=$BIN" + +# ── the fixture: one git repo, four text files of an UNINDEXED extension, and one indexed control ────── +# main.cpp tracked, indexed — proves the indexed half of the answer still works +# personality.cpp.bak untracked, IGNORED by `*.bak` → must not be served (the defect) +# kept.bak matches `*.bak` but force-added, TRACKED → must be served (git's rule, rg's blind spot) +# stale.scm tracked, unsupported-ext → must be served +# draft.scm UNTRACKED and NOT ignored → must be served (untracked ≠ ignored) +# Every text file carries the shared needle so one grep shows the whole served set; only the indexed file +# lacks a `#include` line, so the regex arm's oracle set differs from the literal arm's by exactly it. +R="$TMP/repo"; mkdir -p "$R" +printf '*.bak\n' >"$R/.gitignore" +printf 'int rwIgnMainSymbol( int a ) { return a; }\n// rwIgnNeedleShared indexed\n' >"$R/main.cpp" +printf '#include \nrwIgnNeedleShared hidden\nrwIgnNeedleHiddenOnly\n' >"$R/personality.cpp.bak" +printf '#include \nrwIgnNeedleShared tracked\n' >"$R/kept.bak" +printf '#include \nrwIgnNeedleShared stale\n' >"$R/stale.scm" +printf '#include \nrwIgnNeedleShared draft\n' >"$R/draft.scm" +( + cd "$R" || exit 1 + git init -q . >/dev/null 2>&1 + git config user.email gate@example.invalid + git config user.name gate + git add .gitignore main.cpp stale.scm >/dev/null 2>&1 + git add -f kept.bak >/dev/null 2>&1 # tracked THOUGH .gitignore names it — the point of arm 0/A + git commit -qm fixture >/dev/null 2>&1 +) +AUX_FILES="personality.cpp.bak kept.bak stale.scm draft.scm" +HIDDEN="personality.cpp.bak" + +# ── extractors ───────────────────────────────────────────────────────────────────────────────────────── +# served(): every in the answer, indexed block AND unindexed block, sorted unique. +served(){ + python3 -c ' +import re, sys +xml = sys.stdin.read().split( "-->", 1 )[ -1 ] +print( "\n".join( sorted( set( re.findall( r"/dev/null | sort -u ); } +# oracle_all ERE: every file under the tree that contains ERE, ignore rules NOT applied (the --no-ignore +# and non-git expectation). +oracle_all(){ ( cd "$1" && grep -rlE --exclude-dir=.git -- "$2" . 2>/dev/null | sed 's|^\./||' | sort -u ); } +# the number of aux files git does not ignore — from git check-ignore, so C/D never learn it from ripwire +AUX_UNIGNORED=0 +for f in $AUX_FILES; do ( cd "$R" && git check-ignore -q "$f" ) || AUX_UNIGNORED=$(( AUX_UNIGNORED + 1 )); done +AUX_TOTAL="$( printf '%s\n' $AUX_FILES | grep -c . )" + +rw(){ "$BIN" "$R" "$@" --grep-in=any --limit=100000 --no-cache 2>/dev/null; } + +# ── 0) presence guards ───────────────────────────────────────────────────────────────────────────────── +( cd "$R" && git check-ignore -q "$HIDDEN" ) \ + && ok "(0) git ignores $HIDDEN (the fixture is what the arms assume)" \ + || no "(0) git does NOT ignore $HIDDEN — every later arm is vacuous" +( cd "$R" && git check-ignore -q kept.bak ) \ + && no "(0) git ignores kept.bak though it is tracked — the fixture's force-add did not take" \ + || ok "(0) git does not ignore the TRACKED kept.bak despite the *.bak rule" +[ "$AUX_UNIGNORED" -eq $(( AUX_TOTAL - 1 )) ] \ + && ok "(0) $AUX_UNIGNORED of $AUX_TOTAL aux files survive git's ignore rules (exactly the hidden one does not)" \ + || no "(0) expected $(( AUX_TOTAL - 1 )) unignored aux files, git says $AUX_UNIGNORED" +if command -v rg >/dev/null 2>&1; then + ( cd "$R" && rg -l rwIgnNeedleShared . 2>/dev/null | grep -q "$HIDDEN" ) \ + && no "(0) rg served $HIDDEN — the independent tool disagrees with git about the fixture" \ + || ok "(0) rg does not open $HIDDEN (independent tool agrees)" +else + ok "(0) rg absent — independent-tool cross-check skipped" +fi + +# ── A) --grep: the served set is exactly git's not-ignored set; the hidden file appears nowhere ─────── +A_OUT="$( rw --grep=rwIgnNeedleShared )" +A_RW="$( printf '%s' "$A_OUT" | served )" +A_OR="$( oracle rwIgnNeedleShared )" +if [ "$A_RW" = "$A_OR" ]; then + ok "(A) --grep serves exactly the files git does not ignore: $( printf '%s' "$A_OR" | tr '\n' ' ' )" +else + no "(A) --grep's served set differs from git's not-ignored set" + printf ' ripwire: %s\n' "$( printf '%s' "$A_RW" | tr '\n' ' ' )" + printf ' git : %s\n' "$( printf '%s' "$A_OR" | tr '\n' ' ' )" +fi +printf '%s' "$A_OUT" | grep -q "$HIDDEN" \ + && no "(A) the gitignored $HIDDEN appears in the --grep answer" \ + || ok "(A) the gitignored $HIDDEN appears nowhere in the --grep answer" +# the hidden file's PRIVATE needle: a zero here is the whole point, and the legend's complete= governs it +[ "$( rw --grep=rwIgnNeedleHiddenOnly | served | grep -c . )" -eq 0 ] \ + && ok "(A) a needle that lives only in the ignored file answers zero files" \ + || no "(A) a needle that lives only in the ignored file still finds it" + +# ── B) --regex, the pattern the report was filed with ───────────────────────────────────────────────── +B_RW="$( rw --regex='^#include' | served )" +B_OR="$( oracle '^#include' )" +if [ "$B_RW" = "$B_OR" ]; then + ok "(B) --regex='^#include' serves exactly git's not-ignored set: $( printf '%s' "$B_OR" | tr '\n' ' ' )" +else + no "(B) --regex's served set differs from git's not-ignored set" + printf ' ripwire: %s\n' "$( printf '%s' "$B_RW" | tr '\n' ' ' )" + printf ' git : %s\n' "$( printf '%s' "$B_OR" | tr '\n' ' ' )" +fi + +# ── C) unindexed_files_scanned= describes what was served ───────────────────────────────────────────── +C_N="$( printf '%s' "$A_OUT" | attr unindexed_files_scanned )" +[ "${C_N:-x}" = "$AUX_UNIGNORED" ] \ + && ok "(C) unindexed_files_scanned=$C_N = the $AUX_UNIGNORED aux files git does not ignore" \ + || no "(C) unindexed_files_scanned=${C_N:-absent}, want $AUX_UNIGNORED — the counter still counts the file the scan must not read" + +# ── D) --skipped agrees with the grep root, and the hidden file is in NO class ───────────────────────── +D_OUT="$( "$BIN" "$R" --skipped --no-cache 2>/dev/null )" +D_UNS="$( printf '%s' "$D_OUT" | attr unsupported_ext )" +D_IGN="$( printf '%s' "$D_OUT" | attr ignored )" +[ "${D_UNS:-x}" = "$AUX_UNIGNORED" ] \ + && ok "(D) --skipped unsupported_ext=$D_UNS = the $AUX_UNIGNORED aux files git does not ignore" \ + || no "(D) --skipped unsupported_ext=${D_UNS:-absent}, want $AUX_UNIGNORED" +[ "${D_UNS:-x}" = "${C_N:-y}" ] \ + && ok "(D) unsupported_ext= ($D_UNS) and unindexed_files_scanned= ($C_N) describe the same population" \ + || no "(D) unsupported_ext=${D_UNS:-absent} disagrees with unindexed_files_scanned=${C_N:-absent}" +printf '%s' "$D_OUT" | grep -q "p=\"$HIDDEN\"" \ + && no "(D) --skipped still rows $HIDDEN ($( printf '%s' "$D_OUT" | grep -oE "p=\"$HIDDEN\" why=\"[^\"]*\"" | head -1 | grep -oE 'why="[^"]*"' ))" \ + || ok "(D) --skipped has no row for $HIDDEN — requested absence of an unread language is no class" +printf '%s' "$D_OUT" | grep -q '' \ + && ok "(D) the histogram counts the tracked .bak only (files=1)" \ + || no "(D) the histogram still counts the ignored file: $( printf '%s' "$D_OUT" | grep -oE '' | head -1 )" +[ "${D_IGN:-x}" = "0" ] \ + && ok "(D) ignored=0 — an ignored file of an unindexed extension is not counted as would-have-been-indexed" \ + || no "(D) ignored=${D_IGN:-absent}: the accounting invariant (indexed+oversize+excluded+ignored) now counts a file that could never have been indexed" + +# ── E) --no-ignore is a real escape hatch ────────────────────────────────────────────────────────────── +E_OUT="$( rw --grep=rwIgnNeedleShared --no-ignore )" +E_RW="$( printf '%s' "$E_OUT" | served )" +E_OR="$( oracle_all "$R" rwIgnNeedleShared )" +[ "$E_RW" = "$E_OR" ] \ + && ok "(E) --no-ignore serves every text file, ignored or not: $( printf '%s' "$E_OR" | tr '\n' ' ' )" \ + || { no "(E) --no-ignore did not restore the full set"; printf ' ripwire: %s\n tree : %s\n' "$( printf '%s' "$E_RW" | tr '\n' ' ' )" "$( printf '%s' "$E_OR" | tr '\n' ' ' )"; } +E_N="$( printf '%s' "$E_OUT" | attr unindexed_files_scanned )" +E_UNS="$( "$BIN" "$R" --skipped --no-ignore --no-cache 2>/dev/null | attr unsupported_ext )" +[ "${E_N:-x}" = "$AUX_TOTAL" ] && [ "${E_UNS:-x}" = "$AUX_TOTAL" ] \ + && ok "(E) under --no-ignore both counts grow to $AUX_TOTAL (unindexed_files_scanned=$E_N, unsupported_ext=$E_UNS)" \ + || no "(E) under --no-ignore the counts are unindexed_files_scanned=${E_N:-absent} unsupported_ext=${E_UNS:-absent}, want $AUX_TOTAL" + +# ── F) a NON-GIT root is unchanged ───────────────────────────────────────────────────────────────────── +cp -R "$R" "$TMP/nogit"; rm -rf "$TMP/nogit/.git" +F_RW="$( "$BIN" "$TMP/nogit" --grep=rwIgnNeedleShared --grep-in=any --limit=100000 --no-cache 2>/dev/null | served )" +F_OR="$( oracle_all "$TMP/nogit" rwIgnNeedleShared )" +[ "$F_RW" = "$F_OR" ] \ + && ok "(F) a non-git root serves every text file (no ignore rules to honour)" \ + || { no "(F) a non-git root lost files — the feature shrank a corpus it cannot explain"; printf ' ripwire: %s\n' "$( printf '%s' "$F_RW" | tr '\n' ' ' )"; } + +# ── G) the MCP grep verb serves the same population ─────────────────────────────────────────────────── +mcp_text(){ + printf '%s\n' '{"jsonrpc":"2.0","id":1,"method":"initialize"}' "$1" \ + | "$BIN" --mcp 2>/dev/null | tail -1 | python3 -c ' +import sys, json +r = json.load( sys.stdin ) +print( "__ERROR__:" + r[ "error" ].get( "message", "" ) if "error" in r else r[ "result" ][ "content" ][ 0 ][ "text" ] ) +' +} +call(){ printf '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"%s","arguments":%s}}' "$1" "$2"; } +mcp_text "$( call grep '{"path":"'"$R"'","pattern":"rwIgnNeedleShared"}' )" >"$TMP/mcp.json" +G_RW="$( python3 -c ' +import json, sys +j = json.load( open( sys.argv[ 1 ] ) ) +files = { h.get( "file", "" ) for h in j.get( "hits", [] ) } | { r.get( "file", "" ) for r in j.get( "unindexed", {} ).get( "rows", [] ) } +print( "\n".join( sorted( files ) ) ) +print( "scanned=%s" % j.get( "unindexed_files_scanned", "absent" ) ) +' "$TMP/mcp.json" 2>/dev/null )" +G_SET="$( printf '%s\n' "$G_RW" | grep -v '^scanned=' )" +G_N="$( printf '%s\n' "$G_RW" | grep '^scanned=' | cut -d= -f2 )" +[ "$G_SET" = "$A_OR" ] \ + && ok "(G) MCP grep serves exactly git's not-ignored set" \ + || { no "(G) MCP grep's served set differs from git's not-ignored set"; printf ' mcp: %s\n' "$( printf '%s' "$G_SET" | tr '\n' ' ' )"; head -c 300 "$TMP/mcp.json"; echo; } +[ "${G_N:-x}" = "$AUX_UNIGNORED" ] \ + && ok "(G) MCP unindexed_files_scanned=$G_N agrees with the CLI" \ + || no "(G) MCP unindexed_files_scanned=${G_N:-absent}, want $AUX_UNIGNORED" + +# ── H) determinism ───────────────────────────────────────────────────────────────────────────────────── +rw --grep=rwIgnNeedleShared >"$TMP/d1"; rw --grep=rwIgnNeedleShared >"$TMP/d2" +diff -q "$TMP/d1" "$TMP/d2" >/dev/null && ok "(H) two runs byte-identical" || no "(H) output is nondeterministic" + +# ── I) MUTATION self-tests: each assertion must be able to see its own regression ───────────────────── +# (A) equality: the oracle with the hidden file appended is the pre-fix served set — must NOT equal. +MUT_A="$( printf '%s\n%s\n' "$A_OR" "$HIDDEN" | sort -u )" +[ "$MUT_A" = "$A_OR" ] \ + && no "(I) mutation (hidden file added to the served set): still equals the oracle — (A) is decoration" \ + || ok "(I) mutation (hidden file added to the served set) correctly FAILS assertion (A)" +# (A) absence: the probe must SEE the hidden file when it is legitimately served (the --no-ignore answer). +printf '%s' "$E_OUT" | grep -q "$HIDDEN" \ + && ok "(I) mutation control: the absence probe sees $HIDDEN in the --no-ignore answer — (A) is not vacuous" \ + || no "(I) mutation control: the absence probe cannot see $HIDDEN even when it is served" +# (C)/(D) counters: the --no-ignore count differs from the default expectation, so the comparison can fire. +[ "${E_N:-x}" = "$AUX_UNIGNORED" ] \ + && no "(I) mutation (count under --no-ignore) still equals the default expectation — (C) is decoration" \ + || ok "(I) mutation (count under --no-ignore = ${E_N:-absent}) correctly FAILS assertion (C)" +# (B) regex: perturbing one path in the served set must break the oracle comparison. +MUT_B="$( printf '%s\n' "$B_OR" | sed '1s/$/.moved/' )" +[ "$MUT_B" = "$B_OR" ] \ + && no "(I) mutation (one path renamed): still equals the oracle — (B) is decoration" \ + || ok "(I) mutation (one path renamed) correctly FAILS assertion (B)" + +# ── J) G4: well-formed XML ───────────────────────────────────────────────────────────────────────────── +if command -v xmllint >/dev/null 2>&1; then + printf '%s' "$A_OUT" | xmllint --noout - 2>/dev/null && ok "(J) xml well-formed (xmllint)" || no "(J) xml malformed" + printf '%s' "$D_OUT" | xmllint --noout - 2>/dev/null && ok "(J) --skipped xml well-formed (xmllint)" || no "(J) --skipped xml malformed" +else + ok "(J) xmllint absent — skipped" +fi + +[ "$fail" -eq 0 ] && echo "ALL PASS" || echo "SOME FAILED" +exit "$fail" diff --git a/test/regression.sh b/test/regression.sh index b9bad5ad..9515431b 100755 --- a/test/regression.sh +++ b/test/regression.sh @@ -265,7 +265,7 @@ else RIPWIRE_BIN="$BIN" bash "$ROOT/test/codexdoctorcheck.sh" 2>&1 | sed 's/^/ | /' fi # retired: cacheexclkeycheck — the per-configuration auto-cache key it pinned is a registered NEGATIVE (docs/EVALS.md, "The auto-cache key ignores --exclude", RUN 2026-09-03: a 158K-file root with >= 12 gate configurations thrashed the 2 GiB sweep); the retry design keeps ONE superset blob per root and will bring its own gate -for _g in a9disclosurecheck abicheck accessshapecheck ackonlycheck adaptivecheck adaptivecutshapecheck affectedcheck agentloopclaudecheck agentloopcodexcheck agentloopeditsuitecheck agentloopfollowupcheck agentloopgradercheck agentlooplockcheck agentloopopencodecheck agenttablecheck aiderbytescheck anchorbodycheck anchorcheck archcheck archmetricscheck argvdiffcheck arisefollowupcheck ariseshimcheck aritycheck artifactcheck atcheck atomscheck attrvocabcheck baselinecheck baselinedirtycheck baselineportcheck bashsourcecheck batchcheck binoverridecheck blindspotcheck bm25boundcheck bm25check bodiesshowncheck bodydialectcheck budgetpolicycheck bundleidcheck cachefuzzcheck cachehashcheck cacheidentitycheck cacheisolationcheck cachelintcheck cacheoffsetcheck cachesplitcheck callerscheck callformcheck candheadcheck candidatescheck canoncheck ccheck ccjsoncheck chacheck chaconecheck chainguardcheck chainidcheck churndecaycheck churnjoincheck churnjsonstampcheck clicheck clonebandcheck clonecachecheck clonededupcheck cloneidiomcheck clonelexcheck clsrecvcheck cochangeboostcheck cochangecliocheck cochangesurprisecheck codexinstallhonestycheck codexplugincheck codexwrapcheck collectioncapcheck columnarattrcheck columnarcheck columnarcommacheck commentcoherencecheck communitydrillcheck communitylabelcheck compactlegendcheck compactroutecheck completecheck composelangcheck connectcheck connectcorecheck connectjoincheck constcheck contextratiocheck coplintcheck cppbenchcheck cppoperatorcheck cppqualcheck crossdirincludecheck crossrefcheck crossrefdegradecheck csharpcheck csharpcondcheck cudacheck cyclecutcheck deadcheck deadfiltercheck deadprecisioncheck deckcheck deckclaimcheck deeptailcheck defaultceilingcheck defoverdeclcheck degradedhintcheck dependencypincheck deplangscheck depsprecisecheck detailcheck didyoumeancheck dispatchordercheck dmmcheck docanchorcheck docdemotecheck docdriftcheck docdriftcommentcheck docmdcachecheck docmentioncheck docscommandscheck doctorcheck donelegendcheck droppedpositivecheck duprowcheck dynmapsimdcheck editcheckcheck editchecknotecheck edithandlehintcheck editpayloadbinarycheck editplancheck editplanpayloadconfinecheck editplanrecheckcheck editplanrollbackmsgcheck editpreviewcheck editroundtripcheck edittargetfileabscheck eliximportcheck elixircheck emittertruthcheck emptycorpuscheck emptyvaluerefusecheck ensembleavailcheck ensemblecheck essentialcxcheck estchargecheck evalcheck evictioncheck exemplarcheck exemplarconfcheck exercisescheck expandcallscheck expandmodecheck expandrangecheck expandsibscheck expandtokencheck expandtopk0check externalvetocheck fficheck fieldaffinitycheck fieldnarrowcheck fieldusescheck filerootcheck fileselectorrefusecheck fillordercheck fixedbufsweep flagscheck flagsnoisecheck flagsurfacecheck flagtablecheck flipcheck floormarkcheck fnptrcheck forautobodycheck forbudgetmonotoncheck forcalibfactscheck forcompresscheck fordisclosurecheck forlenscheck formatgatecheck formaxtokenscheck fornotesbudgetcheck fornotesjsoncheck forrankordercheck forrootlegendcheck freshclonecheck freshnesscheck g1configcheck gateabilitycheck gateexitcheck genrecallcheck githardencheck gitignorecheck gitquotepathcheck gitstampcheck goinstcheck gointerfacecheck graphlegendbudgetcheck graphqueryrefusecheck grepanchorcheck grepandcheck grepbytescheck grepcheck grepcontextcheck grepcorpuscheck grepfastcheck grepfollowupcheck grepscancheck grepseamcheck greptiercheck guardmsgcheck hasacheck headsnapcachecheck hermesinstallcheck historyoraclecheck hookcheck hostilecheck hotspotsincecheck htmlcolorcheck htmlhostcheck htmlrendercheck identitycheck impactimportcheck impactpartitioncheck importnarrowcheck includeanglecheck includeprecisecheck indexoutcheck infraportcheck isolateprovenancecheck javarubycheck jslangcheck jsmetricscheck jsnestedcheck jsoncheck jsonlangcheck jsonparitycheck jsonredactcheck jsonrefusallegendcheck jsonwalkcheck jsshapecheck jsverbscheck knownitemcheck landingcheck langcensuscheck langcheck layerquerycheck layoutcheck lb3namecheck legendcostcheck legendcoveragecheck legenddriftcheck legobundlecheck legocheck lintbudgetcheck lintcatalogcheck lintcheck lintdedupcheck lintpayloadcapcheck lintprecisioncheck lintrulescheck lintscopecheck lintselectcheck localitycheck localscountcheck loopconservationcheck lpincheck luacheck luarequirecheck macroedgecheck manifestcheck mapdiffcheck matchcapturecheck matchgrammarcheck maxfilesizecheck mcpattrparitycheck mcpaudit4hardencheck mcpclidiffcheck mcpcodexmetacheck mcpcontractcheck mcpdegradedhintcheck mcpeditcheck mcpeditkindcheck mcpeditmodecheck mcpeditpresencecheck mcpeditracecheck mcpflagshipcheck mcpforparitycheck mcpframehonestycheck mcpgrepdegradedcheck mcphandlecheck mcpincrementalcheck mcpmanifestcheck mcprangeedgecheck mcpreadloopcheck mcpredactcheck mcpreloadcheck mcpremotecheck mcprobustcheck mcpslicecheck mcpstalecheck mcpstrictschemacheck mcptoolprunecheck mcptranchecheck mcpverbscheck mcpw2fixcheck mcpw3fixcheck mcpwatchercheck mdembedcheck mdsectioncheck mentioncheck mentionsverbcheck mergechurncheck mergescoutcheck mergescoutlonglinecheck metalcheck meterdisclosurecheck metricscheck modifierguardcheck moduleconstcheck morecontractcheck mrowalkcheck multirootcheck multiswecheck namedfileinputcheck nameinfocheck namingcalibrationcheck namingconsistencycheck naminglenscheck naminglocalscheck narrowcheck narrowlangcheck neighbourcapcheck nestedimportcheck nestedqualcheck nestprofilecheck nextverbcheck nongitqmetricscheck nonlocalstatecheck notecanoncheck notescheck nsfiltercheck nulbytecheck numericrefusecheck objcfieldcheck objcsniffcheck opencodewrapcheck optremarkscheck ordercheck outlinecheck overbudgetcommentcheck ownerscheck packcallersharecheck packtaskcheck packtaskmonotoncheck packtaskquotacheck padscalecheck paginationcheck pagingsweepcheck panellegendcheck pargatescheck parsehealthcheck partitioncheck patterncheck perfharnesscheck phpcheck pincensuscheck planlanescheck planlintcheck pmccheck portablebuildcheck portablecachecheck postingscheck ppaltcheck pranchorcheck prbudgetcheck prcheck prcontextcheck prconvergecheck precedencecheck preproccondcheck prmaskanchorcheck prnestedcapcheck probecheck propcostcheck prrefsafecheck prrenamecheck pyimportprecisecheck pyshapecheck qackconcurrencycheck qackorigincheck qchurncheck qchurnmemocheck qdrefpaircheck qextractionkeycheck qoriginoraclecheck qrevtokencheck qrowlocatorcheck qschemetripcheck qsnapcachecheck qsnapprefetchcheck qualifiedresolvecheck qualitycheck qualitycrosslangcheck qualityexcludecheck qualitykeycheck qualitykindscheck qualityorigincheck qualitypanelcheck qualityscopecheck qualitysignalcheck qualitystalecheck qualitysymcheck qualnewcheck querycheck queryfilescancheck racymtimecheck radixsimdcheck rangecomposecheck rankbycheck reachcheck readabilitycheck readmedriftcheck readmeexamplecheck recallanchorcheck recallboundarycheck recallbudgetcheck recallbufcheck recallevalcheck recallparitycheck recallpassagecheck recallrankdepthcheck recallrelcheck recalltablecheck recalltotalcheck receiptpostcheck redactcheck redactfixcheck refusaltailcheck regexbombcheck regexcheck regexrefusecheck registermacrocheck relevancefloorcheck relinkcheck reportcheck resolvecheck resolverhonestycheck retrievalqualitycheck reusefirstworkflowcheck ripwirepubliccheck rootrelcheck rootrelemitcheck routecheck routeedgecheck routehookcheck routeoncecheck routingreportcheck rubyconstcheck rubymetricscheck rubyrecvcheck rubyrequirecheck rubyscopecheck rubysettercheck runhintcheck runtracecheck rustimportprecisecheck rustqualcheck safedeletecheck sarifcheck savecachecheck scipcheck scipjoincheck scorecardcheck scoutheadconflictcheck scoutkeycheck seedboundscheck selectorchaincheck selectorhonestycheck selectorrefusecheck selectorscopecheck selfcontainedcheck shadowcheck shapingflagcheck shellgateindexcheck showcasecapturecheck sibliftcheck sigredactcheck sincecheck sincecochangecheck sincewindowcheck singledefcheck situdiffcheck skilldescbudgetcheck skillevalcheck skillevalsplitcheck skillinstallcheck skillroutingjudgedcheck skillscanreadcheck skilltruthcheck skippedcheck skipreasoncheck slicecheck slicediffcheck sliceflowcheck sliceflowsenscheck spectimingcheck staleackcheck statgatecheck sublistcountcheck substrfiltercheck subtokencheck svectorcheck swiftcheck swiftmemberscheck swiftshapecheck taskechocheck termmargincheck testedreachcheck testgatecheck testgatelegendbudgetcheck testgatepagecheck testgaterefusecheck testmacrocheck testrowruncheck testscopecheck textdocscheck tokenbudgetcheck tomllangcheck toolcallroutecheck tornreadcheck tracecheck tracehopcheck traceminecheck treecheck truncvocabcheck tsimportprecisecheck tsshapecheck type3check type3clonecheck typerefcheck unreachablecheck unresolvedcheck usescheck usesselectorcheck usingdeclcheck utf8scrubcheck vendoredassetcheck vendoredbundlecheck vendorpatchcheck verifycheck versioncheck w2verbscheck w3fixbudgetcheck w3fixlegendcheck weaksignalcheck withgraphcheck withprofilecheck wrapverbscheck writetargetcheck xmlwellformed yamllangcheck zonecheck zoneconsistencycheck zoomcheck; do +for _g in a9disclosurecheck abicheck accessshapecheck ackonlycheck adaptivecheck adaptivecutshapecheck affectedcheck agentloopclaudecheck agentloopcodexcheck agentloopeditsuitecheck agentloopfollowupcheck agentloopgradercheck agentlooplockcheck agentloopopencodecheck agenttablecheck aiderbytescheck anchorbodycheck anchorcheck archcheck archmetricscheck argvdiffcheck arisefollowupcheck ariseshimcheck aritycheck artifactcheck atcheck atomscheck attrvocabcheck baselinecheck baselinedirtycheck baselineportcheck bashsourcecheck batchcheck binoverridecheck blindspotcheck bm25boundcheck bm25check bodiesshowncheck bodydialectcheck budgetpolicycheck bundleidcheck cachefuzzcheck cachehashcheck cacheidentitycheck cacheisolationcheck cachelintcheck cacheoffsetcheck cachesplitcheck callerscheck callformcheck candheadcheck candidatescheck canoncheck ccheck ccjsoncheck chacheck chaconecheck chainguardcheck chainidcheck churndecaycheck churnjoincheck churnjsonstampcheck clicheck clonebandcheck clonecachecheck clonededupcheck cloneidiomcheck clonelexcheck clsrecvcheck cochangeboostcheck cochangecliocheck cochangesurprisecheck codexinstallhonestycheck codexplugincheck codexwrapcheck collectioncapcheck columnarattrcheck columnarcheck columnarcommacheck commentcoherencecheck communitydrillcheck communitylabelcheck compactlegendcheck compactroutecheck completecheck composelangcheck connectcheck connectcorecheck connectjoincheck constcheck contextratiocheck coplintcheck cppbenchcheck cppoperatorcheck cppqualcheck crossdirincludecheck crossrefcheck crossrefdegradecheck csharpcheck csharpcondcheck cudacheck cyclecutcheck deadcheck deadfiltercheck deadprecisioncheck deckcheck deckclaimcheck deeptailcheck defaultceilingcheck defoverdeclcheck degradedhintcheck dependencypincheck deplangscheck depsprecisecheck detailcheck didyoumeancheck dispatchordercheck dmmcheck docanchorcheck docdemotecheck docdriftcheck docdriftcommentcheck docmdcachecheck docmentioncheck docscommandscheck doctorcheck donelegendcheck droppedpositivecheck duprowcheck dynmapsimdcheck editcheckcheck editchecknotecheck edithandlehintcheck editpayloadbinarycheck editplancheck editplanpayloadconfinecheck editplanrecheckcheck editplanrollbackmsgcheck editpreviewcheck editroundtripcheck edittargetfileabscheck eliximportcheck elixircheck emittertruthcheck emptycorpuscheck emptyvaluerefusecheck ensembleavailcheck ensemblecheck essentialcxcheck estchargecheck evalcheck evictioncheck exemplarcheck exemplarconfcheck exercisescheck expandcallscheck expandmodecheck expandrangecheck expandsibscheck expandtokencheck expandtopk0check externalvetocheck fficheck fieldaffinitycheck fieldnarrowcheck fieldusescheck filerootcheck fileselectorrefusecheck fillordercheck fixedbufsweep flagscheck flagsnoisecheck flagsurfacecheck flagtablecheck flipcheck floormarkcheck fnptrcheck forautobodycheck forbudgetmonotoncheck forcalibfactscheck forcompresscheck fordisclosurecheck forlenscheck formatgatecheck formaxtokenscheck fornotesbudgetcheck fornotesjsoncheck forrankordercheck forrootlegendcheck freshclonecheck freshnesscheck g1configcheck gateabilitycheck gateexitcheck genrecallcheck githardencheck gitignorecheck grepignorecheck gitquotepathcheck gitstampcheck goinstcheck gointerfacecheck graphlegendbudgetcheck graphqueryrefusecheck grepanchorcheck grepandcheck grepbytescheck grepcheck grepcontextcheck grepcorpuscheck grepfastcheck grepfollowupcheck grepscancheck grepseamcheck greptiercheck guardmsgcheck hasacheck headsnapcachecheck hermesinstallcheck historyoraclecheck hookcheck hostilecheck hotspotsincecheck htmlcolorcheck htmlhostcheck htmlrendercheck identitycheck impactimportcheck impactpartitioncheck importnarrowcheck includeanglecheck includeprecisecheck indexoutcheck infraportcheck isolateprovenancecheck javarubycheck jslangcheck jsmetricscheck jsnestedcheck jsoncheck jsonlangcheck jsonparitycheck jsonredactcheck jsonrefusallegendcheck jsonwalkcheck jsshapecheck jsverbscheck knownitemcheck landingcheck langcensuscheck langcheck layerquerycheck layoutcheck lb3namecheck legendcostcheck legendcoveragecheck legenddriftcheck legobundlecheck legocheck lintbudgetcheck lintcatalogcheck lintcheck lintdedupcheck lintpayloadcapcheck lintprecisioncheck lintrulescheck lintscopecheck lintselectcheck localitycheck localscountcheck loopconservationcheck lpincheck luacheck luarequirecheck macroedgecheck manifestcheck mapdiffcheck matchcapturecheck matchgrammarcheck maxfilesizecheck mcpattrparitycheck mcpaudit4hardencheck mcpclidiffcheck mcpcodexmetacheck mcpcontractcheck mcpdegradedhintcheck mcpeditcheck mcpeditkindcheck mcpeditmodecheck mcpeditpresencecheck mcpeditracecheck mcpflagshipcheck mcpforparitycheck mcpframehonestycheck mcpgrepdegradedcheck mcphandlecheck mcpincrementalcheck mcpmanifestcheck mcprangeedgecheck mcpreadloopcheck mcpredactcheck mcpreloadcheck mcpremotecheck mcprobustcheck mcpslicecheck mcpstalecheck mcpstrictschemacheck mcptoolprunecheck mcptranchecheck mcpverbscheck mcpw2fixcheck mcpw3fixcheck mcpwatchercheck mdembedcheck mdsectioncheck mentioncheck mentionsverbcheck mergechurncheck mergescoutcheck mergescoutlonglinecheck metalcheck meterdisclosurecheck metricscheck modifierguardcheck moduleconstcheck morecontractcheck mrowalkcheck multirootcheck multiswecheck namedfileinputcheck nameinfocheck namingcalibrationcheck namingconsistencycheck naminglenscheck naminglocalscheck narrowcheck narrowlangcheck neighbourcapcheck nestedimportcheck nestedqualcheck nestprofilecheck nextverbcheck nongitqmetricscheck nonlocalstatecheck notecanoncheck notescheck nsfiltercheck nulbytecheck numericrefusecheck objcfieldcheck objcsniffcheck opencodewrapcheck optremarkscheck ordercheck outlinecheck overbudgetcommentcheck ownerscheck packcallersharecheck packtaskcheck packtaskmonotoncheck packtaskquotacheck padscalecheck paginationcheck pagingsweepcheck panellegendcheck pargatescheck parsehealthcheck partitioncheck patterncheck perfharnesscheck phpcheck pincensuscheck planlanescheck planlintcheck pmccheck portablebuildcheck portablecachecheck postingscheck ppaltcheck pranchorcheck prbudgetcheck prcheck prcontextcheck prconvergecheck precedencecheck preproccondcheck prmaskanchorcheck prnestedcapcheck probecheck propcostcheck prrefsafecheck prrenamecheck pyimportprecisecheck pyshapecheck qackconcurrencycheck qackorigincheck qchurncheck qchurnmemocheck qdrefpaircheck qextractionkeycheck qoriginoraclecheck qrevtokencheck qrowlocatorcheck qschemetripcheck qsnapcachecheck qsnapprefetchcheck qualifiedresolvecheck qualitycheck qualitycrosslangcheck qualityexcludecheck qualitykeycheck qualitykindscheck qualityorigincheck qualitypanelcheck qualityscopecheck qualitysignalcheck qualitystalecheck qualitysymcheck qualnewcheck querycheck queryfilescancheck racymtimecheck radixsimdcheck rangecomposecheck rankbycheck reachcheck readabilitycheck readmedriftcheck readmeexamplecheck recallanchorcheck recallboundarycheck recallbudgetcheck recallbufcheck recallevalcheck recallparitycheck recallpassagecheck recallrankdepthcheck recallrelcheck recalltablecheck recalltotalcheck receiptpostcheck redactcheck redactfixcheck refusaltailcheck regexbombcheck regexcheck regexrefusecheck registermacrocheck relevancefloorcheck relinkcheck reportcheck resolvecheck resolverhonestycheck retrievalqualitycheck reusefirstworkflowcheck ripwirepubliccheck rootrelcheck rootrelemitcheck routecheck routeedgecheck routehookcheck routeoncecheck routingreportcheck rubyconstcheck rubymetricscheck rubyrecvcheck rubyrequirecheck rubyscopecheck rubysettercheck runhintcheck runtracecheck rustimportprecisecheck rustqualcheck safedeletecheck sarifcheck savecachecheck scipcheck scipjoincheck scorecardcheck scoutheadconflictcheck scoutkeycheck seedboundscheck selectorchaincheck selectorhonestycheck selectorrefusecheck selectorscopecheck selfcontainedcheck shadowcheck shapingflagcheck shellgateindexcheck showcasecapturecheck sibliftcheck sigredactcheck sincecheck sincecochangecheck sincewindowcheck singledefcheck situdiffcheck skilldescbudgetcheck skillevalcheck skillevalsplitcheck skillinstallcheck skillroutingjudgedcheck skillscanreadcheck skilltruthcheck skippedcheck skipreasoncheck slicecheck slicediffcheck sliceflowcheck sliceflowsenscheck spectimingcheck staleackcheck statgatecheck sublistcountcheck substrfiltercheck subtokencheck svectorcheck swiftcheck swiftmemberscheck swiftshapecheck taskechocheck termmargincheck testedreachcheck testgatecheck testgatelegendbudgetcheck testgatepagecheck testgaterefusecheck testmacrocheck testrowruncheck testscopecheck textdocscheck tokenbudgetcheck tomllangcheck toolcallroutecheck tornreadcheck tracecheck tracehopcheck traceminecheck treecheck truncvocabcheck tsimportprecisecheck tsshapecheck type3check type3clonecheck typerefcheck unreachablecheck unresolvedcheck usescheck usesselectorcheck usingdeclcheck utf8scrubcheck vendoredassetcheck vendoredbundlecheck vendorpatchcheck verifycheck versioncheck w2verbscheck w3fixbudgetcheck w3fixlegendcheck weaksignalcheck withgraphcheck withprofilecheck wrapverbscheck writetargetcheck xmlwellformed yamllangcheck zonecheck zoneconsistencycheck zoomcheck; do [ -f "$ROOT/test/$_g.sh" ] || continue if RIPWIRE_BIN="$BIN" bash "$ROOT/test/$_g.sh" >/dev/null 2>&1; then ok "absorb gate ($_g.sh)" From aa324788e03005a2c28b2755676f5b0947dcdf60 Mon Sep 17 00:00:00 2001 From: joyful-ii-V-I Date: Wed, 9 Sep 2026 10:13:41 -0400 Subject: [PATCH 2/3] =?UTF-8?q?fix(grepignorecheck):=20scrub=20the=20priva?= =?UTF-8?q?te=20tree's=20name=20out=20of=20the=20gate=20header=20=E2=80=94?= =?UTF-8?q?=20ripwirepubliccheck=20arm=201?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The header's measured-defect paragraph named the private validation corpus and its directory; CI run 34359885383 (Release clang shard 3/4) failed ripwirepubliccheck.sh arm 1 on it, the one gate that sweeps every tracked file for exactly this. Fixed forward, not rewritten: the name stays in this branch's history at 4c249fd0, which makes the branch a squash-merge candidate at landing (the owner's call, as for the tgrep and codeburn lanes this round). Wording only; the gate's arms and its fixture are untouched. Co-Authored-By: Claude Fable 5.1 --- test/grepignorecheck.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/test/grepignorecheck.sh b/test/grepignorecheck.sh index 23fc2446..e557bfb8 100755 --- a/test/grepignorecheck.sh +++ b/test/grepignorecheck.sh @@ -8,9 +8,9 @@ # ignore verdict was ever consulted: collectSources classified the extension first (recordPreSizeDrop, which # rowed any grammar-less, non-asset extension as unsupported-ext) and tested the ignore set only on what # survived. A file that was BOTH gitignored AND of an unindexed extension was therefore never asked, and -# grep read it and served it. Measured 2026-09-09 on the canyonraid48 corpus: -# ripwire --regex='^#include' --grep-in=any → four hits inside canyon/personality.cpp.bak -# git check-ignore -v canyon/personality.cpp.bak → .gitignore:78:canyon/*.bak +# grep read it and served it. Measured 2026-09-09 on a private validation corpus: +# ripwire --regex='^#include' --grep-in=any → four hits inside /personality.cpp.bak +# git check-ignore -v /personality.cpp.bak → .gitignore:78:/*.bak # rg '^#include' → does not open it # The header's own counters agreed with the leak: unindexed_files_scanned= counted the file, and the # skipped verb's unsupported_ext= counted it too, so nothing disclosed that an ignored file had been read. From 4e4e0703658e3f63a7e79d813a3f037a6bc6efbd Mon Sep 17 00:00:00 2001 From: joyful-ii-V-I Date: Wed, 9 Sep 2026 15:03:53 -0400 Subject: [PATCH 3/3] =?UTF-8?q?chore(gates):=20re-derive=20the=20gate=20co?= =?UTF-8?q?unt=20after=20rebasing=20onto=20595c8196=20=E2=80=94=20569=20->?= =?UTF-8?q?=20570=20at=20the=20eight=20stated=20sites?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The loop in test/regression.sh names 570 with grepignorecheck absorbed; the number is derived from the loop on this tip, never carried from an earlier base (this lane and the tgrep lane both derived 568 against the same 567 main, each correct in isolation, and identical text at all eight sites merges cleanly and silently). Sites: README.md x2, docs/EVALS.md x3 (incl. the §8 prose line), present/deck5_ripwire_build.js x3. test/manifestcheck.sh green at 570. Co-Authored-By: Claude Fable 5.1 --- README.md | 4 ++-- docs/EVALS.md | 6 +++--- present/deck5_ripwire_build.js | 6 +++--- 3 files changed, 8 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 293e7634..5fa2dafc 100644 --- a/README.md +++ b/README.md @@ -1800,9 +1800,9 @@ wrong, and it has. These are the results that say so, all in-tree, all published ### In the tests
-569 gate scripts, five contracts no unit test can hold, and the house rule: write the gate before the code it measures +570 gate scripts, five contracts no unit test can hold, and the house rule: write the gate before the code it measures -`test/regression.sh` names **569 gate scripts** and is the authoritative list; +`test/regression.sh` names **570 gate scripts** and is the authoritative list; `python3 test/pargates.py . ./build/ripwire -j 6` runs the same set in parallel. On top of them sit the contracts that do not fit a unit test: two runs byte-identical, warm output identical to cold, output that pipes clean through `xmllint --noout`, a sanitizer build with `-fno-sanitize-recover=all`, and a diff --git a/docs/EVALS.md b/docs/EVALS.md index 52026bb7..355930e7 100644 --- a/docs/EVALS.md +++ b/docs/EVALS.md @@ -21,7 +21,7 @@ section, and it is not an afterthought. | **Co-change / known-item evals** | `--eval`, `--eval-retrieval` (see `bench/ANSWERQUALITY.md`) | Whether the tool surfaces the other files a real historical commit touched; and known-item retrieval across four rankers. | | **Ensemble calibration harness** | `bench/ensemblecal/` | Whether `--ensemble`'s four evidence families are actually orthogonal, how often each fires, how stable each is across commits — and the preset ladder derived from that (§9). | | **Differential argv harness** | `test/argvdiffcheck.sh` | That a refactor changed *nothing observable*: two binaries, every argv vector, stdout + stderr + exit code byte-identical. | -| **The gate suite** | `test/regression.sh`, `test/pargates.py` | 569 gate scripts plus the determinism, cache-transparency and golden contracts. | +| **The gate suite** | `test/regression.sh`, `test/pargates.py` | 570 gate scripts plus the determinism, cache-transparency and golden contracts. | | **`--quality-delta`** | `src/quality.h` | Ten measured code-quality failure modes, reported only where a change made them worse. | ### The labeling protocol (why the held-out eval is allowed to disagree with the ranker) @@ -5579,7 +5579,7 @@ copy here would be exactly the dialect divergence that gate exists to catch. Com tags, wrap, stable-order defaults), seven individually invoked standalone gates (`g1freshcheck`, `skillscan`, `htmlexport`, `compresscheck`, `handoffcheck`, `releaseinstallcheck`, `taskroutecheck`), and a single loop -naming **569 gate scripts**, all of which exist on disk. +naming **570 gate scripts**, all of which exist on disk. `python3 test/pargates.py . ./build/ripwire -j 6` runs the same scripts in parallel so a full verification fits in one sitting. It does not modify `regression.sh`. @@ -6491,7 +6491,7 @@ Listed because the reason is more useful than the silence. shipped**. See `bench/locbench/anchorhop_calib.json`. The mention anchor's reproducible numbers are the ablations in §4. - **A single round gate-count.** Two in-tree numbers disagree (`test/pargates.py`'s docstring says - ~210; `test/argvdiffcheck.sh` says 200+), while the loop in `test/regression.sh` names 569. The + ~210; `test/argvdiffcheck.sh` says 200+), while the loop in `test/regression.sh` names 570. The loop is the authority; the stale docstrings are a known drift. `test/manifestcheck.sh` asserts this very number against the loop's actual length, so it cannot go stale silently again. - **"282 argv vectors."** The gate asserts a floor of ≥250 assembled from five sources; 282 was a diff --git a/present/deck5_ripwire_build.js b/present/deck5_ripwire_build.js index 02050af1..54d4aa2e 100644 --- a/present/deck5_ripwire_build.js +++ b/present/deck5_ripwire_build.js @@ -708,7 +708,7 @@ function row(s, y, h, cols, opts={}){ kicker(s, "// how it stays true", AMBER); title(s, "Proven, not promised"); const cards = [ - ["569 gate scripts", "the suite runs on every push — plus determinism, cache-transparency and golden contracts; the gate count itself is gated against the runner's own loop"], + ["570 gate scripts", "the suite runs on every push — plus determinism, cache-transparency and golden contracts; the gate count itself is gated against the runner's own loop"], ["byte-identical, always", "two runs over the same tree produce the same bytes; warm equals cold. Enforced in CI, twice — Release AND a plain flavour, because NDEBUG once blinded a whole class of checks"], ["differential refactoring", "a refactor must prove it changed nothing observable: two binaries, hundreds of argv vectors, stdout + stderr + exit codes byte-identical"], ["held-out labels, authored blind", "eval labels were written by reading source before the ranker ever ran on them — so the eval is allowed to say the ranker is wrong. It has."], @@ -732,7 +732,7 @@ function row(s, y, h, cols, opts={}){ title(s, "Claims you can trust, because we publish what failed", { size: 32 }); card(s, MX, 1.72, 3.86, 1.72); - stat(s, "569", "gate scripts named by test/regression.sh — and the COUNT itself is gated against the runner's own loop, so it cannot go stale quietly", + stat(s, "570", "gate scripts named by test/regression.sh — and the COUNT itself is gated against the runner's own loop, so it cannot go stale quietly", MX+0.15, 1.86, 3.56, CYAN, { bsize: 42, bh: 0.66, lsize: 9.5 }); card(s, 4.68, 1.72, 3.86, 1.72, CARD2); stat(s, "8", "registered NEGATIVES — changes built, gated green, measured against a band written before the code, and reverted rather than tuned", @@ -971,7 +971,7 @@ function row(s, y, h, cols, opts={}){ ["179 long flags · 29 slides", "bash test/deckclaimcheck.sh"], ["every --flag named here exists", "bash test/deckcheck.sh"], ["74.7% fewer element bytes", "bash test/showcasecapturecheck.sh"], - ["569 gate scripts", "bash test/manifestcheck.sh"], + ["570 gate scripts", "bash test/manifestcheck.sh"], ["46 repos · 69 papers · 237 surveyed","bash test/readmedriftcheck.sh"], ["the ten moments, any row", "ripwire . --callers=SYM | wc -c"], ["the head-to-head table", "bench/headtohead/r4-2026-08-06/"],