From 023671b563d479c18b87349bf9247a76ba621c84 Mon Sep 17 00:00:00 2001 From: Alberto Serrano-Calva Date: Thu, 6 Aug 2026 12:50:09 -0400 Subject: [PATCH 1/2] fix(bp-155): decide the L0a oversize cut over the canonical body (A1.2) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `_l0a_chunks` (core/ingest/code_corpus.py) decided the whole<->windowed split on the HEADER-BEARING length (`len(header + body) <= max_chars`), so the path's length participated in a chunk-identity decision: a rename or a move that crossed the budget flipped a slice whole<->windowed and minted a spurious atom (issue #31, bp-151's parked residue, ruled by Amendment A1.2 of dn-vector-membership-store). The fix is the one token the plan pins: `len(full) <= max_chars` becomes `len(body) <= max_chars`. Nothing else in the block moves — `text=full` stays `text=full`, so L0a embed text keeps its coordinate header (D0/R7). The KNOWN RESIDUE comment is replaced with one recording the decision is now canonical-body-scoped, per A1.2, and why. Converts bp-151's deliberate tripwire, `test_l0a_oversize_threshold_is_the_one_rename_residue` (its docstring named its own re-entry: "if #31 is ruled that way, this expectation becomes 0 and this test reddens"), to `test_l0a_oversize_cut_is_canonical_body_scoped`, asserting the residue is gone while keeping the straddle precondition (the fixture still crosses the OLD threshold, so the test has teeth). Measured (2026-08-06, real chunkers over all 580 tracked .py files, not a fixture): - Aggregate rename cost: 0 (was 3 at bp-151's 45c4a15 measurement; 23 on this tree pre-fix, confirming the measurement has teeth and the tree has grown since 45c4a15). A1.5's first falsifier does not fire. - L0a groups whose canonical body changed under the new rule: 72 across 57 files (was ~123/95 at 45c4a15; the tree has moved). Verified every one falls inside the predicted affected band (max_chars - len(header) <= len(body) <= max_chars) — A1.5's second falsifier does not fire. One additional group, core/ingest/code_corpus.py::_l0a_chunks, differs too, but that is this commit's own comment edit to the function whose body includes that comment, not a rule-driven change. Side effect, noted: for slices in the affected band, the OLD rule routed the body through `chunk_text`, whose `_blocks` calls `.strip()` on the block — silently stripping the first line's leading indentation for nested symbols even though the body itself fit under budget. The new rule takes the whole branch directly, so those 72 slices' canonical bodies (and embed text) are now the exact, unmangled source for the first time. No max_chars retune (A1.5's third falsifier), no other chunker change. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_011LZZQPyGsoeGL73cbbEp3U --- core/ingest/code_corpus.py | 12 ++++++------ tests/unit/test_code_corpus.py | 24 +++++++++++++++--------- 2 files changed, 21 insertions(+), 15 deletions(-) diff --git a/core/ingest/code_corpus.py b/core/ingest/code_corpus.py index 4167250..b5cee69 100644 --- a/core/ingest/code_corpus.py +++ b/core/ingest/code_corpus.py @@ -144,12 +144,12 @@ def _l0a_chunks(path: str, lines: list[str], shape: FileShape, *, body = "\n".join(lines[i - 1] for i in owned[key]) full = f"{header}\n{body}" # identity = the header-free body (D0); the header rides only on the embed text. - # KNOWN RESIDUE (issue #31, parked): this ONE cut is still decided over header-bearing - # length, so a rename that crosses the budget flips a slice whole↔windowed and mints 1 - # atom — the L1 mechanism surviving here. Deciding on len(body) is out of D0's bounds - # (§9: no other chunker behavior changes); it is the orchestrator's call, pinned by - # test_l0a_oversize_threshold_is_the_one_rename_residue. - if len(full) <= max_chars: + # The whole↔windowed cut is decided over the CANONICAL body (Amendment A1.2, + # dn-vector-membership-store): len(body), not len(full). This closes issue #31 — a + # rename that crossed the budget used to flip a slice whole↔windowed on path length + # alone, minting a spurious atom. The decision is now path-independent; the embed text + # emitted below is unchanged (`text=full`) so L0a text still keeps its header (D0/R7). + if len(body) <= max_chars: out.append(CodeChunk(LAYER_CODE_AST, key, ls, le, text=full, canonical_body=body)) else: # oversized slice: hard-split the body via the ONE window machinery, re-headered for piece in chunk_text(body, max_chars=max_chars, overlap_chars=overlap_chars): diff --git a/tests/unit/test_code_corpus.py b/tests/unit/test_code_corpus.py index a530e17..e084fd6 100644 --- a/tests/unit/test_code_corpus.py +++ b/tests/unit/test_code_corpus.py @@ -272,22 +272,28 @@ def test_every_chunk_pairs_a_headered_embed_text_with_a_header_free_canonical_bo assert path not in c.canonical_body # the mutable coordinate stays out -def test_l0a_oversize_threshold_is_the_one_rename_residue(): - """PARKED — issue #31, the single case where §8(h) does NOT hold as built. The oversize cut is - decided over the HEADER-BEARING length (`len(header + body) <= max_chars`), so a slice sitting - at the budget flips whole↔windowed when the path lengthens and mints one atom. Characterization, - not endorsement: deciding that cut over the canonical body is out of D0's bounds (§9, no other - chunker behavior changes) and is the orchestrator's call. RE-ENTRY: if #31 is ruled that way, - this expectation becomes 0 and this test reddens — that redness is the tripwire.""" +def test_l0a_oversize_cut_is_canonical_body_scoped(): + """bp-151's deliberate tripwire, formerly `test_l0a_oversize_threshold_is_the_one_rename_ + residue`: it pinned issue #31, the single case where §8(h) did NOT hold as built — the + oversize cut was decided over the HEADER-BEARING length (`len(header + body) <= max_chars`), + so a slice sitting at the budget flipped whole↔windowed when the path lengthened, minting one + spurious atom. Its docstring named its own re-entry verbatim: "if #31 is ruled that way, this + expectation becomes 0 and this test reddens — that redness is the tripwire." Amendment A1.2 + (dn-vector-membership-store) ruled it that way 2026-08-06; bp-155 landed the one-token fix + (`len(body) <= max_chars`) and this test reddened exactly as designed. Converted here to + assert the residue is GONE, guarding that the cut stays canonical-body-scoped going forward.""" here, moved = "a/m.py", "a/much_longer_module_name.py" body = "def f():\n y = 1\n\n return y" budget = len(f"# {here}:f()") + 1 + len(body) # exactly at the budget at the short path - # PRECONDITION: the rename really straddles the threshold — that IS the mechanism under test. + # PRECONDITION: the rename still straddles the threshold under the OLD (header-bearing) rule — + # that IS the mechanism under test. Without this, "0 minted" could mean the fixture never + # crossed the budget at either path, and the test would prove nothing. assert len(f"# {here}:f()") + 1 + len(body) <= budget < len(f"# {moved}:f()") + 1 + len(body) a = _at(here, body + "\n", LAYER_CODE_AST, max_chars=budget) b = _at(moved, body + "\n", LAYER_CODE_AST, max_chars=budget) assert len(a) == 1 # whole at the short path - assert len({c.content_hash for c in b} - {c.content_hash for c in a}) == 1 # ← issue #31 + assert len(b) == 1 # ...and now still whole at the long one + assert len({c.content_hash for c in b} - {c.content_hash for c in a}) == 0 # issue #31, closed # ── the STRUCTURAL CODE mint (F-CI1: no provenance parameter anywhere) ────────────────── From bfe933f35221c2e14fe7b51bd24ca3e0550c31d8 Mon Sep 17 00:00:00 2001 From: Alberto Serrano-Calva Date: Thu, 6 Aug 2026 12:51:29 -0400 Subject: [PATCH 2/2] =?UTF-8?q?docs(bp-155):=20seal=20the=20journal=20?= =?UTF-8?q?=E2=80=94=20the=20L0a=20canonical=20cut,=20measured?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Records the build: the one-token fix, the converted tripwire, the aggregate rename cost re-measured at 0 (A1.5's first falsifier did not fire), the 72/57 in-band boundary-change census (A1.5's second falsifier did not fire), the exact gate results with diff-innocence proven on the 5 known-red tests, and the .strip()-indentation side-finding surfaced while verifying Item 3. Read-map + Follow-through block per the checkpoint contract. --- docs/build-plans/bp-155/journal.md | 110 +++++++++++++++++++++++++++++ 1 file changed, 110 insertions(+) diff --git a/docs/build-plans/bp-155/journal.md b/docs/build-plans/bp-155/journal.md index 0d541a8..e8a9faf 100644 --- a/docs/build-plans/bp-155/journal.md +++ b/docs/build-plans/bp-155/journal.md @@ -1,5 +1,115 @@ # bp-155 — journal +## 2026-08-06 — build complete, all three items, PR open (SEAL) + +**Status.** Items 1–3 built and green on `build/bp-155-l0a-canonical-cut` (base `7c42a30`, +origin/main). The measured aggregate rename cost is **0** — D0 is complete across the tree, not +577/580 of it. Not a status flip — the owner's merge is the gate. + +**Completed.** + +- **Item 1 — the oversize cut decides over the canonical body.** `_l0a_chunks` + (`core/ingest/code_corpus.py:152`): `if len(full) <= max_chars:` → `if len(body) <= max_chars:`. + Nothing else in the block moved — `text=full` stays `text=full` (D0/R7). The KNOWN RESIDUE + comment (issue #31) is replaced by one recording the decision is canonical-body-scoped per + Amendment A1.2, and why (`:146-151`). bp-151's deliberate tripwire, + `test_l0a_oversize_threshold_is_the_one_rename_residue`, is converted to + `test_l0a_oversize_cut_is_canonical_body_scoped` (`tests/unit/test_code_corpus.py:275`): + same straddle-precondition fixture (the rename still crosses the OLD header-bearing threshold), + the mint-1 assertion flipped to mint-0, and the docstring records that this was bp-151's + tripwire and that it reddened exactly as designed when the fix landed. Confirmed by inversion: + stashing the fix and re-running the whole suite reproduces the original 1-atom residue at this + fixture; restoring it returns to 0. +- **Item 2 — aggregate rename cost re-measured at 0.** Script measurement (not a committed test — + matching bp-151's own precedent, whose 11,096/2,373/3 ladder was also ad hoc, not shipped as a + full-tree pytest test) over all **580** tracked `.py` files, real chunkers, real bytes: for each + file, derive at its own path and at a length-changed moved path (same directory, longer + basename), sum new `(layer, content_hash)` atoms across all three layers. **Result: 0.** Teeth + confirmed by re-running the identical script against the pre-fix code (`git stash` the one file): + **23** new atoms across 23 files — nonzero, and larger than bp-151's original 3 because the tree + has grown since `45c4a15`. A1.5's first falsifier does not fire. +- **Item 3 — boundary-change census: 72 groups across 57 files, all in-band.** Comparing L0a + `(path, qualname)` groups' canonical-body hashes before vs. after the fix (same script pattern, + stash/restore), **73** groups changed across **58** files; **72/57** of those are rule-driven, + and **1** (`core/ingest/code_corpus.py::_l0a_chunks` itself) is a measurement artifact — this + commit's own comment edit inside that function's body, not a rule effect (confirmed: its body + differs only in the reworded comment text, and every other changed group is untouched by any + source edit). Verified **all 72** genuine changes fall inside the predicted band + (`max_chars - len(header) <= len(body) <= max_chars`) — A1.5's second falsifier does not fire. + The tree has moved since `45c4a15`'s **123/95** measurement; this is what it reads now, not a + discrepancy. + +**A finding surfaced during Item 3's inspection, not a defect — recorded for the reviewer.** For +groups in the affected band, the OLD rule routed the body through `chunk_text` +(`core/kernel/ingest/chunk.py`), whose `_blocks` calls `.strip()` on each block even when the +whole body is a single block that fits under budget — silently stripping the **first line's +leading indentation** for any nested symbol (a class method's body starts with its indent). E.g. +`config/secrets_backend.py::VaultClient.mint_token`: old routing rendered +`'def mint_token(self, role...'` (dedented); the raw body is +`' def mint_token(self, role...'` (indented, correct). The new rule takes the whole branch +directly for these 72 slices, so their canonical bodies (and embed text) are the exact, +unmangled source for the first time — a side benefit of the fix, not a new behavior to chase. + +**Gate — exact results.** +- `ruff check .` — clean (0 errors) after fixing one E501 introduced by the converted test's + trailing comment. +- `mypy core agents eval ops scheduler scripts` — `Success: no issues found in 262 source files`. +- `mypy` (argless) — exits 1 as designed; tail `Found 69 errors in 20 files (checked 563 source + files)` — **69**, unmoved from the pinned baseline. +- `python -m ops.type_gate` — exit 0; Tier-2 membership OK, bare-ignore scan OK, one parked + non-fatal shim report (pre-existing, finding-0223, unrelated). +- `pytest -q` — **5 failed, 2427 passed, 15 skipped** in 224.14s. The 5: `test_dream_v2_live.py` + (1), `test_worktree_enforcement.py` (3: `test_a_deny_cross_worktree`, + `test_c_unsafe_direction_narrow_not_loosened`, `test_d_no_pointer_is_no_plan_not_main_fallback`), + `test_core_self_containment.py::test_core_imports_nothing_outside_core` (1) — exactly the three + known-red classes (finding-0103, e2e live, issue #13/finding-0280), same test names, same count + bp-151 reported. **Diff-innocence proven**: stashed both changed files back to a byte-identical + `origin/main` (`git diff --stat origin/main HEAD` empty), re-ran the full suite — **same 5 + failures, same names**, 249.76s. `test_scheduler_live.py`'s known flake did not fire either run. + +**In-flight.** Nothing. Working tree = the two write-scope files (committed `023671b`) plus this +journal entry. + +**Next action.** None for the builder. For the reviewer: audit the Item 3 in-band verification +(the `.strip()` side-finding above) and confirm the PR body's numbers against this entry. + +**Open questions.** None raised to an issue — the aggregate rename cost came back 0 (A1.5's first +falsifier did not fire), so there is no fourth path-dependent site to file. + +**Context-manifest delta.** Read beyond §2: `core/kernel/ingest/chunk.py`'s `_blocks` (load-bearing +for the Item 3 side-finding — the `.strip()` call is why "pieces stayed 1→1" for every in-band +group despite the canonical body changing). Nothing proved irrelevant beyond the manifest's own +scope. + +```read-map +docs/design-notes/vector-membership-store.md:512: A1.2's licence, verbatim — the one line that bounds this entire change +core/ingest/code_corpus.py:147: the replaced KNOWN RESIDUE comment — why the decision moved, per A1.2 +core/ingest/code_corpus.py:152: the one-token change itself — len(full) -> len(body) +core/kernel/ingest/chunk.py:33: _blocks' .strip() — why in-band groups keep pieces=1 but change hash (the side-finding) +tests/unit/test_code_corpus.py:275: the converted tripwire — same straddle fixture, mint-0 assertion, docstring records the redness-as-designed +docs/build-plans/bp-155/journal.md:1: this entry — the measured ladder (0 rename cost, 72/57 in-band boundary changes) and the .strip() side-finding +``` + +## Follow-through +- **Built?** Yes — Item 1 (the fix + converted tripwire), Item 2 (aggregate rename cost + re-measured at 0), Item 3 (boundary census at 72/57, all in-band) — all three plan items. +- **Wired / delivered (or why dormant)?** Live on the derivation path: `_l0a_chunks` is called + unconditionally by `derive_code_chunks`, which `CodeCorpusSync._embed_and_land` calls — the + next `code_sync` derives path-independent L0a boundaries with no switch to flip. No flag, none + wanted — this is a correction, not a feature. +- **Does a consumer use it?** Yes, immediately, and by design the 72 affected groups' stored rows + (wherever they exist) go stale the same way bp-151's did — derived ids no longer match stored + ones for those slices. No migration here (§9); bp-153's rebuild reconciles. Existing rows keep + serving retrieval meanwhile. +- **Track state (what remains on this track)?** D0 is now complete (11,096 → 3 → **0**). Next on + the revised order (A1.3): **bp-152** (membership store + path-free atom id), then **bp-153** + (the one rebuild, which must re-embed the 72 groups this plan moved). Neither is un-blocked by + anything this plan left undone — the aggregate came back 0. +- **Opened a new track/finding?** No. No issue filed — A1.5's falsifiers did not fire. The + `.strip()` side-finding is recorded here and in the PR body for the reviewer's awareness, not + filed as a defect (it is a strict improvement with no observed downside, folded into this + plan's own measured numbers rather than a separate track). + ## Pre-build notes for whoever picks this up - ⚑⚑ **The change is `len(full)` → `len(body)` in ONE `if`. Nothing else in that block moves.**