Repository navigation
feat: quiet-break isolation gate + delivery redaction/restraint (E6-E11) - #40
Merged
Merged
Conversation
added 16 commits
September 22, 2026 18:15
stale_detector flagged 20 outdated version strings across 17 docs (en/ru/zh). Updated to the pyproject version; stale_detector now reports 0 drift.
Measures what execution-derived TESTS edges buy an LLM across four tasks.
- e17_extract.py: AST extraction of functions/tests (fixes qualified-name
match that returned FUNC NOT FOUND for every class method).
- e17_pilot_experiment.py: specificity- and complexity-selected panel,
unique/foreign/coverage-matched decoy, edge-case and impact questions.
- tooling: line-budget prompt shards for isolated subagent generation,
blind test-hidden judging, objective impact recall/precision.
- report v1-v4: reasoning tasks hit a ceiling (B never beats A); impact
('which tests cover F') is decisive for the signal (A=0.00, B=1.00).
- 13 extraction tests including adversarial cases.
Result recorded in AGENT_DIARY.md and KNOWN_ISSUES.md.
Negate the first single-line if inside 10 panel functions, run the graph-linked tests (arm B) and coverage-matched decoy tests (arm D), restore via git checkout. - B killed the mutant 6/10, D 0/10 (Fisher exact p=0.011). - 4/10 linked tests did not catch a strong mutation (weak assertions, e.g. TestCypherLexer asserts only len(tokens) > 0). - Conclusion: the signal identifies verifying tests far better than decoys, but its value is bounded by assertion strength, not coverage.
10 functions x 4 mutations (negate-if, flip-compare, bool-flip, int+1, str-append); every graph-linked test and decoy run under every mutant. - 24 tests profiled; decoy tests killed 0/104 mutants (control clean). - kill-rate predictors (Spearman): assertion specificity rho=+0.54 p=0.006, assert count rho=+0.47 p=0.019, density/LOC not significant. - What an assertion compares against matters more than how many it has. TESTS edge = reach; mutation kill-rate = verification.
…rate 24 functions (providers/mcp excluded), 55 tests, 4 mutations each; split by function; decoys killed 0/55. - v6 correlation (exact_compare rho=+0.54) did NOT replicate: train rho=+0.13, held-out rho=-0.01; rule precision 0.59 / recall 0.67. - AST features (assert count/density/specificity/LOC) do not predict whether a test catches a behaviour change; TEST_VERIFIES needs real mutation runs. - Report marks v6 CORRECTED; history preserved.
Two independent causes of search_code >15s timeouts (2026-09-22): - llama_runner watchdog unloads the embedder after EMBEDDER_IDLE_TIMEOUT (120s, free RAM) but RemoteEmbedder never brought it back -> :8080 connection refused (WinError 10061) -> per-chunk retries -> timeout. Add LlamaRunner.is_port_up()/ensure_embedder_started() and revive the server once on embed failure, before per-item retries. - _maybe_hot_reload awaited a full reindex inside search_code; a batch of new/changed files (22) pushed the call past the 15s budget even with a live embedder. Run it as a deduped background task. - Also fix the inverted model key in the (currently unreferenced) LlamaRunner.embed (bge-m3 -> DEFAULT_EMBEDDING_MODEL). Tests: tests/test_embedder_idle_recovery.py (10).
Detect commits that silently isolate a graph node, read-only (git diff + persisted PropertyGraph, no index mutation). Absolute zero-inbound-CALLS is unusable (1535/4081 = 38% of repo), so two delta signals: removed_last_caller (a symbol still defined loses its last caller) and new_orphan (a newly added function nothing calls). Exposed as graph_query(action=isolation) / CLI; fail-open (unavailable/empty) when git or graph is absent. Controls: unit 10/10, real-git E2E 4/4, CLI 1899ms 0 false positives.
Delivery notes (hooks -> python -m src.cli -> model context) bypass the commit secret-gate, so a key pasted into a note as evidence would leak into every agent context the note fires on. Add an in-house prefix-anchored redactor (src/core/redact.py) and apply it at the single CLI choke point (success and error JSON). Anchored on structural prefixes, not entropy, so paths, commit hashes and digests survive byte-identical. Not a security boundary (novel/split-line formats pass) - documented as such. Design credited to Tom Jones, crystal-memory scripts/redact.py (Apache-2.0, not vendored). Tests: 8/8 (unit + CLI integration).
Repeated identical advisory notes numb the reader (Tom Jones crystal-memory; our E3/E4). Add src/core/restraint.py: a findings signature (kind+symbol+file) with a per-project cooldown and 2^strikes backoff, state in data_root (atomic replace). Opt-in via graph_query isolation kwargs={restraint:true}; blocking gates never use it (a blocked commit is a hard stop, not nagging). Tests 4/4 (22/22 with gate+redact); CLI E2E: call1 deliver, immediate call2 suppressed (cooldown 597s).
Blind mapper against Tom's catalogue arrival index (tjonesit/crystals @3e30ed2): the dispositional arrival symptom (#16 agent won't use tools) now reaches family A in 5/5 runs including both VALID runs (longcat-2.0, qwen3.7-plus, controls 6/6). deepseek x3 invalid (NC3 UI-spacing lure, same NONE-control failure as E7). New risk noted: the arrival layer can manufacture a false positive on an adjacent domain.
Local-clone clean state via verify_clean_state.sh --no-clone on committed revision 44d451f: 1800 passed / 0 failed / 13 skipped / 98 deselected (822s); lock-drift negative control PASSED; guard inventory ALL PROVEN (3); revision gate VALID. Note: when a branch is not pushed, the default GitHub clone verifies origin, not this branch - clone the local repo and use --no-clone.
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Advanced Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
added 7 commits
September 23, 2026 06:25
test_graph_stage_e4 hardcoded file_path D:/Project/... which is not absolute on POSIX, so get_tests_for_symbol normalized it to <root>/D:/... and found no node -> 0 covering tests. Passed on Windows, failed on ubuntu (2 tests). Build absolute paths from tmp_path.as_posix() instead; fixture is now platform-independent.
Fix 808864c turned the ubuntu CI green; full run success. Only remaining red is GitHub Advanced Security (Copilot Autofind), which fails with CAPIError 400 model-not-supported on GitHub's side, not our code.
…visory) Project-local opencode plugin: on git commit, run stale_detector (block on objective doc drift) and graph_query isolation (advisory; graph gate has FP risk). Advisory uses before-stash/after-deliver by callID (E8 lesson). CLI invoked as python -m src.cli with cwd=project (CLI ignores --project). Env switches MSCODEBASE_PY/STALE_GATE/ISOLATION_GATE/GATE_LOG. Activated after Zed reload; global install is out of workspace and needs explicit approval.
In opencode 1.18.23 the plugin factory context does not provide a callable shell helper (docs describe v2 where it exists as BunShell). The hook loaded (init logged) but threw "TypeError: ... is not a function" on the shell call. Call python through node:child_process.execFile with an argv array (no shell quoting) instead of the context shell. node:fs already worked, so node builtins are available in the plugin runtime.
Fresh init after restart; git commit --dry-run -> stale drift=0, isolation count=0. Negative control (staged new_orphan added in a separate git add step) -> isolation count=1 and the advisory note delivered into the command output; test file removed, tree clean. Note: the hook runs before the command, so git add && git commit in one command leaves the file unstaged at hook time and the gate cannot see it.
…ackticks auto_update_docs(action=verify) raised IndexError "string index out of range": the empty-content guard ran before the $ prefix was stripped, so a lone $ (or $ ) in backticks left content empty and content[0] blew up. Re-check for empty content after stripping $. Adds a regression test.
The old verifier collected only definitions and scanned venv via rglob, so it reported 1366 broken with ~3% real (params/fields/stdlib). Replace the reference check with src/core/doc_reference_l1.py: vocabulary = identifiers + string literals from src/tests/scripts/tools (+root scripts); scope = live docs (root docs minus ledgers, docs/** minus archive/generated/research/blog/ISSUES/ investigations); whitelist stdlib/typing, external APIs, env vars, model names, commit scopes. auto_update_docs(action=verify) now delegates to it. Effect (measured): broken 1366 -> 0. Also fixes WISDOM naming a non-existent _symbol_resolver (actual: _build_symbol_resolver). Tests 4/4. Limitation: a stale tool name that still appears anywhere in code/tests is masked; tool names need a registry check, not a vocabulary check.
Owner
Author
|
CI green on all jobs (clean-state, ubuntu, windows, transport-equiv, manifest-parity, docker-build). Adds in this push:
Test time (measured): full suite ~200s warm locally (1817 passed), clean-state ~13m cold in CI. xdist not installed; parallelism is the main lever but needs an isolated correctness experiment before adoption. |
added 5 commits
September 24, 2026 22:51
xdist -n 4: 1817 passed in 169s vs 200s serial (~15%), no failures. Not worth default complexity; CI cold clean-state (~13m) is dominated by install/caches, not CPU. Named next levers: cache pip in clean-state and investigate test_temporal_facts_generator (~42s).
…+ 4 dev)
Authoritative count derived from code (server_tools.py tool_classes, tools_reg.py and
dev_tools.py decorators, server_tools.py inline decorators): 65 registered tools.
Fix contradicting numbers in README (61/66), AGENTS (64, 31 core) and
docs/{en,ru,zh}/ARCHITECTURE (64/61/58/46, 28/31/20 core); describe the visible set
via the MSCODEBASE_MCP_TOOLS allowlist instead of a stale magic number. L1 checker
clean (0 broken); tests 12/12.
Verified claims against code and corrected: tests badge 1856 -> 1889 (_count_tests), last-updated date, DI services 18 -> 14 (di_container.py has 14 add_singleton calls), tests 1773 -> 1889, and the tool-count line (32+16+13+4=65, +execute_script -> 66). Confirmed correct: 29 EdgeType members and 13 structural_search patterns. Open: AGENTS.md lists get_repo_map/... as "not registered" while they are registered tool_classes (hidden by allowlist) and README lists them - needs a decision.
Fixed: source-of-truth path src/core/intelligence_layer.py does not exist -> src/core/intelligence/layer.py; CI matrix "(3.10-3.12)" -> Python 3.14-only. Confirmed correct: RuntimeCoordinator/can_execute, ProjectContext.capture() (project_context.py:164), apply_file_move (indexer.py:659), root files. Open: the "Not registered (consolidated)" block lists get_repo_map/get_hotspots/ verify_action/get_task_status etc. as not exposed, but they are registered tool_classes (hidden by the default allowlist) - needs an owner decision.
…ounts Mechanism: only core tool_classes in the MSCODEBASE_MCP_TOOLS allowlist are registered via mcp.tool(); intel(16)/inline(13)/dev(4) are always registered. So the AGENTS claim was right in spirit (hidden -> tool not found) but the list was wrong (verify_action/get_task_status are in the default allowlist; bootstrap_ pipeline was missing). Rewrite the block as "Hidden core tools (default allowlist)"; add a visibility note to README; fix the startup log hardcode in server_tools.py (intel 14->16, inline 12->13) that printed "62 total" while 65 register.
added 2 commits
September 25, 2026 00:20
- Layer 2 "Bridge" marked deprecated (code: not used, LSP server removed 2026-07-20). - server.py "~220 lines" -> 68 (actual line count). - core "30 files" -> 54 top-level (123 incl. subpackages). - DI container "15+/18 services" -> 14. - README: lsp_main.py "does not work" -> "removed" (file deleted). Confirmed correct: tool counts 32/16/13/4=65 and module paths.
- remote_embedder described as ONNX-only -> llama.cpp GGUF primary with ONNX/LM Studio/Ollama fallback. - "15 node labels" -> 16 (NodeLabel members). - DI "Registered Services (11)" -> 14 (add Path, Indexer, GitUrlSourceFactoryKey). - Tool groups: Intelligence 14 -> 16 (add restore/supersede), Lifecycle 3 -> 4 (add get_action_receipt).
ManSio
marked this pull request as ready for review
September 25, 2026 14:12
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft PR to run CI on the experiment branch (ci.yml triggers only on main or a PR into main).
Contents since origin/main (16 commits):
New core: src/core/quiet_break_gate.py, src/core/redact.py, src/core/restraint.py;
tests: tests/test_quiet_break_gate.py, tests/test_redact.py, tests/test_restraint.py.
Local verification: clean-state 1800 passed / 0 failed (822s); ruff 0.15.22 clean.
This branch also carries the fix for the current red lint on main (graph_adapter.py:557, 929f0fd).