Skip to content

feat(calibration): per-repo loosening loop — repos earn overrides on their own evidence#8264

Merged
JSONbored merged 1 commit into
mainfrom
feat/per-repo-loosening-loop
Jul 23, 2026
Merged

feat(calibration): per-repo loosening loop — repos earn overrides on their own evidence#8264
JSONbored merged 1 commit into
mainfrom
feat/per-repo-loosening-loop

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

Closes #8217.

Adds the per-repo apply path on top of #8216's override precedence: repos whose own signal history is dense enough now earn repo-scoped overrides from their own evidence, instead of inheriting only the global knob.

  • runPerRepoKnobLoosening(env, knob): slices the trailing 90-day corpus by repo (sliceCorpusByRepo, contributor calibration: pure per-repo corpus slicing + density stats #8215), checks per-repo density eligibility (computeRepoCorpusDensity), and evaluates evaluateKnobLoosening on each eligible repo's own slice with the knob's unchanged split discipline.
  • Bounded work per tick: at most PER_REPO_LOOSENING_MAX_REPOS_PER_TICK (10) repos, resumed via a rotating per_repo_loosening_cursor:<knobId> system flag, so a large install never starves the dispatch tick.
  • Applies write the repo-scoped key (<overrideFlagKey>:repo:<owner/repo>) that calibration: per-repo knob overrides — storage scheme, resolution seam, and full settings wiring #8216's getKnobOverrideForRepo already consumes, with the same non-best-effort write + one structured alert per applied step, metadata.scope: "repo".
  • loadKnobStatus now reports repoOverrides so the knobs endpoint shows every earned repo override beside the global state.
  • Wired into the calibration tick after the global loop, double-gated by each knob's own autotune env var.

Validation

  • Full local gate npm run test:ci green (TEST_CI_EXIT=0), plus npm audit --audit-level=moderate.
  • Rebased onto current main (import-union conflict with the feat(calibration): reliability curves beside the ladder — knobs status + advisor surfacing #8262 reliability-curve merge resolved); post-rebase tsc --incremental false clean and the affected suites (knob-loosening-run, job-dispatch) pass 31/31.
  • 100% line+branch coverage on every changed range: density-ineligible repos, cursor rotation wraparound, apply refusal arms, and the empty-corpus fallbacks each have both sides exercised.

…their own evidence (#8217)

Epic #8211 track B capstone: on the same calibration tick, each live
knob's per-repo pass evaluates every repo whose OWN labeled slice clears
the knob's sample floors (computeRepoCorpusDensity — same split, same
minimums) from the repo's CURRENT resolved value, and applies to the
repo-scoped key under the identical discipline as the global loop:
smallest candidate step, visible improved + held-out non-regressed on the
REPO slice, hard minimum, double flag gating, one error-level alert per
applied step (ev stays per-knob, repo in the body — the
Sentry-fingerprint discipline). Sparse repos inherit global untouched.

Bounded work per tick: at most 10 eligible repos in deterministic order
with a rotating system_flags cursor, so a large-fleet future never turns
the tick into a stampede. Fail-safe per repo AND per tick, including
non-Error throws. 100% line+branch on the module.

Closes #8217
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@JSONbored JSONbored self-assigned this Jul 23, 2026
@JSONbored
JSONbored merged commit 264f26a into main Jul 23, 2026
10 checks passed
@JSONbored
JSONbored deleted the feat/per-repo-loosening-loop branch July 23, 2026 16:46
@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 23, 2026
@codecov

codecov Bot commented Jul 23, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.63%. Comparing base (904ec7d) to head (d615317).
⚠️ Report is 1 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8264      +/-   ##
==========================================
- Coverage   92.15%   91.63%   -0.53%     
==========================================
  Files         786      786              
  Lines       78814    78860      +46     
  Branches    23790    23800      +10     
==========================================
- Hits        72630    72261     -369     
- Misses       5062     5524     +462     
+ Partials     1122     1075      -47     
Flag Coverage Δ
shard-1 53.35% <100.00%> (-0.51%) ⬇️
shard-2 49.47% <4.25%> (-0.80%) ⬇️
shard-3 56.66% <4.25%> (-0.64%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/queue/job-dispatch.ts 99.32% <100.00%> (+<0.01%) ⬆️
src/services/knob-loosening-run.ts 100.00% <100.00%> (ø)

... and 3 files with indirect coverage changes

JSONbored added a commit that referenced this pull request Jul 23, 2026
orb-v3.3.0 shipped stable, so the beta cutter is idle until the manifest's
declared target moves forward. The calibration-expansion epic's merges since
(per-repo loosening #8264, reviewer votes #8265, tightening #8267, AMS loop
#8270, counterfactual replay #8271) are feats — the inferred bump is minor,
so the next automated snapshot targets orb-v3.4.0-beta.1.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

calibration: per-repo loosening evaluation — repos earn overrides on their own evidence

1 participant