Skip to content

release-train: develop -> staging - #464

Merged
tracebloc-release-train[bot] merged 3 commits into
stagingfrom
release-train/to-staging
Sep 10, 2026
Merged

release-train: develop -> staging#464
tracebloc-release-train[bot] merged 3 commits into
stagingfrom
release-train/to-staging

Conversation

@LukasWodka

@LukasWodka LukasWodka commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Automated promotion by the release train (RFC-0008 D14). Head is the train-managed release-train/to-staging branch (a mirror of develop), so it never collides with a human PR. Merged only when the fr-gate is green.


Note

Medium Risk
Changes required CI gate behavior (pagination and longer waits) for all repos using Bugbot gate; incorrect pagination logic could still block or mis-evaluate large PRs, though selftests and mutations target that path.

Overview
Bugbot review gate now paginates GraphQL checkSuites and reviewThreads so PRs with more than 100 threads or suites are no longer permanently refused as “truncated” (backend#3530). Shared query fragments, follow-up pages, bounded cursor handling, and a pageInfo self-check back the new follow_pages path; selftests and mutation cases cover it.

The reusable workflow raises wait budgets: default wait-seconds 900 → 1500 and job timeout-minutes 20 → 30, with rationale tied to slow/outlier Bugbot runs and gate timeouts mistaken for findings.

Fleet inventory updates: docs marked private; Bugbot issue refs normalized to .github#277; new org-config repo entry (private seed of .github operational logic, protection/callers aligned with peers).

Reviewed by Cursor Bugbot for commit 1316a49. Bugbot is set up for automated code reviews on this repo. Configure here.

LukasWodka and others added 3 commits September 10, 2026 12:57
…y bare #277 citations (#461)

* inventory: docs is private; declare org-config (seed pending); qualify the bare #277 citations

Three inventory-only changes, each of which the Monday audit would
otherwise report on its own:

- docs: `visibility: private`. The repo was made private on 2026-09-10;
  the audit compares the declared visibility with the org's and reads
  the mismatch as drift.
- org-config: a new entry. The repo exists since 2026-09-10 and an active
  repo absent from the inventory is a finding. Its callers, copy and
  quality files are `exempt` under one shared reason, `org_config_seed_pending`:
  its develop holds only a README until the seed PR merges, so nothing can
  be required of it yet. The PR that follows the seed flips them to
  `required`. Protection cells describe what was applied today: develop
  byte-for-byte the shape of this repo's develop (baseline + selftests),
  main as a working branch like rfcs and claude-skills.
- The three bare `#277` citations become `.github#277`. reason-citations
  resolves a bare `#N` against the repo the inventory lives in and refuses
  when GITHUB_REPOSITORY disagrees, so the file could not be evaluated
  from any other host. Same referent, now written down.

Verified locally: caller-drift audit against the live org exits 0 with
docs and org-config OK; reason-citations passes with GITHUB_REPOSITORY set
to either host; selftest-caller-drift 197/197; selftest-reason-citations
37/37; house-rules clean.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* inventory: say which branch the seed-pending reason is about

Bugbot: a repo off the train is audited on its DEFAULT branch, not on
`develop`, so a reason that names `develop` is only true while `develop`
IS the default. It is (set at creation), so the flip instruction holds;
the reason now says so and names the condition under which it stops
holding, instead of leaving the reader to infer it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
…t Bugbot's tail (backend#3530) (#460)

A head with more than one page on a connection was permanently refused:
`require_complete` compared the first page against totalCount, correctly,
on every re-run, and nothing the author could do changed it (client#1017
on the rollup; backend#3388 carries 70 review threads today). `fetch`
now follows `pageInfo.hasNextPage` on the two top-level connections with
follow-up queries built from the same field fragments, joins the pages,
and hands `require_complete` the whole list -- the truncation test stays
and measures the whole set. A cursor that repeats or outruns totalCount
is refused, never followed for ever. `connections_missing_pageinfo` is
the query self-check beside the totalCount one, refused in main().

`wait-seconds` 900 -> 1500 and the job ceiling 20 -> 30 minutes: Bugbot
took 15m22s on client-runtime#544 and the gate gave up at 15m08s; over
57 recent runs on five repos p99 is 9.9m and max 10.3m, and 5 of the 24
gate failures in that window were this timeout rather than a finding.

Selftest: two-page join, truncation after the join, a repeating cursor,
a Bugbot suite on the second page, the pageInfo stripper detected. Three
mutations registered.

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
…463)

The seed merged (tracebloc/org-config#1, 2026-09-10), so org-config's
default branch now carries every caller, the add-to-kanban copy,
CLAUDE.md and .cursor/BUGBOT.md. The `seed pending` exemptions said
they would become drift the moment that happened; this flips all
twelve cells to `required` and drops the now-unused shared reason.

Verified against the live org: caller-drift exits 0 with org-config OK
on every column; reason-citations clean under both hosts;
selftest-caller-drift 197/197; house-rules clean.

Co-authored-by: Claude <noreply@anthropic.com>
@LukasWodka

Copy link
Copy Markdown
Contributor Author

bugbot run

@tracebloc-release-train tracebloc-release-train Bot added the gate-nudge Toggled by the release train to (re-)fire the fr-gate label Sep 10, 2026

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 1316a49. Configure here.

Comment thread scripts/tests/bugbot-gate-selftest.py
@tracebloc-release-train tracebloc-release-train Bot removed the gate-nudge Toggled by the release train to (re-)fire the fr-gate label Sep 10, 2026
@LukasWodka

Copy link
Copy Markdown
Contributor Author

Promoted with 1 open Medium/Low Bugbot finding(s), per the severity policy in release-train's README (High stops the line; Medium/Low are recorded and ship, at both hops):

Selftest loop breaks after first connection, hiding broken stripper

This is a second look at once-reviewed code -- it passed per-feature review on the source branch, and has NOT had functional review yet (that happens on staging). Fix forward on develop if any is real.

What the train did with each:

@tracebloc-release-train
tracebloc-release-train Bot merged commit ea181d1 into staging Sep 10, 2026
74 of 78 checks passed
@tracebloc-release-train
tracebloc-release-train Bot deleted the release-train/to-staging branch September 10, 2026 15:22
@LukasWodka

Copy link
Copy Markdown
Contributor Author

Functional review — staging batches of 2026-09-10 (10:43 and 14:49 hops) — PASS (FR reviewer: LukasWodka; self-signoff per RFC-BACKEND-1405 D6)

Decision: PASS, by the reviewer's instruction at 15:45 UTC. All 132 cards that sat at FR on staging — the 86 held since the 10:43 hop plus the 46 this hop pushed — are advanced to Ready for prod.

What changed since the hold. The 12:00 hold was not about this batch: the edge refused every training envelope with NEVER_STEADY_STATE because chart 1.9.109 (kubelet reservation) plus the runtime's v3 envelope contract left a 2-core / 8 GiB node 100 m and 682 MiB short of the smallest envelope. The interim control-plane trim (tracebloc/client#1032, chart 1.9.112, render 650 m / 2272 MiB with the collector counted) merged at 14:47 UTC and reached staging with this hop at 15:37 UTC; rc v1.9.112-rc.1 is published.

Evidence at signoff.

  • Staging tips: every repo in the hop is on its 15:20–15:37 UTC mirror; the e2e-test-agent mirror was HELD by the train on a Bugbot High (teardown ceiling) and its harness stays at the 11:22 tip — that affects nothing under review here.
  • Artifact channels (fr-assist, 11:34 UTC run 34471966935): all OK, 1 SKIPPED (credentials).
  • Frontend smoke on staging (11:47 UTC, 34473123260): green.
  • Journey battery on the corrected chart is IN FLIGHT at signoff (dispatched 15:39 UTC on the staging harness): baseline https://github.com/tracebloc/e2e-test-agent/actions/runs/34497041338 and the 16-task fleet https://github.com/tracebloc/e2e-test-agent/actions/runs/34497050908. The verdict that matters is Train a model scheduling on the 2-core k3d node. It will be appended below when the runs finish; if it is red, the prod hop does not proceed and the cards come back.

Things this evidence cannot tell you (README → Functional review): interactive/TTY behaviour, per-ticket behaviour, anything at or after a failing step, and any step a run skipped.

@LukasWodka

Copy link
Copy Markdown
Contributor Author

Correction to the evidence paragraph above (15:50 UTC). The two journey runs named there (34497041338, 34497050908) were dispatched with chart_source/chart_ref blank, which installs the published chart from the Helm index — the index is stable-only, so they exercise chart 1.9.109, the customer path as it is on prod today, not the 1.9.112 candidate on staging. They are a control, not the proof. The proof runs are the candidate-chart baselines dispatched with chart_ref=v1.9.112-rc.1: https://github.com/tracebloc/e2e-test-agent/actions/runs/34498185079 (tabular_classification) and the image_classification twin dispatched a minute later. The fleet workflow has no chart_ref input, so the candidate is proven on the baseline cell only. Verdicts will be appended here; the prod hop waits for them.

@LukasWodka

Copy link
Copy Markdown
Contributor Author

Journey verdict appended (16:20 UTC) — the fix is confirmed on staging's candidate chart.

run chart installed node arithmetic (k3d-tracebloc-server-0) Train a model
34498185079 tabular_classification client-1.9.112-rc.1 (candidate) allocatable 1900m / 5538Mi, requested 750m / 2284Mi, leaves 1150m / 3254Mi → envelope fits ✅ experiment COMPLETED, leaderboard found
34498269145 image_classification client-1.9.112-rc.1 (candidate) same: leaves 1150m / 3254Mi → fits ✅ experiment COMPLETED, leaderboard found
34497041338 control, published chart client-1.9.109 (prod today) requested 1000m / 3148Mi, leaves 900m / 2390Mi → refused UnschedulableEnvelope at 15:54 UTC
34497050908 control fleet, published chart client-1.9.109 same refusal on 14 legs; 1 leg died earlier on a cosign download inside the installer ❌ 15/16 red at Train

Same node, same runtime image, same envelope; only the chart differs. The published chart (prod) still refuses; the candidate admits and trains to completion. Prod hop proceeds so 1.9.112 supersedes 1.9.109.

@tracebloc-release-train
tracebloc-release-train Bot restored the release-train/to-staging branch September 10, 2026 18:02
@tracebloc-release-train
tracebloc-release-train Bot deleted the release-train/to-staging branch September 10, 2026 18:02
@tracebloc-release-train
tracebloc-release-train Bot restored the release-train/to-staging branch September 10, 2026 19:09
@tracebloc-release-train
tracebloc-release-train Bot deleted the release-train/to-staging branch September 10, 2026 19:09
@tracebloc-release-train
tracebloc-release-train Bot restored the release-train/to-staging branch September 10, 2026 20:42
@tracebloc-release-train
tracebloc-release-train Bot deleted the release-train/to-staging branch September 10, 2026 20:42
LukasWodka added a commit that referenced this pull request Sep 11, 2026
…connection (#466)

The pageInfo stripper loop in the Bugbot gate selftest ended in a `break`,
so only the first member of PAGED_TOPLEVEL (checkSuites) was ever exercised.
Its reviewThreads branch pasted a fixed 6-space indentation into str.replace,
a needle that is a substring of the 14-space checkSuites line, so even when
reached it stripped checkSuites' pageInfo again and the detector answered
['checkSuites']. A pageInfo self-check blind to reviewThreads alone passed
the suite.

The loop now mirrors the totalCount stripper: literal member list beside the
module's dict (a removed member fails), a name-anchored indentation-agnostic
regex with its substitution count asserted, the detector required to name
exactly the stripped connection, and the visit list compared to the literals
so a reintroduced break reddens the suite. A registry mutation pins the
reviewThreads-only regression; it was UNCAUGHT against the old suite and is
caught now.

Tracked internally (Bugbot finding on the staging promotion #464).

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant