release-train: develop -> staging

[LukasWodka]

Automated promotion by the release train (RFC-0008 D14). Head is the train-managed release-train/to-staging branch (a mirror of develop), so it never collides with a human PR. Merged only when the fr-gate is green.

---

Note

Medium Risk
Touches registry selection, image-refresh reconcile guards, and default resource requests that affect scheduling and training fit; changes are heavily tested but alter production pull/update behavior on edges using ghcr.io or custom registries.

Overview
Chart 1.9.112 adds images.traceblocRegistry so tracebloc control-plane images (jobs-manager, pods-monitor, resource-monitor, requests-proxy) can be pulled from docker.io or ghcr.io without re-homing every image via global.imageRegistry. A shared tracebloc.tbRegistry helper drives pod image refs, post-install NOTES, and the image-refresh CronJob. Refresh now resolves digests on that registry (not a hard-coded docker.io), gates on chart-rendered IMAGE_REGISTRY_RESOLVABLE (docker.io/ghcr.io only), and keeps imagePullPolicy aligned via tracebloc.imageRefreshResolvable. mysql-client and third-party images are unchanged.

An interim control-plane resource trim (backend#2461) lowers default requests (and some limits) so the rendered footprint fits under the installer’s 3 GiB reserve—fixing training pods stuck Pending on small nodes. Embedded installer footprint constants, envelope/schedulability guards, and Helm unit tests are updated accordingly; jobs-manager/pods-monitor now use lower memory requests while keeping prior limits for OOM headroom.

CI: the unit-test job drops pre-baked Google Chrome apt sources before apt-get update to avoid unrelated mirror hash failures. INSTALL.md documents traceblocRegistry and GHCR egress.

^{Reviewed by Cursor Bugbot for commit b3c66bd. Bugbot is set up for automated code reviews on this repo. Configure here.}

[LukasWodka]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit b3c66bd. Configure here.}

[LukasWodka]

Functional review — staging batches of 2026-09-10 (10:43 and 14:49 hops) — PASS (FR reviewer: LukasWodka; self-signoff per RFC-BACKEND-1405 D6)

Decision: PASS, by the reviewer's instruction at 15:45 UTC. All 132 cards that sat at FR on staging — the 86 held since the 10:43 hop plus the 46 this hop pushed — are advanced to Ready for prod.

What changed since the hold. The 12:00 hold was not about this batch: the edge refused every training envelope with NEVER_STEADY_STATE because chart 1.9.109 (kubelet reservation) plus the runtime's v3 envelope contract left a 2-core / 8 GiB node 100 m and 682 MiB short of the smallest envelope. The interim control-plane trim (#1032, chart 1.9.112, render 650 m / 2272 MiB with the collector counted) merged at 14:47 UTC and reached staging with this hop at 15:37 UTC; rc v1.9.112-rc.1 is published.

Evidence at signoff.

• Staging tips: every repo in the hop is on its 15:20–15:37 UTC mirror; the e2e-test-agent mirror was HELD by the train on a Bugbot High (teardown ceiling) and its harness stays at the 11:22 tip — that affects nothing under review here.
• Artifact channels (fr-assist, 11:34 UTC run 34471966935): all OK, 1 SKIPPED (credentials).
• Frontend smoke on staging (11:47 UTC, 34473123260): green.
• Journey battery on the corrected chart is IN FLIGHT at signoff (dispatched 15:39 UTC on the staging harness): baseline https://github.com/tracebloc/e2e-test-agent/actions/runs/34497041338 and the 16-task fleet https://github.com/tracebloc/e2e-test-agent/actions/runs/34497050908. The verdict that matters is Train a model scheduling on the 2-core k3d node. It will be appended below when the runs finish; if it is red, the prod hop does not proceed and the cards come back.

Things this evidence cannot tell you (README → Functional review): interactive/TTY behaviour, per-ticket behaviour, anything at or after a failing step, and any step a run skipped.

[LukasWodka]

Correction to the evidence paragraph above (15:50 UTC). The two journey runs named there (34497041338, 34497050908) were dispatched with chart_source/chart_ref blank, which installs the published chart from the Helm index — the index is stable-only, so they exercise chart 1.9.109, the customer path as it is on prod today, not the 1.9.112 candidate on staging. They are a control, not the proof. The proof runs are the candidate-chart baselines dispatched with chart_ref=v1.9.112-rc.1: https://github.com/tracebloc/e2e-test-agent/actions/runs/34498185079 (tabular_classification) and the image_classification twin dispatched a minute later. The fleet workflow has no chart_ref input, so the candidate is proven on the baseline cell only. Verdicts will be appended here; the prod hop waits for them.

[LukasWodka]

Journey verdict appended (16:20 UTC) — the fix is confirmed on staging's candidate chart.

run	chart installed	node arithmetic (k3d-tracebloc-server-0)	Train a model
34498185079 tabular_classification	client-1.9.112-rc.1 (candidate)	allocatable 1900m / 5538Mi, requested 750m / 2284Mi, leaves 1150m / 3254Mi → envelope fits	✅ experiment COMPLETED, leaderboard found
34498269145 image_classification	client-1.9.112-rc.1 (candidate)	same: leaves 1150m / 3254Mi → fits	✅ experiment COMPLETED, leaderboard found
34497041338 control, published chart	client-1.9.109 (prod today)	requested 1000m / 3148Mi, leaves 900m / 2390Mi → refused	❌ UnschedulableEnvelope at 15:54 UTC
34497050908 control fleet, published chart	client-1.9.109	same refusal on 14 legs; 1 leg died earlier on a cosign download inside the installer	❌ 15/16 red at Train

Same node, same runtime image, same envelope; only the chart differs. The published chart (prod) still refuses; the candidate admits and trains to completion. Prod hop proceeds so 1.9.112 supersedes 1.9.109.