feat(sirius): add complete public SQL GPU acceptance campaign

[aunjgr]

Validate canonical public SQL and the complete local release gates for embedded
Sirius GPU execution.

What type of PR is this?

• Feature
• Test and CI

Which issue(s) this PR fixes:

Tracks #28968 and parent #28966; implements D from #29690 after merged C #29757.

What this PR does / why we need it:

The Go runner executes canonical Q1–Q22 over immutable SF1/SF10 inputs, fully
drains public results and compares exact typed schema, values, duplicates and
ordering against independently executed native MatrixOne. The controller owns
separate LOG/TN/CN processes, bounded private spools and joined failure cleanup.
All 744 timing executions are unprofiled. A separate 264-execution campaign
captures actual CUDA allocation/free events. Logical credits/pool tags, physical
allocations, residency, RSS and Go heap retain their source and scope.

Shared CTE occurrences use independent destructive MO readers on the same
statement snapshot. Merged Sirius #30
preserves retained VARCHAR identity casts and rejects reused/unreachable MO
bindings. D pins merged native 955b6881, preserves ABI v1 and uses the verified
release SDK. Optional unprovisioned Flight profiles are N/A.

Early consumer and provisioning failures cancel the unread query before driver
close and preserve error ownership. Deterministic fixtures prove one physical
connection, immutable SQL/config admission, complete-spool publication, late
errors, failed-query cleanup, eight source identities and typed loaded row
multisets. Frontend tests check actual query/account/physical-OID evidence and
heading privacy. No dependency or CI-policy changes are introduced.

Release validation

Preserved release evidence
contains full manifests, safe query records, raw timing repetitions, six allocation
profiles, lifecycle/sanitizer proofs, fresh numeric samples and artifact hashes.
These records are preserved under fork tag evidence/sirius-d-4230ce3d14 and
removed from the current source tree. The validation note documents exporting
them to a temporary workspace; the unchanged CLI reproduces
local_acceptance_complete: true.

• 744 unprofiled timing + 264 separate resource executions pass, covering
  exact Q1–Q22 SF1/SF10 results and streams 1/2/4. Embedded queries completed GPU
  work without fallback, with healthy terminals and zero retained input/result
  credit. All owned campaign processes joined before publication.
• All six allocation profiles pass with zero unexplained warmed growth and
  reconciled logical pool baselines. Actual ordered CUDA spans and successor
  windows support post-close proof; lifetime high-water marks keep their scope.
• Ten lifecycle cases pass, including bounded credit/admission, held-borrow
  cancellation, writer/numeric error ownership and healthy reuse. Fatal quarantine
  closure is scoped to joined test-owned process death.
• Sanitizer controls reconcile four raw exit statuses of 99. The GPU memory
  check passes 34,509 assertions/30 cases with one CUB registration diagnostic,
  matched by Native A. The ABI check passes 1,964 assertions/10 cases with one
  41,386,248-byte process-global cuDF default pinned allocation, matched by the
  no-GPU-work initialization control and all six stable allocation profiles.
  Raw counts/statuses are retained; no suppression or waiver is used.
• All ten numeric ratios pass 1.1, using fresh serial Native A/B/current
  executions, matched operands and 21 samples per case. Maximum ratio 1.026527;
  checked-add64 0.843248. Actual historical sources/binaries remain identified.
• Owning normal/race, actual all-22 native preparation, GPU data/cancellation,
  same-process Sirius/cuVS coexistence and 82 Python contract tests pass. The
  final main merge's affected type/function/planner/compiler/frontend suites pass.

Scale	Native MO median suite	Embedded GPU median suite
SF1	1.804130 s	4.789524 s
SF10	27.947777 s	52.826967 s

The GPU route was slower overall on the recorded RTX 3070. Timing evidence remains
separate from allocation profiling and retains all per-query and Q9 repetitions.

Both complete campaigns use one frozen release provenance record at MO
9814347c1c, Sirius 955b6881, SDK 29bca11a, with actual rebuilt combined service
and runner hashes. Native proof reports preserve their original execution
contexts and explicit source qualifications. Historical development and the
failed first release attempt remain separate. The quota monitor's retirement
race was fixed by one stat snapshot, tolerating only removed files; permission
and quota failures remain fatal. Full release schedules restarted from the
beginning. No partial capture contributes to acceptance.

Delivery head 688bb85222 normally merges authoritative main 02a2823c20.
Cleanup commit 70ec74b7d1 removes all 54 generated evidence files (about 123,000
lines) from the current PR tree. Code, benchmark definitions and regression
fixtures are unchanged by that cleanup. Earlier Git history and capture
provenance are preserved, with the fork tag keeping the snapshot fetchable.
The affected frontend/compiler/planner/vector/comparison suites pass after the
main merge, including applicable race checks; full pre-push SCA passes. Original MO 9814347c1c
capture identities, binaries and measurements remain preserved. Subsequent main
changes address COM_SET_OPTION, prepared/ENUM/window consumers and vector/branch
index filtering; the canonical immutable TAE campaign routes remain qualified.

Required CI passed at eba41acd67 in replacement run
38163597900.
The two substantive P2 review findings are corrected in 78d87489da:

• Final summaries reconnect mandatory native/GPU and declared optional Flight
  oracle cells, checking retained cardinality, physical descriptors and schema
  identity. The CLI verifies the archive checksums and parses the same bytes it
  hashed. Full-row live comparison remains the value oracle; unordered/peer
  spool hashes may differ.
• The existing controller monitors the whole workspace during readiness,
  worker execution, profiler finalization, owned export and shutdown. Quota or
  deadline failure kills and joins the owner; outer adopted-child cleanup still
  gates publication. Initial PID-generation publication failure also retains
  spawn cleanup. Finalization/export retain their three-hour deadlines.

Review-fix evidence
records six previously accepted parity mutations and two quota-owner red proofs.
All 82 owning Linux Python tests pass. The actual untouched archive passes
with identical acceptance output; native Q1 count 4→5 and OID 61→62 mutations
fail, even after rehashing. Real 1 MiB writers crossing a 512 KiB quota during
finalization/export are killed and joined before parsing/publication. A 1 KiB
export reaches parsing after join. These tests use small process barriers and
reused fixtures, without new GPU measurements or a framework.

Old captures preserve their actual collector source and do not retrospectively
certify the new continuous disk-quota monitor. Fresh CI for this fix head and
reviewer approval still gate merge; no performance or policy waiver is used.

Remaining readiness

The complete implementation and local evidence are ready for review.
Reviewer approval follows that handoff; it is required before final acceptance
and merge. The earlier draft wording incorrectly combined those two stages and
has been corrected. acceptance_complete remains false in the immutable capture
report; CI and reviewer decisions are external and are not inferred by the local
validator. Both P2 threads are answered and resolved; XuPeng-SH is requested to review the corrections.

After current required CI, reviewer approval and the user merge, E implements
the approved embedded-only design. E is stopped until D merges. The owner
confirmed that no customers use the sidecar; F and its migration/compatibility
requirements are dropped, and the abandoned sidecar repository stays untouched.
Docker remains separate; both tracking issues remain open.

The frozen release tar export has SHA-256
28cfb19a161bc89aa917bf9add42e496b425a3b71f63399670d6daa43cfb269e.
After removing checked-in captures, all 82 Python tests pass and external-archive
acceptance is identical. The source tree contains one concise validation note
with export/check commands; raw generated reports remain outside it.

The coverage job in run 38200331638
failed two inherited UT fixtures, both reproduced on clean authoritative main.
Commit 84955f2e93 corrects their inputs/synchronization without changing
production execution:

• TERM acknowledgment now occurs after the handler returns, preserving one
  outstanding signal, all 200 deliveries, and main/subshell PID checks. The
  sender is joined before its exit trap is disarmed. All 100 coverage-mode and
  100 race-mode focused executions pass; the full optools coverage package passes.
• Partition Top-N rejects tuple/interval order keys; ANY remains a positive
  control for the domainless NULL comparator added on main. The full partition
  and comparator coverage packages pass, including pool cleanup assertions.

Fresh authoritative main also merged TRUNCATE lifecycle regression coverage
(#29827). Commit 435b1a81ce updates that fixture to provide the transaction
snapshot now inspected by admission. The retained case supplies its held lock;
fresh owners supply an empty snapshot. All wait/cancel/FastFail and zero wait
bookkeeping assertions remain. The failure reproduced on exact clean main;
the corrected full compiler coverage package, 100 focused race executions, and
targeted static checks pass.

No sleeps, retries, skips, longer deadlines, weakened assertions, or GPU changes
are introduced. Targeted static checks pass. Full pre-push SCA passes; fresh required CI and reviewer approval still gate
merge. Main #29919 is normally merged: full compiler/planner/Substrait coverage,
all 22 exact-profile embedded candidates, and the authenticated multi-CN
privilege-cache test pass. Mock optimizer plan serializations vary between runs,
so no whole-inventory byte-equivalence or new GPU/performance claim is made.
The retained D captures preserve their original MO 9814347c1c execution
identity, measurements, and existing source qualifications.

[XuPeng-SH]

mo-bot review · mo-bot-local · done

Commit: 688bb85222b892fca964fb8becdb610131aaaf34

Assessment: approve

Reviewed the complete 37-file delta against the approved D campaign design. Both previous P2 findings are resolved in the current implementation with matching regression evidence. No actionable unresolved defect remains within this scope.

Model: gpt-6-astra / xhigh

Session: 01a12d8f-7dea-77c2-a565-38d45b5564e4 · slot 0 · generation 0

Validation:

• Verified the assigned HEAD and clean, unchanged source checkout. Separately assessed implementation/tooling, tests and documentation.

• Traced public SQL execution, cancellation before unread-result close, exact row comparison, schema publication, session reset, independent shared-CTE readers, process cleanup and allocation accounting through their existing owners.

• All five source/test hashes in preserved review-fixes.json match the current checkout. Reused its 82 passing Linux Python tests, archive cardinality/schema counterexamples, and live finalization/export quota probes with kill-and-join assertions.

• Verified that the public runner, schema helper, CTE expansion, bridge and Sirius pin are unchanged from the recorded release capture. Reused qualified evidence for 744 timing executions, 264 resource executions, six allocation profiles, ten lifecycle cases, matched sanitizer controls and ten numeric regression gates.

• Frozen evidence for the assigned HEAD reports successful required CI, Ubuntu unit tests, arm64 static analysis, coverage, PROXY/PESSIMISTIC multi-CN tests and Connector/J reset tests. Skipped checks were not counted as passes.

• Recorded evidence attribution and source checks in /home/xupeng/.local/state/mo-bot/jobs/33485d3f4eb50886f3039c3f/attempt-1/review-evidence-audit.json.

Limitations:

• No native/GPU/sanitizer or test-suite reruns were performed. Measurements retain their actual capture revision, 9814347, and the documented source qualifications; they are not measurements of the current HEAD.

• The native submodule is uninitialized locally. Assessment of the unchanged Sirius pin reused the substantive frozen review and release evidence.

• The recorded GPU route was slower overall on the RTX 3070. This approval establishes neither a speedup nor absence of ordinary-SQL performance regression.

• Flight remains unprovisioned. Default cutover, retirement and Docker work are outside D. Full linked-issue bodies were not supplied; scope was checked against the frozen PR and approved design, without claiming closure of the parent issues.

[XuPeng-SH]

mo-bot review · mo-bot-local · review

Commit: 688bb85222b892fca964fb8becdb610131aaaf34

Assessment: approve

Reviewed the complete 37-file delta against the approved D campaign design. Both previous P2 findings are resolved in the current implementation with matching regression evidence. No actionable unresolved defect remains within this scope.

Model: gpt-6-astra / xhigh

Session: 01a12d8f-7dea-77c2-a565-38d45b5564e4 · slot 0 · generation 0

Validation:

• Verified the assigned HEAD and clean, unchanged source checkout. Separately assessed implementation/tooling, tests and documentation.

• Traced public SQL execution, cancellation before unread-result close, exact row comparison, schema publication, session reset, independent shared-CTE readers, process cleanup and allocation accounting through their existing owners.

• All five source/test hashes in preserved review-fixes.json match the current checkout. Reused its 82 passing Linux Python tests, archive cardinality/schema counterexamples, and live finalization/export quota probes with kill-and-join assertions.

• Verified that the public runner, schema helper, CTE expansion, bridge and Sirius pin are unchanged from the recorded release capture. Reused qualified evidence for 744 timing executions, 264 resource executions, six allocation profiles, ten lifecycle cases, matched sanitizer controls and ten numeric regression gates.

• Frozen evidence for the assigned HEAD reports successful required CI, Ubuntu unit tests, arm64 static analysis, coverage, PROXY/PESSIMISTIC multi-CN tests and Connector/J reset tests. Skipped checks were not counted as passes.

• Recorded evidence attribution and source checks in /home/xupeng/.local/state/mo-bot/jobs/33485d3f4eb50886f3039c3f/attempt-1/review-evidence-audit.json.

Limitations:

• No native/GPU/sanitizer or test-suite reruns were performed. Measurements retain their actual capture revision, 9814347, and the documented source qualifications; they are not measurements of the current HEAD.

• The native submodule is uninitialized locally. Assessment of the unchanged Sirius pin reused the substantive frozen review and release evidence.

• The recorded GPU route was slower overall on the RTX 3070. This approval establishes neither a speedup nor absence of ordinary-SQL performance regression.

• Flight remains unprovisioned. Default cutover, retirement and Docker work are outside D. Full linked-issue bodies were not supplied; scope was checked against the frozen PR and approved design, without claiming closure of the parent issues.

[mergify]

Merge Queue Status

• ✅ Entered queue — 2026-10-12 06:49 UTC · Rule: main · triggered by rule Automatic queue on approval for main
• ✅ Checks skipped · PR is already up-to-date
• ✅ Merged — 2026-10-12 06:49 UTC · at 7af631bdf79e7beb4bb3588e2bfd03ba4852b6cc · squash

This pull request spent 12 seconds in the queue, including 1 second running CI.

Required conditions to merge

• #review-threads-unresolved = 0 [🛡 GitHub branch protection]
  • feat(sirius): add complete public SQL GPU acceptance campaign #29862
• github-review-approved [🛡 GitHub branch protection]
  • feat(sirius): add complete public SQL GPU acceptance campaign #29862
• any of [🛡 GitHub branch protection]:
  • check-success = CI Required
  • check-neutral = CI Required
  • check-skipped = CI Required