Skip to content

feat(numeric): complete embedded MO aggregates and exact capability - #27

Merged
aunjgr merged 1 commit into
matrixorigin:upstream-dev-mergefrom
aunjgr:feature/28968-mo-decimal-complete
Oct 8, 2026
Merged

aunjgr merged 1 commit into
matrixorigin:upstream-dev-mergefrom
aunjgr:feature/28968-mo-decimal-complete

Conversation

@aunjgr

@aunjgr aunjgr commented Oct 8, 2026

Copy link
Copy Markdown
Collaborator

Description

Native B completes embedded MO exact-decimal execution after #26. SUM/AVG/MIN/MAX now retain MO's declared types through import, binding, local/merge aggregation and final publication. Equality partitioning and probing share a scale-independent exact key; grouping, signed ordering and outer-join NULLs retain Decimal256 values. The production C ABI appends numeric statuses 12/13 and enables SIRIUS_CAP_MO_EXACT_DECIMAL_V1 (mask 16u, total mask 31u) while preserving ABI-v1 struct layouts.

  • Use column-backed signed 128/256-bit partial states with counts, existing reservation/spill ownership and final precision/NULL checks even for one partial. AVG rounds once at finalization; exact and ordinary slots share group ordinals.
  • Validate the reachable typed plan before input work, including read filters/projections and conditional producers. Preserve immutable bindings and decline unsupported coercions/modifiers. The exact path retains MO's logical optimization and avoids DuckDB coefficient-carrier folding.
  • Retain host/device scratch through stream completion or fatal quarantine. Verify Decimal256 child validity before publishing results, using admitted codec scratch. Preserve the numeric error owner through producer cancellation during cleanup.
  • Compute bounded segmented tile prefixes directly, avoiding new CUB scan registrations and a host synchronization. Domain proofs remove only infallible initial error polling; final publication remains checked.

The approved Native B boundary keeps these consumers together because import descriptors, raw state schemas, partition/probe keys and publication must agree before the complete capability can be advertised. MO lowering, public acceptance, default cutover and service retirement remain separate C–F increments. C starts after this PR merges and is pinned.

The contract, ownership graph, supported-shape limits and validation map give the review order. Semantic authority is the approved MO #29449 blob 42a89f09a1d168d02b9583cb3ea7b4de6dbb5634; #29690 records the delivery split.

Validation

Frozen mo Pixi release build, CUDA 13.3.73 / GCC 14.4 / cuDF-RMM 26.08, RTX 3070 driver 615.71.09. Production tests use bounded 1 GiB GPU/host/disk configuration.

Selection Terminal result
CPU arithmetic and aggregate boundaries 52,645 assertions / 8 cases passed
Binding/preparation and ordinary controls 831 / 80 passed
Native ownership/cancellation/error controls 606 / 78 passed
Full affected GPU/helper/admission/result group 32,349 / 30 passed
Production numeric C ABI, under memcheck 1,468 / 8 passed; zero sanitizer errors
Numeric and result-codec group, under memcheck 21,294 / 26 passed; zero sanitizer errors
SDK exporter and independent C consumer 8 exporter tests plus real GPU result/release/shutdown/reuse passed
SDK provenance and pinned hooks Clean source revision; ABI/header/literal/consumer fingerprints verified; all changed-file hooks passed

All raw benchmark runs are retained: three complete invocations, each with a three-second global warmup, two per-case warmups and seven measured repetitions at 262,144 rows. Pooled exact/control ratios are 0.7931 for safe SUM64→128 and 0.8165 for evaluator add64. The unchanged checked-add64 primitive's historical comparison is +18.15%, with run medians 0.027530–0.033940 ms; fresh matched confirmation remains required in D under the 10% policy. No rollout exception is claimed.

Full leak checking of the existing nonnumeric control selection is not clean: it reports a 41,386,248-byte cuDF default pinned pool and a generic CUB EmptyKernel registration diagnostic in the unchanged converter path. That selection's 11,034 assertions / 3 cases pass. The new aggregate CUDA object has no CUB scan/empty-kernel symbols; the two new scan diagnostics were eliminated. D still owns full process-baseline, lifecycle/resource and real public SQL acceptance.

Checklist

  • Read CONTRIBUTING.md and satisfy the reviewability checklist; coherent Native B scope is justified above.
  • Cover changes with CPU, GPU, production C ABI, allocation-failure, spill/restore and negative preparation tests.
  • Document configuration impact: this PR adds no configuration options.
  • Update repository design/validation documentation and preserve Native A's historical record.

References

Refs matrixorigin/matrixone#28968, matrixorigin/matrixone#28966, matrixorigin/matrixone#29449, matrixorigin/matrixone#29690.
Follows #25, #26 and merged matrixorigin/duckdb-substrait#3 / matrixorigin/duckdb-substrait#4. The importer pin remains 95d9ce8d78490db3991ab6145653716aa3ec42c9.

@aunjgr
aunjgr marked this pull request as ready for review October 8, 2026 10:22
@aunjgr
aunjgr merged commit 5ea60cd into matrixorigin:upstream-dev-merge Oct 8, 2026
6 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant