fix(embedding): preserve exact decimals in NULL predicates and true joins

docs/mo-exact-null-predicates.md

@@ -0,0 +1,62 @@
+# Exact-decimal NULL predicates and condition-free joins
+
+The MatrixOne public numeric fixture found that IS NOT NULL enters cuDF's
+AST unary path with an exact decimal carrier. Decimal256's private four-limb
+STRUCT is not a supported AST unary operand. Ordinary IS NULL can reach the
+same path inside a larger boolean expression.
+
+The invariant is that SQL nullness depends only on the canonical top-level
+validity mask, for every physical decimal width. Coefficient values and private
+limb masks do not define another SQL value. The negation is an unsupported AST
+operation, a NULL boolean result, or a predicate that inspects carrier limbs.
+
+The expression evaluator owns this boundary. Materialize exact decimal NULL
+predicates with cuDF is_null/is_valid on the caller's stream and resource. When
+an enclosing expression requests AST mode, retain the resulting boolean column
+through the existing temporary-column owner and reference it from the parent
+AST. The standalone AST translator declines the unsupported private carrier.
+No numeric URI, function identity, capability or C ABI layout changes.
+
+The change adds no wait, registry, worker, copy of coefficients or independent
+allocation owner. The child and result retain their existing task lifetime;
+result allocation uses the task resource. Errors and cancellation unwind through
+the existing evaluator/task cleanup. The boolean result is one byte per row,
+bounded by the admitted input batch. IS NOT NULL uses is_valid directly.
+
+The focused GPU regression covers Decimal64/128/256, all-valid and mixed-NULL
+columns, a nonzero-offset all-NULL slice, empty input and typed NULL literals.
+It checks nonnullable BOOL8 values independently under materialize, interpreted
+AST and JIT AST strategies, both directly and inside NOT. Standalone translation
+must decline the private carriers. Existing ordinary unary tests remain controls.
+
+MO C keeps the original public CASE/COALESCE plus NULL-predicate query and adds
+an all-width NULL-predicate control. C must pin the merged native fix before
+delivery. Native and small public tests do not establish D's full-data or
+performance acceptance.
+
+Exact preparation also skips DuckDB's ordinary optimizer, leaving JOIN ON true
+as an unsupported ANY_JOIN. Preserve that join's kind, output projection maps
+and mark binding while lowering the literal TRUE condition to equal constant
+TINYINT keys in the existing GPU join. The join planner already materializes
+computed keys and excludes those private columns from public output. FALSE,
+NULL and arbitrary predicates do not enter this transformation. No scalar-row
+assumption or new cross-product operator is needed.
+
+The production C ABI regression covers INNER/LEFT/RIGHT/FULL joins with both
+sides present, either side empty and both empty. It independently checks
+multiplicity, duplicate right values, right NULLs and Decimal256 high limbs.
+FALSE and NULL controls retain rejection before input or GPU work starts.
+Existing join tasks, partition/concat owners and resource admission retain all
+buffers and errors. The additional key costs one byte per input row per side
+plus the existing materialization/hash scratch, charged to those owners. Full
+query performance remains D's gate.
+
+Local development validation on 2026-10-09 passes all 22 MO native preparations
+without starting readers, and the complete public MySQL numeric fixture,
+including exact values/metadata/errors, prepared division, NULL predicates and
+scalar aggregates on empty/all-NULL input. This used an explicitly marked
+development SDK; MO's delivered gitlink remains merged Sirius #28 until this
+fix merges. The standalone exact/ordinary GPU selection and native binding
+suite pass. Clean SDK fingerprints and final native test counts are recorded
+in the PR delivery evidence. None of these checks establishes SF1/SF10 parity,
+full leak acceptance or a passed performance rollout gate.

src/expression_evaluator/gpu_expression_translator.cpp

@@ -678,6 +678,13 @@ std::optional<expr_ref> gpu_expression_translator::add_expression(
 std::optional<expr_ref> gpu_expression_translator::add_expression(
   sirius::ast::unary_op const& alt, cudf::ast::table_reference const table_src)
 {
+  if ((alt.op == sirius::ast::unary_op::kind::op_is_null ||
+       alt.op == sirius::ast::unary_op::kind::op_is_not_null) &&
+      alt.child->return_type().is_mo_decimal()) {
+    // The evaluator materializes these from the canonical validity mask;
+    // a standalone cuDF AST cannot consume the private exact carriers.
+    return std::nullopt;
+  }
   auto child_expr = add_expression(*alt.child, table_src);
   if (!child_expr) { return std::nullopt; }
 

src/expression_evaluator/specializations/operator.cpp

@@ -254,6 +254,20 @@ evaluate_result expression_evaluator::evaluate(sirius::ast::unary_op const& alt,
       "[expression_evaluator] evaluate called on an unsupported TRY operator expression.");
   }
 
+  if ((alt.op == sirius::ast::unary_op::kind::op_is_null ||
+       alt.op == sirius::ast::unary_op::kind::op_is_not_null) &&
+      alt.child->return_type().is_mo_decimal()) {
+    // cuDF AST unary operators cannot consume MO's exact decimal carriers
+    // (Decimal256 is a private STRUCT). SQL nullness is the canonical
+    // top-level mask, independent of coefficient limbs and physical width.
+    auto child  = evaluate(*alt.child, evaluation_mode::MATERIALIZE);
+    auto result = alt.op == sirius::ast::unary_op::kind::op_is_null
+                    ? cudf::is_null(child.get_column_view(), _stream, _mr)
+                    : cudf::is_valid(child.get_column_view(), _stream, _mr);
+    if (mode == evaluation_mode::AST) return materialize_as_ast_column(std::move(result));
+    return evaluate_result(std::move(result));
+  }
+
   auto const ast_op_count = alt.cudf_ast_op_count();
 
   if (_strategy != expression_evaluator_strategy::MATERIALIZE &&

src/numeric/decimal_plan.cpp

@@ -13,6 +13,7 @@
 #include <duckdb/planner/expression/bound_cast_expression.hpp>
 #include <duckdb/planner/expression/bound_columnref_expression.hpp>
 #include <duckdb/planner/expression/bound_comparison_expression.hpp>
+#include <duckdb/planner/expression/bound_constant_expression.hpp>
 #include <duckdb/planner/expression/bound_function_expression.hpp>
 #include <duckdb/planner/expression/bound_operator_expression.hpp>
 #include <duckdb/planner/expression_iterator.hpp>
@@ -603,12 +604,34 @@ void rewrite_exact_comparisons(duckdb::unique_ptr<duckdb::LogicalOperator>& plan
     });
   if (plan->type == duckdb::LogicalOperatorType::LOGICAL_ANY_JOIN) {
     auto& join = plan->Cast<duckdb::LogicalAnyJoin>();
-    plan       = duckdb::LogicalComparisonJoin::CreateJoin(context,
-                                                     join.join_type,
-                                                     duckdb::JoinRefType::REGULAR,
-                                                     std::move(join.children[0]),
-                                                     std::move(join.children[1]),
-                                                     std::move(join.condition));
+    if (join.condition->GetExpressionClass() == duckdb::ExpressionClass::BOUND_CONSTANT &&
+        join.condition->return_type.id() == duckdb::LogicalTypeId::BOOLEAN &&
+        !join.condition->Cast<duckdb::BoundConstantExpression>().value.IsNull() &&
+        join.condition->Cast<duckdb::BoundConstantExpression>().value.GetValue<bool>()) {
+      // Exact plans skip DuckDB's ordinary optimizer. JOIN ON true otherwise
+      // remains an unsupported ANY_JOIN. Equal constant keys use the existing
+      // GPU join, retaining multiplicity and outer NULLs even for empty sides.
+      auto replacement = duckdb::make_uniq<duckdb::LogicalComparisonJoin>(join.join_type);
+      duckdb::JoinCondition condition;
+      condition.comparison = duckdb::ExpressionType::COMPARE_EQUAL;
+      condition.left =
+        duckdb::make_uniq<duckdb::BoundConstantExpression>(duckdb::Value::TINYINT(1));
+      condition.right =
+        duckdb::make_uniq<duckdb::BoundConstantExpression>(duckdb::Value::TINYINT(1));
+      replacement->conditions.push_back(std::move(condition));
+      replacement->children             = std::move(join.children);
+      replacement->left_projection_map  = std::move(join.left_projection_map);
+      replacement->right_projection_map = std::move(join.right_projection_map);
+      replacement->mark_index           = join.mark_index;
+      plan                              = std::move(replacement);
+    } else {
+      plan = duckdb::LogicalComparisonJoin::CreateJoin(context,
+                                                       join.join_type,
+                                                       duckdb::JoinRefType::REGULAR,
+                                                       std::move(join.children[0]),
+                                                       std::move(join.children[1]),
+                                                       std::move(join.condition));
+    }
   }
   if (plan->type == duckdb::LogicalOperatorType::LOGICAL_COMPARISON_JOIN) {
     auto& join = plan->Cast<duckdb::LogicalComparisonJoin>();

test/cpp/embedding/test_decimal_expression_gpu.cpp

@@ -2,11 +2,13 @@
 #include "decimal_import_fixture.hpp"
 #include "expression/ast/utils.hpp"
 #include "expression_evaluator/expression_evaluator.hpp"
+#include "expression_evaluator/gpu_expression_translator_internal.hpp"
 #include "numeric/decimal_error.hpp"
 #include "numeric/exact_decimal_gpu.hpp"
 #include "sirius_c.h"
 
 #include <cudf/column/column_factories.hpp>
+#include <cudf/copying.hpp>
 
 #include <rmm/cuda_stream.hpp>
 
@@ -124,6 +126,58 @@ std::unique_ptr<cudf::column> values(decimal_type type,
 }
 }  // namespace
 
+TEST_CASE("MO exact null predicates consume canonical validity in every evaluator strategy",
+          "[decimal_import_gpu]")
+{
+  rmm::cuda_stream stream;
+  auto mr = cudf::get_current_device_resource_ref();
+  for (uint16_t bits : {64, 128, 256}) {
+    auto type      = logical_type::make_mo_decimal({bits, 15, 2}, true);
+    auto all_valid = values(type.mo_decimal_type(), stream, mr);
+    auto nullable  = values(type.mo_decimal_type(), stream, mr, true);
+    auto empty     = make_decimal_literal(type.mo_decimal_type(), {}, true, 0, stream, mr);
+    auto slice     = cudf::slice(nullable->view(), {1, 2}, stream);
+    std::vector<cudf::column_view> inputs{
+      all_valid->view(), nullable->view(), slice[0], empty->view()};
+    std::vector<std::vector<int64_t>> nulls{{0, 0}, {0, 1}, {1}, {}};
+    for (auto strategy : {expression_evaluator_strategy::MATERIALIZE,
+                          expression_evaluator_strategy::AST_INTERPRET,
+                          expression_evaluator_strategy::AST_JIT}) {
+      for (bool is_null : {false, true}) {
+        for (bool parent_not : {false, true}) {
+          INFO("bits=" << bits << " strategy=" << int(strategy) << " is_null=" << is_null
+                       << " parent_not=" << parent_not);
+          auto op = is_null ? ast::unary_op::kind::op_is_null : ast::unary_op::kind::op_is_not_null;
+          auto expression = std::make_unique<ast::node>(ast::unary_op{op, reference(0, type)});
+          gpu_expression_translator translator(stream.view(), mr);
+          CHECK_FALSE(translator.translate_expression(*expression).has_value());
+          if (parent_not)
+            expression = std::make_unique<ast::node>(
+              ast::unary_op{ast::unary_op::kind::op_not, std::move(expression)});
+          expression_evaluator evaluator(*expression, mr, stream.view(), strategy);
+          for (std::size_t i = 0; i < inputs.size(); ++i) {
+            auto expected = nulls[i];
+            if (is_null == parent_not)
+              for (auto& value : expected)
+                value = !value;
+            auto result = evaluator.evaluate(cudf::table_view{{inputs[i]}});
+            CHECK(result->view().column(0).type().id() == cudf::type_id::BOOL8);
+            expect(result->view().column(0), 1, expected, stream.view());
+          }
+        }
+      }
+    }
+    for (bool is_null : {false, true}) {
+      auto op = is_null ? ast::unary_op::kind::op_is_null : ast::unary_op::kind::op_is_not_null;
+      ast::node expression(
+        ast::unary_op{op, std::make_unique<ast::node>(ast::constant{sirius::null_value{}, type})});
+      expression_evaluator evaluator(expression, mr, stream.view());
+      auto result = evaluator.evaluate(cudf::table_view{{all_valid->view()}});
+      expect(result->view().column(0), 1, {is_null, is_null}, stream.view());
+    }
+  }
+}
+
 TEST_CASE("MO imported scalar signatures execute on the GPU without coercion",
           "[decimal_import_gpu]")
 {

test/cpp/embedding/test_native_numeric.cpp

@@ -635,6 +635,106 @@ TEST_CASE("production exact join keys and outer NULLs preserve values across sca
     CHECK(missing == (outer ? 1 : 0));
   }
 }
+TEST_CASE("production exact true joins preserve multiplicity payloads and empty outer sides",
+          "[native_numeric]")
+{
+  engine engine;
+  decimal_type domain{256, 65, 0};
+  coefficient high;
+  high.words[4] = 1;  // 2^128: passthrough must retain the high limbs.
+  for (auto kind : {substrait::JoinRel::JOIN_TYPE_INNER,
+                    substrait::JoinRel::JOIN_TYPE_LEFT,
+                    substrait::JoinRel::JOIN_TYPE_RIGHT,
+                    substrait::JoinRel::JOIN_TYPE_OUTER}) {
+    for (bool left_empty : {false, true}) {
+      for (bool right_empty : {false, true}) {
+        INFO("kind=" << kind << " left_empty=" << left_empty << " right_empty=" << right_empty);
+        plan_case plan(domain, domain, "mo_decimal_equal", false, true);
+        auto project =
+          plan.plan.mutable_relations(0)->mutable_root()->mutable_input()->mutable_project();
+        project->clear_expressions();
+        *project->add_expressions() = field(1);
+        project->mutable_common()->mutable_emit()->set_output_mapping(0, 2);
+        auto read              = project->input();
+        auto join              = project->mutable_input()->mutable_join();
+        *join->mutable_left()  = read;
+        *join->mutable_right() = read;
+        join->mutable_right()->mutable_read()->mutable_named_table()->set_names(1, "2");
+        join->set_type(kind);
+        join->mutable_expression()->mutable_literal()->set_boolean(true);
+        query run(engine, plan);
+        run.register_extra(domain, true);
+        auto status = run.prepare();
+        INFO(run.prepare_error);
+        REQUIRE(status == SIRIUS_OK);
+        run.publish(
+          plan,
+          left_empty ? std::vector<coefficient>{} : std::vector<coefficient>{small(1), small(2)});
+        run.write(
+          run.extra_inputs[0],
+          domain,
+          right_empty ? std::vector<coefficient>{} : std::vector<coefficient>{high, high, small(0)},
+          !right_empty);
+        std::vector<bool> nulls;
+        auto values = run.results(domain.bytes(), &nulls);
+        bool preserve_left =
+          kind == substrait::JoinRel::JOIN_TYPE_LEFT || kind == substrait::JoinRel::JOIN_TYPE_OUTER;
+        bool preserve_right = kind == substrait::JoinRel::JOIN_TYPE_RIGHT ||
+                              kind == substrait::JoinRel::JOIN_TYPE_OUTER;
+        size_t rows  = left_empty || right_empty ? 0 : 6;
+        size_t valid = left_empty || right_empty ? 0 : 4;
+        if (right_empty && !left_empty && preserve_left) rows = 2;
+        if (left_empty && !right_empty && preserve_right) {
+          rows  = 3;
+          valid = 2;
+        }
+        REQUIRE(values.size() == rows);
+        size_t observed_valid = 0;
+        for (size_t i = 0; i < values.size(); ++i) {
+          if (nulls[i]) continue;
+          ++observed_valid;
+          for (int word = 0; word < 8; ++word)
+            CHECK(values[i].words[word] == high.words[word]);
+        }
+        CHECK(observed_valid == valid);
+      }
+    }
+  }
+}
+TEST_CASE("production exact true-join lowering does not admit FALSE or NULL predicates",
+          "[native_numeric]")
+{
+  engine engine;
+  decimal_type domain{256, 65, 0};
+  for (bool null_predicate : {false, true}) {
+    plan_case plan(domain, domain, "mo_decimal_equal", false, true);
+    auto project =
+      plan.plan.mutable_relations(0)->mutable_root()->mutable_input()->mutable_project();
+    project->clear_expressions();
+    *project->add_expressions() = field(1);
+    project->mutable_common()->mutable_emit()->set_output_mapping(0, 2);
+    auto read              = project->input();
+    auto join              = project->mutable_input()->mutable_join();
+    *join->mutable_left()  = read;
+    *join->mutable_right() = read;
+    join->mutable_right()->mutable_read()->mutable_named_table()->set_names(1, "2");
+    join->set_type(substrait::JoinRel::JOIN_TYPE_LEFT);
+    auto predicate = join->mutable_expression()->mutable_literal();
+    if (null_predicate)
+      predicate->mutable_null()->mutable_bool_()->set_nullability(
+        substrait::Type::NULLABILITY_NULLABLE);
+    else
+      predicate->set_boolean(false);
+    query run(engine, plan);
+    run.register_extra(domain, true);
+    REQUIRE(run.prepare() != SIRIUS_OK);
+    sirius_query_execution_stats stats{sizeof(stats), SIRIUS_ABI_VERSION};
+    sirius_error error{};
+    ok(sirius_query_get_execution_stats(run.handle, &stats, &error), error);
+    CHECK(stats.gpu_tasks_started == 0);
+    CHECK(stats.mo_input_units == 0);
+  }
+}
 TEST_CASE("production Decimal256 ordering retains signed coefficients and NULL position",
           "[native_numeric]")
 {