[fix](expr opt) Preserve cast boundaries during arithmetic simplification by morrySnow · Pull Request #67733 · apache/doris

morrySnow · 2026-09-09T10:19:04Z

Problem

Arithmetic simplification can move a CAST or TRY_CAST from around an arithmetic expression to each operand. That changes where type conversion happens. With large exact integers converted to DOUBLE, the rewritten expression loses precision before subtraction and returns the wrong value.

For example, subtracting 9223372036854775800 from 9223372036854775807 in BIGINT produces the exact value 7. Converting that result to DOUBLE must still produce 7, but converting both operands first rounds them to the same floating-point value and produces 0.

Root cause

The arithmetic flattener recognized a cast containing addition, subtraction, multiplication, or division and recursively propagated the cast target type to every operand. This rewrite assumes that conversion distributes over arithmetic, which is not generally true. Besides floating-point rounding, the transformation can alter overflow, conversion-error, and TRY_CAST null behavior.

Reproduction

CREATE TABLE t (
    id INT NOT NULL,
    k BIGINT NOT NULL
)
DUPLICATE KEY(id)
DISTRIBUTED BY HASH(id) BUCKETS 1
PROPERTIES ("replication_num" = "1");

INSERT INTO t VALUES
    (1, 9223372036854775807),
    (2, 9223372036854775806);

SELECT id,
       k - 9223372036854775800 AS exact_delta,
       TRY_CAST(k - 9223372036854775800 AS DOUBLE) + CAST(0 AS DOUBLE) AS try_expr,
       CAST(k - 9223372036854775800 AS DOUBLE) + CAST(0 AS DOUBLE) AS cast_expr
FROM t
ORDER BY id;

Before this change, exact_delta is 7/6, while both converted expressions incorrectly return 0/0.

Fix

Treat explicit cast expressions as semantic boundaries and atomic operands during arithmetic flattening. The simplifier can continue optimizing arithmetic below and above a cast, but it no longer distributes the conversion into the cast's operands.

This deliberately chooses correctness over the optimization enabled by cross-cast flattening; no target-type whitelist is used because conversion distribution is not safe across all values and operation types.

Tests

Added expression-rule coverage for both CAST and TRY_CAST around a large-integer subtraction.
Ran SimplifyArithmeticRuleTest: 5 tests passed.
Deployed the FE to a local sandbox. The reproduction now returns 7/6 for both converted expressions, and the physical expression keeps the BIGINT subtraction inside the cast.

Arithmetic flattening distributed a cast over its child operands when a constant was present. This can change the evaluation type and move rounding, overflow, error, or null behavior before the original operation. Treat explicit cast expressions as atomic operands during flattening. Arithmetic inside and outside each cast can still be simplified without crossing the conversion boundary. Issue Number: None Tests: - SimplifyArithmeticRuleTest - sandbox SQL reproduction near the BIGINT upper bound

hello-stephen · 2026-09-09T10:19:10Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

morrySnow · 2026-09-09T10:19:12Z

run buildall

hello-stephen · 2026-09-09T14:37:41Z

FE UT Coverage Report

Increment line coverage 100.00% (6/6) 🎉
Increment coverage report
Complete coverage report

hello-stephen · 2026-09-09T17:15:54Z

TPC-H: Total hot run time: 16926 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 167e4e9d4e418e703e43a3697f01ae960baf6e1a, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17556	3020	3015	3015
q2	2123	264	242	242
q3	10205	857	515	515
q4	4672	246	200	200
q5	7675	555	382	382
q6	137	110	93	93
q7	528	514	385	385
q8	9243	857	952	857
q9	3573	2436	2399	2399
q10	6504	873	711	711
q11	398	192	176	176
q12	617	262	201	201
q13	18117	1540	1205	1205
q14	159	155	142	142
q15	q16	435	389	364	364
q17	1369	868	869	868
q18	3057	2270	2238	2238
q19	1280	901	789	789
q20	386	287	205	205
q21	5566	1710	1834	1710
q22	329	268	229	229
Total cold run time: 93929 ms
Total hot run time: 16926 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	3327	3296	3274	3274
q2	503	393	403	393
q3	2201	2291	2218	2218
q4	1203	1181	888	888
q5	2179	2124	2092	2092
q6	161	122	87	87
q7	1031	917	849	849
q8	1573	1367	1378	1367
q9	3126	3080	3059	3059
q10	1881	1787	1633	1633
q11	347	262	252	252
q12	452	430	346	346
q13	1494	1526	1154	1154
q14	178	165	167	165
q15	q16	393	393	357	357
q17	3556	3286	3191	3191
q18	4805	4462	4772	4462
q19	855	791	935	791
q20	1021	973	818	818
q21	3854	3091	3266	3091
q22	398	338	325	325
Total cold run time: 34538 ms
Total hot run time: 30812 ms

hello-stephen · 2026-09-09T17:22:24Z

TPC-DS: Total hot run time: 82068 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 167e4e9d4e418e703e43a3697f01ae960baf6e1a, data reload: false

query5	4257	422	351	351
query6	388	139	126	126
query7	4931	433	237	237
query8	287	126	120	120
query9	8690	2875	2891	2875
query10	426	222	180	180
query11	5367	1063	926	926
query12	120	72	69	69
query13	1211	437	314	314
query14	6110	2212	2100	2100
query14_1	1983	1973	1983	1973
query15	180	119	113	113
query16	906	371	353	353
query17	792	466	370	370
query18	2330	337	229	229
query19	166	140	111	111
query20	83	77	70	70
query21	200	100	87	87
query22	5305	5378	5467	5378
query23	6779	6249	6198	6198
query23_1	6006	6049	6075	6049
query24	7256	1114	755	755
query24_1	777	778	768	768
query25	428	307	254	254
query26	1248	244	135	135
query27	2755	404	251	251
query28	4687	1518	1504	1504
query29	930	451	346	346
query30	256	156	131	131
query31	827	401	323	323
query32	135	82	76	76
query33	455	229	177	177
query34	1006	821	478	478
query35	405	416	357	357
query36	573	561	543	543
query37	123	81	71	71
query38	1014	870	827	827
query39	494	470	484	470
query39_1	461	460	469	460
query40	208	92	79	79
query41	60	57	57	57
query42	78	73	77	73
query43	243	246	209	209
query44	986	546	556	546
query45	119	104	104	104
query46	766	864	553	553
query47	752	759	696	696
query48	317	340	237	237
query49	555	239	189	189
query50	767	270	193	193
query51	8059	8053	7906	7906
query52	71	65	61	61
query53	204	197	146	146
query54	224	162	169	162
query55	67	61	58	58
query56	180	176	175	175
query57	778	682	656	656
query58	181	168	161	161
query59	1225	1218	1087	1087
query60	232	186	176	176
query61	109	111	110	110
query62	357	222	177	177
query63	174	149	141	141
query64	2703	703	623	623
query65	1622	1574	1582	1574
query66	1882	282	202	202
query67	9680	9830	9619	9619
query68	3034	1127	703	703
query69	350	229	201	201
query70	669	626	619	619
query71	252	178	160	160
query72	2349	1668	1488	1488
query73	656	645	324	324
query74	2011	1217	1141	1141
query75	1190	1098	964	964
query76	2381	721	520	520
query77	256	243	217	217
query78	4026	3692	3244	3244
query79	2727	829	578	578
query80	1592	353	280	280
query81	503	152	134	134
query82	621	124	97	97
query83	296	214	187	187
query84	302	109	89	89
query85	812	340	285	285
query86	399	174	168	168
query87	1013	952	890	890
query88	2891	2132	2130	2130
query89	284	197	176	176
query90	1942	133	130	130
query91	136	120	98	98
query92	82	70	70	70
query93	1629	996	720	720
query94	657	300	231	231
query95	489	329	226	226
query96	819	580	274	274
query97	1083	1050	1019	1019
query98	165	143	131	131
query99	411	348	301	301
Total cold run time: 178383 ms
Total hot run time: 82068 ms

hello-stephen · 2026-09-09T17:26:46Z

ClickBench: Total hot run time: 14.81 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 167e4e9d4e418e703e43a3697f01ae960baf6e1a, data reload: false

query1	0.00	0.00	0.00
query2	0.08	0.04	0.04
query3	0.25	0.10	0.10
query4	1.60	0.08	0.09
query5	0.17	0.16	0.17
query6	1.26	0.66	0.68
query7	0.03	0.00	0.01
query8	0.04	0.03	0.03
query9	0.29	0.21	0.21
query10	0.37	0.35	0.38
query11	0.16	0.12	0.11
query12	0.15	0.12	0.11
query13	0.31	0.31	0.30
query14	0.44	0.44	0.45
query15	0.36	0.36	0.34
query16	0.21	0.23	0.23
query17	0.73	0.70	0.71
query18	0.18	0.16	0.16
query19	1.19	1.22	1.12
query20	0.02	0.02	0.02
query21	15.45	0.16	0.12
query22	5.08	0.04	0.04
query23	16.17	0.26	0.10
query24	2.98	0.32	0.25
query25	0.09	0.05	0.04
query26	0.79	0.16	0.12
query27	0.04	0.03	0.04
query28	3.63	0.56	0.28
query29	12.46	3.22	2.57
query30	0.25	0.12	0.12
query31	2.76	0.37	0.17
query32	3.52	0.32	0.23
query33	1.38	1.56	1.55
query34	15.34	2.15	1.75
query35	1.76	1.74	1.74
query36	0.45	0.30	0.29
query37	0.05	0.03	0.03
query38	0.04	0.03	0.03
query39	0.03	0.02	0.03
query40	0.12	0.08	0.08
query41	0.08	0.03	0.03
query42	0.03	0.03	0.03
query43	0.03	0.03	0.03
Total cold run time: 90.37 s
Total hot run time: 14.81 s

hello-stephen · 2026-09-09T17:35:37Z

FE Regression Coverage Report

Increment line coverage 100.00% (6/6) 🎉
Increment coverage report
Complete coverage report

morrySnow requested review from 924060929, englefly and starocean999 as code owners September 9, 2026 10:19

morrySnow changed the title ~~[fix](fe) Preserve cast boundaries during arithmetic simplification~~ [fix](expr opt) Preserve cast boundaries during arithmetic simplification Sep 9, 2026

morrySnow added dev/3.1.x dev/4.0.x dev/4.1.x dev/4.2.x and removed dev/3.1.x dev/4.0.x dev/4.1.x labels Sep 9, 2026

Navigation Menu

Uh oh!

[fix](expr opt) Preserve cast boundaries during arithmetic simplification - #67733

[fix](expr opt) Preserve cast boundaries during arithmetic simplification#67733
morrySnow wants to merge 1 commit into
apache:masterfrom
morrySnow:fix/preserve-arithmetic-cast-semantics

morrySnow commented Sep 9, 2026

Uh oh!

hello-stephen commented Sep 9, 2026

Uh oh!

morrySnow commented Sep 9, 2026

Uh oh!

hello-stephen commented Sep 9, 2026

Uh oh!

hello-stephen commented Sep 9, 2026

Uh oh!

hello-stephen commented Sep 9, 2026

Uh oh!

hello-stephen commented Sep 9, 2026

Uh oh!

hello-stephen commented Sep 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

morrySnow commented Sep 9, 2026

Problem

Root cause

Reproduction

Fix

Tests

Uh oh!

hello-stephen commented Sep 9, 2026

Uh oh!

morrySnow commented Sep 9, 2026

Uh oh!

hello-stephen commented Sep 9, 2026

FE UT Coverage Report

Uh oh!

hello-stephen commented Sep 9, 2026

Uh oh!

hello-stephen commented Sep 9, 2026

Uh oh!

hello-stephen commented Sep 9, 2026

Uh oh!

hello-stephen commented Sep 9, 2026

FE Regression Coverage Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants