fix(trino): support inline SQL UDFs (WITH FUNCTION ... BEGIN ... END) by rusackas · Pull Request #41803 · apache/superset

rusackas · 2026-07-06T18:23:43Z

SUMMARY

Trino queries that declare inline SQL UDFs (WITH FUNCTION ... RETURNS ... BEGIN ... END SELECT ...) fail to parse in SQL Lab, even though they run fine in Trino itself. Two things go wrong: sqlglot splits statements on every semicolon (including the ones inside BEGIN ... END routine bodies), and it has no grammar for FUNCTION specifications in a WITH clause. The upstream issue (tobymao/sqlglot#5178) was closed as "low priority, PRs welcome", so this fixes it on our side with a custom Trino dialect in superset/sql/dialects/trino.py, following the same pattern as our Firebolt/Dremio/Pinot dialects.

The dialect does two things: keeps routine bodies intact when chunking statements (tracking BEGIN/CASE/IF/LOOP/REPEAT/WHILE ... END nesting), and parses inline function specifications into opaque InlineUDF nodes that regenerate verbatim. Trino doesn't allow queries inside SQL UDF bodies, so the opaque representation hides no table references from the security checks... table extraction, mutation detection, and limit handling all keep working on the main query. The new code paths only activate on syntax that fails to parse today (WITH FUNCTION, CREATE [OR REPLACE] FUNCTION), so existing queries are untouched — a CTE literally named function still parses as a regular CTE, and there's a test pinning that.

A procedural IF statement whose condition is parenthesized (IF (a > b) THEN) is disambiguated from the scalar IF() function by checking whether the matching close paren is followed by THEN; a WITH clause with a normal CTE before the inline function (WITH cte AS (...), FUNCTION f() ...) is also handled.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

Before (from #26162): sql parse error: Expecting (. Line 2, Col: 29. — after, the query from the Trino docs parses, splits, and round-trips correctly.

TESTING INSTRUCTIONS

Run pytest tests/unit_tests/sql/dialects/trino_tests.py. Or with a Trino 458+ database connected, paste the example from the issue into SQL Lab and run it:

WITH FUNCTION meaning_of_life()
  RETURNS tinyint
  BEGIN
    DECLARE a tinyint DEFAULT CAST(6 AS tinyint);
    DECLARE b tinyint DEFAULT CAST(7 AS tinyint);
    RETURN a * b;
  END
SELECT meaning_of_life()

ADDITIONAL INFORMATION

Has associated issue: Fixes Trino SQL routines with multiple ; characters cannot run on Superset SQL Lab Editor #26162
Required feature flags:
Changes UI
Includes DB Migration (follow approval process in SIP-59)
- Migration is atomic, supports rollback & is backwards-compatible
- Confirm DB migration upgrade and downgrade tested
- Runtime estimates and downtime expectations provided
Introduces new feature or API
Removes existing feature or API

🤖 Generated with Claude Code

bito-code-review · 2026-07-06T18:23:53Z

Code Review Agent Run #216c6b

Actionable Suggestions - 0

Review Details

Files reviewed - 4 · Commit Range: 9982dba..9982dba
- superset/sql/dialects/__init__.py
- superset/sql/dialects/trino.py
- superset/sql/parse.py
- tests/unit_tests/sql/dialects/trino_tests.py
Files skipped - 0
Tools
- MyPy (Static Code Analysis) - ✔︎ Successful
- Astral Ruff (Static Code Analysis) - ✔︎ Successful
- Whispers (Secret Scanner) - ✔︎ Successful
- Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

/review - Manually triggers a full AI review.
/pause - Pauses automatic reviews on this pull request.
/resume - Resumes automatic reviews.
/resolve - Marks all Bito-posted review comments as resolved.
/abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by

codecov · 2026-07-06T18:36:58Z

Codecov Report

❌ Patch coverage is 18.93939% with 107 lines in your changes missing coverage. Please review.
✅ Project coverage is 65.16%. Comparing base (f999afa) to head (e896050).
⚠️ Report is 5 commits behind head on master.

Files with missing lines	Patch %	Lines
superset/sql/dialects/trino.py	17.69%	107 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #41803      +/-   ##
==========================================
+ Coverage   56.62%   65.16%   +8.53%     
==========================================
  Files        2774     2775       +1     
  Lines      156694   156831     +137     
  Branches    35771    35804      +33     
==========================================
+ Hits        88725   102193   +13468     
+ Misses      67138    52670   -14468     
- Partials      831     1968    +1137

Flag	Coverage Δ
hive	`38.65% <18.93%> (-0.04%)`	⬇️
mysql	`57.71% <18.93%> (?)`
postgres	`57.76% <18.93%> (?)`
presto	`40.57% <18.93%> (-0.04%)`	⬇️
python	`59.15% <18.93%> (+17.29%)`	⬆️
sqlite	`57.37% <18.93%> (?)`
unit	`100.00% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

netlify · 2026-07-07T13:31:52Z

✅ Deploy Preview for superset-docs-preview ready!

Name	Link
🔨 Latest commit	`e896050`
🔍 Latest deploy log	https://app.netlify.com/projects/superset-docs-preview/deploys/6a5fa9b2db60fd0007f47883
😎 Deploy Preview	https://deploy-preview-41803--superset-docs-preview.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.
🤖 Make changes	Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

bito-code-review · 2026-07-07T16:30:33Z

Code Review Agent Run #3ae36b

Actionable Suggestions - 0

Review Details

Files reviewed - 1 · Commit Range: 9982dba..2d6390d
- tests/unit_tests/sql/dialects/trino_tests.py
Files skipped - 0
Tools
- MyPy (Static Code Analysis) - ✔︎ Successful
- Astral Ruff (Static Code Analysis) - ✔︎ Successful
- Whispers (Secret Scanner) - ✔︎ Successful
- Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

/review - Manually triggers a full AI review.
/pause - Pauses automatic reviews on this pull request.
/resume - Resumes automatic reviews.
/resolve - Marks all Bito-posted review comments as resolved.
/abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by

rusackas · 2026-07-21T00:37:14Z

Ugh. Something's off with things here. The replies to codeant's comments claim set[str]/tuple[str, str]/int annotations got added to trino.py, and that the IF (...) block-vs-scalar-call ambiguity got fixed with a matching test. Checked the head commit and none of that's actually there: BLOCK_OPENERS, AMBIGUOUS_OPENERS, BODY_KEYWORDS, routine_mode, depth, and paren_depth are still bare, and _block_depth_delta/_consume_block still just check whether the next token is (. The PR body's own "Known limitation" section confirms IF (a > b) THEN still fails, so the "fixed" replies are just wrong, not a matter of taste.

Separately, Parser._parse is copied wholesale from sqlglot internals rather than extended through the public FUNCTIONS/TRANSFORMS hooks the other dialects (Dremio, Firebolt) use, so it'll silently drift on the next sqlglot bump. Worth a comment pinning the sqlglot version this was copied from, at least.

bito-code-review · 2026-07-21T05:15:01Z

Code Review Agent Run #d4578c

Actionable Suggestions - 0

Review Details

Files reviewed - 4 · Commit Range: d80525a..32977f5
- superset/sql/dialects/__init__.py
- superset/sql/dialects/trino.py
- superset/sql/parse.py
- tests/unit_tests/sql/dialects/trino_tests.py
Files skipped - 0
Tools
- MyPy (Static Code Analysis) - ✔︎ Successful
- Astral Ruff (Static Code Analysis) - ✔︎ Successful
- Whispers (Secret Scanner) - ✔︎ Successful
- Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

/review - Manually triggers a full AI review.
/pause - Pauses automatic reviews on this pull request.
/resume - Resumes automatic reviews.
/resolve - Marks all Bito-posted review comments as resolved.
/abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by

bito-code-review · 2026-07-21T11:15:09Z

Code Review Agent Run #1f0222

Actionable Suggestions - 0

Additional Suggestions - 1

tests/unit_tests/sql/dialects/trino_tests.py - 1
- Weak test assertion for nested parens · Line 288-306
  
  The test only verifies `len(statements) == 1` but doesn't confirm the `InlineUDF` node exists. Other tests in this file (e.g., `test_inline_udf_is_single_statement` at line 39-45) assert `len(list(statements[0].find_all(InlineUDF))) == 1`. Add similar assertion to verify the IF condition with nested parens is correctly recognized as a block opener, not misparsed as a scalar function.

Review Details

Files reviewed - 1 · Commit Range: 32977f5..abda45a
- tests/unit_tests/sql/dialects/trino_tests.py
Files skipped - 0
Tools
- MyPy (Static Code Analysis) - ✔︎ Successful
- Astral Ruff (Static Code Analysis) - ✔︎ Successful
- Whispers (Secret Scanner) - ✔︎ Successful
- Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

/review - Manually triggers a full AI review.
/pause - Pauses automatic reviews on this pull request.
/resume - Resumes automatic reviews.
/resolve - Marks all Bito-posted review comments as resolved.
/abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by

betodealmeida · 2026-07-21T12:51:25Z

@rusackas have we considered submitting these changes upstream directly to sqlglot? Or at least open a ticket there about the problem?

sqlglot cannot parse Trino SQL routine syntax, so queries declaring inline UDFs failed to parse in SQL Lab: the parser splits statements on every semicolon (including the ones inside BEGIN ... END routine bodies) and has no grammar for FUNCTION specifications in a WITH clause. The upstream issue (tobymao/sqlglot#5178) was closed as low priority, so this extends the Trino dialect on the Superset side. The custom dialect keeps routine bodies intact when splitting statements and parses inline function specifications into opaque InlineUDF nodes that regenerate verbatim. Trino does not allow queries inside SQL UDF bodies, so the opaque representation hides no table references from Superset's security checks. The extensions only activate on syntax that fails to parse today, so existing queries are unaffected. Fixes #26162 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Adds unit tests for the two Trino dialect branches the CI coverage gate flagged as untested: a RETURN body with no following expression, and a statement-terminating semicolon that carries an attached comment or has no trailing statement. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Add type annotations to the module-level constants and local counters in the Trino dialect, and fix the `IF (a > b) THEN` vs. scalar `IF(...)` ambiguity: an IF immediately followed by `(` is now classified as a procedural block only when the matching closing paren is followed by THEN, otherwise as a scalar function call. Adds a regression test for the parenthesized condition case. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

routine_mode only triggered when FUNCTION appeared immediately after WITH, so a WITH clause defining a regular CTE before the inline UDF (e.g. `WITH cte AS (...), FUNCTION f() ...`) never entered routine mode, letting semicolons inside the function body incorrectly split the statement. Detect FUNCTION as a fresh WITH-list entry whenever it follows WITH or a top-level comma, not just at the very start. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Adds cases for nested parens inside an `IF (...)` condition, a scalar function literally named `function` outside a routine specification, and an unbalanced `IF` condition, restoring 100% coverage on superset/sql/dialects/trino.py required by unit-tests-required. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The nested-parens regression test only checked the statement count, not that the IF condition was actually recognized as a block opener rather than a scalar call. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

bito-code-review · 2026-07-21T19:45:19Z

Code Review Agent Run #8f3c61

Actionable Suggestions - 0

Review Details

Files reviewed - 4 · Commit Range: f2209f3..e896050
- superset/sql/dialects/__init__.py
- superset/sql/dialects/trino.py
- superset/sql/parse.py
- tests/unit_tests/sql/dialects/trino_tests.py
Files skipped - 0
Tools
- MyPy (Static Code Analysis) - ✔︎ Successful
- Astral Ruff (Static Code Analysis) - ✔︎ Successful
- Whispers (Secret Scanner) - ✔︎ Successful
- Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

/review - Manually triggers a full AI review.
/pause - Pauses automatic reviews on this pull request.
/resume - Resumes automatic reviews.
/resolve - Marks all Bito-posted review comments as resolved.
/abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by

rusackas · 2026-07-21T23:33:48Z

Good call @betodealmeida. Opened tobymao/sqlglot#7926 upstream, with real grammar support for the whole routine body rather than the opaque approach here. If that lands, we can drop this dialect whenever we bump sqlglot... in the meantime this PR still fixes it for the pinned version.

rusackas · 2026-07-21T23:46:03Z

One thing I learned while upstreaming this: per Trino's actual grammar (rootQuery in SqlBase.g4), inline functions get their own leading WITH clause and can't be comma-mixed with named queries. So the WITH cte AS (SELECT 1), FUNCTION ... case this PR supports (and pins with a test) isn't valid Trino... tobymao/sqlglot#7926 emits WITH <functions> WITH <ctes> instead. Being permissive on input doesn't hurt anything here, but worth knowing if anyone compares the two implementations.

pull-request-size Bot added the size/XL label Jul 6, 2026

dosubot Bot added data:connect:trino Related to Trino sqllab Namespace | Anything related to the SQL Lab labels Jul 6, 2026

codeant-ai-for-open-source Bot reviewed Jul 6, 2026

View reviewed changes

Comment thread superset/sql/dialects/trino.py

rusackas requested review from betodealmeida, eschutho, mistercrunch and villebro July 7, 2026 17:26

rusackas mentioned this pull request Jul 20, 2026

Trino SQL routines with multiple ; characters cannot run on Superset SQL Lab Editor #26162

Open

3 tasks

codeant-ai-for-open-source Bot reviewed Jul 21, 2026

View reviewed changes

Comment thread superset/sql/dialects/trino.py Outdated

rusackas force-pushed the fix/issue-26162-open-pr-test-first branch from 7216898 to 32977f5 Compare July 21, 2026 03:48

github-actions Bot added the preset-io label Jul 21, 2026

codeant-ai-for-open-source Bot reviewed Jul 21, 2026

View reviewed changes

Comment thread superset/sql/dialects/trino.py

rusackas and others added 6 commits July 21, 2026 10:16

test(trino): assert InlineUDF node for nested-parens IF condition

e896050

The nested-parens regression test only checked the statement count, not that the IF condition was actually recognized as a block opener rather than a scalar call. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

rusackas force-pushed the fix/issue-26162-open-pr-test-first branch from de2e8ab to e896050 Compare July 21, 2026 17:17

Uh oh!

Conversation

rusackas commented Jul 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

SUMMARY

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

Uh oh!

bito-code-review Bot commented Jul 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Review Agent Run #216c6b

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov Bot commented Jul 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

netlify Bot commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for superset-docs-preview ready!

Uh oh!

bito-code-review Bot commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Review Agent Run #3ae36b

Uh oh!

rusackas commented Jul 21, 2026

Uh oh!

Uh oh!

bito-code-review Bot commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Review Agent Run #d4578c

Uh oh!

bito-code-review Bot commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Review Agent Run #1f0222

Uh oh!

betodealmeida commented Jul 21, 2026

Uh oh!

Uh oh!

bito-code-review Bot commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Review Agent Run #8f3c61

Uh oh!

rusackas commented Jul 21, 2026

Uh oh!

rusackas commented Jul 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

rusackas commented Jul 6, 2026 •

edited

Loading

bito-code-review Bot commented Jul 6, 2026 •

edited

Loading

codecov Bot commented Jul 6, 2026 •

edited

Loading

netlify Bot commented Jul 7, 2026 •

edited

Loading

bito-code-review Bot commented Jul 7, 2026 •

edited

Loading

bito-code-review Bot commented Jul 21, 2026 •

edited

Loading

bito-code-review Bot commented Jul 21, 2026 •

edited

Loading

bito-code-review Bot commented Jul 21, 2026 •

edited

Loading