feat(trino): support inline SQL UDFs (WITH FUNCTION ... SELECT) [CLAUDE]#7926
Open
rusackas wants to merge 1 commit into
Open
feat(trino): support inline SQL UDFs (WITH FUNCTION ... SELECT) [CLAUDE]#7926rusackas wants to merge 1 commit into
rusackas wants to merge 1 commit into
Conversation
Trino queries can declare SQL UDFs inline in a WITH clause preceding the query (https://trino.io/docs/current/udf/sql.html). These failed to parse (Expecting () and the semicolons inside BEGIN ... END routine bodies split the statement. Closes tobymao#5178. - TrinoParser parses FUNCTION entries in a WITH clause into a new exp.FunctionSpecification node, following the functionSpecification / controlStatement grammar from Trino's SqlBase.g4. A CTE literally named "function" still parses as a CTE. - Routine bodies support the full control statement grammar: RETURN, BEGIN ... END, DECLARE, SET, CASE ... END CASE, IF/ELSEIF/ELSE ... END IF, LOOP, WHILE ... DO, REPEAT ... UNTIL, ITERATE, LEAVE, and loop labels. New statement-level expressions: CaseStatement, IfStatement, LoopStatement, WhileStatement, RepeatStatement, Iterate, Leave; DECLARE, SET, RETURN and BEGIN reuse Declare, Set, Return and Block (with a new begin flag). - The body parser continues across the semicolon-separated chunks produced in Parser._parse, the same way _parse_block does for procedures, so sqlglot.parse no longer splits routine bodies into separate statements. - Routine characteristics reuse the existing property machinery, with Trino generator fixes so they round-trip: DETERMINISTIC instead of IMMUTABLE, SECURITY INVOKER instead of SQL SECURITY INVOKER, and DECLARE ... DEFAULT instead of =. - Inline functions live in their own WITH clause before the query's own WITH clause per Trino's grammar, so TrinoGenerator.with_sql emits WITH <functions> WITH <ctes> when both are present. - walk_in_scope treats FunctionSpecification as opaque so qualify does not try to resolve UDF parameters and variables as outer-scope columns. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
9 tasks
Author
|
CC @betodealmeida :D Thanks for suggesting I do this |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Howdy! I was fixing an Issue on Superset, and it was suggested I file the fix "upstream" so here we go!
Trino supports declaring SQL UDFs inline, in a
WITHclause that precedes the query (https://trino.io/docs/current/udf/sql.html):These fail to parse (
Expecting (), and the semicolons insideBEGIN ... ENDbodies additionally split the statement insqlglot.parse. #5178 was closed as low priority with an invitation for a well-tested PR, so here's an attempt. The same gap bit us over in Apache Superset (apache/superset#26162), and I'd rather fix it here than carry a custom dialect downstream.Parsing follows the
functionSpecification/controlStatementgrammar from Trino's SqlBase.g4.TrinoParser._parse_cterecognizesFUNCTION <name>entries and parses them into a newexp.FunctionSpecificationnode (a CTE literally namedfunctionstill parses as a CTE). Routine bodies support the full control statement grammar (RETURN,BEGIN,DECLARE,SET,CASE,IF/ELSEIF,LOOP,WHILE,REPEAT/UNTIL,ITERATE,LEAVE, loop labels), reusingDeclare/Set/Return/Blockwhere they fit and adding statement-level expressions for the rest. Since routine statements are semicolon-separated, the body parser continues across the chunks produced inParser._parse, the same way_parse_blockalready does for procedures, so multi-statement scripts still split correctly around the UDF query.Routine characteristics reuse the existing property machinery, with Trino generator fixes so they round-trip:
DETERMINISTICinstead ofIMMUTABLE,SECURITY INVOKERinstead ofSQL SECURITY INVOKER, andDECLARE ... DEFAULTinstead of=. Per the grammar, inline functions can't be comma-mixed with named queries, soTrinoGenerator.with_sqlemitsWITH <functions> WITH <ctes>when both are present.walk_in_scopetreatsFunctionSpecificationas opaque soqualifydoesn't try to resolve UDF parameters as outer-scope columns.Not covered:
AS $$...$$bodies forLANGUAGE PYTHONUDFs (would need$$tokenizer support in Trino), andCREATE FUNCTIONwith aBEGINbody, which still splits on inner semicolons today. Happy to follow up on either if there's interest.Disclosure: implemented with Claude Code, but reviewed, tested (
make unit,make unitc,make style) and understood by me. [Edit: it's true, I really look at stuff]