Skip to content

Generate the Flink SQL surface in the binding module (stacked on #63) - #65

Merged
estebanzimanyi merged 3 commits into
MobilityDB:mainfrom
estebanzimanyi:feat/flink-sql-surface
Sep 12, 2026
Merged

estebanzimanyi merged 3 commits into
MobilityDB:mainfrom
estebanzimanyi:feat/flink-sql-surface

Conversation

@estebanzimanyi

Copy link
Copy Markdown
Member

Name every facade method by the class the catalog puts it in

The MeosOps* facades are grouped by the catalog's object model, which gives a
class to every base value MEOS passes by reference: geom_in belongs to
Geometry, the cbuffer, npoint and pose constructors and accessors to Cbuffer,
Npoint and Pose, geo_as_text to Geo. The smoke tests call those methods on the
free-function classes, MeosOpsFreeGeo, MeosOpsFreeCbuffer, MeosOpsFreeNpoint and
MeosOpsFreePose, which hold none of them, so the binding module's tests do not
compile against facades generated from the current catalog. Each call names
the class that holds the method, in both the binding and the benchmark copies.

The wiring javadoc and README name methods the same way and are checked against
the same facades: overlaps_tbox_tbox is on MeosOpsFreeCore, edwithin_tgeo_tgeo
on MeosOpsFreeGeo, tbox_in on MeosOpsTBox, and the windowed-tier examples use
tpoint_length and tnumber_twavg, the MEOS functions for a trajectory's length
and a temporal number's time-weighted average.

Generate the Flink SQL surface in the binding module

The binding module generates the MobilityDB SQL surface for Flink SQL beside
the MeosOps facades: a second generate-sources execution runs
codegen_jvm.py --engine flink-sql over the same staged catalog and JMEOS jar
into target/generated-sql, and MobilityFlinkSql.registerAll registers every
function under its MobilityDB SQL name. The module takes flink-table-api-java
for the function and type classes, and the planner for its tests.

GeneratedSqlSurfaceTest runs the surface end to end on a Flink mini-cluster
against libmeos: accessors, overloads resolved by argument type, timestamps and
intervals crossing as Flink TIMESTAMP_LTZ and INTERVAL values, constructors
from Flink scalars, text input and output, a name Flink reserves called quoted,
a name Flink has a built-in for resolving to the built-in unless qualified, and
MEOS values grouped across a shuffle. The CI floor rises to the 27 tests the
suite holds, and GENERATION.md describes the surface.

Built with tools/refresh-from-master.sh against MobilityDB master 31aa65124a
and a JMEOS jar carrying the flink-sql engine, mvn clean test runs 27 tests,
15 in the binding module (8 of them GeneratedSqlSurfaceTest) and 12 in the
benchmark, none failing or skipped.

Test the arrays the generated SQL surface takes

The generated Flink SQL surface passes a SQL array to a MEOS function
taking a C array. The test builds an intset and a tstzset from arrays of
Flink scalars, measures minDistance between arrays of tgeompoint values
against the closed-form distances 5 and 1, and expects a null element to
raise. GENERATION.md states how an array crosses the surface, and the
suite floor rises to 28.

The MeosOps* facades are grouped by the catalog's object model, which gives a
class to every base value MEOS passes by reference: geom_in belongs to
Geometry, the cbuffer, npoint and pose constructors and accessors to Cbuffer,
Npoint and Pose, geo_as_text to Geo. The smoke tests call those methods on the
free-function classes, MeosOpsFreeGeo, MeosOpsFreeCbuffer, MeosOpsFreeNpoint and
MeosOpsFreePose, which hold none of them, so the binding module's tests do not
compile against facades generated from the current catalog. Each call names
the class that holds the method, in both the binding and the benchmark copies.

The wiring javadoc and README name methods the same way and are checked against
the same facades: overlaps_tbox_tbox is on MeosOpsFreeCore, edwithin_tgeo_tgeo
on MeosOpsFreeGeo, tbox_in on MeosOpsTBox, and the windowed-tier examples use
tpoint_length and tnumber_twavg, the MEOS functions for a trajectory's length
and a temporal number's time-weighted average.
The binding module generates the MobilityDB SQL surface for Flink SQL beside
the MeosOps facades: a second generate-sources execution runs
codegen_jvm.py --engine flink-sql over the same staged catalog and JMEOS jar
into target/generated-sql, and MobilityFlinkSql.registerAll registers every
function under its MobilityDB SQL name. The module takes flink-table-api-java
for the function and type classes, and the planner for its tests.

GeneratedSqlSurfaceTest runs the surface end to end on a Flink mini-cluster
against libmeos: accessors, overloads resolved by argument type, timestamps and
intervals crossing as Flink TIMESTAMP_LTZ and INTERVAL values, constructors
from Flink scalars, text input and output, a name Flink reserves called quoted,
a name Flink has a built-in for resolving to the built-in unless qualified, and
MEOS values grouped across a shuffle. The CI floor rises to the 27 tests the
suite holds, and GENERATION.md describes the surface.

Built with tools/refresh-from-master.sh against MobilityDB master 31aa65124a
and a JMEOS jar carrying the flink-sql engine, mvn clean test runs 27 tests,
15 in the binding module (8 of them GeneratedSqlSurfaceTest) and 12 in the
benchmark, none failing or skipped.
The generated Flink SQL surface passes a SQL array to a MEOS function
taking a C array. The test builds an intset and a tstzset from arrays of
Flink scalars, measures minDistance between arrays of tgeompoint values
against the closed-form distances 5 and 1, and expects a null element to
raise. GENERATION.md states how an array crosses the surface, and the
suite floor rises to 28.
@estebanzimanyi
estebanzimanyi merged commit dfe2912 into MobilityDB:main Sep 12, 2026
2 checks passed
@estebanzimanyi
estebanzimanyi deleted the feat/flink-sql-surface branch September 12, 2026 14:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant