ci: add build workflow for Flink 1.18/1.19/1.20 across JDK 11/17

[LuciferYang]

What

Adds a Build GitHub Actions workflow (.github/workflows/build.yml) that compiles and runs the unit tests across every supported Flink version and JDK. Until now the only workflow was pr-title.yml (semantic PR-title lint), so CI never built or tested the code.

How

• Triggers on push to main and pull_request targeting main.
• Matrix is flink {1.18, 1.19, 1.20} x java {11, 17}, 6 jobs, with fail-fast: false so one red cell doesn't hide the others.
• Each job runs actions/setup-java@v4 (temurin, maven cache) then mvn -B -ntp -pl lance-flink-<ver> verify, and sets timeout-minutes: 30 so a hung test is killed well before GitHub's 6h default.
• concurrency cancels superseded runs; permissions is contents: read.
• The workflow reuses the same short Apache license header as pr-title.yml.

The one failing test

LanceReadOptimizationsTest.testInPredicatePushDown asserts that an IN predicate gets pushed down, but the source never implemented that. LanceDynamicTableSource returns null for BuiltInFunctionDefinitions.IN (the code comments // IN (not supported yet)), so the test failed on every run. It is now @Disabled with the reason inline, which lets the suite go green. Turning it into a real test belongs in a separate PR (see follow-ups); this one only adds the pipeline.

Integration tests are out of scope here

This PR runs compile plus unit tests. mvn verify with the current surefire setup matches *Test/*Tests only, so the *ITCase classes never run. Wiring them in needs its own fix first:

• There is no maven-failsafe-plugin, so *ITCase classes aren't executed by the build at all.
• Adding failsafe locally makes the IT run fail on pre-existing shade/relocation problems. The ITs run after package against the shaded jar and throw NoSuchMethodError on LanceTypeConverter.toArrowSchema(RowType) and LanceNamespace.connect(String, Map, BufferAllocator) (shaded-artifact classpath skew, not a test bug). The namespace createTable path separately needs an Arrow IPC schema stream (InvalidInputException code=13).

I left those out to keep this PR focused on the pipeline.

Follow-ups

• Wire *ITCase in through maven-failsafe-plugin after fixing the shade relocation, so the ITs run against the packaged jar.
• Replace the @Disabled on testInPredicatePushDown with an assertion of the current fallback (accepted 0, remaining 1), and fix the class Javadoc that still lists IN and BETWEEN as covered.

Test plan

• mvn -pl lance-flink-1.18 verify -> BUILD SUCCESS (178 run, 0 failures, 18 skipped)
• mvn -pl lance-flink-1.19 verify -> BUILD SUCCESS
• mvn -pl lance-flink-1.20 verify -> BUILD SUCCESS
• Ran locally on both JDK 17 and JDK 11; all three modules green on each.
• JDK 11 leg also confirmed by this workflow once it runs in CI.

[LuciferYang]

cc @fightBoxing FYI

[fightBoxing]

Thanks for kicking off CI @LuciferYang — long overdue and the workflow itself is clean (concurrency + cancel-in-progress, least-privilege permissions, timeout, fail-fast: false, maven cache, -B -ntp). A few concrete items worth clarifying before we merge, mostly around what this CI actually covers:

1. -pl lance-flink-${{ matrix.flink }} may not run the test you disabled

The repo layout today is:

./pom.xml                             # packaging=pom, aggregates the 3 modules
./src/test/java/.../LanceReadOptimizationsTest.java   ← the file you touched
./lance-flink-1.18/          (module)
./lance-flink-1.19/          (module)
./lance-flink-1.20/          (module)

LanceReadOptimizationsTest (and the rest of the existing tests: FlinkSqlReadWriteTest, LanceCatalogS3Test, LanceSqlITCase, LanceAggregatePushDownTest, LanceNamespaceCatalogITCase, …) all live under root src/test/java, while the three per-Flink modules have no tests of their own.

With mvn -B -ntp -pl lance-flink-1.19 verify, Maven only builds that one child module, so:

• The @Disabled you added likely isn't hit by this pipeline at all — could you share the exact local command that reproduced the IN push-down failure so we can be sure we're disabling something the CI would otherwise fail on?
• More importantly, none of the existing root-level tests are exercised by this workflow, which means a green build here would be misleading.

Two ways to fix, either is fine:

• (a) Move / duplicate the root src/test tests into the appropriate submodule(s), or
• (b) Change the run step to mvn -B -ntp -am -pl lance-flink-${{ matrix.flink }} verify and add a second step (or a separate job) that runs the aggregator: mvn -B -ntp -Dflink.version.profile=${{ matrix.flink }} verify from the root, so the root test tree actually runs.

-am (also-make) is worth adding regardless — right now if lance-flink-1.19 depends on any sibling/root artifact not yet installed, -pl alone will fail on a clean checkout.

2. Matrix coverage — a couple of gaps worth a decision

• Flink 1.17 is still in the official support window; is dropping it intentional?
• JDK 21 is supported by Flink 1.19+ and increasingly common in downstream deployments. Adding "21" to the java matrix (perhaps gated with include: so we don't run 1.18 × 21) would give real signal.
• No paths: filter on push / pull_request, so doc-only changes will burn 6 jobs. Low priority, but a paths-ignore: ['**.md', 'docs/**'] is cheap.

3. Test signal / quality gates

Nice-to-have follow-ups (don't have to be in this PR):

• Upload **/target/surefire-reports/** as an artifact on failure — makes remote debugging 10× easier.
• Run spotless / checkstyle / license-header check in a separate lightweight job (single JDK / Flink) so style issues don't multiply by 6.
• Consider splitting UT vs IT (-DskipITs for the fast lane, a dedicated IT job with verify for the slow lane) once IT count grows.

[LuciferYang]

Addressed the actionable items in two commits (86c3042, 73c5897). Replies inline.

1. -pl <module> does run the root tests

The three per-Flink modules pull the root test tree in through build-helper-maven-plugin's add-test-source, so ../src/test/java gets compiled and run under each module's classpath. It shows up in a per-module run: lance-flink-1.19/target/surefire-reports/ contains LanceReadOptimizationsTest, LanceCatalogS3Test, LanceAggregatePushDownTest, and the rest. So mvn -pl lance-flink-1.19 verify runs the whole root test tree against Flink 1.19, and the matrix runs it once per Flink version instead of skipping it. The green build is real.

The IN failure this PR disables reproduces on this branch without editing anything, by deactivating the @Disabled condition:

mvn -B -ntp -pl lance-flink-1.18 test -Dtest='LanceReadOptimizationsTest' \
  -Djunit.jupiter.conditions.deactivate='org.junit.*DisabledCondition'

org.opentest4j.AssertionFailedError: IN predicate should be accepted ==> expected: <1> but was: <0>
    at ...LanceReadOptimizationsTest$FilterPushDownTests.testInPredicatePushDown(LanceReadOptimizationsTest.java:273)

LanceDynamicTableSource returns null for BuiltInFunctionDefinitions.IN, so the predicate falls through to remainingFilters and acceptedFilters comes back empty.

On -am: the three modules don't depend on each other, and the parent pom resolves through relativePath ../pom.xml, so -pl on its own builds cleanly on a fresh checkout. I verified that with clean local runs on JDK 11, 17, and 21. I'm fine adding -am anyway. One caveat on option (b): there's no flink.version.profile in the poms, so mvn -Dflink.version.profile=... verify from the root wouldn't select a version, and a bare root verify just builds all three modules, which duplicates the matrix.

2. Matrix

• Flink 1.17 was dropped on purpose. The root pom comment has the reasons: it EOL'd in 2024-10, and its bundled Calcite hits an assertion/IAE bug under JDK 8. I'd keep that out of this PR.
• Added JDK 21 for 1.19 and 1.20, gated through include. 1.18 stays on 8/11/17, since those are the JDKs Flink 1.18 supports. I ran all three modules locally on JDK 21 and they pass.
• Added paths-ignore for **.md and docs/**.

3. Signal and gates

• Added surefire-report artifact upload on failure.
• A separate lightweight spotless/checkstyle job and a UT/IT split are good follow-ups. spotless isn't actually bound in the pom yet (it only lives in pluginManagement), so it overlaps with build: add spotless, checkstyle, and enforcer plugin configuration #23/chore: apply initial spotless formatting #24 and is better handled there than by widening this PR.

[LuciferYang]

Added -am in d37bd39 as discussed. Verified all three modules still build green locally on JDK 17 (mvn -B -ntp -am -pl lance-flink-<ver> verify, 178 run / 0 failures each).