Skip to content

feat: add LanceNamespaceCatalog backed by Lance namespace API - #52

Merged
fightBoxing merged 1 commit into
lance-format:mainfrom
fightBoxing:fix/pr14-namespace-catalog
Jul 16, 2026
Merged

feat: add LanceNamespaceCatalog backed by Lance namespace API#52
fightBoxing merged 1 commit into
lance-format:mainfrom
fightBoxing:fix/pr14-namespace-catalog

Conversation

@fightBoxing

Copy link
Copy Markdown
Collaborator

Summary

Closes #14 — Adds Lance namespace catalog support.

Design (addressing @jackye1995's review)

  • No adapter layerLanceNamespaceCatalog extends AbstractCatalog directly using LanceNamespace. No intermediate AbstractLanceNamespaceAdapter or LanceNamespaceAdapter wrappers.
  • No name collisionLanceNamespaceCatalogFactory is a separate Flink CatalogFactory (type=lance-namespace), distinct from the existing LanceCatalogFactory (type=lance).
  • Real backend test — Integration test uses DirectoryNamespace (no mocks).

Usage

CREATE CATALOG my_lance WITH (
    'type' = 'lance-namespace',
    'impl' = 'dir',
    'root' = '/tmp/lance-warehouse'
);
USE CATALOG my_lance;

Changes

  • LanceNamespaceConfig — configuration for namespace integration
  • LanceNamespaceCatalogAbstractCatalog backed by LanceNamespace
  • LanceNamespaceCatalogFactory — Flink SPI CatalogFactory
  • Added lance-namespace-core:0.7.5, lance-namespace-apache-client:0.7.5
  • Fixed arrow-memory-core version alignment (15.0.0 → 18.3.0)
  • LanceNamespaceCatalogITCase — 3 integration tests (2 passing, 1 needs Arrow IPC schema fix)

Verification

[INFO] Lance Flink Connector .............................. SUCCESS
[INFO] Lance Flink Connector - Flink 1.18 ................. SUCCESS
[INFO] Lance Flink Connector - Flink 1.19 ................. SUCCESS
[INFO] Lance Flink Connector - Flink 1.20 ................. SUCCESS

Tests verified with real DirectoryNamespace backend:

  • testListDatabases
  • testCreateAndDropDatabase

- Add LanceNamespaceCatalog extending AbstractCatalog using LanceNamespace directly
- Add LanceNamespaceCatalogFactory (Flink CatalogFactory, type='lance-namespace')
- Add LanceNamespaceConfig for namespace configuration
- Register factory in META-INF/services/org.apache.flink.table.factories.Factory
- Add lance-namespace-core:0.7.5 and lance-namespace-apache-client:0.7.5
- Fix arrow-memory-core version alignment (exclude 15.0.0 from lance-core)
- Integration test with real DirectoryNamespace backend
@fightBoxing
fightBoxing merged commit ffe4fc0 into lance-format:main Jul 16, 2026
2 checks passed
@github-actions github-actions Bot added the enhancement New feature or request label Jul 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant