Add bidirectional Apache Gravitino converter

[LiJie20190102]

Hi all 👋

I'd like to gauge interest in adding a bidirectional converter between Apache Ossie semantic models and Apache Gravitino under converters/gravitino/.

Background

Apache Gravitino is a high-performance, geo-distributed, federated metadata lake (Apache Incubating). It provides a unified REST API to manage metadata across many data sources — Hive, JDBC (MySQL/PostgreSQL/Doris/StarRocks), Iceberg, Paimon, Hudi, Kafka, and file-based sources — through a single "catalog of catalogs" abstraction.

Gravitino already knows about physical metadata (catalogs, schemas, tables, columns, types, partition keys, tags, properties) for the entire big-data ecosystem. What it does not carry is Ossie's semantic layer — metrics, cross-dataset relationships, time-dimension markers, and AI context. A converter between the two would let any Ossie-compatible BI/AI tool access the full inventory of a big-data platform through Gravitino's unified API, and conversely let a semantic model authored in Ossie be materialized as table metadata in a big-data catalog.

This is also a two-Apache-podlings integration — both Gravitino and Ossie are Apache Incubating projects, and a converter between them benefits both ecosystems.

I'd like to contribute this converter myself. I'm a big-data platform engineer with deep experience in the Hive/Spark/Iceberg ecosystem, and I've already studied the existing converters/polaris/ codebase and prototyped the Gravitino REST API integration. I'm happy to follow the "Contributing a New Converter" steps (vendor entry, both directions, tests, README) and would open a PR once the approach is agreed.

What's included

• converters/gravitino/ — a Java/Maven module following the pattern of converters/polaris/.
• GravitinoClient — a thin REST client calling Gravitino's existing API (Java 11 HttpClient + Jackson), shared by both directions:
  • GET /api/metalakes/{m}/catalogs → list catalogs
  • GET /api/metalakes/{m}/catalogs/{c}/schemas → list schemas
  • GET /api/metalakes/{m}/catalogs/{c}/schemas/{s}/tables → list tables
  • GET /api/metalakes/{m}/catalogs/{c}/schemas/{s}/tables/{t} → load table
  • POST /api/metalakes/{m}/catalogs/{c}/schemas/{s}/tables → create table

Import (Gravitino → Ossie)

Reads table/schema metadata from a Gravitino metalake and produces an Ossie semantic model YAML.

Gravitino concept	Ossie concept
Metalake + Catalog	SemanticModel
Schema	dataset grouping within the model
Table	Dataset
table.name	dataset.name
catalog.schema.table	dataset.source
table.columns[]	dataset.fields[]
column.name + column.type	field.name + expression (ANSI_SQL)
column.type = timestamp/date	field.dimension.is_time = true
table.partitionKeys[]	flagged in field.description or custom_extensions
table.properties	dataset.custom_extensions[vendor_name: GRAVITINO]
table.tags	candidate source for dataset.ai_context

CLI: ossie-gravitino-converter import --url URL --metalake M [--catalog C] [--auth ...] -o output.yaml

Export (Ossie → Gravitino)

Takes an Ossie semantic model and creates/updates the corresponding tables in Gravitino via its REST API.

Ossie concept	Gravitino concept
SemanticModel	Metalake + Catalog (1:1, or configurable mapping)
Dataset	Table
dataset.name	table.name
dataset.source	resolved to catalog.schema.table
dataset.fields[]	table.columns[]
field.expression (ANSI_SQL)	column.name (extracted from expression)
field.dimension.is_time	column.type set to timestamp / date
dataset.primary_key	table.primaryKey or table.constraints
dataset.unique_keys	table.constraints (unique)
dataset.custom_extensions[vendor_name: GRAVITINO]	table.properties (round-trip)
dataset.description / field.description	table.comment / column.comment

CLI: ossie-gravitino-converter export --url URL --metalake M --catalog C ossie_model.yaml [--auth ...] [--dry-run]

Design considerations

• Round-trip fidelity: Gravitino → Ossie → Gravitino should be lossless. Gravitino's relational model does not have a native "relationship" or "metric" concept, so Ossie relationships and metrics are serialized into custom_extensions[vendor_name: OSSIE] on the corresponding table, following the round-trip-fidelity pattern used by the Databricks converter ([OSSIE][DATABRICKS] Add Databricks Unity Catalog Metric View converter #224).
• No changes on the Gravitino side: the converter is a pure client of Gravitino's existing REST API. Gravitino does not need to know about Ossie.
• Idempotency: the exporter checks if a table already exists before creating (GET then POST/PUT), so re-running is safe.
• Type inference: Ossie fields are expression-based and don't carry explicit physical types. The exporter uses heuristics (field name patterns + is_time flag) to infer Gravitino column types, similar to PolarisExporter.inferIcebergType(). Gravitino's External Type fallback is used when inference fails.
• Big-data ecosystem gap: Ossie currently has converters for BI/SaaS tools (dbt, Snowflake, Tableau, GoodData, Omni, …) but none for big-data catalog/metadata services. Gravitino is the natural bridge — one converter unlocks Hive, Iceberg, Paimon, Hudi, JDBC, and more.

Approach

I'd follow the existing converter pattern (closest analogue: converters/polaris/):

1. GravitinoClient — REST client authenticating via Gravitino's OAuth or basic auth.
2. GravitinoImporter — walks metalake → catalog → schema → table, maps each table to an Ossie Dataset with fields derived from columns. Time types (timestamp, date, timestamptz) set dimension.is_time.
3. GravitinoExporter — reads an Ossie model, infers column types, and calls Gravitino's create-table API.
4. Reuse OsiYamlGenerator / OsiModelParser (or the shared ossie-common module if refactor(converters): group Java modules and share generated Ossie models #235 lands first).
5. Gravitino tags/properties that carry semantic hints (e.g. description, owner) flow into ai_context or description; non-semantic properties go into custom_extensions[vendor_name: GRAVITINO].
6. CLI + tests (mocked Gravitino API responses with Hive + JDBC-MySQL fixtures) + README, following CONTRIBUTING.md.

Before I open the PR: is there anything you'd want me to keep in mind on approach or scope? And should I wait for #235 (Java shared module) to land, or build against the current converters/polaris/ structure for now?

I'll go ahead and start the implementation either way — just want to make sure I'm aligned with the project's direction.

Thanks!

[wForget]

Before I open the PR: is there anything you'd want me to keep in mind on approach or scope? And should I wait for #235 (Java shared module) to land, or build against the current converters/polaris/ structure for now?

Sorry that PR #235 did not lead to much discussion. Also, the common module it introduced only contains the OSI models generated from core-spec/osi-schema.json. I think you can feel free to move forward with your idea.