feat(observation): add platform-neutral Databricks observed state

[ElliotSun]

Summary

Add the minimal M1 platform-observation foundation without making Databricks semantics part of the core domain model.

Changes

• add immutable platform-neutral ObservedPlatformState, asset, property, and identity models under semapact.observation;
• model provider-local identity as platform + ordered namespace + asset rather than hard-coding catalog/schema into the domain;
• add a Databricks adapter that consumes the official SDK boundary WorkspaceClient.tables.get(...) -> TableInfo and projects it directly into observed state;
• make authentication an explicit caller concern: the observation adapter accepts an already initialized/authenticated WorkspaceClient and does not resolve PATs, Azure/OAuth credentials, profiles, service principals, managed identity, or other auth methods;
• reuse datacontract-cli[databricks] through the semapact[databricks] optional dependency instead of maintaining a separate Databricks dependency envelope;
• keep contract import separate: observation does not call UnityImporter, create ODCS, or reuse ODCS type/logical-type projection;
• keep feat: support ODCS v3.1.0 property-level relationships #10 deliberately minimal: asset type, property name, physical type, and nullability only; owner/tags/constraints/relationships and lineage remain later evidence enrichments;
• make the package root lazy-load public exports and make Delta importer registration conditional on the deltalake extra, so installing Databricks support no longer imports unrelated Delta dependencies;
• preserve deterministic ordering/serialization while leaving the stable fingerprint algorithm for the dedicated follow-up;
• update architecture guidance to distinguish governed desired state from platform-neutral observed state.

Boundary

caller/application auth
        ↓
already initialized WorkspaceClient
        ↓
WorkspaceClient.tables.get(...)
        ↓
TableInfo
       / \
      /   \
datacontract-cli import   SemaPact observation
        │                       │
       ODCS             ObservedPlatformState
governed contract       external read-side state

The shared/reused layer is Databricks connectivity + typed TableInfo, not datacontract-cli's ODCS projection. Credential resolution stays outside the observation adapter.

Core identity stays provider-neutral:

ObservedAssetIdentity
= platform + namespace[] + asset

Databricks adapter:
namespace = (catalog, schema)

Future Snowflake/Fabric adapters can map their own hierarchy without changing the observation domain model.

Observation does not

• create or mutate an ODCS contract;
• resolve or store Databricks credentials/authentication mechanisms;
• invoke lifecycle merge or governance evaluation;
• classify drift;
• write back to a platform;
• infer ODCS logical types or requiredness from runtime metadata;
• include rich governance evidence or lineage in the minimal feat: support ODCS v3.1.0 property-level relationships #10 model;
• generate the stable observed-state fingerprint algorithm.

Existing Unity contract import implementation remains unchanged; CI explicitly verifies that it is still importable from a fresh semapact[databricks] installation.

Dependency validation

The repository keeps its existing frozen uv.lock baseline rather than implicitly re-resolving the whole dependency graph for this feature. A dedicated CI job therefore performs a fresh install from project metadata:

semapact[databricks]
  -> datacontract-cli[databricks]
  -> databricks-sdk

That smoke path currently resolves datacontract-cli 1.1.2 and databricks-sdk 0.133.0, verifies WorkspaceClient, TableInfo, and the existing import_unity_contract boundary, then runs the Databricks observation tests against the official SDK type.

Follow-up

Authentication/client construction is intentionally a separate roadmap story (semapact-project#121). Its initial CLI path may construct WorkspaceClient(host=..., token=...), while this observation API remains unchanged as other Databricks auth mechanisms are added.

Validation

• full repository CI on Python 3.11–3.14;
• package build on Python 3.11–3.14;
• fresh semapact[databricks] dependency-resolution smoke;
• official Databricks SDK TableInfo observation tests: 8/8 passing;
• CodeQL;
• Dependency Review.