feat(datasource): import objects from an OData datasource's $metadata, and import several entity sets from Studio and MCP

.changeset/27745-odata-metadata-import.md

@@ -0,0 +1,22 @@
+---
+'@objectstack/spec': patch
+'@objectstack/driver-odata': patch
+'@objectstack/service-datasource': patch
+'@objectstack/rest': patch
+'@objectstack/runtime': patch
+'@objectstack/mcp': patch
+'@object-ui/app-shell': patch
+'@object-ui/i18n': patch
+---
+
+feat(datasource): import objects from an OData datasource's `$metadata` — EDM types map to field types, the key to the id, and navigations to lookups; Studio's datasource page imports several entity sets at once, and MCP gains `list_remote_tables` / `import_object`
+
+Clause-②: yes (widening)
+Metadata-protocol: none
+
+- **Type matrix (`@objectstack/spec/data`).** `SqlDialect` gains `'odata'`, whose aliases map the EDM primitives (`Edm.String`, `Edm.Int16/32/64`, `Edm.Decimal`, `Edm.Double`, `Edm.Boolean`, `Edm.Date`, `Edm.DateTimeOffset`, `Edm.TimeOfDay`, `Edm.Guid`, `Edm.Binary`, `Edm.Stream`, `Collection(…)`) onto the matrix's canonical types. `Edm.Duration`, the geography / geometry families and a service's own enum or complex types stay unknown, so a draft flags them for review. `Edm.Int64` and `Edm.Decimal` take the same verdict as SQL `bigint` / `numeric` (exact for `number`).
+- **Gate 2 gets looser, for SQL federation too.** `lookup` and `master_detail` are now exact under the `text`, `integer` and `bigint` rows, as they already were under `uuid`: a reference stores the target's id, which on a federated object is the remote key a foreign-key column carries. Before, `isCompatible` refused a lookup over `CustomerID int` or `varchar` as a `type_mismatch` error, so the boot validation (`onMismatch: 'fail'`) refused any federated object that declared one. Nothing that passed before fails now.
+- **Draft (`@objectstack/service-datasource`).** For an OData datasource (the driver reports dialect `odata-4.0`, read as the `odata` vocabulary) `generateObjectDraft` / `importObject` / `refreshCatalog` and the validation map EDM types through that arm. A column whose name is not a snake_case field name drafts as its snake_case spelling with `external.columnMap` carrying the remote name (`CategoryID` → `category_id`); `rename` overrides now land in `columnMap` too, so a renamed field stays bound to its column. A remote table name drafts the same way (`Products` → `products`; a leading capital used to give `_products`). A single-valued navigation becomes a `lookup` on the column its `ReferentialConstraint` names when the constraint references the target's single key and the target is imported on the same datasource (or is the object itself); every other navigation is left out of the draft and named in `review` with its reason — import the target first.
+- **Driver (`@objectstack/driver-odata`).** `introspectSchema()` lists each navigation's `referentialConstraints`.
+- **Studio.** The datasource page's tables tab selects several entity sets and imports them in one go, saving lookup targets before the tables that reference them; the write opt-in is offered only on a datasource that allows writes, and an imported object stays read-only otherwise (single-table dialog included).
+- **MCP.** `list_remote_tables` and `import_object` reach the same `IExternalDatasourceService` calls, registered only for a token whose scopes name `metadata:write`, and refused by the runtime bridge without the capability the REST twins require (`FEDERATION_READ_CAPABILITY` / `FEDERATION_WRITE_CAPABILITY`, now exported from `@objectstack/rest`).

content/docs/data-modeling/odata-datasources.mdx

@@ -138,9 +138,24 @@ What each piece does on this driver:
 - **`id` is the entity key.** A single-property key is read as text (`'4711'`). A
   composite key is the body of its key predicate (`'OrderID=1,ProductID=7'`), which is
   also what `GET /data/<object>/<id>` takes.
-- **Declare each field by hand.** Nothing generates fields from `$metadata` today.
-  Navigation properties are never followed, so a related entity set is an object of its
-  own.
+- **Or import the objects from `$metadata`.** Once the datasource is saved, Studio's
+  datasource page lists its entity sets on the **Tables** tab. **Import** saves one,
+  and **Import selected** saves several, referenced entity sets first. An MCP client
+  whose token carries `metadata:write` reaches the same import through
+  `list_remote_tables` and `import_object`. The draft maps each EDM type to a field
+  type, makes the key the `id`, and writes `columnMap` for every property whose name is
+  not a snake_case field name (`CategoryID` becomes `category_id`). EDM types with no
+  field type, such as `Edm.Duration` or a service's own enum, are listed for review. An
+  imported object is read-only unless the datasource allows writes and you tick
+  **Allow writes to this object**.
+- **Navigation properties are never followed**, so a related entity set is an object of
+  its own. The import turns a single-valued navigation into a `lookup` on its foreign-key
+  property when its `ReferentialConstraint` is one property pair onto the target's key
+  and the target is already imported on the same datasource. Every other navigation is
+  left out and named in the review, so import the target first. The lookup reads the
+  related record by key, like any other lookup. These rules are what
+  `packages/services/service-datasource/src/__tests__/external-object-draft-odata-metadata.test.ts`
+  asserts, over a recorded Northwind `$metadata`.
 - **The service document is read lazily.** `connect()` asks the service nothing, and
   `$metadata` is read once, at the first request. A wrong `serviceRoot` or an unknown
   entity set shows up on the first read of the object, not at boot.
@@ -266,7 +281,7 @@ as a sharing scope. The full mapping is in
 
 - **OData v2.** Only the v4 dialect ships.
 - **`$expand` and navigation properties.** Bind each related entity set as an object of
-  its own.
+  its own; an imported foreign key becomes a `lookup` onto it.
 - **Actions and functions** of the service. `execute` is refused.
 - **Transactions and `$batch`.** Each write is one request.
 - **More than 1000 matching rows** in one read (the cap above).

packages/drivers/driver-odata/src/odata-driver.test.ts

@@ -225,7 +225,7 @@ describe('keys', () => {
 });
 
 describe('introspection', () => {
-  it('lists entity sets as tables, with EDM types, the key and single-valued navigations as lookup candidates', async () => {
+  it('lists entity sets as tables, with EDM types, the key, and single-valued navigations with their referential constraints as lookup candidates', async () => {
     const { driver } = driverFor([]);
     const schema = await driver.introspectSchema();
     expect(schema.dialect).toBe('odata-4.0');
@@ -234,8 +234,17 @@ describe('introspection', () => {
     expect(people.columns.find((c) => c.name === 'UserName')).toEqual({ name: 'UserName', type: 'Edm.String', nullable: false, primaryKey: true });
     expect(people.columns.find((c) => c.name === 'Emails')?.type).toBe('Collection(Edm.String)');
     expect(people.navigationProperties).toEqual([
-      { name: 'BestFriend', targetEntityType: 'Microsoft.OData.SampleService.Models.TripPin.Person', targetEntitySet: 'People', nullable: true },
-      { name: 'Photo', targetEntityType: 'Microsoft.OData.SampleService.Models.TripPin.Photo', targetEntitySet: 'Photos', nullable: true },
+      { name: 'BestFriend', targetEntityType: 'Microsoft.OData.SampleService.Models.TripPin.Person', targetEntitySet: 'People', nullable: true, referentialConstraints: [] },
+      { name: 'Photo', targetEntityType: 'Microsoft.OData.SampleService.Models.TripPin.Photo', targetEntitySet: 'Photos', nullable: true, referentialConstraints: [] },
+    ]);
+    expect(schema.tables.Flights!.navigationProperties).toEqual([
+      {
+        name: 'Airline',
+        targetEntityType: 'Microsoft.OData.SampleService.Models.TripPin.Airline',
+        targetEntitySet: 'Airlines',
+        nullable: false,
+        referentialConstraints: [{ property: 'AirlineCode', referencedProperty: 'AirlineCode' }],
+      },
     ]);
     expect(schema.tables.Flights!.columns.filter((c) => c.primaryKey).map((c) => c.name)).toEqual(['PlanItemId']);
   });

packages/drivers/driver-odata/src/odata-driver.ts

@@ -77,8 +77,20 @@ export interface ODataDriverRuntime {
 export interface ODataIntrospectedTable extends IntrospectedTable {
   /** The qualified entity type. */
   entityType: string;
-  /** Single-valued navigation properties: candidates for a lookup field, never followed by reads. */
-  navigationProperties: Array<{ name: string; targetEntitySet?: string; targetEntityType: string; nullable: boolean }>;
+  /**
+   * Single-valued navigation properties: candidates for a lookup field, never
+   * followed by reads. A lookup needs a column holding the target's key, which
+   * only a `ReferentialConstraint` names (the dependent property on this type →
+   * the principal property on the target); a navigation without one has no
+   * column to back a lookup.
+   */
+  navigationProperties: Array<{
+    name: string;
+    targetEntitySet?: string;
+    targetEntityType: string;
+    nullable: boolean;
+    referentialConstraints: Array<{ property: string; referencedProperty: string }>;
+  }>;
 }
 
 export interface ODataIntrospectedSchema extends IntrospectedSchema {
@@ -245,6 +257,7 @@ export class ODataDriver implements IDataDriver {
             targetEntityType: nav.type,
             ...(set.navigationBindings[nav.name] ? { targetEntitySet: set.navigationBindings[nav.name] } : {}),
             nullable: nav.nullable,
+            referentialConstraints: nav.referentialConstraints.map((rc) => ({ ...rc })),
           })),
       };
     }

packages/mcp/src/index.ts

@@ -25,7 +25,12 @@ export type {
   McpAuthoringToolRegistry,
   McpAuthoringInjection,
 } from './types.js';
-export { registerObjectTools, registerActionTools, grantsAuthoringTools } from './mcp-http-tools.js';
+export {
+  registerObjectTools,
+  registerActionTools,
+  registerFederationTools,
+  grantsAuthoringTools,
+} from './mcp-http-tools.js';
 export type {
   McpDataBridge,
   McpObjectSummary,
@@ -34,6 +39,7 @@ export type {
   McpActionSummary,
   McpActionParamSummary,
   RegisterActionToolsOptions,
+  McpFederationBridge,
 } from './mcp-http-tools.js';
 // The portable `SKILL.md` distributable (ADR-0036 Amendment C) — NOT the
 // `skill` metadata type; that one is projected onto MCP prompts just below.

packages/mcp/src/mcp-federation-tools.test.ts

@@ -0,0 +1,179 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+
+/**
+ * [#27745] The federation pair, `list_remote_tables` and `import_object`: an
+ * agent lists a federated datasource's entity sets and imports one through the
+ * same `IExternalDatasourceService` calls Studio's datasource page makes.
+ *
+ * Pinned here: the pair registers only for a token whose scopes name
+ * `metadata:write` (the authoring family's gate, since an import lands object
+ * metadata) and only on a bridge that carries both members; each call reaches
+ * the bridge with exactly the caller's arguments; a refusal the bridge throws
+ * comes back as the coded tool error.
+ */
+
+import { describe, it, expect, beforeEach } from 'vitest';
+import { MCP_OAUTH_SCOPE_DATA_READ, MCP_OAUTH_SCOPE_METADATA_WRITE } from '@objectstack/spec/ai';
+import { MCPServerRuntime } from './mcp-server-runtime.js';
+import type { McpDataBridge, McpFederationBridge } from './mcp-http-tools.js';
+import { renderSkillMarkdown } from './skill-md.js';
+
+const FEDERATION_TOOLS = ['import_object', 'list_remote_tables'];
+
+function makeBridge(withFederation = true): McpDataBridge & Partial<McpFederationBridge> & { calls: unknown[] } {
+  const calls: unknown[] = [];
+  const data: McpDataBridge = {
+    async listObjects() { return []; },
+    async describeObject() { return null; },
+    async query() { return { records: [] }; },
+    async get() { return null; },
+    async create() { return {}; },
+    async update() { return {}; },
+    async remove() { return {}; },
+  };
+  if (!withFederation) return { ...data, calls };
+  return {
+    ...data,
+    calls,
+    async listRemoteTables(datasource, opts) {
+      calls.push(['listRemoteTables', datasource, opts]);
+      if (datasource === 'locked') {
+        throw Object.assign(new Error('Introspecting an external datasource requires the `manage_platform_settings` capability.'), {
+          code: 'PERMISSION_DENIED',
+          status: 403,
+        });
+      }
+      return [{ name: 'Products', columnCount: 6 }, { name: 'Categories', columnCount: 4 }];
+    },
+    async importObject(datasource, remoteName, opts) {
+      calls.push(['importObject', datasource, remoteName, opts]);
+      return {
+        name: 'products',
+        definition: { name: 'products', datasource, external: { remoteName } },
+        review: [],
+      };
+    },
+  };
+}
+
+async function rpc(runtime: MCPServerRuntime, bridge: unknown, grantedScopes: string[] | undefined, body: unknown) {
+  const req = new Request('http://localhost/api/v1/mcp', {
+    method: 'POST',
+    headers: { 'content-type': 'application/json', accept: 'application/json, text/event-stream' },
+    body: JSON.stringify(body),
+  });
+  const res = await runtime.handleHttpRequest(req, {
+    bridge: bridge as McpDataBridge,
+    parsedBody: body,
+    ...(grantedScopes ? { toolOptions: { grantedScopes } } : {}),
+  });
+  return (await res.json()) as any;
+}
+
+async function toolNames(runtime: MCPServerRuntime, bridge: unknown, grantedScopes?: string[]): Promise<string[]> {
+  const json = await rpc(runtime, bridge, grantedScopes, { jsonrpc: '2.0', id: 1, method: 'tools/list' });
+  return (json.result?.tools ?? []).map((t: { name: string }) => t.name);
+}
+
+async function callTool(runtime: MCPServerRuntime, bridge: unknown, name: string, args: Record<string, unknown>) {
+  const json = await rpc(runtime, bridge, [MCP_OAUTH_SCOPE_METADATA_WRITE], {
+    jsonrpc: '2.0',
+    id: 2,
+    method: 'tools/call',
+    params: { name, arguments: args },
+  });
+  return json.result as { content: Array<{ text: string }>; isError?: boolean };
+}
+
+describe('the federation tools register behind metadata:write, on a bridge that carries them', () => {
+  let runtime: MCPServerRuntime;
+  beforeEach(() => {
+    runtime = new MCPServerRuntime({ name: 't', version: '1.0.0' });
+  });
+
+  it('registers both for a token naming metadata:write', async () => {
+    const names = await toolNames(runtime, makeBridge(), [MCP_OAUTH_SCOPE_METADATA_WRITE]);
+    expect(names.filter((n) => FEDERATION_TOOLS.includes(n)).sort()).toEqual(FEDERATION_TOOLS);
+  });
+
+  it('registers neither for a key or a session (no scope list), nor for a token without metadata:write', async () => {
+    for (const scopes of [undefined, [MCP_OAUTH_SCOPE_DATA_READ]]) {
+      const names = await toolNames(runtime, makeBridge(), scopes);
+      expect(names.filter((n) => FEDERATION_TOOLS.includes(n)), String(scopes)).toEqual([]);
+    }
+  });
+
+  it('registers neither on a bridge without the federation members', async () => {
+    const names = await toolNames(runtime, makeBridge(false), [MCP_OAUTH_SCOPE_METADATA_WRITE]);
+    expect(names.filter((n) => FEDERATION_TOOLS.includes(n))).toEqual([]);
+  });
+
+  it('annotates the listing read-only and the import as a non-destructive write', async () => {
+    const json = await rpc(runtime, makeBridge(), [MCP_OAUTH_SCOPE_METADATA_WRITE], { jsonrpc: '2.0', id: 1, method: 'tools/list' });
+    const byName = new Map((json.result.tools as Array<{ name: string; annotations?: Record<string, unknown> }>).map((t) => [t.name, t]));
+    expect(byName.get('list_remote_tables')?.annotations).toMatchObject({ readOnlyHint: true, destructiveHint: false });
+    expect(byName.get('import_object')?.annotations).toMatchObject({ readOnlyHint: false, destructiveHint: false });
+  });
+});
+
+describe('the federation tools reach the bridge with the caller\'s arguments', () => {
+  let runtime: MCPServerRuntime;
+  beforeEach(() => {
+    runtime = new MCPServerRuntime({ name: 't', version: '1.0.0' });
+  });
+
+  it('list_remote_tables answers the bridge\'s tables with their count', async () => {
+    const bridge = makeBridge();
+    const result = await callTool(runtime, bridge, 'list_remote_tables', { datasource: 'erp' });
+    expect(result.isError).toBeFalsy();
+    expect(JSON.parse(result.content[0]!.text)).toEqual({
+      tables: [{ name: 'Products', columnCount: 6 }, { name: 'Categories', columnCount: 4 }],
+      totalCount: 2,
+    });
+    expect(bridge.calls).toEqual([['listRemoteTables', 'erp', {}]]);
+  });
+
+  it('import_object forwards the name override and the write opt-in, and nothing it was not sent', async () => {
+    const bridge = makeBridge();
+    await callTool(runtime, bridge, 'import_object', { datasource: 'erp', remoteName: 'Products' });
+    await callTool(runtime, bridge, 'import_object', {
+      datasource: 'erp',
+      remoteName: 'Products',
+      name: 'erp_products',
+      writable: true,
+    });
+    expect(bridge.calls).toEqual([
+      ['importObject', 'erp', 'Products', {}],
+      ['importObject', 'erp', 'Products', { name: 'erp_products', writable: true }],
+    ]);
+  });
+
+  it('refuses an unknown argument by name instead of dropping it', async () => {
+    const bridge = makeBridge();
+    const result = await callTool(runtime, bridge, 'import_object', { datasource: 'erp', entitySet: 'Products' });
+    expect(result.isError).toBe(true);
+    expect(result.content[0]!.text).toContain('remoteName');
+    expect(bridge.calls).toEqual([]);
+  });
+
+  it('answers a refusal the bridge throws as the coded tool error', async () => {
+    const result = await callTool(runtime, makeBridge(), 'list_remote_tables', { datasource: 'locked' });
+    expect(result.isError).toBe(true);
+    expect(JSON.parse(result.content[0]!.text).error).toMatchObject({ code: 'PERMISSION_DENIED', status: 403 });
+  });
+});
+
+describe('the generated SKILL.md teaches the pair exactly where it registers', () => {
+  it('names both for a metadata:write grant on a host that carries them, and neither otherwise', () => {
+    const granted = renderSkillMarkdown({ grantedScopes: [MCP_OAUTH_SCOPE_METADATA_WRITE], federationTools: true });
+    for (const name of FEDERATION_TOOLS) expect(granted).toContain(`**${name}(`);
+    for (const options of [
+      { federationTools: true },
+      { grantedScopes: [MCP_OAUTH_SCOPE_DATA_READ], federationTools: true },
+      { grantedScopes: [MCP_OAUTH_SCOPE_METADATA_WRITE] },
+    ]) {
+      const skill = renderSkillMarkdown(options);
+      for (const name of FEDERATION_TOOLS) expect(skill).not.toContain(name);
+    }
+  });
+});