PRDCT-502: add Semantic Layer documentation

[Iamfle4ka]

Supersedes #999 (open since 2026-07-08, 75 commits behind main). That PR's content was accurate when it was written; the product moved underneath it. This is the same page, re-cut on current main with the stale parts corrected — @jordanrburger, take it over or close whichever of the two you prefer.

Docs are the last thing blocking the announcement of a rollout that has been live for a month, so this aims to unblock it rather than start over: the prose, structure, and examples are Jordan's.

What changed vs #999

1. Tool visibility — was wrong, and created a support loop.
#999 says the four semantic tools "automatically appear" once the semantic layer is "enabled for your project". The mcp-semantic-tooling feature flag was deleted from keboola/mcp-server in commit 8ab508e2 (2026-07-10, merged via keboola/mcp-server#619) — two days after #999 opened — and replaced with project_has_semantic_models(), a per-request metastore probe that fails closed (src/keboola_mcp_server/mcp.py:103-117, applied at on_list_tools :1023-1025 and on_call_tool :1117-1125). The tools appear only once the project already contains a semantic model. A reader following #999 would email support, be told it is already on, connect, and still see nothing.

2. Section order follows from that. Building a model now comes before using it via MCP, because the model has to exist first. #999 had it the other way round.

3. The two paths that already exist are no longer missing. #999 presents the two AI Kit plugins as the only way to build a model:

• The Keboola UI has a Semantic Layer section (objects open read-only with an Edit toggle, and each shows the AI guidance it carries).
• The CLI has a full kbagent semantic-layer command group — already documented on main in _data/cli/command-reference.md and listed on /cli/commands/. Omitting it is an add-alongside of something main already documents.

4. get_semantic_schema doesn't return a schema. #999 tells readers (and their agents) to fetch the JSON Schema with it. The tool calls the bare api/v1/schema/{type} endpoint (tools/semantic/tools.py:723 passes no version; clients/metastore.py:94-96 only versions the path when given one), which returns a {"versions": [...]} listing. keboola/cli documents this and calls it an upstream gap it deliberately does not mirror. The table entry now says what the tool actually returns and points at kbagent semantic-layer schema for the schema document.

5. /sl-build and /sl-validate --deep need the kbagent binary on PATH (sl-build.md Step 0 detects it with command -v kbagent). #999's install snippet never mentions it, so those two commands fail for anyone who follows the page literally.

6. Enablement callout. "Private beta — contact support to have it enabled" no longer matches how the feature is reached. The callout now describes availability by stack (multi-tenant yes, single-tenant no — the metastore service is absent there) and tells the reader what to do if their project has no Semantic Layer section. Flagged VERIFY(Jordan) in the source for the plan-level wording, since that rollout was still moving at the last project update.

7. No duplicate bullet on the MCP Server page. main gained - **Semantic layer** – Explore the project's semantic models… in the "Available Tools" list during the 75 commits #999 is behind. #999 appends a second bullet for the same category with different capitalisation — invisible in its diff. Here the existing bullet is extended with the cross-link instead.

8. ai/ai-kit/index.md is deliberately untouched. #999 rewrites the same paragraph, the same install block, and adds the same two plugin sections that open PR #1095 (agent/kbagent-marketplace-repoint) already covers — a head-on collision. #1095 should own that page; this PR owns /ai/semantic-layer/.

9. CLAUDE.md guardrail. main still tells every agent "don't document … semantic layer". Dropped that clause, otherwise merging semantic-layer docs leaves a contributor guide contradicting the shipped docs. Owner's call — it is a stakeholder guardrail, revert the one-liner if you want it kept.

Also: the missing blank line before the following heading in ai/index.md, and sentence-case H2s per CLAUDE.md:43 (the rest of /ai/ is still Title Case — say the word and I'll match the siblings instead).

Fact-check pass

An independent verification pass against keboola/mcp-server, keboola/ai-kit, keboola/cli and this repo's own synced CLI reference caught three things in my own draft, fixed in the second commit:

• "There are six semantic object types" is contradicted by the CLI reference this repo ships — the metastore has a seventh, semantic-reference-data (_data/cli/command-reference.md:2509-2551). The six-type table is now scoped to what makes up a model, with the seventh named separately.
• The kbagent prerequisite was overstated. sl-build.md:69 and sl-validate.md:93 degrade rather than fail: without the binary, /sl-build asks you to describe your tables and --deep checks are skipped.
• powerbi-to-sl push — the README's "or the user directly" had been dropped; restored.

Everything else checked out: the four tool names and their readOnlyHint=True registrations, the gating logic and its fail-closed behaviour, the X-Read-Only-Mode interaction, the constraint severity enum, the six object-type descriptions against the JSON schemas, the CLI command list, both plugin descriptions, and the marketplace plugin names.

The UI section is the one part no public source can confirm (keboola/ui is private) — it is flagged VERIFY(Jordan) in the source, with the "AI guidance" sentence called out specifically.

Deliberately left out

Kai's semantic-layer building (AI-3661): four write tools (apply_semantic_model, update_semantic_objects, delete_semantic_objects, share_semantic_objects), all approval-gated. This section originally claimed they sit "behind a beta flag" — @davidesner's comment below corrects that: there is no flag on Kai's path. The semantic-layer project feature gates only the sidebar UI (verified against the 2026-08-19 #kbc-news-feed announcement), so Kai creates and uses models wherever the metastore service is available, and a project can hold a Kai-built model before the UI section appears. The enablement callout now says so (third commit). Documenting Kai's building itself is PRDCT-671 (blocked by this PR), with David's comment as the spec.

Verification

• npm run build clean — 362 pages.
• node scripts/audit-phase2.mjs — no issues on the touched pages.
• Every link and anchor resolves in dist/, including /ai/mcp-server/#restricting-tool-access.
• Sidebar regenerated with npm run gen:sidebar (not hand-edited).
• Tool names, read-only annotations, the six object types, and the gating logic re-verified against keboola/mcp-server at main today; CLI commands against the repo's own synced reference; plugins against keboola/ai-kit.

🤖 Generated with Claude Code

[linear-code]

PRDCT-502

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
connection-docs	Ready	Preview	Sep 7, 2026 10:53am UTC

[davidesner]

Cross-reference from #1106 (Kai chat controls). While documenting Kai's chat surface I investigated Kai's semantic-layer capability against keboola/ui and the announcements, and it lands on this PR's "Deliberately left out" section. Parking the findings here so whoever picks the follow-up doesn't have to re-derive them.

The beta-flag premise looks out of date

Kai's semantic-layer write tools (apply_semantic_model, update_semantic_objects, delete_semantic_objects, AI-3661) are shipped but behind a beta flag

I can't find a flag on that path. Two sources point the other way:

• The 2026-08-19 announcement in #kbc-news-feed — "Kai can now build and maintain your Semantic Layer for you" — closes with: "Currently semantic layer UI is shown only for projects with semantic-layer project feature, but Kai can create and use semantic layer unconditionally on all projects that have Metastore service available."
• The 2026-07-20 UI announcement confirms the same split from the other side: the semantic-layer project feature gates the sidebar UI ("internal testing only"), not Kai.

So the gate to describe is the one this PR already describes for the MCP tools — stack has Metastore, or nothing works — plus a second, UI-only gate. There's a case where a customer's project has a Kai-written semantic model they cannot see in the UI. That's worth a sentence wherever the enablement callout ends up.

What Kai actually does, if the follow-up wants it

From the bundled semantic-layer-building skill in keboola/ui (packages/kai-agent-sandbox/skills/semantic-layer-building/):

• Four write tools, all approval-gated: apply_semantic_model (create, additive only), update_semantic_objects, delete_semantic_objects, share_semantic_objects. The approval card names the per-object-type counts before anything is written.
• Reads via get_semantic_context / search_semantic_context — hidden on a project with no models, matching the project_has_semantic_models() probe described in point 1 of this PR. On a greenfield project Kai works from the skill's payload reference instead.
• Objects are the same six this PR documents, addressed by uuid or name; an ambiguous name is rejected with the candidates rather than guessed.
• Sharing: org-wide or per-project grants, but only organization admins can widen the scope — per the Aug-19 announcement.
• Kai scaffolds and validates the payload locally (scaffold_payload.py, validate_payload.py) before a single apply_semantic_model call, and --sql-dialect takes the project's real dialect (Snowflake or BigQuery) — the metastore accepts both, so a wrong guess validates clean and describes the project wrongly.

Not adding it to #1106

#1106 covers the chat surface (slash commands, attachments, plan mode, approvals). Per @davidesner this belongs with the semantic-layer docs rather than the Kai pages, so #1106 stays out of it — this comment is the handoff.

One note for whoever lands it: point 9 of this PR drops the CLAUDE.md "don't document … semantic layer" guardrail. That clause is still on main today, so until this merges, agents working on the Kai pages will keep declining to write semantic-layer content.

[Iamfle4ka]

Thanks — confirmed, the beta-flag premise was wrong. I re-read the 2026-08-19 #kbc-news-feed message and your quote is verbatim: the semantic-layer project feature gates only the sidebar UI, Kai works wherever the Metastore service is available.

Acted on it three ways:

• Enablement callout now carries the "model can exist before the UI section appears" sentence, with the sourcing recorded in the VERIFY(Jordan) comment next to it (commit is on the branch with the next push).
• PR description: the "Deliberately left out" section is rewritten — no more beta flag, all four write tools named including share_semantic_objects.
• Follow-up filed: PRDCT-671 "Document Kai's semantic-layer building on /ai/semantic-layer/", blocked by PRDCT-502, with your rundown linked as the spec — so the handoff has a home and won't be re-derived.

🤖 Generated with Claude Code

[davidesner]

Acted on my own comment above rather than leaving all of it to the follow-up: fc8e52e6 is on the branch.

The gap was that "Building a semantic model" listed the UI, the CLI, and the AI Kit plugins, but not Kai, which is the one path that needs nothing installed. The enablement callout mentioned a Kai-built model in passing, so the reader met the idea without ever being told they could ask for one.

What the commit does

/ai/semantic-layer/

• New "With Kai" build path, first of the four. Chat-based building with nothing to install, the explore, draft, validate, approve order, the approval card naming the model and its per-type counts, conversational edit / remove / share, and the scope rules: project-scoped by default, read-only sharing to named sibling projects, organization-wide reserved to an organization admin.
• Intro corrected. It said the definitions are read by "AI assistants connected through the MCP Server". Kai and the CLI read them too, so all three are named now.
• "With the CLI" gained search-context and get-context (_data/cli/command-reference.md:2476,2487). The section had covered authoring only, which left the CLI looking write-only while it mirrors the MCP read tools.

/kai/ now points back, which was the missing half of the cross-link: a Semantic Layer line in "What Kai Can Do" plus a Learn More entry, and a Semantic Layer section in Use Cases with build, add-metric, and share prompts. Concepts, object types, and build paths stay on the one canonical page; the Kai pages describe the capability in a line or two and link out.

Verification

I did not take my own comment at face value. Everything above is verified against keboola/ui, packages/kai-agent-sandbox/skills/semantic-layer-building/SKILL.md at main:

• Four write tools, all approval-gated, and the approval card carries the model name plus per-type counts, so prose should not restate the counts.
• Explore, then draft, then validate, then a single apply_semantic_model call.
• Scope ACL: project default, targeted settable by a project admin, organization reserved to organization-admin. That last one is the detail worth keeping, since a non-admin offering org scope gets a 403.
• Reads (get_semantic_context / search_semantic_context) hidden on a project with no models, matching the project_has_semantic_models() probe in point 1 of the PR description.

The sourcing is recorded in a comment next to the section, in the same style as the other two verification comments on the page.

npm run build clean at 363 pages. scripts/audit-phase2.mjs unchanged: the 29 broken internal links it reports are pre-existing and all under /extend/, /storage/, and /components/. Every new link resolves in dist/, including the /kai/getting-started/#action-approval anchor.

This closes PRDCT-671

Correcting what I wrote a moment ago: I called the follow-up "narrowed", but checked against PRDCT-671 it is satisfied. Its deliverable is "how Kai builds and maintains semantic models" on this page, and of the six points its spec lists, five are now on the page in reader-facing form: the write path with approval gating and the card naming per-type counts, the no-beta-flag situation via the enablement callout, the reads hidden until a model exists, the six object types, and sharing with organization-admin-only widening.

What is left over is tool-level plumbing that does not belong on a page whose reader is learning what a semantic model is: the four tool names, payloadPath / payloadSummary, --sql-dialect, and the name-ambiguity rejection. The tool names would have a home in Tool Permissions, and per AI-3742 the write tools do not appear there yet, so documenting them by name would be documenting something the reader cannot act on.

So PRDCT-671 should close when this PR merges, not carry forward. It is blocked by PRDCT-502, and until #1103 lands the content is only on the branch.

One style note for whoever reviews: the new Semantic Layer bullet in kai/index.md uses a single dash while its six siblings use em dashes. That is deliberate, per the house rule against em dashes in new content. The section-wide cleanup is a separate change, not this one.