Skip to content

docs: add live-data-products concept page#36761

Open
sjwiesman wants to merge 15 commits into
MaterializeInc:mainfrom
sjwiesman:ont
Open

docs: add live-data-products concept page#36761
sjwiesman wants to merge 15 commits into
MaterializeInc:mainfrom
sjwiesman:ont

Conversation

@sjwiesman

Copy link
Copy Markdown
Contributor

Motivation

Materialize's user docs don't currently teach the primary use case in declarative, solution-first terms: that you build the nouns of your business as live SQL views, and together they form your business's ontology. New developers (especially AI/agent platform builders) have to assemble this picture themselves from the mechanical concept pages (Clusters, Sources, Views, Indexes, Sinks). This PR adds a front-door concept page that opens with the worldview and shows it through a worked example.

Description

Adds /concepts/live-data-products/ and wires it into three navigation surfaces:

  • New row in the Concepts table at doc/user/content/concepts/_index.md, positioned second (after Reaction Time).
  • New first bullet in the "Learn more" list in doc/user/content/get-started/_index.md.
  • New "Start here" section on the docs home page (doc/user/content/_index.md), inserted between the headless intro include and the "What's new!" callout.

The page is audience-broad (developers + AI/agent platform builders), agent-forward in §5 but not agent-only elsewhere. It opens with the silo-stitching problem (Customer scattered across CRM/OMS/billing/support/fulfillment), shows a single view that stitches the silos, then walks through composition (Customer → Orders → Stores → at_risk_orders). §4 names strict serializability inline and states that a change propagates through the entire business within about a second. §5 frames the agentic feedback loop (observe → think → act → observe the consequences) and gives agent builders two named primitives (Read, Compose) plus a link to the MCP integration. §6 closes with the reframe that a live data product is the unit of work Materialize is designed for.

Verification

  • `cd doc/user && hugo --quiet --renderToMemory` exits 0.
  • All inline and Learn-more links resolve: `/concepts/reaction-time/`, `/get-started/quickstart/`, `/serve-results/`, `/integrations/mcp-server/`.
  • Concepts sidebar shows Live Data Products second, after Reaction Time.
  • Home page renders the "Start here" section between the intro include and the "What's new!" callout.

@sjwiesman sjwiesman requested a review from a team as a code owner May 27, 2026 21:45
Comment thread doc/user/content/concepts/_index.md Outdated
Comment thread doc/user/content/concepts/live-data-products.md Outdated
Comment thread doc/user/content/get-started/_index.md Outdated
Comment thread doc/user/content/_index.md Outdated
@maheshwarip

maheshwarip commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

I think @kay-kim had a POV here that I agree with - we should make this some of this content the front page for the docs. So our "What is Materialize" page of the docs would look like this:

  1. What is Materialize
  • Materialize lets you build live data products to represent your business, using just SQL.
    • What are live data products
    • How does it work:
      • Real time ingestion from sources
      • Incremental view maintenance
      • Standard SQL & PGWire support for client applications
      • Consistency guaranteed
    • How to use Materialize
      • Cloud
      • Self Managed
      • Emulator

Next, I wouldn't put this in the "concepts" section. Rather, I'd add a new section for "Architecture Patterns". And then let's use most of the content from this page, but call it the "Live Context Graph"

  • What a context graph is
  • How to represent the nouns of your business as data products
  • How to create a compounding ontology of data products
  • How to ensure a tight feedback loop with agents

@sjwiesman, @kay-kim lmk if you agree! IMHO let's make these changes, and let's ship it. I can think of a few ways to make this even better, but let's not let perfection be our enemy!

@maheshwarip

Copy link
Copy Markdown
Contributor

I can make a few of the changes and push an update - give me a few mins!

@maheshwarip

Copy link
Copy Markdown
Contributor

@sjwiesman / @kay-kim lmk how you feel about this structure!

@sjwiesman

Copy link
Copy Markdown
Contributor Author

Structure looks good but there are merge conflicts in your commit 😄

@maheshwarip

Copy link
Copy Markdown
Contributor

I am bad at git

@kay-kim kay-kim left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heh heh ... so, I'll make some updates on Friday per slack discussion.

Image Image

@maheshwarip

Copy link
Copy Markdown
Contributor

I need to git good. Let me get rid of some of these changes and clean it up, sorry!

@kay-kim

kay-kim commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Actually, will have to be next week.

@kay-kim

kay-kim commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Oh ... lol .. I misunderstood what was being reverted. I thought everything was being reverted. I'll:

  • rebase off main
  • update lint tests to skip the avif file
  • update the config.toml ... because for folders that have _index.md .. you don't need to touch the config.toml.

sjwiesman and others added 13 commits June 30, 2026 15:17
Introduce /concepts/live-data-products/ — a new front-door concept
page that teaches Materialize's primary use case: building the nouns
of your business as live SQL views that compose into your business's
ontology.

The page opens with the silo-stitching problem (one entity scattered
across CRM, OMS, billing, support, fulfillment), shows how a single
view stitches them, and walks through composition (Customer → Order →
Store → at_risk_orders). It names strict serializability inline as
the consistency model and "about a second" as the freshness claim.
The closing sections frame the agentic feedback loop (observe → think
→ act → observe the consequences) and the ontology as the unit of
work Materialize is designed for.

Wire the page into navigation:
- Add as the second row in the Concepts table (after Reaction Time).
- Add as the first bullet in get-started/_index.md "Learn more".
- Add a "## Start here" section on the docs home page.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Seth Wiesman <sjwiesman@gmail.com>
@kay-kim kay-kim requested a review from a team as a code owner June 30, 2026 19:19

@kay-kim kay-kim left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving now. Will rebase once the build failures are fixed on main. every PR from about 30 minutes seems to be failing.

@def-

def- commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Retriggered the infra-affected tests

@def- def- left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI changes lgtm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants