Skip to content

fix(core): compute taxonomy counts on demand#2219

Merged
khoinguyenpham04 merged 2 commits into
emdash-cms:mainfrom
logelog:fix/taxonomy-count-prefetch-d1
Jul 24, 2026
Merged

fix(core): compute taxonomy counts on demand#2219
khoinguyenpham04 merged 2 commits into
emdash-cms:mainfrom
logelog:fix/taxonomy-count-prefetch-d1

Conversation

@logelog

@logelog logelog commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Stops layout prefetch from recomputing visible taxonomy term counts when no consumer displays them.

Before this change, every HTML render called getTaxonomyTerms() for every taxonomy, which aggregated the full content-term assignment table even when only term labels were being prefetched. Without an object-cache backend, those scans went directly to D1 on every request.

This separates term-list caching from visible-count caching and adds an includeCounts option, defaulting to true so existing consumers keep their behavior. Layout prefetch opts out, while widgets request counts only when showCount is enabled. Query-count snapshots and documentation are updated.

A regression test reproduces two aggregate queries on current main; this branch performs zero during layout prefetch and computes counts only for explicit consumers.

Closes #2210

Type of change

  • Bug fix
  • Feature (requires maintainer-approved Discussion)
  • Refactor (no behavior change)
  • Translation
  • Documentation
  • Performance improvement
  • Tests
  • Chore (dependencies, CI, tooling)

Checklist

  • I have read CONTRIBUTING.md
  • pnpm typecheck passes
  • pnpm lint passes
  • pnpm test passes (targeted taxonomy tests)
  • pnpm format has been run
  • I have added/updated tests for my changes
  • No user-visible admin strings are introduced
  • I have added a changeset
  • New features link to an approved Discussion (not applicable; this is a bug fix)

Screenshots / test output

  • pnpm --filter emdash exec vitest run tests/unit/taxonomies — 8 files, 68 tests passed
  • pnpm --filter emdash typecheck
  • pnpm lint
  • pnpm format
  • pnpm --filter emdash build
  • pnpm query-counts --target sqlite --skip-build
  • pnpm query-counts --target d1

@changeset-bot

changeset-bot Bot commented Jul 23, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: ea5df32

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 17 packages
Name Type
emdash Patch
@emdash-cms/cloudflare Patch
@emdash-cms/sandbox-workerd Patch
@emdash-cms/plugin-mcp-smoke Patch
@emdash-cms/fixture-perf-site Patch
@emdash-cms/perf-demo-site Patch
@emdash-cms/cache-demo-site Patch
@emdash-cms/do-demo-site Patch
@emdash-cms/do-solo-demo-site Patch
@emdash-cms/admin Patch
@emdash-cms/auth Patch
@emdash-cms/blocks Patch
@emdash-cms/gutenberg-to-portable-text Patch
@emdash-cms/x402 Patch
create-emdash Patch
@emdash-cms/auth-atproto Patch
@emdash-cms/plugin-embeds Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions github-actions Bot added the query-count changed PR diff modifies query-count snapshot files label Jul 23, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Query-count snapshot changes

4 routes changed, total Δ -4 queries.

SQLite

Route Before After Δ
GET /posts/building-for-the-long-term (cold) 18 17 -1
GET /posts/building-for-the-long-term (warm) 18 17 -1

D1

Route Before After Δ
GET /posts/building-for-the-long-term (cold) 29 28 -1
GET /posts/building-for-the-long-term (warm) 18 17 -1

Comparing snapshot files between base and head. Updated automatically on each push.

@pkg-pr-new

pkg-pr-new Bot commented Jul 23, 2026

Copy link
Copy Markdown

Open in StackBlitz

@emdash-cms/admin

npm i https://pkg.pr.new/@emdash-cms/admin@2219

@emdash-cms/auth

npm i https://pkg.pr.new/@emdash-cms/auth@2219

@emdash-cms/auth-atproto

npm i https://pkg.pr.new/@emdash-cms/auth-atproto@2219

@emdash-cms/blocks

npm i https://pkg.pr.new/@emdash-cms/blocks@2219

@emdash-cms/cloudflare

npm i https://pkg.pr.new/@emdash-cms/cloudflare@2219

@emdash-cms/contentful-to-portable-text

npm i https://pkg.pr.new/@emdash-cms/contentful-to-portable-text@2219

emdash

npm i https://pkg.pr.new/emdash@2219

create-emdash

npm i https://pkg.pr.new/create-emdash@2219

@emdash-cms/gutenberg-to-portable-text

npm i https://pkg.pr.new/@emdash-cms/gutenberg-to-portable-text@2219

@emdash-cms/plugin-cli

npm i https://pkg.pr.new/@emdash-cms/plugin-cli@2219

@emdash-cms/plugin-types

npm i https://pkg.pr.new/@emdash-cms/plugin-types@2219

@emdash-cms/registry-client

npm i https://pkg.pr.new/@emdash-cms/registry-client@2219

@emdash-cms/registry-lexicons

npm i https://pkg.pr.new/@emdash-cms/registry-lexicons@2219

@emdash-cms/registry-verification

npm i https://pkg.pr.new/@emdash-cms/registry-verification@2219

@emdash-cms/sandbox-workerd

npm i https://pkg.pr.new/@emdash-cms/sandbox-workerd@2219

@emdash-cms/x402

npm i https://pkg.pr.new/@emdash-cms/x402@2219

@emdash-cms/plugin-ai-moderation

npm i https://pkg.pr.new/@emdash-cms/plugin-ai-moderation@2219

@emdash-cms/plugin-atproto

npm i https://pkg.pr.new/@emdash-cms/plugin-atproto@2219

@emdash-cms/plugin-audit-log

npm i https://pkg.pr.new/@emdash-cms/plugin-audit-log@2219

@emdash-cms/plugin-color

npm i https://pkg.pr.new/@emdash-cms/plugin-color@2219

@emdash-cms/plugin-embeds

npm i https://pkg.pr.new/@emdash-cms/plugin-embeds@2219

@emdash-cms/plugin-field-kit

npm i https://pkg.pr.new/@emdash-cms/plugin-field-kit@2219

@emdash-cms/plugin-forms

npm i https://pkg.pr.new/@emdash-cms/plugin-forms@2219

@emdash-cms/plugin-webhook-notifier

npm i https://pkg.pr.new/@emdash-cms/plugin-webhook-notifier@2219

commit: ea5df32

@logelog
logelog marked this pull request as ready for review July 23, 2026 16:18
@github-actions github-actions Bot added the review/needs-review No maintainer or bot review yet label Jul 23, 2026

@emdashbot emdashbot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the right change for #2210: layout prefetch was burning D1 reads on full taxonomy-count aggregates that most pages never render. The split between term-list caching (CacheNamespace.TAXONOMIES) and count caching (termCountNamespaces(collections)) is clean, includeCounts defaults to true so direct getTaxonomyTerms callers keep the old behavior, and the Tags/Categories widgets now only pay for counts when showCount is on. Query-count snapshots move in the right direction.

I checked:

  • The diff and the changed implementation in packages/core/src/taxonomies/index.ts.
  • Callers of getTaxonomyTerms and getTerm across core, templates, and tests.
  • The prefetch path and widget components.
  • The new regression test that asserts the aggregate query is skipped during prefetch and runs exactly once when counts are requested with no object-cache backend.
  • AGENTS.md conventions: SQL is still builder-only, no new locale filters are needed, query counts only decrease, the changeset is present, and comments explain non-obvious reasons rather than narrating the PR.

No blocking or correctness issues found. The separation of term-list and count caches is sound, and the public API contract for existing callers is preserved.

@MA2153 MA2153 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. If this has the chance of breaking existing flows, the changelog should begin with Breaking Change.

@github-actions

Copy link
Copy Markdown
Contributor

Overlapping PRs

This PR modifies files that are also changed by other open PRs:

This may cause merge conflicts or duplicated work. A maintainer will coordinate.

@github-actions github-actions Bot added review/approved Approved; no new commits since and removed review/needs-review No maintainer or bot review yet labels Jul 23, 2026
@khoinguyenpham04

Copy link
Copy Markdown
Collaborator

@logelog hi, does this PR includes AI generated code? If yes, could you edit the description please? Cheers!

@khoinguyenpham04 khoinguyenpham04 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing this @logelog ! Everything looks good but 1 tiny nit, so could we use a new/versioned cache key here, such as termList:? This avoids briefly returning legacy cached count values after an upgrade for taxonomies with no collections.

@github-actions github-actions Bot added review/needs-rereview Author pushed changes since the last review and removed review/approved Approved; no new commits since labels Jul 24, 2026
@khoinguyenpham04 khoinguyenpham04 added the bot:review Trigger an emdashbot code review on this PR label Jul 24, 2026

@emdashbot emdashbot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the right fix for #2210: the layout prefetch was running expensive aggregate queries to compute visible taxonomy term counts on every render, even though most pages never display them. The split between a term-list cache (CacheNamespace.TAXONOMIES) and a separate count cache (termCountNamespaces(collections)), plus the new includeCounts option defaulting to true, makes the optimization transparent to existing callers.

I re-read the diff and re-traced the caching/invalidation paths:

  • getTermList and getVisibleTermCounts are now independent: the prefetch warms only the term list, and widgets (Categories/Tags) request counts only when showCount is on.
  • withCounts rebuilds a fresh term tree instead of mutating the cached count-free list, so both callers safely share the same term lookup.
  • Object-cache invalidation is consistent: term-only data depends on the taxonomies namespace; count entries additionally depend on each collection's content namespace, which existing write paths already bump.
  • The new regression test asserts the aggregate is skipped during prefetch and runs exactly once when counts are requested without an object-cache backend.
  • Query-count snapshots decrease as expected, and the changeset/docs/template updates are in scope.

No prior emdashbot[bot] findings were left open. I didn't find any new blocking correctness issues or AGENTS.md violations. The public API contract is preserved, and the logged-out query-count trend is downward.

@emdashbot emdashbot Bot removed the bot:review Trigger an emdashbot code review on this PR label Jul 24, 2026
@github-actions github-actions Bot added review/approved Approved; no new commits since and removed review/needs-rereview Author pushed changes since the last review labels Jul 24, 2026

@khoinguyenpham04 khoinguyenpham04 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks @logelog and thanks @MA2153 for the review.

@khoinguyenpham04
khoinguyenpham04 merged commit f81aa68 into emdash-cms:main Jul 24, 2026
72 of 74 checks passed
@emdashbot emdashbot Bot mentioned this pull request Jul 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/core area/docs area/templates overlap query-count changed PR diff modifies query-count snapshot files review/approved Approved; no new commits since size/L

Projects

None yet

3 participants