Render cached package facts in initial HTML by pastelsky · Pull Request #1007 · pastelsky/bundlephobia

pastelsky · 2026-07-19T08:39:52Z

Summary

render cached package facts and package-specific metadata in the initial package-page HTML
use one shared PackageRequest from HTTP/page parsing through package resolution and size-cache lookup
keep PackageSizeService focused on cache lookup, build-result composition, cancellation, and cache writes
represent size-cache work as a typed cache-hit | cache-miss result; SSR cannot enqueue builds
use one PackageCacheMode enum everywhere: cache-first, force-rebuild, or cache-only
retain public force query compatibility across all package APIs while normalizing it immediately to PackageCacheMode.ForceRebuild
build every TypeScript package API URL through one shared encoder and endpoint contract
represent absent package descriptions, repositories, categories, error fields, and exact requested versions explicitly instead of using empty-string sentinels
keep build-on-miss policy visible in the Koa middleware chain

Information flow

API or page package input
  -> createPackageRequest(packageString, cacheMode)
  -> PackageRequest { identity + cacheMode }
  -> resolvePackageRequest / PackageSizeService.findPackageSize
  -> ResolvedPackage { exact version + nullable metadata }
  -> PackageSizeCacheResult { cache-hit | cache-miss }

/api/size cache miss
  -> blacklist / failure cache / build-miss rate limit
  -> PackageSizeService.buildPackageSize

package page SSR cache miss/error
  -> stable client fallback (never a server-side build)

Koa middleware owns HTTP behavior: blacklist checks, failure caching, rate limits, request priority, error formatting, response headers, and recent-search recording. Services own reusable package resolution and package-size operations. /api/size now has one package-size middleware for resolution plus pre-build cache behavior; /api/exports and /api/exports-sizes consume the same PackageRequest and ResolvedPackage rather than reparsing query state.

Before / after examples

For /package/react@18.2.0:

	Before	After
Initial HTML	Loading shell and generic metadata	Cached description, sizes, dependencies, and package metadata
Browser hydration	Browser requests `/api/size` on every visit	Cache-hit SSR supplies the initial result; no duplicate size request
Exact cache hit	Client-only flow	API and SSR share the same cache-first service and avoid npm
Cache miss	Browser enters the build flow	SSR returns a stable fallback; the browser still enters the guarded API build flow

For package request and optional-data code:

Before	After
`requestedPackage`, `packageRequestPolicy`, `resolved`, and `packageSizeLookup` moved independently through Koa state	One `PackageRequest` and one `resolvedPackage`; the typed cache result remains local to `packageSizeMiddleware`
`forceBuild -> cachePolicy -> bypass/read` translations	One `PackageCacheMode` enum passed unchanged through every layer
`peep` and `peek` had endpoint-specific meanings	`cache=cache-only`; every cache miss is HTTP 404
`force=true` was interpreted independently by each endpoint	`force` remains supported on size, exports, and export-size APIs, but the request boundary maps it once to `PackageCacheMode.ForceRebuild`; new callers use `cache=force-rebuild`
Browser, MCP, historical jobs, population scripts, rebuild scripts, and tests hand-built package URLs	All TypeScript callers use `createPackageApiPath`, which owns endpoint names, encoding, cache modes, recording, and limits
Missing npm metadata became `""`	Missing and whitespace-only metadata becomes `null` once, then remains nullable across shared server/client domain types
`version ?? ''` was passed to semver in three middlewares	`getExactRequestedVersion` returns `string
Optional error message/details and cache-log type defaulted to empty values	Error messages are required; optional details and log fields are omitted when absent
Empty category/route strings lived in component state	Category and route absence is explicit; renderers omit absent descriptions and error bodies
Similar-package classification fetched repository READMEs but never used them when scoring	Removed that dead network path; scoring uses only the description, keywords, and package name it actually consumes
Separate package-size lookup, blacklist, and cached-response middleware stages	One `packageSizeMiddleware` owns the complete pre-build size flow; generic `jsonCacheResponseMiddleware` remains only for koa-cash export-size entries
`PackageSizeService` dynamically imported and retained its build adapter	The build middleware statically composes the adapter and passes it into the build operation; the SSR import graph never reaches `BuildService`
Middleware intent had to be inferred from implementation	Every affected middleware has a two-line input/output contract
Interaction-heavy tests asserted private call counts	Tests assert public outcomes: cached availability during npm failure, force-rebuild behavior, nullable metadata, build results, and cancellation

Cache behavior

cache=cache-first (default): return a cache hit, otherwise continue to the guarded build flow.
cache=force-rebuild: bypass cache reads, rebuild, return Cache-Control: max-age=0, and replace the persistent result.
cache=cache-only: return a cache hit or HTTP 404; never build.
exact-version cache hits avoid npm resolution.
BuildService is statically imported only at the build middleware composition boundary, so SSR cache lookup cannot initialize build infrastructure.
rendered document cache durations remain in server/config.ts, beside API cache durations.

Verification

TypeScript: tsc --noEmit
Jest: 18 package-resolution, package-size, package-API contract, queue, and utility tests pass
production Next build passes, including lint and type validation
playwright-cli live page flow:
- /package/react@18.2.0 settles to the package-specific title and renders package description, sizes, dependency count, repository, composition, and exports state
- a cached reload starts with the package-specific title and reports no browser console errors
live Koa checks:
- invalid cache mode returns HTTP 400 with the accepted values
- the removed cache=refresh value returns HTTP 400 with the accepted contract
- exact-version cache=cache-first returns HTTP 200 with Cache-Control: max-age=86400
- versionless cache=cache-first returns HTTP 200 with Cache-Control: max-age=30
- cache=force-rebuild returns HTTP 200 with Cache-Control: max-age=0
- cache=cache-only returns HTTP 404 on a local persistent-cache miss
- legacy force=true returns HTTP 200 with Cache-Control: max-age=0 on size, exports, and export-size APIs
- combining force and cache returns HTTP 400 instead of applying ambiguous precedence
the legacy live fixture suite reaches the running API: four error-behavior cases pass; three old exact-stat/TTL assertions are stale against the current builder output and versionless-request cache policy and are intentionally unchanged here
Node 24 CI

pastelsky · 2026-07-19T10:17:27Z

+
+  context.res.setHeader(
+    'Cache-Control',
+    'public, s-maxage=300, stale-while-revalidate=86400'


Seems odd that this is here - is this not duplication with existing headers and code?

This header is for the rendered Next.js document, not the size API response. The package page is handled by the catch-all Koa route, so the API cache-control middleware never runs for it; without setting this in getServerSideProps, Next serves the SSR document with its default non-shared cache behavior. I added an inline comment making that boundary explicit. The 5-minute fresh / 1-day stale window only caches cache-derived public package HTML.

pastelsky · 2026-07-19T10:47:33Z

+  // run for Next.js page responses handled by the catch-all Koa route.
+  context.res.setHeader(
+    'Cache-Control',
+    'public, s-maxage=300, stale-while-revalidate=86400'


Can we keep this cache in some existing config / new config

Moved both SSR document cache durations into the existing config.CACHE object as PACKAGE_PAGE_SHARED and PACKAGE_PAGE_STALE. getServerSideProps now only constructs the header from those configured values.

pastelsky · 2026-07-19T10:49:02Z

file too small can be inlined?

Removed this standalone file. The request-to-package transforms now live in packageResolution.service.ts alongside the normalized resolved-package transform used by both size and exports workflows. This deletes the tiny module without duplicating the parsing logic in SSR and Koa.

pastelsky

Review notes addressed in follow-up commit 3934389.

pastelsky · 2026-07-19T11:08:34Z

+    expect(build).not.toHaveBeenCalled()
+  })
+
+  it('returns an explicit cache miss without invoking the builder', async () => {


Seems ike implementation detail. remove very coupled test cases

Addressed in 3934389. I removed the interaction-heavy miss, builder call-count, and cache-write assertions. The remaining tests exercise public outcomes: an exact cached result remains available when npm fails, refresh returns a miss instead of cached data, build output preserves nullable metadata, and abort propagates to cancellation.

pastelsky · 2026-07-19T11:10:12Z

+      scoped: requested.scoped,
+      packageString: `${requested.name}@${version}`,
+      description: '',
+      repository: '',


Why empty? Is there a better way to represent nullability or union?

Addressed in 3934389. Error logging now uses a requested-package or resolved-package union. Requested context no longer fabricates description, repository, or a latest version; resolved metadata is explicitly nullable.

Follow-up in 5815af2 removes the remaining empty-string sentinels across this workflow, not only the original error context. Error messages are now required, optional details/log fields are omitted, package metadata and UI category/route state use explicit nullability, renderers omit absent content, and exact-version detection returns string | null instead of calling semver with version ?? "". Whitespace-only npm metadata is normalized to null and covered by a focused test. I also audited every file changed by this PR; no ?? "", || "", empty default, or empty-return sentinel remains.

pastelsky · 2026-07-19T11:11:05Z

+  ctx.state.packageRequestPolicy = {
+    forceBuild: ctx.query.force !== undefined,
+    cacheOnly: ctx.query.peep !== undefined,
+    peekOnly: ctx.query.peek !== undefined,


peep and peek sound similar. are they different enough? can we normalize names to reduce terminology fragmentation throughout the pr and rest of the code.

Addressed in 3934389 without compatibility aliases. The HTTP contract is now one validated cache parameter with prefer, refresh, or only. All internal layers carry the same cacheMode, cache-only misses consistently return 404, and repository callers now use cache=refresh.

Follow-up in 79c83cb makes the values outcome-oriented: cache-first, force-rebuild, and cache-only. The same constant tuple now drives validation and typing, so these names cannot diverge again.

pastelsky · 2026-07-19T11:11:21Z

+  const startedAt = now()
+  const cachePolicy = ctx.state.packageRequestPolicy.forceBuild
+    ? 'bypass'
+    : 'read'


Terminology fragmentation again.

Addressed in 3934389. The forceBuild to cachePolicy to bypass/read translation is gone. PackageRequest owns one cacheMode, the service reads it directly, PackageSizeLookup became PackageSizeCacheResult, and the middleware is now packageSizeCacheMiddleware.

Second pass in 79c83cb also consolidated callers. Browser API methods, MCP proxy calls, historical jobs, bulk population, rebuild comparison, and integration tests now use createPackageApiPath for endpoint names, package encoding, cache mode, record, and limit parameters. The only literal package URLs left are the shell stress fixture using default behavior.

pastelsky · 2026-07-19T11:11:32Z

+import { debug, logger } from '../../init'
+import { packageSizeService } from '../../services/packageSize.service'
+
+const packageSizeLookupMiddleware: Middleware = async (ctx, next) => {


Add 2 line description for all middlewares

Addressed in 3934389. Every affected middleware now has a two-line contract describing what it consumes and what it publishes or controls: request, resolution, size cache, blacklist, cached response, build, errors, exports, and export sizes.

pastelsky · 2026-07-19T11:12:12Z

+      typeof repository === 'string' ? repository : repository.url ?? ''
+    return gitURLParse(rawRepository).toString('https')
+  } catch {
+    return ''


See if using better nullablikity helps

Addressed in 3934389. Repository normalization and description truncation now return string or null. PackageMetadata uses the same nullable shape across server and client, cache/build results preserve null, and UI components choose their own display fallback only when rendering.

pastelsky · 2026-07-19T12:21:16Z

-
-  if (force === 'true') {
-    void cache.setPackageSize({ name, version }, body)
-  }


Are we sure this removal is fine and handled in new api?

Handled in a6a9037. The removed force-only cache write is now superseded by the single awaited cache write in PackageSizeService.buildPackageSize for every successful composed result. That path is used by both PackageCacheMode.ForceRebuild and the legacy force query adapter, so force rebuilds still replace the persistent package result without duplicating cache ownership in middleware.

pastelsky · 2026-07-19T12:23:01Z

we've added a bunch of new middlewares along with older middlewares. is it possible to combine some?

Handled in a6a9037. packageSizeCacheMiddleware is deleted. The size-specific lookup, package policy check, successful/failure cache response, cache-only stop, and miss continuation now live in one packageSizeMiddleware, reducing the /api/size chain to request -> error -> package size -> build rate limit -> build. The older generic cachedResponse middleware was narrowed and renamed to jsonCacheResponseMiddleware because it now exists only for the koa-cash export-size path.

pastelsky · 2026-07-19T12:24:01Z

  })
 }

-async function getReadme(


Is this dead code? Or did we delete by mistake?

Confirmed dead rather than accidentally deleted. getPackageDetails previously fetched or constructed readme and returned it, but getCategory destructured only description and keywords, so README content never entered tokenString or getScore. I kept the removal and added an explanatory comment at the fetch boundary; scoring behavior is unchanged while the unused GitHub, GitLab, or Bitbucket request is gone.

pastelsky · 2026-07-19T12:24:22Z

  ctx.cacheControl = {
    maxAge:
-      force != null
+      cacheMode === 'force-rebuild'


Use enums for this everywhere instead of strings.

Handled in a6a9037. PackageCacheMode is now a string enum and every internal comparison, service default, script caller, rebuild caller, and test uses enum members. Object.values(PackageCacheMode) drives boundary validation, leaving the wire values unchanged.

pastelsky · 2026-07-19T12:24:57Z

    `BUILD EXPORTS SIZES: ${packageString} built in ${time.toFixed()}s`
  )

-  if (force === 'true') {


As far as main service is concerned. please maintain backward compat for just force param for all apis

Handled in a6a9037. packageRequestMiddleware now retains force as the one legacy query adapter for /api/size, /api/exports, and /api/exports-sizes, immediately mapping its presence to PackageCacheMode.ForceRebuild. New callers continue to use cache=force-rebuild. Supplying force and cache together returns 400 to avoid ambiguous precedence. Live checks returned 200 and Cache-Control max-age=0 for all three force endpoints.

pastelsky · 2026-07-19T12:26:00Z

+
+  private async getPackageSizeBuilder(): Promise<PackageSizeBuilder> {
+    if (!this.packageSizeBuilder) {
+      const { buildService } = await import('../api/BuildService')


No inline imports

Handled in a6a9037. The inline dynamic import and mutable builder field are removed. build.middleware.ts statically imports BuildService, composes the PackageSizeBuilder adapter at the HTTP build boundary, and passes it into buildPackageSize. ResultPage SSR imports only PackageSizeService, so its cache-only import graph still cannot initialize BuildService.

pastelsky · 2026-07-19T12:26:47Z

add some explainatory comments in this file (in code) when logic is getting multi-stage

Handled in a6a9037. PackageSizeService now explains the three non-obvious stages inline: exact-version cache lookup before npm resolution, tag or range resolution followed by the immutable-version cache lookup, and the single post-build persistence path that replaces the old middleware force write.

pastelsky · 2026-07-19T13:38:39Z

-      priority
-    )
+    body = await packageSizeService.buildPackageSize(resolvedPackage, {
+      builder: packageSizeBuilder,


This reads like overengineering. rename or simplify.

Replace the ad-hoc per-route middleware wiring for /api/size, /api/exports, and /api/exports-sizes with a single linear pipeline: parse -> error boundary -> blocklist guard -> resolve+serve cache -> (miss only) rate limit -> build+store Each endpoint is now a declarative descriptor (which cache, what to build, whether misses are rate limited) rather than its own chain of middleware, so resolution, caching, and response no longer have their logic spread across several files. New server/pipeline/ module owns the flow; the old resolve / cache-serve / build / exports / exports-sizes / koa-cash middlewares and the package-size service are removed. exports-sizes moves onto the same domain cache as size (dropping koa-cash's 304 support), and the blocklist guard now runs before resolution. Also fold in the earlier cleanup pass: shared cache-mode policy and TTL helpers, a single ResolvedPackage constructor, shared formatSentence, and a cache-only SSR lookup that never blocks on npm. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The descriptor + factory + type-parameterized pipeline was more machinery than three endpoints warrant. Spell the three routes out explicitly in index.ts and give each stage a concrete middleware that reads top to bottom: size: request -> error -> blocklist -> resolve+serve cache -> rate limit -> build exports: request -> error -> blocklist -> resolve -> build exports-sizes: request -> error -> blocklist -> resolve -> serve cache -> rate limit -> build Only small leaf helpers stay shared (cachePolicy, sizeCacheMaxAge, logCache, serveFailureCache) in results/packageCache.ts. Drops the generic PackageResultCache, resolveCachedPackage, packageApiPipeline, PackageEndpoint descriptors, buildAndStore/resolveAndServeCached factories, and the runCancellableBuild helper (client-disconnect cancellation is inlined). The size fast path (serve an exact cached version before hitting npm) lives in a concrete, testable lookupPackageSize; SSR uses readCachedPackageSize. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

This was referenced Jul 19, 2026

Add curated npm package discovery pages #1008

Open

Improve mobile LCP on package result pages #1010

Open

pastelsky commented Jul 19, 2026

View reviewed changes

Comment thread docs/pr-evidence/seo-cache-first-ssr/cache-hit-flow-clean.webm Outdated

pastelsky commented Jul 19, 2026

View reviewed changes

Comment thread docs/pr-evidence/seo-cache-first-ssr/README.md Outdated

pastelsky commented Jul 19, 2026

View reviewed changes

Comment thread server/seo/packageFacts.ts Outdated

pastelsky added 7 commits July 19, 2026 15:40

Render cached package facts in initial HTML

01162bc

Add cache-first SSR browser evidence

0ee1e79

Keep treemap hydration deterministic

aaf12c2

Remove PR evidence assets

f407d48

Reuse cache-only size API during SSR

084407a

Share package analysis across SSR and API

e9c6cbd

Test package analysis cancellation

43873b7

pastelsky force-pushed the codex/seo-cache-first-ssr branch from 690c6af to 43873b7 Compare July 19, 2026 10:14

pastelsky commented Jul 19, 2026

View reviewed changes

pastelsky added 2 commits July 19, 2026 16:09

Clarify package size lookup flow

ca4f5d3

Normalize package request policy

909738f

pastelsky commented Jul 19, 2026

View reviewed changes

pastelsky added 2 commits July 19, 2026 16:33

Refine package workflow boundaries

0bc35f1

Unify package request and cache vocabulary

3934389

pastelsky commented Jul 19, 2026

View reviewed changes

pastelsky added 2 commits July 19, 2026 17:23

Clarify cache modes across package callers

79c83cb

Model optional package data explicitly

5815af2

pastelsky commented Jul 19, 2026

View reviewed changes

Consolidate package cache middleware

a6a9037

pastelsky commented Jul 19, 2026

View reviewed changes

pastelsky force-pushed the codex/seo-cache-first-ssr branch from 1b2696f to c65fbdc Compare July 19, 2026 19:01

Uh oh!

Conversation

pastelsky commented Jul 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Information flow

Before / after examples

Cache behavior

Verification

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pastelsky left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pastelsky commented Jul 19, 2026 •

edited

Loading