Implement updates tracking OWID grapher#6
Open
xrendan wants to merge 1197 commits into
Open
Conversation
🐛 (search) don't fallback to default entities as peers
When searching in the chart editor's "Origin url" field, typing a space now matches a dash in option values, so "child labor" surfaces "/child-labor". Also normalizes the highlighted substring so the bolding stays in sync. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The onChange handler was running .trim() on every keystroke and writing the trimmed value back to the controlled input, which made trailing spaces disappear the instant they were typed — so you couldn't get from "child" to "child labor". Store the raw value and only check .trim() to decide whether the field is effectively empty. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Spaces are allowed while typing as a search convenience (so "child labor" finds "/child-labor" in the dropdown), but the saved value must never contain spaces. On blur, replace any lingering spaces with dashes. Also hide the "↵ Enter to use custom URL" hint while the query contains spaces — a URL with spaces isn't a valid custom URL to commit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Keep all topic pages cards on the right when articles are present. Fixes #5681
The file is broken and shouldn't be used by anything anyway. It's needed for migrations managed by Knex, but we use TypeORM instead for those.
Simpler model: instead of accepting spaces, filtering through them, and normalizing on blur, just replace each space with "-" in onChange. The input never holds a space, the stored value never encodes as %20, and the filter/highlight/custom-URL-hint logic all collapse back to the original form. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Deduplicate GDoc slugs before triggering Buildkite lightning builds and before validating targeted bakes. Add a regression test for duplicate lightning deploy slugs.
✨ reverse order of entities in the No data section
✨ use specific chart label tabs
- Show fallback image only when JS is disabled - Otherwise, render Grapher, which has its own loading indicator This should improve the UX especially on data pages, where the image previously flashed for a brief moment before the Grapher started loading.
Keep inactive key insight slides measurable instead of hiding them with `display: none`, preventing cached interactive charts from briefly rendering titles at zero width when switching slides.
They run them too often, prolonging the feedback loop.
🔨 clarify split between interactive/static captioned chart
🐛 Admin origin URL autocomplete: treat spaces as dashes
Nested DoDs should still work. Fixes #5917
- Use react-query to fetch the multi-dim config - Defer loading the config until the embed scrolls near the viewport, similar to `GrapherWithFallback` - Show a fallback image when JS is disabled - Log multi-dim analytics event only when the multi-dim becomes visible, similar to `Grapher` - Introduce a single site-wide `QueryClient `and a `SiteQueryClientProvider` Fixes #6353
MultiEmbedder is no longer used for multi-dims.
Match the convention used elsewhere in site search (SearchDataResults, SearchWritingResults): on initial fetch render placeholder cards that mirror the LatestHit layout instead of a centered "Loading…" string. Reuses the existing .animate-pulse keyframes from search/skeletons.scss. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Collapses the (type, kicker) pair on PagesChronological records into one flat latestType attribute with five values (article, data-insight, data-update, website-upgrade, announcement). The raw OwidGdocType stays on the record so the dynamic atom feed keeps filtering on type as before. - Adds LatestType + LATEST_TYPE_VALUES in the shared types package. - Derives latestType at index time; logs unrecognized announcement kickers via logErrorAndMaybeCaptureInSentry with a safe fallback to "announcement". - Swaps kicker for latestType in attributesForFaceting. - Simplifies queryLatestPages: filters is always the static base filter; user-selected type and topics each live in their own facetFilters group, dropped per-query to drive disabling counts. - Drops LatestFilter tagged union, encodeFilter, the kicker: URL prefix, and the dual TYPE_META/KICKER_META lookup in favor of a single LATEST_TYPE_META map keyed by latestType. URL changes from ?type=kicker:data-update to ?type=data-update. Requires a PagesChronological reindex on deploy. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Header & mobile nav: rename "Insights" → "Latest", point at /latest - Homepage hero: new "Articles" pill linking to /latest?type=article, backed by a server-side article count - "See all Data Insights" homepage button: → /latest?type=data-insight - Footer "Insights": → /latest?type=data-insight (kept alongside "Latest") - Featured Data Insights, small-screen carousel, and DI permalink page "see all" + breadcrumb: → /latest?type=data-insight - Homepage DI carousel cards: link to per-DI permalink instead of scroll-anchoring on the soon-to-be-deleted /data-insights index - Centralize /latest URL construction in latestUrl() helper (renamed latestFilters.ts → latestUrl.ts) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The paginated /data-insights index is now redundant — every entry point that used to land there now lands on /latest?type=data-insight (commit 237a063). Delete the page, its baking pipeline, sitemap entries, dev preview branch, and orphaned constants/helpers. Add a permanent 301 redirect (bare + wildcard) so external links and search engines land in the right place. Individual /data-insights/{slug} permalinks, the atom feed, and the entire admin UI are unaffected. Existing slug-rename redirects in the DB still take precedence over the new wildcard. Also redirect /latest/page/N (retired SSR pagination, no SPA analog) and collapse /blog/* onto /latest — the existing /blog/* /latest/:splat rule was 404-ing for every input, since /latest has never had a /:slug route. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adopt the searchParamsToState / stateToSearchParams / urlNeedsSanitization pattern from site/search/searchState.ts. Unknown params (e.g. legacy ?topic=Health from old /data-insights links), invalid topic names, and unrecognized type values are now stripped from the URL on first paint via a replaceState. Filter mutations build URLs from validated state instead of copying URLSearchParams from the previous URL, so unknown params can't sneak back in. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Introduce LatestFeedGdoc (subset of ChronologicalGdoc shown on /latest) derived structurally from LATEST_FEED_TYPE_VALUES, plus a checkIsLatestFeedGdoc predicate. Tighten ChronologicalGdoc so content.type is non-optional, eliminating one ! assertion in the indexer. - Split deriveLatestType into a dispatcher that takes LatestFeedGdoc and a deriveAnnouncementLatestType(kicker) helper. The two announcement-only call sites (standalone preview, homepage card) call the helper directly, so the dispatcher is only used by the indexer (gated behind the new predicate). Dispatcher uses ts-pattern shape-match — no `as` casts. - Subset enforcement: LATEST_FEED_TYPE_VALUES `satisfies` the chronological tuple at compile time. LATEST_BASE_FILTER (Algolia query) and LATEST_FEED_TYPES (Set) both derive from it, single source of truth. - Narrow getHomepageAnnouncements return type to OwidGdocMinimalAnnouncementInterface (the SQL already filters type='announcement'); propagate through OwidGdocHomepageMetadata and HomepageIntro. - Rename checkIsChronologicalFeedPost → checkIsChronologicalGdoc and checkIsLatestFeedPost → checkIsLatestFeedGdoc to match the type names and the codebase's checkIsX convention. - Rename typeFacetFilter → latestTypeFacetFilter in queries.ts (it filters on the latestType attribute, not type — LATEST_BASE_FILTER is the type filter). - Fix three misleading comments in pagesChronological.ts: the topic-page branches and isChronologicalGdocInstance gate were claimed to be unreachable / skipping, but topic pages are in fact chronologically indexed (for the atom feed) and reach those branches at runtime — the noop is the actual behavior. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds site/latest/README.md describing the architectural patterns of the new /latest SPA: records carrying their own AttachmentsContext, the two indexing paths (bulk + individual), URL as filter-state source of truth, the batched facet-counting search, the latestType derived field, and the preview-surface role of standalone announcement pages. Mirrors the shape of site/search/README.md and notes the deliberate parallel between the two surfaces. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirror the /search analytics pattern on /latest. Two new GA4 events are added to the registry: SiteLatest fires on filter state change, and SiteLatestResultClick fires when a card link is clicked. A new useLatestAnalytics hook debounces the filter event (skipping default state and consecutive duplicates), and a LatestContext exposes the SiteAnalytics instance to hit components. Announcement cards intentionally don't track clicks — they have no canonical link to their own slug, mirroring how links inside the rich excerpt of an article hit are also untracked. LatestState moves into @ourworldindata/types alongside SearchState. GTM registration of owid.site_latest, owid.site_latest_result_click, latestTopics, and latestType is required before GA4 will collect the new params.
Drop the redundant "Site" prefix to match the dominant naming pattern in SiteAnalytics (logSearch, logInstantSearchClick, logSearchAutocompleteClick, and the new logLatestResultClick). The EventCategory.SiteSearchResultClick GA4 event name is unchanged — the "Site" prefix there is a deliberate namespace.
Point the SiteResources "Latest" link at the data-insight feed and rename the SiteFooter "Insights" entry to "Data Insights" for consistency. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add `parseLatestExcerpt` next to `parseSimpleText` in rawToEnriched. Per block: `parseText` for raw text blocks; a synthetic EnrichedBlockText with a parseError for any non-text block. Surface those parseErrors in `GdocPost._validateSubclass`, mirroring the existing `parsedFaqs.parseErrors` walk. Errors show up in the admin via `gdoc.errors` → `getErrors()`, gating Publish on Error-severity entries.
Rename `getAndLoadPublishedDataInsightsPage` to `getAndLoadLastPublishedDataInsights` and hardcode the limit. The atom feed and `GdocDataInsight.getPublishedDataInsights` only ever needed the most recent insights, never a page offset. Removes the now-unused `LATEST_INDEX_PAGE_SIZE` constant and its export.
Drop a few comments that restate what the code already shows, and shorten the `isChronologicalGdocInstance` doc to a single sentence. Add a brief cross-reference from `checkIsChronologicalGdoc` to its class-narrowing counterpart.
The /atom.xml ?type= param now accepts LatestType slugs (article, data-insight, data-update, website-upgrade, announcement) and filters Algolia on the indexed latestType facet — matching the /latest UI and letting subscribers narrow to announcement subtypes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Match the ~-separated set encoding used by /search and /latest so the atom feed's ?topics= and ?type= params stay coherent across surfaces. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace the hand-rolled content type dropdown with react-aria-components' Select / ListBox / Popover. The trigger's active styling is now driven by `data-pressed` (set while the popover is open) instead of by the current selection.
Use react-aria ToggleButtonGroup for topic pills (keyboard nav, focus
ring) and Select for the type dropdown. Give the "All" pill and "All"
type option specific aria-labels ("All topics" / "All types") while
keeping the visible "All" text. Reset the negative right offset on the
scroll arrow under sm-only since the type dropdown isn't a sibling on
mobile.
LatestTopicFacets isn't memoized and the handlers aren't used in any deps array, so useCallback was purely ceremony.
Bumps the newsletter email input to 16px at the sm breakpoint, matching the existing sm-only convention used on the not-found page form. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…-with-algolia-spa-search 🎉🤖 Revamp Latest page with Algolia-powered SPA search
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
10 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
Links to issues, Figma, Slack, and a technical introduction to the work.
Screenshots / Videos / Diagrams
Add if relevant, i.e. might not be necessary when there are no UI changes.
Testing guidance
Step-by-step instructions on how to test this change
Reminder to annotate the PR diff with design notes, alternatives you considered, and any other helpful context.
Checklist
(delete all that do not apply)
Before merging
If DB migrations exists:
After merging