Implement updates tracking OWID grapher

.bundlemonrc.json

@@ -4,8 +4,8 @@
     "files": [
         {
             "path": "owid.mjs",
-            "maxSize": "820kB",
-            "maxPercentIncrease": 5
+            "maxSize": "785kB",
+            "maxPercentIncrease": 2
         },
         {
             "path": "owid.css",

.claude/settings.json

@@ -1,6 +1,26 @@
 {
     "permissions": {
-        "allow": ["Bash(grep:*)", "Bash(yarn typecheck)", "Bash(git log:*)"],
+        "allow": [
+            "Bash(git log:*)",
+            "Bash(grep:*)",
+            "Bash(yarn typecheck:*)",
+            "Bash(yarn testLintChanged:*)",
+            "Bash(yarn fixLintChanged:*)",
+            "Bash(yarn testFormatChanged:*)",
+            "Bash(yarn fixFormatChanged:*)",
+            "Bash(yarn query:*)",
+
+            "Bash(gh issue list:*)",
+            "Bash(gh issue view:*)",
+            "Bash(gh pr checks:*)",
+            "Bash(gh pr diff:*)",
+            "Bash(gh pr list:*)",
+            "Bash(gh pr view:*)",
+            "Bash(gh repo view:*)",
+            "Bash(gh run list:*)",
+            "Bash(gh run view:*)",
+            "Bash(gh search:*)"
+        ],
         "deny": []
     }
 }

.claude/skills/add-synonym/SKILL.md

@@ -0,0 +1,64 @@
+---
+name: add-synonym
+description: Add search synonyms to the Algolia synonym list. Use when someone wants to add, update, or extend synonym groups for site search.
+metadata:
+    internal: true
+---
+
+# Add Search Synonyms
+
+Add or update synonym entries in `site/search/synonymUtils.ts` so that our Algolia-powered site search returns better results.
+
+## Assumptions
+
+- The user is on a macOS (Apple) laptop.
+- The user may not be a developer — they may not have Node.js, Yarn, or even Homebrew installed. Guide them through any setup with clear explanations.
+
+## Steps
+
+1. **Start from a clean, up-to-date main branch and create a feature branch.** Run:
+    - `git checkout master` to switch to the main branch.
+    - `git pull` to get the latest changes.
+    - If either command fails (e.g. uncommitted changes), explain clearly what's happening and help resolve it. For uncommitted changes, `git stash` is usually the right fix.
+    - Create and checkout a new branch: `git checkout -b add-synonym-<username>-<date>` where `<username>` is the GitHub username and `<date>` is today's date as YYYYMMDD (e.g. `add-synonym-bastianherre-20260413`).
+
+2. Ask the user what synonyms they want to add. They can describe it in natural language (e.g. "add WHO as a synonym for World Health Organization" or "make 'EV' and 'electric vehicle' synonyms").
+
+3. Read `site/search/synonymUtils.ts` to see the current synonym groups and section comments.
+
+4. Determine which existing synonym group the new terms belong in:
+    - Find the group where the requested terms best fit and add the new terms to it.
+    - Never create a new synonym group — always add to an existing one.
+    - Preserve the existing formatting: each group is an array of strings, groups are separated by blank lines between sections.
+
+5. Edit the file. Use double quotes for all strings. Keep entries in the same style as surrounding code.
+
+6. Validate the change. Before running any commands, check that the environment is ready:
+    - Run `which yarn` to check if Yarn is available.
+    - If Yarn is not found, explain to the user: "I need to install a couple of tools to validate the changes. This is a one-time setup." Then:
+        - Check if Node.js is installed (`which node`). If not, install it via `brew install node` (install Homebrew first via its install script if `which brew` fails).
+        - Install Yarn via `corepack enable && corepack prepare yarn@stable --activate`. If corepack is not available, fall back to `npm install -g yarn`.
+        - Run `yarn install` to install project dependencies.
+    - If Yarn is found but `node_modules` doesn't exist, run `yarn install` first.
+    - Then run:
+        - `yarn fixFormatChanged > /dev/null 2>&1 && yarn typecheck` and fix any errors.
+        - `yarn testLintChanged` and fix any errors.
+        - `yarn test run --reporter dot site/search/synonymUtils.test.ts` and fix any errors.
+
+7. **Pause and ask the user to confirm** the changes look correct before proceeding. Show them what was added and wait for their go-ahead.
+
+8. Commit to the feature branch with a message in this exact format:
+    - Prefix: `✨🤖`
+    - Followed by a short description, e.g. `✨🤖 Add "WHO" synonym for "World Health Organization"`
+    - Add `Co-Authored-By: Claude <noreply@anthropic.com>` as a trailer.
+
+9. Push the branch and create a PR:
+    - Push with `git push -u origin HEAD`.
+    - Create a PR with `gh pr create --title "<same as commit message>" --body "" --reviewer edomt`.
+
+## Important notes
+
+- Each synonym group is an array of equivalent terms — all terms in a group are treated as interchangeable for search.
+- Country name synonyms are handled separately via the `countries` utility and should NOT be added to this file. If the user asks for a country synonym, explain this and point them to the country data instead — unless it's a geographic term not covered by the countries list (like "gaza", "palestine" which are already in the file).
+- The synonyms in this file are pushed to Algolia during deployment via `baker/algolia/configureAlgolia.ts`. No manual Algolia action is needed.
+- Common typos can be included as synonyms (see existing examples like "happyness", "c02").

.claude/skills/create-migration/SKILL.md

@@ -0,0 +1,20 @@
+---
+name: create-migration
+description: Create a new database migration file for the OWID MySQL database. Use when the user needs to create a database schema change or migration.
+metadata:
+    internal: true
+---
+
+# Create Database Migration
+
+Create a new database migration file for the OWID MySQL 8 database.
+
+## Steps
+
+1. Run `yarn createDbMigration db/migration/<NewMigrationName>` where `<NewMigrationName>` is a descriptive name for the migration
+2. The generated filename will contain a timestamp prefix, so scan the `db/migration/` directory to find the actual path of the new file
+3. Report the new file path to the user
+
+## Naming Guidelines
+
+Choose a descriptive name for the migration that clearly indicates what schema change is being made (e.g., `AddUserEmailIndex`, `CreateAuditLogTable`, `RemoveDeprecatedColumns`).

.clinerules/GENERAL-RULES.md

@@ -2,8 +2,8 @@
 
 - yarn typecheck: runs the typescript typechecker across all files
 - yarn testLintChanged: run eslint on changed files
-- yarn testPrettierChanged: run prettier on changed files
-- yarn fixPrettierChanged: attempt to fix prettier issues on changed files
+- yarn testFormatChanged: check formatting on changed files
+- yarn fixFormatChanged: fix formatting on changed files
 - yarn test run --reporter dot: run unit tests. Uses vitest, can take one or more test filenames to only run a subset.
 - make migrate: apply migrations
 - make dbtest: run database and api tests

.codex/rules/default.rules

@@ -0,0 +1,14 @@
+# Allow read-only OWID database queries.
+prefix_rule(
+    pattern = ["yarn", "query"],
+    decision = "allow",
+    justification = "OWID documents yarn query as the read-only database query helper for this repository.",
+    match = [
+        "yarn query SELECT 1",
+        "yarn query -s SELECT 1",
+    ],
+    not_match = [
+        "yarn createDbMigration",
+        "yarn test",
+    ],
+)

.codex/skills

@@ -0,0 +1 @@
+../.claude/skills

.dev.vars.example

@@ -1,6 +1,7 @@
 # env vars for local development of /functions/donation/donate.ts Cloudflare pages function
 # rename to .dev.vars and fill in the values from the homonymous "(dev)" keys in 1Password
 
+# Should be set to "production" in prod deployment. Defaults to "production" if unset.
 ENV=development
 
 # https://dashboard.stripe.com/test/apikeys

.devcontainer/Dockerfile

@@ -1,4 +1,4 @@
-ARG VARIANT=22
+ARG VARIANT=24
 FROM mcr.microsoft.com/vscode/devcontainers/javascript-node:dev-${VARIANT}
 
 COPY .tmux.conf /home/node/.tmux.conf

.devcontainer/devcontainer.json

@@ -11,7 +11,7 @@
     // },
 
     // Add the IDs of extensions you want installed when the container is created.
-    "extensions": ["dbaeumer.vscode-eslint", "esbenp.prettier-vscode"],
+    "extensions": ["dbaeumer.vscode-eslint", "oxc.oxc-vscode"],
 
     // Use 'forwardPorts' to make a list of ports inside the container available locally.
     // "forwardPorts": [3000, 27017],

.editorconfig

@@ -7,7 +7,10 @@ charset = utf-8
 indent_size = 4
 indent_style = space
 trim_trailing_whitespace = true
-insert_final_newline = true # somehow required for prettier to add a new line at the end of composer.json
+insert_final_newline = true
 
 [*.md]
 trim_trailing_whitespace = false
+
+[*.{yaml,yml}]
+indent_size = 2

.env.archive

@@ -1,5 +1,7 @@
 IS_ARCHIVE=true
 ARCHIVE_BASE_URL=https://archive.ourworldindata.org
+WIKIPEDIA_ARCHIVE_BASE_URL=https://wikipedia-archive.ourworldindata.org
 ENVIRONMENT=production
 BAKED_BASE_URL=
 GRAPHER_DYNAMIC_THUMBNAIL_URL=
+BESPOKE_BASE_URL=

.env.devcontainer

@@ -1,7 +1,6 @@
 TZ=utc
 
-# Essential settings
-SECRET_KEY='fejwiaof jewiafo jeioa fjieowajf isa fjidosajfgj'
+ENV=development
 
 GRAPHER_DB_NAME=grapher
 GRAPHER_DB_USER=grapher

.env.example-full

@@ -1,7 +1,7 @@
 TZ=utc
 
-# Essential settings
-SECRET_KEY='fejwiaof jewiafo jeioa fjieowajf isa fjidosajfgj'
+# Should be set to "production" in prod deployment. Defaults to "production" if unset.
+ENV=development
 
 # Instance configuration (for running multiple parallel dev environments)
 TMUX_SESSION_NAME=grapher  # optional; tmux session name to avoid conflicts
@@ -69,9 +69,12 @@ DATA_API_URL= # optional
 
 SLACK_BOT_OAUTH_TOKEN= # optional
 SLACK_DI_PITCHES_CHANNEL_ID= # optional; #data-insight-pitches channel id
+SLACK_ALGOLIA_INDEXING_CHANNEL_ID= # optional; channel for Algolia indexing failure reports
 
 # for remote dev on tailscale without localhost + port forwarding
 ADMIN_SERVER_HOST= # optional; e.g: YOURMACHINE.TAILXXX.ts.net (find this in the tailscale admin panel)
 ADMIN_BASE_URL= # optional; e.g: http://YOURMACHINE.TAILXXX.ts.net
 VITE_HOST= # optional; e.g: YOURMACHINE.TAILXXX.ts.net
 VITE_DEV_URL= # optional; e.g: http://YOURMACHINE.TAILXXX.ts.net:8090 (should match VITE_PORT if set)
+
+BESPOKE_BASE_URL= # optional; base URL for bespoke component assets, e.g. "http://localhost:8089" for local dev server

.env.example-grapher

@@ -1,7 +1,7 @@
 TZ=utc
 
-# Essential settings
-SECRET_KEY='fejwiaof jewiafo jeioa fjieowajf isa fjidosajfgj'
+# Should be set to "production" in prod deployment. Defaults to "production" if unset.
+ENV=development
 
 # Instance configuration (for running multiple parallel dev environments)
 TMUX_SESSION_NAME=grapher  # optional; tmux session name to avoid conflicts

.github/actions/setup-node-yarn-deps/action.yml

@@ -1,33 +1,38 @@
 name: Set up node and dependencies
 description: Runs all the setup steps required to have the proper Node version and all dependencies installed
 inputs:
-    runPostinstallScripts:
-        description: "Whether to run postinstall scripts during dependency installation. Usually only needed for native packages, like esbuild, sharp or wranglerd."
-        required: false
-        default: "true"
+  runPostinstallScripts:
+    description: "Whether to run postinstall scripts during dependency installation. Usually only needed for native packages, like esbuild, sharp or wranglerd."
+    required: false
+    default: "true"
+  cacheDependencyPath:
+    description: "Path to the yarn.lock file used for cache key generation."
+    required: false
+    default: "yarn.lock"
 runs:
-    using: composite
-    steps:
-        # Corepack must be enabled _before_ running setup-node, otherwise the caching setup will error
-        - name: Enable corepack
-          run: corepack enable
-          shell: bash
+  using: composite
+  steps:
+    # Corepack must be enabled _before_ running setup-node, otherwise the caching setup will error
+    - name: Enable corepack
+      run: corepack enable
+      shell: bash
 
-        - name: Use Node.js (.nvmrc)
-          uses: actions/setup-node@v6
-          with:
-              node-version-file: ".nvmrc"
-              cache: "yarn"
+    - name: Use Node.js (.nvmrc)
+      uses: actions/setup-node@v6
+      with:
+        node-version-file: ".nvmrc"
+        cache: "yarn"
+        cache-dependency-path: ${{ inputs.cacheDependencyPath }}
 
-        - name: List versions
-          run: |
-              echo "Versions:"
-              echo "Node `node --version`"
-              echo "yarn `yarn --version`"
-          shell: bash
+    - name: List versions
+      run: |
+        echo "Versions:"
+        echo "Node `node --version`"
+        echo "yarn `yarn --version`"
+      shell: bash
 
-        - name: Install dependencies
-          env:
-              YARN_ENABLE_HARDENED_MODE: ${{ (github.event.pull_request && github.event.pull_request.head.repo.full_name != 'owid/owid-grapher' && '1') || '0' }}
-          run: yarn --immutable ${{ inputs.runPostinstallScripts != 'true' && '--mode=skip-build' || '' }}
-          shell: bash
+    - name: Install dependencies
+      env:
+        YARN_ENABLE_HARDENED_MODE: ${{ (github.event.pull_request && github.event.pull_request.head.repo.full_name != 'owid/owid-grapher' && '1') || '0' }}
+      run: yarn --immutable ${{ inputs.runPostinstallScripts != 'true' && '--mode=skip-build' || '' }}
+      shell: bash

.github/pull_request_template.md

@@ -27,10 +27,9 @@ Step-by-step instructions on how to test this change
 
 If DB migrations exists:
 
-- [ ] If columns have been added/deleted, all necessary views were recreated
+- [ ] If columns have been added/deleted, all necessary views were recreated and ETL and Analytics team members have been informed of the incoming changes
 - [ ] The DB type definitions have been updated
 - [ ] The DB types in the ETL have been updated
-- [ ] If tables/views were added/removed, the Datasette export has been updated to take this into account
 - [ ] Update the documentation in db/docs
 
 ### After merging

.github/workflows/bespoke.yml

@@ -0,0 +1,46 @@
+name: Bespoke
+
+on:
+  pull_request:
+  push:
+    branches:
+      - master
+
+defaults:
+  run:
+    working-directory: bespoke
+
+jobs:
+  bespoke-typecheck:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Clone repository
+        uses: actions/checkout@v5
+
+      - uses: ./.github/actions/setup-node-yarn-deps
+        with:
+          runPostinstallScripts: false
+          cacheDependencyPath: bespoke/yarn.lock
+
+      - name: Install bespoke dependencies
+        run: yarn --immutable
+
+      - name: Typecheck
+        run: yarn typecheck
+
+  bespoke-build:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Clone repository
+        uses: actions/checkout@v5
+
+      - uses: ./.github/actions/setup-node-yarn-deps
+        with:
+          runPostinstallScripts: false
+          cacheDependencyPath: bespoke/yarn.lock
+
+      - name: Install bespoke dependencies
+        run: yarn --immutable
+
+      - name: Build
+        run: yarn build