[MaxGhenis]

Max, 2026-10-21: "also like even if we've already published something, we should do a fresh comp review before big pushes like newsletters". This is the promotion-time half of the pre-publish comparables gate (PolicyEngine/policyengine-scorecard#208, PolicyEngine/policyengine-app-v2#1380).

The rule

Every policyengine.org analysis an edition links must have, in PolicyEngine/policyengine-scorecard:

• a comparables search dated on the push date or in the scorecard's promotion_window_days before it (7 by default, counting both ends; the window is Max's call, queued);
• that compared the results files the analysis reads now (the search's results_pins equal the record's current_pins);
• and no gap found by then still pending (a pending gap with no found_on counts as found).

An analysis here is an apps.json app listed with research or tagged policy, or any post, under the rules the scorecard's index publishes (entry_rules). Only a post that is plainly not an analysis (tagged org, event, giving and so on, and none of the analysis tags) may skip the check by declaring no_estimate; policy wins over it, and an app never may. A page that has left the registry but still has a scorecard record is held to that record. The rule applies to pushes on or after 2026-10-26.

Where links lead

Read from policyengine-app-v2's route map (website/src/data/analysisRoutes.json, PolicyEngine/policyengine-app-v2#1380), built from the site's evaluated rewrites and redirects: every path, redirect (tracker, CliffWatch, OBBBA aliases, country-wide zone routes) and deployment host (Vercel, Modal, GitHub Pages narrowed by project path, a deployment shared by two countries narrowed by path) that leads to a research app. And from the scorecard's records: each analysis's page and deployments, so a dashboard taken off the site, like the childcare one, is held to its record on every link, its own Vercel URL included. Links are normalised first: any scheme or none, www., case, ports, HTML entities and percent-encoding, dot segments, a trailing dot on the host, trailing punctuation; posts by slug under any country, through /research/, /blog/ or a dated slug. Mailchimp tracking links and link shorteners are refused. Every source (registries, route map, index) is read at the commit its main branch points to, so a cached copy is never read.

Where it is enforced

• CI: the Fresh comparables for promoted analyses job checks the editions a change adds or modifies under editions/ (on a push, the pushed range; a new branch against the empty tree), each on its own date, so an edition already sent is not re-judged against later data. A changed edition must be named for its date, a social post outside editions/ fails, and a base commit the checkout cannot read fails rather than passing. It downloads its sources only when an edition is due; a failed download, or an index or route map of the wrong shape, fails the job.
• upload-newsletter, for new drafts and --campaign-id updates: refuses an undated file, judges the edition on the later of its date and today, and refuses a stale one before calling Mailchimp.
• /create-newsletter, /upload-draft, CLAUDE.md: the pre-send step, and CLAUDE.md's draft-update recipe now goes through upload-newsletter --campaign-id rather than the content endpoint.

The check confirms the search is fresh and every gap over the threshold has a written cause; it does not put the comparable in the edition.

The 20 October UK edition fails it, run with the cutover moved back, app-v2's current registries and scorecard#208's index:

FAIL 2026-10-20-uk.html
  2026-10-20-uk.html: policyengine.org/uk/middle-east-war-living-standards has no comparables record in the scorecard; run the comparables review (analysis-uk-middle-east-war-living-standards) before sending
  2026-10-20-uk.html: policyengine.org/uk/childcare-100k-threshold: no comparables search between 2026-10-13 and 2026-10-20 (last: none); run a prepromotion search before sending

The Scottish Budget social post (2026-01-27-uk-scottish-budget-social.md), which links without a scheme, fails too: its dashboard has no record.

Tests (tests/test_comparables.py, 76; 106 in all)

• Every way of writing a link finds its entry (Hypothesis, 400 examples): 18 targets (paths, redirects, tracker and country-wide routes, dot segments) × 5 schemes × 5 host forms × 16 trailing forms × 3 encodings.
• Deployment links: Vercel, Modal, GitHub Pages and previews find their app; a shared host leads only to the analyses under the linked path; an analysis off the registry is held to its record through its page and its deployment.
• Which entries are checked: untagged costings, policy over no_estimate, apps never exempt, untagged posts under us, posts under any country, the record a post's block names.
• Freshness: agrees with the scorecard's fresh on its 96 shared cases; an edition passes exactly when its analysis is fresh (Hypothesis); old results pins fail.
• The index and route map: 24 index malformations and 5 route-map malformations each fail with a message, never a pass or a traceback (empty country list, unknown statuses, missing claim, bad found_on, non-string pins, missing page url).
• Sources at main's commit: one API call per repository, the token sent when set, a bad answer fails.
• Dates and running, and the upload command: as before.
• Black, ruff and pytest are clean on Python 3.8 and 3.11; CI runs 3.8 to 3.11.

Order

This merges after scorecard#208, whose index carries the window, pins, entry rules and deployments, and after app-v2#1380, which publishes the route map.

🤖 Generated with Claude Code