Repository navigation
Conversation
Upgrade certifi from 2025.11.12 to 2026.10.19 and hf-xet from 1.2.0 to 1.7.0 in uv.lock. certifi 2026.10.19 removes "emSign ECC Root CA - G3"; hf-xet 1.2.0 compiles in a CA list that includes it, while 1.7.0 reads the operating-system CA store. The lockfile's own package version also moves from 1.58.2 to 1.59.1 to match pyproject.toml. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2 tasks
GitHub's API answered the comment POST in run 39685708223 with an error status and an empty body, which the GitHub CLI reports only as "unexpected end of JSON input", so the job failed although the comparison passed. The step now makes up to three attempts, 20 and 40 seconds apart. Each attempt searches for the existing comment again, so a retry after a write that GitHub applied but reported as failed updates that comment instead of posting a second one. pipefail makes a failed search fail the attempt; before, a failed search left the comment ID empty and the step posted a new comment. The PATCH and POST commands are unchanged. The search command's reviewed text in test_workflow_artifacts.py changes only by the indentation inside the new function and the added "|| return 1". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Calibration fit against the base build: pass
A drop of more than 2.0% fails. Notes
National targets in a worse error band on the head (9)
Flags are for review: builds of the same code move 2 to 12 national targets to a worse band (#598). Bands are of |estimate / target - 1|; no target's estimate or error is published. Income distribution and single ages against the base build: passHBAI FYE 2025 (DWP), period 2024: equivalised income in £ a week, rates and Gini in per cent. A value outside HBAI's 95% interval is marked (outside).
FlagsNone. These measures report and flag but do not fail yet: their same-code spread is unmeasured (#598). Aggregate impact of the data change: passBase dataset Checks: 0 broken by this change, 4 failing on both sides, 0 fixed. Moves flagged for review: 131 (thresholds in Notes
Moves flagged for review
91 more flags are in the report artifact. Receipts by tax head
Spending by benefit and service
Caseloads
Government balance, each component counted once
Model totals
Components the model counts twice
Household income totals
Population
Poverty rates
People in poverty
Inequality
Mean household net income in each side's own deciles
Standard reform costings (change in gov_balance)
Standard reforms: change in each reform's target
Household net income by decileThe datasets hold different households, so households are not matched; compare the mean income in each side's own deciles. Checks failing on both base and head (4)
Counted-once checkCounted-once check not run: not requested. Provenance
Base dataset: Head dataset: Receipts by tax head: head levels
Spending by benefit and service: head levels
Caseloads: head levels
Government balance, each component counted once: head levels
Model totals: head levels
Components the model counts twice: head levels
Household income totals: head levels
Population: head levels
Poverty rates: head levels
People in poverty: head levels
Inequality: head levels
Mean household net income in each side's own deciles: head levels
Standard reform costings (change in gov_balance): head levels
Standard reforms: change in each reform's target: head levels
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #634
Why
On 2026-10-20 it was disclosed that eight intermediate certificate authority certificates operated by eMudhra (emSign) have private keys that anyone can recover from the public certificates (incident site). Anyone can therefore issue a certificate for any domain that is accepted by clients trusting the root "emSign ECC Root CA - G3", which allows interception of HTTPS traffic on the network path. Browsers and current operating systems already block these certificates; software that ships its own CA list does not until it is updated.
What in this repository trusted the affected roots
uv.lockpinned two packages that carry their own CA list. CI installs exactly these versions (uv sync --frozen --all-extrasin the Pull request and Push workflows) and then downloads inputs from Hugging Face, uploads releases to Hugging Face and Google Cloud Storage, and runs the tests withHUGGING_FACE_TOKENand Google Cloud credentials loaded:requests,httpx(throughhuggingface_hub) and the Google Cloud client libraries. It includes "emSign ECC Root CA - G3". With certifi 2025.11.12,requestsreturned HTTP 200 from the test endpointhttps://dv-g3b-emudhra.we-have-a-problem.com/.huggingface_hubuses for Xet-backed file transfers. It compiles in its own CA list; the stringemSign ECC Root CA - G3is present in the Linux x86_64 wheel'shf_xet.abi3.so. It ignores certifi and the operating-system store.No other locked package carries a CA list (no
grpcio,httplib2or similar), and the code sets no custom CA bundle and does not turn off certificate verification. No Vercel project or container image deploys from this repository.Changes
uv.lock: certifi 2025.11.12 → 2026.10.19, which removes "emSign ECC Root CA - G3".uv.lock: hf-xet 1.2.0 → 1.7.0, which verifies certificates against the operating-system CA store (throughrustls-platform-verifier) and no longer contains the emSign root.uv.lock: the project's own version entry moves from 1.58.2 to 1.59.1 to matchpyproject.toml; this is whatuv lockwrites for the currentpyproject.tomland changes no installed package.changelog.d/emudhra-ca-trust-update.fixed.md: the changelog fragment required for a change touv.lock.Generated with
uvx uv@0.13.4 lock --upgrade-package certifi --upgrade-package hf-xet. No other package changed.Checks
uvx uv@0.13.4 lock --locked: passes on this branch (it failed onmainonly because of the stale project version entry).python .github/check_changelog_fragment.py HEAD^1 HEAD: passed.requests2.32.5 againsthttps://dv-g3b-emudhra.we-have-a-problem.com/: HTTP 200 with certifi 2025.11.12, certificate verification error with certifi 2026.10.19.https://pypi.org/andhttps://huggingface.co/returned HTTP 200 with both. The Sectigo cross-signed test endpoint (https://dv-g3b-aaa-emudhra.we-have-a-problem.com/) was rejected by both certifi versions.emSign ECC Root CA - G3; 1.7.0 does not and linksrustls-platform-verifier.huggingface_hub1.1.5 (the locked version) with hf-xet 1.7.0 downloaded a Xet-backed public file (hf-internal-testing/tiny-random-bert,pytorch_model.bin, 540,217 bytes) correctly.The full dataset build and test suite are left to CI.
After merge
Merging adds a
fixedfragment, so, as for any fragment, the Versioning workflow bumps the patch version and the Push workflow rebuilds, tests and uploads a data release using the updated lockfile. No manual step is needed. The PyPI package's dependency ranges are unchanged; users resolve their own certifi and hf-xet versions.Apart from the comment step below, this PR changes no workflow, container definition or other deployment configuration. That step still reads only
GH_TOKEN(the job'sGITHUB_TOKEN, withissues: writeandpull-requests: write) andPR(the pull request number from the event); it adds no environment variables.CI fix: the fit comparison's comment step
The first run's comparison passed, but its "Post or update the pull request comment" step failed with
unexpected end of JSON input: GitHub's API answered the comment POST with an error status and an empty body, which the GitHub CLI reports only as that JSON parse error. The step passed in other pull requests' runs before and after, and this run's comment (56,987 characters) was under GitHub's 65,536-character limit, so the failure came from the API, not from this PR's changes..github/workflows/pull_request.yaml: the step makes up to three attempts, 20 and 40 seconds apart. Each attempt searches for the existing comment again, so a retry after a write that GitHub applied but reported as failed updates that comment instead of posting a second one.set -o pipefailmakes a failed search fail the attempt instead of posting a new comment.policyengine_uk_data/tests/test_workflow_artifacts.py: the search command's reviewed text changes by its new indentation and|| return 1. The PATCH and POST commands are unchanged.The step's script, taken from the workflow file, was run with gh 2.102.0 (the runner's version) against a local server that imitates the comment endpoints. The old script reproduces the CI error. The new one recovers from a failed POST, a failed search and a POST that was applied but reported as failed (it then updates the comment rather than posting another), and exits 1 after three failures. The workflow tests pass and actionlint reports nothing.
CI: the base build's failed download (run 39825717667)
After
39b2448, "Build the base for comparison" failed in its "Download data inputs" step, and "Calibration fit against the base" failed because the base build had produced no fit summary, distribution report or dataset (theFileNotFoundErrorforbase/policyengine_uk_data/storage/enhanced_frs_2024_25.h5).huggingface_hub's file-metadata request (httpx.ConnectError: [Errno 104] Connection reset by peer, thenLocalEntryNotFoundError).huggingface_hub1.1.5, which this PR does not change, does not retry that request after a network error. It was not a certificate rejection, which would readCERTIFICATE_VERIFY_FAILED, and no hf-xet transfer had started.e1acdf9, this branch'suv.lock, the same job definition) in the first run on4f8d527, where it passed;39b2448changed only the comment step and its test. In the same run, the Test job downloaded the same files with the same lockfile.🤖 Generated with Claude Code