A donor-transaction panel, and what it prices about prediction intervals

Artifact for the manuscript. Everything measured in the paper is produced by a probe in probes/, committed as a transcript in outputs/, and turned into a macro or a table by the build. No measured number is typed into the manuscript, and the build fails if one is.

The manuscript source, the numbers.tex it reads, the venue style file and the thirteen-check gate that binds them stay with the manuscript and are not redistributed here. What this repository carries is the half a reader can run without them.

pip install -r probe-requirements.txt
python probes/delivered_coverage.py | diff - outputs/probe_output_delivered_coverage.txt

Every probe answers to that pattern: it prints its transcript on standard output and asserts its own claims on the way. Gate check 9 below is that loop over all thirty, and this is how to run one arm of it by hand. In the manuscript tree the two entry points are ./build.sh, which regenerates and gates and typesets and fails rather than producing a PDF, and python verify_numbers.py --submission for the gate alone.

What is here

donorpanel/	the generator, released separately as a package. Calibrated to one published sector report, with the retention probability each outcome was drawn from as a field
probes/	one file per measurement, each printing its own transcript and asserting its own claims. Every assertion names an input that fails it; several assert the manuscript's own findings, so a result that reverses reddens the build instead of quietly changing the paper
outputs/	the committed transcripts. Check 9 re-runs every probe and compares byte for byte
tables/	generated tables and figures, with hand-audited fixtures in tables/fixtures/ and the reading recorded in AUDIT.md
giftmix.py	the two-component lognormal fit, shared by the validation probe and the misspecification arm
DATASHEET.md	the Gebru et al. question set answered for this generator, with every answer pointing at the file it was read off
parked probes	eight probes and transcripts the manuscript does not cite: the seven ltt_*.py of a finding scoped out to a later paper, and vendor_census.py, whose census the manuscript withdrew rather than reported. They stay because check 9 re-runs them and check 11 refuses a transcript no probe claims, so a scoped-out arm cannot rot on disk. Nothing in the paper reads them
data/	gitignored. Third-party research data is fetched, never committed. probes/real_data_anchor.py prints the URL and SHA-256 of what it read

The thirteen gates, in order

./build.sh regenerates numbers.tex from the committed transcripts before the gate runs, so the gate compares the manuscript against freshly parsed fields rather than against whatever was on disk.

1. Every macro the prose uses is defined.
2. numbers.tex regenerates byte-identically.
3. No macro or table cell is derived by arithmetic on a value parsed out of a transcript. A macro is a parsed field or a count of parsed rows. If the prose needs a ratio, the probe prints the ratio. This check exists because a builder here once divided two already-rounded fields and shipped 4.8 where the probe printed 4.9, and check 2 agreed with it because it re-derived through the same division.
4. Every generated table and figure regenerates byte-identically.
5. Every generated table and figure matches its hand-audited fixture.
6. Every figure in numbers.tex appears verbatim in some committed transcript.
7. No hand-typed decimal in the prose.
8. The manuscript claims no more than the source record permits, and every source it calls verified has its page recorded.
9. Every probe reproduces its committed transcript. This is most of the wall clock; the gate prints per-probe timings and the four slowest.
10. No unresolved submission blocker, no defined-but-unused macro, and no [[TODO-AUTHOR]] placeholder. Three fields were the author's and not the build's to guess --- the postal affiliation, the funding-and-competing-interest declaration and the artifact repository URL --- and each sat behind a marker that typesets, so none could reach a submission unnoticed. All three are filled. This check also fetches, because an identifier that typesets is not an identifier that resolves: the artifact URL, the DOI beside it, and the published tree's agreement with this one, file by file. Resolution and currency are separate properties, and the second is the one that catches a probe added here and not published. An unreachable network reports SKIPPED rather than OK, and --submission refuses on a skip.
11. Every committed transcript belongs to a probe the gate runs. It caught seven orphans once: the LTT arm, scoped out of the manuscript and left in place with nothing checking them, so the directory held transcripts no source was required to still produce. It also caught the eighth, added in this round --- donorpanel/tasks.py holds the interval-score definition both comparative probes call, and its assertions used to sit in a __main__ nothing on any build path invoked. A check nothing runs is the same defect class as an assertion that cannot fail.
12. A second parser agrees with every generated cell. audit_tables.py imports nothing from the extractor and reads each cell off the transcripts by splitting on whitespace and indexing by position, where the extractor uses named regex groups.
13. Every headline claim is worded the same way wherever it is stated, and no phrasing this manuscript has retracted is back in it. This gate reads no number. The twelve above guard decimals, and an abstract sentence that outruns the body carries no wrong decimal: one asserted the universal that every profile reachable with two ranks is reachable with one, which the section it summarises refutes with its own table. Check 13 pins each claim to a wording per region, so editing one region reddens the build until the others are looked at.

Three of the thirteen are independent of the builder they check: 5, 6 and 12. Nine re-derive through the parser they are checking and would agree with a broken one. Check 13 checks no builder and no number; it checks the prose against itself.

🛑 Never accept a fixture to make check 5 green. Checks 2 and 4 run the same parser they are checking. Check 5 and check 12 are the independent oracles, and accepting a fixture unread destroys the first of them. When a fixture legitimately changes, re-derive the changed rows from the transcript, record the reading in tables/fixtures/AUDIT.md, then copy.

🛑 An assertion that cannot fail is worse than no assertion. Every probe assertion here has been run against an input that fails it, and the fixture audit records which.

Two conventions that look like inconsistencies and are not

Float and rational arithmetic disagree on purpose. Feasibility columns are computed the way the libraries compute them, in floating point, because that is what a caller meets. Every line stating the mathematics is computed in rationals. They differ exactly at a floor attainable with equality, and the manuscript reports one such cell.

Figure coordinates are parsed strings wrapped in TeX arithmetic. The scaling from dollars to centimetres happens in the generated source, where a fixture audit can read the probe's own digits off the page. Probes print widths in thousands of dollars because the graphics layer cannot hold a coordinate above sixteen thousand, and a figure that silently saturated would be a figure that lied.

Third-party material

None is redistributed here. Report pages, vendor pages and the real-data archive are recorded by URL, read date, content hash and, where the archive accepted them, a third-party snapshot. See donorpanel/evidence.py for the reasoning and LICENSE for what that means for reuse.

One file is a deliberate exception, and it travels with the manuscript rather than with this repository. A LaTeX source that cannot be typeset by whoever holds it is not a reproducible artifact, so dmlr2e.sty is vendored unmodified beside paper3.tex. It is published by the journal for exactly this purpose, and both arXiv and the venue expect a submission to carry it. Its record is kept here because it is the one third-party file this project hands on at all.

file	dmlr2e.sty
source	https://github.com/JmlrOrg/dmlr-style-file
commit	2f4c3dc023ecb66d12f653a75554b6269256a825, 2023-10-11
SHA-256	e4915bac65956be30152abd62b6f5f355134a26c47304c30d317dbe1a81b74ff
modified	no

Two things follow from using it and are recorded here because they moved the tree. Its \textwidth is 6.0in where the previous geometry preamble gave 6.5in, so tab_flag and the since-withdrawn tab_vendor went to \footnotesize at 2pt column separation; the fixture change was audited as a diff and tables/fixtures/AUDIT.md records that it touched two layout lines and no data row. And \@maketitle prints Reviewed on OpenReview: unconditionally, so \openreview is defined in the preamble and reads under review until the forum link exists.

What the gate needs that is not here

Checks 1, 2, 3, 6, 7, 10 and 13 read the manuscript, and checks 4, 5 and 12 need the two builders that write tables/ and the second parser that re-reads them. None of those files is in this repository, so the gate is not runnable from it; the tables and their hand-audited fixtures ship as the record of what the audit read, not as something to regenerate. Check 9 is the arm that is runnable here, one probe at a time, by the command at the top of this file.

build.sh also runs one check that reads files outside the manuscript directory and cannot be vendored in. It fails loudly rather than skipping, and says so.