Skip to content

R wrapper and dashboard refresh: report the TAXSIM build and record provenance - #1414

Draft
MaxGhenis wants to merge 2 commits into
mainfrom
r-dashboard-provenance
Draft

MaxGhenis wants to merge 2 commits into
mainfrom
r-dashboard-provenance

Conversation

@MaxGhenis

Copy link
Copy Markdown
Contributor

Stacked on #1412. The first commit is #1412's (--version, --provenance, policyengine_taxsim/core/provenance.py); review the second commit. I'll rebase onto main once #1412 merges.

R wrapper

  • policyengine_versions() calls collect_versions() and prints format_version_report(), the same report as policyengine-taxsim --version:
    policyengine-taxsim: 3.1.0
    policyengine-us:     2.35.2
    policyengine-core:   3.32.29
    TAXSIM binary:       cd2026081819 (taxsimtest-osx.exe)
    Python:              3.11.15
    
    (real output from this branch). The returned list keeps policyengine_taxsim, policyengine_us and policyengine_core first, with the same "unknown" for a missing package, and adds taxsim_binary, taxsim_binary_build, taxsim_binary_path, taxsim_binary_sha256, python_version and platform (NA when unavailable). With a policyengine-taxsim installed before Add --version and an opt-in --provenance sidecar to the CLI #1412 it falls back to importlib.metadata and says to run setup_policyengine(force = TRUE).
  • policyengine_calculate_taxes(..., provenance = "run.json") and compare_with_taxsim(..., provenance = "compare.json") write the record build_provenance() builds for the CLI, plus rPackageVersion and rVersion. The path is checked (and parent folders created) before the run. Input and output are R data frames, so each is hashed as the CSV pandas writes for it (to_csv(index=False, lineterminator="\n")). policyengine_calculate_taxes() calls PolicyEngineRunner directly, so its record shows engines: {policyengine: n, taxsim: 0}; compare_with_taxsim() records the TAXSIM binary its TaxsimRunner ran.
  • policyengine_calculate_taxes() now imports the runners with convert = FALSE so it has the pandas results to hash; it converts them with py_to_r() as before.
  • Fix: DESCRIPTION now requires reticulate >= 1.45.0, the first release that supports pandas 3 (reticulate NEWS). On main, with reticulate 1.44.1 and the pandas 3.0.6 that setup_policyengine() installs, policyengine_calculate_taxes() fails with "cannot coerce class 'pandas.DataFrame' to a data.frame" (reproduced locally); with 1.47.0 it works.
  • R package version 0.1.0 → 0.2.0, so rPackageVersion distinguishes this code.
  • New tests/testthat suite (11 tests, 68 expectations): unit tests need no Python; integration tests run against the policyengine-taxsim virtualenv. New CI job r-package creates that virtualenv from this checkout and runs the suite with POLICYENGINETAXSIM_REQUIRE_PYTHON=true, so a missing environment fails instead of skipping.
  • R package README, r-package/README.md, root README "Recording versions", and the docs page's R tab (r-version-pin block) updated.

Dashboard refresh

provenance_{year}.json (and summary.metadata, which shares the dict) gains policyengineTaxsimVersion and taxsimBinaryBuild, from collect_versions(taxsim_path=ROOT / "resources/taxsimtest/taxsimtest-linux.exe"). Existing keys and values are unchanged.

collect_versions() runs in a child process (refresh_dashboard.py --versions PATH, like --worker), before the batches so a broken environment fails in seconds. Importing anything under policyengine_taxsim imports policyengine-us (~26 s; verified), and the coordinator is meant not to. A local 52-household smoke run wrote:

"policyengineTaxsimVersion": "3.1.0", ..., "taxsimBinaryBuild": "cd2026081819",
"taxsimBinarySha256": "00a321d2…a89b"

Invariants and how they are tested

  • Every key the refresh wrote before keeps its value, for any collect_versions() output (Hypothesis property test over versions/builds, plus example tests).
  • The refresh coordinator loads no policyengine* module while collecting versions (fresh-interpreter test), and the child reports this checkout's __version__ and the Linux binary's embedded build.
  • Differential: taxsimBinarySha256 from the refresh's digest() equals collect_versions()'s SHA-256 of the same binary; the R record's input hash equals an independent hashlib hash of the same CSV.
  • policyengine_calculate_taxes() returns the same results with and without provenance (integration test). Locally, main's version and this branch's returned identical results for 4 households (2021–2024; single, joint and separate filers) with basic and detailed output.
  • The R record has exactly the CLI record's fields plus rPackageVersion and rVersion (checked locally against policyengine-taxsim policyengine --provenance).
  • The policyengine_versions() list's first three names and their "unknown" convention are unchanged.

Tests run

  • pytest tests/test_dashboard_refresh.py: 15 passed (a bare env without hypothesis or policyengine-us runs 14 with the child-process test skipped, as the refresh workflow's unittest step does).
  • R testthat::test_local(): 11 tests, 68 expectations, 0 failed or skipped, with Python required.
  • Dashboard DocumentationContent tests: 13 passed; the new R-tab test fails on the old block.
  • ruff format --check . clean.

axiom: n/a: R wrapper, docs and dashboard-refresh tooling; no policy change.

🤖 Generated with Claude Code

MaxGhenis and others added 2 commits October 9, 2026 07:54
Federal statistical users asked how to reproduce a run. A TAXSIM-format
output file records no versions, and PyPI does not keep every
policyengine-us release, so a run has to record its versions when it runs.

- `policyengine-taxsim --version` prints the policyengine-taxsim,
  policyengine-us and policyengine-core versions, the build stamp of the
  TAXSIM binary TaxsimRunner would run (read from the file without running
  it) and the Python version.
- `--provenance PATH` on the stdin command and the policyengine, taxsim and
  compare subcommands writes a JSON sidecar: those versions, the binary's
  path and SHA-256, the options (as resolved), SHA-256, path and record
  count of the input and output, per-engine record counts, and every
  installed package version. stdout and output files are byte-identical
  with or without the flag; in stdin mode a pass-through for sys.stdout
  hashes every byte the run writes. The sidecar directory is created and
  probed for writability before the run.
- TAXSIM progress and --logs diagnostics go to stderr, so a stdin run writes
  only TAXSIM-format output to stdout (pre-2021 records used to put three
  progress lines above the CSV header).
- `taxsim --taxsim-path` works: the path is made absolute (it crashed on the
  str click passes), and an already-executable binary is not chmodded.
- README "Recording versions" section and a note in the documentation
  page's Version pinning section.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
policyengine_versions() now prints collect_versions() through
format_version_report() (the `policyengine-taxsim --version` report, with
the bundled TAXSIM binary's build and Python version) and returns the
existing three fields followed by taxsim_binary, taxsim_binary_build,
taxsim_binary_path, taxsim_binary_sha256, python_version and platform. With
a policyengine-taxsim that predates the provenance module it falls back to
importlib.metadata and says how to update.

policyengine_calculate_taxes() and compare_with_taxsim() take a provenance
path, checked before the run, that writes build_provenance() JSON (the
CLI's --provenance record) plus rPackageVersion and rVersion. Inputs and
outputs are hashed as the CSV pandas writes for them. The R package needs
reticulate 1.45.0+, the first release that converts pandas 3 data frames;
main's policyengine_calculate_taxes() fails on reticulate 1.44.1 with the
pandas 3 that setup_policyengine() installs. Adds testthat tests and a CI
job that runs them against a real policyengine-taxsim environment.

scripts/refresh_dashboard.py adds policyengineTaxsimVersion and
taxsimBinaryBuild to provenance_{year}.json and the summary metadata, from
collect_versions(taxsim_path=resources/taxsimtest/taxsimtest-linux.exe) run
in a child process before the batches. Existing keys are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
policyengine-taxsim Ready Ready Preview Oct 9, 2026 12:18pm UTC

Request Review

@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Coordination note from #1418 (agreement with TAXSIM by release): no changes are needed here, but two things now depend on this PR's key names.

If any of these keys is renamed before merge, scripts/alignment_history.py in #1418 needs the same rename.

@MaxGhenis

Copy link
Copy Markdown
Contributor Author

#1422 changes policyengine_calculate_taxes() to call StitchedRunner (pre-2021 records go to TAXSIM-35, like the CLI), adds engine = c("auto", "policyengine"), and rewrites setup_policyengine(). It was built from main, not on this branch, so it can land without #1412. Your r-package CI job and tests/testthat.R are copied into it verbatim and the DESCRIPTION Suggests lines match, so those merge cleanly. A trial merge (git merge-tree, #1422 at 56c8beb with this branch at dfdfc66) conflicts only in R/calculate_taxes.R.

When this branch is rebased after #1422 lands:

  1. @param block and signature: keep both arguments, engine first:
                                              show_progress = TRUE,
                                              engine = c("auto", "policyengine"),
                                              provenance = NULL) {
    
      engine <- match.arg(engine)
  2. engines in the record: list(policyengine = nrow(input_df), taxsim = 0L) is no longer true for pre-2021 rows. Use .engine_counts(input_df$year, .pe_min_year(), engine) from R: compute pre-2021 records with TAXSIM-35 like the command line, and install a pinned release #1422, or runner$engine_counts() from Add --version and an opt-in --provenance sidecar to the CLI #1412 (same definition: year >= pe_min_year). Recording engine in options would also help.
  3. scorp_treatment: runner$scorp_treatment exists on PolicyEngineRunner only; StitchedRunner keeps it in _pe_kwargs. reticulate::import("policyengine_taxsim.core.scorp")$validate_scorp_treatment(NULL) returns the value StitchedRunner resolved for a call without the option.
  4. .make_runner() in R: compute pre-2021 records with TAXSIM-35 like the command line, and install a pinned release #1422 calls reticulate::py_has_attr(runners, "StitchedRunner"). That works on the convert = FALSE module this branch imports.
  5. Stale text: the Details paragraph ("this function computes every record with PolicyEngine, so the file's engines field shows 0 TAXSIM records") and the README sentence "(policyengine_calculate_taxes() computes every record with PolicyEngine)".
  6. Version: both PRs move DESCRIPTION 0.1.0 → 0.2.0. If this lands second, 0.3.0 keeps rPackageVersion telling the two apart.

Your test of engines uses 2023 records only, so it holds either way.

This branch was successfully deployed

1 active deployment
Preview — dfdfc665 Deployed Oct 9, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant