Repository navigation
Conversation
anth-volk
force-pushed
the
v2plan/1b-105-simulations-and-reports
branch
from
October 26, 2026 12:21
e9c7d2e to
9bf0d80
Compare
2 tasks done
anth-volk
force-pushed
the
v2plan/1b-105-simulations-and-reports
branch
from
October 27, 2026 14:38
9bf0d80 to
6282776
Compare
This was referenced Oct 27, 2026
Draft
A simulation is one policy applied to one population: a household, or an economy named by a region code (state/ca) with an optional dataset name, where a null dataset name means the region's default dataset in whichever catalog version a run uses. A null policy means current law. The row holds no catalog version, so the same simulation can run again on a newer one; each run resolves the region and dataset in its own version (item 1.06). Revision 0002 creates the table with: - the population checked against the kind (a household simulation has a household and no region or dataset; an economy simulation has a region and no household); - foreign keys into the Public API's tables in the same database, all ON DELETE RESTRICT: (policy_id, country) to policies (id, country_id), (household_id, country) to households (id, country_id), and dynamic_id to dynamics, with an index on each referencing column; - a unique legacy_simulation_id for simulations migrated from the Public API's v1 database, and a unique (id, country, kind) for the reports' foreign keys; - no UPDATE privilege for the runtime role, since definitions never change. Creating the foreign keys needs the operator's REFERENCES grant on the Public API tables; a test shows the chain fails without it and leaves nothing applied. SimulationStore creates, reads and finds simulations by v1 ID, and create_migrated is idempotent by legacy_simulation_id for the one-time migration (item 2.05). Importing the package now registers every table, so the migration environment always compares the whole schema. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A report is a definition: its baseline simulation, its reform simulation (required for an economy report; a household report may have only a baseline), the year (1900 to 2200) and its options as a JSON object. Composite foreign keys (simulation id, country, kind) into simulation.simulations make both simulations share the report's country and kind; they are ON DELETE RESTRICT, so a simulation a report uses cannot be deleted. A report migrated from the Public API's v1 database keeps its v1 report ID in the unique legacy_report_id. Revision 0002 now creates the reports table too, and revokes UPDATE on both definition tables from the runtime role. ReportStore creates, reads and finds reports by v1 ID; create_migrated is idempotent by legacy_report_id, for the one-time migration (item 2.05). NewReport checks the country, year, options and v1 ID before any write; the database checks them again for any other writer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
NewSimulation and NewReport typed kind as an enum but neither checked nor converted it, so a kind given as plain text (as a mapping of v1 records easily does) passed their checks and then failed with AttributeError when the store read kind.value. Both now convert the kind to its enum member when they are created, and reject an unknown kind with a ValueError that names the field. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The database repeats each check NewSimulation and NewReport make, for any other writer, but only the report year and options checks had a test, so a mistake in another check's expression would have gone unnoticed. New tests insert rows directly as the runtime role and expect each of the other checks to refuse them: the kinds, countries, region code, dataset name, both v1 IDs, and an economy report without a reform. The README's roles table still gave the runtime role UPDATE on every table, and did not say the migrator needs REFERENCES on the Public API tables before revision 0002 deploys; both are corrected. The unused REFERENCED_PUBLIC_API_TABLES constant is gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Revision 0002 passed full check constraint names, such as ck_simulations_kind, which the naming convention prefixed a second time, so the database held ck_simulations_ck_simulations_kind while the table definitions name it ck_simulations_kind. alembic check does not compare check constraints, so nothing noticed; a later revision dropping one by its defined name would have failed. The revision now marks the names as final (op.f), and a test compares every constraint name in the migrated database with the names the table definitions give. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
anth-volk
force-pushed
the
v2plan/1b-105-simulations-and-reports
branch
from
October 27, 2026 21:16
6282776 to
c7404a1
Compare
This was referenced Oct 28, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #899
Same-Repository Draft PR Check
PolicyEngine/policyengine-sim-api, not a personal fork.Summary
Second of four stacked PRs; it builds on #898. Revision
0002adds the two definition tables of thesimulationschema. After it deploys, both exist in staging and production with no rows; nothing writes them until stage 2.simulation.simulations: kind (economyorhousehold), country, policy (null for current law), dynamic, and either a household or a region code (state/ca) with an optional dataset name (null means the region's default dataset in each catalog version). No column refers to a catalog version, so the same simulation can run again on a newer one; each run resolves the region and dataset in its own version (next PR). A check requires the population to match the kind.simulation.reports: baseline and reform simulations of the report's own country and kind (composite foreign keys on simulation, country and kind), the year (1900 to 2200) and options (a JSON object). An economy report needs a reform; a household report may have only a baseline.(policy_id, country)topolicies (id, country_id),(household_id, country)tohouseholds (id, country_id)anddynamic_idtodynamics, allON DELETE RESTRICT, each with an index. Creating them needsREFERENCESon those tables for the migrator role; a test shows the migration fails and applies nothing without it. The runtime role needs no privilege on them.legacy_simulation_idandlegacy_report_id(plain columns, since the Public API's v1 database is separate).SimulationStore.create_migratedandReportStore.create_migratedreturn the existing row when the same v1 ID is migrated again.NewSimulationandNewReportcheck every field before a write (a kind given as text becomes its enum), and the database checks the same fields again for any other writer; a test inserts directly to reach each database check.Environment variables
This PR reads no new environment variable. The migration job from #898 applies revision
0002withSIMULATION_MIGRATION_DATABASE_URLas described there.Waits for
REFERENCESonpolicies,householdsanddynamics(and the catalog tables used by the next PR) in staging and production. Without the grant, the migration job fails and the deploy stops.simulationchain merge one at a time.Testing
Rebased with #898 onto
mainat 2a9212f on 2026-10-28, without conflicts in this branch. Every commit was checked again after the rebase: persistence tests on PostgreSQL 17 with database tests required (100 passed at the tip), the contract library's tests, the entrypoint suite (797 passed), the rules guard and quality guards, Ruff,uv lock --checkand Pyright. One review round found no bugs; its quality findings (the README's roles table, untested database checks, a kind given as text) are fixed here. A later review of the PR above this one found that revision0002gave its check constraints doubled names (ck_simulations_ck_simulations_kind), whichalembic checkdoes not compare; the last commit fixes the names and adds a test comparing every constraint name in the migrated database with the table definitions. The checks above were run locally; this push's GitHub Actions runs report their own results.🤖 Generated with Claude Code