Skip to content

Compare our federal results and line logic against PolicyEngine as a second opinion #30

Closed

Description

@captaincole

In order to catch mistakes in our federal math on returns we have not checked by hand, we can compare our engine against PolicyEngine-US in two ways: compare the numbers it produces, and compare the logic it uses for each federal line.

PolicyEngine-US is an open-source tax and benefits calculator written in Python. Policy researchers use it to model federal and state tax. It is independent of us and of the IRS forms, so when it agrees with our engine to the dollar that is strong evidence, and when it disagrees one of the two has a bug worth finding.

Part 1: compare the numbers

The idea comes from opentax-engine (Invaro). They generate 572 simple returns by combining wages, filing status, children, interest, capital gains and itemized deductions, run both engines, and sort the differences into buckets. Every difference has to be explained with an IRS citation or it fails the build.

Example outcome:

Scenarios run:                 480
Agree to the dollar:           452   (94.2%)
Within $1 (rounding):            9
Explained, with citation:       19
Unexplained:                     0   ← anything here fails the test

Explained differences
- Child tax credit threshold for a qualifying surviving spouse: we use $200,000,
  PolicyEngine uses $400,000. IRS Schedule 8812 instructions say $200,000. (19 scenarios)

Part 2: compare the logic, line by line

PolicyEngine implements each federal line as a small, readable Python formula with its own source file, for example the formula for the taxable amount of social security benefits or for the child tax credit phase-out. We can put our binding for the same line next to their formula and read the two together. This finds bugs the number comparison cannot reach: cases neither of us generated, thresholds that only matter at high income, and places where we both happen to be wrong the same way.

Example outcome: a short review note per form we support, in the form's folder:

Schedule D, line 16 vs PolicyEngine `capital_gains` / `net_capital_gain`
- Same: short-term and long-term netting order, $3,000 loss cap ($1,500 MFS).
- Different: PolicyEngine carries the unused loss forward itself; we expect it as a fact. OK for us.
- Found: we did not treat collectibles gain at 28%. Fixed in #NN.

Two notes. Invaro found PolicyEngine wrong twice, so a disagreement is a lead, not a verdict. And this is federal only. PolicyEngine's California coverage is thinner than ours.

Refs

Activity

  1. added
    enhancementNew feature or request
    ideaIdea gathered from other projects; not scheduled
    on Oct 25, 2026
  2. changed the title [-]Engine: differential test of federal lines against PolicyEngine-US[/-] [+]Compare our federal results against PolicyEngine as a second opinion[/+] on Oct 25, 2026
  3. changed the title [-]Compare our federal results against PolicyEngine as a second opinion[/-] [+]Compare our federal results and line logic against PolicyEngine as a second opinion[/+] on Oct 25, 2026
  4. captaincole commented on Nov 1, 2026

    @captaincole
    OwnerAuthor

    Part 2, first pass: line logic vs PolicyEngine, form by form

    Setup: a sparse checkout of policyengine-us 2.88.9 (commit 10c7f5a), its variables/gov/irs formulas and the parameters/gov/irs values in force for tax year 2025, read next to each form's bindings.ts. Federal only. Three forms have no PolicyEngine counterpart and were skipped: Form 8949 (per-lot listing), Schedule B (sums only) and Schedule H (household employment). Every lead was then given a second, skeptical pass: is it right against the 2025 IRS text in the corpus and our code, and can a wrong number actually print today? A disagreement with PolicyEngine is a lead, not a verdict; the second pass decides.

    What agrees: every 2025 constant both sides touch. Rate schedule brackets and the tax table, standard deduction and the 65+ amounts, the 0/15/20% capital gain ceilings and the QDCG worksheet stacking, CTC $2,200 / $1,700 / phase-out thresholds, HSA limits, IRA and student loan phase-outs, the SALT cap and phase-down (except MFS, below), Additional Medicare and NIIT thresholds, QBI thresholds and phase-in range, and the 2025 AMT exemption and phase-out numbers.

    Leads on our side

    # Form, line What Second pass Fix
    1 1040 line 7a / Schedule D line 21 A net capital loss flows to AGI in full; the $3,000 ($1,500 MFS) cap is unsupported. Found from the 1040 side and the Schedule D side. Confirmed against code and the Schedule D instructions. Next engine PR
    2 Schedule A line 5e MFS phase-down halves the cap first and then takes the full 30%; the worksheet reduces $40,000, floors at $10,000, then halves. Line 7 of the worksheet is also unrounded. Confirmed; recomputed MFS, MAGI $300,000, line 5d $25,000: worksheet $12,500, ours $5,000. The unrounded line is reachable for a single filer (MAGI $500,001 prints $39,999.70). Later: MFS is out of scope. Round when the file is next touched.
    3 Form 8959 line 6 With no filing status, line 5 is blank, line 6 reads it as 0 and line 7 taxes every Medicare dollar. Reproduced (marcus without a status: $2,192, not $392). But a missing filing status already puts the whole return in pending, so no value reaches the UI or the documents today. A latent trap, not a live bug. Later: line 6 should stay blank when line 5 is blank.
    4 Form 8960 line 12 No zero floor; a net capital loss above interest plus dividends gives a negative tax on line 17. Confirmed. The floor is printed on the form itself ("If zero or less, enter -0-"); our catalog label dropped that clause. Reachable with a correct $3,000 cap too: $1,000 interest and a capped loss give line 17 = −$76. Next engine PR
    5 Schedule 1 scoping Only 1099-MISC box 3 turns Schedule 1 on, so an HSA filer without one loses 8889 line 13 and line 16. Confirmed. Cade's decision table has no must_file_schedule_1, so the fallback is the normal path. Correction to the first note: a missing line 13 makes AGI too high; a missing line 8f makes it too low. Next engine PR
    6 Schedule D line 13 When Schedule D is filed, 1099-DIV box 2a goes nowhere: line 13 is a stub and the 1040 line 7a exception turns off. Confirmed; silent, since a stubbed line never becomes a pending input. No scenario has both forms yet, so no golden is wrong. Next engine PR, with #1
    7 Form 8995-A line 34 Without Schedule D, net capital gain is 1040 line 7a; we add it only when Schedule D lines 15 and 16 exist. Confirmed. The deduction stays right for the target user (line 36 never binds); lines 34–36 on the printed form are wrong. Later
    8 Form 8889 line 3 The last-month rule is not modeled; the 55+ catch-up for an unmarried filer belongs on line 3. The last-month rule is an election with a testing period (income plus a 10% tax if coverage lapses), so it belongs behind a decision fact; proration is the right default. The catch-up gap is real and small. Later
    9 Form 8959 scoping The must-file fallback uses $200,000 for every status. Partly. "Who Must File" makes any single W-2 over $200,000 a must-file for every status, so the fallback is a real rule. The gap is MFS with wages between $125,000 and $200,000. Later: MFS is out of scope

    Items 3, 4, 5 and 6 share one shape: a blank or unsupported line is read as zero by the next line, so the error is silent. That is the pattern to fix once, not four times.

    On PolicyEngine's side

    Next

    • One engine PR for items 1, 4, 5 and 6, each with a scenario that reaches it, through the tax-correctness review.
    • Items 2, 3, 7, 8 and 9 wait for scope or for the next touch of their file.
    • Part 1 (running the two engines on the same facts and diffing numbers) is still open.

    Notes by form follow, one comment each.

  5. captaincole commented on Nov 1, 2026

    @captaincole
    OwnerAuthor

    Form 1040 vs PolicyEngine adjusted_gross_income, taxable_income, income_tax_main_rates, capital_gains_tax, income_tax_capped_non_refundable_credits, ctc, income_tax

    Lines 1–16

    • Same: standard deduction ($15,750 / $31,500 / $23,625) and the $2,000 / $1,600 extra per 65+ box; MFS with an itemizing spouse → $0; the Social Security worksheet ($25k/$32k bases, $9k/$12k, 50%/85%, MFS-lived-together 85% rule); the 0% and 15% capital-gain ceilings for every status; every 2025 rate-schedule bracket and subtraction amount (recomputed from their brackets); no Pease limit and no non-itemizer charity deduction in 2025.
    • Different: lines 1a–5b, 8, 10 and 13a are facts or pass-throughs for us, computed by PolicyEngine. Under $100k we use the IRS tax table and round each worksheet line; PolicyEngine taxes exact amounts. Both fine. PolicyEngine models all four Schedule 1-A deductions (senior $6,000 with a 6% phase-out over $75k/$150k; tips $25k and overtime $12.5k/$25k, $100 per $1,000 over $150k/$300k; car-loan interest $10k). We blank line 14 only for a 65+ non-MFS filer. Worth a look: a W-2 box 12 code TP or TT should make line 13b pending too.
    • Found: line 7a copies Schedule D line 16 even when it is a loss. Schedule D line 21 (the smaller of the loss or $3,000, $1,500 MFS) is not modeled, so a net loss over the cap understates AGI. PolicyEngine caps it. Lead carter: IRS ATS scenario 8 moved to California (unverified) #1 in the summary.
    • Not compared: lines 1b–1i, the Schedule D Tax Worksheet (28% / unrecaptured §1250 gain), Forms 2555, 4952, 8814, 4972.

    Lines 17–38

    • Same: nonrefundable credits stop at line 18 tax (our line 22 floors at zero; they take the smaller of tax and credits). AMT sits in tax before credits. NIIT, Form 5329 and Form 4137/8919 taxes come after credits (line 23). The Form 5695 credit-limit worksheet subtracts the same Schedule 3 lines (1, 2, 3, 4, 6d; we also take 6l, which they do not model). 2025 CTC $2,200 per child, $500 ODC, ACTC cap $1,700, phase-out $50 per $1,000 rounded up, $200,000/$400,000 MFJ thresholds all match the 2025 Schedule 8812 instructions.
    • Different: PolicyEngine's income_tax leaves out self-employment tax, Additional Medicare Tax and household employment taxes (they sit in its payroll-tax total); our line 24 follows the form. Withholding (25a–c), estimated payments (26) and excess FICA (Schedule 3 line 11) are inputs on both sides. Our lines 25d and 33 print 0, not blank, when nothing was withheld (cosmetic). Line 25c takes only 8959 line 24 (input gap).
    • Found: PolicyEngine phases out the CTC for a qualifying surviving spouse from $400,000 (ctc/phase_out/threshold.yaml, SURVIVING_SPOUSE). The 2025 Schedule 8812 instructions, "Limits on the CTC and ODC", give $400,000 to married filing jointly only; IRC §24(h)(3) says "in the case of a joint return". Already filed upstream as Federal CTC phase-out threshold for surviving spouses is $400,000 from 2018; 24(h)(3) gives $200,000 PolicyEngine/policyengine-us#9703 with a fix in Fix the 2021 CTC split and surviving-spouse phaseout threshold PolicyEngine/policyengine-us#10306. Nothing on our side; line 19 is still an input gap.
    • Not compared: lines 19, 26–32, 36, 38 (Schedule 8812, EITC, Forms 8863/8839, Schedule 3 Part II, Form 2210).
  6. captaincole commented on Nov 1, 2026

    @captaincole
    OwnerAuthor

    Schedule 1 vs PolicyEngine above_the_line_deductions, adjusted_gross_income, irs_gross_income

    • Same: every 2025 constant both sides touch. HSA limits $4,300/$8,550 (8889 line 3). Educator cap $300 (line 11). Student loan cap $2,500, phase-out from $85,000 (single/HoH/QSS) or $170,000 (MFJ) over $15,000/$30,000 (line 21). IRA $7,000 + $1,000 catch-up at 50; phase-out $79k–$89k, $126k–$146k, spouse route $236k–$246k, MFS $0–$10k, reduction rounded to $10, $200 floor (line 20). Deductible SE tax 50% (line 15). Alimony only for pre-2019 instruments (lines 2a/19a). §461(l) threshold $313,000/$626,000 (line 8p). Unemployment fully taxable in 2025 (line 7). Line 8f = Form 8889 lines 16 + 20.
    • Different: we route all 1099-MISC box 3 to line 8z; PolicyEngine has one miscellaneous_income input. Prizes technically belong on line 8i. Our HSA limit prorates by coverage months; PolicyEngine takes the deduction as an input. Our totals (lines 9, 10, 25, 26) count an unsupported component line as 0; PolicyEngine sums every source. Worth a look.
    • Found: Schedule 1 is filed only when 1099-MISC box 3 > 0 (the fallback in engine/src/filingInfoResolvers.ts). A filer with an HSA deduction (8889 line 13) or taxable HSA distributions (8889 line 16) and no 1099-MISC gets no Schedule 1, so 1040 lines 8 and 10 stay blank. Form 8889 instructions, line 13 ("enter on Schedule 1, Part II, line 13") decide it. Lead Split the engine and the forms out of the agent #5.
    • Second pass: confirmed. A non-filed form gets no fields, the 1040 reads the missing line 10 as 0, and nothing becomes a pending input. Cade's decision table does not include must_file_schedule_1 or must_file_8889, so the fallback is the normal path, and Cade does record HSA contributions after a 1099-SA. A missing line 13 makes AGI too high; a missing line 8f makes it too low. Schedule 2's resolver already derives its must-file from 8959, 8960 and Schedule H, so the same shape fits here: file Schedule 1 when 8889 files or box 3 > 0. No golden changes (jordan already files it; marcus has a code W but no coverage facts).
    • Not compared: lines 1–7, 8a–8v, 12, 14–24z (unsupported on our side); PolicyEngine's loss_ald, adoption and savings-bond exclusions.
  7. captaincole commented on Nov 1, 2026

    @captaincole
    OwnerAuthor

    Schedule 2 vs PolicyEngine alternative_minimum_tax, additional_medicare_tax, net_investment_income_tax

    • Same: lines 11 and 12 copy Forms 8959 and 8960, and the 2025 constants behind them agree (0.9% over $200k/$250k/$125k; 3.8% over $200k/$250k/$125k, surviving spouse $250k). Line 9 takes Schedule H line 8 when there is no FUTA tax, as Schedule H line 27 says. Our totals (1z, 3, 7, 18, 21) add the lines the form names. PolicyEngine has no form totals; it books AMT into income_tax_before_credits, NIIT into income_tax_before_refundable_credits, and Additional Medicare into employee_payroll_tax.
    • Different: line 2 (AMT) is blank for us; PolicyEngine computes all of Form 6251, and its 2025 numbers match the 1040 instructions for line 2 and the 6251 Exemption Worksheet: exemption $88,100/$137,000/$68,500, phased out at 25% above $626,350/$1,252,700, 26% rate up to $239,100. Worth a look: our 1040 line 15, Schedule A line 7 and the QDCG worksheet already give 6251 lines 1, 2a, 10, 13, 20 and 27; private-activity-bond interest facts exist; an ISO-spread fact and the Part II/III bindings are the gap. Line 4 (SE tax) is blank for us (no Schedule SE). Our line 12 reads 0, not blank, for every filer under the NIIT threshold, so line 21 and 1040 line 23 read 0 too (cosmetic, shared with the 8960 note).
    • Found: none.
    • Not compared: lines 1a–1y, 5–6, 8, 13–16, 17a–17z, 19–20. We mark them unsupported; PolicyEngine has only input variables for a few (unreported_payroll_tax, qualified_retirement_penalty, recapture_of_investment_credit).
  8. captaincole commented on Nov 1, 2026

    @captaincole
    OwnerAuthor

    Schedule 3 vs PolicyEngine income_tax_non_refundable_credits, income_tax_capped_non_refundable_credits, energy_efficient_home_improvement_credit

    • Same: line 5b carries Form 5695 line 32. Both sides cap that credit at tax minus the same earlier credits (Schedule 3 lines 1, 2, 3, 4, 6d), in the order the 2025 Form 5695 Credit Limit Worksheet gives. The total cap matches too: we floor 1040 line 22 at line 18 − line 21; PolicyEngine caps all nonrefundable credits at tax before credits. PolicyEngine's credit order (foreign tax, dependent care, elderly, education, saver's, home energy, used then new clean vehicle, CTC, clean energy) agrees with the IRS worksheets.
    • Different: PolicyEngine skips Schedule 3 line 6l (Form 8978) in the worksheet; we include it. PolicyEngine computes lines 1–4, 5a, 6d, 6f and 6m with their phase-outs; we leave them blank until a scenario needs them, and a blank counts as 0 in line 8 and in the 5695 worksheet.
    • Found: none that survived. A first note flagged that PolicyEngine's cap base (regular tax + capital gains tax + AMT) leaves out the excess advance premium tax credit repayment that 1040 line 18 carries. PolicyEngine has no advance-PTC concept at all, so the two agree whenever there is nothing to repay. Scope note, not a bug.
    • Not compared: Part II (lines 9–15). PolicyEngine has no Schedule 3 payments: excess_payroll_tax_withheld is an input with no formula, the premium tax credit sits outside its refundable list, and lines 10, 12 and 13a–13z have no counterpart. Lines 6a–6c, 6g–6l and 6z likewise.
  9. captaincole commented on Nov 1, 2026

    @captaincole
    OwnerAuthor

    Schedule A vs PolicyEngine salt_cap, salt_deduction, medical_expense_deduction, charitable_deduction, interest_deduction

    • Same: medical floor 7.5% of AGI; SALT base cap $40,000 ($20,000 MFS), 30% phase-down above $500,000 ($250,000 MFS) MAGI, floor $10,000 ($5,000 MFS); no §68 (Pease) limit in 2025; no mortgage-insurance line; itemize when the total beats the standard deduction.
    • Different: PolicyEngine takes the larger of income tax or sales tax on 5a; we use a decision and never model the sales-tax amount. PolicyEngine adds Form 2555 and Puerto Rico exclusions back into MAGI; we use AGI. Both fine. Worth a look: PolicyEngine floors AGI at zero before the 7.5% medical floor; we use line 11b as is, so a negative AGI raises line 4. PolicyEngine applies the $750,000/$375,000 acquisition-debt limit to mortgage interest; we take Form 1098 box 1 as is, and the target user often carries a larger mortgage. PolicyEngine applies the Pub. 526 60/50/30/20% AGI limits to gifts; we take lines 11–13 as facts with no limit check.
    • Found: our MFS SALT phase-down runs at 30% against the $20,000 cap. The IRS worksheet (2025 Schedule A instructions, State and Local Tax Deduction Worksheet) reduces $40,000 by 30%, floors at $10,000, then halves for MFS, so 15% in effect. Lead jordan: 1099-Rs as payer-style statements #2.
    • Second pass: confirmed line by line from the worksheet text. Line 1 is $40,000 for every status; line 5 is $500,000 ($250,000 MFS); line 7 is 30% of the excess; line 9 is the larger of line 8 or $10,000 with no MFS variant; line 10 takes half of line 9 for MFS. MFS, MAGI $300,000, line 5d $25,000: worksheet $12,500, ours $5,000. Also: line 7 is a calculated line and ours is unrounded, so a single filer at MAGI $500,001 with $40,000 of tax prints $39,999.70. MFS is out of scope today and only marcus runs Schedule A (single, unaffected), so no golden changes. Fix when the file is next touched: cap = max($10,000, $40,000 − round(30% × excess)), halved for MFS.
    • Not compared: lines 6, 15, 16, 18 (unsupported on our side; PolicyEngine zeroes casualty losses for 2025 and ignores the disaster exception).
  10. captaincole commented on Nov 1, 2026

    @captaincole
    OwnerAuthor

    Schedule D vs PolicyEngine net_capital_gain, capital_gains_tax, filer_loss_limited_net_capital_gains

    • Same: netting order (line 7 + line 15 = line 16; worksheet line 3 = the smaller of lines 15 and 16, zero if either is a loss). The 2025 0% and 15% ceilings match for every status ($48,350 / $96,700 / $64,750 and $533,400 / $600,050 / $566,700; MFS $48,350 / $300,000). Tax is the smaller of the worksheet result and the ordinary tax. Qualified dividends with a net capital loss still go through the worksheet.
    • Different: PolicyEngine takes net short-term and long-term totals as inputs, carryovers included; our lines 6 and 14 stay blank until a prior-year fact exists. PolicyEngine computes tax in cents from bracket math; we use the tax table under $100,000 and round lines 18 and 21 (a few dollars). Worth a look: PolicyEngine models the 28% rate gain and unrecaptured §1250 gain; our lines 18 and 19 stay blank and line 20 always picks the QDCG worksheet, even when a 1099-DIV has box 2b or 2d, which the engine already ingests but nothing reads.
    • Found: (1) Form 1040 line 7a copies Schedule D line 16 with no loss cap; line 21 is unsupported. A $10,000 net loss lowers AGI by $10,000, not $3,000 ($1,500 MFS). Lead carter: IRS ATS scenario 8 moved to California (unverified) #1. (2) When Schedule D is filed, 1099-DIV box 2a goes nowhere: line 13 is a stub and the line 7a Exception 1 path turns off. Schedule D instructions, "Capital Gain Distributions": enter box 2a on line 13. Lead agent: double Luca's output token limit to 8,192 #6.
    • Second pass on (2): confirmed. A stubbed line renders blank and never becomes a pending input; line 15 is a sum that drops it; line 16 and 1040 line 7a follow. No scenario has both a 1099-B and box 2a (carter has box 2a without Schedule D, jordan has Schedule D with box 2a = 0), so no golden is wrong today, but the pairing is common for the target user. Fix: bind line 13 to the sum of 1099-DIV box 2a, the same source the 1040 no-Schedule-D path already uses. Box 2b/2d wait for the 28% and §1250 worksheets.
    • Not compared: lines 4, 5, 11, 12 (Forms 6252, 4684, 6781, 8824, 4797, 2439, K-1s), the QOF box, the line 1a/8a bypass rows, Form 8949 column (g) adjustments.
  11. captaincole commented on Nov 1, 2026

    @captaincole
    OwnerAuthor

    Form 8889 vs PolicyEngine health_savings_account_ald

    • Same: 2025 limits ($4,300 self-only, $8,550 family) match PolicyEngine's parameters and the Form 8889 line 3 instructions. Line 12 floors at zero and line 13 takes the smaller of line 2 and line 12, as the line 13 instructions say.
    • Different: PolicyEngine does not compute the deduction at all; both variables are plain inputs with no formula. We prorate line 3 by coverage months; PolicyEngine has no proration, no catch-up, no employer offset.
    • Found: our line 3 ignores the last-month rule, and for an unmarried filer age 55+ the $1,000 catch-up belongs on line 3 (rule 6), not line 7; our note on line 7 points at the wrong line, and the missing line 7 passes as zero. Lead Home screen features and a single-page return workspace #8.
    • Second pass: the last-month rule is not an automatic correction. The instructions say the taxpayer "may" treat themselves as eligible all year, and doing so binds them to a testing period through the end of 2026, with income inclusion and a 10% tax if coverage lapses. The engine cannot know future eligibility, and the needed fact (coverage on December 1) is not captured; hsa.coverage_months is a count. Proration is the right default, and the rule belongs behind a decision fact (same shape as decisions.scope.must_file_8889). The catch-up gap is real: rule 6 puts it on line 3 for an unmarried 55+ filer, line 7 is only the married family-coverage case, and our line 3 never adds it. No scenario is 55+ with an HSA. Smaller: the 1099-SA sum does not check box 5 (HSA vs MSA); one-line filter, not reachable today.
    • Not compared: lines 4, 6, 7, 10, 14b, 15–21 (Parts II and III).
  12. captaincole commented on Nov 1, 2026

    @captaincole
    OwnerAuthor

    Form 8959 vs PolicyEngine additional_medicare_tax, additional_medicare_tax_withheld

    • Same: thresholds ($200k single/HOH/QSS, $250k MFJ, $125k MFS), rates 0.9%/1.45%, spouses' wages combined on a joint return. PolicyEngine's single max(0, wages + SE − threshold) equals our wages-first split for non-negative SE income.
    • Different: PolicyEngine does not round; we round each line and the CPA goldens agree. PolicyEngine computes withholding as 0.9% over $200k per person and never uses it; our Part V reconciles box 6 against 1.45% of box 5. Worth a look: line 19 takes box 6 only; the instructions add box 12 codes B and N.
    • Found: (1) with no filing status, line 5 blocks, line 6 treats the blank as 0, and line 7 taxes all wages (marcus: $2,192, not $392), reaching Schedule 2 line 11. Lead tests: unit tests for the tax tables, vitest in the pre-commit hook #3. (2) Our must-file fallback uses $200k for every status; "Who Must File" uses the status threshold. Lead Ask The Tax Code: cited answers from the reference corpus #9.
    • Second pass on (1): reproduced in the engine, and the state is reachable (the W-2 is ingested before the scoping questions, the status is recorded later). But the 1040 resolver lists the filing status as a missing input, so the case state carries no money values, document generation refuses, and 1040 lines 16, 22, 24 and 37 are blocked. No wrong dollar reaches a user today. It is a latent trap for the first code path that reads values while inputs are pending. Fix: line 6 stays blank when line 5 is blank (the 1040 line 15 pattern). Line 22 does not share the bug.
    • Second pass on (2): partly. "Who Must File" says any single W-2 with box 5 over $200,000 must file, for every status, so the $200,000 fallback is a real rule, not the single threshold. The gap is MFS with wages between $125,000 and $200,000, and Cade never sets must_file_8959, so it is missed today. Out of scope for a single filer. Fix: use the status threshold when the status is known, else keep $200,000.
    • Not compared: lines 2, 3, 8–17, 23 (unsupported here; PolicyEngine has no RRTA or 4137/8919 legs).
  13. captaincole commented on Nov 1, 2026

    @captaincole
    OwnerAuthor

    Form 8960 vs PolicyEngine net_investment_income_tax, net_investment_income, niit_magi

    • Same: line 1 (1040 line 2b), line 2 (1040 line 3b), thresholds on line 14 ($200,000 single/HOH, $250,000 MFJ/QSS, $125,000 MFS, not indexed), line 15 floor at zero, 3.8% rate.
    • Different: line 4a, we take Schedule 1 lines 3, 5 and 6 as the instructions say; PolicyEngine counts only rental, passive K-1 and trust income (lines 4a and 4b netted). Fine until Schedule C/E land; then line 4b must land too. Line 5a, PolicyEngine caps a net capital loss at $3,000; ours carries the full Schedule D line 16 loss (lead carter: IRS ATS scenario 8 moved to California (unverified) #1, upstream of this form). Line 9a, we deduct Schedule A line 9; PolicyEngine has no Part II deductions. Line 13, PolicyEngine adds back the §911 exclusion; we use AGI. Rounding, we round line 17; PolicyEngine does not.
    • Found: our line 12 has no floor. A net capital loss above interest plus dividends gives a negative line 16 and a negative tax on line 17. PolicyEngine floors NII at zero. Lead tests: tara — IRS ATS scenario 1 moved to California (Schedule H, Form 5695, Schedule 3) #4.
    • Second pass: confirmed, and the floor is printed on the form itself: line 12 reads "Subtract Part II, line 11, from Part I, line 8 ... If zero or less, enter -0-". Our catalog label dropped that clause, which is likely why the generated binding missed it. Line 16 passes the negative through because line 15 is floored, line 17 multiplies it, and Schedule 2 line 12 carries it to 1040 line 23. It stays reachable after the loss cap is fixed: a $400k W-2 filer with $1,000 of interest and a capped −$3,000 loss must file 8960 and gets line 17 = −$76. Fix: floor line 12 at zero like line 15, and restore the catalog label. Only jordan files 8960 (positive line 12), so no golden changes.
    • Not compared: lines 3, 4b, 5b, 5c, 6, 7, 9b, 9c, 10, 18–21, header elections.
  14. captaincole commented on Nov 1, 2026

    @captaincole
    OwnerAuthor

    Form 8995-A vs PolicyEngine qbid_amount, qualified_business_income_deduction, section_199a_net_capital_gain

    • Same: 2025 threshold $197,300 ($394,600 MFJ) and phase-in range $50,000 ($100,000 MFJ); MFS and qualifying surviving spouse use the single figures on both sides, and neither side applies the 2026 range ($75,000/$150,000) to 2025. 20% of QBI, the 50%-of-wages or 25%-of-wages + 2.5%-of-UBIA limit, Part III arithmetic, taxable income before QBI = 1040 line 11a − 12e − 13b floored at 0, income limit = 20% × (that − net capital gain − qualified dividends), deduction = smaller of lines 32 and 36, REIT/PTP at 20% with no wage limit. Jordan's locked Part IV matches the IRS arithmetic.
    • Different: PolicyEngine computes QBI from income sources less SE-tax, health and retirement deductions, nets losses pro rata and applies the SSTB percentage; we take these as inputs or leave them unsupported. PolicyEngine keeps cents; we round each line. Worth a look: we file 8995-A for any REIT dividend; the instructions say Form 8995 unless taxable income exceeds the threshold or the filer is a patron. Same number, different form; the engine has no Form 8995 yet. Single filers at $300k+ taxable income land on 8995-A correctly.
    • Found: our line 34 adds net capital gain only when Schedule D lines 15 and 16 exist. The 2025 8995-A instructions, Line 34: without Schedule D, net capital gain is Form 1040 line 7a. Lead Model evals: document ingest and review judges; e2e on a LAN model #7.
    • Second pass: confirmed, and reachable: a 1099-DIV with box 5 and box 2a and no trades scopes 8995-A, and 1040 line 7a already falls back to the distributions, so lines 34–36 print wrong. The deduction itself stays right unless line 36 binds, which needs REIT dividends above taxable income less capital gains, never the target user. Fix: when Schedule D is absent, take 1040 line 7a, floored at zero. Only jordan files 8995-A and it has Schedule D, so no golden changes.
    • Not compared: Part I entries, lines 2/4/7 inputs, line 14 patron reduction, lines 29/40 carryforward (PolicyEngine has none either), line 38 DPAD (PolicyEngine lacks it).
  15. captaincole commented on Nov 1, 2026

    @captaincole
    OwnerAuthor

    Finding 1: the capital loss limit is missing (1040 line 7a, Schedule D line 21)

    Our position: We agree. This is a real bug and the most important one in the list. A filer with a $20,000 net capital loss gets the full $20,000 off AGI today. The limit is $3,000 ($1,500 if married filing separately). The rest carries to next year.

    Why it happens: 1040 line 7a copies Schedule D line 16 (engine/src/federal/1040/bindings.ts). Schedule D line 21 is still a todo, so it prints blank. The todo note says we had no "smaller of" rule. Bindings are plain functions now, so that reason no longer applies.

    Fix:

    • Schedule D 1.line.21_allowable_loss: when line 16 is below zero, the smaller of the loss or the cap for the filing status. Otherwise blank.
    • 1040 line.7a: line 21 when line 16 is a loss, line 16 when it is a gain.

    Priority: next engine PR.

    Test: a new scenario with a loss over $3,000. No locked scenario has a net loss, so no golden changes.

  16. captaincole commented on Nov 1, 2026

    @captaincole
    OwnerAuthor

    Finding 2: SALT cap phase-down for married filing separately (Schedule A line 5e)

    Our position: We agree the math is wrong for MFS, but it can wait. Our code halves the $40,000 cap first and then takes the full 30% reduction. The worksheet in the Schedule A instructions reduces $40,000, floors the result at $10,000, and halves it only at the end. Example: MFS, MAGI $300,000, line 5d $25,000. The worksheet gives $12,500. We give $5,000.

    A second, smaller problem affects every filer: worksheet line 7 (30% of the excess) is not rounded. MAGI $500,001 prints $39,999.70 where the form wants $40,000.

    Fix: in 0.line.5e_salt_cap, cap = at least $10,000 of ($40,000 minus the rounded 30% reduction). For MFS, halve that cap and round. Use roundDollars on the 30% line.

    Priority: later. MFS is out of scope for now. Fix the rounding the next time this file changes.

    Test: one unit test per case: MFS in the phase-down, and a single filer just over $500,000. Only marcus files Schedule A, and his numbers do not change.

  17. captaincole commented on Nov 1, 2026

    @captaincole
    OwnerAuthor

    Finding 3: Form 8959 line 6 reads a blank line 5 as zero

    Our position: We agree it is a bug, but it cannot print a wrong number today. With no filing status, line 5 (the threshold) is blank. Line 6 treats it as $0, so line 7 taxes every Medicare wage dollar. We reproduced it: marcus without a status shows $2,192 instead of $392.

    It stays hidden because a missing filing status puts the whole return in pending. The UI and the documents show nothing until the status is set. If that gate ever changes, the wrong number shows up.

    Fix: 0.line.6_excess_medicare_wages returns blank when line 5 is blank. This is the same pattern 1040 line 15 uses. We could also list the filing status as a missing input in resolve8959Info. Line 22 does not have this bug.

    Priority: later, but cheap. Pair it with finding 4, because both are a blank line read as zero.

    Test: a unit test with no filing status that expects line 6 and line 7 to stay blank.

  18. captaincole commented on Nov 1, 2026

    @captaincole
    OwnerAuthor

    Finding 4: no zero floor on Form 8960 line 12

    Our position: We agree. Fix it now. The printed form says "If zero or less, enter -0-" on line 12. Our catalog label dropped that part, so the binding never floors. A net capital loss bigger than interest plus dividends gives a negative net investment income, and line 17 becomes a negative tax.

    This does not depend on finding 1. Even with a correct $3,000 cap, $1,000 of interest and a capped loss give line 12 = −$2,000 and line 17 = −$76.

    Fix: wrap 0.line.12 in atLeast(0, …) in engine/src/federal/8960/bindings.ts. Also restore the full label in the catalog.

    Priority: next engine PR, with finding 1.

    Test: the finding 1 scenario, if it has interest income and files Form 8960. Only jordan files 8960 today, and his line 12 is positive, so no golden changes.

  19. captaincole commented on Nov 1, 2026

    @captaincole
    OwnerAuthor

    Finding 5: Schedule 1 does not turn on for an HSA filer

    Our position: We agree. Fix it now. Only 1099-MISC box 3 turns Schedule 1 on. An HSA filer with no 1099-MISC files Form 8889, but the deduction on 8889 line 13 never reaches Schedule 1 line 13. AGI comes out too high, and nothing tells the user. A taxable HSA distribution (line 8f) is lost the same way, and that makes AGI too low.

    Cade's decision table has no must_file_schedule_1 key, so the fallback rule is the normal path, not an edge case.

    Fix: in resolveSchedule1Info (engine/src/filingInfoResolvers.ts), file Schedule 1 when Form 8889 must be filed or 1099-MISC box 3 is above zero. resolveSchedule2Info already derives its must-file from other forms the same way.

    Priority: next engine PR.

    Test: a scenario with an HSA and no 1099-MISC. Check that 1040 line 10 carries the deduction. Jordan already files Schedule 1, so his numbers do not change.

  20. captaincole commented on Nov 1, 2026

    @captaincole
    OwnerAuthor

    Finding 6: capital gain distributions drop out when Schedule D is filed (Schedule D line 13)

    Our position: We agree. Fix it with finding 1. When a filer has 1099-DIV box 2a and also files Schedule D, the box 2a amount goes nowhere. Line 13 is a todo, so it prints blank. The 1040 line 7a exception (distributions only, no Schedule D) turns off. The income disappears, and no pending input warns about it.

    This pairing is common for our target user: a brokerage account with fund distributions and some trades.

    Fix: add capital gain distributions to the Schedule D filing info with sumInvestmentBox on box 2a. Bind 0.line.13_capital_gain_distributions to it and delete the todo. Boxes 2b and 2d (the 25% and 28% worksheets) stay later.

    Priority: next engine PR, with finding 1. Both change Schedule D line 16.

    Test: one scenario with box 2a and at least one trade. Carter has 2a but no Schedule D, and jordan's 2a is zero, so no golden changes.

  21. captaincole commented on Nov 1, 2026

    @captaincole
    OwnerAuthor

    Finding 7: Form 8995-A line 34 ignores gains when there is no Schedule D

    Our position: We agree the line is wrong, but the deduction is still right, so it can wait. Line 34 is qualified dividends plus net capital gain. With no Schedule D, net capital gain is 1040 line 7a (capital gain distributions only). Our binding sets it to zero in that case.

    Lines 34 to 36 print wrong. Line 36 (20% of taxable income minus line 34) limits the deduction only when the QBI amount is bigger than it. That never happens for a filer at our target income, so the final deduction does not change.

    Fix: in 1.line.34, when Schedule D line 16 exists, keep the current path. Otherwise use at least $0 of 1040 line 7a.

    Priority: later. It is a printed-form accuracy fix with no effect on the tax.

    Test: a unit test with 1099-DIV box 2a and box 5 and no trades. Only jordan files 8995-A, and he has Schedule D, so no golden changes.

  22. captaincole commented on Nov 1, 2026

    @captaincole
    OwnerAuthor

    Finding 8: Form 8889 line 3 (last-month rule and the 55+ catch-up)

    Our position: We agree with half of it.

    • Last-month rule: keep proration as the default. The instructions say the taxpayer "may" treat themselves as eligible all year. That choice starts a testing period through the end of 2026. If coverage lapses, they owe income tax plus 10%. The engine cannot know future coverage, so this is the taxpayer's choice. It belongs behind a decision (for example decisions.hsa.last_month_rule), and we also need a fact for coverage on December 1.
    • 55+ catch-up: a real gap. For an unmarried filer age 55 or older, the $1,000 goes on line 3. Line 7 is only for the married, family-coverage case. Our line 3 never adds it, and line 7 is unsupported, so it passes as zero.

    Fix: add an age-55 helper. In 0.line.3_annual_hsa_contribution_limit, add $1,000 × months / 12 for an unmarried 55+ filer. Fix the stale note on line 7. Filtering 1099-SA box 5 to "HSA" is a one-line cleanup.

    Priority: later. No scenario has an HSA filer over 55.

  23. captaincole commented on Nov 1, 2026

    @captaincole
    OwnerAuthor

    Finding 9: Form 8959 must-file uses $200,000 for every filing status

    Our position: Partly right, and it can wait. The "Who Must File" section of the instructions makes any single W-2 with Medicare wages over $200,000 a must-file for every status. So the $200,000 test is a real rule, not a shortcut. It misses one case: married filing separately with wages between $125,000 and $200,000. Those filers owe the tax but do not get the form.

    For MFJ, several W-2s that add up to $200,000 to $250,000 also trigger the form, with $0 tax. That is harmless.

    Cade never sets must_file_8959, so this fallback in resolve8959Info decides every return.

    Fix: when the filing status is known, use its threshold ($125,000 MFS, $250,000 MFJ, $200,000 otherwise) next to the single-W-2 rule. When it is not known, keep $200,000.

    Priority: later. MFS is out of scope.

    Test: a unit test for MFS with $150,000 of wages.

  24. captaincole commented on Nov 1, 2026

    @captaincole
    OwnerAuthor

    Closing for now.

    Part 2 (logic, line by line): done. The notes by form and the nine findings are in the comments above. #67 fixes seven of them. Findings 8 (Form 8889 catch-up and last-month rule) and 9 (Form 8959 MFS threshold) wait for scope. #68 adds a skill so the comparison can be repeated.

    Part 1 (numbers): deferred. Our engine does not cover the full federal return yet, so a generated grid would mostly show known gaps. PolicyEngine also starts from computed amounts (taxable pension income, the HSA deduction, net gains), so it can check the back half of a return (AGI to total tax) but not the front half (documents to income lines). Reopen this, or start a new issue, when the engine's federal coverage is complete. A first step could be our own scenarios run through both engines.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestideaIdea gathered from other projects; not scheduled

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions