feat(scripts): backfill security businesses from ingested executions - #4256
Draft
gilgardosh wants to merge 2 commits into
Draft
feat(scripts): backfill security businesses from ingested executions#4256gilgardosh wants to merge 2 commits into
gilgardosh wants to merge 2 commits into
Conversation
gilgardosh
temporarily deployed
to
accounter-fullstack
August 19, 2026 22:15 — with
GitHub Actions
Inactive
gilgardosh
temporarily deployed
to
accounter-fullstack
August 19, 2026 22:16 — with
GitHub Actions
Inactive
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a one-time backfill utility to align historical Poalim securities data with the new “business per security (ISIN)” model introduced during ingestion, and to re-point eligible legacy trades to the per-security business.
Changes:
- Add
scripts/backfill-security-businesses.tsto (1) create per-ISIN security businesses and linkPOALIM_SECURITY_KEYidentifiers, then (2) re-point non-fee transactions currently on the general foreign-securities business when exactly one resolvable key is present in the description. - Wire the script as
yarn backfill:security-businesses. - Add a changeset documenting the new backfill command and behavior.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| scripts/backfill-security-businesses.ts | New backfill script implementing the two-step historical migration (create security businesses + repoint trades), with dry-run/apply modes. |
| package.json | Adds backfill:security-businesses script entry. |
| .changeset/security-businesses-backfill.md | Documents the new backfill command and its intended behavior/idempotency guarantees. |
Suppressed comments (2)
scripts/backfill-security-businesses.ts:343
- The UPDATE only filters by transaction id. Adding owner_id and the expected current business_id makes the repointing safer (avoids updating a transaction that was reassigned between the SELECT and UPDATE, and makes the intent explicit).
if (apply) {
await client.query(`UPDATE ${SCHEMA}.transactions SET business_id = $1 WHERE id = $2`, [
businessId,
transaction.id,
]);
scripts/backfill-security-businesses.ts:262
- In dry-run mode, identifiersLinked is incremented for every Poalim key in the feed, even if that key is already linked from an earlier run or from ingestion. This can significantly over-report what “--apply” would actually create.
const businessIdByPoalimKey = new Map<string, string>();
for (const group of groups.values()) {
let businessId = businessByIsin.get(group.isin);
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
gilgardosh
force-pushed
the
feat/security-businesses-automation
branch
from
August 20, 2026 09:45
2216167 to
372ead7
Compare
gilgardosh
force-pushed
the
feat/security-businesses-backfill
branch
from
August 20, 2026 09:47
9742d03 to
4442f0b
Compare
gilgardosh
temporarily deployed
to
accounter-fullstack
August 20, 2026 09:48 — with
GitHub Actions
Inactive
gilgardosh
temporarily deployed
to
accounter-fullstack
August 20, 2026 09:48 — with
GitHub Actions
Inactive
gilgardosh
force-pushed
the
feat/security-businesses-automation
branch
2 times, most recently
from
August 20, 2026 13:16
f55d378 to
4b5729a
Compare
gilgardosh
force-pushed
the
feat/security-businesses-backfill
branch
from
August 20, 2026 13:38
4442f0b to
fed3740
Compare
gilgardosh
temporarily deployed
to
accounter-fullstack
August 20, 2026 13:38 — with
GitHub Actions
Inactive
gilgardosh
temporarily deployed
to
accounter-fullstack
August 20, 2026 13:38 — with
GitHub Actions
Inactive
gilgardosh
temporarily deployed
to
accounter-fullstack
August 20, 2026 13:38 — with
GitHub Actions
Inactive
gilgardosh
temporarily deployed
to
accounter-fullstack
August 20, 2026 13:38 — with
GitHub Actions
Inactive
Creates a business per ISIN out of poalim_securities_transactions with a POALIM_SECURITY_KEY identifier per key, then re-points non-fee transactions that currently point at the general foreign-securities business and whose description names exactly one resolvable key. Dry-run by default; --apply writes, --owner limits to one tenant. Idempotent, and reports what it deliberately left alone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Addresses review feedback: the three inserts a security takes ran without a transaction, so a failure part-way through left a business that is not a security - invisible to the ISIN lookup and duplicated by the next run, which is exactly the guarantee the script makes. Also normalizes the currency through formatCurrency. businesses_securities .currency_code is the accounter_schema.currency enum since #4252, and the executions report the feed's own Hebrew labels, so the insert would have failed outright on the first real security. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
gilgardosh
force-pushed
the
feat/security-businesses-backfill
branch
from
August 20, 2026 13:40
fed3740 to
cb23ed0
Compare
gilgardosh
temporarily deployed
to
accounter-fullstack
August 20, 2026 13:40 — with
GitHub Actions
Inactive
gilgardosh
temporarily deployed
to
accounter-fullstack
August 20, 2026 13:40 — with
GitHub Actions
Inactive
Contributor
🚀 Snapshot Release (
|
| Package | Version | Info |
|---|---|---|
@accounter/client |
0.1.0-alpha-20260820133947-eadbfa3872eec9cb96ce0dc92561f34d6eaf0c2b |
npm ↗︎ unpkg ↗︎ |
@accounter/green-invoice-graphql |
0.8.7-alpha-20260820133947-eadbfa3872eec9cb96ce0dc92561f34d6eaf0c2b |
npm ↗︎ unpkg ↗︎ |
@accounter/hashavshevet-mesh |
0.2.13-alpha-20260820133947-eadbfa3872eec9cb96ce0dc92561f34d6eaf0c2b |
npm ↗︎ unpkg ↗︎ |
@accounter/israeli-vat-scraper |
0.1.13-alpha-20260820133947-eadbfa3872eec9cb96ce0dc92561f34d6eaf0c2b |
npm ↗︎ unpkg ↗︎ |
@accounter/modern-poalim-scraper |
0.11.0-alpha-20260820133947-eadbfa3872eec9cb96ce0dc92561f34d6eaf0c2b |
npm ↗︎ unpkg ↗︎ |
@accounter/payper-mesh |
0.2.13-alpha-20260820133947-eadbfa3872eec9cb96ce0dc92561f34d6eaf0c2b |
npm ↗︎ unpkg ↗︎ |
@accounter/scraper-app |
0.0.3-alpha-20260820133947-eadbfa3872eec9cb96ce0dc92561f34d6eaf0c2b |
npm ↗︎ unpkg ↗︎ |
@accounter/server |
0.2.0-alpha-20260820133947-eadbfa3872eec9cb96ce0dc92561f34d6eaf0c2b |
npm ↗︎ unpkg ↗︎ |
@accounter/shaam-uniform-format-generator |
0.2.7-alpha-20260820133947-eadbfa3872eec9cb96ce0dc92561f34d6eaf0c2b |
npm ↗︎ unpkg ↗︎ |
@accounter/shaam6111-generator |
0.1.9-alpha-20260820133947-eadbfa3872eec9cb96ce0dc92561f34d6eaf0c2b |
npm ↗︎ unpkg ↗︎ |
gilgardosh
marked this pull request as draft
August 20, 2026 13:55
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Step 5 of the per-security businesses plan. Stacked on #4255.
Ingestion (#4255) creates a business per security as executions arrive; the history that predates it
has none, and its trades still point at the general foreign-securities business.
yarn backfill:security-businessescloses that gap in two idempotent steps:poalim_securities_transactions, with aPOALIM_SECURITY_KEYidentifier per key seen reporting it, inheriting sort code / IRS code / country / tax category
from the general business — the same thing
ensureSecurityBusinessdoes.currently on the general foreign-securities business, and only non-fee rows.
Dry-run by default;
--applywrites and--owner=<uuid>limits the run. It setsapp.current_business_idper tenant, as the server does per request, so it behaves the same whetheror not the connecting role bypasses RLS.
Verified end-to-end against a local scenario
Two Poalim keys reporting one ISIN, one execution with no ISIN, one plain trade, one fee row, one
description naming two securities:
Resulting counterparties — the trade moved, the fee row and the ambiguous one did not:
ניע"ז קניה 0005129523ניעז עמ׳ תשלום FSEC PYMNT FEE 0005129523ניע"ז 0005129523 0007654321A second
--applycreated and re-pointed nothing (still one security business). The dry run reportedexactly what the apply then did. Scenario data removed afterwards.
yarn lintandyarn prettierclean.🤖 Generated with Claude Code