Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
172 commits
Select commit Hold shift + click to select a range
bce6fc1
Add backtrack: reconstruct season-to-date stats from NHL game logs
nathan-probert Oct 8, 2026
c645538
Add five_gpg to the derived features
nathan-probert Oct 8, 2026
868681c
Add team stats (tgpg, otga) from official box score totals
nathan-probert Oct 8, 2026
3bd85f8
Fix backtrack review findings: debut rows, dead code, and tests
nathan-probert Oct 9, 2026
7cbfb17
Refresh 20232024 dump after the debut-zero re-derive
nathan-probert Oct 9, 2026
d2deffd
Prefer archive names on publish; compare teams by player identity
nathan-probert Oct 9, 2026
7db9677
Add ppg player rate and reconstruct team otshga
nathan-probert Oct 9, 2026
219f92f
Add playoffs to the season record: schedule walk, box scores, game logs
nathan-probert Oct 9, 2026
de6cdf2
Retry box-score fetches so a reset connection cannot kill the build
nathan-probert Oct 9, 2026
148e73a
Record playoff-scope validation: 32/32 endpoint, official-NHL totals,…
nathan-probert Oct 9, 2026
6aa87a5
Re-export 20232024 store dump with playoff games
nathan-probert Oct 9, 2026
f4393b8
Report skater log-merge coverage separately from goalie rows in stats
nathan-probert Oct 9, 2026
61f6c7c
Backfill historic tims and reconstructed lineups into the archive
nathan-probert Oct 11, 2026
0c72669
Backfill historic opponent starting goalies from Rotowire's table
nathan-probert Oct 11, 2026
0edae10
Publish the scored training label and fix the join plan it broke
nathan-probert Oct 11, 2026
9810da6
Store tims and goalie feeds as SQLite tables; publish reads them back
nathan-probert Oct 11, 2026
a59a545
Fix stats lineup query: player_lineups has no season column
nathan-probert Oct 11, 2026
24221bd
Add 20222023 and 20242025 store dumps
nathan-probert Oct 12, 2026
02892f1
Normalize corrupted shift feeds: duplicated team ids and repeated rec…
nathan-probert Oct 12, 2026
cb343f3
Skip one-sided overtime periods in the shift sweep
nathan-probert Oct 12, 2026
c8cee11
Add 20212022 store dump
nathan-probert Oct 13, 2026
8a3927d
Compute hgpg/hppg as the archive's 3-year windows and publish them
nathan-probert Oct 13, 2026
2fe45c9
Regenerate store dumps with hgpg/hppg columns
nathan-probert Oct 13, 2026
a5f12ad
Reconstruct opponent goalie stats from cached game logs
nathan-probert Oct 14, 2026
fb20d89
Update reconstruct.py's reconstruction-status note for the goalie/lin…
nathan-probert Oct 14, 2026
cda88d6
Add 20252026 store dump
nathan-probert Oct 14, 2026
3b2a5c9
Commit API payloads to the store dump so history rebuilds without a r…
nathan-probert Oct 15, 2026
cf3aa34
Add Goalie-Snapshots-backtrack with its own publish pass
nathan-probert Oct 15, 2026
fc2e8cd
Move the opponent-goalie block to Goalie-Snapshots-backtrack
nathan-probert Oct 15, 2026
b2f1444
Add a backtrack-publish dispatch workflow against the deploy CI secrets
nathan-probert Oct 16, 2026
d345e89
Remove the backtrack-publish dispatch workflow - publishing runs locally
nathan-probert Oct 18, 2026
b5b52d4
Move the HTTP cache out of the smartscore subtree to data/cache
nathan-probert Oct 18, 2026
eb91e91
Commit the 2017-18 through 2020-21 store dumps
nathan-probert Oct 19, 2026
9287ab9
Add backtrack model and notes
nathan-probert Oct 19, 2026
1f1389e
Add within-day lambdarank ranker (V3a)
nathan-probert Oct 19, 2026
1194ab2
Record V3b feature experiment (reverted)
nathan-probert Oct 19, 2026
bfbba8b
Record V3c blend experiment (notes only)
nathan-probert Oct 19, 2026
7d9ba53
Add selectable lgbm/extratrees pointwise models (V3d)
nathan-probert Oct 19, 2026
08f0aac
Add seed control and record LGBM seed robustness (V3e)
nathan-probert Oct 19, 2026
758d275
Record V3f LGBM baseline-blend experiment (notes only)
nathan-probert Oct 19, 2026
08e92e7
Add V4 shot/TOI feature block and notes
nathan-probert Oct 19, 2026
2c72637
V4: record completed seed checks only
nathan-probert Oct 19, 2026
e2a5c1c
V4: record chronological-split caveat and next steps
nathan-probert Oct 19, 2026
beaee74
V4: avoid overclaiming the baseline comparison
nathan-probert Oct 19, 2026
2eef069
Add LGBM config options and record V3g
nathan-probert Oct 19, 2026
e4aadd2
Record clean V4 LGBM evaluation (V4 results)
nathan-probert Oct 19, 2026
469df95
Record V5 pick-voting and recency-weight results (notes only)
nathan-probert Oct 19, 2026
8b077e7
Remove lookahead line/PP features (V6)
nathan-probert Oct 19, 2026
abe7767
Make corrected rank ensemble the default (V6b)
nathan-probert Oct 19, 2026
8979a44
Record V6c ensemble multi-seed and injury audit (notes only)
nathan-probert Oct 19, 2026
0f51f75
Audit backtrack evaluation: reproduce 30.358% LGBM, flag same-game li…
Oct 19, 2026
f5c229d
Quantify opponent-goalie identity ablation: immaterial (902/3159)
Oct 19, 2026
bbb3bf2
Verify V6 corrected pipeline: 916/3159, feature provenance audit
Oct 19, 2026
b783823
Consolidate audit docs, V3h notes, and final result
nathan-probert Oct 19, 2026
53a0d44
Record V8 shrinkage experiment (reverted, notes only)
nathan-probert Oct 20, 2026
107f153
Record V8-tune LGBM sweep (notes only)
nathan-probert Oct 20, 2026
5e32994
Record V7 objective experiments: Poisson LGBM and conditional softmax…
nathan-probert Oct 20, 2026
9adacba
Record V7-context team/special-teams experiment (negative, reverted)
nathan-probert Oct 20, 2026
715d463
Record V7-shots raw shot-quality experiment (reverted)
nathan-probert Oct 20, 2026
7472e93
Record V8-own-xG per-shot model experiment (reverted)
nathan-probert Oct 20, 2026
67effa8
Consolidate V7/V8 notes and finalize result
nathan-probert Oct 20, 2026
b6860bd
Record V9 projected-lineup experiment (reverted, notes only)
nathan-probert Oct 20, 2026
215db35
Add standalone lineup projector (V9 stage 1)
nathan-probert Oct 20, 2026
48c5739
Add projected-lineup features (V9 stage 2)
nathan-probert Oct 20, 2026
9362db0
Add probabilistic lineup analysis (V9 stage 3, reverted)
nathan-probert Oct 20, 2026
30de742
Record V9 stage 4 ensemble with lineup features (notes only)
nathan-probert Oct 20, 2026
8bc4ab3
Record V9 stage 5 final numbers (notes only)
nathan-probert Oct 20, 2026
eb201f4
Record V10 real-projection inventory (notes only)
nathan-probert Oct 20, 2026
1463857
Record V11 projection-fidelity sensitivity (notes only)
nathan-probert Oct 20, 2026
cb06e0c
Record V12 identity and diagnostics (rejected, notes only)
nathan-probert Oct 20, 2026
3d89522
Add recent-era backtest flags and record V12-era (notes)
nathan-probert Oct 20, 2026
95614eb
Record V12-qoc quality-of-competition experiment (reverted)
nathan-probert Oct 20, 2026
5ede030
Update FINAL RESULT with post-roadmap findings
nathan-probert Oct 20, 2026
fd9aa76
Record V12 interactions (reverted, notes only)
nathan-probert Oct 20, 2026
cef6a2c
Record V14 Tims-era evaluation and pre-2017 feasibility (notes only)
nathan-probert Oct 20, 2026
fcf07ed
Add tested pre-2017 ingestion runbook (V14b)
nathan-probert Oct 20, 2026
7b4325c
Fix pre-2017 checkpoint query and add ascending backfill note
nathan-probert Oct 20, 2026
3f0cc5c
Record V15 augmented-training evaluation (notes only)
nathan-probert Oct 20, 2026
9c9f315
Commit 2013-14 through 2016-17 store dumps
nathan-probert Oct 20, 2026
96f0ff2
Fix refresh pagination with date windows and document V15b
nathan-probert Oct 20, 2026
2f5481a
V16-collect: projected lineups persistence runbook + script
nathan-probert Oct 21, 2026
3de58b4
V16-calib: confident-miss analysis + within-group discrimination attempt
nathan-probert Oct 21, 2026
493ba96
V16-embed: leak-free player latent factors
nathan-probert Oct 21, 2026
84ed8dd
V16-group3: group-conditional weighting + dedicated g3 ranker
nathan-probert Oct 21, 2026
1e94c4a
V16-group3: group-conditional weighting/dedicated g3 ranker (seed 1/3…
nathan-probert Oct 21, 2026
e56b159
V16-pbp: pass-context first-party xG proxy
nathan-probert Oct 21, 2026
d8b2ef5
V16-pbp: pass-context first-party xG proxy
nathan-probert Oct 21, 2026
ef69863
V16-interact: home x form, rest x role, opponent-relative (rejected/m…
nathan-probert Oct 22, 2026
4f1816c
V16-exposure: multiplicative prior-xG60 x projected-TOI (rejected)
nathan-probert Oct 22, 2026
e7353a0
V16-audit: independent leakage and integrity audit
nathan-probert Oct 22, 2026
0ac2c0a
V16-audit: leakage and integrity audit
Oct 22, 2026
028178c
V16: consolidate rejected experiments, mark embed INVALID, repo hygiene
nathan-probert Oct 22, 2026
d0ee9b9
V16-collect-fix: projected lineups with units + correct scheduling ru…
nathan-probert Oct 22, 2026
535cd53
MODEL_NOTES: record d0ee9b9 collect-fix state (units accumulating in …
nathan-probert Oct 22, 2026
c82b2e7
feat(feed-fidelity): nightly lineup-feed fidelity tracker unit-matchi…
nathan-probert Oct 23, 2026
e2bfc03
docs: spec of Supabase columns needed for in-cloud ML training (16 ne…
nathan-probert Oct 24, 2026
555b20e
docs: spec of Supabase columns needed for in-cloud ML training (16 ne…
nathan-probert Oct 24, 2026
37f0218
feat(backtrack): 16 shot/TOI + club columns migration, backfill scrip…
nathan-probert Oct 24, 2026
0246205
ci: temporary dispatch to apply pending migrations to dev/prod ad hoc
nathan-probert Oct 24, 2026
0b3e0ba
ci: migrate-adhoc as push-triggered (dev default, [prod] marker in co…
nathan-probert Oct 24, 2026
3fc0f11
ci: migrate-adhoc picks env from marker file, not commit text
nathan-probert Oct 24, 2026
8513ed2
fix(backfill): NaN->None needs astype(object); list payload cols (tup…
nathan-probert Oct 24, 2026
9d99995
fix(backtrack): list-wrap payload columns; include name (PostgREST ch…
nathan-probert Oct 24, 2026
23b6344
docs: DAILY_DATA_RUNBOOK - daily day-of data flow, verified 2026-10-24
nathan-probert Oct 24, 2026
409aa3d
ops: DAILY_DATA_RUNBOOK + read-only daily_sanity.py
nathan Oct 24, 2026
0b98040
test(backfill): 1000-row dev spot check (0 mismatches, 4.6e-15 max re…
nathan-probert Oct 24, 2026
4cba2b7
feat(backtrack): forward-path emits 16 shot/TOI+club columns on revie…
nathan-probert Oct 24, 2026
0d98cda
feat(aws): predict-only ML Lambda (MLPicks-{ENV}) — artifact from S3 …
nathan-probert Oct 24, 2026
3df47ef
lint: image-packaged MLPicks exempt from deploy.sh zip list; !Ref lin…
nathan-probert Oct 24, 2026
0c0e930
feat(aws): contract-compat artifact adapter + keys in export, structu…
nathan-probert Oct 24, 2026
edebc17
docs(runbook): exact day-of publish commands, prod-empty warning, mea…
nathan-probert Oct 24, 2026
79eaa3f
Delete dev backtracking snapshot: fail-loud guard + docs
nathan-probert Oct 24, 2026
aa4d34a
Publish team/goalie backtrack to prod, delete the dev copies
nathan-probert Oct 24, 2026
1d7032e
chore: clean repo tree — single results md, delete experiment scripts…
nathan-probert Oct 25, 2026
d82aa91
merge: feat/feed-fidelity (PR #137) into feat/final-clean
nathan-probert Oct 25, 2026
81fd402
merge: feat/supabase-spec (PR #138) into feat/final-clean
nathan-probert Oct 25, 2026
d0c13aa
merge: feat/supabase-shot-cols (PR #141) into feat/final-clean
nathan-probert Oct 25, 2026
98401fe
merge: feat/delete-backtrack-dev (PR #143) into feat/final-clean
nathan-probert Oct 25, 2026
d3e3794
merge: feat/goalie-team-prodfill (PR #144) into feat/final-clean
nathan-probert Oct 25, 2026
4b3f351
merge: feat/daily-data-runbook (PR #139) into feat/final-clean
nathan-probert Oct 25, 2026
607628a
merge: feat/daily-data-runbook-refresh (PR #140) into feat/final-clea…
nathan-probert Oct 25, 2026
3448524
merge: chore/clean-tree (PR #145) into feat/final-clean
nathan-probert Oct 25, 2026
bd7d508
merge: feat/aws-relambda (PR #142) into feat/final-clean
nathan-probert Oct 25, 2026
4fbdbc7
fix(consolidation): keep lineup_project.py (model.py dep per shot-col…
nathan-probert Oct 25, 2026
76c54bc
ci: fix lint gate (ruff 19+1 findings, reformat 10 files) and templat…
nathan-probert Oct 25, 2026
f3394c6
test: gate shot-columns attach off in publish name-preference test
nathan-probert Oct 25, 2026
2cd5a0a
chore: remove migrate-adhoc.yml + spot_check_shot_cols.py — one-shot …
nathan-probert Oct 25, 2026
87253b4
fix(cfn): condition-gate MLPicks resources on MLS3Configured so deplo…
nathan-probert Oct 27, 2026
c3c7a94
ci: ECR Public distribution master + private keep-last-1 Lambda-pull …
nathan-probert Oct 27, 2026
08ad0c0
fix(docker): Lambda entry spec is ml_picks_handler.handler (module ex…
nathan-probert Oct 27, 2026
22c7c54
docs: correct the private-ECR cost claim — free tier is first-year only
nathan-probert Oct 27, 2026
4458995
ci: image-size gate is a 5120MB sanity check, not a 500MB 'free-tier …
nathan-probert Oct 27, 2026
b506e35
feat(infra): replace MLPicks lambda with scheduled GitHub Actions inf…
nathan-probert Oct 27, 2026
34d46cb
feat(infra): per-game EventBridge -> MLPicksKick lambda -> ml-picks.y…
nathan-probert Oct 27, 2026
6ed223f
chore(ci): make CFN rollback failures diagnosable from deploy logs
nathan-probert Oct 27, 2026
d1f2127
fix(cfn): resolve the two template validation errors blocking SmartSc…
nathan-probert Oct 27, 2026
9097d72
chore(ci): rename repo secret GITHUB_DISPATCH_TOKEN -> MLPICKS_DISPAT…
nathan-probert Oct 27, 2026
2783df3
fix(artifact): export train-time preprocessing pipeline with estimato…
nathan-probert Oct 27, 2026
217baa9
fix(picks): do not wipe the Picks table unless the insert is safe; dr…
nathan-probert Oct 27, 2026
def96cf
feat(artifact): supabase-storage download path for the ml-picks job (…
nathan-probert Oct 27, 2026
d281822
merge: resolve conflicts with origin/main (#147 scaffold) taking our …
nathan-probert Oct 27, 2026
ea65542
fix(infra): daily picks read today's live snapshot rows; missing back…
nathan-probert Oct 27, 2026
cd05868
fix(infra): exact shot-block core replaces the stage-1 carry-forward …
nathan-probert Oct 27, 2026
cc04765
feat(infra): runner-side exact features - refresh_store rebuilds raw …
nathan-probert Oct 27, 2026
ab35189
feat(infra): exact same-day rolling/career features - daily backfill …
Oct 28, 2026
ca1f44e
feat(supabase): per-game raw stat columns (shots/goals/points/toi/min…
nathan-probert Oct 28, 2026
a436b8c
feat(infra): exact same-day rolling/career features via published raw…
nathan-probert Oct 28, 2026
07ffeb1
fix(tests): bind run_daily_picks ENV module global to the env_vars fi…
nathan-probert Oct 28, 2026
866238a
refactor(asl): remove the no-op CheckInitialRun Choice; GetTims route…
nathan-probert Oct 30, 2026
1ebcd3d
fix(infra): per-game ML picks dispatch moved from parallel relay to p…
nathan-probert Oct 30, 2026
048f37f
docs(chore): stage-2 20261027 prod-apply runbook (migrate-job diagnos…
nathan-probert Oct 30, 2026
aa2c784
chore(backfill): remove unmapped season key from created rows (PGRST2…
nathan-probert Oct 30, 2026
0c416f1
feat(infra): retrained ML artifact through 2026-10-29 (v2) - nightly …
nathan-probert Oct 30, 2026
110807c
fix(lambda): MLPicksKick carries the import-required env keys (supaba…
nathan-probert Nov 2, 2026
1e27e95
fix(relay): surface GitHub 403 response body + PAT scope hint in MLPi…
nathan-probert Nov 2, 2026
d2336cc
chore(lint): fix CI lint findings on the relay error-capture hardening
nathan-probert Nov 2, 2026
223e253
feat(ci): explicit environment selection in ml-picks.yml (schedule=pr…
nathan-probert Nov 2, 2026
027ef47
fix(infra): one shared history ledger - backfill targets shared Playe…
nathan-probert Nov 3, 2026
30c245d
fix(supabase): retire -dev backtracking tables - migrate no longer re…
nathan-probert Nov 3, 2026
2567682
feat(infra): hybrid ml-picks mode - source-env (read) and write-env (…
nathan-probert Nov 3, 2026
8b5a2f4
feat(ci): unconditional source=prod default in ml-picks resolve (owne…
nathan-probert Nov 3, 2026
1d8136f
feat(infra): ml-picks fills stat on the full slate (no table wipe); l…
nathan-probert Nov 3, 2026
67b13b3
test(unit): one shared run_daily_picks module object per session (fix…
nathan-probert Nov 3, 2026
2cf3f35
docs: verify the stat_update write mode live on dev + prod (row/col i…
nathan-probert Nov 3, 2026
d05b7c9
refactor(ml-picks): three-knob CLI (date/dry/env); stat_update is the…
nathan-probert Nov 3, 2026
966b994
feat(infra): Picks stat column carries the 0-1 decimal form (value/10…
nathan-probert Nov 3, 2026
a32c155
docs(ml-picks): state the 0-1 DB stat convention and fix the stale sc…
nathan-probert Nov 3, 2026
c6f7b66
fix(email): format Picks stat as a true percent (DB stat is a 0-1 fra…
nathan-probert Nov 3, 2026
24ef51b
feat(picks): add legacy_stat column preserving the site weighted-aver…
nathan-probert Nov 4, 2026
555c180
fix(historic-picks): strip legacy_stat from the Historic-Picks upsert…
nathan-probert Nov 4, 2026
d752517
revert: strip legacy_stat from Historic-Picks payload (superseded by …
nathan-probert Nov 4, 2026
44bafa0
feat(historic-picks): add legacy_stat column to Historic-Picks-{env}
nathan-probert Nov 4, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 30 additions & 2 deletions .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -190,8 +190,28 @@ jobs:
# that hardcodes one environment's tables and would silently hit
# prod during a dev deploy. Quote the table name for the suffix to
# match, e.g. "Picks-<env>", or use the __ENV__ placeholder.
if grep -nE '"[^"]*-(dev|prod)"' "$file"; then
echo "::error file=$file::Hardcoded dev/prod table-name suffix. Write the suffix as the __ENV__ placeholder so CI scopes the migration to $ENV."
#
# Exception: a file carrying the BACKTRACK-PROD-PINNED marker. The
# dev backtracking tables were deleted Oct 2026 (rows removed
# 2026-10-24; docs/backtrack_dev_deleted.md) and the standing
# migrate job must never recreate them, so their migrations target
# the '-prod' tables literally instead of via __ENV__ - otherwise a
# PR deploy with ENV=dev would re-CREATE the retired -dev shells.
if grep -q 'BACKTRACK-PROD-PINNED' "$file"; then
pinned=1
else
pinned=0
fi
if grep -nE '"[^"]*-dev"' "$file"; then
echo "::error file=$file::Hardcoded dev table-name suffix. Write the suffix as the __ENV__ placeholder so CI scopes the migration to $ENV (dev backtracking tables are deleted and must never be recreated)."
status=1
fi
if grep -nE '"[^"]*backtrack-__ENV__' "$file"; then
echo "::error file=$file::A backtrack table is scoped via __ENV__: a PR deploy (ENV=dev) would recreate the deleted -dev backtracking shell. Pin the statement to the -prod table literally and mark the file BACKTRACK-PROD-PINNED (see docs/backtrack_dev_deleted.md)."
status=1
fi
if [ "$pinned" -eq 0 ] && grep -nE '"[^"]*-prod"' "$file"; then
echo "::error file=$file::Hardcoded dev/prod table-name suffix. Write the suffix as the __ENV__ placeholder so CI scopes the migration to $ENV (only BACKTRACK-PROD-PINNED files may hardcode -prod: dev backtracking is deleted)."
status=1
fi
done
Expand Down Expand Up @@ -284,14 +304,22 @@ jobs:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-east-1

- name: Run deploy script
run: |
chmod +x build_scripts/deploy.sh
export GITHUB_DISPATCH_TOKEN="${MLPICKS_DISPATCH_TOKEN:-}"
./build_scripts/deploy.sh
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_ACCOUNT_ID: ${{ secrets.AWS_ACCOUNT_ID }}
# Forwarded by deploy.sh's CFN_PARAM_OVERRIDES when set; unset is
# allowed (MLPicksKick relays fail loudly per kick instead).
# Repo secret is MLPICKS_DISPATCH_TOKEN (Actions forbids secret
# names starting with GITHUB_); deploy.sh keeps its script-env
# name GITHUB_DISPATCH_TOKEN, which maps to the Lambda env var.
MLPICKS_DISPATCH_TOKEN: ${{ secrets.MLPICKS_DISPATCH_TOKEN }}

dev-integration:
needs: deploy
Expand Down
277 changes: 253 additions & 24 deletions .github/workflows/ml-picks.yml

Large diffs are not rendered by default.

13 changes: 13 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -46,3 +46,16 @@ data/*.err

# HTTP crawl cache - large, rebuilt from any crawl; never shipped anywhere.
data/cache/

# Model training caches, re-fetched from Supabase (see model.py).
data/ml/

# V16 experiment run residue (deleted in chore/clean-tree; the record lives
# in MODEL_NOTES.md + git history). Never re-track the pick dumps or per-fold
# scraps if they are regenerated.
smartscore/scripts/backtrack/exp_v16_pick*.pkl
smartscore/scripts/backtrack/exp_v16_perfold*.csv

# Mis-named MoneyPuck-derived own-xG cache: no code reads it (V7-shots and
# V8-own-xG were both rejected) and it is raw re-fetchable data.
data/moneypuck/
40 changes: 40 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -105,3 +105,43 @@ backtrack-publish:
@ENV=$(ENV) uv run python $(BACKTRACK)/local_store.py --publish $(SEASON)
@ENV=$(ENV) uv run python $(BACKTRACK)/local_store.py --publish-team $(SEASON)
@ENV=$(ENV) uv run python $(BACKTRACK)/local_store.py --publish-goalies $(SEASON)

# Owner-run ONE-SHOT top-up (stage-2 RAW per-game columns, migration 20261027):
# fills game_shots/game_goals/game_points/game_toi_min for ALL historical rows
# of Player-Snapshots-backtrack-$(ENV) from the local raw store
# (data/raw_nhl.sqlite player_games). Preconditions: the migration landed on
# the target tables (deploy.yml's migrate job, or owner psql) AND the local
# store is loaded/derived (make backtrack-load SEASON=... first). Idempotent
# (PK upserts on (date, player_id)); no DDL is performed here.
# make topup-raw-game-cols ENV=dev # dev first, spot-check, then ENV=prod
# make topup-raw-game-cols ENV=prod SPOT_CHECK=1 # verify-and-exit
# Runbook: docs/ml-picks-runbook.md (stage-2 apply paths).
topup-raw-game-cols:
@uv run python $(BACKTRACK)/topup_raw_game_cols.py --env $(if $(ENV),$(ENV),dev) $(if $(SPOT_CHECK),--spot-check,) $(if $(SKIP_SPOT_CHECK),--skip-spot-check,)
.PHONY: topup-raw-game-cols

# Owner-side OFFLINE store utility (no ML-picks role anymore): since the
# stage-2 raw-column change the picks path reads Supabase alone (the daily
# gap-fill is backfill_missing_days.py), but the local store still feeds
# train_export/publish_shot_cols. This target is the old refresh chain, kept
# for local store maintenance only: dump load + current-season crawl.
backtrack-refresh:
@echo "Refreshing the local raw store (dumps + current season crawl) - owner-side only"
@uv run python $(BACKTRACK)/refresh_store.py $(if $(THROUGH),--through $(THROUGH),)

# ---------------- On-demand ML artifact refresh --------------------------
# Owner-run, on-demand only: training NEVER happens on AWS or in CI, and no
# target here is scheduled. The workflow (.github/workflows/ml-picks.yml) is
# predict-only and reads the artifact this target's upload produces.
# docs: docs/ml-picks-runbook.md
MLS3_BUCKET ?= $(if $(ML_S3_BUCKET),$(ML_S3_BUCKET),smartscore-ml-artifacts)

# Local dry training + export + upload (the on-demand artifact refresh): the
# artifact-fact convention is s3://$(MLS3_BUCKET)/predict/ml/artifact_<through>_<seed>.joblib
# (names sort by vintage; the daily workflow reads the lexically latest .joblib).
# make train-ml-export ENV=prod SEASON=2023 THROUGH=2026-06-14 SEED=0
train-ml-export:
ENV=$(ENV) uv run --with lightgbm==4.7.0 python $(BACKTRACK)/train_export.py \
--env $(ENV) $(if $(SEASON),--season $(SEASON),) $(if $(THROUGH),--through-date $(THROUGH),) \
--seed $(if $(SEED),$(SEED),0) --upload --s3-bucket $(MLS3_BUCKET)
.PHONY: train-ml-export backtrack-refresh
46 changes: 44 additions & 2 deletions build_scripts/deploy.sh
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ LAMBDA_FUNCTIONS=(
"GetGoalies-$ENV"
"GetLineups-$ENV"
"SendEmails-$ENV"
"MLPicksKick-$ENV"
)


Expand Down Expand Up @@ -59,7 +60,33 @@ generate_smartscore_stack() {
exit 1
fi

# (The retired ML pick-pass parameter overrides (MLS3Bucket/MLEcrImageUri)
# are gone with the MLPicks Lambda; the Model pass now runs as the
# scheduled .github/workflows/ml-picks.yml workflow instead.)

# GITHUB_DISPATCH_TOKEN (optional): the fine-grained PAT (Actions: write,
# this repo only) used by the MLPicksKick Lambda to repository_dispatch
# 'ml-picks-game' into ml-picks.yml. Forwarded from the environment when
# set; when unset, create-stack falls back to the template's empty
# default and update-stack keeps the previously stored NoEcho value. The
# relay fails loudly per kick if it ends up empty.
CFN_PARAM_OVERRIDES=(ParameterKey=PosthogApiKey,ParameterValue="$POSTHOG_FEATURE_FLAG_KEY")
if [ -n "${GITHUB_DISPATCH_TOKEN:-}" ]; then
CFN_PARAM_OVERRIDES+=("ParameterKey=GithubDispatchToken,ParameterValue=${GITHUB_DISPATCH_TOKEN}")
fi

if aws cloudformation describe-stacks --stack-name "$STACK_NAME" &>/dev/null; then
# Fail fast with an actionable message when the stack is stuck in a
# (rollback) failed state: UpdateStack (and waiter) cannot recover that
# and every attempt just fails cryptically until the rollback is
# completed with continue-update-rollback (or the stack is deleted).
PRE_STATUS=$(aws cloudformation describe-stacks --stack-name "$STACK_NAME" --query "Stacks[0].StackStatus" --output text)
case "$PRE_STATUS" in
CREATE_FAILED|DELETE_FAILED|ROLLBACK_FAILED|UPDATE_ROLLBACK_FAILED)
echo "Error: stack $STACK_NAME is stuck in $PRE_STATUS; it cannot be updated."
echo "Recover first: aws cloudformation continue-update-rollback --stack-name $STACK_NAME"
exit 1 ;;
esac
echo "Updating CloudFormation stack $STACK_NAME..."
UPDATE_OUTPUT=$(aws cloudformation update-stack \
--stack-name "$STACK_NAME" \
Expand All @@ -72,12 +99,16 @@ generate_smartscore_stack() {
ParameterKey=BrevoSmtpKey,ParameterValue="$BREVO_SMTP_KEY" \
ParameterKey=BrevoFromEmail,ParameterValue="$BREVO_FROM_EMAIL" \
ParameterKey=FeatureSendEmails,ParameterValue="$FEATURE_SEND_EMAILS" \
ParameterKey=PosthogApiKey,ParameterValue="$POSTHOG_FEATURE_FLAG_KEY" \
"${CFN_PARAM_OVERRIDES[@]}" \
--capabilities CAPABILITY_NAMED_IAM 2>&1)

if echo "$UPDATE_OUTPUT" | grep -q "No updates are to be performed."; then
echo "No updates needed. Skipping wait."
else
# Surface the raw UpdateStack response. Its ValidationError text is the
# only place that names an offending resource for template/parameter
# problems; swallowing it made past rollbacks undiagnosable from CI.
echo "$UPDATE_OUTPUT"
echo "Waiting for CloudFormation stack update to complete..."
aws cloudformation wait stack-update-complete --stack-name "$STACK_NAME"
fi
Expand All @@ -94,7 +125,7 @@ generate_smartscore_stack() {
ParameterKey=BrevoSmtpKey,ParameterValue="$BREVO_SMTP_KEY" \
ParameterKey=BrevoFromEmail,ParameterValue="$BREVO_FROM_EMAIL" \
ParameterKey=FeatureSendEmails,ParameterValue="$FEATURE_SEND_EMAILS" \
ParameterKey=PosthogApiKey,ParameterValue="$POSTHOG_FEATURE_FLAG_KEY" \
"${CFN_PARAM_OVERRIDES[@]}" \
--capabilities CAPABILITY_NAMED_IAM

echo "Waiting for CloudFormation stack creation to complete..."
Expand All @@ -106,6 +137,17 @@ generate_smartscore_stack() {

if [[ "$STACK_STATUS" != "CREATE_COMPLETE" && "$STACK_STATUS" != "UPDATE_COMPLETE" ]]; then
echo "CloudFormation stack operation failed with status: $STACK_STATUS."
# Print the most recent failed-resource events so the failing resource
# + its ResourceStatusReason are visible directly in CI logs.
aws cloudformation describe-stack-events --stack-name "$STACK_NAME" --max-items 60 \
--query "StackEvents[?ResourceStatus=='UPDATE_FAILED'].[LogicalResourceId,ResourceType,ResourceStatusReason]" \
--output table &>/dev/null \
&& echo "(failed-resource events printed above if any)" \
|| echo "(describe-stack-events failed; run it manually with ResourceStatus=='UPDATE_FAILED')"
if [[ "$STACK_STATUS" == *ROLLBACK* || "$STACK_STATUS" == *FAILED* ]]; then
echo "Recovery: aws cloudformation continue-update-rollback --stack-name $STACK_NAME"
echo "Then inspect: aws cloudformation describe-stack-events --stack-name $STACK_NAME --max-items 40 --query 'StackEvents[?ResourceStatus==\`UPDATE_FAILED\`].[LogicalResourceId,ResourceType,ResourceStatusReason]' --output text"
fi
exit 1 # Exit with error
fi

Expand Down
Binary file added data/dumps/20132014.sql.gz
Binary file not shown.
Binary file added data/dumps/20142015.sql.gz
Binary file not shown.
Binary file added data/dumps/20152016.sql.gz
Binary file not shown.
Binary file added data/dumps/20162017.sql.gz
Binary file not shown.
85 changes: 85 additions & 0 deletions docs/backtrack_dev_deleted.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
# dev backtracking snapshots deleted (2026-10-24)

All three dev backtrack tables are now deleted (row-deleted; the Postgres
shells remain until a manual DROP). Any script that used to read a dev
backtrack table (language default `ENV=dev` in `config.py`) must be run with
`ENV=prod` / `--env prod` instead. Code resolving a deleted dev table name now
fails loudly via `smartscore/scripts/backtrack/deleted_env_tables.py`.


## What was verified before deletion (all read-only)

- Row counts: dev 617,310; prod 623,358.
- The delta is exactly the 6,048 rows with `date > 2026-06-14`
(2026-09-29..2026-10-24, the 2026-27 season dev never received). No publish
cadence gap inside the dev range.
- Distinct `(date, player_id)` key sets over the FULL dev range
(2013-10-01..2026-06-14): identical — 0 dev-only keys, 0 prod-only keys in
the overlap, no duplicate keys on either side, 0 date-level count diffs.
- 2,000 random overlapping keys: full-row value equality across all 37 columns
(including the 16 backfilled shot/TOI+identity columns) — 0 mismatches
(floats <= 1e-9 rel), 0 NULL-parity diffs.
- Column lists: dev and prod identical (37 = 37) — no dev-only column remained
to migrate.

## Team-Stats / Goalie-Snapshots dev deletions (2026-10-24, second pass)

- `Team-Stats-backtrack-prod` was published from the local store
(`local_store.py --publish-team`, `ENV=prod`, all 14 seasons incl. 2026-27):
34,636 rows, 2013-10-01..2026-10-24.
- `Goalie-Snapshots-backtrack-prod` was published the same way
(`--publish-goalies`): 37,044 rows, 2013-10-01..2026-10-22 (goalies carry no
2026-10-23/24 rows yet because those games had no box scores at publish time
— un-played, the known partially-filled-today source pattern).
- Random 500-key spot-checks per table (value-compare against freshly computed
local values, floats <=1e-9 rel / ints+text exact / NULL-parity): 0
mismatches. Full key-set check: prod is a strict superset of dev — 0
dev-only keys; prod-only keys are exactly the 336 team / 348 goalie 2026-27
rows. The Player-table precedent (6,048 extra 2026-27 rows) replicates.
- After verification, `Team-Stats-backtrack-dev` (34,300 rows) and
`Goalie-Snapshots-backtrack-dev` (36,696 rows) were row-deleted (year-chunked
PostgREST deletes, per-table backup SHA256 verified first); prod counts were
re-checked stable (34,636 / 37,044). Pre-delete gzip backups live in
`data/backups/` with SHA256s in `MANIFEST.md`.

## Remaining manual steps (shell DROP — owner-side, PostgREST cannot DDL)

PostgREST cannot DROP TABLE, so the three empty shells still exist in the
Supabase project (re-verified 0 rows on 2026-11-03). In the dashboard SQL
editor (or psql via SUPABASE_DB_URL) run the DROP SQL verbatim from
**docs/ml-picks-runbook.md, section 11, "Deleting the -dev backtrack table
shells"** (added 2026-11-03):

1. `DROP TABLE IF EXISTS "Player-Snapshots-backtrack-dev";`
2. `DROP TABLE IF EXISTS "Team-Stats-backtrack-dev";`
3. `DROP TABLE IF EXISTS "Goalie-Snapshots-backtrack-dev";`

The standing `migrate` CI job can no longer recreate them: every backtrack
migration in `supabase/migrations/` is hard-pinned to the `-prod` literal
suffix (`BACKTRACK-PROD-PINNED` marker per file), and the deploy.yml
"Guard migration files" step fails any migration that scopes a backtrack
table through `__ENV__` (a PR-deploy `ENV=dev` run used to re-CREATE the
retired `-dev` shells — that is the trap these edits close).

Then, to also reclaim the disk, run `VACUUM FULL` on the (now smaller)
remaining backtrack prod tables, or pause/restore the project (a brief
outage — scheduled maintenance, not a data path blocker).

## Restore / re-derivation paths

- Backups: `data/backups/backtrack-dev-Player-Snapshots-backtrack-dev.csv.gz`
(528 MB gz? see MANIFEST.md for row count + SHA256) plus the other two dev
backtrack tables as separate gzips. Restorable per table by re-upserting the
CSV batches (`on_conflict=date,player_id`).
- Everything the tables were derived from also lives in
`data/raw_nhl.sqlite` (692,514 player_games, 13.4M shifts), so a full
re-derivation is possible with `local_store.py --build --derive --publish`.

## Shrinking the Supabase disk

Postgres only reclaims disk after a VACUUM:
run `VACUUM FULL "Player-Snapshots-backtrack-dev";` (or delete+recreate the
table) in the dashboard SQL editor, or pause/restore the project. Without that,
the freed 617k rows show as dead tuples and the dashboard size will not drop.
Also drop the empty-named table if it should disappear from the dashboard
listing (PostgREST cannot DROP).
Loading
Loading