feat: point-in-time index membership for replays, and --end for a past year on its own - #107
Merged
Merged
Conversation
…t year on its own tools/universe_history.py reconstructs the S&P 500 membership as of any date from the git history of the pinned constituent dataset: a bare, blobless clone under .cache/, the snapshot at the last commit on or before the date, symbols normalised like today's list through the new scan.symbols_from_csv. The backtest's --constituents-asof downloads the union of members over the window and scans a symbol only on the days it was a member; the report opens with a per-year coverage table (members, members with Yahoo history), which is the survivorship bias that remains. --end YYYY-MM-DD replays the sessions on or before a date so a past year is one dispatch. Workflow inputs asof, end and grid (grid=false skips the variants and the other profile's pass); job timeout 360 minutes. First use, 2022 on the index as it was (531 members, 481 with history): the tuned profile at +0.03 R on 175 signals, drawdown -38 R. Closes #95. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
6 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Implements #95.
tools/universe_history.py. Membership as of any past date from the git history ofdatasets/s-and-p-500-companies, the dataset the scanner already pins: a bare, blobless clone under.cache/(under 1 MB, one second, refreshed by a fetch on later runs; a failed refresh serves the cache), the file at the last commit on or before the date, parsed by the newscan.symbols_from_csvso snapshots are normalised exactly like today's list. Dates before the first commit get the first snapshot and are counted. The 2016 snapshot holds 179 symbols that are not members today.--constituents-asof(workflow inputasof): monthly snapshots over the window give the union to download;walk_forwardtakes amemberscallable and scans a symbol only on the days it was a member. The report opens with a per-year coverage table: members and how many have Yahoo history. That gap is the survivorship bias that remains, and a transient Yahoo miss counts the same way, which the wiki says.--end YYYY-MM-DD(inputend): the--dayssessions on or before a date, so one past year is one dispatch.scan_sessionsis the shared window logic.grid(default true) so a long single-profile replay skips the variants and the other profile's pass; job timeout 360 minutes..cache/ignored.First use, locally, 2022 on the index as it was (
--constituents-asof --period 10y --end 2022-12-30 --days 250 --bars 500 --profile tuned; 531 members, 481 with history, 10 minutes):The bear year the two-year replay never contained: expectancy near zero, inverse H&S negative, Wolfe the only positive detector. On this window the breakout-age (#98) and SMA-band (#99) splits show no effect, VIX above 25 was again the best bucket (#101), and 63 % of signals hit the stop within 60 bars. Not written into the wiki tables: the protocol wants a GitHub run for that, one dispatch per year.
Verified:
ruffclean, 134 tests pass (6 new: the commit list of a temporary dataset repository, the snapshot in force including the commit day and dates before history, Yahoo-format symbols, union and coverage, the cached clone and its refresh,walk_forwardmembers and--end,symbols_from_csv).Related Issues
Closes #95
Related: #96 (protocol), #98, #99, #101, #102 (hypotheses that now have a third window)
Checklist
scan.py(no thresholds changed)python -m pytest -q)docs/wiki/CHANGELOG.mdupdated🤖 Generated with Claude Code