Skip to content

feat: backtest excursions by horizon and outcome by feature - #104

Merged
yanivil merged 1 commit into
mainfrom
feat/backtest-horizons-and-features
Sep 8, 2026
Merged

yanivil merged 1 commit into
mainfrom
feat/backtest-horizons-and-features

Conversation

@yanivil

@yanivil yanivil commented Sep 7, 2026

Copy link
Copy Markdown
Owner

Description

Implements #92 on top of #103.

  • Excursions by horizon per window, overall and per pattern: median MFE and MAE within 5 / 10 / 20 / 40 / 60 bars after the fill, in percent and in ATR, plus the share of signals that had reached the target, hit the stop or done neither by then. Medians, because a few runaway winners dominate the means.
  • Features per replay row, measured at the scan day from the history the scan saw (row_features): close vs SMA200, SMA50 vs SMA200, the SMA200's change over 40 bars, distance from the SMA200 in ATR, stop and target distance in ATR, and bars from the pattern's last anchor (handle low, right shoulder, point 5, read from notes the way scan._drop_reason does) to the breakout. Nothing is added to scan.Signal, signals.json or the nightly report.
  • Outcome by feature per window: traded signals bucketed on fixed edges (FEATURE_BUCKETS) by those features plus volume ratio, risk %, reward:risk and breakout age, each bucket with N, hit rate, mean and median R and the month-block interval; empty buckets omitted, rows without a value counted as missing. Fixed edges keep two replays, or the two windows of a --split, comparable bucket by bucket.
  • summarise_rows is the former inner function of breakdown, now shared with the bucket table. JSON output gains features and horizons per window.
  • Wiki 04 describes both tables; changelog.

Verified locally: ruff clean, 124 tests pass (4 new), and a 30-ticker --period 5y --days 250 --bars 500 run renders both tables and writes the JSON with the feature keys on every row.

Related Issues

Closes #92

Checklist

  • Code follows existing project style and type annotations
  • Threshold changes include a one-line "why" comment in scan.py (no thresholds changed)
  • Offline unit tests pass (python -m pytest -q)
  • Random-walk false-positive rate stays under 5% (detection logic untouched)
  • Documentation updated in docs/wiki/
  • CHANGELOG.md updated

🤖 Generated with Claude Code

Each backtest window gains two tables. Excursions by horizon: median MFE and
MAE within 5 / 10 / 20 / 40 / 60 bars after the fill, in percent and in ATR,
with the share of signals at target, at the stop or neither by then, overall
and per pattern. Outcome by feature: traded signals bucketed on fixed edges by
features measured at the scan day from the history the scan saw (close vs
SMA200, SMA50 vs SMA200, SMA200 change over 40 bars, distance from SMA200 in
ATR, stop and target distance in ATR, bars from the last anchor to the
breakout) plus volume ratio, risk %, reward:risk and breakout age, each with
N, hit rate, mean and median R and the month-block interval.

Every replay row carries the features; nothing is added to scan.Signal or the
nightly report. summarise_rows is the former inner function of breakdown, now
shared with the bucket table.

Closes #92.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@yanivil
yanivil merged commit 7579203 into main Sep 8, 2026
5 of 6 checks passed
@yanivil
yanivil deleted the feat/backtest-horizons-and-features branch September 8, 2026 03:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Backtest: MFE / MAE by horizon and outcome by feature bucket

1 participant