codegen: hoist history-subscripted lazy-edge ta.* sites; hold-last source clock for change/mom/roc - #121
Merged
Merged
Conversation
TradingView rule (pinned 2026-09-03 with `lab tv` on NYSE:F 1D, feed
`lab bars NYSE:F 1D --around 2025-10-15 -n 200`): a stateful ta.* call
inside ANY expression operand of a top-level statement -- the RHS of a
Pine v6 lazy `and`/`or`, either ternary arm, nested comparisons -- is
evaluated on EVERY bar. Short-circuiting and branch selection gate only
the value, never the built-in's state, and `[1]` on it is the previous
BAR's value. Pins (TV trades vs model):
c = bar_index % 7 == 3 and close > ta.highest(high,5)[1] every-bar 9/9
(per-call wrong on 8)
... and close > ta.sma(close,5)[1] 25/25 (per-call 28)
... and close > ta.ema(close,5)[1] 23/23 (per-call 27)
v = bar_index % 7 == 3 ? ta.highest(high,5)[1] : na 38/39 (ring 2/39)
A stateful call inside an if/local-block/function body that does not
execute every bar IS execution-gated on TV (pinned separately); that
lowering is untouched. Production instance: robmagnaye14 `bullMSS =
setupAlive and dir == 1 and close > ta.highest(high, 10)[1]` (+ BOS /
bear variants), diverging on ~12 lanes incl. the ETH hard lane.
Lowering: `_lazy_edge_ta_hoist_plan` (codegen/ta.py) scans every
top-level statement expression (VarDecl value except var/varip,
Assignment target/value, TupleAssign value, ExprStmt, the head `if`
condition) and, for each ta.* site below a lazy edge at any depth, emits
`const auto _pf_every_bar_ta_N = <site>;` BEFORE the statement -- in
dynamic mode (bar magnifier / input_tf / script_tf) too, since it wraps
the existing `_use_precalc ? ... : compute/recompute` lowering. A direct
`[k]` on the hoisted call pushes its `_hist_call_*` Series there as well.
`_visit_func_call` / `_visit_subscript` read the local / Series while the
statement is lowered. Nested sites hoist inner-first.
Not hoisted (existing lowering kept): if/for/while/switch bodies, `else if`
conditions (an `if` inside the else local block), user-function bodies,
`var` initializers, request.security payloads, tuple-returning sites, and
the pinned lazy saturated ta.roc(close, 3) clock
(tests/test_lazy_saturated_roc*.py). Block-local arguments cannot occur
in top-level statements, so no site is skipped for that reason.
ta.py `_ta_site_uses_precalc`: hoisted sites are every-bar, so precalc
(the same evaluation) is valid for them again. The `and`-RHS ta.sma /
lazy-edge ta.ema opt-outs (pf-probe-oliver-dual-vol-sma: "eager precalc
is TV-incorrect", #58) encoded the refuted per-call clock for exactly the
top-level shape the lazyand-sma/-ema tapes measured; they now only reach
a static lazy-edge sma/ema that is not a top-level statement operand (UDT
field defaults). The affected pins in test_codegen_validation_fixes.py
and test_security_lazy_ta_under_conditionals.py::
test_chart_context_conditional_ta_is_inline are re-pinned to the every-bar
rule citing the tapes; the security-evaluator lazy rule is untouched.
tests/golden/matrix_eigen_pca.cpp regenerated: its top-level `if ... and
ta.crossover(lam, lamSma)` / crossunder now hoist (the only diff).
New tests/test_lazy_edge_ta_every_bar.py: generated-C++ structure for the
four pin strategies and the robmagnaye MSS/BOS shape, or/not/nesting/
ternary/if-condition/expression-statement shapes, the not-hoisted set,
compile-only checks against the engine headers, and an executable
synthetic-bars driver (no feed) proving 2 hits per shape in both the
precalc and dynamic runs where the per-call model gives 0.
Full suite with the engine env (PINEFORGE_ENGINE_INCLUDE / EIGEN / LIB,
engine 11e61d4): main 9a3f101 2681 passed / 2 skipped / 0 failed; this
branch 2706 passed / 2 skipped / 0 failed (+25 new tests).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dwbd239WT9zfcTLYM5UpUU
Claude-Session: https://claude.ai/code/session_01FjBPpCZh8ZRPCGq3dfkdyE
…hange/mom/roc Eight more cadence-7 ternary pins on NYSE:F 1D (lab tv, range 2025-04-01.. 2026-05-01; `v = bar_index % 7 == 3 ? <call> : na`, value exposed through the entry size; plus the lazy-`and` roc probe), scored per call against three models -- every-bar (A), hold-last source clock (B), ring of executions (C): roc A 0/38 B 38/38 C 0/38 (+ lazy-and entries A 25 / B 39 / C 27 of 39) change A 0/39 B 39/39 C 1/39 call 1 (first execution): no TV entry = na mom A 0/39 B 39/39 C 1/39 cum A 0/39 B 1/39 C 39/39 barssince A 26/39 B 33/39 C 39/39 valuewhen A 25/39 B 39/39 C 39/39 (B == C for a one-execution lookback) crossover A 31/39 B 39/39 C 39/39 cross A 18/39 B 39/39 C 39/39 rising A 26/39 B 20/39 C 39/39 (strictly monotonic over executed samples) None of the eight is every-bar. Measured on Cloud Run against the full population (2026-09-04, like-for-like vs the same lab bundle), the broad every-bar hoist of 334f8a5 fixed the pinned shapes (+21 tiers) but cost 170 tiers and 30 hard-lane regressions; 27 of the 31 regressed sources have ta.crossover/crossunder below a top-level lazy edge, the rest rising/falling, valuewhen, change/roc, and ta.sum. TradingView's clock under lazy evaluation is per ta.* family, so the hoist becomes an ALLOW-LIST: * LAZY_EVERY_BAR_TA = highest, lowest, sma, ema (pinned every-bar; lowest is highest's mirror): hoisted to `_pf_every_bar_ta_N` as in 334f8a5. * LAZY_SOURCE_CLOCK_TA = change, mom, roc (numeric source): computed from the CALL'S OWN `source[length]` history -- written only on bars where the call executes, held on skipped bars, na before the first execution -- through a generated `_PFLazySourceClock` + per-site `Series<double> _pf_lazy_src_hist_N` (held source recorded once per chart bar before the script body; `hist[length-1]` is the source at the most recent execution at or before bar-length; a runtime length indexes the Series at run time). Never precalcs. * LAZY_PER_EXECUTION_TA = cum, barssince, valuewhen, cross, crossover, rising + the exact mirrors crossunder, falling: the native only sees the samples of bars where the call executes, which is exactly the reached-only inline `compute` codegen already emits; now also opted out of precalc (a static `ta.cum` under a lazy edge used to precalc every bar in static mode). * Every other family keeps its existing lowering unchanged (inline compute when reached, precalc when static) until a tape pins it; `_lazy_edge_ta_hoist_plan()["skipped"]` lists them per script (UNPINNED_LAZY_EDGE_REASON). In the 31 regressed sources the only such family is ta.sum (3 sites). The source clock replaces the #64 `_PFLazySaturatedROC3Clock` (literal `ta.roc(close, 3)` under a plain `and` RHS only). Its saturated branch is the same clock (B), but its eager chart `close[3]` fallback before the first execution is refuted by call 1 of the value probes (TV: na, no entry), and the fallback for cadences shorter than the length is replaced by the held-source read. Re-pinned and disclosed: tests/test_lazy_saturated_roc*.py -> tests/test_lazy_source_clock*.py (the shadowed-close / named-length / or / ternary exclusions were conservatism of the literal-shape rule and now route too); the three compile-smoke collision cases follow the new generated names; the matrix_eigen_pca golden returns to main's text (its `ta.crossover`/ `crossunder` under a top-level `if ... and` are per-execution); the ta.rsi half of test_chart_context_conditional_ta_is_hoisted_every_bar stays inline (unpinned family). Executable synthetic-bars coverage (no feed): cadence-4 lazy edges over closes 100..122 give roc 7.272727273 / change 8 / mom hits 2 / one na first execution / cum 330 / barssince 2 in both the precalc and the dynamic run (every-bar would give 5.769 / 6 / 1090 / 7; a ring would still be na), and each stream lifecycle starts from the na state. Full suite with the engine env (PINEFORGE_ENGINE_INCLUDE / EIGEN / LIB, engine 11e61d4): 2703 passed / 2 skipped / 0 failed (334f8a5: 2706 / 2 / 0; main 9a3f101: 2681 / 2 / 0). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dwbd239WT9zfcTLYM5UpUU Claude-Session: https://claude.ai/code/session_01FjBPpCZh8ZRPCGq3dfkdyE
…ION_TA)
The last unpinned lazy-edge family in the 31 regressed sources was
`math.sum` (hexatrades, 3 sites; the analyzer maps it to the TA site
`math::Sum _ta_sum_N`, so the classifier reported the family as `sum`).
Cadence-7 ternary pin on NYSE:F 1D (lab tv, `v = bar_index % 7 == 3 ?
math.sum(close, 3) : na`, Size = round(v*100), 37 entries of 39
candidates):
bar date TV A every B hold C ring
3 2025-04-04 na 29.27 na na
10 2025-04-15 na 28.49 28.61 na
17 2025-04-25 29.07 29.88 28.94 29.07
24 2025-05-06 29.93 30.89 30.52 29.93
31 2025-05-15 31.23 32.00 31.63 31.23
38 2025-05-27 31.50 31.11 31.81 31.50
totals A 0/39 B 1/39 C 39/39
TradingView's math.sum below a lazy edge sums the last `length` EXECUTED
samples and is na until it has `length` of them: the per-execution ring,
i.e. the reached-only inline compute codegen already emits. `sum` joins
LAZY_PER_EXECUTION_TA (inline lowering unchanged; opted out of precalc so
static mode does not turn it every-bar). Not every-bar, not source clock.
Full suite with the engine env (engine 11e61d4): 2703 passed / 2 skipped /
0 failed (unchanged count: one family moved sets, one test extended).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dwbd239WT9zfcTLYM5UpUU
Claude-Session: https://claude.ai/code/session_01FjBPpCZh8ZRPCGq3dfkdyE
Cloud Run, engine + 3e461cd vs the corrected pin: +135 / -30 tiers, with four ETH hard-lane probes going excellent -> weak (louislapis9 100 -> 67.1, oliver1002 100 -> 75.2, quantbyboji 100 -> 87.5, ycelestine77 100 -> 59.4; quantbyboji/oliver1002 also on BTC/ES/NQ, miemomo3 BTC countAbsDelta 0 -> 1). Transpiling each source on main 9a3f101 and on the branch, every changed site is an allow-listed family below a lazy edge WITHOUT a history read: louislapis9 isAdxStrong = adxVal > 30 and adxVal > ta.sma(adxVal, 7) oliver1002 isSpringTest = ... and (volume < ta.sma(volume, 20)) isBOS = ta.crossover(...) and ... and (volume > ta.sma(volume, 20) * 1.2) (the original pf-probe-oliver-dual-vol-sma pin) ycelestine77 baseLong/Short = ta.mom(close, 20) > 0 and ta.change(ta.sma(close, 50)) > 0 quantbyboji five ta.ema(close, 200) under `or` (ema_crossover) + two ta.ema(close, len) under `and`; plus ta.roc(close, 3) > 0 miemomo3 time_exit = hold_days >= 3 and ta.highest(high, n) / entry_price > 1.05 whereas every every-bar tape (highest/sma/ema `[1]` under `and`, the ternary `ta.highest(high, 5)[1]`) and the robmagnaye production shape read `[k]` on the call. TradingView therefore keeps a lazily executed call on the reached-only clock unless the call's own history is referenced, in which case the built-in is evaluated on every bar (the history must be consistent). The hoist now requires BOTH the LAZY_EVERY_BAR_TA family and a direct `Subscript` on the call; other allow-listed sites are listed in `_lazy_edge_ta_hoist_plan()["skipped"]` under NO_HISTORY_READ_REASON and keep their existing lowering (inline compute when reached, precalc when static, the sma/ema opt-outs of main). quantbyboji's roc: the new source clock replaces #64 exactly in the saturated regime and, per the lead's call, keeps #64's eager chart `source[length]` read between executions closer than `length` bars, which no tape distinguishes -- a paired `_pf_lazy_src_chart_N` Series records the chart source once per bar for unshadowed bar-builtin sources (`previous_source(held, eager, length, bar)`; computed sources read the held history there). The only remaining behavioural delta vs main on this probe is the tape-pinned na before a site's first execution (#64 used the eager value; call 1 of every value probe has no TV entry). Result on the five sources: louislapis9, oliver1002, miemomo3 now transpile byte-identically to main; ycelestine77 differs only by the change clock (length 1 == the previous execution == main's ring); quantbyboji only by the roc clock above. robmagnaye keeps its four hoists. Tests: the no-`[k]` re-pins in test_codegen_validation_fixes.py return to main's assertions (docstrings cite both observations); the `[k]` shapes keep the every-bar pins; test_lazy_edge_ta_every_bar.py shapes read `[1]`, plus a test of the five probe shapes staying inline; test_lazy_source_clock*.py pin previous_source / the chart Series / collisions and an executable gap<length case (5.660377358). Full suite with the engine env (engine 11e61d4): 2706 passed / 2 skipped / 0 failed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dwbd239WT9zfcTLYM5UpUU Claude-Session: https://claude.ai/code/session_01FjBPpCZh8ZRPCGq3dfkdyE
…n change)
Four more cadence-7 probes on NYSE:F 1D (lab tv, default range) are the
subscript-free twins of the every-bar tapes, scored per call against the
every-bar (A), hold-last (B) and ring-of-executions (C) models:
c = %7==3 and close > ta.sma(close, 5) A 23/39 B 25/39 C 39/39 (26 entries; [1] twin A 39/39)
c = %7==3 and close > ta.ema(close, 5) A 26/39 C 39/39 (28 entries; [1] twin A 39/39)
v = %7==3 ? ta.sma(close, 5) : na A 2/39 B 1/39 C 39/39 (35 entries: na until five samples)
v = %7==3 ? ta.highest(high, 5) : na A 11/39 B 25/39 C 38/39 (38 entries: TV na with one executed
sample, then the partial maximum)
So the discriminator of 41ce7e7 holds on tape too: a bare call below a lazy
edge runs per execution (reached-only inline compute, #58's precalc
opt-out); only a history-subscripted call is materialised as a continuous
series and evaluated every bar. robmagnaye (four `[1]` hoists) and
blackelonmusk (`close > ta.highest(high, 8)[1]`, `ta.highest(high, 10)[1]`,
`ta.lowest(low, 8)[1]`: three hoists) keep hoisting.
Generated code is byte-identical to 41ce7e7 (comment-only change in
codegen/ta.py). The four bare scripts are pinned structurally in
tests/test_lazy_edge_ta_every_bar.py. Engine note, not codegen:
`ta::Highest` returns na until its window is full, while TV's lazily
executed `ta.highest` returns the partial maximum from the second executed
sample on (calls 2-4 of the bare highest tape).
Quick suite 1839 passed / 560 skipped; engine-env suite unchanged from
41ce7e7 (2706 / 2 / 0) since codegen did not change.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dwbd239WT9zfcTLYM5UpUU
Claude-Session: https://claude.ai/code/session_01FjBPpCZh8ZRPCGq3dfkdyE
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TV rules (all pinned with
lab tvon NYSE:F 1D, 39-call cadence tapes; verified independently by two agents)and/orRHS or a ternary arm in top-level scope, a history-subscripted stateful call (ta.highest/lowest/sma/ema(...)[k]) is evaluated every bar ([1]twins: 9/9, 25/25, 23/23, 38/39). A bare call advances only when reached (sma/ema per-execution FIFO/recursion 39/39 each; highest/lowest over the bar-indexed ring 39/39). Every regressed production shape (louislapis9, oliver1002, ycelestine77, quantbyboji on the ETH hard lane) was a bare call; every every-bar tape carried[1].ta.change/mom/rocunder a lazy edge run on TV's hold-last source clock (the call's own source series, held on skipped bars, na before the first execution): roc 38/38, change 39/39, mom 39/39 (every-bar 0/38).cum/barssince/valuewhen/cross/crossover/rising/falling/math.sumare per-execution (39/39 each) — unchanged lowering, precalc opt-out.Commits
334f8a5 every-bar hoist (superseded) → 3e461cd allow-list + source clock → ebb2367 math.sum → 41ce7e7 hoist keyed on the
[k]read → c3bdba3 bare-twin pins (tests only). Production effect (Cloud Run, full population, like-for-like): robmagnaye14 v2/v5 and blackelonmusk → excellent on every lane; no hard-lane regression attributable to codegen.Full suite with engine env: 2710 passed / 2 pre-existing skips / 0 failed.
🤖 Generated with Claude Code
https://claude.ai/code/session_01Dwbd239WT9zfcTLYM5UpUU