Repository navigation
[codex] Avoid rebuilding unchanged statuses during compilation - #215
Conversation
Local benchmark resultsReferences: parent The standard AirspeedVelocity CI uses Julia 1.12.1 on Ubuntu and disables downstream cases. The appendix below covers the common PSE benchmark names. Separate CI integration jobs use different configurations (for example PlantBiophysics 1 thread / 1,000 steps / fanout 20); the full integrations below use the scientific fixture/oracle family but are not exact replays of those jobs. Longer matched controlsOne matched control block, using trial medians:
XPalm's suite-context no-output case was +8.52%, +9.48%, and +16.32% slower than base in three passes. Two fresh ABBA pairs instead gave +0.1147% and +0.4354%, consistent with the longer control. The context discrepancy remains unexplained; it is not evidence of guaranteed regression-free execution. Allocation improves by 424,576 bytes / 4,588 allocations against base in that case, but the longer control remains 647,776 bytes / 11,216 allocations above parent. Full downstream integrationsOne measured warm scene per mode and revision. Construction, execution and table collection are separate phases. Seconds below are observations, not repeated timing estimates.
XPalm has architecture generation disabled and grows from 8 to 1,197 objects. Reference outputs produce 70,720 rows; even outputs=:none retains 4,456 internal rows. Candidate reference execution plus collection is about 4.30 seconds. A single observation cannot establish the causal overhead of requesting outputs, or exclude a small regression. PlantBiophysics uses one object and 8,760 synthetic hourly forcing steps: all outputs produce 113,880 rows, none produces zero. Its roughly 206 ms table collection dominates the roughly 14 ms execution; execution alone is not the total scenario cost. Checked state, topology, rows and trajectories agree across revisions. XPalm reference floating-point comparisons use rtol=atol=1e-8 with exact discrete checks; checked native cross-revision trajectories agree exactly. Source pins: XPalm Full XPalm execution allocates about 19.5 MB and 205,000 allocations less than base, but about 17.3 MB and 466,000 allocations more than parent. No-output collection adds 980,672 bytes / 34,415 allocations against base. These are allocation totals, not peak resident memory. Common CI benchmark ledgerEach cell gives the median of the available paired pass ratios, followed by their range. There is no pooled overall speedup. Some cases have only one pass; small cases can be timer-sensitive, and overlapping trial distributions are not formal significance tests. Per-trial sample counts vary. Allocation deltas use trial minima, not peak memory. All observed PSE cases are retained, including regressions. All common PSE cases: timings and allocation changes
Additional extended OptionalOne tests retain parent-relative slowdowns: absent sources +11–12%, present sources +23–30%. These are separate from the common-case ledger. The PR description also records the adverse MAESPA initialization observation and test-environment limitations. Raw samples and the complete local audit were preserved in an immutable evidence archive (SHA-256 |
Initial preparation rebuilt status recipes even when all declared fields and shared references were already present. Ordinary applications also prepared temporal/private view storage they did not use. This PR skips that work while preserving the batched preparation introduced in 3c04a39.
Statusuntil a missing default or changed reference requires a staged recipe; preserve supplied statuses and shared/indexed reference identities.mainalso exceeded 60 minutes, while a successful latest-Julia run of this PR took nearly 54 minutes for the tests alone.The runtime change is confined to compilation and its focused tests; the additional workflow change lets the existing suite finish on slower runners. Scientific models and benchmark bodies are unchanged.
Performance
These measurements describe the original optimization revision (
de089172) and its benchmark baselines, before the subsequent merge ofmainand CI-timeout change. They have not been rerun on the updated PR head.Targeted warmed preparation improves from 144.523 to 119.556 ms (17.28%) for width-64 bindings over 256 objects, and from 358.151 to 315.730 ms (11.84%) for fully populated width-128 bindings (30 samples each). These extended fixtures are separate from the standard CI suite.
The longer Many hard-call control improves 8.94% against the then-current
mainbut remains 19.04% slower than the commit preceding the original compilation change. The inherited cold compilation gain is retained: one width-128 first-run observation is 79.513 s before the original change, 8.068 s on thatmainbaseline, and 8.168 s on the optimization revision. Cold compiler-inclusive allocation is about 7.4–7.6 MB higher than thatmainbaseline. No universal speedup or absence of regression is claimed.A separate benchmark comment records CI-common cases and full XPalm / PlantBiophysics integrations, including residual regressions and sampling limits.
Validation of the updated PR head
The PR includes the current
mainand the CI-timeout fix. Fresh CI and downstream integration runs targetc2bc295dd5faa77eec1d7265e829b0d383b4d62f.mainand this PR still use the earlier API. The errors reject tupleoutputs_toand tupleOutputTo(...; vars=...)declarations before the corresponding allocation tests can run. This is the existing API mismatch resolved by Declare distributed outputs and clarify scenario authoring #216, rather than a status-preparation regression. The combined changes in Declare distributed outputs and clarify scenario authoring #216 have since passed all three fresh downstream integrations (XPalm, PlantGeom and PlantBiophysics) on520d903: integration run. This confirms compatibility with the successor API; this PR's own XPalm check remains red. No compatibility shim or skipped test was added here.Earlier local evidence
The original optimization revision completed adapted package suites: PlantSimEngine 4,498/4,498; XPalm 1,397/1,397; PlantBiophysics 369/369. XPalm includes the complete 4,160-step oracle and read-only static PNG comparison. Full XPalm and PlantBiophysics checked results agreed across all three revisions tested at that time.
These were Kaimon-managed runs in a shared pinned environment, not unmodified
Pkg.testinvocations: Aquastale_depsandpersistent_taskssubprocess checks were omitted; PSE ambiguity checking ran in-process, while downstream packages retained their existing disabled ambiguity checks. Optional PlantBiophysics ArchimedLight integration is outside the default runner. This historical local evidence does not assert GitHub CI success on the updated head or compatibility with later downstream changes.MAESPA initialization also preserved all 2,962 checked initial values/container trees and 17 carrier records against the then-current
main, including 38 producer-reference aliases. An inheritedmainfix changes 17 carrier fields relative to the earlier parent, so parent equality is not claimed. No scientific MAESPA timestep was validated. Its single cold initialization was 184.510 s on thatmainbaseline versus 205.888 s on the optimization revision (+11.59%); later fresh-scene observations were +11.14% and +0.16%.