Skip to content

feat(memory): plan-vs-actual — reconstruct a run's memory from the event stream and fail on drift (SKEEP-003 P2, S1.9) - #1074

Merged
michalharakal merged 1 commit into
developfrom
feature/1030-plan-vs-actual
Aug 23, 2026
Merged

michalharakal merged 1 commit into
developfrom
feature/1030-plan-vs-actual

Conversation

@michalharakal

Copy link
Copy Markdown
Contributor

Summary

SKEEP-003 slice S1.9 (milestone M1 #1002, PRD M1-F8 / M1-A8): the memory plan is compared with what a run actually allocated, and a drift beyond the tolerance fails — a plan that quietly diverges from reality is worse than no plan, so this is a CI assertion rather than a report nobody reads.

  • ActualMemory.from(events | RecordingTraceSink) replays the allocation stream and reconstructs: peak live bytes per scope, total churn, allocation counts per scope (a decode loop should add none in FORWARD after warm-up — the M1-A3 signature), and adapter bytes by kind, so the GGUF DEQUANTIZE_TO_FP32 over-allocates: 1.1B Q4_K_M needs >12 GB heap transiently (~4.4 GB legit) #782 class is countable rather than anecdotal.
  • PlanVsActual(plan, actual, tolerance = 10 %) compares the plan's resident sections with the peaks a run reached, renders a planned/actual/drift table marking violations, and check() throws with that table in the message — a CI failure explains itself without a rerun.

Evidence (PlanVsActualTest, 13/13 plan tests):

  • a run that matches the plan is within tolerance and check() is silent;
  • a run allocating twice the plan fails with +100 % and a marked table;
  • the reconstruction is correct about scope mechanics: the forward slab is one allocation and its per-step views are slices, not allocations;
  • adapter bytes are accounted and rendered (adapters: 96 MB — dequantize 96 MB);
  • the tolerance is configurable, defaulting to the PRD's 10 %.

This is the machinery M1-A8 needs; pointing it at the decode sample (once #1032 settles where that lives) closes M0-A1's Llama-1B number.

Test plan

Full local gate (scripts/pr-gate.sh, JDK 25) — all legs passed; results in the first comment.

Closes #1030

🤖 Generated with Claude Code

…ent stream and fail on drift (SKEEP-003 P2)

Milestone M1 (#1002), PRD M1-F8 / M1-A8. A plan that drifts from reality
is worse than no plan, so the comparison is a CI assertion rather than a
report nobody reads.

- ActualMemory.from(events | RecordingTraceSink): replays the allocation
  stream and tracks peak live bytes per scope, total churn, allocation
  counts per scope (a decode loop should add none in FORWARD after
  warm-up) and adapter bytes by kind (the #782 class, now countable).
- PlanVsActual(plan, actual, tolerance = 10 %): compares the plan's
  resident sections with the peaks a run reached, renders the table
  (planned / actual / drift, marking violations), and check() throws with
  that table in the message when any line drifts beyond the tolerance —
  so a CI failure explains itself without a rerun.
- PlanVsActualTest: a run that matches the plan passes; a run that
  allocates twice the plan fails with "+100 %" and a marked table;
  actuals are reconstructed correctly from the event stream (the forward
  slab is one allocation, its per-step views are slices, not allocations);
  adapter bytes are accounted and rendered; the tolerance is
  configurable. 13/13 plan tests.

Closes #1030

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@michalharakal

Copy link
Copy Markdown
Contributor Author

Local gate scripts/pr-gate.sh (JDK 25) on 8b9db55: all legs passed — jvmTest · apiCheck · JS/Wasm · linuxX64Test · assemble · Java consumer API tests.

Targeted: sk.ainet.lang.memory.plan.* 13/13.

@michalharakal
michalharakal merged commit 08a47be into develop Aug 23, 2026
14 checks passed
@michalharakal
michalharakal deleted the feature/1030-plan-vs-actual branch August 23, 2026 19:25
@github-actions

Copy link
Copy Markdown

📖 Documentation Preview

The documentation has been built successfully for this PR.

Generated Files:

  • Operator documentation: docs/modules/operators/_generated_/
  • JSON schema output: operators.json

Artifacts:

  • Download the documentation-preview-1074 artifact to view the complete documentation locally.

This comment will be updated automatically when the PR is updated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[S1.9] M1: plan-vs-actual — allocation-event totals vs MemoryPlan, printed; CI assertion (> 10 % fails)

1 participant