Skip to content

Forest serialization - #63

Merged
jemus42 merged 10 commits into
masterfrom
forest-serialization
Jul 14, 2026
Merged

Forest serialization#63
jemus42 merged 10 commits into
masterfrom
forest-serialization

Conversation

@jemus42

@jemus42 jemus42 commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator

Implements #52

fit <- rpf(mpg ~ ., data = mtcars)
blob <- rpf_marshal(fit)          # plain R list, saveRDS-safe
saveRDS(blob, "model.rds")
restored <- rpf_unmarshal(readRDS("model.rds"))
predict(restored, mtcars)         # bit-identical to original

Hard requirement to get the mlr3extralearner back.
Unfortunately not entirely trivial and a few scenarios are covered:

Mode Usage Blob contains After restore
Prediction-only (default) blob <- rpf_marshal(fit) leaves + bounds predict() works; purification permanently unavailable
Purified lean (recommended) purify(fit); blob <- rpf_marshal(fit) leaves + bounds + purified grids purified predict() and predict_components() work. No training data stored, no re-purification on load
Purify-later (opt-in) blob <- rpf_marshal(fit, include_data = TRUE) leaves + bounds (+ purified grids if purified) + training X/Y everything, including purify() after restore

In particular I wanted the "don't care about purification" path because that's important for settings like the benchmark(s) we did in the past where only prediction is relevant and serialization is potentially needed (for encapsulation with callr as done in my large survival benchmark, or maybe parallelization across mirai or something along those lines). in that setting forcing purification would be wasted time, same as carrying around the information needed for purification later.

jemus42 and others added 10 commits July 14, 2026 00:55
)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…uctor

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Post-review hardening of the marshal boundary:

- set_model: reject leaf interval matrices narrower than feature_size
  (previously an unchecked out-of-bounds read on corrupt blobs)
- set_grid_leaves: reject non-positive grid dims and values matrices whose
  dimensions do not match the grid (previously OOB read or silent zero-fill)
- set_training_data: reject X/Y shape mismatches (previously reached purify
  and read out of bounds)
- rpf_marshal: record packageVersion(); rpf_unmarshal warns when the blob
  was written by a newer package version. Blob format version stays the
  hard compatibility gate.
- export_forest objects no longer store the forest twice in the blob;
  $forest is rebuilt on unmarshal.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@jemus42
jemus42 merged commit dcbc14a into master Jul 14, 2026
9 checks passed
@jemus42
jemus42 deleted the forest-serialization branch July 14, 2026 11:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant