Add composition and composition+persistent-homology benchmark submissions - #363
Open
nkwork9999 wants to merge 1 commit into
Open
Add composition and composition+persistent-homology benchmark submissions#363nkwork9999 wants to merge 1 commit into
nkwork9999 wants to merge 1 commit into
Conversation
- matbench_v0.1_composition_gbm: matbench_glass, composition-only HistGradientBoostingClassifier (mean rocauc 0.8658). - matbench_v0.1_composition_ph_gbm: matbench_log_gvrh and matbench_log_kvrh, composition + persistent-homology structure features with HistGradientBoostingRegressor (mean MAE 0.0813 / 0.0636). All results use official Matbench v0.1 fold IDs and metrics. Each folder's src/ is a standalone, verified-runnable reproduction (dataset download, refit, and scoring reproduce the recorded results.json.gz exactly).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two new algorithm folders under
benchmarks/, both scored on the officialMatbench v0.1 fold IDs and metrics:
matbench_v0.1_composition_gbm—matbench_glass(glass-formingability), composition-only features (156-dim: element fractions + global
composition statistics) with
HistGradientBoostingClassifier. Meanofficial-fold ROC-AUC 0.8658 (± fold std ~0.017).
matbench_v0.1_composition_ph_gbm—matbench_log_gvrhandmatbench_log_kvrh(shear/bulk modulus), the same composition featuresplus a 51-dim persistent-homology structure descriptor (H0/H1
Vietoris-Rips diagrams on the periodic point cloud, via
ripser+persim)with
HistGradientBoostingRegressor. Mean official-fold MAE 0.0813(log_gvrh) and 0.0636 (log_kvrh); the PH features improved over a
composition-only control in all 5 folds on both tasks.
Methodology notes
matbench_v0.1_validation.jsonfold IDs(the local environment could not build
matbench==0.6's pinnedscikit-learn==1.0.1, so results were constructed to theMatbenchBenchmarkJSON schema directly from the raw datasets + officialfold JSON — same fold IDs, same metric functions).
training fold (never the fold's test data); the selected model was refit
on the complete official training fold before recording test predictions.
src/is standalone and was verified to reproduce therecorded
results.json.gzexactly (byte-identical per-fold predictionsand scores) by running fresh from inside that directory alone.
folder's
info.jsonandnotebook.md.Test plan
results.json.gzin both folders validated against theMatbenchBenchmarkschema (gzip integrity, official fold IDs/order,metric keys, finite predictions).
src/reproduction verified standalone (fresh dataset download +refit + score) to match the committed
results.json.gzexactly for allthree tasks.
matbenchpackage itself to build locally (see above), so if the schemaneeds any adjustment for merge I'm happy to iterate.