fix: make predictive_maintenance hold up on the real AI4I CSV - #182
Merged
Merged
Conversation
The example reported F1 0.84, but that figure came from the synthetic fallback only. On the real ai4i2020.csv it scored precision 0.29 and F1 0.43 (10-seed mean): the synthetic generator drew rpm and torque independently and let tool wear run to 253 min, inflating failures to 16% against the real 3.4%, so the real imbalance was never exercised. - Train on the three modes the readings determine (HDF, PWF, OSF) and score against the full Machine failure label. TWF (random tool replacement) and RNF (0.1% random) cannot be predicted, and as positives they taught the net to alarm on high tool wear. - Draw 20% of training samples from the failure pool instead of 50%, and train for 300k iterations instead of 80k. At 50/50 the same target produces about 1.55x the false alarms. - Fit the synthetic generator to the real CSV: log-log rpm/torque relation (corr -0.88), 60/30/10 variant mix, tool replaced between 200 and 240 min. It now yields ~3.75% failures. Real CSV, seeds 1-10: precision 0.66, recall 0.80, F1 0.72 (was 0.29, 0.91, 0.43). Synthetic: F1 0.75. README and docs now report only measured figures, labelled by data source; plot regenerated. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015jDhsfLHi6epg599E5dD1h
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The example reported F1 0.84, but only on the synthetic fallback. On the real
ai4i2020.csvit scored precision 0.29 and F1 0.43 (10-seed mean). The synthetic generator produced 16% failures against the real 3.4%, so the imbalance the docs describe was never exercised.Changes
Machine failurelabel. TWF and RNF are random by construction.Results (seeds 1-10)
A full-batch Adam float MLP of the same shape reaches ~0.77 on the real CSV, so this is close to the architecture's ceiling. Recall is capped near 0.85: 52 of 339 real failures are TWF/RNF only.
Test plan
make releasewith g++ and clang++,-Werrorclean🤖 Generated with Claude Code
https://claude.ai/code/session_015jDhsfLHi6epg599E5dD1h