Refactoring scripts for decision models generation#74
Open
jhozzova wants to merge 5 commits into
Open
Conversation
Add JSON format detection Implement detect_format() function to distinguish between T4 and Legacy KTT JSON schemas. Add device compute capability lookup Implement word-boundary device name matching to automatically extract compute capability from GPU names. Uses regex to avoid false positives (e.g., 'RTX 500' won't match 'RTX 5000'). Falls back to --cc override when device unknown. Add lookup table for common GPUs.
Implement T4 format parser Add T4Parser class to extract tuning parameters, profiling counters, global/local sizes and compilation data from T4 JSON schema. Filters out performance metrics (time, power, energy) and ignored measurements (temperature, frequency, etc.) from profiling counters using exclusion lists. Implement Legacy format parser Add LegacyParser class to extract data from Legacy KTT JSON schema. Handles Configuration.Pairs for tuning params, ProfilingData.Counters for profiling data, GlobalSize/LocalSize and CompilationData from ComputationResults.
Implement build_dataframe() to convert parsed JSON results into training matrices. Combines tuning params, sizes, profiling counters and compilation data into ordered columns with deterministic sorting. Filters out configurations missing profiling data. Splits into X (tuning params) and Y (sizes + profiling counters + compilation data) for sklearn.
Update and document docopt interface for JSON input
Create README.md
Implement main JSON processing pipeline
Add full end-to-end pipeline that loads JSON data, detects format
(T4 vs Legacy), extracts device/compute capability, builds DataFrame,
trains ExtraTreesRegressor, and saves model + metadata.
Pipeline logic:
1. Load JSON file with error handling
2. Detect format using metadata field case
3. Create appropriate parser (T4Parser or LegacyParser)
4. Extract device name and compute capability (with optional override)
5. Extract kernel name for Legacy format (validates single kernel)
6. Build DataFrame with tuning params as X, sizes + counters +
compilation data as Y
7. Split into train/test sets (configurable via --test-size)
8. Train ExtraTreesRegressor with 10 estimators
9. Evaluate model and print metrics
10. Save model as .sav pickle file
11. Save metadata as .sav.metadata.json
Output naming:
- T4 format: {basename}_DT.sav
- Legacy format: {basename}_{kernel}_DT.sav
Metadata format: {"cc": float, "tp": [params], "pc": [targets]}
The orchestration script was used to make it easier to run generate_decision_tree_model.py as that one had multiple confusing and error-prone cmd arguments. Now generate_decision_tree_model.py takes only one (non-optional) argument and that is json file with data. The orchestration script is therefore unnecessary.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I propose refactoring the model generation scripts to consume KTT's native JSON output formats (Legacy and T4) instead of CSV files, with automatic schema inference and metadata extraction. This eliminates the manual column-counting workflow and provides forward compatibility with KTT's evolving output formats.
In the old model generation workflow requires users to:
-t 4:12 -c 13:50to specify which columns contain what data--ccPain points:
Proposed model generation:
.sav+.sav.metadata.json)Tested thoroughly. One known difference is that the new models have tuning parameters always in alphabetical order whereas before the order was the same as in csv file. That should not matter, as tuning parameters are later ordered alphabetically in ProfileBasedSearcher.py itself. The names of global/local size and compilation data are the same as in old model generation, so they can be located by profile-based searcher the same way as before. I have compared models created by those two versions and they have such a high correlation (not the same values, as training data differed) that I can conclude they are functionally the same.