MILAB-6818: export the study setup without its data - #132
Merged
Conversation
deriveTemplateParams carried a fully populated state whenever every file in it was a storage reference, and stripped it otherwise. That made the same export mean one thing for one study and another for the next. Always strip instead: datasets and metadata columns travel as configured, emptied of everything keyed by sample or by group, and the block a template seeds waits for its files while the user re-adds data into the datasets that are already there. Samples go with the files that produced them, so sampleIds and sampleLabels stay behind as well. Drops the portability machinery (isDatasetPortable, collectDatasetHandles) along with it.
AStaroverov
enabled auto-merge
August 25, 2026 12:39
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
deriveTemplateParamsnow always exports the study's setup and never its data.Before, it had two modes: if every file in the block was a storage reference (
index://), the whole state travelled — datasets with files, samples, sample labels, metadata values; otherwise everything keyed by sample or by group was stripped. The result was that the same export meant one thing for one study and another for the next, depending on where its files happened to live.Now there is one mode:
data,sampleGroups,groupLabelsandbarcodeRulesemptied;id/label/global/valueType, without values;sampleLabelColumnLabeltravels;sampleIds,sampleLabels,h5adFilesToPreprocessandseuratFilesToPreprocessstay behind — samples are created by importing files, so a sample arriving without its file is a row with metadata and no data.The block a template seeds is the study as it was set up, waiting for its files: the user re-adds data into the datasets that are already there instead of rebuilding them.
The portability machinery (
isDatasetPortable,collectDatasetHandles) is dropped with it.Not changed
BlockParamsin thekindpackage stays permissive — a fully populated dataset is still a valid param, it is just not something this block produces on export.Greptile Summary
The PR makes template exports deterministic by retaining study configuration while always removing sample-, group-, and file-associated data.
BlockParams— The optional initialization contract used to seed a block. This PR narrows whatderiveTemplateParamsproduces without changing the permissive contract itself.deriveTemplateParams— Converts current block state into template initialization parameters. It now always emits setup-only parameters rather than conditionally exporting a complete portable study.DSAny/DSContent— The dataset object and typed dataset-content union. Their configured fields are retained, while sample- or group-keyed data is cleared.data,sampleGroups, andgroupLabels.MultiplexedFastq— A grouped FASTQ dataset configured with barcode tags and assignment rules. Export retains its configuration but emptiesbarcodeRulesalong with group-bound data.MTColumn— A metadata column definition plus values keyed by sample ID. Export retains its identity, label, global flag, and value type while replacing values with an empty map.index://storage reference that allowed the complete study to be exported. Portability detection is removed because templates no longer export study data.Confidence Score: 5/5
The PR appears safe to merge, with no concrete blocking or non-blocking defect identified in the changed export path.
The exported dataset and metadata shapes remain valid, omitted fields receive empty initialization defaults, and grouped and multiplexing validation accepts the deliberately emptied state.
Important Files Changed
Flowchart
%%{init: {'theme': 'neutral'}}%% flowchart LR S[Current study state] --> D[Retain dataset configuration] D --> X[Clear data, groups, labels, and barcode rules] S --> M[Retain metadata definitions] M --> V[Clear metadata values] S --> L[Retain sample-label column name] X --> T[Setup-only BlockParams] V --> T L --> T T --> N[New study awaiting file imports]Reviews (1): Last reviewed commit: "MILAB-6818: export the study setup witho..." | Re-trigger Greptile
Context used: