Pipeline needs to be able to differentiate between PipelineConfig data and StageConfig data. Current class methods (from_dict()) do not account for this.
Perhaps we need a wrapper class method called from_config which separates these behaviours and calls from_dict or something.
Pipeline needs an attribute to store StageConfig objects. (Mapping object)
Method to insert StageConfigurations if from_dict/from_config are not used to construct the pipeline. This is because users may use from_files() or just the generic Pipeline() constructor but still want to manage StageConfiguration prior to calling Pipeline.run()
We need a simple _create_stage_config() method which can be iteratively called by higher-level functions like the method mentioned in the paragraph above, since this behaviour is expected to be repeated.
Pipeline needs to be able to differentiate between PipelineConfig data and StageConfig data. Current class methods (from_dict()) do not account for this.
Perhaps we need a wrapper class method called from_config which separates these behaviours and calls from_dict or something.
Pipeline needs an attribute to store StageConfig objects. (Mapping object)
Method to insert StageConfigurations if from_dict/from_config are not used to construct the pipeline. This is because users may use from_files() or just the generic Pipeline() constructor but still want to manage StageConfiguration prior to calling Pipeline.run()
We need a simple _create_stage_config() method which can be iteratively called by higher-level functions like the method mentioned in the paragraph above, since this behaviour is expected to be repeated.