Skip to content

[ENH] simplify architecture of containers - #93

Draft
fkiraly wants to merge 20 commits into
mainfrom
simplify-containers
Draft

[ENH] simplify architecture of containers#93
fkiraly wants to merge 20 commits into
mainfrom
simplify-containers

Conversation

@fkiraly

@fkiraly fkiraly commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

The architecture of the BaseContainer, ModelContainer and its child classes is highly duplicative and repetitive.

This PR aims to reduce the container classes to a minimal manifest and make it as extensible to arbitrary containers for sklearn, sklearnex, cuml etc models as possible.

This is achieved by:

  • moving "database-like" information to tags (immutable) or properties (mutable)
  • moving dynamic information (e.g., based on presence of packages) to properties
  • move weird constructor call chain to cleaner inheritance, avoid cascading super.__init__ calls with different sets of variables for estimators
  • using BaseObject from scikit-base tag system, and _safe_import for materialization of blueprints

Currently experimental, with clustering being the scope for trying out new architecture.

Fully aware that parts of the refactor may have to be moved up to parent base classes in the hierarchy eventually, but keeping the scope restricted for now to find the best architecture.

@fkiraly fkiraly added implementing framework Implementing frameworks for new learning tasks enhancement labels Aug 24, 2026
Comment thread pycaret/containers/models/clustering.py Fixed
Comment thread pycaret/containers/models/clustering.py Fixed
Comment thread pycaret/containers/models/clustering.py Fixed
Comment thread pycaret/containers/models/clustering.py Fixed
Comment thread pycaret/containers/models/clustering.py Fixed
Comment thread pycaret/containers/models/clustering.py Fixed
Comment thread pycaret/containers/models/clustering.py Fixed
Comment thread pycaret/containers/models/clustering.py Fixed
Comment thread pycaret/containers/models/clustering.py Fixed
Comment thread pycaret/containers/models/clustering.py Fixed
Comment thread pycaret/containers/models/clustering.py Fixed
Comment thread pycaret/containers/models/clustering.py Fixed
Comment on lines +244 to +245
# if gpu_imported:
# KMeans = get_kmeans()
Comment on lines +367 to +370
# if not gpu_imported:
# args["n_jobs"] = experiment.n_jobs_param
# else:
# DBSCAN = get_dbscan()

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors PyCaret’s model container architecture (starting with clustering) to reduce duplication by moving model metadata to tags/properties and materializing estimator classes dynamically (e.g., via scikit-base’s tag system and _safe_import), aiming to make containers more extensible across backends (sklearn / sklearnex / cuML).

Changes:

  • Refactors clustering model containers to rely on tags/properties for class materialization and parameter/tuning manifests.
  • Updates BaseContainer to inherit from skbase.base.BaseObject and introduces legacy-compatible property-based accessors for class_def and args.
  • Loosens class_def constructor requirements by making it optional in base container/model classes.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
pycaret/containers/models/clustering.py Moves clustering containers toward tag/property-driven manifests and dynamic class materialization.
pycaret/containers/models/base_model.py Makes class_def optional in the base model container constructor.
pycaret/containers/base_container.py Switches to BaseObject and introduces property-based class_def/args access for compatibility and extensibility.
Suppressed comments (2)

pycaret/containers/models/clustering.py:230

  • gpu_imported is set to True unconditionally after _check_soft_dependencies(...). For severity="warning", _check_soft_dependencies returns False when cuml is missing, so this currently misreports GPU availability and may later select a non-importable class path.
        if experiment.gpu_param == "force" or experiment.gpu_param:
            if experiment.gpu_param == "force":
                severity = "error"
            else:
                severity = "warning"

pycaret/containers/models/clustering.py:349

  • Same as above: gpu_imported is set True even when _check_soft_dependencies(..., severity="warning") returns False. Also the log message references KMeans, but this is the DBSCAN container.
        if experiment.gpu_param == "force" or experiment.gpu_param:
            if experiment.gpu_param == "force":
                severity = "error"
            else:
                severity = "warning"

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread pycaret/containers/models/clustering.py
Comment thread pycaret/containers/base_container.py Outdated
Comment thread pycaret/containers/models/base_model.py Outdated
fkiraly and others added 3 commits August 25, 2026 21:41
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

Suppressed comments (1)

pycaret/containers/models/clustering.py:354

  • Same as in KMeans: gpu_imported is set to True even when _check_soft_dependencies(..., severity='warning') returns False, and the log message references KMeans instead of DBSCAN.
            _check_soft_dependencies("cuml", extra=None, severity=severity)

            logger.info("Imported cuml.cluster.KMeans")
            gpu_imported = True

Comment thread pycaret/containers/models/clustering.py
Comment thread pycaret/containers/models/clustering.py
Comment thread pycaret/containers/models/mlflow.db Outdated
@amotl
amotl requested a review from WilliamJudge94 September 4, 2026 18:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement implementing framework Implementing frameworks for new learning tasks

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants