Test extend sklearn

.github/workflows/ci.yml

@@ -118,47 +118,3 @@ jobs:
         with:
           name: python-package-distributions
           path: dist/
-
-  publish-test:
-    needs: build
-    runs-on: ubuntu-latest
-    if: github.event_name == 'workflow_dispatch'
-
-    steps:
-      - uses: actions/checkout@v6
-
-      - name: Set up Python
-        uses: actions/setup-python@v6
-        with:
-          python-version: '3.11'
-
-      - name: Install dependencies
-        run: |
-          python -m pip install --upgrade pip
-          pip install poetry
-
-      - name: Download all the dists
-        uses: actions/download-artifact@v4
-        with:
-          name: python-package-distributions
-          path: dist/
-
-      - name: Check if version already published to TestPyPI
-        id: check-version
-        run: |
-          current_version=$(poetry version -s)
-          status=$(curl -s -o /dev/null -w "%{http_code}" "https://test.pypi.org/pypi/openmodels/${current_version}/json")
-          if [ "$status" == "200" ]; then
-            echo "Version $current_version already exists on TestPyPI, skipping publish."
-            echo "skip=true" >> "$GITHUB_OUTPUT"
-          else
-            echo "skip=false" >> "$GITHUB_OUTPUT"
-          fi
-
-      - name: Publish package to TestPyPI
-        if: steps.check-version.outputs.skip == 'false'
-        env:
-          TEST_PYPI_API_TOKEN: ${{ secrets.TEST_PYPI_API_TOKEN }}
-        run: |
-          poetry config repositories.testpypi https://test.pypi.org/legacy/
-          poetry publish --repository testpypi --username __token__ --password $TEST_PYPI_API_TOKEN

CHANGELOG.md

@@ -5,6 +5,33 @@ All notable changes to the OpenModels project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.1.0-alpha.22] - 2026-08-04
+
+### Removed
+
+- TestPyPI publishing: dropped the `publish-test` job from `.github/workflows/ci.yml` (built a package on every `workflow_dispatch` run and published it to `test.pypi.org` if that version wasn't already there) and the now-unused `testpypi-badge.json`/README badge that displayed the last-published TestPyPI version
+
+### Added
+
+- `roundtrip_fit()` test helper (`openmodels/test_helpers.py`): monkeypatches `fit()`/`fit_transform()`/`fit_predict()` on given estimator classes so their fitted state is replaced with the result of an openmodels serialize→deserialize round-trip, letting any existing test suite double as a round-trip fidelity check
+- `test/test_estimator_conformance.py`: runs scikit-learn's own generic `parametrize_with_checks()` battery against round-tripped instances of every estimator openmodels supports, with a strict, per-(estimator, check) xfail list distinguishing known openmodels gaps from pre-existing sklearn/check fragility
+- `test/upstream/`: reuses scikit-learn's own `cross_decomposition` test suite (`test_pls.py`) unmodified against `PLSRegression`/`PLSCanonical`/`CCA`/`PLSSVD` via a `conftest.py` that registers sklearn's test fixtures as a pytest plugin
+- `test/_estimator_construction.py`: shared registry of minimal constructor arguments for meta-estimators that can't be built with bare defaults (e.g. `estimator=` for `ClassifierChain`, `RFE`, `StackingRegressor`), replacing duplicated per-file special-casing across the smoke test modules
+- scikit-learn 1.9.0 added to the README/docs compatibility matrix
+- `test/test_serializer_base.py`, `test/test_birch_cftree.py`, `test/test_sparse_containers.py`, `test/upstream/cluster/`: regression coverage for the round-trip fixes below
+- 29 stale entries removed from `test/test_estimator_conformance.py`'s `KNOWN_ROUNDTRIP_XFAILS` now that the underlying gaps are fixed
+
+### Fixed
+
+- `PLSRegression`, `CCA`, `PLSCanonical`, and `PLSSVD` were missing `_x_std`/`_y_mean`/`_y_std` from `ATTRIBUTE_EXCEPTIONS`, so `predict()` on a round-tripped model silently used unfitted/default scaling statistics instead of the values learned during `fit()`
+- `test_others.py` estimator discovery now filters through `ALL_ESTIMATORS` so experimental-only estimators that become discoverable as a side effect of importing `sklearn.utils.estimator_checks` (e.g. `HalvingGridSearchCV`) aren't constructed without openmodels actually knowing how to serialize them
+- README: corrected the scikit-learn compatibility workflow description (it runs on-demand via `workflow_dispatch`, not on every push to `main` and weekly, since push/schedule triggers were removed) and fixed a stale placeholder clone URL
+- Numpy `dtype` object attributes (e.g. `SimpleImputer._fit_dtype`) failed to round-trip for any dtype other than `float64`: the type tag used to look up a deserializer was numpy's internal per-dtype subclass name (`Int64DType`, `BoolDType`, ...), which had no matching handler, so the value silently came back as a raw string instead of an `np.dtype`, and `transform()` later crashed with `'str' object has no attribute 'kind'`
+- Dict-valued attributes with non-string keys (e.g. `OrdinalEncoder._missing_indices: dict[int, int]`) lost their key types on deserialize, since JSON forces string keys and there was no handler to coerce them back; `transform()` then crashed indexing with a `str` instead of an `int`. Fixed generically in `SerializerMixin` for any `int`/`float`/`bool`/`str`-keyed dict, not just this one estimator
+- `BisectingKMeans`'s internal `_BisectingTree` centroids (and `KDTree` data) were silently widened from `float32` to `float64` on round-trip, since the top-level attribute-dtype tracking doesn't reach values nested inside these bespoke serializers; `predict()` then crashed with a Cython buffer dtype mismatch
+- `Birch`'s fitted `root_`/`dummy_leaf_` CF-tree was reduced to an empty, structure-less stub on deserialize (only 4 scalar config values were captured, no subclusters/centroids/leaf links); `partial_fit()` on a round-tripped model then crashed trying to concatenate zero leaf centroid arrays. Now serialized and deserialized recursively, including the cross-cutting doubly-linked list of leaf nodes
+- The sparse (de)serializer only recognized `scipy.sparse.csr_matrix`; `csc_matrix` and the newer array-API `csr_array`/`csc_array` containers raised `TypeError: ... is not JSON serializable` instead of round-tripping. Affected any estimator that stores its sparse training/fitted data verbatim (e.g. `KNeighborsClassifier`, `NearestNeighbors`, `DBSCAN`, `KernelRidge`)
+
 ## [0.1.0-alpha.21] - 2026-03-14
 
 ### Added

README.md

@@ -3,7 +3,6 @@
 [![PyPI version](https://badge.fury.io/py/openmodels.svg?cacheBust=1)](https://badge.fury.io/py/openmodels)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 [![Python Versions](https://img.shields.io/pypi/pyversions/openmodels.svg?cacheBust=1)](https://pypi.org/project/openmodels/)
-[![TestPyPI](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/Gnpd/openmodels/refs/heads/main/testpypi-badge.json)](https://test.pypi.org/project/openmodels/)
 
 OpenModels is a flexible and extensible library for serializing and deserializing machine learning models. It's designed to support any serialization format through a plugin-based architecture, providing a safe and transparent solution for exporting and sharing predictive models.
 
@@ -171,13 +170,14 @@ You can pass any compatible `all_estimators` function, list, or dictionary to `S
 
 ## scikit-learn Version Compatibility
 
-OpenModels is automatically tested against the following scikit-learn versions on every push to `main` and weekly via CI:
+OpenModels is tested against the following scikit-learn versions via the on-demand [`scikit-learn compatibility`](https://github.com/Gnpd/openmodels/actions/workflows/sklearn-compat.yml) CI workflow:
 
 | scikit-learn | Status |
 |---|---|
 | 1.6.1 | ✅ Tested |
 | 1.7.2 | ✅ Tested |
 | 1.8.0 | ✅ Tested |
+| 1.9.0 | ✅ Tested |
 
 If you encounter any incompatibility or a use case where the library does not work correctly with your version of scikit-learn, please [open an issue](https://github.com/Gnpd/openmodels/issues/new) — we would greatly appreciate your feedback!
 
@@ -196,7 +196,7 @@ To run the tests:
 1. Clone the repository:
 
    ```bash
-   git clone https://github.com/your-repo/openmodels.git
+   git clone https://github.com/Gnpd/openmodels.git
    cd openmodels
    ```
 

conftest.py

@@ -0,0 +1,25 @@
+# Registers sklearn's own conftest as a plugin - required because some of the sklearn test
+# modules reused wholesale under test/upstream/ (see test/upstream/cross_decomposition/ for the
+# worked example) depend on fixtures sklearn defines there, e.g. `global_random_seed`, which is
+# wired up via a pytest_generate_tests hook. pytest only allows pytest_plugins to be declared in
+# a conftest.py at the true rootdir, which is why this lives here rather than under test/.
+#
+# Recipe for reusing another sklearn test module (e.g. sklearn/svm/tests/test_svm.py):
+#   1. mkdir test/upstream/<package>/ with an empty __init__.py.
+#   2. test/upstream/<package>/conftest.py: a module-scoped autouse fixture wrapping
+#      openmodels.test_helpers.roundtrip_fit around the estimator classes that module's tests
+#      exercise, e.g.:
+#          @pytest.fixture(scope="module", autouse=True)
+#          def _roundtrip_something():
+#              with roundtrip_fit(EstimatorA, EstimatorB):
+#                  yield
+#   3. test/upstream/<package>/test_<module>_upstream.py, a one-liner:
+#          from sklearn.<package>.tests.test_<module> import *  # noqa: F401,F403
+#   4. Run it and skim the output. Not everything in an upstream test file is fair game as a
+#      round-trip check (some test private helper functions unrelated to any estimator - that's
+#      harmless bonus coverage) and not everything that fails is openmodels's fault (some fail
+#      identically against a bare, unpatched estimator - verify with roundtrip_fit removed
+#      before assuming a failure is a real serialization gap; see
+#      test/test_estimator_conformance.py's KNOWN_ROUNDTRIP_XFAILS for the established pattern
+#      of documenting which is which, with reasons).
+pytest_plugins = ["sklearn.conftest"]

docs/supported_models.md

@@ -1,7 +1,7 @@
 # Supported Models
 
 OpenModels currently supports **scikit-learn** estimators. The library has been tested against
-scikit-learn versions **1.6.1**, **1.7.2**, and **1.8.0**.
+scikit-learn versions **1.6.1**, **1.7.2**, **1.8.0**, and **1.9.0**.
 
 ## scikit-learn
 

openmodels/serializers/base.py

@@ -8,7 +8,7 @@
 """
 
 import numpy as np
-from scipy.sparse import csr_matrix  # type: ignore
+from scipy.sparse import csr_matrix, csc_matrix, csr_array, csc_array  # type: ignore
 from scipy.interpolate import interp1d, BSpline  # type: ignore
 from scipy.stats._distn_infrastructure import rv_continuous_frozen  # type: ignore
 import scipy.stats  # type: ignore
@@ -32,7 +32,17 @@ def convert_to_serializable(self, value):
 
         # Recursive case: dict, list, tuple
         if isinstance(value, dict):
-            return {str(k): self.convert_to_serializable(v) for k, v in value.items()}
+            if all(isinstance(k, str) for k in value):
+                return {k: self.convert_to_serializable(v) for k, v in value.items()}
+            # Non-string keys (e.g. int) can't be represented as JSON object keys without
+            # losing their type, so fall back to a self-describing keys/values envelope that
+            # _deserialize_dict can reconstruct exactly.
+            return {
+                "__openmodels_dict__": True,
+                "keys": [self.convert_to_serializable(k) for k in value.keys()],
+                "key_types": [type(k).__name__ for k in value.keys()],
+                "values": [self.convert_to_serializable(v) for v in value.values()],
+            }
 
         if isinstance(value, (list, tuple)):
             return [self.convert_to_serializable(v) for v in value]
@@ -98,6 +108,21 @@ def _deserialize_function(self, data: Dict[str, str]) -> Callable:
         module = __import__(data["module"], fromlist=[data["name"]])
         return getattr(module, data["name"])
 
+    def _deserialize_dict(self, value: Any) -> Any:
+        """Deserialize a dict, restoring non-string key types for the envelope produced by
+        convert_to_serializable's dict branch. Plain string-keyed dicts (the common case,
+        including dicts produced by older openmodels versions) pass through unchanged.
+        """
+        if isinstance(value, dict) and value.get("__openmodels_dict__"):
+            allowed_key_types = {"int": int, "float": float, "bool": bool, "str": str}
+            return {
+                allowed_key_types.get(kt, lambda x: x)(
+                    k
+                ): self.convert_from_serializable(v)
+                for k, kt, v in zip(value["keys"], value["key_types"], value["values"])
+            }
+        return value
+
     # --- Handlers ---
     def _get_serializer_handlers(self):
         """Each mixin extends this list."""
@@ -116,6 +141,7 @@ def _get_deserializer_handlers(self):
             ("str", str),
             ("type", self._deserialize_type),
             ("tuple", tuple),
+            ("dict", self._deserialize_dict),
             ("function", self._deserialize_function),
         ]
 
@@ -282,7 +308,14 @@ def _deserialize_bspline(self, data: Dict[str, Any]) -> BSpline:
     def _get_serializer_handlers(self):
         return [
             (BSpline, self._serialize_bspline),
-            (csr_matrix, self._serialize_csr_matrix),
+            # csr_matrix is the only sparse container openmodels round-trips on the wire, but
+            # any scipy sparse container (the older *_matrix family or the newer array-API
+            # *_array family) is accepted here - _serialize_csr_matrix normalizes it to csr via
+            # the csr_matrix(value) constructor, which accepts any sparse-like input.
+            (
+                (csr_matrix, csc_matrix, csr_array, csc_array),
+                self._serialize_csr_matrix,
+            ),
             (interp1d, self._serialize_interp1d),
             (rv_continuous_frozen, self._serialize_scipy_dist),
         ] + super()._get_serializer_handlers()