Create dedicated tutorial for MLflow wrapper features (DAT-1025)

docs/source/client_api_content.rst

@@ -461,6 +461,61 @@ Organize resources with channels
 
 See also the tutorial notebooks: `upload_data.ipynb <https://github.com/SonanceAI/datamint-python-api/blob/main/notebooks/upload_data.ipynb>`_
 
+Working with Models
+--------------------
+
+``api.models`` is a thin facade over Datamint's MLflow-backed model registry:
+it wraps MLflow's ``RegisteredModel``/``ModelVersion`` objects in
+:py:class:`~datamint.api.endpoints.model_types.Model` /
+:py:class:`~datamint.api.endpoints.model_types.ModelVersion`, so you can
+register, list, and inspect models without knowing MLflow's object model.
+
+Register and list models
++++++++++++++++++++++++++
+
+.. code-block:: python
+
+    # Create a model (or fetch it if it already exists, the default behavior)
+    model = api.models.create("my-model", description="Segmentation model")
+
+    # Look up a model by name; returns None if it doesn't exist
+    model = api.models.get_by_name("my-model")
+
+    # List every registered model
+    all_models = api.models.get_list()
+
+    # Only models with a deployed image
+    deployed_models = api.models.get_list(only_deployed=True)
+
+Models are also created automatically when you pass ``--ai-model`` to
+:doc:`command_line_tools` (``datamint-upload``) with a name that doesn't
+exist yet.
+
+Inspect versions and metrics
+++++++++++++++++++++++++++++
+
+Each :py:class:`~datamint.api.endpoints.model_types.Model` can list its
+:py:class:`~datamint.api.endpoints.model_types.ModelVersion` objects, and each
+version exposes what it was trained for and how it performed:
+
+.. code-block:: python
+
+    model = api.models.get_by_name("my-model")
+
+    versions = model.get_versions()
+    latest = model.get_latest_version()          # highest version number
+    champion = model.get_latest_version(alias="champion")
+
+    print(latest.get_task_type())                 # e.g. "segmentation"
+    print(latest.get_supported_modes())            # e.g. ["auto", "interactive"]
+    print(latest.get_metrics())                    # e.g. {"val/dice": 0.87}
+
+``get_metrics()`` returns ``{}`` for versions with no training run behind
+them (for example, a model registered externally rather than trained through
+a Datamint :mod:`~datamint.lightning.trainers`), rather than raising.
+``Model.get_supported_modes()``/``get_metrics()`` are shortcuts that delegate
+to the latest version when you don't need a specific one.
+
 Deploy a registered model
 +++++++++++++++++++++++++
 
@@ -479,6 +534,9 @@ Use ``api.deploy.start()`` to deploy a model:
     deploy_job = deploy_job.wait()
     print("Deployment complete:", deploy_job.status)
 
+    # Check whether a model has a deployed image
+    model.is_deployed()
+
 Working with Users
 ------------------
 

docs/source/datamint.api.endpoints.rst

@@ -59,6 +59,11 @@ Models API
    :undoc-members:
    :show-inheritance:
 
+.. automodule:: datamint.api.endpoints.model_types
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
 Deploy Model API
 ----------------
 

notebooks/04_experiment_tracking/02_model_registry.ipynb

@@ -0,0 +1,259 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "556c43fc",
+   "metadata": {},
+   "source": [
+    "# Model Registry Tutorial",
+    "",
+    "`api.models` is a facade over Datamint's MLflow-backed model registry. It wraps MLflow's",
+    "`RegisteredModel` / `ModelVersion` objects in plain Python objects (`Model`, `ModelVersion`),",
+    "so you can register, list, and inspect models without learning MLflow's object model.",
+    "",
+    "This notebook covers:",
+    "",
+    "- Registering a model with `api.models.create()`",
+    "- Listing and finding models with `api.models.get_list()` / `get_by_name()`",
+    "- Filtering to deployed models with `only_deployed=True`",
+    "- Inspecting a model's versions with `model.get_versions()` / `get_latest_version()`",
+    "- Reading what a version was trained for: `get_task_type()`, `get_supported_modes()`, `get_annotation_specs()`",
+    "- Reading training metrics with `get_metrics()`",
+    "- Checking deployment status with `is_deployed()`"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ff7cc0e3",
+   "metadata": {},
+   "source": [
+    "## Setup",
+    "",
+    "Update `datamint` first if needed, then configure your API key."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "3d323bd4",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%pip install -U datamint --quiet"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "ef4c457d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from datamint import Api\n",
+    "\n",
+    "api = Api()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "547f3483",
+   "metadata": {},
+   "source": [
+    "## Register A Model",
+    "",
+    "Registering ahead of training gives you a stable name to reference later. `create()` returns the",
+    "existing model instead of raising when one with this name is already registered",
+    "(`exists_ok=True` by default)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "05e434aa",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "MODEL_NAME = \"tutorial_model_registry_demo\"\n",
+    "\n",
+    "model = api.models.create(MODEL_NAME, description=\"Model created for the model registry tutorial\")\n",
+    "model.name, model.description"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7cc966eb",
+   "metadata": {},
+   "source": [
+    "## List And Find Models",
+    "",
+    "`get_list()` returns every registered model; `get_by_name()` returns a single one, or `None` if it",
+    "doesn't exist."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "2571c3f9",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "all_models = api.models.get_list()\n",
+    "print(f\"Registered models: {[m.name for m in all_models]}\")\n",
+    "\n",
+    "found = api.models.get_by_name(MODEL_NAME)\n",
+    "missing = api.models.get_by_name(\"does-not-exist\")\n",
+    "found.name, missing"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c30b3598",
+   "metadata": {},
+   "source": [
+    "## Filter To Deployed Models",
+    "",
+    "Pass `only_deployed=True` to skip models that don't have a deployed image yet. See",
+    "`05_deployment/01_deploy_registered_model.ipynb` for how to deploy one."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "1c605a73",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "deployed_models = api.models.get_list(only_deployed=True)\n",
+    "print(f\"Deployed models: {[m.name for m in deployed_models]}\")\n",
+    "\n",
+    "model.is_deployed()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "fab1e7a0",
+   "metadata": {},
+   "source": [
+    "## Inspect Versions Of A Trained Model",
+    "",
+    "The rest of this notebook needs a model with at least one version behind it, typically one",
+    "registered by a Datamint trainer (see `06_end_to_end`) or by",
+    "`04_experiment_tracking/01_mlflow_manual_logging.ipynb`. Replace `MODEL_NAME` below with one",
+    "you've already trained."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "2b0203fc",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "MODEL_NAME = \"FracAtlas_adapted\"  # replace with a model you've already registered/trained\n",
+    "\n",
+    "model = api.models.get_by_name(MODEL_NAME)\n",
+    "if model is None:\n",
+    "    raise ValueError(f\"Model '{MODEL_NAME}' was not found. Train one first, e.g. via the 06_end_to_end notebooks.\")\n",
+    "\n",
+    "versions = model.get_versions()\n",
+    "print(f\"{MODEL_NAME} has {len(versions)} version(s)\")\n",
+    "\n",
+    "latest = model.get_latest_version()\n",
+    "latest.version, latest.run_id"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d3045992",
+   "metadata": {},
+   "source": [
+    "## What A Version Was Trained For",
+    "",
+    "If the version was logged with the `datamint` MLflow flavor (true for anything trained through a",
+    "Datamint trainer), you can read the task type, supported prediction modes, and annotation specs",
+    "straight from the model artifact, without inspecting the training run."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "3ef28bd9",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "print(\"Task type:\", latest.get_task_type())\n",
+    "print(\"Supported modes:\", latest.get_supported_modes())\n",
+    "print(\"Annotation specs:\", latest.get_annotation_specs())\n",
+    "\n",
+    "# Model.get_supported_modes() is a shortcut that delegates to the latest version\n",
+    "model.get_supported_modes() == latest.get_supported_modes()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "cdfdcb81",
+   "metadata": {},
+   "source": [
+    "## Training Metrics",
+    "",
+    "`get_metrics()` reads the metrics logged for this version's training run. It returns `{}` instead",
+    "of raising when there's no training run behind the version, for example a model registered from",
+    "outside Datamint."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "f984040b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "latest.get_metrics()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "ee63e20b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "model.get_metrics()  # shortcut, uses the latest version"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "5edb693f",
+   "metadata": {},
+   "source": [
+    "## Next Steps",
+    "",
+    "Once you have a model version you're happy with, alias it and deploy it, see",
+    "`05_deployment/01_deploy_registered_model.ipynb`. Registered models are also created",
+    "automatically when you pass `--ai-model <name>` to `datamint-upload` with a name that doesn't",
+    "exist yet."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": ".venv",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.12.13"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

notebooks/04_experiment_tracking/README.md

@@ -1,7 +1,8 @@
 # 04 — Experiment Tracking
 
-Logging experiments with MLflow through the Datamint backend.
+Logging experiments and managing the model registry with MLflow through the Datamint backend.
 
 | Notebook | Level | Description |
 |---|---|---|
 | [01_mlflow_manual_logging](01_mlflow_manual_logging.ipynb) | ![Intermediate](https://img.shields.io/badge/level-intermediate-yellow) | Log metrics, parameters, and model artifacts manually using `mlflow.set_tracking_uri("datamint://...")` |
+| [02_model_registry](02_model_registry.ipynb) | ![Intermediate](https://img.shields.io/badge/level-intermediate-yellow) | Register, list, and inspect models and versions with `api.models` |