Add component registry and feature contracts by nictru · Pull Request #448 · daisybio/drevalpy

nictru · 2026-07-06T07:12:48Z

As the full modularity PR is quite large, I decided to split it into six smaller ones. Also these smaller PRs do not go directly into development, but rather first in a modularity-aggregated branch, which can then hopefully be merged into development relatively easily, as you already reviewed its smaller parts.

Below, you can find an AI-generated overview of the changes in this branch:

Summary

Introduces the component foundation for the modularity refactor: a new drevalpy.components package with registries, featurizer/predictor contracts, and shared feature utilities. This PR is additive only — existing MODEL_FACTORY models, drevalpy.models implementations, experiment workflows, and HPO behavior are unchanged.

This is PR 1 of 6 in the stacked modularity series. Later PRs will wire baselines, literature models, and structured HPO onto this layer.

What’s added

Component registry & contracts

Decorator-based registration for cell-line featurizers, drug featurizers, and predictors (drevalpy/components/registry/)
FeatureKind / FeatureContract for featurizer–predictor compatibility (contracts.py)
Extension loading hooks for third-party components (extensions.py)

Featurizers (implementation, not yet wired into MODEL_FACTORY)

Base classes and built-in featurizers under drevalpy/components/featurizers/ (omics layers, PCA, fingerprints, drug graphs, concat helpers, identity/tissue featurizers, etc.)

Predictor contracts (base only)

predictors/base.py, structured.py, baseline.py — interfaces only; no baseline or literature predictor migrations yet

Shared data & pair-batch utilities

drevalpy/data/features.py and preprocessing.py — feature loading/preprocessing helpers used by featurizers
pair_batch.py, pair_batch_build.py, pair_features.py — pair-level feature batching primitives

Minimal config & deps for the new layer

drevalpy/models/config.py — FeaturizerConfig / ModelConfig types needed by concat featurizers (no ComposedModel or factory wiring yet)
pyproject.toml — makes pydantic a required dependency
setup.cfg — flake8 per-file ignores for the new components/ paths

Registration

register_builtins.py registers featurizers only at this stage (no predictors or literature components yet)

What’s intentionally unchanged

MODEL_FACTORY and all existing drevalpy.models.* model classes
drp_model.py, experiment.py, HPO / YAML grids
Baseline and literature model implementations
CLI and public experiment entry points

Compatibility

All existing imports and MODEL_FACTORY behavior should match development. The new package is present but not used by the default model runtime until later PRs in the stack.

Tests

Focused unit tests for the new layer (45 tests), including:

Registry and metadata validation
Contracts, pair batches, config parsing
Featurizer identity/omics helpers
Feature data and preprocessing utilities

Suggested review focus

Are the component abstractions (Featurizer, Predictor, registries, FeatureContract) the right boundaries for later PRs?
Is the featurizer taxonomy (omics layers, concat, drug views) sensible before baseline/literature migration?
Is ModelConfig in the right place at this stage, or should more config orchestration wait for PR 2?

Drop view/backend/scope/node/edge fields from FeatureContract and match compatibility on feature kind alone. Update featurizer contracts and tests.

Introduce contract normalization/resolvers and attach featurizer and predictor contracts from registry decorators using canonical names.

Remove stale ClassVar imports after decorator contract migration, defer zoo loading until the zoo module exists, and satisfy mypy for registry tests and Ray resource typing.

Introduce view aliases, bracket-aware featurizer parsing, a generic raw pass-through featurizer, required-view PCA, and distinct concat block labels for repeated parametric featurizers.

Skip redundant TOY dataset re-downloads and ensure BPE/SMILESVec embeddings are created after TOY fixtures load, with synthetic fallbacks when BPE featurization fails.

Drop pass-through omics featurizers and methylationPCA in favor of raw[view] and pca[methylation]. Move scaled gene expression out of omics and register proteomics preprocessing as normalizedProteomics.

nictru · 2026-07-06T17:38:52Z

+    @classmethod
+    def distribute_legacy_state(
+        cls,
+        featurizer: ConcatFeaturizersCellLineFeaturizer,
+        state: dict[str, object],
+    ) -> None:
+        """Map legacy flat preprocessing state onto child featurizers when possible."""
+        from drevalpy.components.featurizers.cell_line.normalized_proteomics import (
+            NormalizedProteomicsCellLineFeaturizer,
+        )
+        from drevalpy.components.featurizers.cell_line.pca import PCACellLineFeaturizer
+        from drevalpy.components.featurizers.cell_line.scaled_gene_expression import (
+            ScaledGeneExpressionFeaturizer,
+        )
+
+        if not featurizer._children:
+            featurizer._materialize_children()
+
+        for name, child in featurizer._children:
+            if isinstance(child, ScaledGeneExpressionFeaturizer):
+                child_state = {key: state[key] for key in ("gene_expression_scaler", "fitted") if key in state}
+                if child_state:
+                    child.set_state(child_state)
+            elif isinstance(child, PCACellLineFeaturizer) and "methylation_pca" in state:
+                methylation_pca = state["methylation_pca"]
+                child_state = {
+                    "pca": methylation_pca,
+                    "view": child._view,
+                    "fitted": state.get("fitted"),
+                }
+                if hasattr(methylation_pca, "n_components"):
+                    child_state["n_components"] = int(methylation_pca.n_components)
+                    child_state["output_dim"] = int(methylation_pca.n_components)
+                child.set_state(child_state)
+            elif isinstance(child, NormalizedProteomicsCellLineFeaturizer):
+                child_state = {key: state[key] for key in ("proteomics_transformer",) if key in state}
+                if child_state:
+                    child.set_state(child_state)
+            _ = name
+        if state.get("view_dims") and isinstance(state["view_dims"], dict):
+            featurizer._block_dims = {str(key): int(value) for key, value in state["view_dims"].items()}
+        output_dim = state.get("output_dim")
+        if isinstance(output_dim, int):
+            featurizer._output_dim = output_dim
+        if state.get("fitted"):
+            featurizer._is_fitted = True
+
+    @classmethod
+    def collect_legacy_state(
+        cls,
+        featurizer: ConcatFeaturizersCellLineFeaturizer,
+    ) -> dict[str, object]:
+        """Flatten child featurizer state for legacy sklearn save/load."""
+        if not featurizer._children:
+            featurizer._materialize_children()
+        state: dict[str, object] = {"fitted": featurizer._is_fitted}
+        for _name, child in featurizer._children:
+            child_state = child.get_state()
+            if not isinstance(child_state, dict):
+                continue
+            for key, value in child_state.items():
+                if key != "fitted":
+                    state[key] = value
+        if featurizer._block_dims:
+            state["view_dims"] = dict(featurizer._block_dims)
+        if featurizer._output_dim:
+            state["output_dim"] = featurizer._output_dim
+        return state


Do we need all the legacy stuff here?

nictru · 2026-07-06T17:39:42Z

+        self,
+        *,
+        featurizers: list[Any] | None = None,
+        registry: str = "drug",


why is registry configurable? it should probably always be drug here

Require explicit contracts at registration, mark bpePharmaformer as literature, and drop the unused n_bits hyperparameter from view-based fingerprints.

Drug ID one-hot encoding is handled solely by identity, including in concat specs like fingerprints+identity.

Replace PairBatch with ModelInputBatch, add to_feature_matrix(), and make predictor fit/predict batch-based while introducing MatrixPredictor and BlockPredictor helpers.

Add component registry and feature contracts

37f6a9e

nictru mentioned this pull request Jul 6, 2026

Modular components stack with models/components split #443

Closed

4 tasks

nictru added 7 commits July 6, 2026 11:04

Simplify FeatureContract to kind-only compatibility

d7ea439

Drop view/backend/scope/node/edge fields from FeatureContract and match compatibility on feature kind alone. Update featurizer contracts and tests.

Add decorator-based feature contract registration

cbe9d8f

Introduce contract normalization/resolvers and attach featurizer and predictor contracts from registry decorators using canonical names.

Fix CI lint and type-check failures on component foundation branch.

7de92bb

Remove stale ClassVar imports after decorator contract migration, defer zoo loading until the zoo module exists, and satisfy mypy for registry tests and Ray resource typing.

Add explicit raw[view] and pca[view] omics featurizer grammar.

f1a864f

Introduce view aliases, bracket-aware featurizer parsing, a generic raw pass-through featurizer, required-view PCA, and distinct concat block labels for repeated parametric featurizers.

Fix flaky CI failures for PharmaFormer and Precily global model tests.

28e0f11

Skip redundant TOY dataset re-downloads and ensure BPE/SMILESVec embeddings are created after TOY fixtures load, with synthetic fallbacks when BPE featurization fails.

Fix conftest lint after CI embedding fixture hardening.

bba6ebc

Remove omics package; add normalizedProteomics and raw/pca grammar.

e9ab431

Drop pass-through omics featurizers and methylationPCA in favor of raw[view] and pca[methylation]. Move scaled gene expression out of omics and register proteomics preprocessing as normalizedProteomics.

nictru commented Jul 6, 2026

View reviewed changes

nictru added 3 commits July 7, 2026 16:37

Align drug featurizer registry contracts with cell-line featurizers.

d3e0e41

Require explicit contracts at registration, mark bpePharmaformer as literature, and drop the unused n_bits hyperparameter from view-based fingerprints.

Remove redundant drug oneHot featurizer in favor of identity.

73aed74

Drug ID one-hot encoding is handled solely by identity, including in concat specs like fingerprints+identity.

Unify predictor input around ModelInputBatch.

a8683ff

Replace PairBatch with ModelInputBatch, add to_feature_matrix(), and make predictor fit/predict batch-based while introducing MatrixPredictor and BlockPredictor helpers.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add component registry and feature contracts#448

Add component registry and feature contracts#448
nictru wants to merge 11 commits into
modularity-aggregatedfrom
modularity-1-component-foundation

nictru commented Jul 6, 2026

Uh oh!

nictru Jul 6, 2026

Uh oh!

nictru Jul 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

nictru commented Jul 6, 2026

Summary

What’s added

What’s intentionally unchanged

Compatibility

Tests

Suggested review focus

Uh oh!

nictru Jul 6, 2026

Choose a reason for hiding this comment

Uh oh!

nictru Jul 6, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant