Skip to content
PeterPonyuPublic

About

scRL: reinforcement learning to evaluate fate decisions in single-cell data

Topics

Resources

Contributing

Stars

9 stars

Watchers

1 watching

Forks

Repository files navigation

scRL: Single-Cell Reinforcement Learning

PyPI DOI Python Versions Documentation Status Publish to PyPI GitHub Release License: MIT

Overview

scRL (single-cell Reinforcement Learning) is a computational framework that applies reinforcement learning to single-cell RNA sequencing data to study cellular fate decisions. By treating cell differentiation as a sequential decision-making process, scRL is designed to help analyze when and where fate commitments emerge during development and disease.

Key Features

  • Actor-Critic Architecture: Uses deep reinforcement learning to model cellular differentiation trajectories
  • Grid-Based Embedding: Transforms high-dimensional single-cell data into an interpretable 2D manifold
  • Fate Decision Detection: Identifies early decision points that precede overt lineage commitment
  • Multiple Analysis Modes:
    • Gene Module: Analyzes gene expression patterns and regulatory roles
    • Lineage Module: Evaluates lineage commitments and differentiation potentials
    • Trajectory Module: Simulates and visualizes differentiation pathways
  • Flexible Reward Systems: Supports both contribution-based and decision-based reward patterns

Installation

Requirements

  • Python >= 3.9
  • PyTorch >= 1.13.1
  • CUDA support (optional, for GPU acceleration)

Core dependencies (automatically installed):

  • anndata >= 0.8.0
  • numpy >= 1.23.5
  • pandas >= 1.5.2
  • scikit-learn >= 1.2.1
  • networkx >= 2.8.8
  • matplotlib >= 3.6.3
  • seaborn >= 0.11.2
  • joblib >= 1.2.0
  • tqdm >= 4.64.1
  • pygam >= 0.8.0

Install from PyPI

pip install scrl-fatedecision

Note: For the examples in this README, you'll also need Scanpy with its Leiden clustering extra:

pip install "scanpy[leiden]"

Install from Source

git clone https://github.com/PeterPonyu/scRL.git
cd scRL
pip install -e .

Quick Start

This example follows the Basics of scRL tutorial. grids_from_embedding returns the grid results object gres; project_cluster, align_pseudotime, d_rewards, get_state_value and project_back store their results in gres and return None.

import scRL
import scanpy as sc

# Load and preprocess your single-cell data
adata = sc.datasets.paul15()
sc.pp.normalize_total(adata, target_sum=1e4)
sc.pp.log1p(adata)
sc.pp.scale(adata, max_value=10)
sc.pp.pca(adata, n_comps=50)
sc.tl.tsne(adata, perplexity=50)
sc.pp.neighbors(adata)
sc.tl.leiden(adata)

# 2-D embedding for the grids, cluster labels, and the latent space for the agent
X = adata.obsm['X_tsne']
clusters = adata.obs['leiden']
X_pca = adata.obsm['X_pca']

# Create grid embedding
gres = scRL.grids_from_embedding(X)

# Project cluster information onto grids
scRL.project_cluster(gres, clusters)

# Align pseudotime from the starting cluster
scRL.align_pseudotime(gres, early_cluster='4')

# Lineage rewards from the starting cluster towards the target clusters
scRL.d_rewards(gres, starts=['4'], ends=['0', '3', '5'], mode='Decision')

# Train the actor-critic agent on these rewards
trainer = scRL.trainer(
    algo='ActorCritic',
    gres=gres,
    reward_type='d',
    reward_mode='Decision',
    starts_prob=False,
    X_latent=X_pca,
    num_episodes=5000
)
returns, values = trainer.train()

# Fate decision values: critic state values on the grids, projected back to cells
scRL.get_state_value(gres, trainer, key='decision_value')
scRL.project_back(gres, 'decision_value')
adata.obs['decision_value'] = gres.embedding['decision_value']

The cluster labels come from the tutorial's Leiden clustering, where cluster '4' holds the starting cells and '0', '3' and '5' are the myeloid target clusters. Leiden labels depend on the clustering run, so set early_cluster, starts and ends from the clusters in your own data.

Documentation

Comprehensive documentation is available at https://scrl.readthedocs.io

Tutorials

API Reference

How It Works

scRL integrates three key components:

  1. Grid Embedding Generation: Transforms single-cell data into a structured 2D grid space using boundary scanning algorithms, preserving topological relationships while enabling efficient exploration.

  2. Reinforcement Learning Training: An Actor-Critic agent learns to navigate the grid environment, guided by rewards based on gene expression patterns or lineage commitments. The critic evaluates the strength of fate decisions at each state, while the actor identifies optimal differentiation routes.

  3. Functional Analysis Modules:

    • Gene Module: Explores how individual genes contribute to or determine cell fate
    • Lineage Module: Analyzes commitment to specific cell lineages
    • Trajectory Module: Simulates future differentiation paths using an autoencoder trained alongside the RL agent

Scientific Background

Understanding how cells develop into different types during growth and disease is a central problem in single-cell analysis. scRL frames cell development as a sequential decision-making problem to estimate when cells begin committing to future fates.

Reported Findings

  • Identifies early decision points before cells show obvious lineage commitment
  • Evaluated across hematopoiesis, cancer, organogenesis, and perturbation datasets
  • Includes analyses of regulatory factors associated with fate decisions

Citation

If you use scRL in your research, please cite:

@article{fu2025scrl,
  title={scRL: Utilizing Reinforcement Learning to Evaluate Fate Decisions in Single-Cell Data},
  author={Fu, Zeyu and Chen, Chunlin and Wang, Song and Wang, Junping and Chen, Shilei},
  journal={Biology},
  volume={14},
  number={6},
  pages={679},
  year={2025},
  doi={10.3390/biology14060679}
}

Support

License

This project is licensed under the MIT License - see the LICENSE file for details.

Contributing

Contributions are welcome! Please feel free to submit pull requests or open issues for bugs, questions, or new features.

About

scRL: reinforcement learning to evaluate fate decisions in single-cell data

Topics

Resources

Contributing

Stars

9 stars

Watchers

1 watching

Forks

Releases

Packages

Used by

Contributors

Languages