diff --git a/CHANGELOG.md b/CHANGELOG.md index eebcd41..7368cd1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,6 +14,7 @@ and this project adheres to [Semantic Versioning][]. - `embpy.store.actions.compile_action_table` is now the shared headless action compiler, with explicit RESOLVED / CONTROL / UNRESOLVED status accounting and JSON sidecars. - `embpy.resources.gene.control.ControlPolicy` and status-aware embedding APIs add clearer handling for control and unresolved identifiers. +- `AlphaGenomeWrapper` (Google DeepMind's cloud-API DNA model) and `ScoobyWrapper` (gagneurlab/scooby, single-cell-resolution DNA sequence model) are now registered in `DNA_MODELS`/`MODEL_REGISTRY` under the keys `alphagenome`, `scooby_onek1k`, `scooby_neurips`, and `scooby_epicardioids`, reachable through `BioEmbedder.embed(model=...)` and `list_available_models()` like any other model. Each is gated behind its own optional extra (`embpy[alphagenome]`, `embpy[scooby]`) and standalone pixi environment (`pixi install -e alphagenome` / `-e scooby`) -- neither is pulled in by `embpy[all]` or the default/gpu/dev pixi environments, since AlphaGenome needs an external `ALPHAGENOME_API_KEY` and Scooby installs from a git URL. ### Changed diff --git a/docs/index.md b/docs/index.md index 78a1796..f692932 100644 --- a/docs/index.md +++ b/docs/index.md @@ -24,4 +24,5 @@ notebooks/genes notebooks/proteins notebooks/small_molecules notebooks/cells +notebooks/variant_effects ``` diff --git a/docs/notebooks/variant_effects.ipynb b/docs/notebooks/variant_effects.ipynb new file mode 100644 index 0000000..12d7ce0 --- /dev/null +++ b/docs/notebooks/variant_effects.ipynb @@ -0,0 +1,558 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "vep-00", + "metadata": {}, + "source": [ + "# Variant effects with Borzoi\n", + "\n", + "This notebook shows embpy's lower-level genomics API for sequence-to-function variant-effect work with [Borzoi](https://www.nature.com/articles/s41588-024-02053-6) (Linder et al. 2025, *Nat. Genet.*).\n", + "\n", + "## What this notebook demonstrates\n", + "\n", + "1. **Predicted profiles as an optional extra output.** `BorzoiWrapper.embed()` returns a pooled embedding by default, exactly like every other DNA model wrapper in embpy. Passing `return_profile=True` additionally runs Borzoi's prediction head and returns the full per-track, per-bin coverage profile (RNA-seq / ATAC / ChIP), the same output the Borzoi paper use for variant-effect analysis.\n", + "2. **`SNPEmbedder` with ref/alt as the default output.** `embed_snp()` always returns the reference and alternate-allele embeddings. Delta vectors (`alt - ref`) are computed by default too, but can be turned off (`compute_delta=False`), and a concatenated `[ref, alt]` feature vector is available on request (`compute_concat=True`).\n", + "3. **Profile-based variant-effect prediction.** `SNPEmbedder.predict_variant_effect()` implements the gene-level statistic from the Borzoi paper: sum the predicted (unsquashed) coverage over a gene's exon-overlapping bins for both alleles, then report the log2 fold-change per track.\n", + "\n", + "This notebook uses the lower-level `embpy.models.dna_models.BorzoiWrapper` and `embpy.tl.genomics` API directly rather than the high-level `BioEmbedder`, since variant-effect prediction works on raw genomic coordinates rather than gene/protein identifiers.\n", + "\n", + "**Compute note:** Borzoi's receptive field is 524,288 bp; a forward pass is a few GB of activations and is comfortably faster on GPU. `device=\"auto\"` below will use CUDA if available and fall back to CPU otherwise (slow but correct)." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "vep-01", + "metadata": { + "execution": { + "iopub.execute_input": "2026-06-23T12:29:47.798184Z", + "iopub.status.busy": "2026-06-23T12:29:47.797797Z", + "iopub.status.idle": "2026-06-23T12:34:59.877736Z", + "shell.execute_reply": "2026-06-23T12:34:59.876887Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Using device: cpu\n" + ] + } + ], + "source": [ + "from pathlib import Path\n", + "\n", + "import numpy as np\n", + "import pandas as pd\n", + "import torch\n", + "\n", + "from embpy.models.dna_models import BorzoiWrapper\n", + "from embpy.tl.genomics import (\n", + " SNPContext,\n", + " SNPEmbedder,\n", + " SequenceProvider,\n", + " genomic_to_bin_indices,\n", + ")\n", + "\n", + "OUTPUT_DIR = Path(\"outputs\")\n", + "OUTPUT_DIR.mkdir(exist_ok=True)\n", + "\n", + "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n", + "print(f\"Using device: {device}\")" + ] + }, + { + "cell_type": "markdown", + "id": "vep-02", + "metadata": {}, + "source": [ + "## Load Borzoi\n", + "\n", + "`borzoi_v0` (`johahi/borzoi-replicate-0`) is one of four published replicate folds; any of the `borzoi_v*` / `flashzoi_v*` entries in the DNA registry work the same way. Loading downloads the ~/.cache'd Hugging Face checkpoint on first use." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "vep-03", + "metadata": { + "execution": { + "iopub.execute_input": "2026-06-23T12:35:00.024420Z", + "iopub.status.busy": "2026-06-23T12:35:00.023194Z", + "iopub.status.idle": "2026-06-23T12:35:05.576255Z", + "shell.execute_reply": "2026-06-23T12:35:05.574963Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Loaded johahi/borzoi-replicate-0 | trunk dim = 1536 | bin size = 32 bp\n" + ] + } + ], + "source": [ + "wrapper = BorzoiWrapper(\"johahi/borzoi-replicate-0\")\n", + "wrapper.load(device)\n", + "print(f\"Loaded {wrapper.model_name} | trunk dim = {wrapper.TRUNK_OUTPUT_DIM} | bin size = {wrapper.BIN_SIZE} bp\")" + ] + }, + { + "cell_type": "markdown", + "id": "vep-04", + "metadata": {}, + "source": [ + "## 1. Predicted profiles as an optional extra output\n", + "\n", + "We center a window on a well-studied *cis*-regulatory region (the *SORT1* locus on chr1, harboring the lipid-trait variant rs12740374) and pad/crop it to Borzoi's exact 524,288 bp input length so the profile's bin coordinates line up exactly with genomic coordinates -- this matters later for variant-effect aggregation.\n", + "\n", + "`embed()` behaves like every other wrapper by default (returns only the pooled embedding). Passing `return_profile=True` additionally returns the raw per-track, per-bin coverage profile from `predict_profile()`." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "vep-05", + "metadata": { + "execution": { + "iopub.execute_input": "2026-06-23T12:35:05.578811Z", + "iopub.status.busy": "2026-06-23T12:35:05.578027Z", + "iopub.status.idle": "2026-06-23T12:35:05.977456Z", + "shell.execute_reply": "2026-06-23T12:35:05.976356Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Window length: 524288 bp | variant at offset 262145 | window_start=109012823\n" + ] + } + ], + "source": [ + "CHROM = \"chr1\"\n", + "VARIANT_POS = 109_274_968 # rs12740374, SORT1 locus (GRCh38; verified via Ensembl REST variation lookup)\n", + "REF, ALT = \"G\", \"T\"\n", + "\n", + "provider = SequenceProvider() # falls back to the Ensembl REST API; pass fasta_dir=... for local genomes\n", + "window, snp_offset = provider.get_window(CHROM, VARIANT_POS, context=BorzoiWrapper.SEQUENCE_LENGTH)\n", + "window_start = VARIANT_POS - snp_offset # 0-based genomic coordinate of window[0]\n", + "print(f\"Window length: {len(window)} bp | variant at offset {snp_offset} | window_start={window_start}\")" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "vep-06", + "metadata": { + "execution": { + "iopub.execute_input": "2026-06-23T12:35:05.979982Z", + "iopub.status.busy": "2026-06-23T12:35:05.979217Z", + "iopub.status.idle": "2026-06-23T12:39:20.671671Z", + "shell.execute_reply": "2026-06-23T12:39:20.670274Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Pooled embedding shape: (1536,)\n", + "Profile shape: (7611, 6144) (num_tracks, num_bins)\n" + ] + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
identifierdescription
0CNhs10608+CAGE:Clontech Human Universal Reference Total ...
1CNhs10608-CAGE:Clontech Human Universal Reference Total ...
2CNhs10610+CAGE:SABiosciences XpressRef Human Universal T...
3CNhs10610-CAGE:SABiosciences XpressRef Human Universal T...
4CNhs10612+CAGE:Universal RNA - Human Normal Tissues Bioc...
\n", + "
" + ], + "text/plain": [ + " identifier description\n", + "0 CNhs10608+ CAGE:Clontech Human Universal Reference Total ...\n", + "1 CNhs10608- CAGE:Clontech Human Universal Reference Total ...\n", + "2 CNhs10610+ CAGE:SABiosciences XpressRef Human Universal T...\n", + "3 CNhs10610- CAGE:SABiosciences XpressRef Human Universal T...\n", + "4 CNhs10612+ CAGE:Universal RNA - Human Normal Tissues Bioc..." + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "embedding = wrapper.embed(window, pooling_strategy=\"mean\")\n", + "embedding, profile = wrapper.embed(window, pooling_strategy=\"mean\", return_profile=True)\n", + "\n", + "print(f\"Pooled embedding shape: {embedding.shape}\")\n", + "print(f\"Profile shape: {profile.shape} (num_tracks, num_bins)\")\n", + "\n", + "track_meta = BorzoiWrapper.get_track_metadata()\n", + "display(track_meta[[\"identifier\", \"description\"]].head())" + ] + }, + { + "cell_type": "markdown", + "id": "vep-07", + "metadata": {}, + "source": [ + "Each row of `profile` is one experimental track (RNA-seq, ATAC, ChIP, ...); each column is a 32 bp genomic bin. Let's plot a couple of RNA-seq tracks around the variant to sanity-check the prediction looks like real coverage." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "vep-08", + "metadata": { + "execution": { + "iopub.execute_input": "2026-06-23T12:39:20.674454Z", + "iopub.status.busy": "2026-06-23T12:39:20.673657Z", + "iopub.status.idle": "2026-06-23T12:39:50.125269Z", + "shell.execute_reply": "2026-06-23T12:39:50.124662Z" + } + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA3kAAAGMCAYAAABnMGtWAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAnx5JREFUeJzs3Xd8FGX+B/DPbMmmJ3QIBEKRIr2DNAvIAaL87CIK3omIIirHKYgnWFE5T6QccnqKWBFRLCBIkya9GXrvEAIk2fTs7jy/PzYz2ZrsJrPJZvN5+8JkZ2ee55mym/nO0yQhhAARERERERGFBF1FF4CIiIiIiIi0wyCPiIiIiIgohDDIIyIiIiIiCiEM8oiIiIiIiEIIgzwiIiIiIqIQwiCPiIiIiIgohDDIIyIiIiIiCiEM8oiIiIiIiEIIgzwiIiIiIqIQwiCPiAImKSkJo0aNUl///vvvkCQJv//+e4WVyZVrGUPRzTffjJtvvrmii0EBsmPHDtx0002IioqCJEnYu3cvpk2bBkmSnNarCtd6WZ0+fRqSJGHBggUVXRQiojJhkEcUohYsWABJktR/4eHhaN68OcaNG4eUlJSKLp5fli9fjmnTplVoGRyPpSRJiIqKwo033og33ngDOTk5FVo2qrosFgvuu+8+XL9+He+//z4+//xzNGrUqKKLVSWcPn0ajz32GJo2bYrw8HDUrVsXffv2xdSpU93WFULg888/R9++fREfH4/IyEi0bdsWr732GrKzs93Wv/nmm52+byIiItCuXTvMnDkTsiwDcP9O8vZPeag2b9483HfffWjYsCEkSWLATxTiDBVdACIKrNdeew2NGzdGXl4eNm3ahHnz5mH58uXYv38/IiMjy7Usffv2RW5uLsLCwvzabvny5Zg7d26FB3oDBgzAo48+CgDIysrCxo0b8c9//hP79u3D4sWLK7Rsxfntt98quggUICdOnMCZM2fw0Ucf4fHHH1eXv/zyy5g0aVIFliy0HT9+HF27dkVERAT++te/IikpCZcuXcLu3bvxzjvv4NVXX1XXtdlsGD58OL799lv06dMH06ZNQ2RkJDZu3IhXX30VixcvxurVq1GnTh2nPBo0aIDp06cDAK5evYqvvvoKzz//PFJTU/Hmm2/i888/d1p/4cKFWLVqldvyVq1aAQDeeecdZGZmolu3brh06VIgDgsRBREGeUQhbtCgQejSpQsA4PHHH0eNGjXw73//Gz/++CMeeughj9tkZ2cjKipK87LodDqEh4drnm55ad68OUaMGKG+fvLJJ1FQUIDvv/8eeXl5muxbII69v0F1ZRGo67QiWa1WyLLs8zm7cuUKACA+Pt5pucFggMEQvH/ic3Jyyv0hk5bef/99ZGVlYe/evW41p8o5Ubz77rv49ttvMXHiRMyYMUNd/sQTT+D+++/HsGHDMGrUKPz6669O28XFxbl937Rs2RKzZ8/Ga6+95vQeAGzduhWrVq1yW65Yv369WosXHR1dqv0mosqDzTWJqphbb70VAHDq1CkAwKhRoxAdHY0TJ05g8ODBiImJwcMPPwwAkGUZM2fOROvWrREeHo46depgzJgxSEtLc0pTCIE33ngDDRo0QGRkJG655RYcOHDALW9vffK2bduGwYMHo1q1aoiKikK7du3wwQcfqOWbO3cuAOfmSQqty+ivunXrQpIktxvqxYsXo3PnzoiIiEDNmjUxYsQIXLhwwWkdb8fetamt4z/HvnVWqxWvv/46mjZtCpPJhKSkJLz00kvIz893ysefPnlffPEFunXrhsjISFSrVg19+/Z1qwn8z3/+g9atW8NkMiEhIQFPP/000tPT1ffHjRuH6Ohoj81YH3roIdStWxc2m01d9uuvv6JPnz6IiopCTEwMhgwZ4nZuirtON27cqDZDM5lMSExMxPPPP4/c3Fy3/BcvXowbb7wR4eHhaNOmDX744QeMGjUKSUlJTuv5el15opT15MmTGDhwIKKiopCQkIDXXnsNQgh1PaX/17/+9S/MnDlTPY8HDx4EAKxdu1Y9LvHx8bjrrrtw6NAhp3z69esHALjvvvucrg9PffI8SU9Px3PPPYfExESYTCY0a9YM77zzjtoksDg//vgjhgwZgoSEBJhMJjRt2hSvv/6607kF7NdfmzZtsGvXLvTt2xeRkZF46aWXANgDor/97W+oU6cOwsPD0b59e3z22WdO23v73vDUf0459hcuXMCwYcMQHR2NWrVqYeLEiW7lSk9Px6hRoxAXF4f4+HiMHDnS6TouzokTJ9CgQQOPTWNr166t/p6bm4sZM2agefPmaq2co6FDh2LkyJFYsWIFtm7dWmye4eHh6Nq1KzIzM90CSV80atTIp2uCiEJD8D7mI6KAOHHiBACgRo0a6jKr1YqBAweid+/e+Ne//qU+YR8zZgwWLFiAxx57DOPHj8epU6cwZ84c7NmzB5s3b4bRaAQAvPLKK3jjjTcwePBgDB48GLt378btt9+OgoKCEsuzatUq3HHHHahXrx6effZZ1K1bF4cOHcIvv/yCZ599FmPGjMHFixc9NkMqrzIq8vLycPXqVQD2WqTNmzfjs88+w/Dhw52CPKU8Xbt2xfTp05GSkoIPPvgAmzdvxp49e5xqXTwd+169ernt65kzZ/Dyyy873UA+/vjj+Oyzz3Dvvffi73//O7Zt24bp06fj0KFD+OGHH3zeL8Wrr76KadOm4aabbsJrr72GsLAwbNu2DWvXrsXtt98OwB48vPrqq+jfvz/Gjh2LI0eOYN68edixY4d6vB944AHMnTsXy5Ytw3333aemn5OTg59//hmjRo2CXq8HAHz++ecYOXIkBg4ciHfeeQc5OTmYN28eevfujT179jgFX96u08WLFyMnJwdjx45FjRo1sH37dsyePRvnz593aka7bNkyPPDAA2jbti2mT5+OtLQ0/O1vf0P9+vXdjoWv15U3NpsNf/nLX9CjRw+8++67WLFiBaZOnQqr1YrXXnvNad1PP/0UeXl5eOKJJ2AymVC9enWsXr0agwYNQpMmTTBt2jTk5uZi9uzZ6NWrF3bv3o2kpCSMGTMG9evXx1tvvYXx48eja9eubk3+ipOTk4N+/frhwoULGDNmDBo2bIg//vgDkydPxqVLlzBz5sxit1+wYAGio6MxYcIEREdHY+3atXjllVdgNpudaqwA4Nq1axg0aBAefPBBjBgxAnXq1EFubi5uvvlmHD9+HOPGjUPjxo2xePFijBo1Cunp6Xj22Wd93hdHNpsNAwcORPfu3fGvf/0Lq1evxnvvvYemTZti7NixAOwPfe666y5s2rQJTz75JFq1aoUffvgBI0eO9CmPRo0aYfXq1Vi7dq364MyTTZs2IS0tDc8++6zXmtVHH30Un376KX755Rf06NGj2HyVwNa15paIyI0gopD06aefCgBi9erVIjU1VZw7d0588803okaNGiIiIkKcP39eCCHEyJEjBQAxadIkp+03btwoAIgvv/zSafmKFSucll+5ckWEhYWJIUOGCFmW1fVeeuklAUCMHDlSXbZu3ToBQKxbt04IIYTVahWNGzcWjRo1EmlpaU75OKb19NNPC09fV4EoozcAPP4bNmyYyMvLU9crKCgQtWvXFm3atBG5ubnq8l9++UUAEK+88oq6zNuxd5Wbmys6d+4sEhISxKVLl4QQQuzdu1cAEI8//rjTuhMnThQAxNq1a9Vl/fr1E/369Ss2j2PHjgmdTif+7//+T9hsNqf3lGOmHMfbb7/daZ05c+YIAOKTTz5R169fv7645557nNL59ttvBQCxYcMGIYQQmZmZIj4+XowePdppvcuXL4u4uDin5cUdq5ycHLdl06dPF5IkiTNnzqjL2rZtKxo0aCAyMzPVZb///rsAIBo1aqQu8/W68kYp6zPPPKMuk2VZDBkyRISFhYnU1FQhhBCnTp0SAERsbKy4cuWKUxodOnQQtWvXFteuXVOX7du3T+h0OvHoo4+qy5TP1OLFi522nzp1qttnplGjRk7X+uuvvy6ioqLE0aNHndabNGmS0Ov14uzZs8Xup6fjPmbMGBEZGen0mejXr58AID788EOndWfOnCkAiC+++EJdVlBQIHr27Cmio6OF2Wx22kfle0OhHL9PP/1UXaYc+9dee81p3Y4dO4rOnTurr5cuXSoAiHfffVddZrVaRZ8+fdzS9GT//v0iIiJCABAdOnQQzz77rFi6dKnIzs72uI8//PCD17SuX78uAIi7775bXdavXz/RsmVLkZqaKlJTU8Xhw4fFP/7xDwFADBkyxGM63r4nPYmKivLpe4+IKi821yQKcf3790etWrWQmJiIBx98ENHR0fjhhx/cai+UJ9yKxYsXIy4uDgMGDMDVq1fVf507d0Z0dDTWrVsHAFi9ejUKCgrwzDPPODUFeu6550os2549e3Dq1Ck899xzbk+mfWlWVB5ldHTXXXdh1apVWLVqFX788UdMnjwZK1aswPDhw9VmeDt37sSVK1fw1FNPOfXRGzJkCFq2bIlly5a5pet67F099dRTSE5OxpIlS1C3bl0A9sFoAGDChAlO6/79738HAI/5FGfp0qWQZRmvvPIKdDrnPw3KMVOO43PPPee0zujRoxEbG6vmKUkS7rvvPixfvhxZWVnqeosWLUL9+vXRu3dvAPZa3PT0dDz00ENO50+v16N79+7q+XPk6VhFRESov2dnZ+Pq1au46aabIITAnj17AAAXL15EcnIyHn30Uaf+SP369UPbtm2d0vP1uirJuHHjnI7huHHjUFBQgNWrVzutd88996BWrVrq60uXLmHv3r0YNWoUqlevri5v164dBgwYoJ77slq8eDH69OmDatWqOe1n//79YbPZsGHDhmK3dzzumZmZuHr1Kvr06YOcnBwcPnzYaV2TyYTHHnvMadny5ctRt25dp77BRqMR48ePR1ZWFtavX1/qfXvyySedXvfp0wcnT550yttgMDhdT3q9Hs8884xP6bdu3Rp79+7FiBEjcPr0aXzwwQcYNmwY6tSpg48++khdLzMzEwAQExPjNS3lPbPZ7LT88OHDqFWrFmrVqoWWLVtixowZuPPOOzm9AxH5hM01iULc3Llz0bx5cxgMBtSpUwctWrRwu4k3GAxo0KCB07Jjx44hIyPDqXmgI6VPyJkzZwAAN9xwg9P7tWrVQrVq1Yotm9J0tE2bNr7vUDmX0VGDBg3Qv39/9fWdd96JGjVqYOLEifjll18wdOhQNa8WLVq4bd+yZUts2rTJaZmnY+9o/vz5+PTTTzF//nynplxnzpyBTqdDs2bNnNavW7cu4uPj1XL46sSJE9DpdLjxxhu9ruNt38LCwtCkSROnPB944AHMnDkTP/30E4YPH46srCwsX74cY8aMUYPGY8eOAYDX5m6xsbFOr70dq7Nnz+KVV17BTz/95NZnLiMjw6nsrsdLWbZ79271ta/XVXF0Oh2aNGnitKx58+YA7E3uHDVu3NjpdXHXUKtWrbBy5UpNBp05duwY/vzzT6cA01FJ+3ngwAG8/PLLWLt2rVuAohx3Rf369d0Gkzlz5gxuuOEGt+8jZTRIf69hRXh4uNs+VatWzenaOHPmDOrVq+c2AImnY+5N8+bN8fnnn8Nms+HgwYP45Zdf8O677+KJJ55A48aN0b9/fzWAU4I9T7wFgklJSfjoo48gyzJOnDiBN998E6mpqZV68CoiKj8M8ohCXLdu3dTRNb0xmUxuN1qyLKN27dr48ssvPW7j7cawPAVDGW+77TYAwIYNGzB06FC/t/d07BXbt2/Hs88+i8cffxxPPPGEx3WCdSCFHj16ICkpCd9++y2GDx+On3/+Gbm5uXjggQfUdZTBPT7//HO1htKRax8mT8fKZrNhwIABuH79Ol588UW0bNkSUVFRuHDhAkaNGuXTACKuyvu6cqwRK0+yLGPAgAF44YUXPL6vBKWepKeno1+/foiNjcVrr72mzhW3e/duvPjii27HvSz76O0adx1IRaH09ywver0ebdu2Rdu2bdGzZ0/ccsst+PLLL9G/f381YP3zzz8xbNgwj9v/+eefAOD2gCUqKsrpoVKvXr3QqVMnvPTSS5g1a1ZgdoaIQgaDPCLyqGnTpli9ejV69epV7A2aMrrcsWPHnGouUlNTSxyJsGnTpgCA/fv3O93MuPJ2k1ceZSyJ1WoFALVZopLXkSNH3Gqojhw54vNE1ampqbj33nvRoUMHdXRRR40aNYIsyzh27Jh6IwkAKSkpSE9P93tC7KZNm0KWZRw8eBAdOnTwuI7jvjkex4KCApw6dcrtHN5///344IMPYDabsWjRIiQlJTnVRirnv3bt2sWe/+IkJyfj6NGj+Oyzz9Q5DAF7U1BPZT9+/LhbGq7LfL2uiiPLMk6ePOkUKB09ehQA3EbydOV4nF0dPnwYNWvW1GTqiKZNmyIrK6tUx/7333/HtWvX8P3336Nv377qcmXUXl80atQIf/75J2RZdgrelaaeynFQattdR74sbU2fkvaaNWuQlZXlVJvn6Zj7Q3mgpsxD17t3b8THx+Orr77ClClTPAagCxcuBADccccdxabdrl07jBgxAvPnz8fEiRPRsGHDMpWViEIb++QRkUf3338/bDYbXn/9dbf3rFaresPVv39/GI1GzJ4922l4+JJG5gOATp06oXHjxpg5c6bbDZxjWsoNres65VHGkvz8888AgPbt2wOw3+TVrl0bH374odNUBr/++isOHTqEIUOGlJimzWbDgw8+iIKCAixZssTjnGmDBw/2uA///ve/AcCnfBwNGzYMOp0Or732mlstjHLM+vfvj7CwMMyaNcvpOP7vf/9DRkaGW54PPPAA8vPz8dlnn2HFihW4//77nd4fOHAgYmNj8dZbb8FisbiVKTU1tcRyKzfNjuURQqhTcCgSEhLQpk0bLFy40Kmf4Pr165GcnOy0rq/XVUnmzJnjVKY5c+bAaDSqtb/e1KtXDx06dMBnn33mlNf+/fvx22+/qee+rO6//35s2bIFK1eudHsvPT1dfYDhiafjXlBQgP/85z8+5z948GBcvnwZixYtUpdZrVbMnj0b0dHR6vQQjRo1gl6vd+sj6E9envK2Wq2YN2+eusxms2H27Nk+bb9x40aP16zSX1Jp9hkZGYmJEyfiyJEjmDJlitv6y5Ytw4IFCzBw4MASR9YEgBdeeAEWi0X9nBMRecOaPCLyqF+/fhgzZgymT5+OvXv34vbbb4fRaMSxY8ewePFifPDBB7j33nvVOaimT5+OO+64A4MHD8aePXvw66+/ombNmsXmodPpMG/ePAwdOhQdOnTAY489hnr16uHw4cM4cOCAevPZuXNnAMD48eMxcOBA6PV6PPjgg+VSRkdHjx7FF198AcA+/PzWrVvx2WefoVmzZnjkkUcA2AeOeOedd/DYY4+hX79+eOihh9QpFJKSkvD888+XmM+HH36ItWvX4sknn3Qb5KNOnToYMGAA2rdvj5EjR+K///2v2nRu+/bt+OyzzzBs2DDccsstPu8XYO+XNmXKFLz++uvo06cP7r77bphMJuzYsQMJCQmYPn06atWqhcmTJ+PVV1/FX/7yF9x55504cuQI/vOf/6Br165ukzB36tRJTTc/P9+pqSZg73M3b948PPLII+jUqRMefPBB1KpVC2fPnsWyZcvQq1cvp0DJk5YtW6Jp06aYOHEiLly4gNjYWCxZssRjDe1bb72Fu+66C7169cJjjz2GtLQ0zJkzB23atHEK/Hy9rooTHh6OFStWYOTIkejevTt+/fVXLFu2DC+99JJPzT1nzJiBQYMGoWfPnvjb3/6mTqEQFxeHadOmlbi9L/7xj3/gp59+wh133IFRo0ahc+fOyM7ORnJyMr777jucPn3a6+fjpptuQrVq1TBy5EiMHz8ekiTh888/dwr6SvLEE09g/vz5GDVqFHbt2oWkpCR899132Lx5M2bOnKn2UYuLi8N9992H2bNnQ5IkNG3aFL/88kup5opTDB06FL169cKkSZNw+vRp3Hjjjfj+++/d+hJ6884772DXrl24++670a5dOwDA7t27sXDhQlSvXt1pUKdJkyZhz549eOedd7Blyxbcc889iIiIwKZNm/DFF1+gVatWbnMDenPjjTdi8ODB+Pjjj/HPf/7TaSqckvz888/Yt28fAMBiseDPP//EG2+8AcDev1jZDyIKERUzqCcRBZoyhcKOHTuKXW/kyJEiKirK6/v//e9/RefOnUVERISIiYkRbdu2FS+88IK4ePGiuo7NZhOvvvqqqFevnoiIiBA333yz2L9/v9uQ7d6GQt+0aZMYMGCAiImJEVFRUaJdu3Zi9uzZ6vtWq1U888wzolatWkKSJLdhwrUsozdwmTpBr9eLBg0aiCeeeEKkpKS4rb9o0SLRsWNHYTKZRPXq1cXDDz+sTluh8HbsleHvPf1znArBYrGIV199VTRu3FgYjUaRmJgoJk+e7DR8vRC+TaGg+OSTT9RyV6tWTfTr10+sWrXKaZ05c+aIli1bCqPRKOrUqSPGjh3rNgWGYsqUKQKAaNasmdc8161bJwYOHCji4uJEeHi4aNq0qRg1apTYuXOnuk5x1+nBgwdF//79RXR0tKhZs6YYPXq02Ldvn8eh8L/55hvRsmVLYTKZRJs2bcRPP/0k7rnnHtGyZUu3dH25rjxRynrixAlx++23i8jISFGnTh0xdepUp6knlCkAZsyY4TGd1atXi169eomIiAgRGxsrhg4dKg4ePOh27FDKKRSEsE9jMXnyZNGsWTMRFhYmatasKW666Sbxr3/9SxQUFBS7n5s3bxY9evQQERERIiEhQbzwwgti5cqVbp/xfv36idatW3tMIyUlRTz22GOiZs2aIiwsTLRt29bj9AWpqaninnvuEZGRkaJatWpizJgxYv/+/R6nUCjuM+Xo2rVr4pFHHhGxsbEiLi5OPPLII2LPnj0+TaGwefNm8fTTT4s2bdqIuLg4YTQaRcOGDcWoUaPEiRMn3Na32Wzi008/Fb169RKxsbEiPDxctG7dWrz66qsiKyvLbf3ijpky7cfUqVOdlpc0hYIyvYSnfyXtLxFVPpIQfjx2IyIiCjEdOnRArVq13PrxldaoUaPw3XffOdUOEhERlSf2ySMioirBYrG49TP7/fffsW/fPtx8880VUygiIqIAYJ88IiKqEi5cuID+/ftjxIgRSEhIwOHDh/Hhhx+ibt26bpNnExERVWYM8oiIqEqoVq0aOnfujI8//hipqamIiorCkCFD8Pbbb/s1gAUREVGwY588IiIiIiKiEMI+eURERERERCGEQR4REREREVEIYZBHREREREQUQhjkERERERERhRAGeURERERERCGEQR4REREREVEIYZBHREREREQUQhjkERERERERhRAGeURERERERCGEQR4REREREVEIYZBHREREREQUQhjkERERERERhRAGeURERERERCGEQR4REREREVEIYZBHREREREQUQhjkERERERERhRAGeURERERERCGEQR4REREREVEIYZBHREREREQUQgz+rHzu3DmMHTsW58+fx969e7F3716sW7cOzz//fKDK5zdZlnHx4kXExMRAkqSKLg4REREREZEmhBDIzMxEQkICdDrv9XWSEEL4mujgwYMxfPhwzJgxA/v27YPVakXHjh2RnJysSaG1cP78eSQmJlZ0MYiIiIiIiALi3LlzaNCggdf3/arJu3LlCkaMGIH33nvPvrHBAIPBryQCLiYmBoB9x2NjYyu4NERERERERNowm81ITExUYx5v/IrQDAYDHCv+0tLS4EdFYLlQmmjGxsYyyCMiIiIiopBTUrc0vwZeue+++zBmzBiYzWZ8/PHHGDBgAB5//PEyFZCIiIiIiIi041efPAD4+uuvsXTpUgghMGzYMAwfPjxQZSsVs9mMuLg4ZGRksCaPiIiIiIhChq+xjt9BXrBjkEdUeR1NyUSTmlEw6Dm7CxEREZErX2Mdn/rkTZgwodj3//3vf/tXOiIiD25/fwOmDb0Ro3o1ruiiEBEREVVaPgV5cXFxgS4HEREAIN8qV3QRiIiIiCo1n4K8qVOnBrocRFTFKS3HI03BNS0LERERUWXjV8eXxx9/HNeuXVNfX716FWPGjNG8UERU9VzJzAcA1IsNr+CSEBEREVVufgV5u3btQo0aNdTXNWvWxI4dOzQvFBHZ7TqThqRJyyq6GOXiYnouAMBk5KArRERERGXh192U1Wp1ei2EQEFBgaYFIqIiyefTK7oI5UYuHOc3tMb7JSIiIip/fgV5PXr0wLhx43DmzBmcPn0azzzzDHr06BGoshFVeXqdVNFFKEfC4f8apioEvtp2FiE2WwwRERGRV34Fee+99x5ycnLQtWtXdOvWDXl5eXj//fcDVTaiKk9XGOTJctUJULQOxi5m5OGlH5KRYs7XNF0iIiKiYOXXMHaxsbH45JNPAlUWInJhKAzy8q0yIsL0FVyawApURZtRbz+GFhunZiAiIqKqwa+avPnz5yMjIwMAMG7cOHTp0gUbNmwISMGICNDr7B/RXIutgksSeMLlp1aMhceQ8+8RERFRVeFXkDd37lzExcVh8+bNSE5OxptvvomJEycGqmxEVZ5SC5VXBYI8VYBq9FiTR0RERFWFX0GewWBv3bl27Vo8+uijGDhwoNuIm0SkvWCrycuz2JA0aRlSM7Xr56Y01xQBivIKWJNHREREVYRfQZ5Op8OiRYuwaNEi9O/fHwA4hQJROQi2mjylVuxatvaDmRTXN08I4ffALMrarMkjIiKiqsKvIG/OnDn4+uuvMXr0aDRq1AhHjx7FrbfeGqiyEVV5SjwTbEGeTlJG/Sx9Glez8vHJplPqa1+Ct8aTl+PjjadKXM8T1uQRERFRVeH3PHlLly7Fs88+CwBo3rw5Zs2aFZCCEVFR08U8S3AFKEqQV5amlQs2n8Zrvxx0W15SrLfl5DW/8lGCxwLW5BEREVEV4dcUCo899hgkyX1yZk6rQFQ1lWXaA9d53n0dXbO0tZrnrueUajsiIiKiysavmrwuXbqgc+fO6Ny5M1q3bo3Dhw8jMjIyUGUjqvICNXdcWSk1eGUpn6cHRvY0i0/U3yBPSe2fPx7wazsiIiKiysqvmrynn37a6fXYsWNx5513alogIiqijjgZ5MFeaehdqvJ83UerHKQHg4iIiChI+FWT5yo8PBznz5/XqixE5CJYwxklICtLvOXaXFNNu4TtrDY/R9cM1oNIREREFCB+1eRNmDBB/d1ms2Hnzp1o06aN5oUiosrB3+kMHLk21/S1CajNz8gyUPPuEREREQUrv4K8uLi4og0NBowfPx5333235oUiIjsliAq2QEUpTdlq8rxU5ZWwr0dSMkufKREREVEV4FeQN3Xq1ECVg4g8CK7QzpPSl9CtuWag+h8Wpte/VR2Y8yx4+svd+Pxv3TXOhIiIiCh4+BXk5eTk4LPPPsOxY8dgtVrV5Zwrj6hqUWoYy1KT5zrwijfHUjJxJCUTd7RLKH1mAACBAxfM2HjsahnTISIiIgpufgV5d999N4xGI7p27Qq9Xh+oMhGRIkir8tQ57TScQsHbPHkD3t8AAKUO8hzTUwJLIYTXKRyIiIiIKju/gryzZ8/i4MGDgSoLEbnQYj66QCrLwCteR9cM0L4KUZSnTRYw6BnkERERUWjyawqFli1b4upVNnUiKi/BG9zZf5aluaZblzylT57G1ZeOx1BXGOVxrj0iIiIKZX7V5L355pvo2bMnunbtivDwcHX5J598onnBiCj4laUmzy2tANdaCgD6wiaacrBGz0REREQa8CvIGz16NHr27IkuXbqwTx5ROfDWT63CaVCg8tonx5pBZdoG17n21h2+gptb1GI/PSIiIgoJfgV5aWlpWLhwYaDKQkQugr7CScOYqKi5prMaUWG4ll2gQfoCusIG6rLsvPyxBTuwZOxN6NyoWpnzISIiIqpofvXJa9++PS5cuBCoshCRi2CbBF0RyHJp2QTUnl7hTzjU5DnkUWCzR3y+TulAREREFOz8qslLTU1FmzZt0LNnT6c+ed9//73mBSOiIloHPmUViOJ4S7KsTSgdp3vQqwOvFFXl5VvtvxdYZZy+mo2kmlFlyo+IiIioovkV5I0YMQIjRowIVFmIyEWQxXZuJC3ba3rLQ6MsHGvyjl7OQrc31+D020OQb7EHeS8vTcbRlCycfnuINhkSERERVRC/gryRI0cGqhxE5EGwxniBKJdSW6l1YFuUblHCe86mqb+nmPMAFNXoEREREVV2fvXJI6JyFuxVeWXgbddc+/sFoqvce6uOqr/nFNgAAJFhfj3zIiIiIgpaDPKIKoFgC/UC0UfQse+co7I2CVUHXhGOubgzcOAVIiIiChEM8oiCWLAFd+XBLcjTrE9eUcJ1Yk0O+dmX6xjkERERUYjwu32SLMu4fPkyrFaruqxhw4aaFoqI7IK1taYWxXJLw0uiWoVeQhQdT5NB75atnjEeERERhQi/grwFCxZg/PjxMBqN0BXOKixJEq5cuRKQwhFVdcLbDOEVTCmWVrVsTmlrn6Q9XVGUtk0WTssBwKBjwwYiIiIKDX4Fea+//jp27NiBFi1aBKo8RFSJaBnjKc0pXfv7lXmePDVOLkpX9lBFGoiAlYiIiKgi+PXoumbNmgzwiMpRkFXgqVxHwNQ2bWea9clzaK7pGOQp+2Jge00iIiIKET4FeWazGWazGcOGDcPMmTNx5coVdZnZbA50GYmqLE+1UKFKjbu0nidPqSF0+N0mO60AoGiidCIiIqLKzqfmmvHx8ZAkSW1GNWHCBPW1JEmw2WwBLSRRVRW0oZ0GBXNtlultkJmyBl+egkfnmjw7PUfXJCIiohDhU5Any3LJKxERaSBQtZYCwmNzTYW+MJhUHl4RERERVVZ+9cl76qmnfFpGRNpQaruCbSqFQBTH22TogUjX0+iaSk2eVQ6yg01ERETkJ7+CvK1bt7ot++OPPzQrDBFVDkVTKGhf4xWwKRRQVG6nIK8wRzXIszHIIyIiosrNp+aaixYtwjfffINTp07h7rvvVpdnZGQgOjo6YIUjquqCrQYvkFz76KnLyxj2FdWGCjUtpz55ysArak2eDEAPIiIiosrKpyCvZcuWuOuuu7B7927cdddd6vLY2FjcdtttASscEQWngE6hEKCkHZNVuhk7BpYG1uQRERFRiPApyGvfvj3at2+PIUOGoFatWuqNEQcnIAqsognCK7ggXmjxFaAMdFI0CKa2O+vYJ09trunQ19F1dE2LjQNNERERUeXmV588q9WKwYMHIzIyEpGRkbjjjjtw6dKlQJWNqMoL1uBOi3J5S8N1eSCOgdInz95Pz/67MlVDvpVBHhEREVVufgV5TzzxBHr37o1Lly7h0qVL6N27N5544olAlY2oygvSGC8giiZ+L7907f30nNdjkEdERESVnU/NNRXnzp3Dzz//rL6eNGkSOnTooHWZiMhFsAV7WpZHCG2affqal9Nrp9/tr/KttvIpDBEREVGA+FWTJ4TA5cuX1deXL1/2OiIeEZVdsH68AjN/n1LlpnldnsNvzmkL4Z5tAWvyiIiIqJLzqyZv4sSJ6NixIwYNGgQAWLFiBWbMmBGQghFRYEexDFYB22MPwaOAcBjcRqnJY5BHRERElZtfQd4jjzyCjh074vfffwcA/P3vf0fr1q0DUS4ichBsNeaaDLyiBFcuaWo98Irj9p7SVpbJrMkjIiKiEOFXkAcATZo0gdlsBgA0btxY8wIRUZEgi+3cBCL41DpN4fLTOS/39ViTR0RERJWdX0HeH3/8gXvuuQd169YFAKSkpGDJkiXo2bNnQApHRFVHecSzrnkICIcaRPsvrMkjIiKiys6vgVcmTJiA7777Dnv27MGePXvw3Xff4fnnn/d5+w0bNmDo0KFISEiAJElYunSp0/tCCLzyyiuoV68eIiIi0L9/fxw7dsyfIhKFlGBrphkIroO4BGoKBce8HN9zn0KBo2sSERFR5eZXkJebm4tevXqpr2+66Sbk5eX5vH12djbat2+PuXPnenz/3XffxaxZs/Dhhx9i27ZtiIqKwsCBA/3KgygUBVuop0VA5utk6FrxlK7jZOhFUyiwJo+IiIgqN7+aa0ZHR2P16tXo378/AGDNmjWIioryeftBgwapI3O6EkJg5syZePnll3HXXXcBABYuXIg6depg6dKlePDBBz1ul5+fj/z8fPW10l+QKBQEa0VeIEb9dB2IRet0PaXtWLMnF8Z2bK5JRERElZ1fQd4HH3yAe+65B3q9HgAgyzK+//57TQpy6tQpXL58WQ0gASAuLg7du3fHli1bvAZ506dPx6uvvqpJGYiCTZDGeJoqr3107H/nmLdweB9gc00iIiKq/PwK8rp06YLjx4/jyJEjAIAWLVrAaDRqUhBlkvU6deo4La9Tp47TBOyuJk+ejAkTJqivzWYzEhMTNSkTUUXzNq1ARdOiXO61aspPjUfXLCY5xykUlJ82VuQRERFRJef3FAq5ubmQZRlWqxXJyckAgE6dOmleMF+ZTCaYTKYKy5+IAkuroM+ejIeqvMJlshrkMcojIiKiys2vIO/999/HK6+8gtq1a0Ons4/ZIkkSjh49WuaCOE7LUK9ePXV5SkoKOnToUOb0iSqjQPR904KWpXIdxEXrWstia/Kc9sT+u1UOzmNORERE5Cu/grzZs2fjyJEjSEhI0LwgjRs3Rt26dbFmzRo1qDObzdi2bRvGjh2reX5ElUFRgBKcgUcgB2AJBLc+eR6bawbnsSYiIiLylV9BXv369csU4GVlZeH48ePq61OnTmHv3r2oXr06GjZsiOeeew5vvPEGbrjhBjRu3Bj//Oc/kZCQgGHDhpU6T6LKLFjDjUDM3xeoOQHVUTuF+/F0HHhFFqzJIyIiotDgV5D30ksvYfz48bjjjjsQHh6uLu/bt69P2+/cuRO33HKL+loZMGXkyJFYsGABXnjhBWRnZ+OJJ55Aeno6evfujRUrVjjlRUQVT9Pmmi6plWtzTSHcmouyJo+IiIgqO7+CvC1btmDhwoXYtGmTOo2CJEnYvn27T9vffPPNxT6tlyQJr732Gl577TV/ikUUuoRw/BF8yjK6prfJ0LXLwi0d1zwd4znld6stWA82ERERkW/8CvIWLlyI06dPIz4+PkDFISJHwRpuBDLoDGzaLrWGhf85vsfRNYmIiKiy0/mzcqNGjRjgEZWjoK3B01B5zQXosRWBcM+XffKIiIiosvOrJq9r1664//77ce+99zr1k7vzzjs1LxgRBTPh8P/SpuB5a61H13QM4nwZeIV98oiIiKiy8yvI27VrFwBg3rx56jJJkhjkEQWI0CCYqiwcR8EMWB4eplBw/Z01eURERFTZ+RXkrVu3LlDlICIPgrW5ZiDK5XUgljLmVVzNoIBQm3Eq+cgM8oiIiKiS86tP3s8//wyz2QwA+Ne//oV7770XBw4cCEjBiCh4a/CUcgUm2AvgZOjFTNfAefKIiIgoVPgV5E2ZMgWxsbHYt28fvvjiCwwYMABPPvlkoMpGRIWCtUZPS4Hax+IGdhEefmefPCIiIqrs/AryDAZ7687ffvsNTzzxBMaMGYPs7OyAFIyIgje406JcgWqeWXymrnkVTYYOtU8ep1AgIiKiys2vIM9ms2Hbtm1YsmQJbrnlFgCAxWIJSMGISPuRJrWmRfnUmjY4/9SKp9o6x7yVfeDomkRERBQq/Ary3njjDYwZMwa9e/dGq1atcOTIETRv3jxQZSMiNQAKrsAjkOVxGwGzjHmpA6t4m7LBIciUJPbJIyIiosrPr9E1hw4diqFDh6qvW7RogSVLlmheKCKqekoKxrTJw/trIQQMOok1eURERFTp+VWTR0TlK1jDjeIGM/E7rQDvpeNIoG6ja6KoT54sAL1OgtUWrEediIiIyDcM8oiCmOscbsEiIFMnBDBtb2Th3BfQqNOxJo+IiIgqPQZ5RBRUNB94xaHPnXtzzaLJ0CEEDHqJo2sSERFRpedXkLdjxw5kZmaqr81mM3bu3Kl5oYjILthq8BRKs8eyFM+tltJLVV4gj4Fj0vbmmqzJIyIiosrPryBvzJgxiIyMVF9HRkZyMnSiAGK4oQXh4bfC107NNQWMeomjaxIREVGl51eQJ8sy9Hq9+tpgMMBqtWpeKCKyc51DLlgEpk9e2WsHi03fsWmmQ64oaq0Jg56jaxIREVHl51eQFxYWhmPHjqmvjx49CqPRqHmhiKhycA+aSpGGW5rFv+93+sUk4DwZOmDQ6ViTR0RERJWeX/PkTZ06Fb1798agQYMAACtXrsSnn34akIIRUfBNgh5IRbWWgdlnAQ8BpVP+9nnyGOQRERFRZedXkDdkyBBs3LgRq1evBgD885//RNOmTQNSMCJynI8uuAIPTebH81Jjp/WuCq8vCmvyHJYZ9DrkWS3aFoCIiIionPkV5AFA8+bN0bx580CUhYgqGS3isXILYD1kIxzqDWUhYNLrYONk6ERERFTJ+RTkPfTQQ/j666/RsWNHSJLk9v7u3bs1LxgRBa9ANKkM1CAzjjGka7kda/KUgVfYXJOIiIgqO5+CvIkTJwIAZs6cGciyEJGLYGumWR7cJywva3oOUygUk7YAYOQ8eURERBQCfAryOnfuDAA4ceIE/vrXvzq998knn6Bfv37al4yIgnbYFbcJzEuThttP4fRTa55SFQ4NNmUhoOfAK0RERBQC/JpCYc6cOW7L5s6dq1lhiMhZFazI0zyydR5B0+U94Rywcp48IiIiCgU+1eRt374dW7ZsQWpqKmbNmqUuz8jIQH5+fsAKR0TBKRBhUKADWk9NX4VwrlHUSVKVbCJLREREocWnIO/SpUvYu3cvcnJysGfPHnV5bGwsFixYEKiyEVV5ahPGIIs7lECoLE0rve2Th1Cs1Hm45uM+T17RElkI6CT7pOhERERElZlPQd5dd92FO+64A19++SUeffTRQJeJiApVhYDDdVTNQNakuaYtRFEB7H3ydLAFW0RNRERE5Cef++Tp9XqOrklUzoI13ghksbSfDF2pdfT0nvMk7Hpd1RzRlIiIiEKLXwOvdOrUCZs2bQpUWYjITWBHnCwrTeOhcgiu3JprCuE0T55eJ1WJ2lMiIiIKbT4111Rs3boVCxYsQJMmTRAdHa0u52ToRIERrJVKWpRLDVxdR7wse9IeE3QaSdNDXkIISJIEOVgPOhEREZGP/AryOF0CUfmqSvGGY7NJp+UBPAb2wK+oOadektRlkiQFLmMiIiKiAPIryFMmPb948SIAICEhQfsSEZFKqVUKvmAvcOXSummq82Tr3sfXVJprKr8zxiMiIqLKyq8+eYcOHULr1q3Vf23btsXhw4cDVTaiKi/oYrtCmjTXVFtrOgeMmg+8Ukx6jk047VMo2CM7jrBJRERElZlfQd5TTz2FKVOmIC0tDWlpaZgyZQrGjh0bqLIRVXmMNbQNdIvtkwf76JoA2C+PiIiIKjW/gry0tDQMHz5cff3ggw8iLS1N80IRkV3QToYeiDQDtJOOx9B9dE3XKRSKmmsSERERVVZ+BXl6vR4HDx5UXx88eBB6vV7zQhFRoSAPNrQoXnlOhu6etygaeMVhsBXW5BEREVFl5tfAK2+99Rb69u2Ldu3aAQCSk5Px5ZdfBqRgRBS8wUZAR7zUOr1iplBwnBNPADDoJLflRERERJWNX0HewIEDcejQIWzbtg0A0KNHD9SsWTMgBSOioK/I01SgBl5xysPliDq+Fg4DrwRrcE1ERETkC7+CPACoVasW7rjjjkCUhYhcuDZlDBaOTRxLnYbLz5LyKms+3t4sGl0TRUEeq/KIiIioEvOrT96xY8cwaNAgJCQkoHr16uo/IgqMYA01AjLwivozcHvtaXTNooFZhMPomgErAhEREVHA+RXkjR49GqNGjUK1atWwfv163HvvvZg4cWKgykZU5ZXnICTBQvt58hyaZHrIy7GZqE7H5ppERERU+fkV5JnNZjzwwAPQ6XRo27Yt5s+fj6VLlwaoaERUFIAEV9ChSTNSlyaf6s+ypFlsdu4pO/XJA6BnnzwiIiIKAX4FeUajEQAQExOD06dPIz8/H1evXg1IwYgosE0XK5vSBrpOE567pOE8T57gPHlEREQUEvwaeKVv3764du0axo0bh86dOyMsLAwPPvhgoMpGVOUFa7BR3v3mypag93SEQ36yAOfJIyIiopDgV5A3Y8YMAMDw4cPRp08fZGRkoE2bNgEpGBEF7+iaCi1iIU+NKAOVl3uaomjgFQhIACQJsHHkFSIiIqrE/J5CQZGYmIjExEQty0JELoK2uWZAAi7nn5ql6zQXnut7zvlLkn0aBVbkERERUWXmV588AG5z5HHOPKLAUSqU9p1Lr9ByuNIiBvKWhlsgplFtpsfAzWV0TQkSdBKbaxIREVHl5neQ9+qrrxb7moi0o8QaX247i/ScgootjEfaBUMl1VqWeuAVh81c8xAQWHUwxek9nSRxnjwiIiKq1PwO8jp37qz+fu3aNWzcuFHTAhGRo6Jo49b31mP813uQlW+twPLYaVnR5d6EMjARloDwWEu4t7CW1LG5JmvyiIiIqDLzO8gDgJUrV+L+++9H48aNsWnTJq3LRESFhAAe790YAHA9uwA/7buIdtNWVnCpAqOkPnn5VrlM6XqSYs5Xf5eFfeAVnRR88xISERER+cPnIO/MmTN45ZVX0KhRI7z88stYt24dzp07h++++y6Q5SOq0gSARjWjsPGFW3BT0xqIDTdAFkCBl4DnZGpWOZVLmcC8DGl4GQTFW5KDZ5Wt1YAQ7nm+9EMyAKB1Qqyar06SkFsgY9Oxq8iz2MqUZ7ARQuBSRm5FF4OIiIgCzKcgb8CAAejWrRsyMzPxyy+/YMeOHYiOjkZcXFygy0dUpSm1S4nVI/HV6B74/R+3AABe++WAGoD8uPcClidfwgPzt+DW99bjH4v3VWCJ/efWT86tSaV9wZlrOQAAQ+GE5b6n7/n3G+vFAgD+1rsxokwGtblmZr4VQ+dswoj/bUPLf67A8uRLajm+2nYWOQUV31zWXxabjOx8K77efg49p69FbkFwBq+uNag7Tl/HqavZrFklIiLyk09TKBw/fhz169dHixYtkJSUBKBo0mAiChyrTSBMX/QspnpUGJaM7Yl/fPcn7pqzGQPb1MWsNcectlm86zxG922C5nVifMpDCFHs53ndkSu4uXktp3UCM2cdYDLo8OPeC2ifGIdHeyY5vV8/PgIfPdoF98/fggKrjMw8C2pEm3xO/0pmPiY6BMBP3dIU7RvEo0G1CDz4360APH+vTf4+GTWjTbDJAi/9kIyXfkjGvlduR1yksXQ7Wg7yrTbcNWczXrnjRpy6lo0pP+x3en/dkSsY3LZehZTtwMUM1Igy4fs95zG0XQKuZuXjwf9uxZv/1xb//u0IPvtrN9SINmH7qet48otd6nb9W9XGg10bov+NdSqk3ERERJWJTzV5p06dwowZM7BhwwYkJSVhxIgRyMvLC1ih5s6di6SkJISHh6N79+7Yvn17wPIiCmYFVhlhBuePaedG1fHSoFY4kpKpBni3tKilvt+rWQ3sPJ3mtM21rHzIHoaM/HHvBTSevBxPfbkLNlnAJgvMXXccx1IyAQDn03Lw2Kc7kHwhAxm5FmTmWbDqYIraXDQzz7lWy5xnwbojV7zuz7Wsoj5wSg3elhPX1GUDW9fF3Z3q45UfD2D7qetYuOW0+l7rhFgY9RKy8q1o/vKv6PzGapjzLF7zUvMRAvXjI5BYPcJpuQQJidUjIUkScgprti6k5+K2lrXx8aNd0OeGmgCAjFwL7p+/BQ99tFXdtv1rv6H9q7/h861nSsy/PAkhcPCiGS1eXoHDlzMx/ONtbgHegBvr4Kkvd+PF7/5Epg/Hzxdp2QXoOX0Nmr/8K26Ystxrc+JLGbkYMmsTekxfg3dXHEGfd9dh2Z+XkG+Vse7IFVzMyMOA9zdg4uJ9TgEeAKw+dAWPL9yJ9q/+hgKrjPFf78HgDzaqA+coZFnAnGfBgH+vx4GLGTifloPpvx7C+qOpOHTJrMn+EhERBTufJ0O/7bbbcNtttyEtLQ1ffvklDhw4gMTERDz00EN49913NSvQokWLMGHCBHz44Yfo3r07Zs6ciYEDB+LIkSOoXbu2ZvlQ5ZGVb8UL3+3DW//XFvGRYT5tc/CiGU1qRSHcqA9w6QIr3ybDqHd/FnNbq9r4aVwv3DlnMx7okojXhrWGOdeK2AgDZqw4guQLGfh6+1l0TaqOxjWj0PmN1QCAwW3r4sW/tES/Gb8jzKBTb8aXJ19Gq7rH8d6qowCAWWuO4Zdneqs1XHfO2eyxfC//uB97z6fjq21n0aZ+LPZfsN9EN6sdja5J1fH19rP4dkxP1Ik1YcfpNExcvA/3dW6AUb2SYC0MOp/9Zi+e/WYvAODujvXx9t3tsGL/Zdw/f4tTXm3qxyEirOh8NqkVhXbTfkNCXDjG9GuKrHwrRvdpgpNXsxAXYcTaw1cw5Yf96NmkBqpFGfHLM31gkwVmrj6K2WuPo25cuJpWbIT9qzDGZMBbo7qq+aXlFKBVvVgs2nEWLy5JxuoJ/XDwkhnjv96DjFwL/rl0P+rFhldo7ZIQAkt2X0Dy+XR8tqUo6Jw29Ea8sewQrLLA5km3wmTQwajTIcdixaqDKVi08xzqxoXj+QHNy1yGPy9k4FJG0YO/09eyPdYk7zuX4bYst7DZcY7DqLHhRu/PHzNyLdh0PBU/7bsIAFi04xw6JMar74/5Ypc6LcbCP87ghjrRmL/+JOavPwkAaN8gDtHhBnRIjMdz/Ztjw9FU3NyiNiQAqVn5OJaShaa1o1Az2oSrWfmINBrwwH+3YOaDHdC4ZhRMhuD7Ttlx+jpqRIWhSa3oii4KEREFCUmUobPDrl278Mknn2Du3LmaFah79+7o2rUr5syZAwCQZRmJiYl45plnMGnSpBK3N5vNiIuLQ0ZGBmJjYzUrV2X3494L6NWsJmr62Lxt15k0JNWIRI1oE77ZfhYrD1zG6L5NkG+VcUsL92D7p30XceZqNs5cz8E797SDTgJe++Ug6sdH4PE+TZzWFULgt4Mp6NKomk/N7T7eeBJvLDuE0X0ao2H1SPRqVhPDP9qGX8b3Vvfn+JUsPPzxVqSY87HxhVvQ5911GNKuHuYO7+QxzYMXzVh35Ar0OgnfbD+LMf2aIrFaJCyyjNTMfMRHGJFizkOuxYbaMeGoHWNCvfgI5BbY0KpeTLk1Vx78wUY8P6A5BngJIpImLcPjvRvj5TtuVJf9tO8ixn+9R31dK8aE1Mx8GHQSrLJAjagwXMu2z7nXrkEc/vtIF7y/6igW7TwHAEisHoGsPCvScuy1PA93bwiTQY/03AI82LUhWtWLwZJd55FYPRJfbD2DdUdSUTc2HPlWG7omVcfIm5Kw4Wgq5m+w31TXjjHhSmZRDZ6jW1rUwqC29fDRhpOoGxeOt+9ph/rxEThyOROjF+7EsI711drKWQ91xJ3tE7DuyBXIskD7xHhsOJqK5cmXsfqQ/aa+zw01sfHYVbd8Rt2UhGl3tgZgrx1NvpCBzo2qqe/nWWywygLRJt+eewkh8MeJa3h/1VHsPJOG2jEmPNe/ORpWj4ROB3RMrAZJsvfxS8u2IMygQ5RJj+x8Gy5n5KFmdBjMeRY0qBZZpgcRFpuMge9vwMmr2QDstZ0x4QZ8MqorIsMMyLPYIAScgmPFk5/vQlLNKLz4lxbqMtfr2mKTIQEwFD5osMkCshAw6CSndX/Ycx5fbTuLWQ91xP/N/QN/v705et9QE1EmA2JMBlhlgVyLDd/uOIc3lh1yyqNnkxrYcvIaEqtH4Nx1+6AwEUa9Gvx5MqxDApbutQd5PZpUx4LHukEWAqev5uDZb/bg2BX7AES3tqwNmyyw/miqx3Ta1o9D8oUMdG9cHdtOXVeXd2tcHfd0qo8XlyR73G5o+wTc3LwW0nIKYJMFfj+SioxcC/KsNoy/9QbcUCca8ZFhkGWBq1n5iAjT45NNp6DXSbitZR1EhukRZTLAqNehfnwE4iKNsMkCR1MyceRyJnrfUBNp2QUwGfSoFWOCXmef1iNMr4NVFsiz2pBnsaFmlAl5VhtufMU+4q7y/fh/HeujfWIcjDoddH72Ya1MrDYZAvD4IIyIKFT5GuuUKcjTWkFBASIjI/Hdd99h2LBh6vKRI0ciPT0dP/74o9s2+fn5yM8vuoE0m81ITEwMqiAvM8+CIbM2Ic9iK5xkWRT+0Vb6Ntnn7xKw3zzafxb9DgGnebt0Ogl6nQSDTgdJst+gmgx6hBt1EIXr2v/Z05CFffh9AKgWaYSA/UZXKkxLJ0nQFc4Ppty4Xc3KR5hBh8gwPdJznJt0VY8KgwTlhlDAKgundaLC9LDYBAps9lqiyDA9DDoJOp0EWbbva2bhU/ua0WFqmZX9lgsPhrIsx8sgEQadhIgwPYSw90Gy2NwvZeU+1KjX2W9MYd/PzHwrdBLQq1lRUFAvLhx6nYS6seE4cz0HLerEIDbCgCvmfKRk5uHc9VzoJCCssEZEORfF3UO5lki5YTYWHg+n8+yynk0IpJjz8ePTvdDeoabCUdKkZZhxbzvc1yVRXWa1ybhj9ibc3KI2Plx/AgDw8pBWuLN9Arq9tQYAEKbXocAm49T0wYXNFa34YPUxzN9wEqNuSsIrd9yIlq+sQO0YEza9eKv3HYT9xl/v4SD8sOc82jWIx30fblGvv4e6Jao1RwcvmtE1qTqiSgisUjPzsf5oKoa2r+exFkUIgZ/2XURCfATu+7Co9u/Rno2wcMsZtG8Qhx/H9S42j9JKMeeh+1trMKhNXRy7koXcAhssNlkNaiXJvf9iVJge2QU2hBl0sBUGljrJ/nmSHLYRKPwMOFwjOkmCUW8/1pl5VuRbZdSKMeGrx7ujVozJ55puAJi5+ihmrj7mtlz5LtBJEqyyDLlwQBqjTqd+pgH7NSQLAZNBh+wCG4a2T8Dshzri4Y+3YvPxoia4YQYdrDZZnWDeZND5NB1GbLgB5jwrOjeqhl1n0opdVzlm0SaDOo9kfKTR7btLKzoJaFIrGtUijbAU9pstsMkw51lw5loOosL0yMq3qvscYdSjelQYmtSKwqWMPOQW2JCVb4UQAuY8K6JNBhTYZOglqdjg1lM5dJKENvXjMGd4Rxy7koV1h69gya7zyC6w2Wtv9fa/E3qH7z/792LRZ9b1mZXjS8f3JC/b6CRJfQCg/O1RXqtE0Q/l75KAUJuJhxl0iA03Isxg/ztmkwVsQkAu/Gkr/NthlWWH3+2J6nUSwg26wmtNQJKAyDADDHp7U+yMXAsijXoYDTroJAl6nX1fbELAapNhsdnTN+olWGwCAkLdV0kC9JL9+zrQz/aUwZ+Uv696nQS9S6bKS2WpXHhMAPuDAOW4SwBiwo1F60vO589j/sXMUVrcnaLyHYXCcqvfYY73MxDqPYsvfF3P06jJygM2qfCYKGVRvssc/15Jkr3ve4FVRoFNhsmgh8ng/aGBUi4lTfu1af8suj5sUOZdLWk0auUaU0eYFkXzujp9hFyOZ9G6RfeSBp0OpmJaQlRW6hRLLteop2va03Xs6dj7Gv2YDDqsnXizbyuXE02DPJ1O5zLoQtFADZIkwWrVZrS5ixcvon79+vjjjz/Qs2dPdfkLL7yA9evXY9u2bW7bTJs2Da+++qrb8mAK8qw2GTtOp8Fk1KkBlcUm3G7sJEhuX8jKl4TyxS8A9Y+i8mUebtQjz2JzStPxp06S1HUUJoP9y9ZiE0WBFaD+ga4ZHYacwhtW5Usvp8Cm5qlso5Qv2mRAuNG+Xmphv6saUWGw2AQyci32P8pKcCsDcZFGtX+WTr2WivZV53BcAAkNq0ciK98KvSQh32aDXpKQmWdFntWmHtNa0eGQhf3JeXxkGGQh1OaIlsKbTOWLXhYCTWtFI8ygg8UmF/7hL/6vyrnrOagRHYaL6bmwyUU3KJ7+wDi9dvgSkoWAxSbDKttvXnQ6Sf2DXkQU/rGXUCMqDEk1o7yWKd9qQ5heB281iwcvmqHXSWhR1950LiPHApsQqBZpRFqOBdWjnIMCc54FMSYDJEnCpYxcRJsMiAkv2wAj6TkFCDfq1QAlkLWgGTn2WrMwgw46Ceof7UBy7TcphMDVrAL1hrd2TDgKrDKy8q2ICNMjKkyPApuMML0OFzPykJ5ToP4xd725dPwekCT7Z99a+DAjOtyAmHAD4iPC3Ppt+kKpZbLIQn1QoT5wKfxp0OuglyRYbPYb61yLDTHhBugkCfmFD4oKbDKiTAbUiApTayVFYRpZBVZczbQ/MIqNMNofzBj16gMZQ+GNc3aBDQZd0U2tY+1TgVVWv+dE4U2s8lktKPzsHr+ShepRYagVbVIfYoUb7Z8LIewPopzjDfvNUEauBdEmAy5n5CEm3ABbYT5KbSVgby6uHON8q63E60n5+5hvtX9PXc0qcGoa7Coj14JLGbkw6nVIqhGlltdk0EEWRf1YDXp782pJAsINeuh09m2z821oVMO5RtgmC6Rm5uNCuv0zrNz8K+fY8SbIQxzmNJqp0+i0Tt91zjeZyvewvvDBoeP3OeDy9w1F3+1GvaReT5l5FuRbZTUQU76XXX8aCn9XPud5Fhl5VhsKrPbm7bIQyCmwwVp4bcaGG5FjscJqEw7BKNRrzqjXQa+zX+fKd5TjA1glaPVHcQFTidsWbmpxeDhS9MCo6KGwchyV4El5OKPU/GbmWd0eIiqBpC/8+aZWPpMC9u8WJdhT728kCVab0HSkXOUYOH5fOAZ0AgL6wofXasBV+J59XVFYEywhTK+H0SAhzyLD4vAwyzXIUkjKwwLJ/p1l/9sunI6tLBcFlsUdS+WYwSEQdwz2nV+rJXBKV7mXtNhkp4dxxSkp6A82rg84lOC2uOvZ21ve7kM8LdZJ9rEQgommQV52drbbsh9//BEvv/wymjRpgtWrV5ettIVKE+RVhpo8IiIiIiKisvI1yPOpA0pUVFFNwpYtW/Diiy8iOzsb8+fPx4ABA8pe2kI1a9aEXq9HSkqK0/KUlBTUrVvX4zYmkwkmk+/DqBMREREREYUyn9v4HD58GMOGDcMjjzyCsWPHYteuXZoGeAAQFhaGzp07Y82aNeoyWZaxZs0ap5o9IiIiIiIi8synmrzRo0fj119/xeTJk7FkyRLo9YHr4zJhwgSMHDkSXbp0Qbdu3TBz5kxkZ2fjscceC1ieREREREREocLngVciIyNhNBo9DsBy/fr1Yrb235w5czBjxgxcvnwZHTp0wKxZs9C9e3efts3IyEB8fDzOnTvHPnlERERERBQylPFH0tPTERcX53U9n4K8M2fOFPt+o0aN/C9hgJw/fx6JiYklr0hERERERFQJnTt3Dg0aNPD6fpnnyduzZw86duxYliQ0JcsyLl68iJiY8puwuiRKxM3axdDHc1018DxXHTzXVQPPc9XBc101hPJ5FkIgMzMTCQkJ0Om8D6/iU588ANi5cyfOnDmDm2++GTVq1MCBAwcwZcoUbN68GampqZoUWgs6na7YqLYixcbGhtyFRp7xXFcNPM9VB8911cDzXHXwXFcNoXqei2umqfBpdM133nkH/fv3x4wZM9CzZ0/Mnj0bXbt2RbNmzXDs2LEyF5SIiIiIiIi04VNN3oIFC3Dw4EEkJCTg8OHDaNOmDVauXInbbrst0OUjIiIiIiIiP/hUkxceHo6EhAQAQMuWLdG8eXMGeH4wmUyYOnUqJ22vAniuqwae56qD57pq4HmuOniuqwaeZx8HXmnVqhW+/fZbKKs+8MADTq/btWsX2FISERERERGRT3wK8pKSkryOVClJEk6ePKl5wYiIiIiIiMh/ZZ5CgYiIiIiIiIKHT33yiIiIiIiIqHJgkEdERERERBRCGOQRERERERGFEAZ5REREREREIYRBHhERERERUQhhkEdERERERBRCGOQRERERERGFEAZ5REREREREIYRBHhERERERUQhhkEdERERERBRCDFoneO7cOYwdOxbnz5/H3r17sXfvXqxbtw7PP/+81ll5JMsyLl68iJiYGEiSVC55EhERERERBZoQApmZmUhISIBO572+ThJCCC0zHjx4MIYPH44ZM2Zg3759sFqt6NixI5KTk7XMxqvz588jMTGxXPIiIiIiIiIqb+fOnUODBg28vq95Td6VK1cwYsQIvPfee/YMDAYYDJpn41VMTAwA+47HxsaWW75ERERERESBZDabkZiYqMY83mgefRkMBjhWDqalpUHjysJiKU00Y2NjGeQREREREVHIKalbmuYDr9x3330YM2YMzGYzPv74YwwYMACPP/641tkQERERERGRB5r3yQOAr7/+GkuXLoUQAsOGDcPw4cO1zsIrs9mMuLg4ZGRksCaPiIiIiIhChq+xTkCCvIrEII+IiIio4h28aIbFJqN9YnxFF4UoZPga62jWJ2/ChAnFvv/vf/9bq6yIiIiIKMgNnrURAHD67SEVXBLtnEjNQtNa0RVdDKISaRbkxcXFaZUUEREREVFQ2X7qOu6fvyWkglYKXZoFeVOnTtUqKSIiIiKioJKWU1DRRSAPLqbn4rI5D50aVqvoogQVzUfXfPzxx3Ht2jX19dWrVzFmzBitsyEiIiIiKjc2OaSGsQgZf/tsJ+7+zx8VXYygo3mQt2vXLtSoUUN9XbNmTezYsUPrbIiIiIiIyo3FJld0EcgDmcG3R5oHeVar1em1EAIFBazeJiIiIqLKizV5wSnMoHk4ExI0Pyo9evTAuHHjcObMGZw+fRrPPPMMevTooXU2RERERBSkCqyhV+tltTHIC0ZGvVTRRQhKmgd57733HnJyctC1a1d069YNeXl5eP/997XOhoiIiIio3Fjk0AtcQ4FRz5o8TzQbXVMRGxuLTz75ROtkiYiIiKiSELDXeg24sQ6SJi3Db8/3RfM6MRVWnivmPFhlgYT4iFKnweaawUliRZ5Hmoe+8+fPR0ZGBgBg3Lhx6NKlCzZs2KB1NkRERERUgbadvIblyZeKXUcIe2B0OSOvPIrk1Yj/bcNNb68tUxqWwuaayj4RBTPNg7y5c+ciLi4OmzdvRnJyMt58801MnDhR62yIiIiIyMWYz3fiYnpuueQ14n/b8NSXu4tdJ6fABgCICNOXR5G8One97MfEWji6JmM8qgw0D/IMBnsL0LVr1+LRRx/FwIED3Ubc9NXbb78NSZLw3HPPaVhCIiIiotC08kAKVh1MKZe8imu9qARCys+KblFn1aA/nbVwhxnjUWWgeZCn0+mwaNEiLFq0CP379weAUk2hsGPHDsyfPx/t2rXTuohEREREIctaTn3H/Gm2WNGBkUWDkTGVPnlsrhlcpAp/hBCcNA/y5syZg6+//hqjR49Go0aNcPToUdx6661+pZGVlYWHH34YH330EapVq1bsuvn5+TCbzU7/iIiIiKqq8poc2pdslAFYQiEuUptrapBWToEV/91wQoOUiDwLyDx5S5cuxbPPPgsAaN68OWbNmuVXGk8//TSGDBmi1gQWZ/r06YiLi1P/JSYmlqrcRERERKFAVHi9mbtQqP1Sm2tqsCu/HUjBW8sPlz0hIi80n0Lhscceg+RhLFNfp1X45ptvsHv3buzYscOn9SdPnowJEyaor81mMwM9IiIiqnKU5oR6XfDMG6b2zavYYmiiqE9e0d4cS8nEG8sO4bO/ditVWlR2nELBM82DvC5duqi/5+XlYcmSJejUqZNP2547dw7PPvssVq1ahfDwcJ+2MZlMMJlMpSorERERUaiwFDYn1AXBTa9rbVcIVOTBanOvyVu69wLWH031Oy2lSa3VJsNQgZN5v/DdPjzbvznql2H+QApOmgd5Tz/9tNPrsWPH4s477/Rp2127duHKlStOQaHNZsOGDRswZ84c5OfnQ6+v2CF4iYiIiIJZMAZUwdiE1F/KCJ2OxzffUrpRO3WFkXiBRkHe/zadwv4LGXj/gQ5+bfftzvNoXicGj/dpgm93nsMvf17CQj9rJSk4aR7kuQoPD8f58+d9Wve2225DcnKy07LHHnsMLVu2xIsvvsgAj4iIiKgEchBFecFTkrJTjqtjwJqRaylVWiaDPbArsMqIDCt72eauO47r2QV+B3lA0RyGX247i33n0steGAoKmgd5jv3jbDYbdu7ciTZt2vi0bUxMjNu6UVFRqFGjhs9pEBEREVVlwRDjqYGQgPPPEOB4fLMLSjcXdJhDkKcFo97/Nrq5hRPVR5vs4UAwNPMl7Wge5MXFxRUlbjBg/PjxuPvuu7XOhoiIiIg8CMamkcFXIv8p87E57ktpB7nRF44Wkq9RkGcoRTlSM/MBAHVi7eNgMMYLLZoHeVOnTtU0vd9//13T9IiIiIhCWTAO3BgMtYtlpYzi6DgdhKGM1V9aBXlKzaA/bMJ5IBkdh6kMKZoHeTk5Ofjss89w7NgxWK1FVdj+zpVHRERERL5TbtaDoU9e0dQJ7v3YKislBHLck7LGRVo11wwrxeAtNpcpIRjjhRbNg7y7774bRqMRXbt25UApREREROUsCGI8N8FYJn8p80A77Usp90vrw1GaAE12mcTQ0zzXlUElLXbAaR7knT17FgcPHtQ6WSIiIqIqY8HmU7iQnospQ270e1sRBBGVawkqvkQa0nBnKrKGs6gmz46xUmjRfPbFli1b4urVq1onS0RERFRlTPv5ID7aeEq9EfdHMPbJCyVaBmZaxeOlqYVTgzy1Jk+bslBw0Lwm780330TPnj3RtWtXhIeHq8s/+eQTrbMiIiIiCmlNX1qO028P8WldJfgIhj55CrVFYBCVqbSKBl4pe1rBcDxc5/2TWJcXUjQP8kaPHo2ePXuiS5cu7JNHREREVEZCCL9qarSKH2RZ4MttZ/Bw90bQ+TmKpBDOTQErPqTRjgZd8oICa/JCm+ZBXlpaGhYuXKh1skRERERVkk0WMPgx2bVWtUSpWfn4548HcFOzmmhaK7psiVXmaKiQOk+ehrVwFVmh51rjW1mnUGANpGea98lr3749Lly4oHWyRERERCHJJgv0enstzHkWj+9bbL5FAkVTKGhTLmthQnIZEhQuTQIrMyUG0uL4FpfEF1vPYOOx1LJnUgKb7FyWShrjkRea1+SlpqaiTZs26Nmzp1OfvO+//17rrIiIiIgqPYtNxoX0XBxLyULnRtXc35dlRMBzF5gUcx5qRpugd2hOqVVA9epPBwAUf/PfqEakx+WuzTSDoAtamXk6DGWt1fN0rl5euh9hBh2OvjHI53RKE58VNdcMgZNDbjQP8kaMGIERI0ZonSwRERFRSFLusbPyrQCA5xftdXrfUsyE2d3fWoMX/9ISY29uqnm5Nh2/6lQ+T6pFhmmeb7AKRCjk7diWR+BVNFG9XWVtrkmeaR7kjRw5UuskiYiIiEKW0jfKJtuDuR/2OHd7Kam55rm0nICUq2mtaCRfyPD4XklBiOvboRQ/aFFTGlSVZxx4JSRp3iePiIiIiHxnE0rfN8/vW2zea/IAwGZzrpHRytD29by+52+QElRBTSl5bK5ZxjS1OiylCtCU6S3UKRQqJwannjHIIyIiIqpAwmUADFcFJQV5AYqg1DnuilunhDRCaWL2Yo9DJYxiXftLlmZCdQpeDPKIiIiIKpBak+clUCixJi9AkVRZgjvXFSphDOSdJvsSPIOeBEERKAACEuTJsoyLFy/i7Nmz6j8iIiIicqcEd95utguKGXgFKJrqoDwDBiUv1v2UTUU21/SlpjYUfLzxJJImLavoYpQ7zQdeWbBgAcaPHw+j0Qidzh5DSpKEK1euaJ0VERERUaVXFOR5vt0uqaKuLPPYFUcNAsqSfBWpJhKi8vUNE0FUmxhIW09er+giVAjNg7zXX38dO3bsQIsWLbROmoiIiCjkKAOueIvVSroJt3obsaWMihtFsqSwwHXbUAojHPeltPGRJgF0GYV4bKfSV9HOaZrvds2aNRngEREREflILqFPXkn34rYSBm6hYKbNWZPK0HC2aACW0LyC9LpKVsWqEc2CPLPZDLPZjGHDhmHmzJm4cuWKusxsNmuVDREREVFIUQZO8RrklXDvHbCaPJch9j29V/K2VUNl3E/X0TVDVVUdNVSz5prx8fGQJEl9CjBhwgT1tSRJsNlsWmVFREREFDKUm2xrCZOee+M6umZ53LT7OyF4KNUSabkrFdtcUw3zKq4Q5UDPIK9s5AA9RSIiIiIKZUoNnvepEIq/CQ/YFAo+RCAl3T+HUGznUWl3L5gOS6ifoyraWlP7PnlPPfWUT8uIiIiIqGievH/9dsTj+yU313SegkHrigtP+Zc0cEiIxw2aqcjjVDXq8QBdFY3yNA/ytm7d6rbsjz/+0DobIiIiopCg1Jhdycz3/H4J27tOoaBVzYwW6ajD9Jc9qaDhuY9i6fZQq3NVqsDeJVAPpXPkiM01y2jRokX45ptvcOrUKdx9993q8oyMDERHR2uVDREREVFIsZWxx4s1UM01tUgjVCOHQhUd3GnB3/6VlY2OQV7ZtGzZEnfddRd2796Nu+66S10eGxuL2267TatsiIiIiEKK46iar/180O39kgICtU+exs01i2uSWfLomqEdOGilIo9T0WToFVYETZQ0emZVba6pWZDXvn17tG/fHkOGDEGtWrXUi7aqDltKRERE5AvHIO+Tzafc3i8pEHCbeLwcb9p9vc2r7IEEUELQW9o0S10aZ6VqrSmKfx0qqmooonmfPKvVisGDByMyMhKRkZG44447cOnSJa2zISIiIgoJygDlnRtV8/h+Rd17F9eMr6QmfiEaL7gpfXAXPEcoeEpCWtI8yHviiSfQu3dvXLp0CZcuXULv3r3xxBNPaJ0NERERUUhQavKsZeycpwQOmjfXLMVk6O7rVf5QIhCDyFTsPHnKz8p/bsidZs01FefOncPPP/+svp40aRI6dOigdTZEREREIUGZQsHiZTJ0f+/Bg+mePYiKElClPeaa1eiVIrKvKuemqtK8Jk8IgcuXL6uvL1++zCcERERERF4o90lW2XNNXolNIwN0m1VcsiVlWWVu/Uob3AXB8VGuu2AoC2lP85q8iRMnomPHjhg0aBAAYMWKFZgxY4bW2RARERGFBGVwTK9TIZQ4kqXza80GmigmCPD3AX4oBBIBad4YBMelss9lWEXHVSmR5kHeI488go4dO+L3338HAPz9739H69attc6GiIiIKCQoUyBYvTTX9FVxoz+WKj0t0giF6M4HFT2QShnmQg+JAJzcaR7kAUCTJk1gNpsBAI0bNw5EFkREREQhoaSBV0psGqlxeXxRcpkYORRHuPyskDJo/FCAgovmQd4ff/yBe+65B3Xr1gUApKSkYMmSJejZs6fWWRERERFVekpXPIuX5poVdRMeYq0SNaPpcQmCA1NUoxcEhSHNaD7wyoQJE/Ddd99hz5492LNnD7777js8//zzWmdDREREVKmdupoNwJeaPN9uvrW+RS92nrxgrF6sAGp/tkq5v0rZK2XhqQSaB3m5ubno1auX+vqmm25CXl6e1tkQERERVWq3/Ot3bD15TZ1CIS3HUqp0AnWTXmyyVTAu0PI4qyNbVuCBZGwX2jQP8qKjo7F69Wr19Zo1axAVFaV1NkRERESV0vXsAqRm5gMA0nMKIISAUe996IyKvhkvLv+SBvxgv6/iaXVcyjKiKk9NaNK8T94HH3yAe+65B3q9HgAgyzK+//57rbMhIiKqMs5cy4ZBr0P9+IiKLgppoNPrqxBhtN8n2WR7n7wwvQ4Wm83j+r7ehGtdo1d8RV7xw+4Ll/Uo+Ai3XyonzaYMCTGaB3ldunTB8ePHceTIEQBAixYtYDQatc6GiIioyug343cYdBKOvzW4ootCZbTrzHUAQK7FHtDpdRJsQsBk1CO7wEuQV0HVYMVlWxVr5jztclmPQ1CMrlnZozzyKCBTKOTm5kKWZVitViQnJwMAOnXqFIisiIiIqgSvE2VTpXIx3XmcAp1kH3glTK95D5oyK6qt837t+dxcM4QCCcfArsxBnkbRcunmyavMg8ZQSTQP8t5//3288sorqF27NnQ6+xeWJEk4evSo1lkRERERVSrWwvkSok0GZOVbIQsBmwyYjN6DvBIHsqyAm/RgLBOVDk9VaNI8yJs9ezaOHDmChIQErZMmIiKq0lbsv4y/tKlb0cWgMlDmxFP6EVllASFQfE2ej3fhmt+s8+7fSSCarwZFc00OjhOSNG8bUL9+fQZ4REREAfDkF7s4p1UllDRpGZImLYMQAocvmwHY++IBgE0W9uaahuJq8oo/54FqClk0SbaH90q4Dl0HZgmly1aL46318ZBKMfpIqAyOw3FXPNO8Ju+ll17C+PHjcccddyA8PFxd3rdvX62zIiIiqnJkARQz2j4FMYtNIM9ir8rLzLMCsAd5koRig7xgVLnDgtIJSDAUBAcylAJwKqJ5kLdlyxYsXLgQmzZtUqdRkCQJ27dvL3Hb6dOn4/vvv8fhw4cRERGBm266Ce+88w5atGihdTGJiIgqJZss1FogCn6bjl1Vf1+04yz0Ogm1YkzqPHlWWUAvScU21/T1Jlzrm/WiCbsrNo1g5svgNL5sXxFC/dwoquq3peZB3sKFC3H69GnEx8f7ve369evx9NNPo2vXrrBarXjppZdw++234+DBg5xQnYiICPaRGKny+HzrafX3f/54AMO7N0S0yaAGeTZZADoUG7iXdMpd39fqCilLH7SiUTVDjxYfQa2Du6oayPgiFK9BX2ge5DVq1KhUAR4ArFixwun1ggULULt2bezatctrc8/8/Hzk5+err81mc6nyJiIiqgxsnEqhUsktbJ6psFhlRJuKbr+ssoBOAnTF9KmqqDPuy2ToZUqkkgnE85WgeGYjylYbScFJ8wbgXbt2xf33349vv/0WP/30k/qvNDIyMgAA1atX97rO9OnTERcXp/5LTEwsVV5ERETBZPfZNGTkWtyWn7qajaRJyyqgRFQasktQbrHJiAzTq69tNhkWm4CxDB0tXQfQ0LpWR4vBfkJ1wKCyjkxZkYcllGtbHVXVWk7Na/J27doFAJg3b566TJIk3HnnnX6lI8synnvuOfTq1Qtt2rTxut7kyZMxYcIE9bXZbGagR0REld7d//kDd7ZPwKyHOqJ/qzpYfSgFAHDoElusVCauNa/5VhnpOUXBu00AlgIb9Lri+uT5dxteHs01S8pEuPwMJVrsUzDEvJwMPbRpHuStW7dOk3Sefvpp7N+/H5s2bSp2PZPJBJPJpEmeREREFWnpngt4Z8VhbJl8GwDAnGcPBqpFGtV1lPuxjzacxOi+Tcq7iOSnHIvN6XVmntVpJE2bLOOt5YcBABtfuAV/X7wP209dd9qm5InHAzWFAmvwHAViT4Lh6ITSOaIimjfX/Pnnn9V+cf/6179w77334sCBA36lMW7cOPzyyy9Yt24dGjRooHURiYiIgtJP+y7iUkae+tpik91XKrwfe3P5IWw8llpOJaPSWLLrPPadS3daZs6zwOQQ5FkdavoSq0fC0/gr/t6Da9U8rbjmfMoyb/OzhfLIjZ6Coorez1JMkxcyzTVLmiOwsu9faWke5E2ZMgWxsbHYt28fvvjiCwwYMABPPvmkT9sKITBu3Dj88MMPWLt2LRo3bqx18YiIiIKWa1BnsRV/o/zI/7Z77LdHweGjjScBAI1qRKrLMvOs+Eubuuprm8357IYb9Si1crxpr4qVPx4nhS9jWhVZi+ban7AqntNQpnmQZzDYW4D+9ttveOKJJzBmzBhkZ2f7tO3TTz+NL774Al999RViYmJw+fJlXL58Gbm5uVoXk4iIKOjkW52DPGth0Od48+XWhI43ZkFLqbEb0KqOusyca0GtmKJuJlZZoEeT6hh/azMAQL24CA8pFX+Sg/kSYOBQvIo8PKHcb9JRVR14RfMgz2azYdu2bViyZAluueUWAIDF4ttTxnnz5iEjIwM333wz6tWrp/5btGiR1sUkIiIKOgVWzzV5APBoz0YASm6aRMFDOXtGh+aZ5jyL08TnNlkgMsygBn6v3HEjejWr4ZxOhTXX9D4wR9GgHZ4LpyyWi0mjsgrGXZHKcNbZJy80aT7wyhtvvIExY8agf//+aNWqFY4cOYLmzZv7tC0vMiIiqsqy861Or5XmmwICtaJNJc6nRsFFCXCMDh3t7NMlFAV5qw6moE5cOAyFyyLC9GheJwabj19T1ynx7silmSabawaKh754pWx2qZ6rCp1CoUqexCpD8yBv6NChGDp0qPq6RYsWWLJkidbZEBERhZTdZ9Nw7EqW0zJ1UA5RNLCC68AcnMA4eMmFFbMGvXPDKceavSMpmagZEwa9w4ktS62MlkJlqoDgVnxNaPnnXPkEx6cl+GjeXJOIiIj8l5Hj3rXBdSAWSZLcavJ4Ex28lJo8g8tE544Tn7epHwurTcDgaVjNQhU9uqbHGix/0wqZkCJIP3NlOOkceCU0McgjIiIKAq6BAABYrEpzTXuA5+k+TuadWdBSzk2YS02e0xQKNgGbLJxr8vysrXVt+qfZZOg+pFRRc/hVBM+7UrqpIorr7wiUbkoEv6nXS+icIyrCII+IiCgION7s/W/TKQDAtewC5Fls6g2hpxs/mfdnQUs5NwadhOXj+6jLHfvkWWUBiyxg0Hm/JQvGOKmk4M317Yreh2DtyurtsJRPc83QGxSHimge5O3YsQOZmZnqa7PZjJ07d2qdDRERUUhxbJr5+i8HAdinVLjtvfXFbhdKNSWhRqnJizIZcGNCrLrcMciz2GTYZNmpJtffeCRQ10Bxzfh8beIXSldncc1XK1pZYtjg2xv/BGsAX9E0D/LGjBmDyMiiST8jIyN9ngydiIioqnKdPkFxIT23sLmmfUAO16ZVlf0GLZTJhVV5MeFGp+WOQd6ZazkosMrF98kLTPFK5Eu+Ja1TVZoT+7ubJY2uWR6BC/vihTbNgzxZlqHX69XXBoMBVqu1mC2IiIiowOY5yAPsN2ESJEByvyGrKjfRlZHSXDPaZB/MvFEN+0NwpY9eQlw4ACA9x1J8n7ySmkaqP7W9Fspyabk9jOBlGnRcrxv2zQstmgd5YWFhOHbsmPr66NGjMBqNxWxBRERE3mryXLkHeQEoDGlCOafhRvvt1nP9bwAAhBUOvBIbYYReJyEtp8CpT16wTXjv6RITJVVF+fZ2peKpD1tZ968iA6tQqcmr7OUPFM3nyZs6dSp69+6NQYMGAQBWrlyJTz/9VOtsiIiIQoqlcBh9q4eorai5pjuZUV7QyrPaoNdJaFXP3h9PXxjIOU6hEG7QIbvA5lSTFzzKfm0Fy9UpIcBlKWXiDFAoUDQP8oYMGYKNGzdi9erVAIB//vOfaNq0qdbZEBERhZQCqw1hBh2sBTa394QQkGAP9FzvCXmTGLxsNoHl4/sgqrC5pr6whk6ZDF0WAuFGPbILbMUOvFLi4CYBqpEpLr2iJn6+lYmXqQuND0hpKn+LaiYr99mp3KUPHM2DPABo3rw5mjdvHoikiYiIQpLFJmAy6JDjIchTeJopj33ygpPFJiO7wOpUa6eMtxKm16FJrSjc2T4BX207CwAlDLxSMec4VJrzaSUQAWtFHtpgm+aCtKVZkPfQQw/h66+/RseOHT22Jd+9e7dWWREREYWcApsMk0EPwOL2ntJc0xMGecHpvg+3QBZF/e8AQKfU5Ol1WPv3mwEA3+++AADO8+S5DbxSfF4VMchJSQGg8PBbKCrroDfBUIsWBEWgANAsyJs4cSIAYObMmVolSUREVGUUWGWYjF7GQyu8CZMk95tCdskLPkII7D2XDqBoJE0AapNMx/53ShBYlj55gWoSWVzgUlJeynVaFAxW7IUqSR6GpvWTtjV42h4PT7X8JZfB+SeFFs2CvM6dOwMATpw4gb/+9a9O733yySfo16+fVlkRERGFHHtNnvdBr6XC2zj3Pnm8RQs2jk1uHefE69mkJp6+xXmcgnCjfdop5z55zjfsFXWKfQnQvAUroRxAhMxHziUQr6z4HeiZ5lMozJkzx23Z3Llztc6GiIgopFisSnNNO8eKHQFhH13Tw8grrMkLPtn5RfMDOzbXjAjT4x8DWzqta/KhJs/fU1weffhca+rc33deL1QFav/K87BxfrzQpFlN3vbt27FlyxakpqZi1qxZ6vKMjAzk5+drlQ0REVFIKrDJTgFBmEGHPIt9njXHGz7XPnjskxd8shyCPMeaPE+Umjyj0zx5zutUVKBUXK4ll8h59M1QuEqLmsW69zj09xRpPahN6UbXdCmLNkWhIKFZkHfp0iXs3bsXOTk52LNnj7o8NjYWCxYs0CobIiKikGRxaa7p2GRPuQnz1FyTQV7wyc53bK5Z/N23cs51ZWhb5VZrptEl4Utyvk7vQJ55q0UrTdDmd94hEtxV9vIHimZB3l133YU77rgDX375JR599FGtkiUiIqoS8q3OQZ7N4e5YFsLeVFNyD+p4Ex18DlzMUH/3NOK4I+XtCGNRU123efK0Kpifih14xd/gjtepX8r1c80vkZCkaZ88vV7P0TWJiIhKwT5PXtGNvk12DPKK1uPcVsHP4kdHSavNvq4yYbpHQXyOvU6GrvwMoQu0aPJwT+/5mxa8plVe1H6V6oIKK0qZhNAlpinNB17p1KkTNm3apHWyREREIa3AanPqkye71uShsLkm++QFPYtV9nld5fw5NdV17ZPn49235s3viu03VnwuwdYUsBxaP5aKt49vuTTXLKEMVLlp1lxTsXXrVixYsABNmjRBdHS0upyToRMREXlnsQnEhhfd6LsOtqKMrsk+ecGvwOZ7kLfuSCqAkpt1VgRf+uJ5q6lTA1MPg5VUWp5q8AK0W/6mW6qBV0LglAAlP0gIwo9WudA8yON0CURERP4rbjJ0pfWfp/mcOYVC8CnwoyavfWI89hVOnK7wd568wA3jX/p0g60mT0sem2v6eayC6fiERAAO+znw9LAkVIJZf2ke5CmTnl+8eBEAkJCQoHUWREREIcc+Gbre43uyLNTbfveBV6roHUwQ8yfI+2HsTbiYkVvsOn7399L4mvAUBJSUAy9L33g7V2yu6Tt+B3qmeZ+8Q4cOoXXr1uq/tm3b4vDhw1pnQ0REFFIKXEbXdKSMrinBveaONXnBx2KT0eeGmnikR6MS19XpJDSoFum0zH2evOLTCNQl4FtzTW/bKoOUeB+spLIJ5l1wrf31hevAK5W9Rq8i+zcGI82DvKeeegpTpkxBWloa0tLSMGXKFIwdO1brbIiIiEKKpZiaPGWkTUmSOPBKJZBvldGybgxeH9amXPILln5hnrYNlqtTi3IUBUVFqQmXn76Xx2VkywrEr5DQpHmQl5aWhuHDh6uvH3zwQaSlpWmdDRERUUgpsMpOo2s6EsL+NNrTA2kGecGnwOb9XPrCfZ680vX3KqvihtYvClJ8G2WzomnRpC9Ua80rew2eIjT2QjuaB3l6vR4HDx5UXx88eBB6vecnk0RERGRnr8krprmmw++OguUmmooUWGWEaXjv4+s5DqZrwW0+x4ophqb5B2QfNEq0TKNrus29WdFnq3Qqa7kDRfOBV9566y307dsX7dq1AwAkJyfjyy+/1DobIiKikFLc6Jo2oTTX9NQnjzc2wSa3wIbIsDIEeS537BV1hou7afa5T16Fh3d2WnxMZA/9C8va51Cr41OmprWalICCjeZB3sCBA3Ho0CFs27YNANCjRw/UrFlT62yIiIhCSoFNFDu6pv3GX+IUCpVAZr4VUSbNb7G8cg0UtLokiutvVvK0Dr6tV6louC++Hj+f0ytF4UJtcJxKXnzNBeQbqFatWrjjjjsCkTQREVHIsdpkZOZZEBPu+c+yLIr6abnd0Ff2O7MQlJ1vRbSXc+kLt5Z3JZzjokFOyv9a8Jaja4AYCpdpIGrNNes/WYp0vAXiSh/gyiIUrq1A0LxP3rFjxzBo0CAkJCSgevXq6j8iIiLyLDPPinyrjPrxER7ft8nCPvCKh8nQeYMTfLLyrIg2adgnT7OUtMvYtRbI7f1gG15TA8V91kobYHvbyt8gqzQBaAidGgD8LnSleZA3evRojBo1CtWqVcP69etx7733YuLEiVpnQ0REFDLyrDYAQLjRc2BglWUYdPaZsDiFQvDLLrAiKqwMNXl+zpOnc+3Dp9nomt6b8fk6d1+w9c0rCy0/ayWl5HdzTQ365JV2OoiKFgrXViBoHuSZzWY88MAD0Ol0aNu2LebPn4+lS5dqnQ0REVHIyC2wIcygg87L03uLTUAnSV4GXgl8+cg/eRbZa8DuC38ntlZiPK3jfV/S89pcMwT75AViV4KxuWZlxWDPmeZBntFoBADExMTg9OnTyM/Px9WrV7XOhoiIKGTkWWSEG3SQvLTRsthkGPT299wHXuGNTbDJt9q8jpRaGiX1uwzO7lOhd12W5wAl5dNcM3RqWcmd5gOv9O3bF9euXcO4cePQuXNnhIWF4cEHH9Q6GyIiopCRZ7Uh3Kj3WpNnVWryIKk3ZHd3rI8TqVkceCUI5VtlryOl+sKtuWaJ67tOuaDtsPzFpudtCgWXLnmhcJkWuw/+7p8oPsDyf3TN0nPv5ysQrI8OPAm1GkmtaB7kzZgxAwAwfPhw9OnTBxkZGWjTpo3W2RAREYWMPIs9yPPWTM/eJ0/n1lxTkiQ21wwyQgj7nIdeJrYvXZrFvx+okRCLC+587pMXQnfesofArKy7p9XhKVVNXuicGvIgoJO4JCYmIjExMZBZEBERVXr5FhnhRp3Xm3WLTUBfOPCK452ZTmJzzWCTb5UBoExBnr8xm9onr9Q5eubLaJIl9snTtkgVqjz3xd/AvUwDr6jNUCvn2aqkxQ44zfvkAXCbI49z5hEREXmXZ7Ehwqj3emNntcnQF7bltMqOQR5r8oKNGuSVYeAVV94DKfs7gRpdsyzpufZfC4XLtLjPmt+tNUvYzmITOHstx+/0SsN128p6rhjsOQtIkPfqq68W+5qIfJc0aRnWHblS0cUgogDKs9pgMuqLGXhF2KdQkCQUFAYRgDJvHu9sgkl+4XQYZarJc5tCwfM5thVGHaVprfnRhpNYdTCl2HWK7YKm9oPy0qesFGUKdp4GXilt/0eLreSOZH1nrFPPse9l87085TmQjFa2nbyGzcev4pmv9+D4lUwAHDjGm4AEeZ07d1Z/v3btGjZu3BiIbIiqjAMXMiq6CEQUQMqQ+64369Ui7SNWF9hk1IkNBwDsOZcOAPbmm5LE5ppBJt8iQycBBm+j6GhIuf9XHg74c4P/5vJD+Pu3e4tdp/jmms4//dm2stJyn2yyXPJKAK5l5fu0nuxjwJaWXQBZdq1lLb7pbbC4mJ6LB/67FQ9/vA0/77uIr7adAwBsPXkdAIM9VwEJ8gBg5cqVuP/++9G4cWNs2rQpUNkQhTTlD3aUKaDdZ4moguVZbAg36Nya3W168Vb191b1YnAhPRe7zqQBAGLCjfY+eb7dK1I5UUbW9FYr6wtft1Vu7P3NSakdMudZi13PWnhxlebW2e2GOwSiPn8GoimpBk6pyfO0VrTJgFE3JcFk0CG5hIe8/tbGdXx9FWatPVZCmu7LrmcXYO3hFCSfz8CuM2lqoKhYsPkU3l1xGH+eT0eKOU9dbpMFDl40u63vSWaeBUmTluHgRTOEEPg1+RKOXM5U31+45YzT+uY8Cyw271+AVb2Vg6ZB3pkzZ/DKK6+gUaNGePnll7Fu3TqcO3cO3333nZbZEFUZuRZ7s5+oMAZ5RKEsVxld0+FuvUG1CPUBT9v6cZAkCf1b1Vbf79a4OnSSxGfXQUbrOfIA7zfw6nI/o7yL6bn2zUpo7uvYNNg97xKCCx8uzKRJy7C/mCAmPacgqG7UlQcqJZVo7eEUNH1pudp0FwB+Tb7kdDytNvfjdzE9F3+cuIpGNSJxU9Ma6NSwGnafTfOYhxACc9cdR+PJy5E0aZmaXp7Vhr3n0pGRa/EaaG4/dR35VhvMeRa3MijyLDa89EMyrmcXAAAWbjmNvy7YiaFzNuGeeX9g0AfOrfSm/XwQ//n9BO6csxlTfkhWl3d9czUGz9qI+RtOAgCumPPQbtpKzPv9hFuehy7ZA7qHP96KVq+swNgvd2PgzA3YedpeU3f4shkPd2+ItvXjAADf7TqPQ5fMDsekKK37529B48nLkZVvdXjf8/FYnnzJqcyhQrNvoQEDBqBbt27IzMzEL7/8gh07diA6OhpxcXFaZUFU5ShfruFh2nXgr2xSM/Ox/mhqRReDKKDylNE1HZaNuilJ/f25/jcAAD4e2VVd/pc2dQsHXgmem2ACzqflIj3HommaOQU2XEzPxYLNp5xuVB1r8hyXu97Mbj91HWeuZauvj6ZkIiEuHEIAGbkWCCHwzorDSJq0DOfTcvDj3gvIyreqtSRfbTvjd7BV0tpKcHfgYgbWHb6CR/63zen9FHMeOry2Cv/bdMrnfmlasckCx69kuS1XavJkp2Ptvv0XW88CAI5ezsJP+y7ipulrMPbL3djg8LfMU3PNER9vw/CP7MdBkiT8X6f6WHc41e3YH0vJROPJyzFj5RF12ZEUJUDahmFzN6P9q7/h253nkGex4d0Vh/Hv347g37/Z1z+Zmo0WL6/ARxtPFe6Xs8sZeThw0Yyvtp1Fp9dXYdvJa5i52rn270hKpnp9OAZSALD60BXIskBqZj6uZxcgIS5cvb6+2HYW5jwr3llxGCdSi47xidQs3D9/CwAgLceCPIs97doxJvx3w0kM+Pd6HLpkxu2t62J494bqdnfO2YyODeMBAG2mrcSnm+37tP2UPTC8a84mJF8oCgQVGTn2WsO5647jqS9348ttZ7Hj9HVMWvKn18C6stGseuD48eOoX78+WrRogaSkJAC+NzcgIs+UJg/B9CRTse9cOtonxgc8n65vrgYAnH57SMDzIqoo+YU1eZGFNXcdEuPxeJ8mAIA9/xyAalFh6rpTh96IV+64EQAHXglGfxy/iqQakWVKw6h3vn96f/VRZBdY8d8NJ9ElqTqiTAZYbDLqxdn7aZ5IzUbjycvRvXF1AMBnW85g2p2tIUkScgtsuH/+FlSPCsOOKf3xxdYzmLvuOHo3q4nl+y/hnRWHYdDp8PlWe1O43u+sAwDERRhRJ9YEAFh5IAWNJy/H4id7IsKoR+0YE65l2R9C+jqZ96d/nMZnW85gaLsELPjjFNIKA+HNx6/hp30XAQA/77uIoe0TAACfbLLfrL+x7BDeWHYIJ98ajHs+/AMX0nKxfUp/XEzPxaId5/D8gObFHstLGbmYtab45omu7vvwD+w+m46fxvVCuwbx6nIl1rzvwy048dZgp22m/XQAd3ZIQPWoMBy+ZIZeJ2HXmeuY9vNBdZ3kCxnof2MdAMCstccBFH1+cwqsOHnVHogrNWxdk6rjhe/+xJSl+3Fv5waIDTdg3u8nkWcpqiHcN/V2tH/1NwBAy7ox2HM2XX1v5YHLyMi14D8utWaXC+8t2jWIQ2SYXj1X1SLt3zOPL9yBv7Sph8TqERACeOC/WwHYHzbNXH0M21+6Dd3eWoOnvtyNuzvWR2pWfuF0LkV5NHlpORpWj0STmlFYPaEfHv54G7acvIZZa47h8d6N8fGmUxg2ZzNMRh2sssDkQS0BAF0aVcPOM2mIi7A3Rx9/2w14eel+Nd0bakejX/NaeKBLIpq8tBwA0PeGWthzNh1CAP/dcBL3dm6grn8itejhxpGUTDSvHQOdTsKYL3YCgBoo63USRny8DflWGd/sOId9r9yOuMI+0ZWVZkHeqVOnsGbNGvzvf//DlClTMGjQIOTl5ZW8oQdz587FjBkzcPnyZbRv3x6zZ89Gt27dtCoqUdA7fiUL47/eg4OFzRCSz2fgrg71K7hURbLzrbhr7mb88kxvtKmvXW39L39eRJ8baiEuony/WL/adha9mtVAvxm/4/Drf0G4hkOfE/li47Gr6NeiFqJNBtzfpYHT590xwAPsD1CVZ6icQiHwPtpwEnGRRtzfJREFVhk/7DmPPjfUws4zaagfH4HOjao5rZ+Ra3G6ySyN1gn279X3H2iPXs1qotuba/DfwuZu3+48h4VbzsCol/Duve2cttt26jrG39oMs9bam/EB9hv/xOoRyMyzYvup65j60wF7HvXjsOPMdXy93T54xfDuDfHmsDbqdhm5FmTkWtA1qRp2nLbXbNz34Raf98GxXxZgrz0C7AFrTLgBj/ZshIVbzqgBHgA88/UeTPh2L9b/4xbM33ASsx/qiGe+3gMAWH8sVQ1gkiYtQ5OaUTh5NRsrD1zG4cuZ+EvruujQMB6fbzmDRWN6IMWchzqx4Zj200H8ceIqWtWLxaFLZmTlWzF9+SE0rhmFx/s0we6zacjItWDl/suoFhXm1Izw0U+24z8Pd8JNTWviWlY+ThTW7tlkASEE/vXbEfxx4hoAYMWBy1hx4LK67f91rK8GePXjI9A6IRZ7z6Vj5+nr2HrymrrenHUncCkjD/1vrINokwEx4Qacu56L6lFhSKoRibgII77adhZfbTvrdoxPvDUYep2Eb57ogZrRJjStFYWDl8xoEB+Jw5fNeOC/W/H7EfeWMMM6JKBObDgmD26FD1Yfw/urj+Lr7WfVwYKOpmThaMoxTBjQHKN6JaHdtN9g0El4rn9zPNffHlTfUDsaqw6mOI3Q2qhGJM5cy8FD3Rri6+1ncfZ6Dga2rgOdTsLXT/TAnXM24c/zGXh+QHPERRjx3qqjyCwcV+bFJcl4om8TvDS4FX47cBl9bqiFMIMOVzLz8PLSorIrDzZ0Ogm/PtsHExfvw7hbm+Hh7g3x6CfbcfhyJt5fdQy1Y0xY8Fg3DJ5V1Kz0LzM34p5ODTDtzhtx8KIZT9/SFF9vP4c3h7XBigOX8ePeomvxjWUHYc6zYNeZdGx68ZZKeV8giQA8AkxLS8OXX36J//3vf7h69SoeeughvPvuuz5tu2jRIjz66KP48MMP0b17d8ycOROLFy/GkSNHULt27RK3N5vNiIuLQ0ZGBmJjY8u6K0TlwmqTkZ1vw8FLZjStFYVub61xW0f5Mg8GlzJy0XP6Wnw1ujtualqzVGkcv5KFCd/uxU/jegMAzl3PQZ9312Hi7c0x7tYb1PW6vLEKV7MKcPrtIfhhz3kcuGDGy4W1GL6wyQLvrjiMv7Spi1b1YnHT22vx0aNd1Buza1n56PzGajzQJRGLdp7DsvG91RssIq39uPcCnv1mL9rWj0P7xDjkFsjo3rg6XljyJ0bdlIRpd7b2K72nv9qN5rVj8Gz/G0pemTwSQkCSJFzNykdUmAHZBVa8tfwQNh27irpx4fjzvL1ZYdv6cTiXluPWFLNN/VjoJQmdG1XH7rNp2HsuHe/d1x73lCHQy7PYMHHxPrx7bztEhhmw7M9LePqr3RjTrwnmrz8Jg05ymi8RAJ66uSn63FALPZvWQNKkZU7vPdG3CS6m5+KXPy8BAH4a1wvN68TgenYBpv96GD/vu4gdU/qjVowJG4+l4lJGHl747k881K0hXrurNZbuuYBOjaph3u8nkJ5jgTnPgu2nrmPUTUlY8MdpNKweicgwPeIjjXjq5ma4MSEWfd9dh44N49E1qTru6dQAW05ew90d62Pj8au4uXktSJKEUZ9ux+9HUjHulmZIyynAl4WBTMu6MYiLMGLRmJ7ILbBhyOyN0EsSmtaKRr34cHy6+bS6bxFGvdp/XfFQt0Q1eAWAbS/dhlrRJjR5aTl6N6uJTcevelzP0cDWdbDygD2AeaRHI7Wmc+LtzfGf30+gW+Pq+P1IKno0qY6Ff+2OrSev4XxaLn758yLCjXr0b1UHL/2QjP+N7ILbWtXBsZRMDHh/g5r+fZ0bwKDX4evtRcFbvbhwfDW6B9YcSsFjvRpDr5NwMjULt763Hm8Ma4Ptp66rQbFBJ+G4S22iI5ss0PzlX3FHu3p4//4O+Gr7WbVGzLFlTPL5DAydUzRAohKozXu4Ewa2rgudTlKby7reg6w5lIK/fWavEfvo0S5IPp+OWWuPY/OkW1Er2oR3VhzGA10T0bxODADgy21nsOnYVcwbYR+F/73fjkCSJKw+mIJjVzLx2/P90LhmlNu+WG0yFu86j6+3n1XvGTwRQqgPKe7uWB//fqCD+ln4v471ceBiBo5dyUKYXofuTWrgs8e6qq0Ov95+FpO/T8af027HiuTLeGHJnwCAMX2bYNKglkHVOtHXWCcgQZ6jXbt24ZNPPsHcuXN9Wr979+7o2rUr5syZAwCQZRmJiYl45plnMGnSJLf18/PzkZ9fNLys2WxGYmJiUAV5QojCNu/O7Z6Lu1zsTXDs68tC2H8XArKwN40Qwv5h0+sk6CXJPiKbBDUDZR0ULXJr0iNJUrFlEIXb2H/ay6HXFb+NY9qe9tMxTcD+FFqvDP2sDOFb+GauxQaLTUZOgQ02WcAmC1hlAZNBh2pRYYVDVOsgSfDYJ0VXuH9KWSTYn/zA4RgKAH+eT0eDahGoGW2CBAkChcfZ5Xgrx0AAkGVle+d1ob4ufE8u2j7faoPFJhBu1CNMr0NMuAG7z6Zh6Z4LWOfhSdu/72+PCd/uQ/9WdbD15DX0a14L9eLCMahtPVSLNKoT7eZZbLDaBCLD7IM2SIXHVFfY41Y5dka9Dueu56BuXDiiwgyQXfZPuc6KrjcUriPU8yYXLjt9NQdPfrELbwxrg1ta1i48zs7XiV491s7DNCvHfeEfpzF/w0mser4vIsL0WLH/Mt5YdgjP3nYD7u+aCH1hbUWfd9ahwCZj4wu3oM+79mZEG1+4BULY89TpJHvZ5KLyOp6Ts9dzMHqh/Y/Qy0Na4Y1lh5BYPQLVIsMweVArrNh/CZ9tOYPYcAPMeVY80qMRnujbpOgzKJzTVa61cIMeYQYdZGE/vsrPXIsNEiTUiA5zOiZFnwep8DzZO/FbbDJMRh2MOp1T2ZXPvizs15tybpXPk/IRU5YoaSrpe/r8eftcavG3q7R/SUr7B6gsf7pcR7B0LIvjd54ovIAdXwtRtB7gOFeY/X2LTUaeRYYQQHaBFcevZMGgk7Dl5DX1KXHPJjWQVDMK5lwLliVfUvM/NX2w3zcSLy9Nxhdbz2LSoJZoWD0SVlkgIS4c9eIjIMsC17ILEG3Sw6DTwWjQwaiXkG+RUWCTUWCVYTLoEG0yIN8qQ5IAo16HzDyLeq0rnysATp8BIQSsNoHYCCMkyd6n0Cbbv6/1Oglheh2Meh3iI42Qhf17tsAqo2H1SFzLLsCFtFzEhBtQLz4CEoAL6bloVD0SsgD2nkuDxSagkyREhOlwQ+0YZBdYkZZdAHOeFfGRRkSbDLDJAnVjw2HQ2z87UWEG5FlthX8f7TevusJpJmw2GRuPXUWXpOrYdy4dBTYZp69mIzUzH5n57iNMRobp0blRNVSLDEPduHAUWGUs+OM0mteJxv9GdsX8DScQYdTDYrMfi7jIMPxx/CqSL2RAkoCfxvVWb2y1kJVvxQPzt+CTUV3R/a01mDK4FZrViUakUY/uTWq4rZ9nscGca0G1qDBsP3Ud7RPjkWLOw23vrQfgfJMvywLHU7PcynvvvD8w88EOaFDNvelpnsWGX/68hLs71segDzaq/cEU1SKNSMuxlNjk7fTVbNz8r9/xxrA2uKdTA2TkWnDsSiYe+d92TBt6I0b1agwAmPz9n/h6+zk1eN53Lh13zd2s7otyI//v+9vjYnou/vXbUdxQOxpP3dIUeRYZD3Wz9+H624IdWHP4itpcUN3Xzg3waM9GuJZdgNNXs/FqYQ3csTcHof+/1+NM4aTkLw1uicd7N8FjC3aofcW9dSXIyLFg1aEUp1rdSUv+RHxkGD5cfwKfjOqC7o1roPXUler7dWPDsfWl29zSumLOQ60YE1LM+egxfQ2WjO2JtvXjEebDXIzKQ4xVB1MweuFOvHpna4x06O/ruu5lcx4yci1oWbfke+htJ6+pTTlPvz0Ef55Px2s/H8TiJ3tWWFB0NSsfD8zfgk9HdUPDGpFoM3UlsvKtOP32EAgh8O7KI5j3+wlsmXwr6sVFqNsJIZBTYEOUyYDzaTno/c46tE6IxbLxfSpkP4oTNEGePwoKChAZGYnvvvsOw4YNU5ePHDkS6enp+PHHH922mTZtmsfJ1oMpyDPnWdBu2m8VXYxKyaCTEBmmh0GvUwPaPKtN807t/tIV3nDrJOebdsfAUnmtrBtWuA8FNhn5FhvMeVY0qRmFNvXj0K1xddSMNqFJrSisP5KKO9rXQ724CHy78xw6NayGq1n5+HzLGWQXWLHrTBoKrDLyC0fpCjPoEKbXIafAfqPiqemWEmz4wrXcEopeKw8TdJKEjFzP58CfvBy3MegkWGwCRr1UNEmsA8f2/pIE6AtHFbQVBkDKww7H8ut09mXmPAuqRYbBqLdfQxcz7M2IEuLCER6mx/m0XEQY9Xjq5qZYsvu82uFe8nIcJElCrsWmPt3USUVzloUbdLDJAtkFNrd98MS1H4O34wOExAjkIUV5tiapn3v7g6dwo30qhGuFAyd1blQNbevH4cjlTEy980anmyeLTUan11ahVUIsvh3T0+8yZOZZ8NcFOwDYB+cw6nW4mJ6LK5n5kCSgemQYcgpssMqy+rmSJPtE3WF6HfKs9mDP8RoLN+rUB2iOn33J4Xflc5CRa4EsBCLDDNDroN50Wqz2h3JXs/IRptchLtKICKMeVpuMGtEmJMSH40Rqthos51lkZORakJVvRfM60UiIj0BOvg3XcwqglyREmvSICjMgIkyPAquMzDwL9DoJF9Ptn2WdDsjOt09DIUn2mofL5jxUizSiYfVIxIQbUTvWhLTsAtSvFgFZAH2a1UTduHAkX8hA7RgTWtSNRb7Vhqw8K9rUj3NqniUXppcQX3RTWFFSM/NRMzqsVDfRH288ifrxERjUtp5m5ZFlAZ1OQp7FhtveW4/n+t+AL7edxcPdG+K+LonFbmuxyZjyQzJeGtwK8ZFFzZIPXzajaa1oGPX26+lkahYWbjmDlwa3QphBh3yrDTNWHMGQdvXQsWE1ZORakG+1oXaMvSnfr8mX0Kx2NG5wCVyXJ1/C9lPXMe3O1nh3xWHERxpx+HIm3vq/tur5zrfa0OHVVXj7nra4q0N9LE++hIVbTmP+I13UrgQ5BVYM/2gbpt/dFq3q+X+veepqNhpVj4ROJ+FSRi7iI8Kw91w64iONxaYnywKfbz2DB7slwmTwr/ngFXMe3l15BG/+Xxu/t/XGnGfBv387iqduborahfN4Bpu959KRW2BDz6ZFD0OUwNcbIQRGL9yFx/s0Rg8PD1EqWrkHeTqdzumAOR5ASZJgtRY/FwsAXLx4EfXr18cff/yBnj2L/ti98MILWL9+PbZt2+a2TWWpyVOaEigBgWPNhqe6MWW549N5x8ACsN8YWmVZrS2SBdQbDce8HPl6s6jUkjhup5ck2Hy4XBxX8bSfavkKazIcO207rqOTJBj1kscPojLfitJcxaUi0+mpu7qN8HyMlBovq81eEm+BmxJAaPV0SvnolTa94raXZQFbYY2cQSdBp5NgLRwFy/FGzzUoKuu+KZ97ubDW1dNNsKc8XL9whSiqMXHdrqzHTStKGb3tk9Jnw/G6UcruWCOoBLdKsOrrOXEeSa+odskxYHStHXfe3n09Zbk/h9a3uv3CdQN0yvxJV6mdLSktx1pRX65hb5Qb4PImCmuWDfqiJ/3K94JB5/y9WmCV1eDMapPVBxa+5iMEvO6jxSarN+q+pCUL9yZhRERUxNcgT7OBVzIzM92W/fjjj3j55ZfRpEkTrbJxYzKZYDKZApa+FiRJQmQA5jnTS4BeV74dQYNltjblhiJMw5uB8p5vvKxBSnHb63QSdC4338rNnkYP8Iotk04n+XVuXPfFcWCJktatKMWVEVBuVN33Cyj87Dq8F2bwf5+cml6qvwbHsaEiFRHgAfbrw+AyQqOn7wUATk2+DD4GZI75FPc58DXAU9LS8xImItKEZre1UVFFHSW3bNmCF198EdnZ2Zg/fz4GDBjgUxo1a9aEXq9HSkqK0/KUlBTUrVtXq6ISERERERGFLM0mQweAw4cPY9iwYXjkkUcwduxY7Nq1y+cADwDCwsLQuXNnrFlTNLKgLMtYs2aNU/NNIiIiIiIi8kyzmrzRo0fj119/xeTJk7FkyRLo9aVrEzZhwgSMHDkSXbp0Qbdu3TBz5kxkZ2fjscce06qoREREREREIUvTgVciIyNhNBo9DsBy/fp1n9OaM2eOOhl6hw4dMGvWLHTv3t2nbTMyMhAfH49z584FzcArREREREREZaUMMpmeno64uDiv62kW5J05c6bY9xs1aqRFNiU6f/48EhOLH7KXiIiIiIiosjp37hwaNGjg9f1ymSdvz5496NixY6CzAWDvw3fx4kXExMQEzSh8SsTN2sXQx3NdNfA8Vx0811UDz3PVwXNdNYTyeRZCIDMzEwkJCdDpvA+voumg8Tt37sSZM2dw8803o0aNGjhw4ACmTJmCzZs3IzU1VcusvNLpdMVGtRUpNjY25C408oznumrgea46eK6rBp7nqoPnumoI1fNcXDNNhWaja77zzjvo378/ZsyYgZ49e2L27Nno2rUrmjVrhmPHjmmVDRERERERERVDs5q8BQsW4ODBg0hISMDhw4fRpk0brFy5ErfddptWWRAREREREVEJNKvJCw8PR0JCAgCgZcuWaN68OQO8QiaTCVOnToXJZKroolCA8VxXDTzPVQfPddXA81x18FxXDTzPGg680qpVK3z77bdQknvggQecXrdr106LbIiIiIiIiKgYmgV5SUlJXkezlCQJJ0+e1CIbIiIiIiIiKka5TKFARERERERE5UOzPnlERERERERU8RjkERERERERhRAGeeVg7ty5SEpKQnh4OLp3747t27dXdJGoGBs2bMDQoUORkJAASZKwdOlSp/eFEHjllVdQr149REREoH///m5zQV6/fh0PP/wwYmNjER8fj7/97W/IyspyWufPP/9Enz59EB4ejsTERLz77ruB3jVyMH36dHTt2hUxMTGoXbs2hg0bhiNHjjitk5eXh6effho1atRAdHQ07rnnHqSkpDitc/bsWQwZMgSRkZGoXbs2/vGPf8BqtTqt8/vvv6NTp04wmUxo1qwZFixYEOjdo0Lz5s1Du3bt1Alxe/bsiV9//VV9n+c4NL399tuQJAnPPfecuoznOjRMmzYNkiQ5/WvZsqX6Ps9z6Lhw4QJGjBiBGjVqICIiAm3btsXOnTvV93k/VgJBAfXNN9+IsLAw8cknn4gDBw6I0aNHi/j4eJGSklLRRSMvli9fLqZMmSK+//57AUD88MMPTu+//fbbIi4uTixdulTs27dP3HnnnaJx48YiNzdXXecvf/mLaN++vdi6davYuHGjaNasmXjooYfU9zMyMkSdOnXEww8/LPbv3y++/vprERERIebPn19eu1nlDRw4UHz66adi//79Yu/evWLw4MGiYcOGIisrS13nySefFImJiWLNmjVi586dokePHuKmm25S37daraJNmzaif//+Ys+ePWL58uWiZs2aYvLkyeo6J0+eFJGRkWLChAni4MGDYvbs2UKv14sVK1aU6/5WVT/99JNYtmyZOHr0qDhy5Ih46aWXhNFoFPv37xdC8ByHou3bt4ukpCTRrl078eyzz6rLea5Dw9SpU0Xr1q3FpUuX1H+pqanq+zzPoeH69euiUaNGYtSoUWLbtm3i5MmTYuXKleL48ePqOrwfKx6DvADr1q2bePrpp9XXNptNJCQkiOnTp1dgqchXrkGeLMuibt26YsaMGeqy9PR0YTKZxNdffy2EEOLgwYMCgNixY4e6zq+//iokSRIXLlwQQgjxn//8R1SrVk3k5+er67z44ouiRYsWAd4j8ubKlSsCgFi/fr0Qwn5ejUajWLx4sbrOoUOHBACxZcsWIYT9gYBOpxOXL19W15k3b56IjY1Vz+0LL7wgWrdu7ZTXAw88IAYOHBjoXSIvqlWrJj7++GOe4xCUmZkpbrjhBrFq1SrRr18/NcjjuQ4dU6dOFe3bt/f4Hs9z6HjxxRdF7969vb7P+7GSsblmABUUFGDXrl3o37+/ukyn06F///7YsmVLBZaMSuvUqVO4fPmy0zmNi4tD9+7d1XO6ZcsWxMfHo0uXLuo6/fv3h06nw7Zt29R1+vbti7CwMHWdgQMH4siRI0hLSyunvSFHGRkZAIDq1asDAHbt2gWLxeJ0rlu2bImGDRs6neu2bduiTp066joDBw6E2WzGgQMH1HUc01DW4XdA+bPZbPjmm2+QnZ2Nnj178hyHoKeffhpDhgxxOx8816Hl2LFjSEhIQJMmTfDwww/j7NmzAHieQ8lPP/2ELl264L777kPt2rXRsWNHfPTRR+r7vB8rGYO8ALp69SpsNpvTFwkA1KlTB5cvX66gUlFZKOetuHN6+fJl1K5d2+l9g8GA6tWrO63jKQ3HPKj8yLKM5557Dr169UKbNm0A2M9DWFgY4uPjndZ1PdclnUdv65jNZuTm5gZid8hFcnIyoqOjYTKZ8OSTT+KHH37AjTfeyHMcYr755hvs3r0b06dPd3uP5zp0dO/eHQsWLMCKFSswb948nDp1Cn369EFmZibPcwg5efIk5s2bhxtuuAErV67E2LFjMX78eHz22WcAeD/mC0NFF4CIqKI9/fTT2L9/PzZt2lTRRaEAaNGiBfbu3YuMjAx89913GDlyJNavX1/RxSINnTt3Ds8++yxWrVqF8PDwii4OBdCgQYPU39u1a4fu3bujUaNG+PbbbxEREVGBJSMtybKMLl264K233gIAdOzYEfv378eHH36IkSNHVnDpKgfW5AVQzZo1odfr3UZ1SklJQd26dSuoVFQWynkr7pzWrVsXV65ccXrfarXi+vXrTut4SsMxDyof48aNwy+//IJ169ahQYMG6vK6deuioKAA6enpTuu7nuuSzqO3dWJjY3lDUk7CwsLQrFkzdO7cGdOnT0f79u3xwQcf8ByHkF27duHKlSvo1KkTDAYDDAYD1q9fj1mzZsFgMKBOnTo81yEqPj4ezZs3x/Hjx/mZDiH16tXDjTfe6LSsVatWatNc3o+VjEFeAIWFhaFz585Ys2aNukyWZaxZswY9e/aswJJRaTVu3Bh169Z1Oqdmsxnbtm1Tz2nPnj2Rnp6OXbt2qeusXbsWsiyje/fu6jobNmyAxWJR11m1ahVatGiBatWqldPeVG1CCIwbNw4//PAD1q5di8aNGzu937lzZxiNRqdzfeTIEZw9e9bpXCcnJzv9EVm1ahViY2PVP049e/Z0SkNZh98BFUeWZeTn5/Mch5DbbrsNycnJ2Lt3r/qvS5cuePjhh9Xfea5DU1ZWFk6cOIF69erxMx1CevXq5Tat0dGjR9GoUSMAvB/zSUWP/BLqvvnmG2EymcSCBf/f3v3HVFX/cRx/XYUL3hCl3TsMAhx6pysL3BrrtrQaDKf9tDWIsRSroTHsF2vphiE2GzZSi5krWX1zYuQfaVrbjWnQqIXkJJnTkW0qrR9zyyicKATv7x/O++30BXTDH3B6PjY2zjnv8/l87j3/3Nf93PM5/7EjR45YcXGxTZ482bGqE0aX7u5ua2trs7a2NpNk69evt7a2Njt58qSZXViyd/LkyfbJJ59Ye3u7Pfzww4Mu2Tt79mzbv3+/ffXVVxYMBh1L9nZ1dVliYqI98cQTdvjwYauvrzefz+eKJXvHimeeecYmTZpkTU1NjqW4z549G6lZtmyZpaam2hdffGEHDhywUChkoVAocvziUty5ubn23XffWTgctkAgMOhS3C+99JIdPXrUNm3axFLc19CKFSvsyy+/tOPHj1t7e7utWLHCPB6PNTQ0mBnX2M3+vrqmGdfaLcrKyqypqcmOHz9uX3/9teXk5Jjf77dTp06ZGdfZLVpbWy0qKsrWrl1rx44ds7q6OvP5fLZt27ZIDZ/HhkfIuwZqamosNTXVvF6vZWVlWUtLy/UeEobR2Nhokv7vb/HixWZ2YdneVatWWWJiosXExFh2drZ1dHQ42vjtt9+soKDA4uLiLD4+3pYsWWLd3d2OmkOHDtndd99tMTExlpycbFVVVdfqJcJs0Gssyd5///1ITU9Pj5WUlFhCQoL5fD5buHCh/fLLL452Tpw4YfPnz7cJEyaY3++3srIy6+vrc9Q0NjZaZmameb1eS09Pd/SBq+vJJ5+0tLQ083q9FggELDs7OxLwzLjGbvbPkMe1dof8/Hy76aabzOv1WnJysuXn5zuencZ1do89e/bYrFmzLCYmxmbOnGnvvvuu4zifx4bnMTO7PnOIAAAAAIArjXvyAAAAAMBFCHkAAAAA4CKEPAAAAABwEUIeAAAAALgIIQ8AAAAAXISQBwAAAAAuQsgDAAAAABch5AEAAACAixDyAABjxr333qvnn39+2JqpU6dq48aNI+6ro6NDU6ZMUXd394jbGs7q1auVmZk54nYef/xxvfHGGyMfEABgzCPkAQBc5dtvv1VxcfGI21m5cqWWL1+uiRMnSroQ+u677z4lJiYqNjZW6enpKi8vV19fX+ScLVu2aM6cOUpISFBCQoJycnLU2to64rFcjvLycq1du1Z//PHHNekPADB6EfIAAK4SCATk8/lG1EZnZ6c+/fRTFRUVRfZFR0dr0aJFamhoUEdHhzZu3KgtW7aooqIiUtPU1KSCggI1Njbqm2++UUpKinJzc/XTTz+NaDyXY9asWZo2bZq2bdt21fsCAIxuhDwAwJjy119/qbS0VJMmTZLf79eqVatkZpHj//y5psfjUW1trRYuXCifz6dgMKjdu3cP28eOHTuUkZGh5OTkyL709HQtWbJEGRkZSktL00MPPaTCwkI1NzdHaurq6lRSUqLMzEzNnDlTtbW1GhgY0L59+y75ut555x2lpKTI5/MpLy/PMSNXVFSkRx55RJWVlQoEAoqPj9eyZcvU29vraOPBBx9UfX39JfsCALgbIQ8AMKZ88MEHioqKUmtrq958802tX79etbW1w55TWVmpvLw8tbe3a8GCBSosLNTp06eHrG9ubtYdd9wxbJs//PCDwuGw7rnnniFrzp49q76+Pt14442XbGvHjh3as2ePwuGw2traVFJS4qjZt2+fjh49qqamJn344Yf6+OOPVVlZ6ajJyspSa2urzp8/P2x/AAB3I+QBAMaUlJQUbdiwQTNmzFBhYaGWL1+uDRs2DHtOUVGRCgoKNH36dL322ms6c+bMsPfKnTx5UklJSYMeu+uuuxQbG6tgMKg5c+ZozZo1Q7bz8ssvKykpSTk5OcOO79y5c9q6dasyMzM1d+5c1dTUqL6+Xr/++mukxuv16r333tOtt96q+++/X2vWrNFbb72lgYGBSE1SUpJ6e3sd5wEA/n0IeQCAMeXOO++Ux+OJbIdCIR07dkz9/f1DnnP77bdH/r/hhhsUHx+vU6dODVnf09Oj2NjYQY999NFHOnjwoLZv367PPvtM1dXVg9ZVVVWpvr5eO3fuHLKti1JTUx0/DQ2FQhoYGFBHR0dkX0ZGhuNew1AopDNnzujHH3+M7JswYYKkCzOIAIB/r6jrPQAAAK626Ohox7bH43HMgP2T3+/X77//PuixlJQUSdItt9yi/v5+FRcXq6ysTOPHj4/UVFdXq6qqSnv37nUEzKvt4k9QA4HANesTADD6MJMHABhT9u/f79huaWlRMBh0hKyRmj17to4cOXLJuoGBAfX19TkC4+uvv65XX31V4XD4kvf1XdTZ2amff/45st3S0qJx48ZpxowZkX2HDh1ST0+PoyYuLi4SOiXp8OHDuvnmm+X3+y+rXwCAOzGTBwAYUzo7O/Xiiy9q6dKlOnjwoGpqaq74Q8DnzZunp59+Wv39/ZHwWFdXp+joaN12222KiYnRgQMHtHLlSuXn50dmCtetW6dXXnlF27dv19SpUyP3xsXFxSkuLm7I/mJjY7V48WJVV1frzz//1LPPPqu8vDxNmTIlUtPb26unnnpK5eXlOnHihCoqKlRaWqpx4/73fW1zc7Nyc3Ov6HsBABh7CHkAgDFl0aJF6unpUVZWlsaPH6/nnnvuijz8/O/mz5+vqKgo7d27V/PmzZMkRUVFad26dfr+++9lZkpLS1NpaaleeOGFyHmbN29Wb2+vHnvsMUd7FRUVWr169ZD9TZ8+XY8++qgWLFig06dP64EHHtDbb7/tqMnOzlYwGNTcuXN1/vx5FRQUONo8d+6cdu3apXA4PPI3AAAwpnns7w8XAgAAkqRNmzZp9+7d+vzzz6/3UFRUVKSuri7t2rVryJrNmzdr586damhouHYDAwCMSszkAQAwiKVLl6qrq0vd3d2aOHHi9R7OJUVHR6umpuZ6DwMAMAowkwcAwCh3OTN5AABcRMgDAAAAABfhEQoAAAAA4CKEPAAAAABwEUIeAAAAALgIIQ8AAAAAXISQBwAAAAAuQsgDAAAAABch5AEAAACAixDyAAAAAMBF/gvq7ydgB2vLRQAAAABJRU5ErkJggg==", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "import matplotlib.pyplot as plt\n", + "\n", + "# Track \"identifier\" is a raw file accession (e.g. \"ENCFF281BWX+\"); the\n", + "# human-readable assay/tissue lives in \"description\" as \"ASSAY:detail\".\n", + "rna_tracks = track_meta[track_meta[\"description\"].str.startswith(\"RNA:\")].head(2)\n", + "assert len(rna_tracks) > 0, \"no RNA tracks found in track metadata\"\n", + "\n", + "fig, axes = plt.subplots(len(rna_tracks), 1, figsize=(9, 4), sharex=True, squeeze=False)\n", + "for ax, (track_idx, row) in zip(axes[:, 0], rna_tracks.iterrows()):\n", + " ax.plot(profile[track_idx], lw=0.8)\n", + " ax.set_ylabel(row[\"description\"][:25], fontsize=8)\n", + "axes[-1, 0].set_xlabel(\"bin (32 bp)\")\n", + "fig.suptitle(\"Predicted Borzoi coverage profile around SORT1\")\n", + "fig.tight_layout()\n", + "fig.savefig(OUTPUT_DIR / \"borzoi_profile_example.png\", dpi=120)\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "id": "vep-09", + "metadata": {}, + "source": [ + "## 2. `SNPEmbedder`: ref/alt by default, delta and concat on request\n", + "\n", + "`SNPEmbedder.embed_snp()` always returns `ref_embedding` and `alt_embeddings` (one per alt allele). `delta_embeddings`/`delta_norms` are computed by default, but can be skipped with `compute_delta=False`; a concatenated `[ref, alt]` feature vector is available on request via `compute_concat=True`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "vep-10", + "metadata": { + "execution": { + "iopub.execute_input": "2026-06-23T12:39:50.127549Z", + "iopub.status.busy": "2026-06-23T12:39:50.127362Z", + "iopub.status.idle": "2026-06-23T12:44:01.967382Z", + "shell.execute_reply": "2026-06-23T12:44:01.925035Z" + } + }, + "outputs": [], + "source": [ + "snp = SNPContext(\n", + " chrom=CHROM,\n", + " position=snp_offset, # 1-based offset\n", + " ref_allele=REF,\n", + " alt_alleles=[ALT],\n", + " context_window=BorzoiWrapper.SEQUENCE_LENGTH,\n", + " strand=\"+\",\n", + " variant_id=\"rs12740374\",\n", + ")\n", + "\n", + "embedder = SNPEmbedder(wrapper, pooling_strategy=\"mean\")\n", + "\n", + "# Default: ref + alt + delta, no concat.\n", + "result = embedder.embed_snp(snp, chromosome_sequence=window)\n", + "print(\"ref_embedding:\", result.ref_embedding.shape)\n", + "print(\"alt_embeddings:\", [a.shape for a in result.alt_embeddings])\n", + "print(\"delta_embeddings (default on):\", [d.shape for d in result.delta_embeddings])\n", + "print(\"concat_embeddings (default off):\", result.concat_embeddings)\n", + "\n", + "# Skip delta, request concat instead.\n", + "result_concat = embedder.embed_snp(\n", + " snp, chromosome_sequence=window, compute_delta=False, compute_concat=True\n", + ")\n", + "print()\n", + "print(\"compute_delta=False ->\", result_concat.delta_embeddings)\n", + "print(\"compute_concat=True ->\", [c.shape for c in result_concat.concat_embeddings])" + ] + }, + { + "cell_type": "markdown", + "id": "vep-11", + "metadata": {}, + "source": [ + "## 3. Profile-based variant-effect prediction (Borzoi-style)\n", + "\n", + "The Borzoi paper score a variant's effect on a gene by summing the model's predicted (linear-scale) coverage over the gene's exon-overlapping bins for the reference and alternate alleles, then reporting the log2 fold-change with a pseudocount. `SNPEmbedder.predict_variant_effect()` implements exactly this, generically for any model wrapper exposing `predict_profile()`.\n", + "\n", + "`genomic_to_bin_indices()` converts genomic exon coordinates into the bin indices `predict_variant_effect` aggregates over. For *SORT1*, we use its annotated exon span; in a real workflow these would come from a GTF or `GeneResolver.get_gene_regions(...)`." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "vep-12", + "metadata": { + "execution": { + "iopub.execute_input": "2026-06-23T12:44:01.996328Z", + "iopub.status.busy": "2026-06-23T12:44:01.996091Z", + "iopub.status.idle": "2026-06-23T12:46:13.938803Z", + "shell.execute_reply": "2026-06-23T12:46:13.937935Z" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "151 bins overlap the SORT1 exon used here.\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Reference profile shape: (7611, 6144)\n", + "Effect scores (per track) for alt 'T': (7611,)\n" + ] + }, + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
tracklog2_fold_change
0ENCFF611FWS1.467083
1ENCFF746CXV1.463224
2ENCFF893KHE1.435684
3ENCFF837FTN1.370417
4ENCFF623CXL-1.361386
5ENCFF360QQU1.340232
6ENCFF301WTK-1.280393
7ENCFF891SNS1.252314
8ENCFF905FLR1.175283
9ENCFF107LVE1.071769
\n", + "
" + ], + "text/plain": [ + " track log2_fold_change\n", + "0 ENCFF611FWS 1.467083\n", + "1 ENCFF746CXV 1.463224\n", + "2 ENCFF893KHE 1.435684\n", + "3 ENCFF837FTN 1.370417\n", + "4 ENCFF623CXL -1.361386\n", + "5 ENCFF360QQU 1.340232\n", + "6 ENCFF301WTK -1.280393\n", + "7 ENCFF891SNS 1.252314\n", + "8 ENCFF905FLR 1.175283\n", + "9 ENCFF107LVE 1.071769" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Real SORT1 (GRCh38, chr1, ENST00000256637) exons near the variant, fetched once via\n", + "# the Ensembl REST API (https://rest.ensembl.org/overlap/id/ENSG00000134243?feature=exon);\n", + "# in practice use GeneResolver.get_gene_regions(\"SORT1\", region=\"exons\") or a GTF file.\n", + "sort1_exons = [\n", + " (109_309_575, 109_314_057),\n", + " (109_314_261, 109_314_384),\n", + " (109_314_672, 109_314_778),\n", + "]\n", + "\n", + "bin_indices = genomic_to_bin_indices(\n", + " sort1_exons,\n", + " window_start=window_start,\n", + " bin_size=wrapper.BIN_SIZE,\n", + " profile_offset_bp=wrapper.profile_offset_bp,\n", + " num_bins=profile.shape[1],\n", + ")\n", + "print(f\"{len(bin_indices)} bins overlap the SORT1 exon used here.\")\n", + "\n", + "vep_result = embedder.predict_variant_effect(\n", + " snp,\n", + " chromosome_sequence=window,\n", + " bin_indices=bin_indices,\n", + " undo_squashed_scale=True, # required: the statistic assumes linear-scale coverage\n", + ")\n", + "\n", + "print(f\"Reference profile shape: {vep_result.ref_profile.shape}\")\n", + "print(f\"Effect scores (per track) for alt '{ALT}': {vep_result.effect_scores[0].shape}\")\n", + "\n", + "top_hits = vep_result.top_tracks(alt_index=0, n=10)\n", + "pd.DataFrame(top_hits, columns=[\"track\", \"log2_fold_change\"])" + ] + }, + { + "cell_type": "markdown", + "id": "vep-13", + "metadata": {}, + "source": [ + "The tracks at the top of this table are Borzoi's prediction of which assays (cell types / RNA-seq vs ATAC vs ChIP) are most affected by the variant at the *SORT1* locus." + ] + }, + { + "cell_type": "markdown", + "id": "e8174345", + "metadata": {}, + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "embpy", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.12" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/technical.md b/docs/technical.md index 7197aab..d9bd56b 100644 --- a/docs/technical.md +++ b/docs/technical.md @@ -127,10 +127,12 @@ pixi run -e default verify Common environments: ```bash -pixi install -e default # CPU/dev -pixi install -e mps # Apple Silicon/PyTorch MPS -pixi install -e gpu # Linux CUDA GPU -pixi install -e boltz # Boltz-specific examples +pixi install -e default # CPU/dev +pixi install -e mps # Apple Silicon/PyTorch MPS +pixi install -e gpu # Linux CUDA GPU +pixi install -e boltz # Boltz-specific examples +pixi install -e alphagenome # AlphaGenome cloud API client (opt-in) +pixi install -e scooby # Scooby single-cell DNA model (opt-in, git install) ``` Pip users can start with: @@ -142,6 +144,29 @@ pip install embpy Optional model families may require extra dependencies. Large GPU models should be installed in an environment with compatible PyTorch/CUDA wheels. +Pip extras for individual model families (see `pyproject.toml` for the full, +up-to-date list and per-extra caveats): + +```bash +pip install "embpy[seqmodels]" # Borzoi, Enformer +pip install "embpy[alphagenome]" # AlphaGenome (cloud API; needs ALPHAGENOME_API_KEY) +pip install "embpy[scooby]" # Scooby (installs from git; see pyproject.toml) +pip install "embpy[esm3]" # ESM-3 / ESM-C +pip install "embpy[helical]" # Helical-backed single-cell models +pip install "embpy[state]" # Arc STATE +pip install "embpy[stack]" # Arc STACK +pip install "embpy[all]" # every extra that installs cleanly via plain pip +``` + +`embpy[all]` intentionally excludes extras that need a compiler/CUDA toolchain +or pre-built C library (`caduceus`, `evo2`, `helical`), need torch +pre-installed to build (`minimol`), pin incompatible dependency versions +(`esm3`, `boltz`), install from a git URL (`scooby`), or call an external +hosted API (`alphagenome`) -- install those explicitly. + +`AlphaGenomeWrapper` calls a hosted API rather than loading local weights, so +set `ALPHAGENOME_API_KEY` in the environment before calling `.load()`. + ## Model Keys The active model registry depends on the installed optional dependencies. diff --git a/pixi.toml b/pixi.toml index 0f5ad86..fd24c72 100644 --- a/pixi.toml +++ b/pixi.toml @@ -21,6 +21,8 @@ # pixi shell -e gpu # activate the CUDA 12.4 env # pixi run -e dev test # run the test suite in the dev env # pixi run -e docs build-docs # build the Sphinx docs +# pixi install -e alphagenome # AlphaGenome cloud API client (opt-in) +# pixi install -e scooby # Scooby single-cell DNA model (opt-in, git install) # --------------------------------------------------------------------------- [workspace] @@ -158,6 +160,30 @@ transformers = ">=4.45,<4.51" borzoi-pytorch = ">=0.4.3" enformer-pytorch = ">=0.8.10" +# AlphaGenome (Google DeepMind cloud API client). No local weights are +# downloaded; inference runs against Google's servers and needs +# ALPHAGENOME_API_KEY set in the environment. Pure-Python with no CUDA/build +# requirements -- kept opt-in (rather than folded into default/gpu/dev) so +# the main envs don't gain a hard dependency on an external API account. +[feature.alphagenome.pypi-dependencies] +alphagenome = ">=0.7" + +# Scooby (single-cell-resolution DNA sequence model, gagneurlab/scooby). +# Installed from git because PyPI's `scooby` package name is already taken +# by an unrelated project (a pyvista/VTK system-report utility) -- the real +# package has to be pulled from the gagneurlab fork by URL, not by name. It +# also pulls in a patched `peft` fork via git (scooby's own setup.py) plus +# `borzoi-pytorch`/`enformer-pytorch`, which pin `transformers<4.51.0` -- +# mirrored on the conda side here like `seqmodels`. +# Kept as its own standalone environment (see `[environments]` below) so the +# git-based resolve doesn't slow down or destabilize the main envs. +[feature.scooby.dependencies] +transformers = ">=4.45,<4.51" + +[feature.scooby.pypi-dependencies] +snapatac2-scooby = "*" +scooby = { git = "https://github.com/gagneurlab/scooby.git" } + # --------------------------------------------------------------------------- # Evo 2 (Arc Institute long-context DNA LM, StripedHyena 2 architecture via # Vortex). Linux-64 + CUDA 12 only. @@ -584,6 +610,21 @@ caduceus = { features = [ "jupyter", ] } +# Standalone env for AlphaGenome (cloud API client, no local weights). Kept +# out of default/gpu/dev so those envs don't gain a hard dependency on an +# external API account; CPU-only since there's no local inference. Usage: +# pixi install -e alphagenome && pixi shell -e alphagenome +# export ALPHAGENOME_API_KEY=... +alphagenome = { features = ["cpu", "alphagenome", "morphology", "jupyter"], solve-group = "cpu" } + +# Standalone env for Scooby (single-cell-resolution DNA sequence model). +# Kept separate from `default`/`gpu` because it installs from a git URL (see +# `[feature.scooby]` above), which would slow down/destabilize the main +# solve-groups. GPU only in practice -- Scooby shares Borzoi's trunk, which +# needs a GPU for reasonable runtime. Usage: +# pixi install -e scooby && pixi shell -e scooby +scooby = { features = ["gpu", "scooby", "morphology", "jupyter"] } + # CPU env that also pulls the docs + dev extras. Useful for tutorial work. dev = { features = ["cpu", "morphology", "jump", "scanpy", "cell_eval", "esm3", "seqmodels", "jupyter", "dev", "docs"], solve-group = "cpu" } diff --git a/pyproject.toml b/pyproject.toml index 73c3363..0c928ca 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -93,6 +93,20 @@ optional-dependencies.seqmodels = [ "borzoi-pytorch>=0.4.3", "enformer-pytorch>=0.8.10", ] +# AlphaGenome (Google DeepMind cloud API client). No local weights are +# downloaded; inference runs against Google's servers and requires an +# `ALPHAGENOME_API_KEY` in the environment. Pure-Python, no CUDA/compiler +# needed -- kept out of "all" anyway (see below) since it implies an +# external API account rather than a local model. +optional-dependencies.alphagenome = [ + "alphagenome>=0.7", +] +# Scooby (gagneurlab/scooby): single-cell-resolution DNA sequence model built +# on the Borzoi trunk, conditioned on a precomputed per-cell embedding. +optional-dependencies.scooby = [ + "snapatac2-scooby", + "scooby @ git+https://github.com/gagneurlab/scooby.git", +] # MiniMol molecule GNN. Kept separate because it transitively depends on # `torch-sparse`, `torch-scatter` etc. which fail to build via pip unless # torch is pre-installed (the packages don't declare torch as a build-time @@ -163,6 +177,10 @@ optional-dependencies.morphology = [ # * helical -- transitively needs the igraph C library (louvain) which # pip cannot build; use conda/pixi for helical instead # * minimol -- transitive torch-sparse build chain; use conda/pixi instead +# * scooby -- pulls in `scooby` and `peft` via direct git references (see +# the scooby extra above); install it explicitly if you need it +# * alphagenome -- calls a hosted API and needs an ALPHAGENOME_API_KEY; +# kept opt-in so "all" doesn't imply an external API account # Install those explicitly in a separate env if you need them. optional-dependencies.all = [ "embpy[torch,ppi,lamindb,pertpy,scanpy,morphology,seqmodels]", diff --git a/src/embpy/embedder_registry/dna.py b/src/embpy/embedder_registry/dna.py index 7236015..77e7fb9 100644 --- a/src/embpy/embedder_registry/dna.py +++ b/src/embpy/embedder_registry/dna.py @@ -8,12 +8,12 @@ ``BioEmbedder.embed_gene``'s human/mouse guard and should live next to the DNA registry; they do, now. -The optional ``EvoWrapper`` / ``Evo2Wrapper`` gating uses the -``_HAVE_*`` flag pattern so the entire dict can be constructed even -when the optional dependency is missing -- the wrapper slot for -unavailable models is ``None`` and the per-call dispatch (see -``BioEmbedder._discover_models``) drops the entry from the user-facing -listing. +The optional ``EvoWrapper`` / ``Evo2Wrapper`` / ``AlphaGenomeWrapper`` / +``ScoobyWrapper`` gating uses the ``_HAVE_*`` flag pattern so the entire +dict can be constructed even when the optional dependency is missing -- +the wrapper slot for unavailable models is ``None`` and the per-call +dispatch (see ``BioEmbedder._discover_models``) drops the entry from the +user-facing listing. """ from __future__ import annotations @@ -45,6 +45,22 @@ _HAVE_EVO2 = False Evo2Wrapper = None # type: ignore +try: + from ..models.alphagenome_models import AlphaGenomeWrapper + + _HAVE_ALPHAGENOME = True +except ImportError: + _HAVE_ALPHAGENOME = False + AlphaGenomeWrapper = None # type: ignore + +try: + from ..models.scooby_models import ScoobyWrapper + + _HAVE_SCOOBY = True +except ImportError: + _HAVE_SCOOBY = False + ScoobyWrapper = None # type: ignore + DNA_MODELS: dict[str, tuple[type[BaseModelWrapper] | None, str | None]] = { "enformer_human_rough": (EnformerWrapper, "EleutherAI/enformer-official-rough"), @@ -141,6 +157,18 @@ CaduceusWrapper, "kuleshov-group/caduceus-ps_seqlen-131k_d_model-256_n_layer-16", ), + # AlphaGenome (Google DeepMind cloud API client) -- pip install embpy[alphagenome] + "alphagenome": (AlphaGenomeWrapper if _HAVE_ALPHAGENOME else None, "alphagenome"), + # Scooby (gagneurlab/scooby) -- pip install embpy[scooby]. Single-cell-resolution + # DNA sequence model conditioned on a precomputed per-cell embedding; the model + # path selects the checkpoint (and, via ScoobyWrapper.KNOWN_CHECKPOINTS, its + # architecture hyperparameters). + "scooby_onek1k": (ScoobyWrapper if _HAVE_SCOOBY else None, "lauradmartens/onek1k-scooby"), + "scooby_neurips": (ScoobyWrapper if _HAVE_SCOOBY else None, "johahi/neurips-scooby"), + "scooby_epicardioids": ( + ScoobyWrapper if _HAVE_SCOOBY else None, + "lauradmartens/epicardioids-scooby", + ), } diff --git a/src/embpy/models/__init__.py b/src/embpy/models/__init__.py index 0ad9ca0..2bbc148 100644 --- a/src/embpy/models/__init__.py +++ b/src/embpy/models/__init__.py @@ -63,6 +63,18 @@ singlecell_info, ) +# AlphaGenome is optional (requires: pip install alphagenome) +try: + from .alphagenome_models import AlphaGenomeWrapper +except ImportError: + AlphaGenomeWrapper = None # type: ignore + +# Scooby is optional (requires: pip install snapatac2-scooby && pip install git+https://github.com/gagneurlab/scooby.git) +try: + from .scooby_models import ScoobyWrapper +except ImportError: + ScoobyWrapper = None # type: ignore + __all__ = [ "BorzoiWrapper", "EnformerWrapper", @@ -100,4 +112,7 @@ "get_singlecell_wrapper", "list_singlecell_models", "singlecell_info", + "AlphaGenomeWrapper", + "ScoobyWrapper" ] + diff --git a/src/embpy/models/alphagenome_models.py b/src/embpy/models/alphagenome_models.py new file mode 100644 index 0000000..7536e91 --- /dev/null +++ b/src/embpy/models/alphagenome_models.py @@ -0,0 +1,334 @@ +from __future__ import annotations + +import logging +import os +from collections.abc import Sequence +from typing import Any + +import numpy as np +import pandas as pd +import torch + +from .base import BaseModelWrapper + +logger = logging.getLogger(__name__) + +try: + from alphagenome.data import genome + from alphagenome.models import dna_client, dna_output, variant_scorers +except ImportError: + genome = None # type: ignore + dna_client = None # type: ignore + dna_output = None # type: ignore + variant_scorers = None # type: ignore + +ORGANISMS = {"human": "HOMO_SAPIENS", "mouse": "MUS_MUSCULUS"} + + +class AlphaGenomeWrapper(BaseModelWrapper): + """Wrapper for Google DeepMind's AlphaGenome sequence model (cloud API). + + This class handles: + 1. Resolving the API key from ``ALPHAGENOME_API_KEY`` and creating a + ``dna_client.DnaClient`` (no local weights to load). + 2. Predicting per-track profiles for a raw DNA sequence and pooling + them into a fixed-length embedding (analogue of the local DNA + model wrappers' ``embed``). + 3. Scoring a genomic variant's predicted effect on gene expression + (log2 fold-change between REF and ALT alleles), following the same + VCF-style chrom/pos/ref/alt convention and log2FC statistic as + ``embpy.tl.genomics.snp_utils.profile_variant_effect_score`` / + ``BorzoiWrapper``-based variant scoring elsewhere in this codebase. + + Parameters + ---------- + model_path_or_name : str, optional + Label only; AlphaGenome exposes a single hosted model per API + version (kept for interface consistency with ``BaseModelWrapper``). + model_version : str, optional + Name of an ``alphagenome.models.dna_client.ModelVersion`` member + (e.g. ``"FOLD_0"``). ``None`` uses the server default (all folds). + organism : str, optional + ``"human"`` or ``"mouse"``. Default ``"human"``. + context_window : int, optional + Sequence length (bp) used to build a default interval centered on + a variant position when explicit interval bounds are not supplied + to :meth:`predict_variant_effect`. Defaults to 1,048,576 (1 Mb, + AlphaGenome's maximum context). + **kwargs : Any + Forwarded to ``BaseModelWrapper.__init__``. + """ + + model_type = "dna" + available_pooling_strategies = ["mean", "max", "none"] + + DEFAULT_CONTEXT_WINDOW = 1_048_576 + + def __init__( + self, + model_path_or_name: str = "alphagenome", + model_version: str | None = None, + organism: str = "human", + context_window: int = DEFAULT_CONTEXT_WINDOW, + **kwargs: Any, + ): + super().__init__(model_path_or_name, **kwargs) + if organism not in ORGANISMS: + raise ValueError(f"Unknown organism '{organism}'. Choose from {list(ORGANISMS)}.") + self.organism = organism + self.model_version = model_version + self.context_window = context_window + self.client: Any = None + + @staticmethod + def _resolve_api_key() -> str: + key = os.environ.get("ALPHAGENOME_API_KEY") + if not key: + raise ValueError( + "ALPHAGENOME_API_KEY environment variable is not set. Source the " + "AlphaGenome API key file into the environment before calling " + "AlphaGenomeWrapper.load()." + ) + return key + + def load(self, device: torch.device | None = None) -> None: + """Create the AlphaGenome API client. There are no local weights to load. + + Parameters + ---------- + device : torch.device, optional + Accepted for interface consistency with ``BaseModelWrapper``; + AlphaGenome inference runs on Google's servers, not locally. + + Raises + ------ + ImportError + If the ``alphagenome`` package is not installed. + ValueError + If ``ALPHAGENOME_API_KEY`` is not set in the environment. + """ + if dna_client is None: + raise ImportError("alphagenome not installed; cannot load AlphaGenomeWrapper. Run `pip install alphagenome`.") + if self.client is not None: + logging.warning("AlphaGenome client already initialized.") + return + + api_key = self._resolve_api_key() + model_version = getattr(dna_client.ModelVersion, self.model_version) if self.model_version else None + self.client = dna_client.create(api_key, model_version=model_version) + self.model = self.client + self.device = device + logging.info(f"AlphaGenome API client ready (organism={self.organism}).") + + def _organism_enum(self) -> Any: + return getattr(dna_client.Organism, ORGANISMS[self.organism]) + + def _default_interval(self, chrom: str, pos: int) -> Any: + half = self.context_window // 2 + return genome.Interval(chromosome=chrom, start=max(pos - half, 0), end=pos + half) + + def embed( + self, + input: str, + pooling_strategy: str = "mean", + output_type: str = "RNA_SEQ", + ontology_terms: Sequence[str] | None = None, + **kwargs: Any, + ) -> np.ndarray: + """Predict a track for a DNA sequence and pool it into an embedding. + + Runs ``predict_sequence`` for a single output modality (default + RNA-seq) and pools the per-position track values across the + sequence, yielding one value per output track -- the AlphaGenome + analogue of ``BorzoiWrapper.embed``'s bin-pooled trunk embedding. + + Parameters + ---------- + input : str + Raw DNA sequence string (up to 1,048,576 bp). + pooling_strategy : str, default "mean" + "mean", "max", or "none" (returns the raw ``(positions, tracks)`` + array without pooling). + output_type : str, default "RNA_SEQ" + Name of an ``alphagenome.models.dna_output.OutputType`` member + (e.g. "RNA_SEQ", "ATAC", "DNASE", "CAGE", "CHIP_HISTONE"). + ontology_terms : sequence of str, optional + Restrict predictions to specific tissue/cell-type ontology + CURIEs (e.g. ``"UBERON:0002107"`` for liver). ``None`` returns + all tracks for the output type. + **kwargs : Any + Ignored; accepted for interface consistency. + + Returns + ------- + np.ndarray + Shape ``(num_tracks,)``, or ``(num_positions, num_tracks)`` if + ``pooling_strategy="none"``. + + Raises + ------ + RuntimeError + If the client hasn't been initialized (``load()`` not called). + ValueError + If ``pooling_strategy`` is invalid. + """ + if self.client is None: + raise RuntimeError("AlphaGenome client not initialized. Call load() first.") + if pooling_strategy not in self.available_pooling_strategies: + raise ValueError(f"Invalid pooling: '{pooling_strategy}'") + + out_type = getattr(dna_output.OutputType, output_type) + prediction = self.client.predict_sequence( + sequence=input, + organism=self._organism_enum(), + requested_outputs=[out_type], + ontology_terms=ontology_terms, + ) + track_data = getattr(prediction, output_type.lower()) + values = np.asarray(track_data.values, dtype=np.float32) # (positions, tracks) + + if pooling_strategy == "none": + return values + elif pooling_strategy == "mean": + return values.mean(axis=0) + else: + return values.max(axis=0) + + def embed_batch( + self, + inputs: Sequence[str], + pooling_strategy: str = "mean", + **kwargs: Any, + ) -> list[np.ndarray]: + """Sequentially embed a batch of DNA sequences (one API call each). + + AlphaGenome's client has no native multi-sequence batching for + ``predict_sequence``, so this loops calling :meth:`embed` per input, + mirroring ``APIEmbeddingWrapper``'s per-item API call pattern. + + Parameters + ---------- + inputs : Sequence[str] + DNA sequence strings. + pooling_strategy : str, default "mean" + Forwarded to :meth:`embed`. + **kwargs : Any + Forwarded to :meth:`embed`. + + Returns + ------- + list[np.ndarray] + One embedding per input, in order. + """ + if self.client is None: + raise RuntimeError("AlphaGenome client not initialized. Call load() first.") + return [self.embed(seq, pooling_strategy=pooling_strategy, **kwargs) for seq in inputs] + + def get_track_metadata(self, output_type: str | None = None) -> pd.DataFrame: + """Return AlphaGenome's output track metadata table. + + Analogue of ``BorzoiWrapper.get_track_metadata``, but fetched live + from the API since AlphaGenome has no bundled local track table. + + Parameters + ---------- + output_type : str, optional + Restrict to one ``dna_output.OutputType`` name (e.g. + "RNA_SEQ"). ``None`` concatenates metadata for every output + type. + + Returns + ------- + pandas.DataFrame + """ + if self.client is None: + raise RuntimeError("AlphaGenome client not initialized. Call load() first.") + metadata = self.client.output_metadata(organism=self._organism_enum()) + if output_type is not None: + return getattr(metadata, output_type.lower()) + frames = [df for df in vars(metadata).values() if isinstance(df, pd.DataFrame)] + return pd.concat(frames, ignore_index=True) if frames else pd.DataFrame() + + def predict_variant_effect( + self, + chrom: str, + pos: int, + ref: str, + alt: str, + interval_start: int | None = None, + interval_end: int | None = None, + output_type: str = "RNA_SEQ", + ontology_terms: Sequence[str] | None = None, + variant_id: str = "", + ) -> tuple[np.ndarray, list[str]]: + """Score a variant's predicted effect on gene expression. + + Uses AlphaGenome's ``GeneMaskLFCScorer``, which sums predicted + coverage over each gene's exons for the REF and ALT alleles and + returns the log2 fold-change -- the same statistic and the same + VCF-style chrom/pos/ref/alt input convention as + ``embpy.tl.genomics.snp_utils.profile_variant_effect_score`` / + ``SNPEmbedder.predict_variant_effect``, which underlie + ``BorzoiWrapper``-based variant scoring elsewhere in this codebase. + + Parameters + ---------- + chrom, pos, ref, alt : str, int, str, str + VCF-style variant fields. ``pos`` is 1-based. + interval_start, interval_end : int, optional + 0-based, half-open genomic window to score within. If either is + ``None``, a window of ``self.context_window`` bp centered on + ``pos`` is used. + output_type : str, default "RNA_SEQ" + Assay to score; one of ``alphagenome.models.dna_output.OutputType`` + names. + ontology_terms : sequence of str, optional + Restrict scoring to specific tissue/cell-type ontology CURIEs + (e.g. ``"UBERON:0002107"`` for liver). ``None`` scores all + tracks for the output type. + variant_id : str, optional + Optional identifier (e.g. rsID) attached to the variant. + + Returns + ------- + scores : np.ndarray + Shape ``(num_tracks,)``: ``log2(ALT/REF)`` gene-exon coverage + fold-change per track. + track_names : list[str] + Track identifiers aligned with ``scores``. + + Raises + ------ + RuntimeError + If the client hasn't been initialized (``load()`` not called). + """ + if self.client is None: + raise RuntimeError("AlphaGenome client not initialized. Call load() first.") + + if interval_start is None or interval_end is None: + interval = self._default_interval(chrom, pos) + else: + interval = genome.Interval(chromosome=chrom, start=interval_start, end=interval_end) + + variant = genome.Variant( + chromosome=chrom, + position=pos, + reference_bases=ref, + alternate_bases=alt, + name=variant_id, + ) + scorer = variant_scorers.GeneMaskLFCScorer(requested_output=getattr(dna_output.OutputType, output_type)) + + scored = self.client.score_variant( + interval=interval, + variant=variant, + variant_scorers=[scorer], + organism=self._organism_enum(), + ) + df = variant_scorers.tidy_scores(scored) + if ontology_terms is not None: + df = df[df["ontology_curie"].isin(set(ontology_terms))] + + scores = df["raw_score"].to_numpy(dtype=np.float32) + track_names = df["track_name"].tolist() + return scores, track_names diff --git a/src/embpy/models/dna_models.py b/src/embpy/models/dna_models.py index 09cc19a..e09061b 100644 --- a/src/embpy/models/dna_models.py +++ b/src/embpy/models/dna_models.py @@ -634,9 +634,13 @@ def embed_batch( try: from borzoi_pytorch import Borzoi + from borzoi_pytorch.pytorch_borzoi_model import TRACKS_DF as _BORZOI_TRACKS_DF + from borzoi_pytorch.pytorch_borzoi_utils import undo_squashed_scale as _undo_squashed_scale except ImportError: logging.warning("borzoi_pytorch not installed; BorzoiWrapper will be nonfunctional.") Borzoi = None # type: ignore + _BORZOI_TRACKS_DF = None # type: ignore + _undo_squashed_scale = None # type: ignore class BorzoiWrapper(BaseModelWrapper): @@ -669,6 +673,10 @@ class BorzoiWrapper(BaseModelWrapper): SEQUENCE_LENGTH = 524_288 NUM_CHANNELS = 4 ALPHABET_MAP = {"A": 0, "C": 1, "G": 2, "T": 3} + # Borzoi predicts coverage tracks at 32 bp resolution (see borzoi_pytorch's + # `Borzoi.crop`, which crops the trunk to `target_length` bins before the + # final head convs). + BIN_SIZE = 32 def __init__(self, model_path_or_name: str = "johahi/borzoi-replicate-0", **kwargs): """ @@ -792,8 +800,12 @@ def embed( self, input: str, pooling_strategy: str = "mean", + return_profile: bool = False, + is_human: bool = True, + track_indices: Sequence[int] | None = None, + undo_squashed_scale: bool = False, **kwargs: Any, - ) -> np.ndarray: + ) -> np.ndarray | tuple[np.ndarray, np.ndarray]: """ Compute Borzoi embeddings for a single DNA sequence. @@ -808,13 +820,23 @@ def embed( The DNA sequence string. pooling_strategy : str, default "mean" “mean” or “max” pooling over genomic bins. + return_profile : bool, default False + If True, also run the model's prediction head (see + :meth:`predict_profile`) and return ``(embedding, profile)`` + instead of just the embedding. This costs a second forward pass. + is_human, track_indices, undo_squashed_scale + Forwarded to :meth:`predict_profile` when ``return_profile=True``; + ignored otherwise. **kwargs : Any Currently unused but accepted for interface consistency. Returns ------- np.ndarray - A 1D NumPy array of length hidden_dim representing the pooled Borzoi embedding. + A 1D NumPy array of length hidden_dim representing the pooled + Borzoi embedding, or, if ``return_profile=True``, a tuple + ``(embedding, profile)`` where ``profile`` is the array returned + by :meth:`predict_profile`. Raises ------ @@ -839,13 +861,118 @@ def embed( trunk = embs.squeeze(0) # (hidden_dim, num_bins) if pooling_strategy == "none": - return trunk.T.cpu().numpy() # (num_bins, hidden_dim) + pooled_np = trunk.T.cpu().numpy() # (num_bins, hidden_dim) elif pooling_strategy == "mean": - pooled = trunk.mean(dim=1) # (hidden_dim,) + pooled_np = trunk.mean(dim=1).to(torch.float32).cpu().numpy() # (hidden_dim,) else: - pooled = trunk.max(dim=1).values # (hidden_dim,) + pooled_np = trunk.max(dim=1).values.to(torch.float32).cpu().numpy() + + if not return_profile: + return pooled_np + + profile = self.predict_profile( + input, + is_human=is_human, + track_indices=track_indices, + undo_squashed_scale=undo_squashed_scale, + ) + return pooled_np, profile + + @property + def profile_offset_bp(self) -> int: + """bp offset of profile bin 0 relative to the start of the model's input window. + + Borzoi crops both ends of its receptive field before the prediction + head, so the first predicted bin does not start at position 0 of the + 524,288 bp input. This is only exact when the sequence passed to + :meth:`predict_profile` is exactly ``SEQUENCE_LENGTH`` long (i.e. no + additional padding/cropping was applied by :meth:`_preprocess_sequence`). + """ + if self.model is None: + raise RuntimeError("Borzoi model not loaded. Call load() first.") + crop_length = self.model.crop.target_length + return (self.SEQUENCE_LENGTH - crop_length * self.BIN_SIZE) // 2 + + def predict_profile( + self, + input: str, + is_human: bool = True, + track_indices: Sequence[int] | None = None, + undo_squashed_scale: bool = False, + ) -> np.ndarray: + """Predict Borzoi's full coverage profile (RNA-seq/ATAC/ChIP tracks). + + Unlike :meth:`embed`, which pools the pre-head trunk embedding, this + runs the model's human/mouse head (``model.forward``) to obtain the + actual predicted per-track, per-bin coverage. + + Parameters + ---------- + input : str + Raw DNA sequence string. + is_human : bool, default True + Use the human head (7,611 tracks) or the mouse head (2,608 tracks). + track_indices : sequence of int, optional + Restrict the output to these track indices (saves memory if only + a handful of assays are of interest). Default: all tracks. + undo_squashed_scale : bool, default False + Borzoi targets are stored on a "squashed" (soft-clipped, + power-transformed) scale during training; set True to invert + that transform and recover approximate linear-scale coverage, + which is required before summing bins for variant-effect scoring. - return pooled.to(torch.float32).cpu().numpy() + Returns + ------- + np.ndarray + Array of shape ``(num_tracks, num_bins)``. With the default + settings ``num_bins == model.crop.target_length`` and each bin + spans :attr:`BIN_SIZE` (32) bp; see :attr:`profile_offset_bp` for + how bin 0 maps back to genomic coordinates. + """ + if self.model is None or self.device is None: + raise RuntimeError("Borzoi model not loaded. Call load() first.") + + one_hot = self._preprocess_sequence(input).to(self.device) # (1, 4, 524288) + + with torch.no_grad(): + model: Any = self.model + tracks = model.forward(one_hot, is_human=is_human) + + if not isinstance(tracks, torch.Tensor) or tracks.dim() != 3: + raise RuntimeError( + f"Unexpected Borzoi profile output: {type(tracks)}, " + f"shape={getattr(tracks, 'shape', None)}" + ) + + tracks = tracks.squeeze(0) # (num_tracks, num_bins) + if track_indices is not None: + idx = torch.as_tensor(list(track_indices), dtype=torch.long, device=tracks.device) + tracks = tracks.index_select(0, idx) + if undo_squashed_scale: + if _undo_squashed_scale is None: + raise ImportError("borzoi_pytorch not installed; cannot undo squashed scale.") + tracks = _undo_squashed_scale(tracks.unsqueeze(0)).squeeze(0) + + return tracks.to(torch.float32).cpu().numpy() + + @staticmethod + def get_track_metadata() -> Any: + """Return Borzoi's bundled track metadata (one row per output channel). + + Columns include ``identifier``, ``description``, ``file``, + ``strand_pair``, ``sum_stat`` and ``scale`` -- the same + ``targets.txt`` table used internally by ``borzoi_pytorch`` to decode + and unsquash predictions. Useful for mapping :meth:`predict_profile` + track indices to assay names (e.g. "RNA:liver" or "ATAC:PBMC"). + + Returns + ------- + pandas.DataFrame + A copy of the bundled track annotation table. + """ + if _BORZOI_TRACKS_DF is None: + raise ImportError("borzoi_pytorch not installed; cannot load track metadata.") + return _BORZOI_TRACKS_DF.copy() # Conservative default chosen for a 80 GB H100 / A100. # Borzoi's first conv expands a (B, 4, 524288) input into roughly diff --git a/src/embpy/models/scooby_models.py b/src/embpy/models/scooby_models.py new file mode 100644 index 0000000..17b51a1 --- /dev/null +++ b/src/embpy/models/scooby_models.py @@ -0,0 +1,478 @@ +import logging +from collections.abc import Sequence +from typing import Any, Literal + +import numpy as np +import pandas as pd +import torch +import torch.nn.functional as F + +from .base import BaseModelWrapper + + +try: + from scooby.modeling import Scooby + from scooby.utils.utils import undo_squashed_scale as _scooby_undo_squashed_scale +except ImportError: + logging.warning("scooby not installed; ScoobyWrapper will be nonfunctional.") + Scooby = None # type: ignore + _scooby_undo_squashed_scale = None # type: ignore + + +class ScoobyWrapper(BaseModelWrapper): + """ + Wrapper for the Scooby model (via the `scooby` package, gagneurlab/scooby). + + Scooby predicts single-cell-resolution scRNA-seq/scATAC-seq coverage + profiles from DNA sequence, conditioned on a precomputed single-cell + embedding (e.g. a scPoli/scVI latent vector for the cell(s) of + interest). This class handles: + + 1. Padding/center-cropping an arbitrary-length DNA string to exactly + 524,288 bp (identical convention to :class:`BorzoiWrapper`). + 2. Converting one or more precomputed cell embeddings into the + per-cell decoder conv weights via `Scooby.forward_cell_embs_only`. + 3. Running the cell-conditioned decoder to obtain per-track, + per-bin coverage predictions, optionally pseudobulk-aggregated + across cells of the same cell type (summed on the linear/unsquashed + scale, matching `scooby.utils.utils.get_pseudobulk_profile_pred`). + """ + + model_type = "dna" + available_pooling_strategies = ["mean", "max", "median", "none"] + + SEQUENCE_LENGTH = 524_288 + NUM_CHANNELS = 4 + ALPHABET_MAP = {"A": 0, "C": 1, "G": 2, "T": 3} + # Same trunk/backbone as Borzoi -> identical bin resolution. + BIN_SIZE = 32 + + # Verified from each checkpoint's safetensors header (see module + # docstring above), not from scooby's generic scripts/config_*.yaml + # (whose defaults are examples for the multiome bone-marrow dataset and + # do not match the actual released checkpoints for OneK1K/Epicardioids). + KNOWN_CHECKPOINTS: dict[str, dict[str, Any]] = { + "lauradmartens/onek1k-scooby": dict( + cell_emb_dim=10, + n_tracks=2, + use_transform_borzoi_emb=True, + clip_soft=5.0, + ), + "johahi/neurips-scooby": dict( + cell_emb_dim=14, + n_tracks=3, + use_transform_borzoi_emb=True, + clip_soft=5.0, + ), + "lauradmartens/epicardioids-scooby": dict( + cell_emb_dim=50, + n_tracks=3, + use_transform_borzoi_emb=True, + clip_soft=5.0, + ), + } + + def __init__( + self, + model_path_or_name: str = "lauradmartens/onek1k-scooby", + cell_emb_dim: int | None = None, + n_tracks: int | None = None, + embedding_dim: int = 1920, + use_transform_borzoi_emb: bool | None = None, + clip_soft: float | None = None, + **kwargs, + ): + """ + Initialize the ScoobyWrapper. + + Parameters + ---------- + model_path_or_name : str, optional + HuggingFace model identifier or local path for Scooby weights. + Defaults to "lauradmartens/onek1k-scooby" (the OneK1K/Yazar et + al. 2022 PBMC checkpoint -- the same cohort used elsewhere in + this pipeline for third-cohort eQTL replication). Also accepts + "johahi/neurips-scooby" and "lauradmartens/epicardioids-scooby", + or any other Scooby checkpoint (local or on the Hub) provided + `cell_emb_dim`/`n_tracks` are given explicitly. + cell_emb_dim, n_tracks, use_transform_borzoi_emb, clip_soft + Checkpoint architecture/postprocessing hyperparameters. Any left + as ``None`` are looked up in :attr:`KNOWN_CHECKPOINTS` by + `model_path_or_name`; if not found there, they must be supplied + explicitly (this keeps the wrapper usable for future + fine-tuned checkpoints, e.g. a Cardinal G&H/UKB-specific Scooby). + **kwargs : Any + Reserved for future use (none currently). + + Raises + ------ + ValueError + If a required hyperparameter cannot be resolved from + :attr:`KNOWN_CHECKPOINTS` and was not passed explicitly. + """ + super().__init__(model_path_or_name, **kwargs) + known = self.KNOWN_CHECKPOINTS.get(model_path_or_name, {}) + + def _resolve(name: str, value: Any, default: Any = None) -> Any: + if value is not None: + return value + if name in known: + return known[name] + if default is not None: + return default + raise ValueError( + f"'{name}' could not be inferred for checkpoint '{model_path_or_name}' " + f"(not in ScoobyWrapper.KNOWN_CHECKPOINTS). Pass it explicitly to " + "ScoobyWrapper(...)." + ) + + self.cell_emb_dim: int = _resolve("cell_emb_dim", cell_emb_dim) + self.n_tracks: int = _resolve("n_tracks", n_tracks) + self.use_transform_borzoi_emb: bool = _resolve( + "use_transform_borzoi_emb", use_transform_borzoi_emb, default=True + ) + self.clip_soft: float = _resolve("clip_soft", clip_soft, default=5.0) + self.embedding_dim = embedding_dim + + @property + def TRACK_NAMES(self) -> list[str]: + """Track identifiers aligned with the model's track axis. + + Derived from `scooby.utils.utils.get_outputs`, which selects RNA + tracks via the boolean mask ``[1, 1, 0]`` and ATAC via ``[0, 0, 1]`` + over each (RNA+, RNA-, ATAC) track triplet -- i.e. for + ``n_tracks == 3`` (multiome checkpoints) the track order is + ``["RNA:+", "RNA:-", "ATAC"]``; for ``n_tracks == 2`` (RNA-only + checkpoints, e.g. OneK1K) it is ``["RNA:+", "RNA:-"]``. + """ + if self.n_tracks == 2: + return ["RNA:+", "RNA:-"] + if self.n_tracks == 3: + return ["RNA:+", "RNA:-", "ATAC"] + return [f"track_{i}" for i in range(self.n_tracks)] + + def load(self, device: torch.device): + """ + Load the Scooby model onto the specified device. + + Uses `scooby.modeling.Scooby.from_pretrained`, which (like Borzoi) + is a HuggingFace `transformers.PreTrainedModel`: it downloads + `config.json` (Borzoi backbone hyperparameters only) plus + `model.safetensors`, and forwards `cell_emb_dim`/`embedding_dim`/ + `n_tracks`/`use_transform_borzoi_emb` to `Scooby.__init__` to build + the cell-state decoder before loading weights. + + Parameters + ---------- + device : torch.device + The target device for model inference. + + Raises + ------ + ImportError + If `scooby` is not installed. + RuntimeError + If the model fails to load for any other reason. + """ + if self.model is not None: + logging.warning(f"Scooby '{self.model_name}' already loaded.") + return + if Scooby is None: + raise ImportError("scooby not installed; cannot load ScoobyWrapper.") + + logging.info(f"Loading Scooby '{self.model_name}' …") + try: + scooby_model: Any = Scooby.from_pretrained( + self.model_name, + cell_emb_dim=self.cell_emb_dim, + embedding_dim=self.embedding_dim, + n_tracks=self.n_tracks, + return_center_bins_only=True, + disable_cache=False, + use_transform_borzoi_emb=self.use_transform_borzoi_emb, + ) + self.model = scooby_model.to(device).eval() + self.device = device + logging.info( + f"Scooby '{self.model_name}' loaded on {device} " + f"(cell_emb_dim={self.cell_emb_dim}, n_tracks={self.n_tracks})." + ) + except Exception as e: + logging.error(f"Failed to load Scooby '{self.model_name}': {e}") + self.model = None + self.device = None + raise RuntimeError(f"Could not load Scooby '{self.model_name}'.") from e + + def _preprocess_sequence(self, sequence: str) -> torch.Tensor: + """ + Internal method to preprocess a DNA sequence for Scooby. + + Identical convention to `BorzoiWrapper._preprocess_sequence`, + including the channel-first output layout: (1, 4, SEQUENCE_LENGTH). + + Note: `Scooby.forward_seq_to_emb`'s docstring claims a channel-last + `(batch_size, seq_len, 4)` input, but that is stale/inaccurate -- + its first op, `self.conv_dna(x)`, is Borzoi's `nn.Conv1d(4, 512, + 15)`, which requires channel-first `(batch, 4, seq_len)`. Confirmed + against scooby's own callers: `scripts/train_rna_only.py` does + `inputs = inputs.permute(0, 2, 1)` (dataset yields channel-last, + permuted to channel-first) before `scooby(inputs, ...)`, and + `docs/notebooks/Evaluate_Model.ipynb` does + `seqs = x[0].cuda().permute(0,2,1)` for the same reason. + + Parameters + ---------- + sequence : str + Raw DNA sequence (e.g., "ACGT..."). + + Returns + ------- + torch.Tensor + Float tensor of shape (1, 4, SEQUENCE_LENGTH). + """ + seq = sequence.upper() + idx = torch.tensor([self.ALPHABET_MAP.get(b, 0) for b in seq], dtype=torch.long).unsqueeze(0) # (1, L_in) + oh = F.one_hot(idx, num_classes=self.NUM_CHANNELS).permute(0, 2, 1).float() # (1, 4, L_in) + + L_in = oh.shape[2] + L_tar = self.SEQUENCE_LENGTH + + if L_in < L_tar: + pad_total = L_tar - L_in + pad_left = pad_total // 2 + pad_right = pad_total - pad_left + oh = F.pad(oh, (pad_left, pad_right), mode="constant", value=0.0) + logging.debug(f"Padded Scooby indices from {L_in}→{L_tar} with zeros.") + elif L_in > L_tar: + trim = (L_in - L_tar) // 2 + oh = oh[:, :, trim : trim + L_tar] + logging.warning(f"Truncated Scooby indices from {L_in}→{L_tar} (center‐crop).") + + if oh.shape[2] != L_tar: + raise ValueError(f"Preprocessing error: final length {oh.shape[2]} != {L_tar}") + + return oh # shape: (1, 4, SEQUENCE_LENGTH) + + def _prepare_cell_embeddings(self, cell_embeddings: Any) -> torch.Tensor: + """Coerce a caller-supplied cell embedding (or embeddings) into + the (1, num_cells, cell_emb_dim) tensor `forward_cell_embs_only` expects. + + Parameters + ---------- + cell_embeddings : array-like + Either a single embedding vector of shape ``(cell_emb_dim,)`` + (e.g. a scPoli/scVI latent vector for one cell, or a + precomputed centroid representing a cell type), or a matrix of + shape ``(num_cells, cell_emb_dim)`` (one row per single cell, + e.g. all cells belonging to one annotated cell type -- see + :meth:`predict_profile` with ``aggregate="pseudobulk"``). + + Returns + ------- + torch.Tensor + Float tensor of shape (1, num_cells, cell_emb_dim). + """ + arr = torch.as_tensor(np.asarray(cell_embeddings), dtype=torch.float32) + if arr.dim() == 1: + arr = arr.unsqueeze(0) # (1, cell_emb_dim) + if arr.dim() != 2 or arr.shape[-1] != self.cell_emb_dim: + raise ValueError( + f"cell_embeddings must have shape (cell_emb_dim,) or (num_cells, cell_emb_dim) " + f"with cell_emb_dim={self.cell_emb_dim}; got shape {tuple(arr.shape)}." + ) + return arr.unsqueeze(0) # (1, num_cells, cell_emb_dim) + + @property + def profile_offset_bp(self) -> int: + """bp offset of profile bin 0 relative to the start of the model's + input window (identical mechanism to `BorzoiWrapper.profile_offset_bp` + since Scooby shares Borzoi's trunk/crop).""" + if self.model is None: + raise RuntimeError("Scooby model not loaded. Call load() first.") + crop_length = self.model.crop.target_length + return (self.SEQUENCE_LENGTH - crop_length * self.BIN_SIZE) // 2 + + def predict_profile( + self, + input: str, + cell_embeddings: Any, + aggregate: Literal["pseudobulk", "none"] = "pseudobulk", + undo_squashed_scale: bool = True, + track_indices: Sequence[int] | None = None, + ) -> np.ndarray: + """Predict Scooby's cell-conditioned coverage profile. + + Parameters + ---------- + input : str + Raw DNA sequence string. + cell_embeddings : array-like + A single precomputed cell embedding of shape ``(cell_emb_dim,)`` + (one cell, or a prototype/centroid representing a cell type), + or ``(num_cells, cell_emb_dim)`` for multiple single cells + (e.g. all cells of one annotated cell type). Must live in the + same embedding space the loaded checkpoint was trained to + condition on (for the default OneK1K checkpoint: a 10-D scPoli + latent space fit on that cohort's highly-variable genes -- + projecting new cells into this space requires the trained + scPoli reference model, not an independent PCA/scVI fit; see + module notes). + aggregate : {"pseudobulk", "none"}, default "pseudobulk" + Only relevant when multiple cells are passed. ``"pseudobulk"`` + sums the per-cell *unsquashed* predicted coverage across cells + (matching `scooby.utils.utils.get_pseudobulk_profile_pred`), + returning a single ``(num_tracks, num_bins)`` array -- directly + compatible with `embpy.tl.genomics.snp_utils.profile_variant_effect_score`/ + `SNPEmbedder.predict_variant_effect`, exactly like `BorzoiWrapper.predict_profile`. + ``"none"`` returns per-cell profiles, shape + ``(num_cells, num_tracks, num_bins)``. + undo_squashed_scale : bool, default True + Invert Scooby's soft-clipped, power-transformed training scale + to recover approximate linear-scale coverage. Defaults to + ``True`` here (unlike `BorzoiWrapper.predict_profile`, which + defaults to `False`) because pseudobulk aggregation is only + valid on the linear scale -- scooby's own evaluation code + (`get_pseudobulk_profile_pred`) always unsquashes before summing. + track_indices : sequence of int, optional + Restrict the output to these track indices (see + :attr:`TRACK_NAMES`). + + Returns + ------- + np.ndarray + ``(num_tracks, num_bins)`` if a single cell embedding was given + or ``aggregate="pseudobulk"``; ``(num_cells, num_tracks, num_bins)`` + if multiple cells were given and ``aggregate="none"``. Each bin + spans :attr:`BIN_SIZE` (32) bp; see :attr:`profile_offset_bp`. + """ + if self.model is None or self.device is None: + raise RuntimeError("Scooby model not loaded. Call load() first.") + + one_hot = self._preprocess_sequence(input).to(self.device) # (1, 4, SEQUENCE_LENGTH) + cell_emb = self._prepare_cell_embeddings(cell_embeddings).to(self.device) + num_cells = cell_emb.shape[1] + + with torch.no_grad(): + model: Any = self.model + conv_weights, conv_biases = model.forward_cell_embs_only(cell_emb) + raw = model.forward_sequence_w_convs(one_hot, conv_weights, conv_biases) # (1, num_bins, num_cells * n_tracks) + + if not isinstance(raw, torch.Tensor) or raw.dim() != 3: + raise RuntimeError(f"Unexpected Scooby output: {type(raw)}, shape={getattr(raw, 'shape', None)}") + + num_bins = raw.shape[1] + per_cell = raw.squeeze(0).view(num_bins, num_cells, self.n_tracks).permute(1, 2, 0) # (num_cells, n_tracks, num_bins) + + if undo_squashed_scale: + if _scooby_undo_squashed_scale is None: + raise ImportError("scooby not installed; cannot undo squashed scale.") + per_cell = _scooby_undo_squashed_scale(per_cell, clip_soft=self.clip_soft) + + if track_indices is not None: + idx = torch.as_tensor(list(track_indices), dtype=torch.long, device=per_cell.device) + per_cell = per_cell.index_select(1, idx) + + if num_cells == 1: + return per_cell.squeeze(0).to(torch.float32).cpu().numpy() # (n_tracks, num_bins) + + if aggregate == "pseudobulk": + pooled = per_cell.sum(dim=0) # (n_tracks, num_bins) -- sum, not mean; see get_pseudobulk_profile_pred + return pooled.to(torch.float32).cpu().numpy() + if aggregate == "none": + return per_cell.to(torch.float32).cpu().numpy() # (num_cells, n_tracks, num_bins) + raise ValueError(f"Invalid aggregate='{aggregate}'. Choose 'pseudobulk' or 'none'.") + + def get_track_metadata(self) -> pd.DataFrame: + """Return a minimal track-annotation table (one row per output channel). + + Mirrors `BorzoiWrapper.get_track_metadata`'s ``identifier`` column + (used by `SNPEmbedder.predict_variant_effect`) but, since Scooby's + tracks are cell-embedding-conditioned RNA(+ATAC) strand channels + rather than fixed named bulk assays, only ``identifier`` is + populated (see :attr:`TRACK_NAMES`). + """ + return pd.DataFrame({"identifier": self.TRACK_NAMES}) + + def embed( + self, + input: str, + pooling_strategy: str = "mean", + **kwargs: Any, + ) -> np.ndarray: + """ + Compute the cell-embedding-independent trunk embedding for a DNA sequence. + + This pools `Scooby.forward_seq_to_emb`'s output (the same + fine-tuned Borzoi trunk representation the cell-state decoder is + applied to) -- i.e. it does *not* depend on `cell_embeddings`, + analogous to `BorzoiWrapper.embed`. + + Parameters + ---------- + input : str + The DNA sequence string. + pooling_strategy : str, default "mean" + "mean" or "max" pooling over genomic bins. + **kwargs : Any + Currently unused but accepted for interface consistency. + + Returns + ------- + np.ndarray + A 1D NumPy array of length `embedding_dim`. + """ + if self.model is None or self.device is None: + raise RuntimeError("Scooby model not loaded. Call load() first.") + if pooling_strategy not in self.available_pooling_strategies: + raise ValueError(f"Invalid pooling: '{pooling_strategy}'") + + one_hot = self._preprocess_sequence(input).to(self.device) # (1, 4, SEQUENCE_LENGTH) + + with torch.no_grad(): + model: Any = self.model + trunk = model.forward_seq_to_emb(one_hot) # (1, embedding_dim, num_bins) + if not isinstance(trunk, torch.Tensor) or trunk.dim() != 3: + raise RuntimeError(f"Unexpected Scooby trunk output: {type(trunk)}, shape={getattr(trunk, 'shape', None)}") + + trunk = trunk.squeeze(0) # (embedding_dim, num_bins) + if pooling_strategy == "none": + pooled_np = trunk.T.cpu().numpy() + elif pooling_strategy == "mean": + pooled_np = trunk.mean(dim=1).to(torch.float32).cpu().numpy() + elif pooling_strategy == "max": + pooled_np = trunk.max(dim=1).values.to(torch.float32).cpu().numpy() + else: # "median" + pooled_np = trunk.median(dim=1).values.to(torch.float32).cpu().numpy() + + return pooled_np + + def embed_batch( + self, + inputs: Sequence[str], + pooling_strategy: str = "mean", + **kwargs: Any, + ) -> list[np.ndarray]: + """ + Compute Scooby trunk embeddings for a batch of sequences. + + Scooby is only validated with batch size 1 (see module docstring), + so unlike `BorzoiWrapper.embed_batch` this does not attempt to + concatenate sequences into one forward pass -- it simply loops + `embed()` over `inputs`. + + Parameters + ---------- + inputs : Sequence[str] + List of raw DNA sequence strings. + pooling_strategy : str, default "mean" + Forwarded to :meth:`embed`. + **kwargs : Any + Forwarded to :meth:`embed`. + + Returns + ------- + list[np.ndarray] + One 1D embedding per input sequence. + """ + del kwargs + return [self.embed(seq, pooling_strategy=pooling_strategy) for seq in inputs] diff --git a/src/embpy/tl/genomics/__init__.py b/src/embpy/tl/genomics/__init__.py index b5c75d7..9ad18bd 100644 --- a/src/embpy/tl/genomics/__init__.py +++ b/src/embpy/tl/genomics/__init__.py @@ -3,9 +3,12 @@ SNPEmbedder, SNPEmbeddingResult, SequenceProvider, + VariantEffectResult, download_hg38_per_chrom, download_hg38_single_fasta, embed_vcf, + genomic_to_bin_indices, + profile_variant_effect_score, ) __all__ = [ @@ -13,7 +16,10 @@ "SNPEmbedder", "SNPEmbeddingResult", "SequenceProvider", + "VariantEffectResult", "download_hg38_per_chrom", "download_hg38_single_fasta", "embed_vcf", + "genomic_to_bin_indices", + "profile_variant_effect_score", ] diff --git a/src/embpy/tl/genomics/snp_utils.py b/src/embpy/tl/genomics/snp_utils.py index fc2ee4e..abdb268 100644 --- a/src/embpy/tl/genomics/snp_utils.py +++ b/src/embpy/tl/genomics/snp_utils.py @@ -1,6 +1,7 @@ from __future__ import annotations import logging +from collections.abc import Sequence from dataclasses import dataclass, field from typing import Any, Literal @@ -245,19 +246,29 @@ class SNPEmbeddingResult: alt_sequences : list[str] Alternate context sequences, one per alt allele. ref_embedding : np.ndarray - Embedding of the reference sequence. + Embedding of the reference sequence. Always populated. alt_embeddings : list[np.ndarray] - Embeddings of each alternate sequence. + Embeddings of each alternate sequence. Always populated. delta_embeddings : list[np.ndarray] - ``alt_emb - ref_emb`` for each alternate allele. + ``alt_emb - ref_emb`` for each alternate allele. Opt-in via ``compute_delta=True``. + concat_embeddings : list[np.ndarray] + ``concatenate([ref_emb, alt_emb])`` for each alternate allele -- + handy as a single feature vector for downstream ML. Opt-in via + ``compute_concat=True``. delta_norms : list[float] L2 norm of each delta embedding (scalar summary of effect size). + Empty when ``compute_delta=False``. cosine_similarities : list[float] Cosine similarity between reference and each alternate embedding. model_name : str Name of the model used. pooling_strategy : str Pooling strategy used. + compute_delta : bool + Whether ``delta_embeddings``/``delta_norms`` were (or should be) + auto-computed from ``ref_embedding``/``alt_embeddings``. + compute_concat : bool + Whether ``concat_embeddings`` were (or should be) auto-computed. """ snp: SNPContext @@ -266,16 +277,23 @@ class SNPEmbeddingResult: ref_embedding: np.ndarray alt_embeddings: list[np.ndarray] delta_embeddings: list[np.ndarray] = field(default_factory=list) + concat_embeddings: list[np.ndarray] = field(default_factory=list) delta_norms: list[float] = field(default_factory=list) cosine_similarities: list[float] = field(default_factory=list) model_name: str = "" pooling_strategy: str = "mean" + compute_delta: bool = False + compute_concat: bool = False def __post_init__(self) -> None: # Auto-compute deltas and summaries if not provided - if not self.delta_embeddings: + if self.compute_delta and not self.delta_embeddings: self.delta_embeddings = [a - self.ref_embedding for a in self.alt_embeddings] - if not self.delta_norms: + if self.compute_concat and not self.concat_embeddings: + self.concat_embeddings = [ + np.concatenate([self.ref_embedding, a]) for a in self.alt_embeddings + ] + if not self.delta_norms and self.delta_embeddings: self.delta_norms = [float(np.linalg.norm(d)) for d in self.delta_embeddings] if not self.cosine_similarities: ref_norm = float(np.linalg.norm(self.ref_embedding)) @@ -362,6 +380,157 @@ def _apply_snp(sequence: str, offset: int, ref: str, alt: str) -> str: return sequence[:offset] + alt + sequence[offset + len(ref) :] +@dataclass +class VariantEffectResult: + """Profile-based variant-effect prediction (Borzoi-style). + + Holds the predicted reference/alternate coverage profiles plus a + per-track effect score, following the gene-level statistic used for + RNA-seq/ATAC/ChIP variant-effect prediction in the Borzoi paper + (Linder et al. 2025, *Nat. Genet.*): summed predicted coverage over a set of bins + (typically a gene's exon-overlapping bins) on the linear (unsquashed) + scale, compared between alleles via a log2 fold-change with pseudocount. + + Attributes + ---------- + snp : SNPContext + The input variant descriptor. + ref_profile : np.ndarray + Reference predicted profile, shape ``(num_tracks, num_bins)``. + alt_profiles : list[np.ndarray] + Alternate predicted profiles, one per alt allele. + bin_indices : np.ndarray, optional + Bins the effect score was aggregated over (``None`` means "all bins"). + effect_scores : list[np.ndarray] + Per-track log2 fold-change effect score, one array (shape + ``(num_tracks,)``) per alt allele. + track_names : list[str], optional + Track identifiers aligned with the channel axis of the profiles, + if the model wrapper exposes ``get_track_metadata()``. + model_name : str + Name of the model used. + """ + + snp: SNPContext + ref_profile: np.ndarray + alt_profiles: list[np.ndarray] + bin_indices: np.ndarray | None = None + effect_scores: list[np.ndarray] = field(default_factory=list) + track_names: list[str] | None = None + model_name: str = "" + + def top_tracks(self, alt_index: int = 0, n: int = 10) -> list[tuple[str, float]]: + """Return the ``n`` tracks with the largest absolute effect size. + + Parameters + ---------- + alt_index + Which alt allele's effect scores to rank (default: the first). + n + Number of tracks to return. + + Returns + ------- + list[(str, float)] + ``(track_name, effect_score)`` pairs sorted by ``|effect_score|`` + descending. Track names fall back to stringified indices if + ``track_names`` is unavailable. + """ + scores = self.effect_scores[alt_index] + order = np.argsort(-np.abs(scores))[:n] + names = self.track_names if self.track_names is not None else [str(i) for i in range(len(scores))] + return [(names[i], float(scores[i])) for i in order] + + +def genomic_to_bin_indices( + intervals: list[tuple[int, int]], + window_start: int, + bin_size: int, + profile_offset_bp: int = 0, + num_bins: int | None = None, +) -> np.ndarray: + """Convert genomic intervals (e.g. exon coordinates) into profile bin indices. + + Use this to translate a gene's exon coordinates into the bin indices + needed by :func:`profile_variant_effect_score` / ``predict_variant_effect``. + + Parameters + ---------- + intervals + List of 0-based, half-open ``(start, end)`` genomic intervals, in the + same coordinate system as ``window_start`` (i.e. ``start``/``end`` + are absolute chromosome coordinates if ``window_start`` is too). + window_start + 0-based genomic coordinate of the first base of the sequence window + passed to the model. + bin_size + Model bin width in bp (e.g. ``BorzoiWrapper.BIN_SIZE`` == 32). + profile_offset_bp + bp offset of profile bin 0 relative to ``window_start`` (e.g. + ``BorzoiWrapper.profile_offset_bp``); Borzoi crops both ends of its + receptive field before predicting, so bin 0 is not at the window + start. + num_bins + If given, drop any bins outside ``[0, num_bins)``. + + Returns + ------- + np.ndarray + Sorted, de-duplicated array of bin indices overlapping ``intervals``. + """ + bins: set[int] = set() + for start, end in intervals: + rel_start = start - window_start - profile_offset_bp + rel_end = end - window_start - profile_offset_bp + b0 = int(rel_start // bin_size) + b1 = -(-int(rel_end) // bin_size) # ceil division + for b in range(max(b0, 0), b1): + if num_bins is None or 0 <= b < num_bins: + bins.add(b) + return np.array(sorted(bins), dtype=int) + + +def profile_variant_effect_score( + ref_profile: np.ndarray, + alt_profile: np.ndarray, + bin_indices: Sequence[int] | None = None, + pseudocount: float = 1.0, +) -> np.ndarray: + """Gene-level variant-effect statistic from predicted coverage profiles. + + Sums the (linear-scale) predicted coverage over ``bin_indices`` for both + alleles, then returns the log2 fold-change with a pseudocount -- the + statistic used for RNA-seq/ATAC/ChIP variant-effect scoring in the + Borzoi papers. + + Parameters + ---------- + ref_profile, alt_profile + Arrays of shape ``(num_tracks, num_bins)`` as returned by + ``BorzoiWrapper.predict_profile`` (pass ``undo_squashed_scale=True`` + when predicting so the values are on the linear scale this statistic + assumes). + bin_indices + Bins to sum over, e.g. from :func:`genomic_to_bin_indices`. ``None`` + sums over the entire profile. + pseudocount + Added to both sums before taking the log2 ratio. + + Returns + ------- + np.ndarray + Shape ``(num_tracks,)``: ``log2((alt_sum + pseudocount) / (ref_sum + pseudocount))``. + """ + if bin_indices is not None: + idx = np.asarray(bin_indices, dtype=int) + ref_sum = ref_profile[:, idx].sum(axis=1) + alt_sum = alt_profile[:, idx].sum(axis=1) + else: + ref_sum = ref_profile.sum(axis=1) + alt_sum = alt_profile.sum(axis=1) + return np.log2((alt_sum + pseudocount) / (ref_sum + pseudocount)) + + class SNPEmbedder: """Compute variant-effect embeddings for SNPs using any DNA/protein model. @@ -445,10 +614,18 @@ def embed_snp( snp: SNPContext, chromosome_sequence: str, pooling_strategy: str | None = None, + compute_delta: bool = False, + compute_concat: bool = False, **kwargs: Any, ) -> SNPEmbeddingResult: """Embed a single SNP and return full result. + By default the result always carries the reference and every + alternate-allele embedding (``ref_embedding``/``alt_embeddings``). + Delta vectors (``alt - ref``) re opt-in via + ``compute_delta=True``. Concatenated ``[ref, alt]`` + feature vectors are opt-in via ``compute_concat=True``. + Parameters ---------- snp @@ -460,6 +637,10 @@ def embed_snp( or a pre-sliced region for efficiency. pooling_strategy Overrides ``self.pooling_strategy`` for this call only. + compute_delta + Whether to compute ``delta_embeddings``/``delta_norms``. + compute_concat + Whether to compute ``concat_embeddings`` (``[ref, alt]`` per allele). **kwargs Additional arguments forwarded to the model's ``embed`` method (e.g. ``target_layer``, ``layer_name``). @@ -492,6 +673,8 @@ def embed_snp( alt_embeddings=[np.asarray(e, dtype=np.float32) for e in alt_embs], model_name=getattr(self.wrapper, "model_name", ""), pooling_strategy=pool, + compute_delta=compute_delta, + compute_concat=compute_concat, ) def embed_snps_batch( @@ -607,6 +790,81 @@ def embed_snp_from_vcf_row( **kwargs, ) + def predict_variant_effect( + self, + snp: SNPContext, + chromosome_sequence: str, + bin_indices: Sequence[int] | None = None, + pseudocount: float = 1.0, + **kwargs: Any, + ) -> VariantEffectResult: + """Profile-based variant-effect prediction (Borzoi-style). + + Requires a model wrapper that implements ``predict_profile`` (e.g. + :class:`~embpy.models.dna_models.BorzoiWrapper`). Computes the + reference and alternate predicted coverage profiles and aggregates + them into a per-track log2 fold-change effect score via + :func:`profile_variant_effect_score` -- the same statistic used for + gene-level variant-effect prediction in the Borzoi papers. + + Parameters + ---------- + snp + Variant descriptor. For exact bin alignment with + :func:`genomic_to_bin_indices`, set ``snp.context_window`` equal + to the model's fixed input length (e.g. + ``BorzoiWrapper.SEQUENCE_LENGTH``). + chromosome_sequence + Chromosome (or pre-sliced region) sequence; ``snp.position`` is + relative to its start. + bin_indices + Output bins to aggregate over (e.g. a gene's exon bins, see + :func:`genomic_to_bin_indices`). ``None`` sums over all bins. + pseudocount + Forwarded to :func:`profile_variant_effect_score`. + **kwargs + Forwarded to the wrapper's ``predict_profile`` (e.g. + ``track_indices``, ``undo_squashed_scale``). + + Returns + ------- + VariantEffectResult + """ + if not hasattr(self.wrapper, "predict_profile"): + raise TypeError( + f"{type(self.wrapper).__name__} does not implement predict_profile(); " + "profile-based variant-effect prediction requires a model such as " + "BorzoiWrapper." + ) + + ref_ctx, alt_ctxs, _ = self._build_sequences(snp, chromosome_sequence) + + ref_profile = np.asarray(self.wrapper.predict_profile(ref_ctx, **kwargs)) + alt_profiles = [np.asarray(self.wrapper.predict_profile(a, **kwargs)) for a in alt_ctxs] + + scores = [ + profile_variant_effect_score(ref_profile, alt_profile, bin_indices, pseudocount) + for alt_profile in alt_profiles + ] + + track_names: list[str] | None = None + get_track_metadata = getattr(self.wrapper, "get_track_metadata", None) + if callable(get_track_metadata): + try: + track_names = list(get_track_metadata()["identifier"]) + except Exception as exc: + logging.debug(f"Could not load track metadata: {exc}") + + return VariantEffectResult( + snp=snp, + ref_profile=ref_profile, + alt_profiles=alt_profiles, + bin_indices=np.asarray(bin_indices) if bin_indices is not None else None, + effect_scores=scores, + track_names=track_names, + model_name=getattr(self.wrapper, "model_name", ""), + ) + def embed_vcf( vcf_path: str, diff --git a/src/embpy/tl/snp_utils.py b/src/embpy/tl/snp_utils.py index f695f79..cab78cd 100644 --- a/src/embpy/tl/snp_utils.py +++ b/src/embpy/tl/snp_utils.py @@ -1,2 +1,7 @@ # Backward compatibility -- code lives in tl/genomics/snp_utils.py -from .genomics.snp_utils import * # noqa: F401,F403 +# (re-exports private helpers too, since existing tests import e.g. `_apply_snp` +# directly from this module). +from .genomics import snp_utils as _snp_utils + +globals().update({k: v for k, v in vars(_snp_utils).items() if not k.startswith("__")}) +del _snp_utils diff --git a/tests/embpy/models/test_alphagenome_models.py b/tests/embpy/models/test_alphagenome_models.py new file mode 100644 index 0000000..e5d1230 --- /dev/null +++ b/tests/embpy/models/test_alphagenome_models.py @@ -0,0 +1,297 @@ +"""Tests for AlphaGenomeWrapper (Google DeepMind cloud API client) using mocks.""" + +from __future__ import annotations + +from unittest.mock import MagicMock, patch + +import numpy as np +import pandas as pd +import pytest + +from embpy.models import alphagenome_models as ag +from embpy.models.alphagenome_models import AlphaGenomeWrapper + + +def _mock_dna_client() -> MagicMock: + """A MagicMock standing in for the `alphagenome.models.dna_client` module.""" + mock = MagicMock() + mock.ModelVersion.FOLD_0 = "FOLD_0_ENUM" + mock.Organism.HOMO_SAPIENS = "HUMAN_ENUM" + mock.Organism.MUS_MUSCULUS = "MOUSE_ENUM" + return mock + + +class TestAlphaGenomeWrapperInit: + def test_init_defaults(self): + w = AlphaGenomeWrapper() + assert w.model_name == "alphagenome" + assert w.model_type == "dna" + assert w.organism == "human" + assert w.model_version is None + assert w.context_window == AlphaGenomeWrapper.DEFAULT_CONTEXT_WINDOW + assert w.client is None + + def test_init_mouse_organism(self): + w = AlphaGenomeWrapper(organism="mouse") + assert w.organism == "mouse" + + def test_init_invalid_organism_raises(self): + with pytest.raises(ValueError, match="Unknown organism"): + AlphaGenomeWrapper(organism="fly") + + def test_init_custom_context_window(self): + w = AlphaGenomeWrapper(context_window=1024) + assert w.context_window == 1024 + + +class TestAlphaGenomeWrapperLoad: + def test_load_without_package_raises(self): + w = AlphaGenomeWrapper() + with patch.object(ag, "dna_client", None): + with pytest.raises(ImportError, match="not installed"): + w.load() + + def test_load_without_api_key_raises(self, monkeypatch): + monkeypatch.delenv("ALPHAGENOME_API_KEY", raising=False) + w = AlphaGenomeWrapper() + with patch.object(ag, "dna_client", _mock_dna_client()): + with pytest.raises(ValueError, match="ALPHAGENOME_API_KEY"): + w.load() + + def test_load_creates_client(self, monkeypatch): + monkeypatch.setenv("ALPHAGENOME_API_KEY", "test-key") + mock_client_module = _mock_dna_client() + sentinel_client = MagicMock() + mock_client_module.create.return_value = sentinel_client + + w = AlphaGenomeWrapper() + with patch.object(ag, "dna_client", mock_client_module): + w.load() + + assert w.client is sentinel_client + assert w.model is sentinel_client + mock_client_module.create.assert_called_once_with("test-key", model_version=None) + + def test_load_resolves_model_version(self, monkeypatch): + monkeypatch.setenv("ALPHAGENOME_API_KEY", "test-key") + mock_client_module = _mock_dna_client() + + w = AlphaGenomeWrapper(model_version="FOLD_0") + with patch.object(ag, "dna_client", mock_client_module): + w.load() + + mock_client_module.create.assert_called_once_with("test-key", model_version="FOLD_0_ENUM") + + def test_load_already_initialized_is_noop(self, monkeypatch): + monkeypatch.setenv("ALPHAGENOME_API_KEY", "test-key") + mock_client_module = _mock_dna_client() + + w = AlphaGenomeWrapper() + existing_client = MagicMock() + w.client = existing_client + with patch.object(ag, "dna_client", mock_client_module): + w.load() + + mock_client_module.create.assert_not_called() + assert w.client is existing_client + + +class TestAlphaGenomeWrapperEmbed: + def test_embed_without_load_raises(self): + w = AlphaGenomeWrapper() + with pytest.raises(RuntimeError, match="not initialized"): + w.embed("ACGT") + + def test_embed_batch_without_load_raises(self): + w = AlphaGenomeWrapper() + with pytest.raises(RuntimeError, match="not initialized"): + w.embed_batch(["ACGT"]) + + def test_embed_invalid_pooling_raises(self): + w = AlphaGenomeWrapper() + w.client = MagicMock() + with pytest.raises(ValueError, match="Invalid pooling"): + w.embed("ACGT", pooling_strategy="invalid") + + def _mock_prediction(self, values: np.ndarray) -> MagicMock: + prediction = MagicMock() + prediction.rna_seq.values = values + return prediction + + def test_embed_mean_pooling(self): + w = AlphaGenomeWrapper() + w.client = MagicMock() + values = np.array([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]], dtype=np.float32) + w.client.predict_sequence.return_value = self._mock_prediction(values) + + with patch.object(ag, "dna_client", _mock_dna_client()), patch.object(ag, "dna_output", MagicMock()): + result = w.embed("ACGT", pooling_strategy="mean") + + np.testing.assert_allclose(result, values.mean(axis=0)) + + def test_embed_max_pooling(self): + w = AlphaGenomeWrapper() + w.client = MagicMock() + values = np.array([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]], dtype=np.float32) + w.client.predict_sequence.return_value = self._mock_prediction(values) + + with patch.object(ag, "dna_client", _mock_dna_client()), patch.object(ag, "dna_output", MagicMock()): + result = w.embed("ACGT", pooling_strategy="max") + + np.testing.assert_allclose(result, values.max(axis=0)) + + def test_embed_none_pooling_returns_raw_array(self): + w = AlphaGenomeWrapper() + w.client = MagicMock() + values = np.array([[1.0, 2.0], [3.0, 4.0]], dtype=np.float32) + w.client.predict_sequence.return_value = self._mock_prediction(values) + + with patch.object(ag, "dna_client", _mock_dna_client()), patch.object(ag, "dna_output", MagicMock()): + result = w.embed("ACGT", pooling_strategy="none") + + np.testing.assert_allclose(result, values) + + def test_embed_passes_organism_enum(self): + w = AlphaGenomeWrapper(organism="mouse") + w.client = MagicMock() + values = np.zeros((2, 2), dtype=np.float32) + w.client.predict_sequence.return_value = self._mock_prediction(values) + mock_dna_client = _mock_dna_client() + + with patch.object(ag, "dna_client", mock_dna_client), patch.object(ag, "dna_output", MagicMock()): + w.embed("ACGT") + + assert w.client.predict_sequence.call_args.kwargs["organism"] == "MOUSE_ENUM" + + def test_embed_batch_calls_embed_per_sequence(self): + w = AlphaGenomeWrapper() + w.client = MagicMock() + values = np.zeros((2, 2), dtype=np.float32) + w.client.predict_sequence.return_value = self._mock_prediction(values) + + with patch.object(ag, "dna_client", _mock_dna_client()), patch.object(ag, "dna_output", MagicMock()): + results = w.embed_batch(["ACGT", "TTTT", "GGGG"]) + + assert len(results) == 3 + assert w.client.predict_sequence.call_count == 3 + + def test_embed_batch_empty_returns_empty(self): + w = AlphaGenomeWrapper() + w.client = MagicMock() + assert w.embed_batch([]) == [] + + +class TestAlphaGenomeWrapperTrackMetadata: + def test_get_track_metadata_without_load_raises(self): + w = AlphaGenomeWrapper() + with pytest.raises(RuntimeError, match="not initialized"): + w.get_track_metadata() + + def test_get_track_metadata_specific_output_type(self): + w = AlphaGenomeWrapper() + w.client = MagicMock() + rna_df = pd.DataFrame({"identifier": ["a", "b"]}) + metadata = MagicMock() + metadata.rna_seq = rna_df + w.client.output_metadata.return_value = metadata + + with patch.object(ag, "dna_client", _mock_dna_client()): + result = w.get_track_metadata(output_type="RNA_SEQ") + + assert result is rna_df + + def test_get_track_metadata_concatenates_all(self): + import types + + w = AlphaGenomeWrapper() + w.client = MagicMock() + df1 = pd.DataFrame({"identifier": ["a"]}) + df2 = pd.DataFrame({"identifier": ["b"]}) + metadata = types.SimpleNamespace(rna_seq=df1, atac=df2, not_a_frame="ignored") + w.client.output_metadata.return_value = metadata + + with patch.object(ag, "dna_client", _mock_dna_client()): + result = w.get_track_metadata() + + assert len(result) == 2 + assert set(result["identifier"]) == {"a", "b"} + + +class TestAlphaGenomeWrapperVariantEffect: + def _mock_variant_scorers(self, df: pd.DataFrame) -> MagicMock: + mock = MagicMock() + mock.tidy_scores.return_value = df + return mock + + def test_predict_variant_effect_without_load_raises(self): + w = AlphaGenomeWrapper() + with pytest.raises(RuntimeError, match="not initialized"): + w.predict_variant_effect("chr1", 100, "A", "T") + + def test_predict_variant_effect_default_interval(self): + w = AlphaGenomeWrapper(context_window=100) + w.client = MagicMock() + w.client.score_variant.return_value = MagicMock() + df = pd.DataFrame( + { + "raw_score": [0.5, -0.3], + "track_name": ["trackA", "trackB"], + "ontology_curie": ["UBERON:1", "UBERON:2"], + } + ) + mock_genome = MagicMock() + + with ( + patch.object(ag, "dna_client", _mock_dna_client()), + patch.object(ag, "dna_output", MagicMock()), + patch.object(ag, "genome", mock_genome), + patch.object(ag, "variant_scorers", self._mock_variant_scorers(df)), + ): + scores, track_names = w.predict_variant_effect("chr1", 100, "A", "T") + + np.testing.assert_allclose(scores, [0.5, -0.3]) + assert track_names == ["trackA", "trackB"] + # default interval centered on pos=100 with context_window=100 -> half=50 + mock_genome.Interval.assert_called_once_with(chromosome="chr1", start=50, end=150) + + def test_predict_variant_effect_explicit_interval(self): + w = AlphaGenomeWrapper() + w.client = MagicMock() + w.client.score_variant.return_value = MagicMock() + df = pd.DataFrame({"raw_score": [1.0], "track_name": ["t"]}) + mock_genome = MagicMock() + + with ( + patch.object(ag, "dna_client", _mock_dna_client()), + patch.object(ag, "dna_output", MagicMock()), + patch.object(ag, "genome", mock_genome), + patch.object(ag, "variant_scorers", self._mock_variant_scorers(df)), + ): + w.predict_variant_effect("chr1", 100, "A", "T", interval_start=10, interval_end=20) + + mock_genome.Interval.assert_called_once_with(chromosome="chr1", start=10, end=20) + + def test_predict_variant_effect_ontology_filter(self): + w = AlphaGenomeWrapper() + w.client = MagicMock() + w.client.score_variant.return_value = MagicMock() + df = pd.DataFrame( + { + "raw_score": [0.5, -0.3], + "track_name": ["trackA", "trackB"], + "ontology_curie": ["UBERON:1", "UBERON:2"], + } + ) + + with ( + patch.object(ag, "dna_client", _mock_dna_client()), + patch.object(ag, "dna_output", MagicMock()), + patch.object(ag, "genome", MagicMock()), + patch.object(ag, "variant_scorers", self._mock_variant_scorers(df)), + ): + scores, track_names = w.predict_variant_effect( + "chr1", 100, "A", "T", ontology_terms=["UBERON:1"] + ) + + assert track_names == ["trackA"] + np.testing.assert_allclose(scores, [0.5]) diff --git a/tests/embpy/models/test_dna_models.py b/tests/embpy/models/test_dna_models.py index daf5842..52cb389 100644 --- a/tests/embpy/models/test_dna_models.py +++ b/tests/embpy/models/test_dna_models.py @@ -196,6 +196,90 @@ def test_embed_with_mocked_model(self): assert result.shape == (hidden_dim,) assert not np.isnan(result).any() + def test_predict_profile_without_load_raises(self): + w = BorzoiWrapper() + with pytest.raises(RuntimeError, match="not loaded"): + w.predict_profile("ACGT") + + def test_predict_profile_returns_tracks_by_bins(self): + w = BorzoiWrapper() + w.device = torch.device("cpu") + + num_tracks, num_bins = 7611, 50 + mock_model = MagicMock() + mock_model.forward.return_value = torch.rand(1, num_tracks, num_bins) + w.model = mock_model + + profile = w.predict_profile("ACGT") + assert isinstance(profile, np.ndarray) + assert profile.shape == (num_tracks, num_bins) + mock_model.forward.assert_called_once() + assert mock_model.forward.call_args.kwargs.get("is_human") is True + + def test_predict_profile_track_subset(self): + w = BorzoiWrapper() + w.device = torch.device("cpu") + + mock_model = MagicMock() + mock_model.forward.return_value = torch.rand(1, 10, 5) + w.model = mock_model + + profile = w.predict_profile("ACGT", track_indices=[0, 2, 4]) + assert profile.shape == (3, 5) + + def test_predict_profile_undo_squashed_scale_without_package_raises(self): + w = BorzoiWrapper() + w.device = torch.device("cpu") + mock_model = MagicMock() + mock_model.forward.return_value = torch.rand(1, 4, 5) + w.model = mock_model + + with patch("embpy.models.dna_models._undo_squashed_scale", None): + with pytest.raises(ImportError): + w.predict_profile("ACGT", undo_squashed_scale=True) + + def test_embed_return_profile_runs_second_forward(self): + w = BorzoiWrapper() + w.device = torch.device("cpu") + + hidden_dim, num_bins = 32, 10 + num_tracks = 7611 + mock_model = MagicMock() + mock_model.get_embs_after_crop.return_value = torch.randn(1, hidden_dim, num_bins) + mock_model.forward.return_value = torch.rand(1, num_tracks, num_bins) + w.model = mock_model + + embedding, profile = w.embed("ACGT", return_profile=True) + assert embedding.shape == (hidden_dim,) + assert profile.shape == (num_tracks, num_bins) + + def test_embed_default_does_not_call_forward(self): + w = BorzoiWrapper() + w.device = torch.device("cpu") + mock_model = MagicMock() + mock_model.get_embs_after_crop.return_value = torch.randn(1, 32, 10) + w.model = mock_model + + w.embed("ACGT") + mock_model.forward.assert_not_called() + + def test_profile_offset_bp_computed_from_crop(self): + w = BorzoiWrapper() + mock_model = MagicMock() + mock_model.crop.target_length = 16352 # 16384 - 32, as in real Borzoi + w.model = mock_model + assert w.profile_offset_bp == (524_288 - 16352 * 32) // 2 + + def test_profile_offset_bp_without_load_raises(self): + w = BorzoiWrapper() + with pytest.raises(RuntimeError, match="not loaded"): + _ = w.profile_offset_bp + + def test_get_track_metadata_returns_dataframe(self): + df = BorzoiWrapper.get_track_metadata() + assert "identifier" in df.columns + assert len(df) > 0 + class TestEvo2Wrapper: """Tests for the Evo2Wrapper (mocked, since evo2 may not be installed).""" diff --git a/tests/embpy/models/test_scooby_models.py b/tests/embpy/models/test_scooby_models.py new file mode 100644 index 0000000..285aafe --- /dev/null +++ b/tests/embpy/models/test_scooby_models.py @@ -0,0 +1,373 @@ +"""Tests for ScoobyWrapper (gagneurlab/scooby single-cell DNA model) using mocks.""" + +from __future__ import annotations + +from unittest.mock import MagicMock, patch + +import numpy as np +import pytest +import torch + +from embpy.models import scooby_models as sm +from embpy.models.scooby_models import ScoobyWrapper + + +class TestScoobyWrapperInit: + def test_init_defaults_resolve_onek1k_checkpoint(self): + w = ScoobyWrapper() + assert w.model_name == "lauradmartens/onek1k-scooby" + assert w.model_type == "dna" + assert w.cell_emb_dim == 10 + assert w.n_tracks == 2 + assert w.use_transform_borzoi_emb is True + assert w.clip_soft == 5.0 + assert w.embedding_dim == 1920 + + def test_init_neurips_checkpoint(self): + w = ScoobyWrapper(model_path_or_name="johahi/neurips-scooby") + assert w.cell_emb_dim == 14 + assert w.n_tracks == 3 + + def test_init_epicardioids_checkpoint(self): + w = ScoobyWrapper(model_path_or_name="lauradmartens/epicardioids-scooby") + assert w.cell_emb_dim == 50 + assert w.n_tracks == 3 + + def test_init_unknown_checkpoint_without_hparams_raises(self): + with pytest.raises(ValueError, match="could not be inferred"): + ScoobyWrapper(model_path_or_name="someorg/custom-scooby") + + def test_init_unknown_checkpoint_with_explicit_hparams(self): + w = ScoobyWrapper(model_path_or_name="someorg/custom-scooby", cell_emb_dim=20, n_tracks=3) + assert w.cell_emb_dim == 20 + assert w.n_tracks == 3 + assert w.use_transform_borzoi_emb is True # default fallback + assert w.clip_soft == 5.0 # default fallback + + def test_init_explicit_hparams_override_known_checkpoint(self): + w = ScoobyWrapper(clip_soft=3.0) + assert w.clip_soft == 3.0 + assert w.cell_emb_dim == 10 # still resolved from KNOWN_CHECKPOINTS + + def test_track_names_two_tracks(self): + w = ScoobyWrapper() # onek1k -> n_tracks=2 + assert w.TRACK_NAMES == ["RNA:+", "RNA:-"] + + def test_track_names_three_tracks(self): + w = ScoobyWrapper(model_path_or_name="johahi/neurips-scooby") + assert w.TRACK_NAMES == ["RNA:+", "RNA:-", "ATAC"] + + def test_track_names_generic_fallback(self): + w = ScoobyWrapper(model_path_or_name="someorg/custom-scooby", cell_emb_dim=5, n_tracks=4) + assert w.TRACK_NAMES == ["track_0", "track_1", "track_2", "track_3"] + + def test_get_track_metadata(self): + w = ScoobyWrapper() + df = w.get_track_metadata() + assert list(df["identifier"]) == ["RNA:+", "RNA:-"] + + +class TestScoobyWrapperLoad: + def test_load_without_package_raises(self): + w = ScoobyWrapper() + with patch.object(sm, "Scooby", None): + with pytest.raises(ImportError, match="not installed"): + w.load(torch.device("cpu")) + + def test_load_already_loaded_is_noop(self): + w = ScoobyWrapper() + w.model = MagicMock() + mock_scooby_cls = MagicMock() + with patch.object(sm, "Scooby", mock_scooby_cls): + w.load(torch.device("cpu")) + mock_scooby_cls.from_pretrained.assert_not_called() + + def test_load_success(self): + w = ScoobyWrapper() + device = torch.device("cpu") + mock_scooby_cls = MagicMock() + mock_instance = MagicMock() + mock_instance.to.return_value = mock_instance + mock_instance.eval.return_value = mock_instance + mock_scooby_cls.from_pretrained.return_value = mock_instance + + with patch.object(sm, "Scooby", mock_scooby_cls): + w.load(device) + + assert w.model is mock_instance + assert w.device is device + mock_scooby_cls.from_pretrained.assert_called_once_with( + "lauradmartens/onek1k-scooby", + cell_emb_dim=10, + embedding_dim=1920, + n_tracks=2, + return_center_bins_only=True, + disable_cache=False, + use_transform_borzoi_emb=True, + ) + + def test_load_failure_wraps_in_runtime_error(self): + w = ScoobyWrapper() + mock_scooby_cls = MagicMock() + mock_scooby_cls.from_pretrained.side_effect = OSError("network error") + + with patch.object(sm, "Scooby", mock_scooby_cls): + with pytest.raises(RuntimeError, match="Could not load Scooby"): + w.load(torch.device("cpu")) + + assert w.model is None + assert w.device is None + + +class TestScoobyWrapperPreprocess: + def test_preprocess_pads_short_sequence(self): + w = ScoobyWrapper() + result = w._preprocess_sequence("ACGT") + assert result.shape == (1, 4, w.SEQUENCE_LENGTH) + + def test_preprocess_truncates_long_sequence(self): + w = ScoobyWrapper() + long_seq = "A" * (w.SEQUENCE_LENGTH + 100) + result = w._preprocess_sequence(long_seq) + assert result.shape == (1, 4, w.SEQUENCE_LENGTH) + + def test_preprocess_exact_length(self): + w = ScoobyWrapper() + seq = "A" * w.SEQUENCE_LENGTH + result = w._preprocess_sequence(seq) + assert result.shape == (1, 4, w.SEQUENCE_LENGTH) + + +class TestScoobyWrapperCellEmbeddings: + def test_prepare_single_embedding_is_unsqueezed(self): + w = ScoobyWrapper() # cell_emb_dim=10 + emb = np.random.rand(10) + result = w._prepare_cell_embeddings(emb) + assert result.shape == (1, 1, 10) + + def test_prepare_multi_cell_embeddings(self): + w = ScoobyWrapper() + emb = np.random.rand(5, 10) + result = w._prepare_cell_embeddings(emb) + assert result.shape == (1, 5, 10) + + def test_prepare_wrong_dim_raises(self): + w = ScoobyWrapper() + emb = np.random.rand(7) # wrong last-dim size (expects 10) + with pytest.raises(ValueError, match="cell_embeddings must have shape"): + w._prepare_cell_embeddings(emb) + + +class TestScoobyWrapperProfileOffset: + def test_profile_offset_bp_without_load_raises(self): + w = ScoobyWrapper() + with pytest.raises(RuntimeError, match="not loaded"): + _ = w.profile_offset_bp + + def test_profile_offset_bp_computed_from_crop(self): + w = ScoobyWrapper() + mock_model = MagicMock() + mock_model.crop.target_length = 16352 # matches Borzoi's crop, same trunk + w.model = mock_model + assert w.profile_offset_bp == (524_288 - 16352 * 32) // 2 + + +class TestScoobyWrapperPredictProfile: + def _mock_model(self, raw_tensor: torch.Tensor) -> MagicMock: + model = MagicMock() + model.forward_cell_embs_only.return_value = (MagicMock(), MagicMock()) + model.forward_sequence_w_convs.return_value = raw_tensor + return model + + def test_without_load_raises(self): + w = ScoobyWrapper() + with pytest.raises(RuntimeError, match="not loaded"): + w.predict_profile("ACGT", cell_embeddings=np.random.rand(10)) + + def test_single_cell_returns_tracks_by_bins(self): + w = ScoobyWrapper() # n_tracks=2 + w.device = torch.device("cpu") + num_bins = 5 + raw = torch.randn(1, num_bins, 1 * w.n_tracks) # (1, num_bins, num_cells*n_tracks) + w.model = self._mock_model(raw) + + with patch.object(sm, "_scooby_undo_squashed_scale", lambda x, clip_soft: x): + profile = w.predict_profile("ACGT", cell_embeddings=np.random.rand(10)) + + assert isinstance(profile, np.ndarray) + assert profile.shape == (w.n_tracks, num_bins) + + def test_pseudobulk_aggregation_sums_across_cells(self): + w = ScoobyWrapper() + w.device = torch.device("cpu") + num_bins, num_cells = 5, 3 + raw = torch.randn(1, num_bins, num_cells * w.n_tracks) + w.model = self._mock_model(raw) + + with patch.object(sm, "_scooby_undo_squashed_scale", lambda x, clip_soft: x): + profile = w.predict_profile( + "ACGT", cell_embeddings=np.random.rand(num_cells, 10), aggregate="pseudobulk" + ) + + assert profile.shape == (w.n_tracks, num_bins) + + def test_aggregate_none_returns_per_cell(self): + w = ScoobyWrapper() + w.device = torch.device("cpu") + num_bins, num_cells = 5, 3 + raw = torch.randn(1, num_bins, num_cells * w.n_tracks) + w.model = self._mock_model(raw) + + with patch.object(sm, "_scooby_undo_squashed_scale", lambda x, clip_soft: x): + profile = w.predict_profile( + "ACGT", cell_embeddings=np.random.rand(num_cells, 10), aggregate="none" + ) + + assert profile.shape == (num_cells, w.n_tracks, num_bins) + + def test_invalid_aggregate_raises(self): + w = ScoobyWrapper() + w.device = torch.device("cpu") + num_bins, num_cells = 5, 2 + raw = torch.randn(1, num_bins, num_cells * w.n_tracks) + w.model = self._mock_model(raw) + + with patch.object(sm, "_scooby_undo_squashed_scale", lambda x, clip_soft: x): + with pytest.raises(ValueError, match="Invalid aggregate"): + w.predict_profile( + "ACGT", cell_embeddings=np.random.rand(num_cells, 10), aggregate="bogus" + ) + + def test_track_indices_subsets_output(self): + w = ScoobyWrapper() + w.device = torch.device("cpu") + num_bins = 5 + raw = torch.randn(1, num_bins, w.n_tracks) + w.model = self._mock_model(raw) + + with patch.object(sm, "_scooby_undo_squashed_scale", lambda x, clip_soft: x): + profile = w.predict_profile( + "ACGT", cell_embeddings=np.random.rand(10), track_indices=[0] + ) + + assert profile.shape == (1, num_bins) + + def test_undo_squashed_scale_false_skips_transform(self): + w = ScoobyWrapper() + w.device = torch.device("cpu") + num_bins = 5 + raw = torch.randn(1, num_bins, w.n_tracks) + w.model = self._mock_model(raw) + + with patch.object(sm, "_scooby_undo_squashed_scale") as mock_undo: + w.predict_profile("ACGT", cell_embeddings=np.random.rand(10), undo_squashed_scale=False) + + mock_undo.assert_not_called() + + def test_undo_squashed_scale_without_package_raises(self): + w = ScoobyWrapper() + w.device = torch.device("cpu") + num_bins = 5 + raw = torch.randn(1, num_bins, w.n_tracks) + w.model = self._mock_model(raw) + + with patch.object(sm, "_scooby_undo_squashed_scale", None): + with pytest.raises(ImportError): + w.predict_profile("ACGT", cell_embeddings=np.random.rand(10), undo_squashed_scale=True) + + def test_unexpected_output_shape_raises(self): + w = ScoobyWrapper() + w.device = torch.device("cpu") + raw = torch.randn(1, 5) # wrong: only 2 dims, expected 3 + w.model = self._mock_model(raw) + + with pytest.raises(RuntimeError, match="Unexpected Scooby output"): + w.predict_profile("ACGT", cell_embeddings=np.random.rand(10)) + + +class TestScoobyWrapperEmbed: + def test_embed_without_load_raises(self): + w = ScoobyWrapper() + with pytest.raises(RuntimeError, match="not loaded"): + w.embed("ACGT") + + def test_embed_batch_without_load_raises(self): + w = ScoobyWrapper() + with pytest.raises(RuntimeError, match="not loaded"): + w.embed_batch(["ACGT"]) + + def test_embed_invalid_pooling_raises(self): + w = ScoobyWrapper() + w.model = MagicMock() + w.device = torch.device("cpu") + with pytest.raises(ValueError, match="Invalid pooling"): + w.embed("ACGT", pooling_strategy="invalid") + + def _mock_trunk_model(self, trunk_tensor: torch.Tensor) -> MagicMock: + model = MagicMock() + model.forward_seq_to_emb.return_value = trunk_tensor + return model + + def test_embed_mean_pooling(self): + w = ScoobyWrapper() + w.device = torch.device("cpu") + embedding_dim, num_bins = 1920, 10 + trunk = torch.randn(1, embedding_dim, num_bins) + w.model = self._mock_trunk_model(trunk) + + result = w.embed("ACGT", pooling_strategy="mean") + assert isinstance(result, np.ndarray) + assert result.shape == (embedding_dim,) + assert not np.isnan(result).any() + + def test_embed_max_pooling(self): + w = ScoobyWrapper() + w.device = torch.device("cpu") + trunk = torch.randn(1, 16, 10) + w.model = self._mock_trunk_model(trunk) + + result = w.embed("ACGT", pooling_strategy="max") + assert result.shape == (16,) + + def test_embed_median_pooling(self): + w = ScoobyWrapper() + w.device = torch.device("cpu") + trunk = torch.randn(1, 16, 10) + w.model = self._mock_trunk_model(trunk) + + result = w.embed("ACGT", pooling_strategy="median") + assert result.shape == (16,) + + def test_embed_none_pooling_returns_per_bin(self): + w = ScoobyWrapper() + w.device = torch.device("cpu") + embedding_dim, num_bins = 16, 10 + trunk = torch.randn(1, embedding_dim, num_bins) + w.model = self._mock_trunk_model(trunk) + + result = w.embed("ACGT", pooling_strategy="none") + assert result.shape == (num_bins, embedding_dim) + + def test_embed_unexpected_trunk_shape_raises(self): + w = ScoobyWrapper() + w.device = torch.device("cpu") + w.model = self._mock_trunk_model(torch.randn(1, 16)) # wrong: 2 dims + + with pytest.raises(RuntimeError, match="Unexpected Scooby trunk output"): + w.embed("ACGT") + + def test_embed_batch_calls_embed_per_sequence(self): + w = ScoobyWrapper() + w.device = torch.device("cpu") + trunk = torch.randn(1, 1920, 10) + w.model = self._mock_trunk_model(trunk) + + results = w.embed_batch(["ACGT", "TTTT"]) + assert len(results) == 2 + assert w.model.forward_seq_to_emb.call_count == 2 + + def test_embed_batch_empty_returns_empty(self): + w = ScoobyWrapper() + w.model = MagicMock() + w.device = torch.device("cpu") + assert w.embed_batch([]) == [] diff --git a/tests/embpy/test_registry_split.py b/tests/embpy/test_registry_split.py index 75f9d8d..1a8e74d 100644 --- a/tests/embpy/test_registry_split.py +++ b/tests/embpy/test_registry_split.py @@ -131,6 +131,10 @@ "CaduceusWrapper", "kuleshov-group/caduceus-ps_seqlen-131k_d_model-256_n_layer-16", ), + ("alphagenome", "AlphaGenomeWrapper", "alphagenome"), + ("scooby_onek1k", "ScoobyWrapper", "lauradmartens/onek1k-scooby"), + ("scooby_neurips", "ScoobyWrapper", "johahi/neurips-scooby"), + ("scooby_epicardioids", "ScoobyWrapper", "lauradmartens/epicardioids-scooby"), # --- Protein --- ("esm1b", "ESM2Wrapper", "facebook/esm-1b"), ("esm1v_1", "ESM2Wrapper", "facebook/esm1v_t33_650M_UR90S_1"), diff --git a/tests/embpy/test_snp_utils.py b/tests/embpy/test_snp_utils.py index 4341668..71c7ff3 100644 --- a/tests/embpy/test_snp_utils.py +++ b/tests/embpy/test_snp_utils.py @@ -12,6 +12,9 @@ SNPEmbeddingResult, SNPEmbedder, SequenceProvider, + VariantEffectResult, + genomic_to_bin_indices, + profile_variant_effect_score, _apply_snp, _extract_context, ) @@ -370,6 +373,40 @@ def test_output_embeddings_are_float32(self): assert result.ref_embedding.dtype == np.float32 assert result.alt_embeddings[0].dtype == np.float32 + def test_ref_and_alt_always_present(self): + """ref/alt embeddings are always returned regardless of compute_delta/concat.""" + result = SNPEmbedder(_make_wrapper()).embed_snp( + _make_snp(), chromosome_sequence=CHR_SEQ, compute_delta=False, compute_concat=False + ) + assert result.ref_embedding is not None + assert len(result.alt_embeddings) == 1 + + def test_compute_delta_false_skips_delta(self): + result = SNPEmbedder(_make_wrapper()).embed_snp( + _make_snp(), chromosome_sequence=CHR_SEQ, compute_delta=False + ) + assert result.delta_embeddings == [] + assert result.delta_norms == [] + + def test_compute_delta_true_is_default(self): + result = SNPEmbedder(_make_wrapper()).embed_snp(_make_snp(), chromosome_sequence=CHR_SEQ) + assert len(result.delta_embeddings) == 1 + assert len(result.delta_norms) == 1 + + def test_compute_concat_default_false(self): + result = SNPEmbedder(_make_wrapper()).embed_snp(_make_snp(), chromosome_sequence=CHR_SEQ) + assert result.concat_embeddings == [] + + def test_compute_concat_true_builds_ref_alt_concat(self): + hidden_dim = 64 + result = SNPEmbedder(_make_wrapper(hidden_dim)).embed_snp( + _make_snp(), chromosome_sequence=CHR_SEQ, compute_concat=True + ) + assert len(result.concat_embeddings) == 1 + assert result.concat_embeddings[0].shape == (2 * hidden_dim,) + np.testing.assert_allclose(result.concat_embeddings[0][:hidden_dim], result.ref_embedding) + np.testing.assert_allclose(result.concat_embeddings[0][hidden_dim:], result.alt_embeddings[0]) + class TestEmbedSNPFromVCFRow: @@ -641,4 +678,136 @@ def test_rest_fallback_attempted_when_no_files(self): provider.get_chromosome("chr17") except Exception: pass - m.assert_called() \ No newline at end of file + m.assert_called() + + +class TestGenomicToBinIndices: + + def test_single_interval_within_one_bin(self): + bins = genomic_to_bin_indices([(100, 110)], window_start=0, bin_size=32) + assert list(bins) == [3] # 100//32 = 3, ceil(110/32) = 4 -> bin 3 only + + def test_interval_spanning_multiple_bins(self): + bins = genomic_to_bin_indices([(0, 70)], window_start=0, bin_size=32) + assert list(bins) == [0, 1, 2] + + def test_window_start_offset(self): + # window starts at genomic position 1000; interval at 1032-1064 -> bin 1 + bins = genomic_to_bin_indices([(1032, 1064)], window_start=1000, bin_size=32) + assert list(bins) == [1] + + def test_profile_offset_shifts_bins(self): + bins = genomic_to_bin_indices( + [(32, 64)], window_start=0, bin_size=32, profile_offset_bp=32 + ) + assert list(bins) == [0] + + def test_multiple_intervals_deduplicated_and_sorted(self): + bins = genomic_to_bin_indices( + [(64, 96), (0, 32), (0, 32)], window_start=0, bin_size=32 + ) + assert list(bins) == [0, 2] + + def test_num_bins_clips_out_of_range(self): + bins = genomic_to_bin_indices( + [(0, 320)], window_start=0, bin_size=32, num_bins=5 + ) + assert all(b < 5 for b in bins) + + def test_negative_relative_start_clamped_to_zero(self): + bins = genomic_to_bin_indices([(-100, 32)], window_start=0, bin_size=32) + assert bins[0] == 0 + + +class TestProfileVariantEffectScore: + + def test_no_change_gives_zero_score(self): + ref = np.ones((3, 10), dtype=np.float32) * 5 + alt = ref.copy() + scores = profile_variant_effect_score(ref, alt) + np.testing.assert_allclose(scores, 0.0, atol=1e-6) + + def test_doubling_coverage_gives_score_near_one(self): + ref = np.ones((2, 1), dtype=np.float32) * 100 + alt = ref * 2 + scores = profile_variant_effect_score(ref, alt, pseudocount=1.0) + # log2((200+1)/(100+1)) ~ log2(2) for large counts + assert np.allclose(scores, np.log2(201 / 101), atol=1e-5) + + def test_bin_indices_restrict_aggregation(self): + ref = np.array([[1.0, 1.0, 100.0]]) + alt = np.array([[1.0, 1.0, 200.0]]) + score_all = profile_variant_effect_score(ref, alt) + score_subset = profile_variant_effect_score(ref, alt, bin_indices=[0, 1]) + assert score_subset[0] != score_all[0] + assert pytest.approx(score_subset[0], abs=1e-6) == 0.0 + + def test_output_shape_matches_num_tracks(self): + ref = np.random.rand(5, 8).astype(np.float32) + alt = np.random.rand(5, 8).astype(np.float32) + scores = profile_variant_effect_score(ref, alt) + assert scores.shape == (5,) + + +class TestPredictVariantEffect: + + def _profile_wrapper(self, num_tracks: int = 4, num_bins: int = 6) -> MagicMock: + wrapper = MagicMock() + wrapper.model_name = "mock_profile_model" + rng = np.random.default_rng(0) + wrapper.predict_profile.side_effect = lambda *a, **kw: rng.random( + (num_tracks, num_bins) + ).astype(np.float32) + del wrapper.get_track_metadata # simulate models without track metadata + return wrapper + + def test_raises_without_predict_profile(self): + wrapper = MagicMock(spec=["embed"]) + embedder = SNPEmbedder(wrapper) + with pytest.raises(TypeError, match="predict_profile"): + embedder.predict_variant_effect(_make_snp(), chromosome_sequence=CHR_SEQ) + + def test_returns_variant_effect_result(self): + wrapper = self._profile_wrapper() + embedder = SNPEmbedder(wrapper) + result = embedder.predict_variant_effect(_make_snp(), chromosome_sequence=CHR_SEQ) + assert isinstance(result, VariantEffectResult) + assert result.ref_profile.shape == (4, 6) + assert len(result.alt_profiles) == 1 + assert len(result.effect_scores) == 1 + assert result.effect_scores[0].shape == (4,) + + def test_multi_allelic_returns_one_score_per_alt(self): + wrapper = self._profile_wrapper() + embedder = SNPEmbedder(wrapper) + snp = _make_snp(alts=["T", "G"]) + result = embedder.predict_variant_effect(snp, chromosome_sequence=CHR_SEQ) + assert len(result.alt_profiles) == 2 + assert len(result.effect_scores) == 2 + + def test_track_names_from_metadata_when_available(self): + import pandas as pd + + wrapper = MagicMock() + wrapper.model_name = "mock_profile_model" + rng = np.random.default_rng(0) + wrapper.predict_profile.side_effect = lambda *a, **kw: rng.random((3, 4)).astype(np.float32) + wrapper.get_track_metadata.return_value = pd.DataFrame( + {"identifier": ["a", "b", "c"]} + ) + embedder = SNPEmbedder(wrapper) + result = embedder.predict_variant_effect(_make_snp(), chromosome_sequence=CHR_SEQ) + assert result.track_names == ["a", "b", "c"] + + def test_top_tracks_ranks_by_absolute_score(self): + snp = _make_snp() + result = VariantEffectResult( + snp=snp, + ref_profile=np.zeros((4, 2)), + alt_profiles=[np.zeros((4, 2))], + effect_scores=[np.array([0.1, -5.0, 2.0, -0.5])], + track_names=["w", "x", "y", "z"], + ) + top = result.top_tracks(n=2) + assert top[0][0] == "x" + assert top[1][0] == "y" \ No newline at end of file