diff --git a/CHANGELOG.md b/CHANGELOG.md index eb7a130e7..856b98000 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ Each release's notes. From 1.0.0 on, a major or minor release has hand-written n ## Unreleased +- summary.json has a new section, `acmg_sf`: the ACMG secondary-findings card, built from step 06's ClinVar hits on every run, with or without the VEP cache. Before, the card existed only inside step 23's clinical section, so a run without the VEP cache had no card at all. The new section's state follows step 06. Without step 23's clinical VCF it has `high_impact_checked: false` and a `high_impact_note` that says why and what it can miss (a predicted loss-of-function variant not in ClinVar). A clinical VCF that cannot be read no longer costs the ClinVar rows of this card. `sections.clinical.data.acmg_sf` stays as it was, and the reports still read the card there, so they show it only when step 23 ran. The first run after the update lists the new section under "New since" once. - A third-party outage no longer fails the Freshness check of a pull request. Since 2026-11-18 rest.ensembl.org has not answered, and the pull-request job's dry run of `og dev freshness` failed on every pull request that touched `versions.env` or `og-fresh`. `og dev freshness --unreachable-warns` now lists a lookup whose upstream did not answer (no connection, a timeout, HTTP 5xx or 429 after the retries, or HTTP 403) under "Upstreams that did not answer" as a warning, and its database row reads `NO ANSWER`; it does not change the exit. The 403 counts because rest.ensembl.org came back on 2026-11-19 answering 403 to GitHub's runners and 200 elsewhere, which still failed the pull-request job. A 404, an empty answer or an answer in an odd shape still exits 1. `freshness.yml` passes the flag only in the pull-request dry run. Without the flag the report and the exit are as before, and the monthly report on main still counts an outage as an error in the issue. - Three statements in the docs were wrong. Troubleshooting said exit code 137 confirms a kill for memory, and checked it with `docker ps -a --latest`. 137 is any SIGKILL, and the step scripts run their containers with `--rm`, so that command shows an older, unrelated container. The page now says how to confirm a kill for memory: `docker events --filter event=oom`, the kernel log, or `.State.OOMKilled` for a container kept without `--rm`. The run-state and troubleshooting pages said `bin/og` does not run on a Mac and told Mac users to build `og` with cargo or run it in a container. `bin/og` runs the native macOS build that `scripts/setup.sh --og` installs, and the pages now say so; on a Mac, `og run headroom` reports the machine's memory as unknown. Troubleshooting, the FAQ and the lessons-learned page said the step scripts run their containers as root, and lessons-learned gave `:latest` images and `chmod 777` as current fixes. The step scripts run their containers as you; those entries are now marked as past fixes and name what the pipeline does today. Documentation only: no option, output or behaviour changes. - Lint's ShellCheck job now checks every tracked shell script, not only `scripts/`: the e2e cases and their library, the fake-docker and parity tests, `tests/*.sh`, `bin/og` and the fake tools, 482 files in all, at every ShellCheck level. The 265 findings these files hold today are listed in `tests/shellcheck-baseline.txt` and still pass; any other finding fails lint. A listed finding is matched by its file, its code and the text of its line, so it keeps matching when lines above it move, and a new line with the same mistake still fails. Each run plants an unquoted `$1` in a new test script and on a new line of a listed script, and fails if either passes or if the file search misses the new script or `bin/og`. The `scripts/` check at warning level stays as it was. Part of #411 (plat-10). diff --git a/docs/06-clinvar-screen.md b/docs/06-clinvar-screen.md index 522b0fc9a..335f542ed 100644 --- a/docs/06-clinvar-screen.md +++ b/docs/06-clinvar-screen.md @@ -121,6 +121,8 @@ The [runtime table](hardware-requirements.md#runtime-per-step) gives the measure Both reports (step 24 and `generate-report.sh`) read the hits file and show gene, genotype (het, hom, or "one copy, other copy not called ref" for the half of a split `1/2` call), significance, review status, stars and the combined-allele note ("combined allele of ...", "part of a combined allele: ...", both followed by "; unphased, so the parts may sit on different haplotypes" on an unphased het call, or "possibly one allele with ...: check ClinVar for a combined record") for each hit, the best-reviewed first, with the ClinVar file's release date, the gnomAD frequency and the low-confidence mark. The headline count leaves the low-confidence calls out and a line beside it counts them. When the cDNA check table is there, both print its result beside each hit (the HTML hits table has a "cDNA read check" column) and the ClinVar card counts the flagged hits. Older versions of this step wrote `clinvar/isec/0002.vcf`; that file holds only the sample's side of the intersection, with no gene or significance, and nothing reads it any more. Rerun step 6 to get the hits file. +summary.json also has an ACMG secondary-findings card built from these hits, `sections.acmg_sf`. It follows this step, not step 23: it is there whenever the hits file is, also in a run without the VEP cache, and is not run (or failed) with this step's reason otherwise. It holds the hits in the 84 genes of the ACMG SF v3.3 list with each gene's rule applied, step 05's class 4 and 5 SVs in the genes whose rule allows them, and the deletions that count as a second hit. Step 23's rare HIGH-impact records join it when step 23 left its clinical VCF. Without that VCF the card says `high_impact_checked: false`, and `high_impact_note` names why (step 23 skipped, with the run's reason, failed, or its VCF unreadable) and what that can miss: a predicted loss-of-function variant that is not in ClinVar, alone or as the second hit of a recessive gene. [Step 23](23-clinical-filter.md#secondary-findings-genes-acmg-sf-v33) gives the rules. The reports still show the card from step 23's section, so they show it only when step 23 ran. + `${SAMPLE}_clinvar_hits.tsv` is the table to read or load elsewhere. To list the hits from the VCF yourself: ```bash diff --git a/docs/23-clinical-filter.md b/docs/23-clinical-filter.md index 28388bf82..697ad788f 100644 --- a/docs/23-clinical-filter.md +++ b/docs/23-clinical-filter.md @@ -90,6 +90,8 @@ The Nextflow `CLINICAL_FILTER` module applies the same tiers, ranking, frequency Both reports list, beside the clinical filter's counts, the secondary findings in the 84 genes of the ACMG SF v3.3 list (Lee et al., Genet Med 2025;27(8):101454; the genes in which the ACMG recommends reporting pathogenic variants found by chance, such as BRCA1, LDLR, MYBPC3 or TTN): the step 6 ClinVar hits in those genes, the clinical filter's records with a HIGH-impact consequence on the ranked transcript in them, and step 5's AnnotSV class 4 and 5 structural variants in the genes whose rule allows them. Each gene's rule from Table 1 of the list is a row of `assets/acmg_sf.tsv`: one variant in a dominant gene, two in a recessive one (MUTYH, BTD, ATP7B and others; a homozygote, or two variants of unknown phase), HFE only as homozygous p.C282Y, GLA and OTC also in a heterozygous female, ABCD1 hemizygous, homozygous or with two heterozygous variants. A variant the rule does not report is listed apart with the reason; a pseudodeficiency allele is not counted, and a BTD finding with the p.Asp444His hypomorph says partial, not profound. Of the reported HIGH-impact records the reports count as "rare HIGH impact" only the rare ones with a sound call; a weak call or an unknown frequency is listed with its mark and counted apart, and a common one is left out. The list version is `ACMG_SF_VERSION` in `og summary collect` (`rust/crates/og-summary`). It is a list to review with a clinician: a rare HIGH-impact variant is not a ClinVar classification. [Interpreting results](interpreting-results.md#carrier-status-is-not-disease) gives every rule. Step 17 (CPSR) reports the ACMG secondary findings with its own classification. +summary.json holds the card twice. `sections.clinical.data.acmg_sf` is the card as both reports read it. It is there whenever this step's summary table or clinical VCF can be read, also one left by an earlier run (then, when this step did not run ok, the clinical section is stale). `sections.acmg_sf` is the same card as a section of step 6: it is written whenever step 6's hits are there, so a run without the VEP cache still gets the ClinVar and SV findings. Without this step's clinical VCF it has `high_impact_checked: false` and a `high_impact_note` that names why (this step skipped, with the run's reason, failed, or its VCF unreadable) and what that can miss: a predicted loss-of-function variant that is not in ClinVar, alone or as the second hit of a recessive gene. With the clinical VCF the two cards are equal, except when that VCF is from an earlier run (this step not ok in this run): then `sections.acmg_sf` adds a `high_impact_note` with the earlier VCF's date. A clinical VCF that cannot be read, or whose HIGH-impact records cannot be parsed, makes the clinical section's card unavailable, while `sections.acmg_sf` keeps its ClinVar rows with `high_impact_checked: false` and says the VCF could not be read. The two cards then differ. + The `table_text` column of `assets/acmg_sf.tsv` quotes each gene's rule from Table 1 of the paper (doi:10.1016/j.gim.2025.101454), such as "All P and LP" or "P and LP (2 variants)". summary.json gives that text as the rule of each finding, and the reports use it in the reason a variant is not reported. These are short cited quotations of a factual reporting rule: six phrases, the longest 44 characters. "(See text)" after a rule points to the paper's section on the dilated cardiomyopathy (DCM) genes. The `note` column says what the table's footnotes add, in our own words. ## Runtime diff --git a/docs/24-html-report.md b/docs/24-html-report.md index a46341805..b78de3d17 100644 --- a/docs/24-html-report.md +++ b/docs/24-html-report.md @@ -91,7 +91,7 @@ The cards, by chapter (cards not listed here are named in the chapter table abov - **ClinVar Screening** — hit count by review stars, the ClinVar file date, and the hits table best-reviewed first with a Stars column. The count's badge is yellow when there is a hit and gray when there is none, never green. Each hit shows its gnomAD allele frequency from step 13's VEP output, and a line says where the frequencies came from and how many hits are above 1%, or why every frequency is unknown: a common allele with a P/LP record is often low-penetrance, so read every hit with the frequency the card shows, and look up a hit whose frequency is unknown. A low-confidence line counts the hits whose call is under step 6's GQ or DP floor: they are listed but not counted in the total. When step 6's cDNA read check ran, the card also counts the hits it flagged - **Clinical Filter** and **slivar** — variant counts from steps 23 and 31 -- **Secondary-Findings Genes (ACMG SF v3.3)** — the step 06 ClinVar hits, the clinical filter's rare HIGH-impact records and step 05's class 4 and 5 SVs in the 84 genes of the ACMG list, each gene's reporting rule applied (`assets/acmg_sf.tsv`, for example HFE homozygous C282Y only), and the variants a rule does not report listed apart with the reason +- **Secondary-Findings Genes (ACMG SF v3.3)** — the step 06 ClinVar hits, the clinical filter's rare HIGH-impact records and step 05's class 4 and 5 SVs in the 84 genes of the ACMG list, each gene's reporting rule applied (`assets/acmg_sf.tsv`, for example HFE homozygous C282Y only), and the variants a rule does not report listed apart with the reason. The card is shown when step 23 ran. summary.json also writes it as its own section, `acmg_sf`, from step 06's hits, with or without step 23 ([step 06](06-clinvar-screen.md#output)) - **Cancer Predisposition** — CPSR status and the classification breakdown (read by column name from `${SAMPLE}.cpsr.grch38.classification.tsv.gz`) - **Structural Variants** — Manta, Delly, CNVpytor and the SURVIVOR consensus (2+ callers) counts; then step 05's class 4 and 5 SVs (AnnotSV) with their caller support, exon-edge and frequency checks. A call step 05's rules moved to class 3 (a coding tandem repeat, an immunoglobulin or T-cell receptor locus, the MHC, a pseudogene named by a disease gene's alias, or read depth that does not show the change) is listed apart with AnnotSV's class, the rule and the reason, and the class 4-5 count leaves it out - **Repeat Expansions** — a table of every locus, the five key loci (HTT, FMR1, C9orf72, ATXN1, DMPK) first, with its repeat count, Stranger's status and notes; and counts of the loci outside the normal range (step 9b, Stranger), of those short reads could not size (not counted as normal), and the loci in an intermediate range (`bin/str_intermediate_ranges.tsv`). When Stranger flags RFC1, or another locus where the motif decides (`expansion_hunter/${SAMPLE}_str_motifs.tsv`), the card adds the verdict of the read-level motif count (step 9b). A locus the repeat catalog gives no thresholds (its `NoThresholds` field in `expansion_hunter/${SAMPLE}_eh_catalog.json`: MUC1, CEL, TYMS and PLIN4 with the STRchive catalog, among others) is marked not graded, with the catalog's reason, never normal. Any other locus outside Stranger's catalog has no status and is counted apart. Without step 9b the card says Stranger did not run; a Stranger file older than the ExpansionHunter result it came from is not read, and the card says to rerun step 9b. Repeat-expansion calls from short reads can be wrong at some loci diff --git a/docs/results/secondary-findings.md b/docs/results/secondary-findings.md index 94e96b430..ee96e62c4 100644 --- a/docs/results/secondary-findings.md +++ b/docs/results/secondary-findings.md @@ -14,7 +14,7 @@ The most common "pathogenic" finding in any genome is **heterozygous carrier sta **Your own carrier list (step 24 report).** The ClinVar card of both reports is followed by a carrier list for this one sample, with no partner needed: every step 6 ClinVar hit in the 3,292 recessive and X-linked genes of `assets/carrier_genes.tsv` (the list of the [couple screen](../39-couple.md#the-carrier-list)). An allele that list marks as a hypomorph (for example BTD p.Asp444His or the GALT Duarte variants) or a pseudodeficiency allele (HEXA c.739C>T and c.745C>T, which lower the enzyme test result but cause no Tay-Sachs disease) is listed as set aside, never as a carrier finding on its own. -**Secondary findings follow each gene's ACMG rule.** The secondary-findings card (step 23's section of the report) applies the reporting rule of each of the 84 ACMG SF v3.3 genes (Lee et al., Genet Med 2025;27(8):101454, Table 1), written gene by gene in `assets/acmg_sf.tsv`: +**Secondary findings follow each gene's ACMG rule.** The secondary-findings card (step 23's section of the report; in summary.json also `sections.acmg_sf`, written from step 6's hits on every run, VEP cache or not) applies the reporting rule of each of the 84 ACMG SF v3.3 genes (Lee et al., Genet Med 2025;27(8):101454, Table 1), written gene by gene in `assets/acmg_sf.tsv`: - a dominant gene (BRCA1, LDLR, MYH7 and most others): one pathogenic or likely pathogenic variant is reported; - a recessive gene (MUTYH, ATP7B, BTD, CYP27A1, GAA, RPE65, CASQ2, TRDN): two variants are needed, a homozygote or two different ones even when their phase is unknown. One heterozygous MUTYH variant is listed apart as not reported, with the reason; @@ -31,6 +31,8 @@ The most common "pathogenic" finding in any genome is **heterozygous carrier sta Examples: an exon deletion of MUTYH plus a pathogenic MUTYH variant; three alpha-globin genes plus an HBA2 point variant; one SMN1 copy plus an SMN1 variant (short reads place few SMN1 variants reliably, so check with a targeted test); and the 1q21.1 deletion plus one of the two RBM8A low-expression alleles (c.-21G>A, rs139428292, or c.67+32G>C, rs201779890), the cause of TAR syndrome (thrombocytopenia with absent radius; Albers et al., Nat Genet 2012;44(4):435-439, PMID 22366785). On its own an RBM8A low-expression allele stays set aside. The [couple screen](../39-couple.md) (step 39) applies the same pairing between partners: a hypomorph allele of the list in one partner and a full loss of function in the same gene in the other (a ClinVar pathogenic variant, a Delly deletion) is a shared gene with its risk for a child; the allele in both partners, or in one alone, stays set aside. In the secondary-findings card a called deletion counts as one of the two variants a recessive gene needs, unless read-backed phase puts it on the variant's copy. The line under the carrier list names the deletion callers it read and the ones it could not read. A deletion AnnotSV ranks below class 4 is not counted. +Without a readable clinical VCF from step 23 (no VEP cache, step 23 skipped or failed with no VCF left, or a VCF that cannot be read) the card in `sections.acmg_sf` holds the ClinVar hits and the SVs only, and its `high_impact_note` says so: a predicted loss-of-function variant that is not in ClinVar is not screened, alone or as the second hit of a recessive gene. A clinical VCF left by an earlier run is still read when step 23 failed in this run, as the clinical section reads it; that section then shows as stale, and the card's `high_impact_note` gives the date of that earlier VCF and step 23's result in this run. + The TTN rule (truncating variants only) and the LMNA rule (missense only with cardiac evidence) are printed beside the finding, not applied. It is still a list to review with a clinician. --- diff --git a/rust/crates/og-render/src/chapters.rs b/rust/crates/og-render/src/chapters.rs index ea46924ed..7ea79341b 100644 --- a/rust/crates/og-render/src/chapters.rs +++ b/rust/crates/og-render/src/chapters.rs @@ -266,8 +266,21 @@ pub fn step_of(key: &str) -> &'static str { .map_or("?", |s| s.step) } +/// Summary keys no frame section lists, because the section of another key +/// already shows what they hold: `(key, the key whose section shows it)`. +/// `acmg_sf` is the secondary-findings card the clinical section also holds +/// as `clinical.data.acmg_sf`, and the reports read it there. Listing it in +/// the clinical-filter frame would change that section's step, state and +/// file date before any reader shows it. It moves into the frame when the +/// readers switch to `sections.acmg_sf` (i411-display-sf). +pub const SHOWN_WITH: &[(&str, &str)] = &[("acmg_sf", "clinical")]; + /// The section that shows summary key `key`. pub fn section_of(key: &str) -> Option<&'static Sec> { + let key = SHOWN_WITH + .iter() + .find(|(k, _)| *k == key) + .map_or(key, |(_, with)| *with); CHAPTERS .iter() .flat_map(|c| c.sections.iter()) @@ -283,12 +296,19 @@ pub fn section(sid: &str) -> Option<&'static Sec> { } /// What is wrong with a chapter table against the summary keys: a key no -/// section shows, a key two sections show, a key no collector writes. +/// section shows, a key two sections show, a key no collector writes. A key +/// of `SHOWN_WITH` counts as shown where its partner key is. pub fn key_problems(chapters: &[Chapter], keys: &[&str]) -> Vec { - let seen: Vec<&str> = chapters + let mut seen: Vec<&str> = chapters .iter() .flat_map(|c| c.sections.iter().flat_map(|s| s.keys.iter().copied())) .collect(); + let carried: Vec<&str> = SHOWN_WITH + .iter() + .filter(|(_, with)| seen.contains(with)) + .map(|(k, _)| *k) + .collect(); + seen.extend(carried); let mut out: Vec = keys .iter() .filter(|k| !seen.contains(k)) diff --git a/rust/crates/og-render/src/index.rs b/rust/crates/og-render/src/index.rs index 5f7797a74..3f74d8619 100644 --- a/rust/crates/og-render/src/index.rs +++ b/rust/crates/og-render/src/index.rs @@ -450,6 +450,13 @@ pub fn build( if !matches!(s.get("state"), Some(Value::Str(st)) if st == "ok" || st == "stale") { continue; } + // The ACMG card of its own section is the card the clinical section + // also holds (`clinical.data.acmg_sf`), which the index already reads. + // Skip it so a gene is not listed twice, until the report readers + // move from the nested card to this section (i411-display-sf). + if key == "acmg_sf" { + continue; + } let field = |k: &str| s.get(k).cloned().unwrap_or(Value::Null); let mut add = |term: &str, txt: &str, locus: Option<&str>| { let items = terms.entry(term.to_owned()).or_default(); diff --git a/rust/crates/og-render/tests/port_report_core.rs b/rust/crates/og-render/tests/port_report_core.rs index b8d8d44ca..6f846eeb6 100644 --- a/rust/crates/og-render/tests/port_report_core.rs +++ b/rust/crates/og-render/tests/port_report_core.rs @@ -177,7 +177,7 @@ fn the_release_stands_next_to_the_commit_in_both_reports() { #[test] fn every_summary_key_is_in_exactly_one_section() { let keys: Vec<&str> = summary_keys().map(|s| s.key).collect(); - assert_eq!(keys.len(), 48); + assert_eq!(keys.len(), 49); assert_eq!(key_problems(&CHAPTERS, &keys), Vec::::new()); // Red controls: a key taken out, and a key in two sections, are caught. let cut: Vec<&str> = keys.iter().copied().filter(|k| *k != "telomere").collect(); @@ -202,6 +202,25 @@ fn every_summary_key_is_in_exactly_one_section() { let mut missing = key_problems(&short, &keys); missing.sort(); assert_eq!(missing, ["missing consumer_export", "missing viral"]); + // acmg_sf is shown with clinical, not as a frame key of its own: listing + // it in a section as well is a key shown twice, and the search index + // anchors it to the clinical-filter section. + let mut also = CHAPTERS.to_vec(); + also.push(Chapter { + id: "x", + title: "X", + question: "?", + sections: &[Sec { + id: "x", + title: "X", + keys: &["acmg_sf"], + }], + }); + assert_eq!(key_problems(&also, &keys), ["twice acmg_sf".to_owned()]); + assert_eq!( + og_render::chapters::section_of("acmg_sf").map(|s| s.id), + Some("clinical-filter") + ); } #[test] diff --git a/rust/crates/og-summary/src/collect/acmg_card.rs b/rust/crates/og-summary/src/collect/acmg_card.rs index 9506a9040..dabb42721 100644 --- a/rust/crates/og-summary/src/collect/acmg_card.rs +++ b/rust/crates/og-summary/src/collect/acmg_card.rs @@ -1,2 +1,140 @@ -//! The summary section of the ACMG card, computed from the step 06 hits. -//! Empty until its unit of the 1.0.0 plan lands. +//! The ACMG secondary-findings card as its own summary section, +//! `acmg_sf`, whose state follows step 06: ok when step 06's ClinVar hits +//! VCF is there, not run (or failed) with step 06's reason otherwise. +//! +//! The card's main input needs no VEP: step 06's P/LP hits, step 05's +//! class 4 and 5 SVs and step 35's deletions, with each gene's rule from +//! `assets/acmg_sf.tsv`. Only the rare HIGH-impact source needs step 23's +//! clinical VCF, so a run without the VEP cache still gets the card. The +//! verdicts are [`acmg::acmg_sf`]'s, the one function the clinical section +//! also calls: with the clinical VCF when it is there and readable, without +//! it otherwise. Without it the card says `high_impact_checked: false` and +//! `high_impact_note` names why and what that can miss. With a clinical +//! VCF left by an earlier run (step 23 not ok in this run, the VCF older +//! than the run's start) the card reads it and its `high_impact_note` says +//! the HIGH-impact source is from that earlier run. + +use super::acmg::{self, ClinicalVcf}; +use super::clinvar::HitsVcf; +use super::two_hits::DeletionInputs; +use pycompat::json::Value; + +/// Why step 23's rare HIGH-impact records are not in the card. +#[derive(Debug, Clone, PartialEq)] +pub enum NoHighImpact { + /// The run status gives step 23's result (`failed (exit 1)`, + /// `skipped (needs VEP, ...)`, or `ok` with no clinical VCF left). + Step23(String), + /// No run status names step 23 (a Nextflow report, a folder without + /// `logs/run_status.tsv`). + NotRun, + /// The clinical VCF is there and cannot be read. + Unreadable(String), +} + +impl NoHighImpact { + /// `high_impact_note`: why, then what the card can miss. + pub fn note(&self) -> String { + let why = match self { + NoHighImpact::Step23(r) if r.starts_with("failed") => { + format!("step 23 (clinical filter) {r}") + } + NoHighImpact::Step23(r) if r.starts_with("skipped") => { + format!("step 23 (clinical filter) was {r}") + } + NoHighImpact::Step23(r) => { + format!("step 23 (clinical filter) was {r} but wrote no clinical VCF") + } + NoHighImpact::NotRun => { + "step 23 (clinical filter) did not run; it needs VEP's annotation (step 13) and \ + the VEP cache" + .to_owned() + } + NoHighImpact::Unreadable(e) => format!("step 23's clinical VCF cannot be read: {e}"), + }; + format!( + "Rare HIGH-impact variants were not screened: {why}. This card holds the ClinVar P/LP \ + hits and the class 4 and 5 structural variants only, so it can miss a predicted \ + loss-of-function variant that is not in ClinVar, alone or as the second hit of a \ + recessive gene." + ) + } +} + +/// What the card reads: the step 06 hits, the clinical VCF when step 23 +/// left one, the deletions of the two-hit rule, step 23's result word +/// from the run status, and the clinical VCF's file date when it is from +/// an earlier run. +pub struct Inputs<'a> { + pub assets: &'a str, + pub hits: &'a HitsVcf, + pub clinical: Option<&'a Result>, + pub deletions: &'a DeletionInputs, + pub step23: Option<&'a str>, + /// The clinical VCF's file date (UTC minute) when step 23 was not ok in + /// this run and the VCF was written before the run started. + pub earlier_vcf: Option, +} + +/// `high_impact_note` for a card that read a clinical VCF of an earlier run. +pub fn earlier_note(date: &str, step23: Option<&str>) -> String { + let result = step23.filter(|r| !r.is_empty()).unwrap_or("not run"); + format!( + "Rare HIGH-impact variants come from the clinical VCF of an earlier run ({date}); step \ + 23 (clinical filter) was {result} in this run." + ) +} + +/// The data of the `acmg_sf` section. +#[derive(Debug, Clone, PartialEq)] +pub struct AcmgCard(pub Value); + +impl AcmgCard { + /// The card. An `Err` is the section's note (a hits record whose POS + /// is no integer); a rules list or a hits VCF that cannot be read gives + /// the card's `unavailable` form, as in the clinical section. + pub fn read(i: &Inputs) -> Result { + // A clinical VCF that cannot be read costs the HIGH-impact source + // only, never the ClinVar rows. + let (clinical, missing) = match i.clinical { + Some(Ok(c)) => match &c.high { + Ok(_) => (i.clinical, None), + Err(e) => (None, Some(NoHighImpact::Unreadable(e.clone()))), + }, + Some(Err(e)) => (None, Some(NoHighImpact::Unreadable(e.clone()))), + None => ( + None, + Some( + i.step23 + .map_or(NoHighImpact::NotRun, |r| NoHighImpact::Step23(r.to_owned())), + ), + ), + }; + let mut v = acmg::acmg_sf(&acmg::Inputs { + assets: i.assets, + hits: Some(&i.hits.read), + clinical, + deletions: i.deletions, + })?; + if let Value::Object(m) = &mut v { + let checked = m.get("high_impact_checked").cloned(); + match (missing, &i.earlier_vcf) { + (Some(why), _) if checked == Some(Value::Bool(false)) => { + m.insert("high_impact_note".into(), Value::Str(why.note())); + } + (None, Some(date)) if checked == Some(Value::Bool(true)) => { + m.insert( + "high_impact_note".into(), + Value::Str(earlier_note(date, i.step23)), + ); + } + _ => {} + } + } + Ok(AcmgCard(v)) + } + + pub fn to_json(&self) -> Value { + self.0.clone() + } +} diff --git a/rust/crates/og-summary/src/collect/group_b.rs b/rust/crates/og-summary/src/collect/group_b.rs index 81426f5bb..ab338db0e 100644 --- a/rust/crates/og-summary/src/collect/group_b.rs +++ b/rust/crates/og-summary/src/collect/group_b.rs @@ -1,15 +1,18 @@ //! Group b of `og summary collect`: the clinical sections. ClinVar, the //! carrier list, PharmCAT, CPIC, pypgx, Cyrius, CPSR, the clinical filter -//! with its ACMG secondary-findings tier, slivar, and the `cyp2d6` block. +//! with its ACMG secondary-findings tier, the ACMG card of its own +//! (`acmg_sf`, step 06), slivar, and the `cyp2d6` block. //! //! Each input is read once: the ClinVar hits VCF feeds the ClinVar -//! section, the ACMG tier, the carrier list and CPSR's call floor; the -//! clinical VCF feeds the record count and the ACMG tier; AnnotSV's table +//! section, the ACMG card, the ACMG tier, the carrier list and CPSR's call +//! floor; the clinical VCF feeds the record count, the ACMG card and the +//! ACMG tier; AnnotSV's table //! and step 35's copy-number tables feed both two-hit readings. The //! sections are read one at a time, so the peak memory is the biggest //! section's, not the sum. use super::acmg::{self, ClinicalVcf}; +use super::acmg_card::{self, AcmgCard}; use super::carrier::Carrier; use super::clinical::{Clinical, Slivar}; use super::clinvar::{ClinVar, Floor, HitsVcf}; @@ -22,7 +25,7 @@ use super::two_hits::{self, DeletionInputs}; use super::{Folder, Input, Read, read_at, with_mtime}; /// The sections `og summary collect --only b` reads, in summary.json's order. -pub const GROUP_B: [Spec; 9] = [ +pub const GROUP_B: [Spec; 10] = [ Spec { key: "clinvar", title: "ClinVar screen", @@ -33,6 +36,11 @@ pub const GROUP_B: [Spec; 9] = [ title: "Carrier list", step: "06", }, + Spec { + key: "acmg_sf", + title: "ACMG secondary findings", + step: "06", + }, Spec { key: "pharmcat", title: "PharmCAT", @@ -134,10 +142,39 @@ pub fn read(input: &Input, folder: &Folder) -> Vec
{ Some(h) => ClinVar::read(h, d, &side, &s) .and_then(|data| with_mtime(&h.path, h.rel.clone(), Data::ClinVar(data))), }; + let clinical_vcf = clinical_vcf_at.as_ref().map(|(v, _)| ClinicalVcf::read(v)); + // The card follows step 06: it needs the hits, not step 23. + let step23 = folder + .run + .as_ref() + .and_then(|r| r.steps.get("23")) + .map(String::as_str); + // A clinical VCF older than this run, with step 23 not ok in it, is an + // earlier run's: the card says so, as the clinical section goes stale. + let earlier_vcf = match (&folder.run, &clinical_vcf_at) { + (Some(run), Some((p, _))) if step23 != Some("ok") => run + .started_epoch + .zip(pycompat::mtime::mtime(p).ok()) + .filter(|&(started, m)| super::model::is_stale(m, started)) + .and_then(|(_, m)| super::time::utc_minute(m)), + _ => None, + }; + let acmg_sf: Read = match &hits { + None => Ok(None), + Some(h) => AcmgCard::read(&acmg_card::Inputs { + assets: &input.assets, + hits: h, + clinical: clinical_vcf.as_ref(), + deletions: &deletions, + step23, + earlier_vcf, + }) + .and_then(|c| with_mtime(&h.path, h.rel.clone(), Data::AcmgSf(c))), + }; let clinical: Read = match clinical_table.clone().or_else(|| clinical_vcf_at.clone()) { None => Ok(None), Some((p, rel)) => { - let vcf = clinical_vcf_at.as_ref().map(|(v, _)| ClinicalVcf::read(v)); + let vcf = clinical_vcf; let tier = acmg::Inputs { assets: &input.assets, hits: hits.as_ref().map(|h| &h.read), @@ -155,7 +192,7 @@ pub fn read(input: &Input, folder: &Folder) -> Vec
{ let rest = read_rest(input, &s, index, &find, &hits, &deletions); let [carrier, pharmcat, cpic, pypgx, cyrius, cpsr, slivar] = rest; let reads = [ - clinvar, carrier, pharmcat, cpic, pypgx, cyrius, cpsr, clinical, slivar, + clinvar, carrier, acmg_sf, pharmcat, cpic, pypgx, cyrius, cpsr, clinical, slivar, ]; GROUP_B .iter() diff --git a/rust/crates/og-summary/src/collect/post.rs b/rust/crates/og-summary/src/collect/post.rs index 305bc87ac..b528b9b59 100644 --- a/rust/crates/og-summary/src/collect/post.rs +++ b/rust/crates/og-summary/src/collect/post.rs @@ -32,7 +32,7 @@ use pycompat::value::truthy; use std::path::Path; /// summary.json's sections, in its order. -pub const ORDER: [&str; 48] = [ +pub const ORDER: [&str; 49] = [ "coverage", "callability", "sex_check", @@ -43,6 +43,7 @@ pub const ORDER: [&str; 48] = [ "clinvar", "couple", "carrier", + "acmg_sf", "pharmcat", "cpic", "pypgx", @@ -85,8 +86,9 @@ pub const ORDER: [&str; 48] = [ /// Sections that are no step of their own, so never in `not_run`: the read /// count is written by alignment, which a run from a BAM or a VCF never -/// does; the couple screen and the carrier list come with other steps. -const NOT_A_STEP: [&str; 3] = ["read_count", "couple", "carrier"]; +/// does; the couple screen, the carrier list and the ACMG secondary-findings +/// card come with other steps (39 and 06), whose own sections say not run. +const NOT_A_STEP: [&str; 4] = ["read_count", "couple", "carrier", "acmg_sf"]; /// What no step assesses. const NOT_ASSESSED: [&str; 5] = [ diff --git a/rust/crates/og-summary/src/collect/sections.rs b/rust/crates/og-summary/src/collect/sections.rs index a354c126f..8b5e5f73f 100644 --- a/rust/crates/og-summary/src/collect/sections.rs +++ b/rust/crates/og-summary/src/collect/sections.rs @@ -24,6 +24,7 @@ pub enum Data { Json(Value), ClinVar(super::clinvar::ClinVar), Carrier(super::carrier::Carrier), + AcmgSf(super::acmg_card::AcmgCard), PharmCat(super::pgx::PharmCat), Cpic(super::pgx::Cpic), Pypgx(super::pgx::Pypgx), @@ -46,6 +47,7 @@ impl Data { Data::Json(v) => v.clone(), Data::ClinVar(d) => d.to_json(), Data::Carrier(d) => d.to_json(), + Data::AcmgSf(d) => d.to_json(), Data::PharmCat(d) => d.to_json(), Data::Cpic(d) => d.to_json(), Data::Pypgx(d) => d.to_json(), diff --git a/rust/crates/og-summary/tests/screen_acmg_card_section.rs b/rust/crates/og-summary/tests/screen_acmg_card_section.rs new file mode 100644 index 000000000..539218b06 --- /dev/null +++ b/rust/crates/og-summary/tests/screen_acmg_card_section.rs @@ -0,0 +1,446 @@ +//! The ACMG secondary-findings card is its own section, `acmg_sf`, and +//! follows step 06: a run without the VEP cache (no step 23, no +//! `clinical/` folder) still gets it from the ClinVar hits. +//! +//! 1. Red first: a folder with step 06's hits (a BRCA2 P/LP heterozygote, +//! two MUTYH heterozygotes) and no `clinical/` folder. Before the +//! section existed it got no card at all. +//! 2. Agreement: every summary-collect parity folder of group b +//! (`tests/parity/gen/summary-clinical.sh`) that has both the hits and +//! the clinical VCF, collected as it is and again without `clinical/`. +//! Without step 23 the card may lose only what the HIGH-impact source +//! brings: (a) every ClinVar row of the card without VEP is in the card +//! with VEP, unchanged; (b) every gene reported without VEP is reported +//! with VEP; (c) a row not reported without VEP stays as it is, or is +//! reported with VEP because a HIGH-impact row of its gene joined it. + +use og_summary::collect::{Group, Input, group_b, post}; +use pycompat::json::Value; +use std::path::{Path, PathBuf}; +use std::process::Command; + +fn repo() -> PathBuf { + Path::new(env!("CARGO_MANIFEST_DIR")).join("../../..") +} + +fn scratch(name: &str) -> PathBuf { + let d = std::env::temp_dir().join(format!("og-screen-acmg-{name}-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&d); + std::fs::create_dir_all(&d).unwrap(); + d +} + +fn put(p: &Path, text: &str) { + std::fs::create_dir_all(p.parent().unwrap()).unwrap(); + std::fs::write(p, text).unwrap(); +} + +fn input(dir: &Path, assets: &Path, group: Group) -> Input { + Input { + sample: "S".into(), + sample_dir: dir.to_str().unwrap().to_owned(), + declared_sex: None, + assets: assets.to_str().unwrap().to_owned(), + group, + tables: og_summary::collect::group_c::util::default_tables(), + } +} + +/// The sections of group b, as summary.json writes them. +fn sections_b(dir: &Path, assets: &Path) -> Value { + let v = group_b::collect(&input(dir, assets, Group::B)) + .unwrap() + .to_json(); + at(&v, &["sections"]).clone() +} + +fn at<'a>(v: &'a Value, path: &[&str]) -> &'a Value { + path.iter().fold(v, |v, k| match v { + Value::Object(m) => m + .get(*k) + .unwrap_or_else(|| panic!("no {k} in {path:?}: {v:?}")), + _ => panic!("{path:?} is no object at {k}"), + }) +} + +fn get<'a>(v: &'a Value, k: &str) -> Option<&'a Value> { + match v { + Value::Object(m) => m.get(k), + _ => None, + } +} + +fn s(v: &Value) -> &str { + match v { + Value::Str(x) => x, + _ => panic!("no string: {v:?}"), + } +} + +fn rows(card: &Value, list: &str) -> Vec { + match get(card, list) { + Some(Value::Array(a)) => a.clone(), + None => Vec::new(), + Some(x) => panic!("{list} is no list: {x:?}"), + } +} + +fn gene_variant(r: &Value) -> (String, String) { + ( + s(at(r, &["gene"])).to_owned(), + s(at(r, &["variant"])).to_owned(), + ) +} + +const HITS_HEAD: &str = "##fileformat=VCFv4.2\n##PGP_CALL_FLOOR=GQ=20,DP=8\n\ +#CHROM\tPOS\tID\tREF\tALT\tQUAL\tFILTER\tINFO\tFORMAT\tS\n"; +const PL: &str = "CLNREVSTAT=criteria_provided,_multiple_submitters,_no_conflicts"; + +/// Step 06's hits: a BRCA2 P/LP heterozygote and two MUTYH heterozygotes. +fn hits() -> String { + format!( + "{HITS_HEAD}\ +chr13\t32340300\t1\tC\tT\t.\tPASS\tGENEINFO=BRCA2:675;CLNSIG=Pathogenic;{PL}\tGT:GQ:DP\t0/1:60:35\n\ +chr1\t45331556\t2\tC\tT\t.\tPASS\tGENEINFO=MUTYH:4595;CLNSIG=Pathogenic;{PL}\tGT:GQ:DP\t0/1:50:30\n\ +chr1\t45332400\t3\tG\tA\t.\tPASS\tGENEINFO=MUTYH:4595;CLNSIG=Likely_pathogenic;{PL}\tGT:GQ:DP\t0/1:55:32\n" + ) +} + +#[test] +fn a_run_without_step_23_gets_the_card_from_the_clinvar_hits() { + let d = scratch("red"); + let sd = d.join("S"); + put(&sd.join("clinvar/S_clinvar_hits.vcf"), &hits()); + let assets = repo().join("assets"); + let secs = sections_b(&sd, &assets); + assert!( + get(&secs, "clinical").is_some_and(|c| s(at(c, &["state"])) == "missing"), + "no clinical section: step 23 did not run" + ); + let sec = get(&secs, "acmg_sf").expect("summary.json has no acmg_sf section"); + assert_eq!(s(at(sec, &["state"])), "ok", "{sec:?}"); + assert_eq!(s(at(sec, &["title"])), "ACMG secondary findings"); + assert_eq!( + s(at(sec, &["source"])), + "clinvar/S_clinvar_hits.vcf", + "the section's file is step 06's" + ); + let card = at(sec, &["data"]); + assert_eq!(at(card, &["genes_on_list"]), &Value::Int(84)); + let reported: Vec<(String, String)> = rows(card, "clinvar_hits") + .iter() + .map(|r| { + let rule = s(at(r, &["rule"])).to_owned(); + (s(at(r, &["gene"])).to_owned(), rule) + }) + .collect(); + assert_eq!( + reported, + [ + ("BRCA2".to_owned(), "All P and LP".to_owned()), + ("MUTYH".to_owned(), "P and LP (2 variants)".to_owned()), + ("MUTYH".to_owned(), "P and LP (2 variants)".to_owned()), + ], + "BRCA2 and the MUTYH pair are reported under their rules" + ); + assert!(rows(card, "not_reported").is_empty(), "{card:?}"); + assert_eq!(at(card, &["clinvar_checked"]), &Value::Bool(true)); + assert_eq!(at(card, &["high_impact_checked"]), &Value::Bool(false)); + let note = s(at(card, &["high_impact_note"])); + assert!( + note.starts_with( + "Rare HIGH-impact variants were not screened: step 23 (clinical filter) did not run" + ) && note.contains("not in ClinVar"), + "{note}" + ); + + // The run status gives step 23's reason; a failed step 23 is named. + let skip = + "skipped (needs VEP, data not installed: vep_cache/homo_sapiens/116_GRCh38/info.txt)"; + for (word, want) in [ + (skip, format!("step 23 (clinical filter) was {skip}")), + ( + "failed (exit 1)", + "step 23 (clinical filter) failed (exit 1)".to_owned(), + ), + ] { + put( + &sd.join("logs/run_status.tsv"), + &format!("meta\tstarted_utc\t2020-01-01 00:00 UTC\nstep\t06\tok\nstep\t23\t{word}\n"), + ); + let secs = sections_b(&sd, &assets); + let note = s(at(&secs, &["acmg_sf", "data", "high_impact_note"])).to_owned(); + assert!(note.contains(&want), "{note}"); + } +} + +#[test] +fn the_section_follows_step_06_and_is_never_a_not_run_line_of_its_own() { + let d = scratch("state"); + let sd = d.join("S"); + std::fs::create_dir_all(&sd).unwrap(); + let assets = repo().join("assets"); + let secs = sections_b(&sd, &assets); + assert_eq!(s(at(&secs, &["acmg_sf", "state"])), "missing"); + put( + &sd.join("logs/run_status.tsv"), + "meta\tstarted_utc\t2020-01-01 00:00 UTC\nstep\t06\tfailed (exit 2)\n", + ); + let secs = sections_b(&sd, &assets); + assert_eq!(s(at(&secs, &["acmg_sf", "state"])), "failed"); + assert!(s(at(&secs, &["acmg_sf", "note"])).starts_with("step 06 failed")); + // The whole summary: ClinVar screen names the failed step once. + let all = post::collect(&input(&sd, &assets, Group::All)).unwrap(); + let not_run: Vec<&str> = match at(&all, &["not_run"]) { + Value::Array(a) => a.iter().map(s).collect(), + x => panic!("{x:?}"), + }; + assert!( + not_run.contains(&"ClinVar screen (step 06 failed)"), + "{not_run:?}" + ); + assert!( + !not_run + .iter() + .any(|l| l.starts_with("ACMG secondary findings")), + "{not_run:?}" + ); +} + +/// The parity folders of group b, written by their generator. +fn parity_folders(d: &Path) -> PathBuf { + let script = repo().join("tests/parity/gen/summary-clinical.sh"); + let st = Command::new("bash") + .arg(&script) + .current_dir(d) + .env("NO_ORACLE", "1") + .status() + .expect("bash runs"); + assert!(st.success(), "{} failed", script.display()); + for f in ["cb_vep.vcf.gz", "cb_vep.vcf.gz.tbi"] { + std::fs::copy( + repo().join("tests/parity/inputs/summary-collect").join(f), + d.join(f), + ) + .unwrap(); + } + d.join("assets") +} + +/// Copies `from` to `to`, all but its `clinical/` folder (symlinks kept). +fn copy_without_clinical(from: &Path, to: &Path) { + let st = Command::new("cp") + .arg("-PR") + .arg(from) + .arg(to) + .status() + .unwrap(); + assert!(st.success()); + std::fs::remove_dir_all(to.join("clinical")).unwrap(); +} + +/// The three rules of the module doc, for one folder: the reasons it +/// breaks them, none when it keeps them. +fn agreement(with: &Value, without: &Value) -> Vec { + let mut bad = Vec::new(); + let high_genes: Vec = rows(with, "high_impact") + .iter() + .map(|r| gene_variant(r).0) + .collect(); + let with_cv = rows(with, "clinvar_hits"); + let without_cv = rows(without, "clinvar_hits"); + // (a) every ClinVar row without VEP, unchanged with VEP. + for r in &without_cv { + if !with_cv.contains(r) { + bad.push(format!( + "(a) ClinVar row without VEP not in the card with VEP: {r:?}" + )); + } + } + // (a') a ClinVar row only with VEP is one (c) explains. + let reported_with: Vec<(String, String)> = ["clinvar_hits", "high_impact", "structural"] + .iter() + .flat_map(|l| rows(with, l)) + .map(|r| gene_variant(&r)) + .collect(); + let not_without: Vec<(String, String)> = rows(without, "not_reported") + .iter() + .map(gene_variant) + .collect(); + for r in &with_cv { + if !without_cv.contains(r) { + let gv = gene_variant(r); + let moved = not_without.contains(&gv) && high_genes.contains(&gv.0); + // A row whose fields grew (a two-hit line) is the same row. + let same = without_cv.iter().any(|w| gene_variant(w) == gv); + if !(moved || same) { + bad.push(format!( + "(a) ClinVar row only with VEP, not explained: {r:?}" + )); + } + } + } + // (b) every gene reported without VEP is reported with VEP. + for (g, _) in ["clinvar_hits", "structural"] + .iter() + .flat_map(|l| rows(without, l)) + .map(|r| gene_variant(&r)) + { + if !reported_with.iter().any(|(h, _)| *h == g) { + bad.push(format!("(b) {g} reported without VEP, not with VEP")); + } + } + // (c) a not-reported row stays, or is reported because a HIGH-impact + // row of its gene joined it. + let not_with = rows(with, "not_reported"); + for r in rows(without, "not_reported") { + let gv = gene_variant(&r); + let stays = not_with.contains(&r); + let joined = reported_with.contains(&gv) && high_genes.contains(&gv.0); + if !(stays || joined) { + bad.push(format!( + "(c) not reported without VEP, neither kept nor joined: {r:?}" + )); + } + } + bad +} + +#[test] +fn without_the_clinical_vcf_the_card_loses_only_what_the_high_impact_source_brings() { + let d = scratch("agree"); + let assets = parity_folders(&d); + let mut compared = Vec::new(); + let mut moved = 0; + let mut names: Vec<_> = std::fs::read_dir(&d) + .unwrap() + .map(|e| e.unwrap().path()) + .filter(|p| p.join("clinvar/S_clinvar_hits.vcf").is_file() && p.join("clinical").is_dir()) + .collect(); + names.sort(); + for dir in names { + let name = dir.file_name().unwrap().to_str().unwrap().to_owned(); + let bare = d.join(format!("{name}.noclinical")); + copy_without_clinical(&dir, &bare); + let with = sections_b(&dir, &assets); + let without = sections_b(&bare, &assets); + let wsec = at(&with, &["acmg_sf"]); + let nsec = at(&without, &["acmg_sf"]); + if s(at(wsec, &["state"])) != "ok" { + // A hits VCF that cannot be read is the same note either way. + assert_eq!(wsec, nsec, "{name}"); + continue; + } + let (w, n) = (at(wsec, &["data"]), at(nsec, &["data"])); + assert_eq!( + at(n, &["high_impact_checked"]), + &Value::Bool(false), + "{name}" + ); + assert!( + s(at(n, &["high_impact_note"])).contains("did not run") + || s(at(n, &["high_impact_note"])).contains("wrote no clinical VCF"), + "{name}: {n:?}" + ); + if at(w, &["high_impact_checked"]) == &Value::Bool(false) { + // The clinical VCF is there and cannot be read: the card keeps + // its ClinVar rows and says why. + assert!( + s(at(w, &["high_impact_note"])).contains("clinical VCF cannot be read"), + "{name}: {w:?}" + ); + assert_eq!(rows(w, "clinvar_hits"), rows(n, "clinvar_hits"), "{name}"); + continue; + } + assert!(get(w, "high_impact_note").is_none(), "{name}: {w:?}"); + // The clinical section's own tier, with the same inputs, agrees. + assert_eq!( + Some(w), + get(at(&with, &["clinical", "data"]), "acmg_sf"), + "{name}: the card and the clinical section's tier differ" + ); + let bad = agreement(w, n); + assert!(bad.is_empty(), "{name}:\n{}", bad.join("\n")); + let not_with = rows(w, "not_reported"); + let n_moved = rows(n, "not_reported") + .iter() + .filter(|r| !not_with.contains(r)) + .count(); + eprintln!( + "{name}: ClinVar rows {} without VEP, {} with; not reported {} without, {} with; {n_moved} joined", + rows(n, "clinvar_hits").len(), + rows(w, "clinvar_hits").len(), + rows(n, "not_reported").len(), + not_with.len() + ); + moved += n_moved; + compared.push(name); + } + let _ = std::fs::remove_dir_all(&d); + assert!( + compared.len() >= 5, + "only {compared:?} compared: the generator lost its folders" + ); + assert!( + moved > 0, + "no row moved from not reported to reported with VEP: the folders no longer test rule (c)" + ); +} + +/// A clinical VCF left by an earlier run (step 23 not ok in this run, the +/// VCF written before the run started) is still read, and the card says +/// its HIGH-impact source is from that earlier run. With step 23 ok in +/// this run, or a VCF of this run, the card has no such note. +#[test] +fn a_clinical_vcf_of_an_earlier_run_is_named_in_the_card() { + let d = scratch("earlier"); + let assets = parity_folders(&d); + let mut names: Vec<_> = std::fs::read_dir(&d) + .unwrap() + .map(|e| e.unwrap().path()) + .filter(|p| p.join("clinvar/S_clinvar_hits.vcf").is_file() && p.join("clinical").is_dir()) + .collect(); + names.sort(); + let dir = names + .into_iter() + .find(|p| { + let secs = sections_b(p, &assets); + s(at(&secs, &["acmg_sf", "state"])) == "ok" + && at(&secs, &["acmg_sf", "data", "high_impact_checked"]) == &Value::Bool(true) + }) + .expect("a parity folder with a readable clinical VCF"); + let now = std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .unwrap() + .as_secs(); + let status = |started: u64, step23: &str| { + put( + &dir.join("logs/run_status.tsv"), + &format!("meta\tstarted_epoch\t{started}\nstep\t06\tok\nstep\t23\t{step23}\n"), + ); + sections_b(&dir, &assets) + }; + // The run started after the VCF was written, and step 23 failed in it. + let secs = status(now + 3600, "failed (exit 1)"); + let card = at(&secs, &["acmg_sf", "data"]); + assert_eq!(at(card, &["high_impact_checked"]), &Value::Bool(true)); + let note = s(at(card, &["high_impact_note"])); + assert!( + note.starts_with( + "Rare HIGH-impact variants come from the clinical VCF of an earlier run (" + ) && note.ends_with("step 23 (clinical filter) was failed (exit 1) in this run."), + "{note}" + ); + assert_eq!(s(at(&secs, &["clinical", "state"])), "stale"); + // Step 23 ok in this run, or a VCF written after the run started (the + // generator dates its files in 2021, so a run of 1970 is before them): + // no note. + for (started, step23) in [(now + 3600, "ok"), (1, "failed (exit 1)")] { + let secs = status(started, step23); + assert!( + get(at(&secs, &["acmg_sf", "data"]), "high_impact_note").is_none(), + "{started} {step23}: {secs:?}" + ); + } + let _ = std::fs::remove_dir_all(&d); +} diff --git a/tests/e2e/acmg-sf-rules-1-reports.sh b/tests/e2e/acmg-sf-rules-1-reports.sh index 7660081d8..3ce2098aa 100755 --- a/tests/e2e/acmg-sf-rules-1-reports.sh +++ b/tests/e2e/acmg-sf-rules-1-reports.sh @@ -9,6 +9,13 @@ # or "not made" line. Control: the same render without the assets # mount (what generate-report.sh did before it mounted assets/) fails # the same checks. +# 1b. No step 23 (a run without the VEP cache): og summary collect on the +# sample's step 06, 05 and 35 outputs with no clinical/ folder still +# writes the ACMG card (sections.acmg_sf) with the 84 rules, says the +# HIGH-impact source was not screened, and keeps every ClinVar row the +# full run's card has. Control: without step 06's hits the section is +# not run. The reports still read the card from the clinical section +# until they move to sections.acmg_sf. # 2. Nextflow: HTML_REPORT on the sample's ClinVar hits and clinical VCF, # with the two assets staged as main.nf stages them, applies the rules, # and its carrier list equals the bash one. Control: the same task @@ -75,6 +82,37 @@ check_eq "control: the text check fails without assets/" "$CT" red if [ -s "${CTL}/summary.json" ] && ! summary_ok "${CTL}/summary.json" > /dev/null; then CS=red; else CS=green; fi check_eq "control: the summary check fails without assets/" "$CS" red +# --- 1b. bash, no step 23 ---------------------------------------------------------- +# card JSON: the ACMG card of a summary.json, or null. +card() { jq -c '.sections.acmg_sf' "$1" 2> /dev/null; } +check "the full run's card is ok and screened the HIGH-impact records" \ + jq -e '.sections.acmg_sf | .state == "ok" and .data.genes_on_list == 84 and .data.high_impact_checked == true' "$SUM" +NC="${CASE_TMP}/no-step23" +rm -rf "$NC" +mkdir -p "${NC}/${SAMPLE}" +for d in clinvar annotsv sv_merged paralogs; do + [ ! -d "${SD}/${d}" ] || cp -R "${SD}/${d}" "${NC}/${SAMPLE}/" +done +"${REPO}/bin/og" summary collect --sample "$SAMPLE" --sample-dir "${NC}/${SAMPLE}" --assets "${REPO}/assets" \ + --out "${NC}/summary.json" > "${NC}/collect.log" 2>&1 +check_eq "og summary collect without clinical/ exits 0" "$?" 0 +echo " card without step 23: $(card "${NC}/summary.json" | cut -c1-300)" +check_eq "without step 23 the clinical section is not run" \ + "$(jq -r '.sections.clinical.state' "${NC}/summary.json" 2> /dev/null)" missing +check "without step 23 the card is ok with the 84 rules" \ + jq -e '.sections.acmg_sf | .state == "ok" and .data.genes_on_list == 84 and (.data | has("unavailable") | not)' "${NC}/summary.json" +check "and says the HIGH-impact source was not screened" \ + jq -e '.sections.acmg_sf.data | .high_impact_checked == false and (.high_impact_note | startswith("Rare HIGH-impact variants were not screened"))' "${NC}/summary.json" +# shellcheck disable=SC2016 # a jq program: $f and $n are jq variables +check "every ClinVar row of the card without step 23 is in the full run's card, unchanged" \ + jq -en --slurpfile f "$SUM" --slurpfile n "${NC}/summary.json" \ + '($n[0].sections.acmg_sf.data.clinvar_hits - $f[0].sections.acmg_sf.data.clinvar_hits) | length == 0' +rm -rf "${NC}/${SAMPLE}/clinvar" +"${REPO}/bin/og" summary collect --sample "$SAMPLE" --sample-dir "${NC}/${SAMPLE}" --assets "${REPO}/assets" \ + --out "${NC}/control.json" > /dev/null 2>&1 +check_eq "control: without step 06's hits the card is not run" \ + "$(jq -r '.sections.acmg_sf.state' "${NC}/control.json" 2> /dev/null)" missing + # --- 2. Nextflow -------------------------------------------------------------------- if ! command -v nextflow >/dev/null; then fail "nextflow is not on PATH"; finish; fi NP="${CASE_TMP}/project" diff --git a/tests/e2e/nf-wiring-1-report-tables.sh b/tests/e2e/nf-wiring-1-report-tables.sh index 3a5abadb5..96625a5f4 100755 --- a/tests/e2e/nf-wiring-1-report-tables.sh +++ b/tests/e2e/nf-wiring-1-report-tables.sh @@ -11,8 +11,9 @@ # hits, a VEP file with a gnomAD AF on each hit (3% on the planted one), # CPSR's table and its cDNA read check, step 02's read count and the # clinical filter's and slivar's summary tables. The read_count, clinvar, -# cpsr, clinical and slivar sections of the two summary.json files must -# be equal, with the AF, the CPSR classes and the flagged row in them. +# cpsr, clinical, slivar and acmg_sf sections of the two summary.json +# files must be equal, with the AF, the CPSR classes and the flagged row +# in them. # Control: the module as it was (no place for these tables) stops. # 3. main.nf passes them on: a stub run of every tool (-profile test_all) # stages each table in each HTML_REPORT task whose sample has the step's @@ -203,6 +204,11 @@ sys.exit(bad) PY } check "the five sections of the Nextflow summary equal the bash ones" same_sections "${SD}/summary.json" "$NF_SUM" +# The ACMG card (step 06's section) reads the same hits, clinical VCF and deletions in both. +# shellcheck disable=SC2016 # a jq program: $b, $n and $x are jq variables +check "the ACMG card of the Nextflow summary equals the bash one" \ + jq -en --slurpfile b "${SD}/summary.json" --slurpfile n "$NF_SUM" \ + '($b[0].sections.acmg_sf | del(.file_date)) as $x | $x.state == "ok" and $x == ($n[0].sections.acmg_sf | del(.file_date))' nf_py() { python3 -c "import json, sys; d = json.load(open(sys.argv[1]))['sections']; print($1)" "$NF_SUM" 2>/dev/null; } check_eq "the ClinVar hits' frequencies come from the VEP file" \ "$(nf_py "d['clinvar']['data']['frequency']['source']")" "vep/${S}_vep.vcf.gz" diff --git a/tests/parity/gen/summary-clinical.sh b/tests/parity/gen/summary-clinical.sh index 6d5cfbac3..2d0f7907e 100644 --- a/tests/parity/gen/summary-clinical.sh +++ b/tests/parity/gen/summary-clinical.sh @@ -31,14 +31,17 @@ NEW=202103040506.07 # after it NEWER=202103040507.00 # The helper modules of bin/collect_summary.py, as they were when the -# collector was last the summary's writer. +# collector was last the summary's writer. NO_ORACLE=1 leaves them out: +# og-summary's tests read the folders alone (tests/screen_acmg_card_section.rs). HELPERS=7c1470bc50933f48a1a0dde3c0e0b2b87bc48718 -git -C "$root" cat-file -e "${HELPERS}^{commit}" 2> /dev/null \ - || git -C "$root" fetch -q --depth=1 origin "$HELPERS" -mkdir -p pymods -for m in pgx_outside_calls clip_check annotsv_postfilter paraphase_report traits_report; do - git -C "$root" show "${HELPERS}:bin/${m}.py" > "pymods/${m}.py" -done +if [ "${NO_ORACLE:-}" != 1 ]; then + git -C "$root" cat-file -e "${HELPERS}^{commit}" 2> /dev/null \ + || git -C "$root" fetch -q --depth=1 origin "$HELPERS" + mkdir -p pymods + for m in pgx_outside_calls clip_check annotsv_postfilter paraphase_report traits_report; do + git -C "$root" show "${HELPERS}:bin/${m}.py" > "pymods/${m}.py" + done +fi mkdir -p assets for a in acmg_sf.tsv carrier_genes.tsv paralog_loci.tsv; do diff --git a/tests/parity/golden/summary-collect/all-bad/all-bad.json b/tests/parity/golden/summary-collect/all-bad/all-bad.json index 24bc28580..13c65ca8c 100644 --- a/tests/parity/golden/summary-collect/all-bad/all-bad.json +++ b/tests/parity/golden/summary-collect/all-bad/all-bad.json @@ -1,759 +1,768 @@ { - "schema_version": 2, - "sample": "S", - "generated_utc": "2026-11-15 00:19 UTC", - "run": { - "status_file": false, - "started_utc": null, - "declared_sex": null - }, - "databases": { - "clinvar_release": null, - "cpsr_clinvar_release": null, - "hla_database": null - }, - "manifest": [], - "sections": { - "coverage": { - "title": "Coverage (mosdepth)", - "step": "16b", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "ValueError: could not convert string to float: 'abc'", - "data": {} - }, - "callability": { - "title": "Gene callability (mosdepth)", - "step": "16b", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "S_callability.bed.gz is not mosdepth's thresholds table of one depth", - "data": {} - }, - "sex_check": { - "title": "Sex check (indexcov)", - "step": "16", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "indexcov-indexcov.ped has no sex column or no sample row", - "data": {} - }, - "sample_qc": { - "title": "Sample identity and contamination (somalier, VerifyBamID2)", - "step": "33", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "S_sample_qc.tsv has no inferred_sex row", - "data": {} - }, - "mosaic_sex_loss": { - "title": "Mosaic loss of chromosome Y or X", - "step": "33", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "S_sample_qc.tsv has no mosaic_call row (detected, not_detected, gain, not_estimated)", - "data": {} - }, - "read_count": { - "title": "Reads kept by alignment", - "step": "02", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "S_read_count.tsv has no whole number for input_reads", - "data": {} - }, - "variants": { - "title": "Variant calling", - "step": "03", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "S.vcf.gz: Compressed file ended before the end-of-stream marker was reached", - "data": {} - }, - "clinvar": { - "title": "ClinVar screen", - "step": "06", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "ValueError: invalid literal for int() with base 10: 'x'", - "data": {} - }, - "couple": { - "title": "Couple carrier screen", - "step": "39", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "S_couple_carrier.tsv line 2 is not a row of S's couple table", - "data": {} - }, - "carrier": { - "title": "Carrier list", - "step": "06", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "ValueError: invalid literal for int() with base 10: 'x'", - "data": {} - }, - "pharmcat": { - "title": "PharmCAT", - "step": "07", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "S.report.json: Expecting value: line 1 column 21 (char 20)", - "data": {} - }, - "cpic": { - "title": "CPIC drug-gene recommendations", - "step": "27", - "state": "ok", - "source": "cpic/S_phenotypes.tsv", - "file_date": "2021-03-04 05:06 UTC", - "note": null, - "data": { - "genes": [], - "non_normal": 0, - "ambiguous": 0, - "not_called": 0, - "unclassified": 0, - "parse_failed": true, - "warnings": [] - } - }, - "pypgx": { - "title": "pypgx", - "step": "32", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "KeyError: 'Gene'", - "data": {} - }, - "cyrius": { - "title": "Cyrius (CYP2D6)", - "step": "21", - "state": "ok", - "source": "cyrius/S_cyp2d6.tsv", - "file_date": "2021-03-04 05:06 UTC", - "note": null, - "data": { - "genotype": "none", - "filter": "" - } - }, - "parascopy": { - "title": "SMN1/SMN2 copy number (Parascopy, short reads)", - "step": "35", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "S_smn_copy_number.tsv lacks the chrom, start, end, agCN, agCN_filter, agCN_qual, psCN, psCN_qual or psCN_filter column", - "data": {} - }, - "paralogs": { - "title": "Paralog genes (Paraphase, PacBio HiFi)", - "step": "35b", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "S_paraphase_copy_number.tsv lacks the region, gene, copy_number or status column", - "data": {} - }, - "hla": { - "title": "HLA typing (T1K)", - "step": "08", - "state": "ok", - "source": "hla_t1k/S_hla_genotype.tsv", - "file_date": "2021-03-04 05:06 UTC", - "note": null, - "data": { - "loci": [ - { - "gene": "HLA-A", - "alleles": [ - "A*02:01" - ], - "quality": [ - "x" - ], - "low_confidence": true, - "withheld_from_pharmcat": true, - "candidates": [ - 1 - ], - "one_allele_type": false - } - ], - "derived": { - "coeliac": { - "status": "not_determined", - "reason": "HLA-DQA1 not typed; HLA-DQB1 not typed", - "dq2_5": null, - "dq2_5_phase": null, - "dq2_2": null, - "dq8": null, - "dqb1_02_dose": null - }, - "alleles": [ - { - "allele": "A*31:01", - "kind": "drug", - "association": "carbamazepine hypersensitivity reactions in Europeans", - "pmids": "21428769", - "citation": "McCormack 2011, N Engl J Med", - "state": "not_determined", - "reason": "HLA-A low confidence" - }, - { - "allele": "B*15:02", - "kind": "drug", - "association": "carbamazepine Stevens-Johnson syndrome in Han Chinese", - "pmids": "15057820", - "citation": "Chung 2004, Nature", - "state": "not_determined", - "reason": "HLA-B not typed" - }, - { - "allele": "B*57:01", - "kind": "drug", - "association": "abacavir hypersensitivity; flucloxacillin liver injury", - "pmids": "18256392,19483685", - "citation": "Mallal 2008, N Engl J Med; Daly 2009, Nat Genet", - "state": "not_determined", - "reason": "HLA-B not typed" - }, - { - "allele": "B*58:01", - "kind": "drug", - "association": "allopurinol severe skin reactions", - "pmids": "15743917", - "citation": "Hung 2005, Proc Natl Acad Sci U S A", - "state": "not_determined", - "reason": "HLA-B not typed" - }, - { - "allele": "B*27", - "kind": "disease", - "association": "ankylosing spondylitis", - "pmids": "4123836,4688372", - "citation": "Brewerton 1973, Lancet; Schlosstein 1973, N Engl J Med", - "state": "not_determined", - "reason": "HLA-B not typed" - }, - { - "allele": "C*06:02", - "kind": "disease", - "association": "psoriasis", - "pmids": "16642438", - "citation": "Nair 2006, Am J Hum Genet", - "state": "not_determined", - "reason": "HLA-C not typed" - }, - { - "allele": "DQB1*06:02", - "kind": "disease", - "association": "narcolepsy with cataplexy", - "pmids": "11179016", - "citation": "Mignot 2001, Am J Hum Genet", - "state": "not_determined", - "reason": "HLA-DQB1 not typed" - }, - { - "allele": "DRB1*15:01", - "kind": "disease", - "association": "multiple sclerosis", - "pmids": "12557126", - "citation": "Barcellos 2003, Am J Hum Genet", - "state": "not_determined", - "reason": "HLA-DRB1 not typed" - }, - { - "allele": "DRB1*01:01", - "kind": "disease", - "association": "rheumatoid arthritis, shared epitope allele", - "pmids": "2446635", - "citation": "Gregersen 1987, Arthritis Rheum", - "state": "not_determined", - "reason": "HLA-DRB1 not typed" - }, - { - "allele": "DRB1*04:01", - "kind": "disease", - "association": "rheumatoid arthritis, shared epitope allele", - "pmids": "2446635", - "citation": "Gregersen 1987, Arthritis Rheum", - "state": "not_determined", - "reason": "HLA-DRB1 not typed" - }, - { - "allele": "DRB1*04:04", - "kind": "disease", - "association": "rheumatoid arthritis, shared epitope allele", - "pmids": "2446635", - "citation": "Gregersen 1987, Arthritis Rheum", - "state": "not_determined", - "reason": "HLA-DRB1 not typed" - }, - { - "allele": "DRB1*04:05", - "kind": "disease", - "association": "rheumatoid arthritis, shared epitope allele", - "pmids": "2446635", - "citation": "Gregersen 1987, Arthritis Rheum", - "state": "not_determined", - "reason": "HLA-DRB1 not typed" - }, - { - "allele": "DRB1*04:08", - "kind": "disease", - "association": "rheumatoid arthritis, shared epitope allele", - "pmids": "2446635", - "citation": "Gregersen 1987, Arthritis Rheum", - "state": "not_determined", - "reason": "HLA-DRB1 not typed" - }, - { - "allele": "DRB1*14:02", - "kind": "disease", - "association": "rheumatoid arthritis, shared epitope allele", - "pmids": "2446635", - "citation": "Gregersen 1987, Arthritis Rheum", - "state": "not_determined", - "reason": "HLA-DRB1 not typed" - } - ] - } - } - }, - "prs": { - "title": "Polygenic risk scores", - "step": "25", - "state": "missing", - "source": null, - "file_date": null, - "note": null, - "data": {} - }, - "ancestry_fine": { - "title": "Fine-grained ancestry (ADMIXTURE, RFMix)", - "step": "26b", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "S_ancestry_fine.tsv has no rows or lacks a column of ['code', 'fraction', 'group', 'high', 'level', 'low', 'se']", - "data": {} - }, - "manta": { - "title": "Structural variants (Manta)", - "step": "04", - "state": "missing", - "source": null, - "file_date": null, - "note": null, - "data": {} - }, - "delly": { - "title": "Structural variants (Delly)", - "step": "19", - "state": "missing", - "source": null, - "file_date": null, - "note": null, - "data": {} - }, - "cnvpytor": { - "title": "Copy number (CNVpytor)", - "step": "18", - "state": "missing", - "source": null, - "file_date": null, - "note": null, - "data": {} - }, - "sv_consensus": { - "title": "SV consensus merge", - "step": "22", - "state": "missing", - "source": null, - "file_date": null, - "note": null, - "data": {} - }, - "annotsv": { - "title": "Structural variants, class 4 and 5 (AnnotSV)", - "step": "05", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "S_sv_class4_5_checked.tsv lacks a column of ['ACMG_class', 'Caller', 'Common', 'Exon_edge', 'Population_AF', 'SV_chrom', 'SV_end', 'SV_start', 'SV_type']", - "data": {} - }, - "expansions": { - "title": "Repeat expansions (ExpansionHunter)", - "step": "09", - "state": "ok", - "source": "expansion_hunter/S_eh.vcf", - "file_date": "2021-03-04 05:06 UTC", - "note": null, - "data": { - "records": 2, - "key_loci": [ - { - "locus": "HTT", - "repeat_count": "not in output" - }, - { - "locus": "FMR1", - "repeat_count": "not in output" - }, - { - "locus": "C9ORF72", - "repeat_count": "not in output" - }, - { - "locus": "ATXN1", - "repeat_count": "not in output" - }, - { - "locus": "DMPK", - "repeat_count": "not in output" - } - ], - "loci": [ - { - "locus": "CFTR_T", - "repeat_count": "7/9", - "status": "", - "normal_max": "", - "pathologic_min": "", - "support": "SPANNING/SPANNING", - "adsp": "8/9", - "ci": "7-7/9-9", - "sizing": "sized", - "allele_sizing": "sized/sized", - "zone": "", - "zone_label": "", - "zone_citation": "" - }, - { - "locus": "CFTR_TG", - "repeat_count": "11/12", - "status": "", - "normal_max": "", - "pathologic_min": "", - "support": "SPANNING/FLANKING", - "adsp": "8/0", - "ci": "11-11/12-12", - "sizing": "flanking_only", - "allele_sizing": "sized/flanking_only", - "zone": "", - "zone_label": "", - "zone_citation": "" - } - ], - "stranger": false, - "stranger_outdated": false, - "unsized": [ - "CFTR_TG" - ], - "intermediate": [], - "readings": [ - { - "locus": "CFTR", - "repeat": "intron 9 (legacy IVS8) TG and poly-T tract", - "source": "Deignan et al. 2020, PMID 32404922", - "status": "not_sized", - "genotype": ".", - "phase": ".", - "reading": "ExpansionHunter did not size both tracts (TG 11/12, flanking_only; poly-T 7/9, sized): not read." - }, - { - "locus": "UGT1A1", - "repeat": "promoter TA repeat", - "source": "Gammal et al. 2016, PMID 26417955", - "status": "not_sized", - "genotype": ".", - "pharmcat": ".", - "agreement": "not_compared", - "reading": "not in step 09's output: STR_EXTRA_LOCI=yes ./scripts/setup.sh --strchive-data adds the repeat to the catalog (docs/09-str-expansions.md). PharmCAT (step 7) still reads UGT1A1 *28." - }, - { - "locus": "SLC6A4", - "repeat": "5-HTTLPR promoter length polymorphism", - "status": "not_tested", - "genotype": ".", - "reading": "not tested by any step. CPIC's 2023 SSRI guideline (Bousman et al. 2023, PMID 37032427) found that the data on SLC6A4 do not support its use in antidepressant prescribing, so a result would carry no recommendation.", - "source": "Bousman et al. 2023, PMID 37032427" - } - ] - } - }, - "telomere": { - "title": "Telomere content (TelomereHunter)", - "step": "10", - "state": "ok", - "source": "telomere/S/S_summary.tsv", - "file_date": "2021-03-04 05:06 UTC", - "note": null, - "data": { - "tel_content": "1.5", - "intratelomeric_reads": 40, - "read_check": "depth unknown (no total_reads or read_lengths in the summary): not checked" - } - }, - "roh": { - "title": "Runs of homozygosity", - "step": "11", - "state": "ok", - "source": "vcf/S_roh.txt", - "file_date": "2021-03-04 05:06 UTC", - "note": null, - "data": { - "segments": 0, - "total_mb": 0.0, - "largest_mb": 0.0, - "autosomal_over_5mb": [], - "masked_mb": 0.0, - "sex_chromosome_segments": 0, - "long_mb": 0.0, - "f_roh": null, - "relationship": null, - "upd_flag": null, - "scope": { - "source": "segments_only", - "autosomes": [], - "unmasked_bp": 0, - "partial": true, - "note": "Partial or unverified chromosome scope: no relationship band or UPD flag. F_ROH uses only the listed chromosomes, not callable bases." - } + "schema_version": 2, + "sample": "S", + "generated_utc": "2026-11-19 23:58 UTC", + "run": { + "status_file": false, + "started_utc": null, + "declared_sex": null + }, + "databases": { + "clinvar_release": null, + "cpsr_clinvar_release": null, + "hla_database": null + }, + "manifest": [], + "sections": { + "coverage": { + "title": "Coverage (mosdepth)", + "step": "16b", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "ValueError: could not convert string to float: 'abc'", + "data": {} + }, + "callability": { + "title": "Gene callability (mosdepth)", + "step": "16b", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "S_callability.bed.gz is not mosdepth's thresholds table of one depth", + "data": {} + }, + "sex_check": { + "title": "Sex check (indexcov)", + "step": "16", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "indexcov-indexcov.ped has no sex column or no sample row", + "data": {} + }, + "sample_qc": { + "title": "Sample identity and contamination (somalier, VerifyBamID2)", + "step": "33", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "S_sample_qc.tsv has no inferred_sex row", + "data": {} + }, + "mosaic_sex_loss": { + "title": "Mosaic loss of chromosome Y or X", + "step": "33", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "S_sample_qc.tsv has no mosaic_call row (detected, not_detected, gain, not_estimated)", + "data": {} + }, + "read_count": { + "title": "Reads kept by alignment", + "step": "02", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "S_read_count.tsv has no whole number for input_reads", + "data": {} + }, + "variants": { + "title": "Variant calling", + "step": "03", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "S.vcf.gz: Compressed file ended before the end-of-stream marker was reached", + "data": {} + }, + "clinvar": { + "title": "ClinVar screen", + "step": "06", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "ValueError: invalid literal for int() with base 10: 'x'", + "data": {} + }, + "couple": { + "title": "Couple carrier screen", + "step": "39", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "S_couple_carrier.tsv line 2 is not a row of S's couple table", + "data": {} + }, + "carrier": { + "title": "Carrier list", + "step": "06", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "ValueError: invalid literal for int() with base 10: 'x'", + "data": {} + }, + "acmg_sf": { + "title": "ACMG secondary findings", + "step": "06", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "ValueError: invalid literal for int() with base 10: 'x'", + "data": {} + }, + "pharmcat": { + "title": "PharmCAT", + "step": "07", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "S.report.json: Expecting value: line 1 column 21 (char 20)", + "data": {} + }, + "cpic": { + "title": "CPIC drug-gene recommendations", + "step": "27", + "state": "ok", + "source": "cpic/S_phenotypes.tsv", + "file_date": "2021-03-04 05:06 UTC", + "note": null, + "data": { + "genes": [], + "non_normal": 0, + "ambiguous": 0, + "not_called": 0, + "unclassified": 0, + "parse_failed": true, + "warnings": [] + } + }, + "pypgx": { + "title": "pypgx", + "step": "32", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "KeyError: 'Gene'", + "data": {} + }, + "cyrius": { + "title": "Cyrius (CYP2D6)", + "step": "21", + "state": "ok", + "source": "cyrius/S_cyp2d6.tsv", + "file_date": "2021-03-04 05:06 UTC", + "note": null, + "data": { + "genotype": "none", + "filter": "" + } + }, + "parascopy": { + "title": "SMN1/SMN2 copy number (Parascopy, short reads)", + "step": "35", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "S_smn_copy_number.tsv lacks the chrom, start, end, agCN, agCN_filter, agCN_qual, psCN, psCN_qual or psCN_filter column", + "data": {} + }, + "paralogs": { + "title": "Paralog genes (Paraphase, PacBio HiFi)", + "step": "35b", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "S_paraphase_copy_number.tsv lacks the region, gene, copy_number or status column", + "data": {} + }, + "hla": { + "title": "HLA typing (T1K)", + "step": "08", + "state": "ok", + "source": "hla_t1k/S_hla_genotype.tsv", + "file_date": "2021-03-04 05:06 UTC", + "note": null, + "data": { + "loci": [ + { + "gene": "HLA-A", + "alleles": [ + "A*02:01" + ], + "quality": [ + "x" + ], + "low_confidence": true, + "withheld_from_pharmcat": true, + "candidates": [ + 1 + ], + "one_allele_type": false + } + ], + "derived": { + "coeliac": { + "status": "not_determined", + "reason": "HLA-DQA1 not typed; HLA-DQB1 not typed", + "dq2_5": null, + "dq2_5_phase": null, + "dq2_2": null, + "dq8": null, + "dqb1_02_dose": null + }, + "alleles": [ + { + "allele": "A*31:01", + "kind": "drug", + "association": "carbamazepine hypersensitivity reactions in Europeans", + "pmids": "21428769", + "citation": "McCormack 2011, N Engl J Med", + "state": "not_determined", + "reason": "HLA-A low confidence" + }, + { + "allele": "B*15:02", + "kind": "drug", + "association": "carbamazepine Stevens-Johnson syndrome in Han Chinese", + "pmids": "15057820", + "citation": "Chung 2004, Nature", + "state": "not_determined", + "reason": "HLA-B not typed" + }, + { + "allele": "B*57:01", + "kind": "drug", + "association": "abacavir hypersensitivity; flucloxacillin liver injury", + "pmids": "18256392,19483685", + "citation": "Mallal 2008, N Engl J Med; Daly 2009, Nat Genet", + "state": "not_determined", + "reason": "HLA-B not typed" + }, + { + "allele": "B*58:01", + "kind": "drug", + "association": "allopurinol severe skin reactions", + "pmids": "15743917", + "citation": "Hung 2005, Proc Natl Acad Sci U S A", + "state": "not_determined", + "reason": "HLA-B not typed" + }, + { + "allele": "B*27", + "kind": "disease", + "association": "ankylosing spondylitis", + "pmids": "4123836,4688372", + "citation": "Brewerton 1973, Lancet; Schlosstein 1973, N Engl J Med", + "state": "not_determined", + "reason": "HLA-B not typed" + }, + { + "allele": "C*06:02", + "kind": "disease", + "association": "psoriasis", + "pmids": "16642438", + "citation": "Nair 2006, Am J Hum Genet", + "state": "not_determined", + "reason": "HLA-C not typed" + }, + { + "allele": "DQB1*06:02", + "kind": "disease", + "association": "narcolepsy with cataplexy", + "pmids": "11179016", + "citation": "Mignot 2001, Am J Hum Genet", + "state": "not_determined", + "reason": "HLA-DQB1 not typed" + }, + { + "allele": "DRB1*15:01", + "kind": "disease", + "association": "multiple sclerosis", + "pmids": "12557126", + "citation": "Barcellos 2003, Am J Hum Genet", + "state": "not_determined", + "reason": "HLA-DRB1 not typed" + }, + { + "allele": "DRB1*01:01", + "kind": "disease", + "association": "rheumatoid arthritis, shared epitope allele", + "pmids": "2446635", + "citation": "Gregersen 1987, Arthritis Rheum", + "state": "not_determined", + "reason": "HLA-DRB1 not typed" + }, + { + "allele": "DRB1*04:01", + "kind": "disease", + "association": "rheumatoid arthritis, shared epitope allele", + "pmids": "2446635", + "citation": "Gregersen 1987, Arthritis Rheum", + "state": "not_determined", + "reason": "HLA-DRB1 not typed" + }, + { + "allele": "DRB1*04:04", + "kind": "disease", + "association": "rheumatoid arthritis, shared epitope allele", + "pmids": "2446635", + "citation": "Gregersen 1987, Arthritis Rheum", + "state": "not_determined", + "reason": "HLA-DRB1 not typed" + }, + { + "allele": "DRB1*04:05", + "kind": "disease", + "association": "rheumatoid arthritis, shared epitope allele", + "pmids": "2446635", + "citation": "Gregersen 1987, Arthritis Rheum", + "state": "not_determined", + "reason": "HLA-DRB1 not typed" + }, + { + "allele": "DRB1*04:08", + "kind": "disease", + "association": "rheumatoid arthritis, shared epitope allele", + "pmids": "2446635", + "citation": "Gregersen 1987, Arthritis Rheum", + "state": "not_determined", + "reason": "HLA-DRB1 not typed" + }, + { + "allele": "DRB1*14:02", + "kind": "disease", + "association": "rheumatoid arthritis, shared epitope allele", + "pmids": "2446635", + "citation": "Gregersen 1987, Arthritis Rheum", + "state": "not_determined", + "reason": "HLA-DRB1 not typed" } - }, - "haplogroup": { - "title": "Mitochondrial haplogroup", - "step": "12", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "S_haplocheck.txt has no Contamination Status column", - "data": {} - }, - "y_haplogroup": { - "title": "Y-chromosome haplogroup (Yleaf)", - "step": "37", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "S_y_haplogroup.txt has no Hg column or no sample row", - "data": {} - }, - "y_tree": { - "title": "Y placement on the current public tree", - "step": "37", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "S_y_tree.txt lacks a column of Hg, Level, Tree", - "data": {} - }, - "y_str": { - "title": "Y-STR markers (HipSTR)", - "step": "45", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "S_y_str.tsv has a confidence other than high, low, no call or not callable", - "data": {} - }, - "consumer_export": { - "title": "Raw-data export (23andMe v5 layout)", - "step": "40", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "S_consumer_export.tsv has no whole-number 'sites' row", - "data": {} - }, - "bloodtype": { - "title": "Blood groups (predicted)", - "step": "41", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "S_bloodtype.tsv has no system, phenotype and confidence columns, or no row", - "data": {} - }, - "lpa": { - "title": "Lp(a): LPA KIV-2 repeat count", - "step": "46", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "S_lpa_kiv2.tsv has no status row (estimated, too little depth or not assessed)", - "data": {} - }, - "vntyper": { - "title": "MUC1 VNTR frameshifts (VNtyper)", - "step": "52", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "S_vntyper.tsv has no status row (positive, possible, negative, not assessed)", - "data": {} - }, - "locityper": { - "title": "Locityper second opinion at hard loci", - "step": "53", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "S_locityper.tsv lacks a row or one of the columns locus, region, status, genotype, haplotype_1, haplotype_2, quality, total_reads, unexplained_reads, weighted_distance, warnings, filter, reason", - "data": {} - }, - "mito": { - "title": "Mitochondrial variants", - "step": "20", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "KeyError: 'length'", - "data": {} - }, - "traits": { - "title": "Traits panel", - "step": "42", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "S_traits.tsv line 2 does not have one field per column", - "data": {} - }, - "archaic": { - "title": "Archaic ancestry (IBDmix)", - "step": "43", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "S_archaic.tsv has no percentage for neanderthal_percent", - "data": {} - }, - "ancient": { - "title": "Ancient populations (AADR PCA)", - "step": "44", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "S_ancient.tsv has no rows or lacks a column of ['country', 'dates', 'distance', 'group', 'locality', 'n', 'rank']", - "data": {} - }, - "viral": { - "title": "Viral DNA in the unmapped reads", - "step": "47", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "S_viral.tsv has no rows or lacks a column of virus, accession, reads, breadth, relative_depth, copies_per_cell, call", - "data": {} - }, - "exomiser": { - "title": "Symptom-driven ranking (Exomiser)", - "step": "48", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "S_exomiser_top.tsv lacks a column of rank, gene, moi, exomiser_score, phenotype_score, variant, genotype, consequence, hgvs, acmg, disease", - "data": {} - }, - "chip": { - "title": "Clonal haematopoiesis (CHIP)", - "step": "49", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "S_chip.tsv lacks a column of gene, protein_change, vaf, alt_reads, depth, status, note", - "data": {} - }, - "mei": { - "title": "Mobile element insertions (MEGAnE)", - "step": "50", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "S_mei.tsv is missing beside S_mei_sites.tsv", - "data": {} - }, - "mosaic": { - "title": "Mosaic chromosomal alterations (MoChA)", - "step": "51", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "S_mocha.tsv has 1 reported calls, S_mocha_summary.tsv says 2", - "data": {} - }, - "cpsr": { - "title": "Cancer predisposition (CPSR)", - "step": "17", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "S.cpsr.grch38.classification.tsv.gz: Compressed file ended before the end-of-stream marker was reached", - "data": {} - }, - "clinical": { - "title": "Clinical variant filter", - "step": "23", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "ValueError: invalid literal for int() with base 10: 'x'", - "data": {} - }, - "slivar": { - "title": "Variant prioritization (slivar)", - "step": "31", - "state": "unreadable", - "source": null, - "file_date": null, - "note": "S_prioritized.vcf.gz: Compressed file ended before the end-of-stream marker was reached", - "data": {} + ] + } + } + }, + "prs": { + "title": "Polygenic risk scores", + "step": "25", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "ancestry_fine": { + "title": "Fine-grained ancestry (ADMIXTURE, RFMix)", + "step": "26b", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "S_ancestry_fine.tsv has no rows or lacks a column of ['code', 'fraction', 'group', 'high', 'level', 'low', 'se']", + "data": {} + }, + "manta": { + "title": "Structural variants (Manta)", + "step": "04", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "delly": { + "title": "Structural variants (Delly)", + "step": "19", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "cnvpytor": { + "title": "Copy number (CNVpytor)", + "step": "18", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "sv_consensus": { + "title": "SV consensus merge", + "step": "22", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "annotsv": { + "title": "Structural variants, class 4 and 5 (AnnotSV)", + "step": "05", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "S_sv_class4_5_checked.tsv lacks a column of ['ACMG_class', 'Caller', 'Common', 'Exon_edge', 'Population_AF', 'SV_chrom', 'SV_end', 'SV_start', 'SV_type']", + "data": {} + }, + "expansions": { + "title": "Repeat expansions (ExpansionHunter)", + "step": "09", + "state": "ok", + "source": "expansion_hunter/S_eh.vcf", + "file_date": "2021-03-04 05:06 UTC", + "note": null, + "data": { + "records": 2, + "key_loci": [ + { + "locus": "HTT", + "repeat_count": "not in output" + }, + { + "locus": "FMR1", + "repeat_count": "not in output" + }, + { + "locus": "C9ORF72", + "repeat_count": "not in output" + }, + { + "locus": "ATXN1", + "repeat_count": "not in output" + }, + { + "locus": "DMPK", + "repeat_count": "not in output" + } + ], + "loci": [ + { + "locus": "CFTR_T", + "repeat_count": "7/9", + "status": "", + "normal_max": "", + "pathologic_min": "", + "support": "SPANNING/SPANNING", + "adsp": "8/9", + "ci": "7-7/9-9", + "sizing": "sized", + "allele_sizing": "sized/sized", + "zone": "", + "zone_label": "", + "zone_citation": "" + }, + { + "locus": "CFTR_TG", + "repeat_count": "11/12", + "status": "", + "normal_max": "", + "pathologic_min": "", + "support": "SPANNING/FLANKING", + "adsp": "8/0", + "ci": "11-11/12-12", + "sizing": "flanking_only", + "allele_sizing": "sized/flanking_only", + "zone": "", + "zone_label": "", + "zone_citation": "" + } + ], + "stranger": false, + "stranger_outdated": false, + "unsized": [ + "CFTR_TG" + ], + "intermediate": [], + "readings": [ + { + "locus": "CFTR", + "repeat": "intron 9 (legacy IVS8) TG and poly-T tract", + "source": "Deignan et al. 2020, PMID 32404922", + "status": "not_sized", + "genotype": ".", + "phase": ".", + "reading": "ExpansionHunter did not size both tracts (TG 11/12, flanking_only; poly-T 7/9, sized): not read." + }, + { + "locus": "UGT1A1", + "repeat": "promoter TA repeat", + "source": "Gammal et al. 2016, PMID 26417955", + "status": "not_sized", + "genotype": ".", + "pharmcat": ".", + "agreement": "not_compared", + "reading": "not in step 09's output: STR_EXTRA_LOCI=yes ./scripts/setup.sh --strchive-data adds the repeat to the catalog (docs/09-str-expansions.md). PharmCAT (step 7) still reads UGT1A1 *28." + }, + { + "locus": "SLC6A4", + "repeat": "5-HTTLPR promoter length polymorphism", + "status": "not_tested", + "genotype": ".", + "reading": "not tested by any step. CPIC's 2023 SSRI guideline (Bousman et al. 2023, PMID 37032427) found that the data on SLC6A4 do not support its use in antidepressant prescribing, so a result would carry no recommendation.", + "source": "Bousman et al. 2023, PMID 37032427" + } + ] + } + }, + "telomere": { + "title": "Telomere content (TelomereHunter)", + "step": "10", + "state": "ok", + "source": "telomere/S/S_summary.tsv", + "file_date": "2021-03-04 05:06 UTC", + "note": null, + "data": { + "tel_content": "1.5", + "intratelomeric_reads": 40, + "read_check": "depth unknown (no total_reads or read_lengths in the summary): not checked" + } + }, + "roh": { + "title": "Runs of homozygosity", + "step": "11", + "state": "ok", + "source": "vcf/S_roh.txt", + "file_date": "2021-03-04 05:06 UTC", + "note": null, + "data": { + "segments": 0, + "total_mb": 0.0, + "largest_mb": 0.0, + "autosomal_over_5mb": [], + "masked_mb": 0.0, + "sex_chromosome_segments": 0, + "long_mb": 0.0, + "f_roh": null, + "relationship": null, + "upd_flag": null, + "scope": { + "source": "segments_only", + "autosomes": [], + "unmasked_bp": 0, + "partial": true, + "note": "Partial or unverified chromosome scope: no relationship band or UPD flag. F_ROH uses only the listed chromosomes, not callable bases." } + } + }, + "haplogroup": { + "title": "Mitochondrial haplogroup", + "step": "12", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "S_haplocheck.txt has no Contamination Status column", + "data": {} + }, + "y_haplogroup": { + "title": "Y-chromosome haplogroup (Yleaf)", + "step": "37", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "S_y_haplogroup.txt has no Hg column or no sample row", + "data": {} + }, + "y_tree": { + "title": "Y placement on the current public tree", + "step": "37", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "S_y_tree.txt lacks a column of Hg, Level, Tree", + "data": {} + }, + "y_str": { + "title": "Y-STR markers (HipSTR)", + "step": "45", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "S_y_str.tsv has a confidence other than high, low, no call or not callable", + "data": {} + }, + "consumer_export": { + "title": "Raw-data export (23andMe v5 layout)", + "step": "40", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "S_consumer_export.tsv has no whole-number 'sites' row", + "data": {} + }, + "bloodtype": { + "title": "Blood groups (predicted)", + "step": "41", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "S_bloodtype.tsv has no system, phenotype and confidence columns, or no row", + "data": {} + }, + "lpa": { + "title": "Lp(a): LPA KIV-2 repeat count", + "step": "46", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "S_lpa_kiv2.tsv has no status row (estimated, too little depth or not assessed)", + "data": {} + }, + "vntyper": { + "title": "MUC1 VNTR frameshifts (VNtyper)", + "step": "52", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "S_vntyper.tsv has no status row (positive, possible, negative, not assessed)", + "data": {} + }, + "locityper": { + "title": "Locityper second opinion at hard loci", + "step": "53", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "S_locityper.tsv lacks a row or one of the columns locus, region, status, genotype, haplotype_1, haplotype_2, quality, total_reads, unexplained_reads, weighted_distance, warnings, filter, reason", + "data": {} + }, + "mito": { + "title": "Mitochondrial variants", + "step": "20", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "KeyError: 'length'", + "data": {} + }, + "traits": { + "title": "Traits panel", + "step": "42", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "S_traits.tsv line 2 does not have one field per column", + "data": {} + }, + "archaic": { + "title": "Archaic ancestry (IBDmix)", + "step": "43", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "S_archaic.tsv has no percentage for neanderthal_percent", + "data": {} + }, + "ancient": { + "title": "Ancient populations (AADR PCA)", + "step": "44", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "S_ancient.tsv has no rows or lacks a column of ['country', 'dates', 'distance', 'group', 'locality', 'n', 'rank']", + "data": {} + }, + "viral": { + "title": "Viral DNA in the unmapped reads", + "step": "47", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "S_viral.tsv has no rows or lacks a column of virus, accession, reads, breadth, relative_depth, copies_per_cell, call", + "data": {} + }, + "exomiser": { + "title": "Symptom-driven ranking (Exomiser)", + "step": "48", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "S_exomiser_top.tsv lacks a column of rank, gene, moi, exomiser_score, phenotype_score, variant, genotype, consequence, hgvs, acmg, disease", + "data": {} + }, + "chip": { + "title": "Clonal haematopoiesis (CHIP)", + "step": "49", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "S_chip.tsv lacks a column of gene, protein_change, vaf, alt_reads, depth, status, note", + "data": {} + }, + "mei": { + "title": "Mobile element insertions (MEGAnE)", + "step": "50", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "S_mei.tsv is missing beside S_mei_sites.tsv", + "data": {} + }, + "mosaic": { + "title": "Mosaic chromosomal alterations (MoChA)", + "step": "51", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "S_mocha.tsv has 1 reported calls, S_mocha_summary.tsv says 2", + "data": {} + }, + "cpsr": { + "title": "Cancer predisposition (CPSR)", + "step": "17", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "S.cpsr.grch38.classification.tsv.gz: Compressed file ended before the end-of-stream marker was reached", + "data": {} + }, + "clinical": { + "title": "Clinical variant filter", + "step": "23", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "ValueError: invalid literal for int() with base 10: 'x'", + "data": {} + }, + "slivar": { + "title": "Variant prioritization (slivar)", + "step": "31", + "state": "unreadable", + "source": null, + "file_date": null, + "note": "S_prioritized.vcf.gz: Compressed file ended before the end-of-stream marker was reached", + "data": {} + } + }, + "cyp2d6": { + "calls": { + "PharmCAT": null, + "pypgx": null, + "Cyrius": "none" }, - "cyp2d6": { - "calls": { - "PharmCAT": null, - "pypgx": null, - "Cyrius": "none" - }, - "agree": null, - "consensus": null, - "probable": null - }, - "not_run": [ - "Polygenic risk scores", - "Structural variants (Manta)", - "Structural variants (Delly)", - "Copy number (CNVpytor)", - "SV consensus merge" - ], - "not_assessed": [ - "Mosaic and low-fraction small variants (only chrM heteroplasmy is reported, and with step 49 the CHIP genes)", - "Methylation and imprinting", - "Repeat expansions outside the ExpansionHunter catalog", - "Phase: two variants in one gene are compound-het candidates, not confirmed", - "Variants of uncertain significance (not interpreted)", - "Polygenic score percentiles: no ancestry reference panel was used, so the scores are raw sums", - "HBA1/HBA2 copy number (alpha-thalassaemia deletions): neither step 35's read-depth count (short reads) nor step 35b (Paraphase, PacBio HiFi) made a call", - "MUC1 VNTR frameshifts (ADTKD-MUC1): step 52 (VNtyper, opt-in) gave no reading; no standard caller sees them", - "Autosomal mosaic chromosomal alterations (large gains, losses and copy-neutral LOH in a share of the blood cells): step 51 (MoChA) did not assess this sample" - ] + "agree": null, + "consensus": null, + "probable": null + }, + "not_run": [ + "Polygenic risk scores", + "Structural variants (Manta)", + "Structural variants (Delly)", + "Copy number (CNVpytor)", + "SV consensus merge" + ], + "not_assessed": [ + "Mosaic and low-fraction small variants (only chrM heteroplasmy is reported, and with step 49 the CHIP genes)", + "Methylation and imprinting", + "Repeat expansions outside the ExpansionHunter catalog", + "Phase: two variants in one gene are compound-het candidates, not confirmed", + "Variants of uncertain significance (not interpreted)", + "Polygenic score percentiles: no ancestry reference panel was used, so the scores are raw sums", + "HBA1/HBA2 copy number (alpha-thalassaemia deletions): neither step 35's read-depth count (short reads) nor step 35b (Paraphase, PacBio HiFi) made a call", + "MUC1 VNTR frameshifts (ADTKD-MUC1): step 52 (VNtyper, opt-in) gave no reading; no standard caller sees them", + "Autosomal mosaic chromosomal alterations (large gains, losses and copy-neutral LOH in a share of the blood cells): step 51 (MoChA) did not assess this sample" + ] } diff --git a/tests/parity/golden/summary-collect/all-default-out/history-bad/summary.json b/tests/parity/golden/summary-collect/all-default-out/history-bad/summary.json new file mode 100644 index 000000000..519ac21be --- /dev/null +++ b/tests/parity/golden/summary-collect/all-default-out/history-bad/summary.json @@ -0,0 +1,534 @@ +{ + "schema_version": 2, + "sample": "S", + "generated_utc": "2026-11-19 23:58 UTC", + "run": { + "status_file": false, + "started_utc": null, + "declared_sex": null + }, + "databases": { + "clinvar_release": null, + "cpsr_clinvar_release": null, + "hla_database": null + }, + "manifest": [ + [ + "run", + "written_utc", + "2021-03-04 05:06:07 UTC" + ] + ], + "sections": { + "coverage": { + "title": "Coverage (mosdepth)", + "step": "16b", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "callability": { + "title": "Gene callability (mosdepth)", + "step": "16b", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "sex_check": { + "title": "Sex check (indexcov)", + "step": "16", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "sample_qc": { + "title": "Sample identity and contamination (somalier, VerifyBamID2)", + "step": "33", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "mosaic_sex_loss": { + "title": "Mosaic loss of chromosome Y or X", + "step": "33", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "read_count": { + "title": "Reads kept by alignment", + "step": "02", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "variants": { + "title": "Variant calling", + "step": "03", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "clinvar": { + "title": "ClinVar screen", + "step": "06", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "couple": { + "title": "Couple carrier screen", + "step": "39", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "carrier": { + "title": "Carrier list", + "step": "06", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "acmg_sf": { + "title": "ACMG secondary findings", + "step": "06", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "pharmcat": { + "title": "PharmCAT", + "step": "07", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "cpic": { + "title": "CPIC drug-gene recommendations", + "step": "27", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "pypgx": { + "title": "pypgx", + "step": "32", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "cyrius": { + "title": "Cyrius (CYP2D6)", + "step": "21", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "parascopy": { + "title": "SMN1/SMN2 copy number (Parascopy, short reads)", + "step": "35", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "paralogs": { + "title": "Paralog genes (Paraphase, PacBio HiFi)", + "step": "35b", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "hla": { + "title": "HLA typing (T1K)", + "step": "08", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "prs": { + "title": "Polygenic risk scores", + "step": "25", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "ancestry_fine": { + "title": "Fine-grained ancestry (ADMIXTURE, RFMix)", + "step": "26b", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "manta": { + "title": "Structural variants (Manta)", + "step": "04", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "delly": { + "title": "Structural variants (Delly)", + "step": "19", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "cnvpytor": { + "title": "Copy number (CNVpytor)", + "step": "18", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "sv_consensus": { + "title": "SV consensus merge", + "step": "22", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "annotsv": { + "title": "Structural variants, class 4 and 5 (AnnotSV)", + "step": "05", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "expansions": { + "title": "Repeat expansions (ExpansionHunter)", + "step": "09", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "telomere": { + "title": "Telomere content (TelomereHunter)", + "step": "10", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "roh": { + "title": "Runs of homozygosity", + "step": "11", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "haplogroup": { + "title": "Mitochondrial haplogroup", + "step": "12", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "y_haplogroup": { + "title": "Y-chromosome haplogroup (Yleaf)", + "step": "37", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "y_tree": { + "title": "Y placement on the current public tree", + "step": "37", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "y_str": { + "title": "Y-STR markers (HipSTR)", + "step": "45", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "consumer_export": { + "title": "Raw-data export (23andMe v5 layout)", + "step": "40", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "bloodtype": { + "title": "Blood groups (predicted)", + "step": "41", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "lpa": { + "title": "Lp(a): LPA KIV-2 repeat count", + "step": "46", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "vntyper": { + "title": "MUC1 VNTR frameshifts (VNtyper)", + "step": "52", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "locityper": { + "title": "Locityper second opinion at hard loci", + "step": "53", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "mito": { + "title": "Mitochondrial variants", + "step": "20", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "traits": { + "title": "Traits panel", + "step": "42", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "archaic": { + "title": "Archaic ancestry (IBDmix)", + "step": "43", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "ancient": { + "title": "Ancient populations (AADR PCA)", + "step": "44", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "viral": { + "title": "Viral DNA in the unmapped reads", + "step": "47", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "exomiser": { + "title": "Symptom-driven ranking (Exomiser)", + "step": "48", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "chip": { + "title": "Clonal haematopoiesis (CHIP)", + "step": "49", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "mei": { + "title": "Mobile element insertions (MEGAnE)", + "step": "50", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "mosaic": { + "title": "Mosaic chromosomal alterations (MoChA)", + "step": "51", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "cpsr": { + "title": "Cancer predisposition (CPSR)", + "step": "17", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "clinical": { + "title": "Clinical variant filter", + "step": "23", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "slivar": { + "title": "Variant prioritization (slivar)", + "step": "31", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + } + }, + "cyp2d6": { + "calls": { + "PharmCAT": null, + "pypgx": null, + "Cyrius": null + }, + "agree": null, + "consensus": null, + "probable": null + }, + "not_run": [ + "Coverage (mosdepth)", + "Gene callability (mosdepth)", + "Sex check (indexcov)", + "Sample identity and contamination (somalier, VerifyBamID2)", + "Mosaic loss of chromosome Y or X", + "Variant calling", + "ClinVar screen", + "PharmCAT", + "CPIC drug-gene recommendations", + "pypgx", + "Cyrius (CYP2D6)", + "SMN1/SMN2 copy number (Parascopy, short reads)", + "Paralog genes (Paraphase, PacBio HiFi)", + "HLA typing (T1K)", + "Polygenic risk scores", + "Fine-grained ancestry (ADMIXTURE, RFMix)", + "Structural variants (Manta)", + "Structural variants (Delly)", + "Copy number (CNVpytor)", + "SV consensus merge", + "Structural variants, class 4 and 5 (AnnotSV)", + "Repeat expansions (ExpansionHunter)", + "Telomere content (TelomereHunter)", + "Runs of homozygosity", + "Mitochondrial haplogroup", + "Y-chromosome haplogroup (Yleaf)", + "Y placement on the current public tree", + "Y-STR markers (HipSTR)", + "Raw-data export (23andMe v5 layout)", + "Blood groups (predicted)", + "Lp(a): LPA KIV-2 repeat count", + "MUC1 VNTR frameshifts (VNtyper)", + "Locityper second opinion at hard loci", + "Mitochondrial variants", + "Traits panel", + "Archaic ancestry (IBDmix)", + "Ancient populations (AADR PCA)", + "Viral DNA in the unmapped reads", + "Symptom-driven ranking (Exomiser)", + "Clonal haematopoiesis (CHIP)", + "Mobile element insertions (MEGAnE)", + "Mosaic chromosomal alterations (MoChA)", + "Cancer predisposition (CPSR)", + "Clinical variant filter", + "Variant prioritization (slivar)" + ], + "not_assessed": [ + "Mosaic and low-fraction small variants (only chrM heteroplasmy is reported, and with step 49 the CHIP genes)", + "Methylation and imprinting", + "Repeat expansions outside the ExpansionHunter catalog", + "Phase: two variants in one gene are compound-het candidates, not confirmed", + "Variants of uncertain significance (not interpreted)", + "Polygenic score percentiles: no ancestry reference panel was used, so the scores are raw sums", + "HBA1/HBA2 copy number (alpha-thalassaemia deletions): neither step 35's read-depth count (short reads) nor step 35b (Paraphase, PacBio HiFi) made a call", + "Mobile element insertions (Alu, L1 and SVA, the FKTN, TAF1 and BBS1 SVA insertions among them): not looked for (step 50, MEGAnE, is opt-in)", + "MUC1 VNTR frameshifts (ADTKD-MUC1): step 52 (VNtyper, opt-in) gave no reading; no standard caller sees them", + "Autosomal mosaic chromosomal alterations (large gains, losses and copy-neutral LOH in a share of the blood cells): step 51 (MoChA) did not assess this sample" + ] +} diff --git a/tests/parity/golden/summary-collect/all-empty/all-empty.json b/tests/parity/golden/summary-collect/all-empty/all-empty.json new file mode 100644 index 000000000..b6d46c0f2 --- /dev/null +++ b/tests/parity/golden/summary-collect/all-empty/all-empty.json @@ -0,0 +1,528 @@ +{ + "schema_version": 2, + "sample": "S", + "generated_utc": "2026-11-19 23:58 UTC", + "run": { + "status_file": false, + "started_utc": null, + "declared_sex": null + }, + "databases": { + "clinvar_release": null, + "cpsr_clinvar_release": null, + "hla_database": null + }, + "manifest": [], + "sections": { + "coverage": { + "title": "Coverage (mosdepth)", + "step": "16b", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "callability": { + "title": "Gene callability (mosdepth)", + "step": "16b", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "sex_check": { + "title": "Sex check (indexcov)", + "step": "16", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "sample_qc": { + "title": "Sample identity and contamination (somalier, VerifyBamID2)", + "step": "33", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "mosaic_sex_loss": { + "title": "Mosaic loss of chromosome Y or X", + "step": "33", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "read_count": { + "title": "Reads kept by alignment", + "step": "02", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "variants": { + "title": "Variant calling", + "step": "03", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "clinvar": { + "title": "ClinVar screen", + "step": "06", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "couple": { + "title": "Couple carrier screen", + "step": "39", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "carrier": { + "title": "Carrier list", + "step": "06", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "acmg_sf": { + "title": "ACMG secondary findings", + "step": "06", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "pharmcat": { + "title": "PharmCAT", + "step": "07", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "cpic": { + "title": "CPIC drug-gene recommendations", + "step": "27", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "pypgx": { + "title": "pypgx", + "step": "32", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "cyrius": { + "title": "Cyrius (CYP2D6)", + "step": "21", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "parascopy": { + "title": "SMN1/SMN2 copy number (Parascopy, short reads)", + "step": "35", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "paralogs": { + "title": "Paralog genes (Paraphase, PacBio HiFi)", + "step": "35b", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "hla": { + "title": "HLA typing (T1K)", + "step": "08", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "prs": { + "title": "Polygenic risk scores", + "step": "25", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "ancestry_fine": { + "title": "Fine-grained ancestry (ADMIXTURE, RFMix)", + "step": "26b", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "manta": { + "title": "Structural variants (Manta)", + "step": "04", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "delly": { + "title": "Structural variants (Delly)", + "step": "19", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "cnvpytor": { + "title": "Copy number (CNVpytor)", + "step": "18", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "sv_consensus": { + "title": "SV consensus merge", + "step": "22", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "annotsv": { + "title": "Structural variants, class 4 and 5 (AnnotSV)", + "step": "05", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "expansions": { + "title": "Repeat expansions (ExpansionHunter)", + "step": "09", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "telomere": { + "title": "Telomere content (TelomereHunter)", + "step": "10", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "roh": { + "title": "Runs of homozygosity", + "step": "11", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "haplogroup": { + "title": "Mitochondrial haplogroup", + "step": "12", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "y_haplogroup": { + "title": "Y-chromosome haplogroup (Yleaf)", + "step": "37", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "y_tree": { + "title": "Y placement on the current public tree", + "step": "37", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "y_str": { + "title": "Y-STR markers (HipSTR)", + "step": "45", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "consumer_export": { + "title": "Raw-data export (23andMe v5 layout)", + "step": "40", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "bloodtype": { + "title": "Blood groups (predicted)", + "step": "41", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "lpa": { + "title": "Lp(a): LPA KIV-2 repeat count", + "step": "46", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "vntyper": { + "title": "MUC1 VNTR frameshifts (VNtyper)", + "step": "52", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "locityper": { + "title": "Locityper second opinion at hard loci", + "step": "53", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "mito": { + "title": "Mitochondrial variants", + "step": "20", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "traits": { + "title": "Traits panel", + "step": "42", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "archaic": { + "title": "Archaic ancestry (IBDmix)", + "step": "43", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "ancient": { + "title": "Ancient populations (AADR PCA)", + "step": "44", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "viral": { + "title": "Viral DNA in the unmapped reads", + "step": "47", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "exomiser": { + "title": "Symptom-driven ranking (Exomiser)", + "step": "48", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "chip": { + "title": "Clonal haematopoiesis (CHIP)", + "step": "49", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "mei": { + "title": "Mobile element insertions (MEGAnE)", + "step": "50", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "mosaic": { + "title": "Mosaic chromosomal alterations (MoChA)", + "step": "51", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "cpsr": { + "title": "Cancer predisposition (CPSR)", + "step": "17", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "clinical": { + "title": "Clinical variant filter", + "step": "23", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "slivar": { + "title": "Variant prioritization (slivar)", + "step": "31", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + } + }, + "cyp2d6": { + "calls": { + "PharmCAT": null, + "pypgx": null, + "Cyrius": null + }, + "agree": null, + "consensus": null, + "probable": null + }, + "not_run": [ + "Coverage (mosdepth)", + "Gene callability (mosdepth)", + "Sex check (indexcov)", + "Sample identity and contamination (somalier, VerifyBamID2)", + "Mosaic loss of chromosome Y or X", + "Variant calling", + "ClinVar screen", + "PharmCAT", + "CPIC drug-gene recommendations", + "pypgx", + "Cyrius (CYP2D6)", + "SMN1/SMN2 copy number (Parascopy, short reads)", + "Paralog genes (Paraphase, PacBio HiFi)", + "HLA typing (T1K)", + "Polygenic risk scores", + "Fine-grained ancestry (ADMIXTURE, RFMix)", + "Structural variants (Manta)", + "Structural variants (Delly)", + "Copy number (CNVpytor)", + "SV consensus merge", + "Structural variants, class 4 and 5 (AnnotSV)", + "Repeat expansions (ExpansionHunter)", + "Telomere content (TelomereHunter)", + "Runs of homozygosity", + "Mitochondrial haplogroup", + "Y-chromosome haplogroup (Yleaf)", + "Y placement on the current public tree", + "Y-STR markers (HipSTR)", + "Raw-data export (23andMe v5 layout)", + "Blood groups (predicted)", + "Lp(a): LPA KIV-2 repeat count", + "MUC1 VNTR frameshifts (VNtyper)", + "Locityper second opinion at hard loci", + "Mitochondrial variants", + "Traits panel", + "Archaic ancestry (IBDmix)", + "Ancient populations (AADR PCA)", + "Viral DNA in the unmapped reads", + "Symptom-driven ranking (Exomiser)", + "Clonal haematopoiesis (CHIP)", + "Mobile element insertions (MEGAnE)", + "Mosaic chromosomal alterations (MoChA)", + "Cancer predisposition (CPSR)", + "Clinical variant filter", + "Variant prioritization (slivar)" + ], + "not_assessed": [ + "Mosaic and low-fraction small variants (only chrM heteroplasmy is reported, and with step 49 the CHIP genes)", + "Methylation and imprinting", + "Repeat expansions outside the ExpansionHunter catalog", + "Phase: two variants in one gene are compound-het candidates, not confirmed", + "Variants of uncertain significance (not interpreted)", + "Polygenic score percentiles: no ancestry reference panel was used, so the scores are raw sums", + "HBA1/HBA2 copy number (alpha-thalassaemia deletions): neither step 35's read-depth count (short reads) nor step 35b (Paraphase, PacBio HiFi) made a call", + "Mobile element insertions (Alu, L1 and SVA, the FKTN, TAF1 and BBS1 SVA insertions among them): not looked for (step 50, MEGAnE, is opt-in)", + "MUC1 VNTR frameshifts (ADTKD-MUC1): step 52 (VNtyper, opt-in) gave no reading; no standard caller sees them", + "Autosomal mosaic chromosomal alterations (large gains, losses and copy-neutral LOH in a share of the blood cells): step 51 (MoChA) did not assess this sample" + ] +} diff --git a/tests/parity/golden/summary-collect/all-failed/all-failed.json b/tests/parity/golden/summary-collect/all-failed/all-failed.json new file mode 100644 index 000000000..e8350a12e --- /dev/null +++ b/tests/parity/golden/summary-collect/all-failed/all-failed.json @@ -0,0 +1,528 @@ +{ + "schema_version": 2, + "sample": "S", + "generated_utc": "2026-11-19 23:58 UTC", + "run": { + "status_file": true, + "started_utc": "2020-02-02 02:02 UTC", + "declared_sex": null + }, + "databases": { + "clinvar_release": null, + "cpsr_clinvar_release": null, + "hla_database": null + }, + "manifest": [], + "sections": { + "coverage": { + "title": "Coverage (mosdepth)", + "step": "16b", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "callability": { + "title": "Gene callability (mosdepth)", + "step": "16b", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "sex_check": { + "title": "Sex check (indexcov)", + "step": "16", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "sample_qc": { + "title": "Sample identity and contamination (somalier, VerifyBamID2)", + "step": "33", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "mosaic_sex_loss": { + "title": "Mosaic loss of chromosome Y or X", + "step": "33", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "read_count": { + "title": "Reads kept by alignment", + "step": "02", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "variants": { + "title": "Variant calling", + "step": "03", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "clinvar": { + "title": "ClinVar screen", + "step": "06", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "couple": { + "title": "Couple carrier screen", + "step": "39", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "carrier": { + "title": "Carrier list", + "step": "06", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "acmg_sf": { + "title": "ACMG secondary findings", + "step": "06", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "pharmcat": { + "title": "PharmCAT", + "step": "07", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "cpic": { + "title": "CPIC drug-gene recommendations", + "step": "27", + "state": "failed", + "source": null, + "file_date": null, + "note": "step 27 failed in the run of 2020-02-02 02:02 UTC; its log is in logs/", + "data": {} + }, + "pypgx": { + "title": "pypgx", + "step": "32", + "state": "failed", + "source": null, + "file_date": null, + "note": "step 32 failed in the run of 2020-02-02 02:02 UTC; its log is in logs/", + "data": {} + }, + "cyrius": { + "title": "Cyrius (CYP2D6)", + "step": "21", + "state": "failed", + "source": null, + "file_date": null, + "note": "step 21 failed in the run of 2020-02-02 02:02 UTC; its log is in logs/", + "data": {} + }, + "parascopy": { + "title": "SMN1/SMN2 copy number (Parascopy, short reads)", + "step": "35", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "paralogs": { + "title": "Paralog genes (Paraphase, PacBio HiFi)", + "step": "35b", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "hla": { + "title": "HLA typing (T1K)", + "step": "08", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "prs": { + "title": "Polygenic risk scores", + "step": "25", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "ancestry_fine": { + "title": "Fine-grained ancestry (ADMIXTURE, RFMix)", + "step": "26b", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "manta": { + "title": "Structural variants (Manta)", + "step": "04", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "delly": { + "title": "Structural variants (Delly)", + "step": "19", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "cnvpytor": { + "title": "Copy number (CNVpytor)", + "step": "18", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "sv_consensus": { + "title": "SV consensus merge", + "step": "22", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "annotsv": { + "title": "Structural variants, class 4 and 5 (AnnotSV)", + "step": "05", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "expansions": { + "title": "Repeat expansions (ExpansionHunter)", + "step": "09", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "telomere": { + "title": "Telomere content (TelomereHunter)", + "step": "10", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "roh": { + "title": "Runs of homozygosity", + "step": "11", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "haplogroup": { + "title": "Mitochondrial haplogroup", + "step": "12", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "y_haplogroup": { + "title": "Y-chromosome haplogroup (Yleaf)", + "step": "37", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "y_tree": { + "title": "Y placement on the current public tree", + "step": "37", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "y_str": { + "title": "Y-STR markers (HipSTR)", + "step": "45", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "consumer_export": { + "title": "Raw-data export (23andMe v5 layout)", + "step": "40", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "bloodtype": { + "title": "Blood groups (predicted)", + "step": "41", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "lpa": { + "title": "Lp(a): LPA KIV-2 repeat count", + "step": "46", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "vntyper": { + "title": "MUC1 VNTR frameshifts (VNtyper)", + "step": "52", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "locityper": { + "title": "Locityper second opinion at hard loci", + "step": "53", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "mito": { + "title": "Mitochondrial variants", + "step": "20", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "traits": { + "title": "Traits panel", + "step": "42", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "archaic": { + "title": "Archaic ancestry (IBDmix)", + "step": "43", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "ancient": { + "title": "Ancient populations (AADR PCA)", + "step": "44", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "viral": { + "title": "Viral DNA in the unmapped reads", + "step": "47", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "exomiser": { + "title": "Symptom-driven ranking (Exomiser)", + "step": "48", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "chip": { + "title": "Clonal haematopoiesis (CHIP)", + "step": "49", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "mei": { + "title": "Mobile element insertions (MEGAnE)", + "step": "50", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "mosaic": { + "title": "Mosaic chromosomal alterations (MoChA)", + "step": "51", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "cpsr": { + "title": "Cancer predisposition (CPSR)", + "step": "17", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "clinical": { + "title": "Clinical variant filter", + "step": "23", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "slivar": { + "title": "Variant prioritization (slivar)", + "step": "31", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + } + }, + "cyp2d6": { + "calls": { + "PharmCAT": "failed (step 27)", + "pypgx": "failed (step 32)", + "Cyrius": "failed (step 21)" + }, + "agree": null, + "consensus": null, + "probable": null + }, + "not_run": [ + "Coverage (mosdepth)", + "Gene callability (mosdepth)", + "Sex check (indexcov)", + "Sample identity and contamination (somalier, VerifyBamID2)", + "Mosaic loss of chromosome Y or X", + "Variant calling", + "ClinVar screen", + "PharmCAT", + "CPIC drug-gene recommendations (step 27 failed)", + "pypgx (step 32 failed)", + "Cyrius (CYP2D6) (step 21 failed)", + "SMN1/SMN2 copy number (Parascopy, short reads)", + "Paralog genes (Paraphase, PacBio HiFi)", + "HLA typing (T1K)", + "Polygenic risk scores", + "Fine-grained ancestry (ADMIXTURE, RFMix)", + "Structural variants (Manta)", + "Structural variants (Delly)", + "Copy number (CNVpytor)", + "SV consensus merge", + "Structural variants, class 4 and 5 (AnnotSV)", + "Repeat expansions (ExpansionHunter)", + "Telomere content (TelomereHunter)", + "Runs of homozygosity", + "Mitochondrial haplogroup", + "Y-chromosome haplogroup (Yleaf)", + "Y placement on the current public tree", + "Y-STR markers (HipSTR)", + "Raw-data export (23andMe v5 layout)", + "Blood groups (predicted)", + "Lp(a): LPA KIV-2 repeat count", + "MUC1 VNTR frameshifts (VNtyper)", + "Locityper second opinion at hard loci", + "Mitochondrial variants", + "Traits panel", + "Archaic ancestry (IBDmix)", + "Ancient populations (AADR PCA)", + "Viral DNA in the unmapped reads", + "Symptom-driven ranking (Exomiser)", + "Clonal haematopoiesis (CHIP)", + "Mobile element insertions (MEGAnE)", + "Mosaic chromosomal alterations (MoChA)", + "Cancer predisposition (CPSR)", + "Clinical variant filter", + "Variant prioritization (slivar)" + ], + "not_assessed": [ + "Mosaic and low-fraction small variants (only chrM heteroplasmy is reported, and with step 49 the CHIP genes)", + "Methylation and imprinting", + "Repeat expansions outside the ExpansionHunter catalog", + "Phase: two variants in one gene are compound-het candidates, not confirmed", + "Variants of uncertain significance (not interpreted)", + "Polygenic score percentiles: no ancestry reference panel was used, so the scores are raw sums", + "HBA1/HBA2 copy number (alpha-thalassaemia deletions): neither step 35's read-depth count (short reads) nor step 35b (Paraphase, PacBio HiFi) made a call", + "Mobile element insertions (Alu, L1 and SVA, the FKTN, TAF1 and BBS1 SVA insertions among them): not looked for (step 50, MEGAnE, is opt-in)", + "MUC1 VNTR frameshifts (ADTKD-MUC1): step 52 (VNtyper, opt-in) gave no reading; no standard caller sees them", + "Autosomal mosaic chromosomal alterations (large gains, losses and copy-neutral LOH in a share of the blood cells): step 51 (MoChA) did not assess this sample" + ] +} diff --git a/tests/parity/golden/summary-collect/all-history-bad/all-history-bad.json b/tests/parity/golden/summary-collect/all-history-bad/all-history-bad.json new file mode 100644 index 000000000..519ac21be --- /dev/null +++ b/tests/parity/golden/summary-collect/all-history-bad/all-history-bad.json @@ -0,0 +1,534 @@ +{ + "schema_version": 2, + "sample": "S", + "generated_utc": "2026-11-19 23:58 UTC", + "run": { + "status_file": false, + "started_utc": null, + "declared_sex": null + }, + "databases": { + "clinvar_release": null, + "cpsr_clinvar_release": null, + "hla_database": null + }, + "manifest": [ + [ + "run", + "written_utc", + "2021-03-04 05:06:07 UTC" + ] + ], + "sections": { + "coverage": { + "title": "Coverage (mosdepth)", + "step": "16b", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "callability": { + "title": "Gene callability (mosdepth)", + "step": "16b", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "sex_check": { + "title": "Sex check (indexcov)", + "step": "16", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "sample_qc": { + "title": "Sample identity and contamination (somalier, VerifyBamID2)", + "step": "33", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "mosaic_sex_loss": { + "title": "Mosaic loss of chromosome Y or X", + "step": "33", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "read_count": { + "title": "Reads kept by alignment", + "step": "02", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "variants": { + "title": "Variant calling", + "step": "03", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "clinvar": { + "title": "ClinVar screen", + "step": "06", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "couple": { + "title": "Couple carrier screen", + "step": "39", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "carrier": { + "title": "Carrier list", + "step": "06", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "acmg_sf": { + "title": "ACMG secondary findings", + "step": "06", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "pharmcat": { + "title": "PharmCAT", + "step": "07", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "cpic": { + "title": "CPIC drug-gene recommendations", + "step": "27", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "pypgx": { + "title": "pypgx", + "step": "32", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "cyrius": { + "title": "Cyrius (CYP2D6)", + "step": "21", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "parascopy": { + "title": "SMN1/SMN2 copy number (Parascopy, short reads)", + "step": "35", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "paralogs": { + "title": "Paralog genes (Paraphase, PacBio HiFi)", + "step": "35b", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "hla": { + "title": "HLA typing (T1K)", + "step": "08", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "prs": { + "title": "Polygenic risk scores", + "step": "25", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "ancestry_fine": { + "title": "Fine-grained ancestry (ADMIXTURE, RFMix)", + "step": "26b", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "manta": { + "title": "Structural variants (Manta)", + "step": "04", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "delly": { + "title": "Structural variants (Delly)", + "step": "19", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "cnvpytor": { + "title": "Copy number (CNVpytor)", + "step": "18", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "sv_consensus": { + "title": "SV consensus merge", + "step": "22", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "annotsv": { + "title": "Structural variants, class 4 and 5 (AnnotSV)", + "step": "05", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "expansions": { + "title": "Repeat expansions (ExpansionHunter)", + "step": "09", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "telomere": { + "title": "Telomere content (TelomereHunter)", + "step": "10", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "roh": { + "title": "Runs of homozygosity", + "step": "11", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "haplogroup": { + "title": "Mitochondrial haplogroup", + "step": "12", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "y_haplogroup": { + "title": "Y-chromosome haplogroup (Yleaf)", + "step": "37", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "y_tree": { + "title": "Y placement on the current public tree", + "step": "37", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "y_str": { + "title": "Y-STR markers (HipSTR)", + "step": "45", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "consumer_export": { + "title": "Raw-data export (23andMe v5 layout)", + "step": "40", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "bloodtype": { + "title": "Blood groups (predicted)", + "step": "41", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "lpa": { + "title": "Lp(a): LPA KIV-2 repeat count", + "step": "46", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "vntyper": { + "title": "MUC1 VNTR frameshifts (VNtyper)", + "step": "52", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "locityper": { + "title": "Locityper second opinion at hard loci", + "step": "53", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "mito": { + "title": "Mitochondrial variants", + "step": "20", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "traits": { + "title": "Traits panel", + "step": "42", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "archaic": { + "title": "Archaic ancestry (IBDmix)", + "step": "43", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "ancient": { + "title": "Ancient populations (AADR PCA)", + "step": "44", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "viral": { + "title": "Viral DNA in the unmapped reads", + "step": "47", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "exomiser": { + "title": "Symptom-driven ranking (Exomiser)", + "step": "48", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "chip": { + "title": "Clonal haematopoiesis (CHIP)", + "step": "49", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "mei": { + "title": "Mobile element insertions (MEGAnE)", + "step": "50", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "mosaic": { + "title": "Mosaic chromosomal alterations (MoChA)", + "step": "51", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "cpsr": { + "title": "Cancer predisposition (CPSR)", + "step": "17", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "clinical": { + "title": "Clinical variant filter", + "step": "23", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, + "slivar": { + "title": "Variant prioritization (slivar)", + "step": "31", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + } + }, + "cyp2d6": { + "calls": { + "PharmCAT": null, + "pypgx": null, + "Cyrius": null + }, + "agree": null, + "consensus": null, + "probable": null + }, + "not_run": [ + "Coverage (mosdepth)", + "Gene callability (mosdepth)", + "Sex check (indexcov)", + "Sample identity and contamination (somalier, VerifyBamID2)", + "Mosaic loss of chromosome Y or X", + "Variant calling", + "ClinVar screen", + "PharmCAT", + "CPIC drug-gene recommendations", + "pypgx", + "Cyrius (CYP2D6)", + "SMN1/SMN2 copy number (Parascopy, short reads)", + "Paralog genes (Paraphase, PacBio HiFi)", + "HLA typing (T1K)", + "Polygenic risk scores", + "Fine-grained ancestry (ADMIXTURE, RFMix)", + "Structural variants (Manta)", + "Structural variants (Delly)", + "Copy number (CNVpytor)", + "SV consensus merge", + "Structural variants, class 4 and 5 (AnnotSV)", + "Repeat expansions (ExpansionHunter)", + "Telomere content (TelomereHunter)", + "Runs of homozygosity", + "Mitochondrial haplogroup", + "Y-chromosome haplogroup (Yleaf)", + "Y placement on the current public tree", + "Y-STR markers (HipSTR)", + "Raw-data export (23andMe v5 layout)", + "Blood groups (predicted)", + "Lp(a): LPA KIV-2 repeat count", + "MUC1 VNTR frameshifts (VNtyper)", + "Locityper second opinion at hard loci", + "Mitochondrial variants", + "Traits panel", + "Archaic ancestry (IBDmix)", + "Ancient populations (AADR PCA)", + "Viral DNA in the unmapped reads", + "Symptom-driven ranking (Exomiser)", + "Clonal haematopoiesis (CHIP)", + "Mobile element insertions (MEGAnE)", + "Mosaic chromosomal alterations (MoChA)", + "Cancer predisposition (CPSR)", + "Clinical variant filter", + "Variant prioritization (slivar)" + ], + "not_assessed": [ + "Mosaic and low-fraction small variants (only chrM heteroplasmy is reported, and with step 49 the CHIP genes)", + "Methylation and imprinting", + "Repeat expansions outside the ExpansionHunter catalog", + "Phase: two variants in one gene are compound-het candidates, not confirmed", + "Variants of uncertain significance (not interpreted)", + "Polygenic score percentiles: no ancestry reference panel was used, so the scores are raw sums", + "HBA1/HBA2 copy number (alpha-thalassaemia deletions): neither step 35's read-depth count (short reads) nor step 35b (Paraphase, PacBio HiFi) made a call", + "Mobile element insertions (Alu, L1 and SVA, the FKTN, TAF1 and BBS1 SVA insertions among them): not looked for (step 50, MEGAnE, is opt-in)", + "MUC1 VNTR frameshifts (ADTKD-MUC1): step 52 (VNtyper, opt-in) gave no reading; no standard caller sees them", + "Autosomal mosaic chromosomal alterations (large gains, losses and copy-neutral LOH in a share of the blood cells): step 51 (MoChA) did not assess this sample" + ] +} diff --git a/tests/parity/golden/summary-collect/all-history/all-history.json b/tests/parity/golden/summary-collect/all-history/all-history.json index 5c8f30782..434216079 100644 --- a/tests/parity/golden/summary-collect/all-history/all-history.json +++ b/tests/parity/golden/summary-collect/all-history/all-history.json @@ -1,7 +1,7 @@ { "schema_version": 2, "sample": "S", - "generated_utc": "2026-11-16 16:16 UTC", + "generated_utc": "2026-11-19 23:58 UTC", "run": { "status_file": true, "started_utc": "2020-01-01 00:00 UTC", @@ -12794,6 +12794,139 @@ } } }, + "acmg_sf": { + "title": "ACMG secondary findings", + "step": "06", + "state": "ok", + "source": "clinvar/S_clinvar_hits.vcf", + "file_date": "2021-03-04 05:06 UTC", + "note": null, + "data": { + "version": "ACMG SF v3.3", + "genes_on_list": 84, + "rules": "assets/acmg_sf.tsv", + "clinvar_hits": [ + { + "gene": "MUTYH", + "variant": "chr1:45331556 C>T", + "genotype": "het", + "significance": "Pathogenic", + "stars": 2, + "low_confidence": false, + "rule": "P and LP (2 variants)", + "rule_note": null, + "severity": null + }, + { + "gene": "BRCA1", + "variant": "chr17:43057100 C>T", + "genotype": "het", + "significance": "Pathogenic", + "stars": 3, + "low_confidence": false, + "rule": "All P and LP", + "rule_note": null, + "severity": null + }, + { + "gene": "GLA", + "variant": "chrX:101398000 C>T", + "genotype": "1", + "significance": "Pathogenic", + "stars": 0, + "low_confidence": false, + "rule": "All hemi, het, homozygous P and LP", + "rule_note": null, + "severity": null + } + ], + "clinvar_checked": true, + "high_impact": [ + { + "gene": "MUTYH", + "variant": "chr1:45332000 G>A", + "genotype": "het", + "consequence": "stop_gained", + "transcript": "MANE", + "frequency": "rare", + "frequency_unknown": false, + "weak_call": false, + "call_note": null, + "gq": 40, + "dp": 30, + "ad": "15,15", + "rule": "P and LP (2 variants)", + "rule_note": null, + "severity": null + }, + { + "gene": "MYBPC3", + "variant": "chr11:47332000 G>A", + "genotype": "het", + "consequence": "splice_donor_variant,intron_variant", + "transcript": "ANY", + "frequency": "rare", + "frequency_unknown": false, + "weak_call": false, + "call_note": null, + "gq": null, + "dp": null, + "ad": null, + "rule": "All P and LP", + "rule_note": null, + "severity": null + }, + { + "gene": "MSH2", + "variant": "chr2:47414000 A>AT", + "genotype": "het", + "consequence": "frameshift_variant", + "transcript": "CANONICAL", + "frequency": "rare", + "frequency_unknown": false, + "weak_call": true, + "call_note": "alt reads 0.07 of the depth on a heterozygote", + "gq": 50, + "dp": 30, + "ad": "28,2", + "rule": "All P and LP", + "rule_note": "A deletion of the 3' end of EPCAM silences MSH2 and counts as an MSH2 finding.", + "severity": null + } + ], + "high_impact_checked": true, + "high_impact_rare": 2, + "high_impact_weak": 1, + "high_impact_frequency_unknown": 0, + "structural": [], + "structural_checked": true, + "structural_note": null, + "not_reported": [ + { + "gene": "HFE", + "variant": "chr6:26092913 G>A", + "genotype": "het", + "source": "ClinVar", + "reason": "HFE is reported only for: HFE p.C282Y homozygotes only" + }, + { + "gene": "BTD", + "variant": "chr3:15645186 G>C", + "genotype": "het", + "source": "ClinVar", + "reason": "one heterozygous variant; BTD is reported with P and LP (2 variants)" + }, + { + "gene": "HFE", + "variant": "chr6:26093000 C>T", + "genotype": "hom", + "source": "HIGH impact", + "reason": "HFE is reported only for: HFE p.C282Y homozygotes only" + } + ], + "two_hit_note": null + } + }, "pharmcat": { "title": "PharmCAT", "step": "07", @@ -15563,6 +15696,7 @@ "new_since": { "previous_utc": "2021-01-01 00:00 UTC", "sections": [ + "acmg_sf", "ancestry_fine", "ancient", "annotsv", diff --git a/tests/parity/golden/summary-collect/all-inf/all-inf.json b/tests/parity/golden/summary-collect/all-inf/all-inf.json index 99b10ea8d..52705535b 100644 --- a/tests/parity/golden/summary-collect/all-inf/all-inf.json +++ b/tests/parity/golden/summary-collect/all-inf/all-inf.json @@ -1,7 +1,7 @@ { "schema_version": 2, "sample": "S", - "generated_utc": "2026-11-13 23:22 UTC", + "generated_utc": "2026-11-19 23:58 UTC", "run": { "status_file": true, "started_utc": "2020-01-01 00:00 UTC", @@ -137,6 +137,15 @@ "note": null, "data": {} }, + "acmg_sf": { + "title": "ACMG secondary findings", + "step": "06", + "state": "missing", + "source": null, + "file_date": null, + "note": null, + "data": {} + }, "pharmcat": { "title": "PharmCAT", "step": "07", diff --git a/tests/parity/golden/summary-collect/all-nf/all-nf.json b/tests/parity/golden/summary-collect/all-nf/all-nf.json index 9008bdbca..c75ef9c64 100644 --- a/tests/parity/golden/summary-collect/all-nf/all-nf.json +++ b/tests/parity/golden/summary-collect/all-nf/all-nf.json @@ -1,7 +1,7 @@ { "schema_version": 2, "sample": "S", - "generated_utc": "2026-11-15 22:16 UTC", + "generated_utc": "2026-11-19 23:58 UTC", "run": { "status_file": true, "started_utc": "2020-01-01 00:00 UTC", @@ -12603,6 +12603,139 @@ } } }, + "acmg_sf": { + "title": "ACMG secondary findings", + "step": "06", + "state": "ok", + "source": "clinvar/S_clinvar_hits.vcf", + "file_date": "2021-03-04 05:06 UTC", + "note": null, + "data": { + "version": "ACMG SF v3.3", + "genes_on_list": 84, + "rules": "assets/acmg_sf.tsv", + "clinvar_hits": [ + { + "gene": "MUTYH", + "variant": "chr1:45331556 C>T", + "genotype": "het", + "significance": "Pathogenic", + "stars": 2, + "low_confidence": false, + "rule": "P and LP (2 variants)", + "rule_note": null, + "severity": null + }, + { + "gene": "BRCA1", + "variant": "chr17:43057100 C>T", + "genotype": "het", + "significance": "Pathogenic", + "stars": 3, + "low_confidence": false, + "rule": "All P and LP", + "rule_note": null, + "severity": null + }, + { + "gene": "GLA", + "variant": "chrX:101398000 C>T", + "genotype": "1", + "significance": "Pathogenic", + "stars": 0, + "low_confidence": false, + "rule": "All hemi, het, homozygous P and LP", + "rule_note": null, + "severity": null + } + ], + "clinvar_checked": true, + "high_impact": [ + { + "gene": "MUTYH", + "variant": "chr1:45332000 G>A", + "genotype": "het", + "consequence": "stop_gained", + "transcript": "MANE", + "frequency": "rare", + "frequency_unknown": false, + "weak_call": false, + "call_note": null, + "gq": 40, + "dp": 30, + "ad": "15,15", + "rule": "P and LP (2 variants)", + "rule_note": null, + "severity": null + }, + { + "gene": "MYBPC3", + "variant": "chr11:47332000 G>A", + "genotype": "het", + "consequence": "splice_donor_variant,intron_variant", + "transcript": "ANY", + "frequency": "rare", + "frequency_unknown": false, + "weak_call": false, + "call_note": null, + "gq": null, + "dp": null, + "ad": null, + "rule": "All P and LP", + "rule_note": null, + "severity": null + }, + { + "gene": "MSH2", + "variant": "chr2:47414000 A>AT", + "genotype": "het", + "consequence": "frameshift_variant", + "transcript": "CANONICAL", + "frequency": "rare", + "frequency_unknown": false, + "weak_call": true, + "call_note": "alt reads 0.07 of the depth on a heterozygote", + "gq": 50, + "dp": 30, + "ad": "28,2", + "rule": "All P and LP", + "rule_note": "A deletion of the 3' end of EPCAM silences MSH2 and counts as an MSH2 finding.", + "severity": null + } + ], + "high_impact_checked": true, + "high_impact_rare": 2, + "high_impact_weak": 1, + "high_impact_frequency_unknown": 0, + "structural": [], + "structural_checked": true, + "structural_note": null, + "not_reported": [ + { + "gene": "HFE", + "variant": "chr6:26092913 G>A", + "genotype": "het", + "source": "ClinVar", + "reason": "HFE is reported only for: HFE p.C282Y homozygotes only" + }, + { + "gene": "BTD", + "variant": "chr3:15645186 G>C", + "genotype": "het", + "source": "ClinVar", + "reason": "one heterozygous variant; BTD is reported with P and LP (2 variants)" + }, + { + "gene": "HFE", + "variant": "chr6:26093000 C>T", + "genotype": "hom", + "source": "HIGH impact", + "reason": "HFE is reported only for: HFE p.C282Y homozygotes only" + } + ], + "two_hit_note": null + } + }, "pharmcat": { "title": "PharmCAT", "step": "07", diff --git a/tests/parity/golden/summary-collect/all/all.json b/tests/parity/golden/summary-collect/all/all.json index 04eb170a4..7fb5eeaf0 100644 --- a/tests/parity/golden/summary-collect/all/all.json +++ b/tests/parity/golden/summary-collect/all/all.json @@ -1,7 +1,7 @@ { "schema_version": 2, "sample": "S", - "generated_utc": "2026-11-16 16:16 UTC", + "generated_utc": "2026-11-19 23:58 UTC", "run": { "status_file": true, "started_utc": "2020-01-01 00:00 UTC", @@ -12799,6 +12799,139 @@ } } }, + "acmg_sf": { + "title": "ACMG secondary findings", + "step": "06", + "state": "ok", + "source": "clinvar/S_clinvar_hits.vcf", + "file_date": "2021-03-04 05:06 UTC", + "note": null, + "data": { + "version": "ACMG SF v3.3", + "genes_on_list": 84, + "rules": "assets/acmg_sf.tsv", + "clinvar_hits": [ + { + "gene": "MUTYH", + "variant": "chr1:45331556 C>T", + "genotype": "het", + "significance": "Pathogenic", + "stars": 2, + "low_confidence": false, + "rule": "P and LP (2 variants)", + "rule_note": null, + "severity": null + }, + { + "gene": "BRCA1", + "variant": "chr17:43057100 C>T", + "genotype": "het", + "significance": "Pathogenic", + "stars": 3, + "low_confidence": false, + "rule": "All P and LP", + "rule_note": null, + "severity": null + }, + { + "gene": "GLA", + "variant": "chrX:101398000 C>T", + "genotype": "1", + "significance": "Pathogenic", + "stars": 0, + "low_confidence": false, + "rule": "All hemi, het, homozygous P and LP", + "rule_note": null, + "severity": null + } + ], + "clinvar_checked": true, + "high_impact": [ + { + "gene": "MUTYH", + "variant": "chr1:45332000 G>A", + "genotype": "het", + "consequence": "stop_gained", + "transcript": "MANE", + "frequency": "rare", + "frequency_unknown": false, + "weak_call": false, + "call_note": null, + "gq": 40, + "dp": 30, + "ad": "15,15", + "rule": "P and LP (2 variants)", + "rule_note": null, + "severity": null + }, + { + "gene": "MYBPC3", + "variant": "chr11:47332000 G>A", + "genotype": "het", + "consequence": "splice_donor_variant,intron_variant", + "transcript": "ANY", + "frequency": "rare", + "frequency_unknown": false, + "weak_call": false, + "call_note": null, + "gq": null, + "dp": null, + "ad": null, + "rule": "All P and LP", + "rule_note": null, + "severity": null + }, + { + "gene": "MSH2", + "variant": "chr2:47414000 A>AT", + "genotype": "het", + "consequence": "frameshift_variant", + "transcript": "CANONICAL", + "frequency": "rare", + "frequency_unknown": false, + "weak_call": true, + "call_note": "alt reads 0.07 of the depth on a heterozygote", + "gq": 50, + "dp": 30, + "ad": "28,2", + "rule": "All P and LP", + "rule_note": "A deletion of the 3' end of EPCAM silences MSH2 and counts as an MSH2 finding.", + "severity": null + } + ], + "high_impact_checked": true, + "high_impact_rare": 2, + "high_impact_weak": 1, + "high_impact_frequency_unknown": 0, + "structural": [], + "structural_checked": true, + "structural_note": null, + "not_reported": [ + { + "gene": "HFE", + "variant": "chr6:26092913 G>A", + "genotype": "het", + "source": "ClinVar", + "reason": "HFE is reported only for: HFE p.C282Y homozygotes only" + }, + { + "gene": "BTD", + "variant": "chr3:15645186 G>C", + "genotype": "het", + "source": "ClinVar", + "reason": "one heterozygous variant; BTD is reported with P and LP (2 variants)" + }, + { + "gene": "HFE", + "variant": "chr6:26093000 C>T", + "genotype": "hom", + "source": "HIGH impact", + "reason": "HFE is reported only for: HFE p.C282Y homozygotes only" + } + ], + "two_hit_note": null + } + }, "pharmcat": { "title": "PharmCAT", "step": "07", diff --git a/tests/schema/summary.schema.json b/tests/schema/summary.schema.json index 3f7fd9b66..1579d8676 100644 --- a/tests/schema/summary.schema.json +++ b/tests/schema/summary.schema.json @@ -1345,6 +1345,66 @@ } ] }, + "acmg_sf": { + "allOf": [ + { + "$ref": "#/definitions/section" + }, + { + "properties": { + "data": { + "type": "object", + "properties": { + "version": { + "type": "string" + }, + "unavailable": { + "type": "string" + }, + "clinvar_hits": { + "type": "array", + "items": { + "type": "object", + "required": [ + "gene", + "variant", + "genotype" + ] + } + }, + "clinvar_checked": { + "type": "boolean" + }, + "high_impact": { + "type": "array" + }, + "high_impact_checked": { + "type": "boolean" + }, + "high_impact_note": { + "type": "string" + }, + "structural": { + "type": "array" + }, + "not_reported": { + "type": "array", + "items": { + "type": "object", + "required": [ + "gene", + "variant", + "source", + "reason" + ] + } + } + } + } + } + } + ] + }, "cpsr": { "allOf": [ {