Skip to content
Merged
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Each release's notes. From 1.0.0 on, a major or minor release has hand-written n

## Unreleased

- Step 41 (blood groups) no longer reads a VCF that does not cover a gene as the reference allele. Without a BAM, a system with no VCF record at any of its sites now reads `not assessed` (phenotype and confidence) instead of the reference phenotype at confidence medium, so a VCF with no chr9 record no longer reads ABO as O. A call with a genotype quality (`GQ`) under 20 or a read depth (`DP`, else `MIN_DP`) under 8 keeps its genotype but gives its system confidence low, and the note and the evidence file name the values. An ABO A allele now reads at confidence low, because no variant shows it: it is what is left of a functional allele with neither c.802G>A nor c.803G>C. Duffy reads the VCF's phase: a GATA box phased with one Fyb in one phase set goes on the chromosome the VCF gives (on Fya that is Fy(a-b+)). When H reads H- and ABO is `not assessed` or `not determined`, the ABO row no longer carries the Bombay note about its alleles. The notes now say what the DNA cannot check: the ABO alleles outside the five variants, RHD null and DEL alleles behind a D+, and null alleles the table does not hold. The table keeps its columns; `not assessed` is a new value in the phenotype and confidence columns.
- A third-party outage no longer fails the Freshness check of a pull request. Since 2026-11-18 rest.ensembl.org has not answered, and the pull-request job's dry run of `og dev freshness` failed on every pull request that touched `versions.env` or `og-fresh`. `og dev freshness --unreachable-warns` now lists a lookup whose upstream did not answer (no connection, a timeout, HTTP 5xx or 429 after the retries, or HTTP 403) under "Upstreams that did not answer" as a warning, and its database row reads `NO ANSWER`; it does not change the exit. The 403 counts because rest.ensembl.org came back on 2026-11-19 answering 403 to GitHub's runners and 200 elsewhere, which still failed the pull-request job. A 404, an empty answer or an answer in an odd shape still exits 1. `freshness.yml` passes the flag only in the pull-request dry run. Without the flag the report and the exit are as before, and the monthly report on main still counts an outage as an error in the issue.
- Three statements in the docs were wrong. Troubleshooting said exit code 137 confirms a kill for memory, and checked it with `docker ps -a --latest`. 137 is any SIGKILL, and the step scripts run their containers with `--rm`, so that command shows an older, unrelated container. The page now says how to confirm a kill for memory: `docker events --filter event=oom`, the kernel log, or `.State.OOMKilled` for a container kept without `--rm`. The run-state and troubleshooting pages said `bin/og` does not run on a Mac and told Mac users to build `og` with cargo or run it in a container. `bin/og` runs the native macOS build that `scripts/setup.sh --og` installs, and the pages now say so; on a Mac, `og run headroom` reports the machine's memory as unknown. Troubleshooting, the FAQ and the lessons-learned page said the step scripts run their containers as root, and lessons-learned gave `:latest` images and `chmod 777` as current fixes. The step scripts run their containers as you; those entries are now marked as past fixes and name what the pipeline does today. Documentation only: no option, output or behaviour changes.
- Lint's ShellCheck job now checks every tracked shell script, not only `scripts/`: the e2e cases and their library, the fake-docker and parity tests, `tests/*.sh`, `bin/og` and the fake tools, 482 files in all, at every ShellCheck level. The 265 findings these files hold today are listed in `tests/shellcheck-baseline.txt` and still pass; any other finding fails lint. A listed finding is matched by its file, its code and the text of its line, so it keeps matching when lines above it move, and a new line with the same mistake still fails. Each run plants an unquoted `$1` in a new test script and on a new line of a listed script, and fails if either passes or if the file search misses the new script or `bin/og`. The `scripts/` check at warning level stays as it was. Part of #411 (plat-10).
Expand Down
30 changes: 22 additions & 8 deletions docs/41-bloodtype.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ Pinned in `versions.env`; [Image versions](versions.md) lists the current tags.

- `${SAMPLE}/vcf/${SAMPLE}.vcf.gz`: one sample, from any caller. Its records near the marker sites are left-aligned first, so an indel need not be written the way the table writes it. Split and whole multi-allelic records, all-sites VCFs (a reference record and then the variant at one position) and gVCF reference blocks are read too: every record that covers a site counts, whatever their order (see below)
- the reference (`REF_FASTA`, `reference/GRCh38_no_alt_analysis_set.fasta` by default) with its `.fai`, for the left-alignment; `--reference` in the Nextflow pipeline
- `${SAMPLE}/aligned/${SAMPLE}_sorted.bam` with its `.bai` (`ALIGN_DIR` as for the other BAM steps). Optional: without it RhD is not determined, and a site the VCF has no record for is read as the reference allele at confidence medium.
- `${SAMPLE}/aligned/${SAMPLE}_sorted.bam` with its `.bai` (`ALIGN_DIR` as for the other BAM steps). Optional: without it RhD is not determined, and a site the VCF has no record for is read as the reference allele at confidence medium. A system with no VCF record at any of its sites is then not assessed (see [Without the BAM](#without-the-bam)).

## Command

Expand All @@ -54,17 +54,30 @@ With `run-all.sh`: `TOOLS=...,bloodtype`. It is opt-in and needs no data, so a r
| RhD | read depth over RHD; c.487-19_504dup with c.609G>A | Copies = 2 x RHD depth / depth of the two flanks outside the RHD deletion (0 under 0.5, 1 under 1.5, else 2). The RHD pseudogene (RHD*08N.01) is an inactive copy. Exons 3 to 7 without RHD reads in a gene that is there is the pattern of an RHD-RHCE hybrid gene: RhD is then not determined |
| RhCE | c.307C>T (C/c), c.676G>C (E/e) | one allele per copy of the variant |
| Kell | c.578C>T (K/k), c.841C>T (Kp), c.1790T>C (Js) | the same |
| Duffy | c.125G>A (Fya/Fyb), c.-67T>C (GATA box), c.265C>T (FY*X) | The GATA box stops expression on red cells; with no phase it is placed on a Fyb chromosome first, where it usually lies (FY*02N.01). FY*X gives a weak Fyb (`b+w`) |
| Duffy | c.125G>A (Fya/Fyb), c.-67T>C (GATA box), c.265C>T (FY*X) | The GATA box stops expression on red cells. With one Fyb and one GATA box, both phased calls in one phase set (the same `PS`, or no `PS` on either), it goes on the chromosome the VCF gives: on the Fya chromosome that is Fy(a-b+). Else it is placed on a Fyb chromosome first, where it usually lies (FY*02N.01). FY*X gives a weak Fyb (`b+w`) |
| Kidd, MNS, Lutheran, Diego, Dombrock, Colton | one variant each pair (c.838G>A; c.59T>C and c.143C>T; c.230G>A; c.2561C>T; c.793G>A; c.134C>T) | one allele per copy |
| HPA-1, -2, -3, -4, -5, -6, -15 | ITGB3 c.176T>C, GP1BA c.482C>T, ITGA2B c.2621T>G, ITGB3 c.506G>A, ITGA2 c.1600G>A, ITGB3 c.1544G>A, CD109 c.2108C>A | one allele per copy; GRCh38 carries HPA-15b. HPA-6b is a low-frequency antigen, written HPA-6bw in the ISBT table |
| HNA-1 | FCGR3B c.108G>C, c.114C>T, c.244G>A, c.316G>A (a or b); c.194A>G (checked); c.233C>A (c) | FCGR3B\*01 (HNA-1a) and \*02 (HNA-1b) differ at five bases. GRCh38 carries the \*02 base at four of them and the \*01 base at c.194, so the copies of HNA-1a are counted on the four, which must agree (else not determined), and c.194 is only checked: a c.194 that does not match is named in the note. FCGR3B\*03 (HNA-1c) is \*02 with c.233C>A and carries HNA-1b too. Without a call at c.233 an HNA-1b allele reads \*02 or \*03. Confidence is always low: FCGR3B and FCGR3A are near copies, and FCGR3B has from none to four or more copies, which the step does not count |
| Vel, Langereis, Junior | SMIM1 c.64_80del; 36 ABCB6 null rows; 23 ABCG2 null rows | Read from the null rows alone: two null copies lack the antigen (Vel-, Lan-, Jr(a-)). One copy is a carrier, still positive. Two different null alleles, each once, are taken to sit on different chromosomes (no phase), so the antigen is lacking at confidence low |
| H, Secretor | 34 FUT1 null rows; 14 FUT2 null rows (c.461G>A among them, often written c.428G>A) | The same rule. Two FUT1 null copies give H- on red cells: `H- (Bombay, Oh)` with a non-secretor, `H- (para-Bombay)` with a secretor. The ABO row then says the red cells carry little or no A or B and usually group as O, whatever the ABO alleles are |
| H, Secretor | 34 FUT1 null rows; 14 FUT2 null rows (c.461G>A among them, often written c.428G>A) | The same rule. Two FUT1 null copies give H- on red cells: `H- (Bombay, Oh)` with a non-secretor, `H- (para-Bombay)` with a secretor. The ABO row then says the red cells carry little or no A or B and usually group as O, whatever the ABO alleles are; an ABO row that reads `not assessed` or `not determined` has no alleles to qualify and gets no such note |
| Kell null (K0), Kidd null (Jk(a-b-)) | 57 KEL null rows; 40 SLC14A1 null rows, c.342-1G>A and c.871T>C among them | Each null copy silences one chromosome. With no phase it is placed on a chromosome with the antigens of the row's `null_on` column (k, Kpb and Jsb for KEL; Jka or Jkb, as the ISBT table finds it): one KEL null with K/k reads K+k-. Two null copies give K-k- Kp(a-b-) Js(a-b-) (lacking Ku) or Jk(a-b-) (lacking Jk3), also when an antigen site has no call. Two different nulls with different `null_on` alleles are each named with their own in the note |
| HPA-7 to -14, -16 to -35 | one variant each in ITGB3, ITGA2B, ITGA2, GP1BB or GP9 | one allele per copy; the a antigen is on the reference allele, the low-prevalence b antigen on the other |
| HNA-3, -4, -5 | SLC44A2 c.461G>A, ITGAM c.230G>A, ITGAL c.2372G>C | one allele per copy; GRCh38 carries HNA-3b |

A site the VCF has no record for is the reference allele when the BAM has at least 8 reads there at MAPQ 20, and not determined below that. Without the BAM it is read as reference, and the confidence says so. The step leaves out of `samtools bedcov` the rows on a contig the BAM has no `@SQ` line for (a BAM cut to a few chromosomes), because `bedcov` would stop on them. Those sites then have no depth row: the ones with no VCF record are not determined, and RhD is not determined without chr1. A record at a site whose REF is not the table's REF is a no call, because it says nothing about the table's ALT. There are two exceptions. A 0/0 call over a longer REF that starts with the table's REF is the reference allele. A 0/0 record over a shorter REF (an all-sites VCF's `C . 0/0`, or a gVCF block that starts at a deletion marker such as ABO c.1061delC) says nothing past its own base, so the site reads as one with no record: the reads decide.
A site the VCF has no record for is the reference allele when the BAM has at least 8 reads there at MAPQ 20, and not determined below that. Without the BAM it is read as reference, and the confidence says so (see [Without the BAM](#without-the-bam)). The step leaves out of `samtools bedcov` the rows on a contig the BAM has no `@SQ` line for (a BAM cut to a few chromosomes), because `bedcov` would stop on them. Those sites then have no depth row: the ones with no VCF record are not determined, and RhD is not determined without chr1. A record at a site whose REF is not the table's REF is a no call, because it says nothing about the table's ALT. There are two exceptions. A 0/0 call over a longer REF that starts with the table's REF is the reference allele. A 0/0 record over a shorter REF (an all-sites VCF's `C . 0/0`, or a gVCF block that starts at a deletion marker such as ABO c.1061delC) says nothing past its own base, so the site reads as one with no record: the reads decide. A gVCF reference block (ALT `.`, `<*>` or `<NON_REF>`) whose `INFO/END` reaches the table's last REF base is a reference call there, wherever the block starts, with the block's `GQ` and `DP` (else `MIN_DP`) (see [Weak calls](#weak-calls)).

### Without the BAM

A VCF lists where the genome differs from the reference. A site with no record is either the reference allele or a site the VCF does not cover, and only the reads tell the two apart. So without the BAM:

- A system with a record at one of its sites at least is read, and its sites with no record count as the reference allele at confidence medium. The record shows that the VCF covers the gene.
- A system with no record at any of its sites is `not assessed` (phenotype and confidence), and the note names the sites' chromosome. A VCF with no chr9 record would otherwise read ABO as O, the group GRCh38 carries.

A person who matches GRCh38 at every site of a system (group O at ABO, HPA-1a/1a, most null rows) also has no record there in a variants-only VCF, so that system is not assessed either. A gVCF covers every site with a record or a reference block, and needs no BAM for this. RhD needs the BAM in every case.

### Weak calls

A call keeps its genotype, but it is weak when its genotype quality (FORMAT `GQ`) is under 20 or its read depth (FORMAT `DP`, else a gVCF block's `MIN_DP`) is under 8, the same depth a site with no record needs. A system that uses a weak call has confidence low, and its note names the site and the values (`KEL c.578C>T: weak call (GQ 0 under 20, DP 1 under 8)`). The evidence file adds the same values to the site's note. A record without `GQ` or `DP` is not checked. The allele balance of a call (the share of reads that carry the ALT) is not checked.

### Indels written another way

Expand Down Expand Up @@ -99,8 +112,9 @@ Confidence:

- **high**: every site was called in the VCF or covered by reads
- **medium**: a site was read as reference with no BAM depth to confirm it, or one of the system's null rows was not determined (for example too few reads there)
- **low**: a site in a region where short reads are unreliable, named in the note (RHCE exon 2 for C/c, which is RHD-like on a C allele; GYPA, GYPB and GYPE, which are near copies), markers that disagree, another allele at a site of a two-antigen pair, or two different null alleles taken to sit on different chromosomes
- **low**: a site in a region where short reads are unreliable, named in the note (RHCE exon 2 for C/c, which is RHD-like on a C allele; GYPA, GYPB and GYPE, which are near copies), markers that disagree, another allele at a site of a two-antigen pair, two different null alleles taken to sit on different chromosomes, a weak call (see [Weak calls](#weak-calls)), or an ABO A allele (no variant shows A: it is a functional allele with neither c.802G>A nor c.803G>C, so a rare O allele that keeps c.261 or a weak A subgroup reads A too)
- **not determined**: a needed site had no call or too few reads; RhD without the BAM, with flanks under 10x, or with the hybrid pattern
- **not assessed**: without the BAM, no VCF record at any of the system's sites

Row order: first the systems where the genome predicts the person lacks a high-prevalence antigen, then the others in the table's order. The high-prevalence antigens are the ones in the marker table's `high_prevalence` column: ISBT's high-prevalence antigens Vel, Lan, Jra, H, k, Kpb, Jsb, Ku, Jk3, Lub, Dib and Coa, and for platelets HPA-1a and the a antigen of HPA-4 and of HPA-6 to HPA-35 except HPA-15, whose b antigen is a low-prevalence one. Such a row's note ends with `lacks <antigen>, carried by most people (high prevalence)`. A genome with only the reference alleles lacks none of them, so its rows keep the table's order.

Expand All @@ -121,9 +135,9 @@ Not measured on a whole genome. bcftools reads the VCF's records near the marker

## Notes

- RhD here is the number of RHD copies, not their kind: weak D and partial D (RHD missense alleles) are not assessed, and neither are the RHD-CE-D hybrids that keep exons 3 to 7. The common RhD-negative allele, the deletion of the whole gene (RHD*01N.01), is what the depth sees.
- Phase is not read. ABO assumes A and B sit on different chromosomes, Duffy places the GATA box on Fyb first.
- Rare and null alleles outside the table (null alleles of two or more variants, ABO's weak subgroups, cis-AB) are not assessed. A high-prevalence antigen reads positive when none of the table's null alleles is found, so a null allele the table does not hold goes unseen. Neither are HNA-2 (CD177), HNA-1d and HNA-1null (no FCGR3B gene) assessed.
- RhD here is the number of RHD copies, not their kind: weak D and partial D (RHD missense alleles) are not assessed, and neither are RHD null alleles other than RHD*08N.01, the RHD-CE-D hybrids that keep exons 3 to 7, and DEL alleles. The common RhD-negative allele, the deletion of the whole gene (RHD*01N.01), is what the depth sees. A D+ row's note says so.
- Phase is read for Duffy only (see the table). ABO assumes A and B sit on different chromosomes, and two different null alleles of one system are taken to sit on different chromosomes.
- Rare and null alleles outside the table (null alleles of two or more variants, ABO's weak subgroups, cis-AB) are not assessed. A high-prevalence antigen reads positive when none of the table's null alleles is found, so a null allele the table does not hold goes unseen: the note of each system read from null rows (Vel, Langereis, Junior, H, secretor, Kell, Kidd) says so while the antigen reads present. The ABO row's note names the ABO alleles outside its five variants. Neither are HNA-2 (CD177), HNA-1d and HNA-1null (no FCGR3B gene) assessed.
- Every null row's allele was looked up in gnomAD v4 (genomes and exomes, 2026-11-03): 214 alleles, 53 of them not in gnomAD at all. Three were dropped because they are far more common than their null phenotype allows: FUT2 c.772G>A (rs602662, about 0.5 of chromosomes) and c.335C>T (rs200157007, 0.14 in South Asians) sit on the same haplotype as the common non-secretor allele c.461G>A, so with no phase they turned Se/se carriers into non-secretors; SLC14A1 c.810G>A (rs17675299, 0.08 in Finns) is far too common for a Kidd null. The rows kept that pass 1% in some population are known nulls of that population: FUT2 c.461G>A (the common non-secretor allele), SMIM1 c.64_80del (Vel), ABCG2 c.376C>T (Jr(a-), East Asians), SLC14A1 c.871T>C (Finns), and FUT2 c.812delC (rs1799761, 0.015 in African genomes).
- Another ALT at a null row's position (for example G>T at the position of KEL c.1546C>T, rs8176034) is not that null allele: the note names it, and the confidence stays as the other sites set it.
- Secretor status: weak secretor alleles (such as FUT2 c.418A>T, often written c.385A>T) and a deletion of the whole FUT2 gene are not assessed; the row's note says so.
Expand Down
2 changes: 1 addition & 1 deletion docs/vcf-first.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Many providers send a VCF, the file that lists where your DNA differs from the r

## What runs on a VCF alone

With only `sample`, `vcf` and `vcf_index` in the [samplesheet](nextflow.md#samplesheet-format), these tools run: `pharmcat`, `cpic`, `clinvar`, `vep`, `vcfanno`, `slivar`, `clinical_filter`, `cpsr`, `roh`, `prs`, `ancestry`, `mito_haplogroup` and `html_report`.
With only `sample`, `vcf` and `vcf_index` in the [samplesheet](nextflow.md#samplesheet-format), these tools run: `pharmcat`, `cpic`, `clinvar`, `vep`, `vcfanno`, `slivar`, `clinical_filter`, `cpsr`, `roh`, `prs`, `ancestry`, `mito_haplogroup` and `html_report`. The blood groups run too when `--tools` names `bloodtype` (it is opt-in), with RhD not determined. Without reads, a site the VCF does not list may be the reference allele or a site the file does not cover, so a blood group system with no record at any of its sites reads "not assessed" ([Step 41](41-bloodtype.md#without-the-bam)).

Everything that reads the aligned reads has nothing to work on, so it does not run: coverage (`mosdepth`, and `multiqc`, which reads it), structural variants and copy-number changes (`manta`, `delly`, `cnvpytor`, `duphold`, `annotsv`, `survivor_merge`), HLA typing, repeat expansions, telomere length, mitochondrial heteroplasmy and the read-depth CYP2D6 calls (`cyrius`, `pypgx`). If your provider can send the BAM or FASTQ, get it. The [Vendor Compatibility Guide](vendor-guide.md) says what each provider delivers.

Expand Down
Loading
Loading