Repository navigation
Conversation
…es of its records Two variants phased on one copy, a combined allele counted with its own parts, or a record with no alt copy called could complete the two hits a recessive ACMG SF gene needs. The tier now counts haplotypes: a phased heterozygote needs a "|" GT, a non-empty PS and every copy called; a half call or a "|" GT without PS is phase unknown. Each finding of a two-hit gene gets pair (biallelic or possibly biallelic) and phase fields, and a ClinVar finding that is a combined allele keeps its combined_allele note. Bead: clin-01 Bead: clin-02 Bead: clin-03 Bead: clin-04 Bead: clin-06 Bead: clin-29
…bly biallelic Nothing says whether the deletion is real. A full loss of function called homozygous hits both copies either way, so it stays biallelic with a reason that says so; a hypomorph is biallelic only with a real deletion. Bead: clin-32
…ut an alt copy Bead: clin-29 Bead: clin-04
… record the goldens The summary-collect goldens change only by the new pair, phase and combined_allele fields and the two-hit reason wording. Bead: clin-01 Bead: D14
… phase A present genotype was used whatever its quality. The reader now marks a call with GQ under 20 or DP (else MIN_DP) under 8 as weak, keeping its genotype, and keeps the haplotype of the marker's ALT and the phase set of a phased call, for the rules to use. Bead: trait-31 Bead: D10
…ssing chr9 as type O A weak call gives its system confidence low and is named in the note. Without a BAM, a system with no VCF record at any of its sites reads not assessed instead of the reference phenotype at confidence medium. Duffy places a GATA box phased with one Fyb on the chromosome the VCF gives. An ABO A allele reads low, since no variant shows it. The ABO, RhD and null-row notes say what the DNA cannot check. The bloodtype parity goldens record these changes on purpose. Bead: trait-31 Bead: trait-28 Bead: trait-32 Bead: trait-33 Bead: trait-35 Bead: trait-36 Bead: trait-37 Bead: D10
… and fix the position count The step page, the script and module headers, the VCF-only page and the e2e case now match the new rules. The crate doc said the VCF is read at 318 positions; the marker table has 271. Bead: trait-39 Bead: trait-33
…s a reference call A reference block that starts before a marker and covers it through INFO/END was ignored, so without a BAM a gVCF could read a system as not assessed although it covers it. Such a block is now a reference call at the marker, with its GQ and MIN_DP. ABO's c.796C>A call now counts in the ABO confidence and notes, and the VCF-only page says the blood groups are opt-in. Bead: trait-28 Bead: D10
Red first: summary.json has no ACMG card when clinical/ is missing, and the agreement test compares every group b parity folder with and without its clinical VCF. summary-clinical.sh gets NO_ORACLE=1 so the test can write the folders without the oracle's helper modules. Bead: D11 Bead: D14
… step 06 sections.acmg_sf follows step 06: ok when the ClinVar hits VCF is there, not run or failed with step 06's reason otherwise. The verdicts come from the same acmg::acmg_sf the clinical section calls. Without step 23's clinical VCF the card says high_impact_checked false and high_impact_note names why and what it can miss; an unreadable clinical VCF costs only the HIGH-impact source. The clinical section keeps its nested card for the readers that still use it. The full-summary parity goldens gain the section. Bead: D11 Bead: D14
… in e2e acmg-sf-rules-1-reports gains a cell without clinical/: the card is there with the 84 rules, says the HIGH-impact source was not screened and keeps every ClinVar row of the full run's card. nf-wiring-1-report-tables compares the card of the bash and the Nextflow summaries. Bead: D11 Bead: D14
Bead: D11
…ssed systems and new RhD notes Without a BAM, the platelet case's control VCF had no record at the HPA-1, HNA-3 and HNA-4 sites, which now read not assessed; it now plants 0/0 there. The copy-number case compared step 41's whole RhD row with the tree before the engine, notes included; it now compares the phenotype, genotype and confidence, the engine's result. Bead: trait-28 Bead: trait-35
A homozygous hypomorphic allele inside a deletion is one copy, so the two-hit verdict for RBM8A is now 'possibly biallelic'. The og-render clinical test still asserted the old 'biallelic' and was the one red test of Guard rust on this branch. Bead: clin-32
…ll-floor # Conflicts: # CHANGELOG.md # tests/e2e/cn-engine-1-parity.sh
Step 10 warned and ran TelomereHunter without -b when the band file was missing, and the Nextflow module did so whenever --cytoband was unset (the default), so reads were sorted by hg19 band ends on GRCh38 positions. The script now installs assets/reference/cytoBand.hg38.txt first; the workflow passes that copy when --cytoband is unset. Bead: trait-46 Bead: trait-47
TelomereHunter 1.1.0 is Python 2.7 code. TelomereHunter2 1.0.12 gives the same tel_content on the planted test BAM (5347.255643 with 1,100 reads, 5337.585253 with 1,098, both versions). Its image has no samtools, so the image test builds telo.bam with the samtools image (new input telobam). It needs more than 1,024 open files on a BAM with many contigs; the script and module raise the limit. Bead: trait-49
The step script and module called the value a biological age baseline. Reworded, the step doc covers TelomereHunter2 and the bands, and tests/test_telomere_wording.sh fails when a file that describes step 10 uses the phrase (with a planted control). Bead: trait-48 Bead: trait-49
…off an unread ABO row Also correct the gVCF block wording (DP, else MIN_DP) and the platelet control label.
…dings card When step 23 is not ok in this run and its clinical VCF predates the run, sections.acmg_sf still reads that VCF. The card now says so in high_impact_note, with the VCF's date and step 23's result, as the clinical section goes stale.
… keep it out of the search index
# Conflicts: # CHANGELOG.md
# Conflicts: # CHANGELOG.md
# Conflicts: # CHANGELOG.md
# Conflicts: # CHANGELOG.md # tests/parity/golden/summary-collect/all-history/all-history.json # tests/parity/golden/summary-collect/all/all.json
…d agreement test GeiserX#531 (i411-sf-haplotype-count) gives each reported row of a recessive gene the gene's own verdict, `pair` and `phase`. GeiserX#533 (screen-acmg-card-section) checks that every ClinVar row of the card without VEP is in the card with VEP, unchanged. Together, cb-full fails: with step 23's clinical VCF a HIGH-impact MUTYH row joins, and the gene's phase sentence names that pair instead of the deletion. The row's own fields are the same. Rule (a) now compares the row without `pair` and `phase` only when a HIGH-impact row of its gene joined it; any other change still fails.
…st show in the annotation The m.315 C>CC call is the m.310 T>TC insertion ClinVar and gnomAD list, and m.308 CC>C is m.302 AC>A; today neither meets its record. A run whose NuMT filter had no autosomal depth must say so in the table. Both tests fail on main. Bead: lin-35 Bead: lin-32
… a NuMT filter without depth og mito annotate matched an indel only on its exact trimmed spelling, and looked for ClinVar's record at most one base away, so m.315 C>CC never met the m.310 T>TC record of the same poly-C insertion. When the exact spelling finds nothing, both sides are now moved to the left end of their repeat on the rCRS (embedded, GRCh38's chrM, MD5 c68f5267...) and matched there. An exact match still wins, so every table that matched before is unchanged; an allele whose REF is not the rCRS is not moved. ClinVar's chrM records are now read whole, since a record can sit anywhere in its repeat. --numt-default adds a '# numt' header line: the NuMT filter used a default depth (0) and marked nothing. Without it the table is as before. Bead: lin-35 Bead: lin-32
…mal depth Step 20 already warned; it now passes --numt-default to og mito annotate when neither mosdepth output nor AUTOSOMAL_COVERAGE gives a depth. The Nextflow MITO_VARIANTS prints its message as a WARNING on stderr, and MITO_VARIANTS_ANNOTATE passes --numt-default when mosdepth is not in --tools, the one case where MITO_VARIANTS has no depth files. The fake-docker case checks both paths and fails on main. Bead: lin-32
Bead: lin-35 Bead: lin-32
og report charts --summary IN --charts-json OUT writes what bin/render_report.py --charts-json writes, byte for byte, and prints the same Sections line, so both run with the same arguments. The specs are built once (charts::build) for the HTML report (charts::figure) and the live page (charts::json). Bead: D1
Every non-browser check of tests/test_report_charts.py now runs on og report charts in rust/crates/og-render/tests/charts.rs, each with its red control, plus frozen hashes of the oracle's charts JSON on the fixture and five changed copies. The Python file keeps only the live page's browser check, which needs the Python live server. Bead: D1
Bead: D1
… script 04a EVERYTHING=true reads assets/everything.tsv: every tool row goes into TOOLS (html_report aside: run-all.sh writes the report itself), every switch row is set to its value, and TOOLS or a switch set to another value stops the run with exit 2. A step skipped because its data is not installed, or because a panel is set to none, stops the run with exit 2 before Nextflow and names the setup command. Skips that come from the sample stay skips: no BAM, no gVCF, a female sample for steps 37 and 45, a BAM that is not PacBio HiFi for 35b. Step 37 on a female sample is now listed as skipped, as step 45 was; the pipeline skipped it already. Step 35b runs og paraphase check-bam on the BAM first, so a BAM that is not PL:PACBIO, or a FASTQ row, is a skip instead of a failed step. TIDDIT=true runs 04a-tiddit.sh after the pipeline, so every optional step is reachable from one command. EVERYTHING and TIDDIT join SWITCHES, the replay list of og recheck, the output contract, and step 04a joins assets/steps.tsv and RUN_ALL_STEPS, which the tests keep equal to what run-all.sh records. Bead: D8
assets/everything.tsv lists every optional item: each --tools name of main.nf, each run-all.sh switch, each setup.sh flag, each false-by-default or opt-in parameter of nextflow.config and each optional image of versions.env, as tool, switch, setup, input, image or excluded with the reason. og dev everything-check fails when one of them has no row, when a row names an item that is gone, when conf/everything.config sets other tools or parameters than the rows, or when conf/test_all.config misses a tool. Guard runs it; its planted cases are og-dev unit tests. -profile everything (conf/everything.config) gives the Nextflow path the same tools and switches as EVERYTHING=true. Bead: D8
Every data folder stubbed: every tool row runs and every script step runs after the pipeline. Without Yleaf's tables it exits 2 before Nextflow and names setup.sh --yleaf-data. A female sample still runs with the Y steps skipped; a BAM that is not PacBio HiFi skips Paraphase. TOOLS, KIR=false or ARCHAIC_DIR=none with EVERYTHING=true are refused. Bead: D8
og view serve does what assets/viewer/serve.py did: it builds the igv.js config from the sample's files, checks the locus against the reference, and serves the page, igv.js, the config and byte ranges of the files on 127.0.0.1 only, with the viewer's Content-Security-Policy. Two refusals are new. A locus whose end comes before its start is refused at start (serve.py accepted chr1:8-2). A request with a Forwarded, X-Forwarded-* or X-Real-IP header gets 403: the viewer has no password, so a reverse proxy in front of it would hand the sample to anyone who reaches the proxy. An ssh port forward adds no such header. sample_files picks Manta's diploid SVs before step 22's consensus, for the live server to reuse. The cargo tests carry the server checks of tests/test_viewer.py and the page check in headless Chrome. Bead: plat-11 Bead: D12 Bead: D1
scripts/owngenome-view.sh now runs og view serve (bin/og, or OG) and no longer needs python3 on the host. Its default SV track is Manta's diploid SVs, else step 22's consensus: the consensus keeps only calls two callers agree on, and the reads are where a single-caller call gets checked. tests/test_owngenome_view.sh checks the launcher with a python3 that fails first on PATH: the pinned igv.js and its sha256, the printed address, Manta's SV VCF first, 403 for a foreign Host and a forwarded request, and exit 2 for a reversed locus. Against the old launcher it fails (python3 was called). Bead: plat-11 Bead: D12 Bead: D1
…nd needs no python3 docs/viewer.md now names og view serve, drops the python3 requirement, says every account on the machine can reach 127.0.0.1, documents the reversed-locus and forwarded-request refusals and Manta's SVs first, and lists the tests that check it. docs/live-site.md says the live viewer still takes step 22's consensus first until its server moves to og. assets/viewer/serve.py keeps only what bin/owngenome_live_viewer.py imports (sample_files, build_config, send_file, LIVE_CSP and what they call); its server is gone. tests/test_viewer.py keeps its config and report-link checks and the helpers tests/test_live_viewer.py imports; the server, launcher and browser checks moved to Rust and shell. Both are deletions plus their docstrings: no Python line was added. Bead: plat-11 Bead: D12 Bead: D1
…iserX#531 adds GeiserX#533 (screen-acmg-card-section) recorded the new sections.acmg_sf card in the all, all-nf and all-history goldens before GeiserX#531 (i411-sf-haplotype-count) gave each row of the card its gene's `pair` and `phase` and kept the `combined_allele` note. The merged og writes the card with those fields, the same card as sections.clinical.data.acmg_sf, so oracle-diff reported 3 items that differ. The goldens are og's output for the merged tree; only those card rows and the masked generated_utc change.
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars.
|
… too c9961b9 let a HIGH-impact row of a gene change that gene's pair and phase in the og-summary agreement test. The e2e case acmg-sf-rules-1-reports checks the same rule on a real run and still demanded strict equality, so the two copies of the rule disagreed. The jq check now ignores pair and phase only when a HIGH-impact row of that gene is in the full run's card; any other change still fails. Checked with jq on hand-made cards: a pair change with a HIGH-impact row passes, and a pair change without one, a changed field and a lost row all fail. The old check failed the first.
…epth A successful task's stderr stays in its work directory, so the MITO_VARIANTS WARNING never reached a Nextflow user. The workflow now says it with log.warn when mito_variants runs without mosdepth.
…and open Manta's SVs first on the live site too og's main holds stderr's lock while og view serve runs, so a handler thread that logged with eprintln! blocked forever before it closed its connection: no request was logged, every socket stayed in CLOSE_WAIT, and the viewer stopped answering once it ran out of file descriptors. The handlers now send their lines over a channel and the thread that owns stderr writes them. tests/test_owngenome_view.sh checks that a request is logged and that the server closes the connection; both fail on the previous build. The live site's viewer picks its SV track with serve.sample_files, so the order swap there makes both viewers open Manta's diploid SVs first.
… figures Old checks 7 and 8 test what bin/render_report.py still ships: each chart once in its section below the caveat, every anchor resolving, and two renders giving the same charts file. og report render does not embed the figures yet, so they stay in Python until html.rs does. The charts.rs header and the changelog now say which checks moved.
left_allele trimmed first, so m.314 CG>CCG passed as C>CC on the rCRS and met m.310's record. The docs now say the table writes the trimmed spelling, not the moved one.
…index GRIDSS (script 04b, after the pipeline) needs the classic BWA index, which setup.sh does not build; an EVERYTHING=true run now refuses up front instead of failing hours in. A refused run puts the last run's run_status.tsv back and posts no run-finished event. pipeline-overview.md lists TIDDIT=true.
# Conflicts: # CHANGELOG.md
# Conflicts: # CHANGELOG.md
# Conflicts: # CHANGELOG.md
…r attributes were set in The live page sets the <html> data-pgp-* attributes as its files arrive, so two renders can carry the same attributes in another order. CI failed once on T7's and once on GeiserX#536's Guard on 'child with no act, a search typed' and passed on a rerun. Each element of the clone now gets its attributes back sorted by name before outerHTML: an order swap passes, while a changed value, an added or a missing attribute still differ (checked in Chromium on a planted order swap, value change, added and missing attribute).
…rpus summaries GeiserX#537 ported the charts but left its corpus rows out of the parity list, so CI held the charts only on the six frozen summaries. One report-charts case per report-render summary, compared as JSON, and the oracle row at 65095e9. scripts/ci/oracle-diff.sh report-charts: all 616 items in 154 cases agree; a planted key rename in og's charts file makes both cases it touches differ.
It writes the chart specs as render_report.py --charts-json does (GeiserX#537); no step calls it yet.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
GeiserX#539