Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 26 additions & 31 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ jobs:
runs-on: ubuntu-latest
permissions:
contents: write
checks: read # the tagged commit's check runs
id-token: write # the build attestation
attestations: write
steps:
Expand Down Expand Up @@ -92,44 +93,38 @@ jobs:
fetch-depth: 0
persist-credentials: false

- name: Generate release notes
id: notes
# A release is named after the version the pipeline reports, so the tag
# must equal manifest.version in nextflow.config at the tagged commit,
# and a major or minor release needs its curated notes,
# docs/releases/<tag>.md, which become the release body.
- name: The tag matches manifest.version, and the notes exist
env:
TAG: ${{ steps.tag.outputs.tag }}
run: |
# Verify the tag exists in the repo
if ! git rev-parse "${TAG}" >/dev/null 2>&1; then
echo "ERROR: Tag ${TAG} does not exist in this repository" >&2
exit 1
fi

# Find previous tag
PREV_TAG=$(git tag --sort=-version:refname | grep '^v' | grep -v "^${TAG}$" | head -1)

if [ -n "$PREV_TAG" ]; then
RANGE="${PREV_TAG}..${TAG}"
else
RANGE="${TAG}"
fi

scripts/ci/check-release.sh tag "$TAG" --notes-out "${RUNNER_TEMP}/notes.md"
{
echo "notes<<EOF"
echo "## What's Changed"
echo ""
git log "$RANGE" --pretty=format:"- %s" --no-merges 2>/dev/null || echo "- Initial release"
echo ""
echo ""
echo "## Docker Images"
echo ""
echo "All image versions are listed in [\`versions.env\`](versions.env)."
echo ""
echo "To set up: \`./scripts/setup.sh /path/to/data\`"
echo ""
echo "## og binaries"
echo ""
echo "og-linux-x86_64 and og-linux-aarch64 are this release's Rust binaries, with their SHA256SUMS and a build attestation. \`./scripts/setup.sh --og\` on a checkout of this tag downloads and checks them."
echo "EOF"
} >> "$GITHUB_OUTPUT"
} >> "${RUNNER_TEMP}/notes.md"

# Only a commit that main's CI passed is released: it must be on main,
# and every check run on it (this job left out) must have finished green.
- name: The tagged commit is on main and its checks passed
env:
TAG: ${{ steps.tag.outputs.tag }}
GH_TOKEN: ${{ github.token }}
REPO: ${{ github.repository }}
run: |
sha=$(git rev-parse "${TAG}^{commit}")
if ! git merge-base --is-ancestor "$sha" origin/main; then
echo "ERROR: ${TAG} (${sha}) is not a commit on main" >&2
exit 1
fi
gh api --paginate "repos/${REPO}/commits/${sha}/check-runs?per_page=100" \
--jq '.check_runs[] | [.name, .status, (.conclusion // "")] | @tsv' > "${RUNNER_TEMP}/runs.tsv"
scripts/ci/check-release.sh checks "${RUNNER_TEMP}/runs.tsv" --ignore "$GITHUB_JOB"

- uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
with:
Expand All @@ -148,7 +143,7 @@ jobs:
with:
tag_name: ${{ steps.tag.outputs.tag }}
name: ${{ steps.tag.outputs.tag }}
body: ${{ steps.notes.outputs.notes }}
body_path: ${{ runner.temp }}/notes.md
draft: false
files: |
dist/og-linux-x86_64
Expand Down
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Changelog

Each release's notes. From 1.0.0 on, a major or minor release has hand-written notes under `docs/releases/`, and the release workflow uses them as the release body. Older releases list their commits on GitHub.

## Unreleased

- Four VCF filters of steps 04, 05 and 07 now run in the Rust `og` binary: `og sv inv-filter` (step 04), `og sv svlen-fix` (step 05), `og pgx refblock-fill` and `og pgx mito-merge` (step 07). They replace `bin/manta_inv_filter.py`, `bin/sv_svlen_fix.py`, `bin/pgx_refblock_fill.py` and `bin/pgx_mito_merge.py`, which are gone. Their output is the same, byte for byte, except for these fixes. Steps 04, 05 and 07 now need `og`: `scripts/setup.sh <genome_dir>` builds it, and `scripts/setup.sh --og` builds only it.
Expand All @@ -14,3 +16,8 @@
- The CYP2D6 depth check and the RFC1 motif count, ported to the Rust `og` binary: `og pgx cyp2d6-bed`, `og pgx cyp2d6-measure` and `og pgx cyp2d6-check` (from `bin/cyp2d6_depth_check.py`) and `og str motif` (from `bin/rfc1_motif.py`) read BAMs with htslib, without pysam. On 67 cases they write what the Python writes, byte for byte, with one fix: a mosdepth regions file that lists one region twice with two depths is now refused (exit 2); the Python averaged the two. Steps 21, 32 and 9b still run the Python for now; no option, output file or format changes.
- Genome map, first pieces, in the Rust `og` binary. `og map build` writes one person's map document: the chromosome bands, the runs of homozygosity, and every finding with a place on the genome (its own position, or its gene's span). A result the page keeps closed (a finding to act on, and APOE on every page) is tagged with its id on every row that could reveal it, so the page's own closed-result rule hides those rows too. `og map genes` builds the gene and exon files once per GENCODE release, offline. No pipeline output, option or page changes yet; the live page starts drawing the map in a later change. The format is in `rust/crates/og-map/README.md`.
- Rust building blocks for the ports that look things up by position. `og-io` reads the lines of a region from a bgzipped file through its `.tbi` or `.csi` index, as `tabix` prints them, reading only the blocks the index points at. `og-intervals` reads bases from an uncompressed FASTA through its `.fai`, as `samtools faidx` prints them, and refuses a compressed FASTA or a `.fai` that does not match the file. It also adds sorted interval sets (union, intersection, lookups by binary search, BED files such as the PAR regions) and reciprocal overlap. Nothing calls them yet, so no pipeline output, option or file changes.

## Releases

- 1.0.0 (not released yet): [docs/releases/v1.0.0.md](docs/releases/v1.0.0.md)
- 0.2.0 to 0.8.2: https://github.com/GeiserX/OwnGenome/releases
8 changes: 6 additions & 2 deletions bin/render_report.py
Original file line number Diff line number Diff line change
Expand Up @@ -1578,7 +1578,8 @@ def head(key, title=None):
m = s["manifest"]
if m:
commit = next((r[2] for r in m if r[0] == "run" and r[1] == "git_commit"), "unknown")
w(f"Run manifest: run_manifest.tsv (pipeline commit {commit}, "
release = next((r[2] for r in m if r[0] == "run" and r[1] == "pipeline_version"), None)
w(f"Run manifest: run_manifest.tsv ({f'OwnGenome {release}, ' if release else ''}pipeline commit {commit}, "
f"{sum(1 for r in m if r[0] == 'image')} images)")
w("=" * 80)
w(" DISCLAIMER: This is NOT a clinical report. Discuss findings with a")
Expand Down Expand Up @@ -3375,6 +3376,8 @@ def html_page(s, parts, index=None):
S = s["sections"]
m = s["manifest"]
commit = next((r[2] for r in m if r[0] == "run" and r[1] == "git_commit"), None) if m else None
# The OwnGenome release the run declared (nextflow.config's manifest.version), when the manifest records it.
release = next((r[2] for r in m if r[0] == "run" and r[1] == "pipeline_version"), None) if m else None
db = s["databases"]
ns = s.get("new_since") or {}
changed = set(ns.get("sections") or [])
Expand Down Expand Up @@ -3421,7 +3424,8 @@ def html_page(s, parts, index=None):
f" Sample: <strong>{E(s['sample'])}</strong> &nbsp;|&nbsp; Generated: {E(s['generated_utc'])}"
+ (f" &nbsp;|&nbsp; Latest run: {E(s['run']['started_utc'])}" if s["run"]["started_utc"] else ""),
" </div>", ' <div class="meta">',
f" Pipeline commit: {E(commit or 'unknown')} &nbsp;|&nbsp; ClinVar file: {E(db['clinvar_release'] or 'unknown')}"
(f" OwnGenome release: {E(release)} &nbsp;|&nbsp;" if release else "")
+ f" Pipeline commit: {E(commit or 'unknown')} &nbsp;|&nbsp; ClinVar file: {E(db['clinvar_release'] or 'unknown')}"
f" &nbsp;|&nbsp; ClinVar in CPSR's bundle: {E(db['cpsr_clinvar_release'] or 'unknown')}"
f" &nbsp;|&nbsp; HLA database: {E(db['hla_database'] or 'unknown')}",
" </div>"] + ([f' <div class="meta">{E(recheck_line(s["recheck"]))}</div>'] if s.get("recheck") else []) + [
Expand Down
Loading
Loading