Repository navigation
Conversation
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Duplicate-file validation, host-genome symlinks, cleanup tracking/recovery, and documentation accuracy have unresolved issues.
Review effort: Balanced
Findings: 1
Open (7)
What changed in this PR
Adds a ZymoBIOMICS vignette alongside host-genome screening, QC summaries, terminal cleanup, validation, and workflow reliability improvements.
Changes:
- Adds optional DNA host screening and QC/provenance summaries.
- Improves temporary-directory isolation, dRep execution, and cleanup.
- Expands documentation, analysis artifacts, validation tests, and CI coverage.
| File | Description |
|---|---|
workflow/Snakefile |
Adds QC and cleanup targets. |
workflow/scripts/sample_qc_summary.py |
Generates per-sample QC summaries. |
workflow/scripts/cleanup_cruft.py |
Implements manifest-driven cleanup. |
workflow/rules/RNA.smk |
Isolates temporary paths and removes intermediates. |
workflow/rules/DNA.smk |
Adds host screening, QC, and scratch improvements. |
workflow/rules/cleanup.smk |
Adds terminal cleanup rule. |
test/test_valid_input.py |
Tests sample-sheet validation. |
test/__init__.py |
Initializes the test package. |
src/utils.py |
Strengthens input and host-genome validation. |
src/run.sh |
Detects Slurm scratch support. |
src/run.py |
Adds host and scratch container binds. |
run_tests.sh |
Adds unit-test runner. |
mkdocs.yml |
Adds manual and walkthrough navigation. |
metamorph |
Exposes --host-genome. |
docs/walkthrough.md |
Adds an end-to-end workflow walkthrough. |
docs/usage/run.md |
Documents host screening, QC, and cleanup. |
docs/manual/zymo-vignette.md |
Adds the ZymoBIOMICS analysis vignette. |
docs/manual/zymo_vignette_scripts/zymo_single_rep_bc.json |
Stores replicate dissimilarities. |
docs/manual/zymo_vignette_scripts/zymo_refmap_results.json |
Stores reference-mapping results. |
docs/manual/zymo_vignette_scripts/zymo_refmap_analysis.py |
Analyzes reference mapping. |
docs/manual/zymo_vignette_scripts/zymo_fulldepth_results.json |
Stores full-depth results. |
docs/manual/zymo_vignette_scripts/zymo_fulldepth_analysis2.py |
Computes coverage-normalized estimates. |
docs/manual/zymo_vignette_scripts/zymo_fulldepth_analysis.py |
Analyzes full-depth classifications. |
docs/manual/zymo_vignette_scripts/zymo_convergence.py |
Computes convergence metrics. |
docs/manual/zymo_vignette_scripts/zymo_convergence_results.json |
Stores convergence results. |
docs/manual/zymo_vignette_scripts/zymo_asm_coverage_results.json |
Stores assembly coverage results. |
docs/manual/zymo_vignette_scripts/vendor_bars_pixel_extracted.json |
Stores extracted vendor benchmarks. |
docs/manual/zymo_vignette_scripts/refmap_refstats/ERR16012165.refstats |
Adds replicate mapping statistics. |
docs/manual/zymo_vignette_scripts/refmap_refstats/ERR16012164.refstats |
Adds replicate mapping statistics. |
docs/manual/zymo_vignette_scripts/refmap_refstats/ERR16012163.refstats |
Adds replicate mapping statistics. |
docs/manual/zymo_vignette_scripts/refmap_refstats/ERR16012162.refstats |
Adds replicate mapping statistics. |
docs/manual/zymo_vignette_scripts/refmap_refstats/ERR16012161.refstats |
Adds replicate mapping statistics. |
docs/manual/zymo_vignette_scripts/refmap_refstats/ERR16012160.refstats |
Adds replicate mapping statistics. |
docs/manual/zymo_vignette_scripts/refmap_refstats/ERR16012159.refstats |
Adds replicate mapping statistics. |
docs/manual/zymo_vignette_scripts/dehost_read_counts.json |
Stores dehosting counts. |
docs/manual/zymo_vignette_scripts/build_refmap_chart.py |
Builds reference-mapping chart. |
docs/manual/zymo_vignette_scripts/build_mash_heatmap.py |
Builds Mash heatmap. |
docs/manual/zymo_vignette_scripts/build_dehost_chart.py |
Builds dehosting chart. |
docs/manual/zymo_vignette_scripts/build_convergence_chart.py |
Builds convergence chart. |
docs/manual/zymo_vignette_scripts/build_composition_chart.py |
Builds composition chart. |
docs/manual/introduction.md |
Adds conceptual pipeline introduction. |
config/cluster.json |
Adds resources for new rules. |
config/cleanup.json |
Defines default cleanup actions. |
.tests/dummy_host_genome.fasta |
Supports host-genome dry runs. |
.github/workflows/main.yaml |
Expands CI dry-run matrix. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+18
to
+35
| input: | ||
| all_targets | ||
| output: | ||
| touch(cruft_sentinel) | ||
| params: | ||
| rname = "cleanup_cruft", | ||
| # workpath (not the metagenome_results/ subdir) so manifest patterns | ||
| # can reach anything under --output, e.g. dna/, rna/, logfiles/. | ||
| project_dir = workpath, | ||
| manifest_path = join(workpath, "config", "cleanup.json"), | ||
| threads: int(cluster["cleanup_cruft"].get("threads", default_threads)), | ||
| singularity: metawrap_container, | ||
| shell: | ||
| """ | ||
| python3 workflow/scripts/cleanup_cruft.py \\ | ||
| --manifest {params.manifest_path} \\ | ||
| --project-dir {params.project_dir} | ||
| """ |
Comment on lines
+83
to
+86
| log(f"compress_indexed: bgzip -l 9 {path}") | ||
| if not dry_run: | ||
| subprocess.run(["bgzip", "-f", "-l", "9", path], check=True) | ||
|
|
Comment on lines
+83
to
+91
| ```text | ||
| job count | ||
| ----------------------------- ------- | ||
| all 1 | ||
| bowtie2_dehost 2 | ||
| dna_centrifuger 2 | ||
| metawrap_read_qc_skipBmtagger 2 | ||
| total 7 | ||
| ``` |
…ing container bind mounts' resolve links Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
…Centrifuge to Centrifuger' typo Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
…irectory' docs update Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
…by real path' symlink resolve Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
rroutsong
changed the title
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
end to end tested output:
/data/OpenOmics/dev/metamorph/remove_cruft_at_end/enzymatic_community_data/output_full_depth