Nextflow

[dimalvovs]

This pull request introduces several improvements and updates to the Nextflow pipeline, focusing on parameter management, reproducibility, and workflow configuration. The most significant changes are the addition of new parameters (such as asynchronous_updates and seed), improved parameter passing and output tracking, and enhanced configuration for various compute environments. There are also clarifications and fixes in the workflow automation and documentation.

Parameter handling and reproducibility:

• Added new parameters to the pipeline, including seed (for reproducibility) and asynchronous_updates, and ensured they are passed throughout the workflow (main.nf, nextflow.config). [1] [2] [3]
• Changed the way the output directory prefix is constructed in COGAPS to use a hash of key parameters, improving traceability of results. [1] [2]
• The full set of parameters used for each run is now saved to a params.yml file in the output directory.

Workflow and configuration enhancements:

• Updated nextflow.config to add and refine profiles for different compute environments (e.g., jhpce, jhu_discovery, igs), including resource settings and queue options.
• Fixed the resourceLimits definition to use a closure, and adjusted CPU/memory allocations for high-resource processes. [1] [2]

Workflow automation and documentation:

• Clarified and updated comments in GitHub Actions workflows to better describe when images are built and pushed, and under what conditions artifacts are used in downstream workflows. [1] [2] [3] [4]
• Updated the README.md to reflect the new and reordered parameters, improving documentation accuracy.

Dependency management:

• Ensured that the sparseMatrixStats R package is installed in the Docker image, supporting Nextflow module requirements.

Code cleanup:

• Removed unused variables (such as args) from process scripts for clarity and maintainability. [1] [2] [3]