Create documentation by antoniasaracco · Pull Request #59 · nf-core/datasync

antoniasaracco · 2026-07-22T18:19:58Z

This closes #24

…for-pipeline docs: document the datasync pipeline

atrigila

All good! minor comments

Co-authored-by: Anabella Trigila <18577080+atrigila@users.noreply.github.com>

julianSchwab

Well done. This is a quite detailed description already that guides users well! However, I think it would help to include a description of the checksum files and an example of those.

nf-core-bot · 2026-07-24T15:02:55Z

Warning

Newer version of the nf-core template is available.

Your pipeline is using an old version of the nf-core template: 4.0.2.
Please update your pipeline to the latest version.

For more documentation on how to update your pipeline, please see the Synchronisation documentation.

Co-authored-by: Anabella Trigila <18577080+atrigila@users.noreply.github.com>

atrigila

Thank you for addressing all comments!

Co-authored-by: Anabella Trigila <18577080+atrigila@users.noreply.github.com>

apeltzer

Looks good to me - I checked in @atrigila comments and also looked at most of the URls with x-refs, looks fine for me really 👍🏻

In case we still spot something, we can check once more before making the first release

github-actions · 2026-07-28T14:54:40Z

Warning

Newer version of the nf-core template is available.

Your pipeline is using an old version of the nf-core template: 4.0.2.
Please update your pipeline to the latest version.

For more documentation on how to update your pipeline, please see the Synchronisation documentation.

github-actions · 2026-07-28T14:55:23Z

`nf-core pipelines lint` overall result: Passed ✅ ⚠️

Posted for pipeline commit 71318c9

+| ✅ 200 tests passed       |+
#| ❔   5 tests were ignored |#
#| ❔   1 tests had warnings |#
!| ❗  22 tests had warnings |!

Details

❗ Test warnings:

files_exist - File not found: conf/igenomes.config
files_exist - File not found: conf/igenomes_ignored.config
readme - README contains the placeholder zenodo.XXXXXXX. This should be replaced with the zenodo doi (after the first release).
pipeline_todos - TODO string in nextflow.config: Specify your pipeline's command line flags
pipeline_todos - TODO string in nextflow.config: Optionally, you can add a pipeline-specific nf-core config at https://github.com/nf-core/configs
pipeline_todos - TODO string in nextflow.config: Update the field with the details of the contributors to your pipeline. New with Nextflow version 24.10.0
pipeline_todos - TODO string in README.md: If applicable, make list of people who have also contributed
pipeline_todos - TODO string in README.md: Add citation for pipeline after first release. Uncomment lines below and update Zenodo doi and badge at the top of this file.
pipeline_todos - TODO string in README.md: Add bibliography of tools and data used in your pipeline
pipeline_todos - TODO string in awsfulltest.yml: You can customise AWS full pipeline tests as required
pipeline_todos - TODO string in nextflow.config: Specify any additional parameters here
pipeline_todos - TODO string in main.nf: Optionally add in-text citation tools to this list.
pipeline_todos - TODO string in main.nf: Optionally add bibliographic entries to this list.
pipeline_todos - TODO string in main.nf: Only uncomment below if logic in toolCitationText/toolBibliographyText has been filled!
pipeline_todos - TODO string in CONTRIBUTING.md: Add any pipeline specific contribution guidelines here, such as coding styles, procedures, checklists etc.
pipeline_todos - TODO string in test.config: Specify the paths to your test data on nf-core/test-datasets
pipeline_todos - TODO string in test.config: Give any required params for the test so that command line flags are not needed
pipeline_todos - TODO string in test_full.config: Specify the paths to your full test data ( on nf-core/test-datasets or directly in repositories, e.g. SRA)
pipeline_todos - TODO string in test_full.config: Give any required params for the test so that command line flags are not needed
pipeline_todos - TODO string in base.config: Check the defaults for all processes
pipeline_todos - TODO string in base.config: Customise requirements for specific processes.
pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your preferred methods description, e.g. add publication citation for this pipeline

❔ Tests ignored:

files_exist - File is ignored: .github/workflows/linting_comment.yml
files_unchanged - File ignored due to lint config: .github/workflows/branch.yml
files_unchanged - File does not exist: .github/workflows/linting_comment.yml
files_unchanged - File ignored due to lint config: .github/workflows/linting.yml
multiqc_config - multiqc_config

❔ Tests fixed:

rocrate_readme_sync - Mismatch fixed: RO-Crate description updated from README.md.

✅ Tests passed:

files_exist - File found: .gitattributes
files_exist - File found: .gitignore
files_exist - File found: .nf-core.yml
files_exist - File found: .prettierignore
files_exist - File found: .prettierrc.yml
files_exist - File found: CHANGELOG.md
files_exist - File found: CITATIONS.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_exist - File found: nextflow_schema.json
files_exist - File found: nextflow.config
files_exist - File found: README.md
files_exist - File found: .github/.dockstore.yml
files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
files_exist - File found: .github/ISSUE_TEMPLATE/config.yml
files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
files_exist - File found: .github/workflows/branch.yml
files_exist - File found: .github/workflows/nf-test.yml
files_exist - File found: .github/actions/get-shards/action.yml
files_exist - File found: .github/actions/nf-test/action.yml
files_exist - File found: .github/workflows/linting.yml
files_exist - File found: assets/email_template.html
files_exist - File found: assets/email_template.txt
files_exist - File found: assets/sendmail_template.txt
files_exist - File found: assets/nf-core-datasync_logo_light.png
files_exist - File found: conf/modules.config
files_exist - File found: conf/test.config
files_exist - File found: conf/test_full.config
files_exist - File found: docs/CONTRIBUTING.md
files_exist - File found: docs/images/nf-core-datasync_logo_light.png
files_exist - File found: docs/images/nf-core-datasync_logo_dark.png
files_exist - File found: docs/output.md
files_exist - File found: docs/README.md
files_exist - File found: docs/README.md
files_exist - File found: docs/usage.md
files_exist - File found: nf-test.config
files_exist - File found: tests/default.nf.test
files_exist - File found: main.nf
files_exist - File found: assets/multiqc_config.yml
files_exist - File found: conf/base.config
files_exist - File found: .github/workflows/awstest.yml
files_exist - File found: .github/workflows/awsfulltest.yml
files_exist - File found: modules.json
files_exist - File found: ro-crate-metadata.json
files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
files_exist - File not found check: .github/workflows/push_dockerhub.yml
files_exist - File not found check: .markdownlint.yml
files_exist - File not found check: .nf-core.yaml
files_exist - File not found check: .yamllint.yml
files_exist - File not found check: bin/markdown_to_html.r
files_exist - File not found check: conf/aws.config
files_exist - File not found check: docs/images/nf-core-datasync_logo.png
files_exist - File not found check: lib/Checks.groovy
files_exist - File not found check: lib/Completion.groovy
files_exist - File not found check: lib/NfcoreTemplate.groovy
files_exist - File not found check: lib/Utils.groovy
files_exist - File not found check: lib/Workflow.groovy
files_exist - File not found check: lib/WorkflowMain.groovy
files_exist - File not found check: lib/WorkflowDatasync.groovy
files_exist - File not found check: parameters.settings.json
files_exist - File not found check: pipeline_template.yml
files_exist - File not found check: Singularity
files_exist - File not found check: lib/nfcore_external_java_deps.jar
files_exist - File not found check: .travis.yml
nextflow_config - Found nf-schema plugin
nextflow_config - Config variable found: manifest.name
nextflow_config - Config variable found: manifest.nextflowVersion
nextflow_config - Config variable found: manifest.description
nextflow_config - Config variable found: manifest.version
nextflow_config - Config variable found: manifest.homePage
nextflow_config - Config variable found: timeline.enabled
nextflow_config - Config variable found: trace.enabled
nextflow_config - Config variable found: report.enabled
nextflow_config - Config variable found: dag.enabled
nextflow_config - Config variable found: process.cpus
nextflow_config - Config variable found: process.memory
nextflow_config - Config variable found: process.time
nextflow_config - Config variable found: params.outdir
nextflow_config - Config variable found: params.input
nextflow_config - Config variable found: manifest.mainScript
nextflow_config - Config variable found: timeline.file
nextflow_config - Config variable found: trace.file
nextflow_config - Config variable found: report.file
nextflow_config - Config variable found: dag.file
nextflow_config - Config variable (correctly) not found: params.nf_required_version
nextflow_config - Config variable (correctly) not found: params.container
nextflow_config - Config variable (correctly) not found: params.singleEnd
nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
nextflow_config - Config variable (correctly) not found: params.name
nextflow_config - Config variable (correctly) not found: params.enable_conda
nextflow_config - Config variable (correctly) not found: params.max_cpus
nextflow_config - Config variable (correctly) not found: params.max_memory
nextflow_config - Config variable (correctly) not found: params.max_time
nextflow_config - Config variable (correctly) not found: params.validationFailUnrecognisedParams
nextflow_config - Config variable (correctly) not found: params.validationLenientMode
nextflow_config - Config variable (correctly) not found: params.validationSchemaIgnoreParams
nextflow_config - Config variable (correctly) not found: params.validationShowHiddenParams
nextflow_config - Config variable (correctly) not found: validation.failUnrecognisedParams
nextflow_config - Config variable (correctly) not found: validation.failUnrecognisedHeaders
nextflow_config - Config timeline.enabled had correct value: true
nextflow_config - Config report.enabled had correct value: true
nextflow_config - Config trace.enabled had correct value: true
nextflow_config - Config dag.enabled had correct value: true
nextflow_config - Config manifest.name began with nf-core/
nextflow_config - Config variable manifest.homePage began with https://github.com/nf-core/
nextflow_config - Config dag.file ended with .html
nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
nextflow_config - Config manifest.version ends in dev: 1.0dev
nextflow_config - Config params.custom_config_version is set to master
nextflow_config - Config params.custom_config_base is set to https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Lines for loading custom profiles found
nextflow_config - nextflow.config contains configuration profile test
nextflow_config - Config default value correct: params.custom_config_version= master
nextflow_config - Config default value correct: params.custom_config_base= https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Config default value correct: params.publish_dir_mode= copy
nextflow_config - Config default value correct: params.max_multiqc_email_size= 25.MB
nextflow_config - Config default value correct: params.validate_params= true
nextflow_config - Config default value correct: params.pipelines_testdata_base_path= https://raw.githubusercontent.com/nf-core/test-datasets/datasync/
nf_test_content - 'tests/default.nf.test' contains outdir parameter
nf_test_content - 'tests/default.nf.test' snapshots a 'versions.yml' file
nf_test_content - 'tests/default.nf.test' snapshots a 'versions.yml' file
nf_test_content - 'tests/main_full.nf.test' contains outdir parameter
nf_test_content - 'tests/main_full.nf.test' snapshots a 'versions.yml' file
nf_test_content - 'tests/main_full.nf.test' snapshots a 'versions.yml' file
nf_test_content - 'tests/edge.nf.test' contains outdir parameter
nf_test_content - 'tests/edge.nf.test' snapshots a 'versions.yml' file
nf_test_content - 'tests/edge.nf.test' snapshots a 'versions.yml' file
nf_test_content - 'tests/nextflow.config' contains modules_testdata_base_path
nf_test_content - 'tests/nextflow.config' contains pipelines_testdata_base_path
nf_test_content - 'nf-test.config' sets a testsDir
nf_test_content - 'nf-test.config' sets a workDir
nf_test_content - 'nf-test.config' sets a configFile
files_unchanged - .gitattributes matches the template
files_unchanged - .prettierrc.yml matches the template
files_unchanged - CODE_OF_CONDUCT.md matches the template
files_unchanged - LICENSE matches the template
files_unchanged - .github/.dockstore.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/bug_report.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/config.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
files_unchanged - .github/PULL_REQUEST_TEMPLATE.md matches the template
files_unchanged - assets/email_template.html matches the template
files_unchanged - assets/email_template.txt matches the template
files_unchanged - assets/sendmail_template.txt matches the template
files_unchanged - assets/nf-core-datasync_logo_light.png matches the template
files_unchanged - docs/images/nf-core-datasync_logo_light.png matches the template
files_unchanged - docs/images/nf-core-datasync_logo_dark.png matches the template
files_unchanged - docs/README.md matches the template
files_unchanged - .gitignore matches the template
files_unchanged - .prettierignore matches the template
actions_nf_test - '.github/workflows/nf-test.yml' is triggered on expected events
actions_nf_test - '.github/workflows/nf-test.yml' checks minimum NF version
actions_awstest - '.github/workflows/awstest.yml' is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml does not use -profile test
readme - README Nextflow minimum version badge matched config. Badge: 25.10.4, Config: 25.10.4
readme - README nf-core template version badge found.
pipeline_if_empty_null - No ifEmpty(null) strings found
plugin_includes - No wrong validation plugin imports have been found
pipeline_name_conventions - Name adheres to nf-core convention
template_strings - Did not find any Jinja template strings (0 files)
schema_lint - Schema lint passed
schema_lint - Schema title + description lint passed
schema_lint - Input mimetype lint passed: 'text/csv'
schema_params - Schema matched params returned from nextflow config
system_exit - No System.exit calls found
actions_schema_validation - Workflow validation passed: fix_linting.yml
actions_schema_validation - Workflow validation passed: awstest.yml
actions_schema_validation - Workflow validation passed: clean-up.yml
actions_schema_validation - Workflow validation passed: awsfulltest.yml
actions_schema_validation - Workflow validation passed: linting.yml
actions_schema_validation - Workflow validation passed: branch.yml
actions_schema_validation - Workflow validation passed: nf-test.yml
actions_schema_validation - Workflow validation passed: pr-comment.yml
actions_schema_validation - Workflow validation passed: template-version-comment.yml
actions_schema_validation - Workflow validation passed: release-announcements.yml
actions_schema_validation - Workflow validation passed: download_pipeline.yml
merge_markers - No merge markers found in pipeline files
modules_json - Only installed modules found in modules.json
modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'
local_component_structure - local subworkflows directory structure is correct 'subworkflows/local/TOOL/SUBTOOL'
base_config - conf/base.config found and not ignored.
modules_config - conf/modules.config found and not ignored.
modules_config - RCLONE_COPY found in conf/modules.config and Nextflow scripts.
modules_config - RCLONE_CHECK found in conf/modules.config and Nextflow scripts.
modules_config - RCLONE_CHECKSUM found in conf/modules.config and Nextflow scripts.
modules_config - MULTIQC found in conf/modules.config and Nextflow scripts.
nfcore_yml - Repository type in .nf-core.yml is valid: pipeline
nfcore_yml - nf-core version in .nf-core.yml is set to the latest version: 4.0.2
rocrate_readme_sync - RO-Crate description matches the README.md.
container_configs - conf/containers_conda_lock_files_amd64.config is up to date
container_configs - conf/containers_conda_lock_files_arm64.config is up to date
container_configs - conf/containers_docker_amd64.config is up to date
container_configs - conf/containers_docker_arm64.config is up to date
container_configs - conf/containers_singularity_https_amd64.config is up to date
container_configs - conf/containers_singularity_https_arm64.config is up to date
container_configs - conf/containers_singularity_oras_amd64.config is up to date
container_configs - conf/containers_singularity_oras_arm64.config is up to date

Run details

nf-core/tools version 4.0.2
Run at 2026-07-28 15:21:47

Already taken into account - so can move on

antoniasaracco added 10 commits July 17, 2026 16:50

docs: document datasync pipeline

1fa3a48

docs: expand rclone remote configuration guidance

447fa14

docs: clarify tested rclone configuration scope

1946fbe

Merge pull request #2 from antoniasaracco/codex/create-documentation-…

9483a25

…for-pipeline docs: document the datasync pipeline

Update README.md

be56cb8

Update usage.md

ca5f192

Update CHANGELOG.md

ceb7d63

Update README.md

f10bc2c

Update CHANGELOG.md

c0cb403

Update README.md

7dea7d8

antoniasaracco marked this pull request as draft July 22, 2026 18:27

antoniasaracco marked this pull request as ready for review July 22, 2026 18:28

antoniasaracco requested review from atrigila and delfiterradas July 22, 2026 18:28

Update output.md

6ff8774

atrigila reviewed Jul 22, 2026

View reviewed changes

antoniasaracco and others added 10 commits July 22, 2026 16:46

Update README.md

eb934c8

Add files via upload

4f16da8

Update README.md

40af01f

Update README.md

fdb0a8d

Co-authored-by: Anabella Trigila <18577080+atrigila@users.noreply.github.com>

Update README.md

51b9fdf

Update README.md

b558b3c

Update README.md

c3c93c9

Co-authored-by: Anabella Trigila <18577080+atrigila@users.noreply.github.com>

Update README.md

fd9b00c

Update docs/usage.md

110eaf8

Co-authored-by: Anabella Trigila <18577080+atrigila@users.noreply.github.com>

Update docs/usage.md

cfd37ed

Co-authored-by: Anabella Trigila <18577080+atrigila@users.noreply.github.com>

julianSchwab previously requested changes Jul 23, 2026

View reviewed changes

Comment thread docs/usage.md

Comment thread docs/usage.md

Comment thread docs/usage.md Outdated

atrigila reviewed Jul 23, 2026

View reviewed changes

Comment thread docs/images/metromap_style_pipeline_workflow_components.drawio.png Outdated

Comment thread docs/output.md

antoniasaracco added 2 commits July 23, 2026 19:39

Update usage.md

65d43d6

Update usage.md

673cd47

antoniasaracco and others added 17 commits July 24, 2026 12:03

Update docs/output.md

083f023

Co-authored-by: Anabella Trigila <18577080+atrigila@users.noreply.github.com>

Update docs/output.md

70519cc

Co-authored-by: Anabella Trigila <18577080+atrigila@users.noreply.github.com>

Update docs/output.md

1983a4f

Co-authored-by: Anabella Trigila <18577080+atrigila@users.noreply.github.com>

Update docs/usage.md

e9a292a

Co-authored-by: Anabella Trigila <18577080+atrigila@users.noreply.github.com>

Delete docs/images/datasync-metromap.drawio.png

e5dd017

Add files via upload

359a1b0

Delete docs/images/datasync-metromap.drawio.png

2b553c3

Add files via upload

c6dcf3d

Delete docs/images/datasync-metromap.drawio.png

2ff8c91

Add files via upload

d11002a

Delete docs/images/datasync-metromap.drawio.png

a720a7b

Add files via upload

c38ce42

Delete docs/images/datasync-metromap.drawio.png

316efd3

Add files via upload

1a69b5e

Update README.md

589fb05

Update output.md

3fe2a48

Update output.md

1da035b

antoniasaracco requested a review from atrigila July 24, 2026 16:22

atrigila approved these changes Jul 24, 2026

View reviewed changes

Apply suggestions from code review

426b319

Co-authored-by: Anabella Trigila <18577080+atrigila@users.noreply.github.com>

apeltzer approved these changes Jul 28, 2026

View reviewed changes

antoniasaracco added 4 commits July 28, 2026 11:58

Merge branch 'nf-core:dev' into create_documentation

06a59a6

Update README.md

35778f5

Update README.md

3c365be

Update usage.md

71318c9

apeltzer merged commit 71a8dc6 into nf-core:dev Jul 28, 2026
4 checks passed

Uh oh!

Conversation

antoniasaracco commented Jul 22, 2026

Uh oh!

atrigila left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

julianSchwab left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nf-core-bot commented Jul 24, 2026

Uh oh!

atrigila left a comment

Choose a reason for hiding this comment

Uh oh!

apeltzer left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Jul 28, 2026

Uh oh!

github-actions Bot commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

nf-core pipelines lint overall result: Passed ✅ ⚠️

❗ Test warnings:

❔ Tests ignored:

❔ Tests fixed:

✅ Tests passed:

Run details

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

github-actions Bot commented Jul 28, 2026 •

edited

Loading

`nf-core pipelines lint` overall result: Passed ✅ ⚠️