Add ci nftests #381

nschcolnicov · 2024-08-19T16:51:48Z

PR checklist

github-actions · 2024-08-19T16:53:26Z

`nf-core lint` overall result: Passed ✅ ⚠️

Posted for pipeline commit e843703

+| ✅ 213 tests passed       |+
#| ❔   1 tests were ignored |#
!| ❗   3 tests had warnings |!

❗ Test warnings:

pipeline_todos - TODO string in main.nf: Optionally add in-text citation tools to this list.
pipeline_todos - TODO string in main.nf: Optionally add bibliographic entries to this list.
pipeline_todos - TODO string in main.nf: Only uncomment below if logic in toolCitationText/toolBibliographyText has been filled!

❔ Tests ignored:

nextflow_config - Config default ignored: params.fastp_known_mirna_adapters

✅ Tests passed:

files_exist - File found: .gitattributes
files_exist - File found: .gitignore
files_exist - File found: .nf-core.yml
files_exist - File found: .editorconfig
files_exist - File found: .prettierignore
files_exist - File found: .prettierrc.yml
files_exist - File found: CHANGELOG.md
files_exist - File found: CITATIONS.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_exist - File found: nextflow_schema.json
files_exist - File found: nextflow.config
files_exist - File found: README.md
files_exist - File found: .github/.dockstore.yml
files_exist - File found: .github/CONTRIBUTING.md
files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
files_exist - File found: .github/ISSUE_TEMPLATE/config.yml
files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
files_exist - File found: .github/workflows/branch.yml
files_exist - File found: .github/workflows/ci.yml
files_exist - File found: .github/workflows/linting_comment.yml
files_exist - File found: .github/workflows/linting.yml
files_exist - File found: assets/email_template.html
files_exist - File found: assets/email_template.txt
files_exist - File found: assets/sendmail_template.txt
files_exist - File found: assets/nf-core-smrnaseq_logo_light.png
files_exist - File found: conf/modules.config
files_exist - File found: conf/test.config
files_exist - File found: conf/test_full.config
files_exist - File found: docs/images/nf-core-smrnaseq_logo_light.png
files_exist - File found: docs/images/nf-core-smrnaseq_logo_dark.png
files_exist - File found: docs/output.md
files_exist - File found: docs/README.md
files_exist - File found: docs/README.md
files_exist - File found: docs/usage.md
files_exist - File found: main.nf
files_exist - File found: assets/multiqc_config.yml
files_exist - File found: conf/base.config
files_exist - File found: conf/igenomes.config
files_exist - File found: .github/workflows/awstest.yml
files_exist - File found: .github/workflows/awsfulltest.yml
files_exist - File found: modules.json
files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
files_exist - File not found check: .github/workflows/push_dockerhub.yml
files_exist - File not found check: .markdownlint.yml
files_exist - File not found check: .nf-core.yaml
files_exist - File not found check: .yamllint.yml
files_exist - File not found check: bin/markdown_to_html.r
files_exist - File not found check: conf/aws.config
files_exist - File not found check: docs/images/nf-core-smrnaseq_logo.png
files_exist - File not found check: lib/Checks.groovy
files_exist - File not found check: lib/Completion.groovy
files_exist - File not found check: lib/NfcoreTemplate.groovy
files_exist - File not found check: lib/Utils.groovy
files_exist - File not found check: lib/Workflow.groovy
files_exist - File not found check: lib/WorkflowMain.groovy
files_exist - File not found check: lib/WorkflowSmrnaseq.groovy
files_exist - File not found check: parameters.settings.json
files_exist - File not found check: pipeline_template.yml
files_exist - File not found check: Singularity
files_exist - File not found check: lib/nfcore_external_java_deps.jar
files_exist - File not found check: .travis.yml
nextflow_config - Config variable found: manifest.name
nextflow_config - Config variable found: manifest.nextflowVersion
nextflow_config - Config variable found: manifest.description
nextflow_config - Config variable found: manifest.version
nextflow_config - Config variable found: manifest.homePage
nextflow_config - Config variable found: timeline.enabled
nextflow_config - Config variable found: trace.enabled
nextflow_config - Config variable found: report.enabled
nextflow_config - Config variable found: dag.enabled
nextflow_config - Config variable found: process.cpus
nextflow_config - Config variable found: process.memory
nextflow_config - Config variable found: process.time
nextflow_config - Config variable found: params.outdir
nextflow_config - Config variable found: params.input
nextflow_config - Config variable found: params.validationShowHiddenParams
nextflow_config - Config variable found: params.validationSchemaIgnoreParams
nextflow_config - Config variable found: manifest.mainScript
nextflow_config - Config variable found: timeline.file
nextflow_config - Config variable found: trace.file
nextflow_config - Config variable found: report.file
nextflow_config - Config variable found: dag.file
nextflow_config - Config variable (correctly) not found: params.nf_required_version
nextflow_config - Config variable (correctly) not found: params.container
nextflow_config - Config variable (correctly) not found: params.singleEnd
nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
nextflow_config - Config variable (correctly) not found: params.name
nextflow_config - Config variable (correctly) not found: params.enable_conda
nextflow_config - Config timeline.enabled had correct value: true
nextflow_config - Config report.enabled had correct value: true
nextflow_config - Config trace.enabled had correct value: true
nextflow_config - Config dag.enabled had correct value: true
nextflow_config - Config manifest.name began with nf-core/
nextflow_config - Config variable manifest.homePage began with https://github.com/nf-core/
nextflow_config - Config dag.file ended with .html
nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
nextflow_config - Config manifest.version ends in dev: 2.3.2dev
nextflow_config - Config params.custom_config_version is set to master
nextflow_config - Config params.custom_config_base is set to https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Lines for loading custom profiles found
nextflow_config - nextflow.config contains configuration profile test
nextflow_config - Config default value correct: params.protocol= custom
nextflow_config - Config default value correct: params.umitools_extract_method= string
nextflow_config - Config default value correct: params.umitools_method= dir
nextflow_config - Config default value correct: params.skip_umi_extract_before_dedup= true
nextflow_config - Config default value correct: params.mature= https://github.com/nf-core/test-datasets/raw/smrnaseq/miRBase/mature.fa
nextflow_config - Config default value correct: params.hairpin= https://github.com/nf-core/test-datasets/raw/smrnaseq/miRBase/hairpin.fa
nextflow_config - Config default value correct: params.save_aligned_mirna_quant= true
nextflow_config - Config default value correct: params.three_prime_adapter= AGATCGGAAGAGCACACGTCTGAACTCCAGTCA
nextflow_config - Config default value correct: params.trim_fastq= true
nextflow_config - Config default value correct: params.fastp_min_length= 17
nextflow_config - Config default value correct: params.fastp_max_length= 100
nextflow_config - Config default value correct: params.min_trimmed_reads= 10
nextflow_config - Config default value correct: params.save_merged= true
nextflow_config - Config default value correct: params.phred_offset= 33
nextflow_config - Config default value correct: params.custom_config_version= master
nextflow_config - Config default value correct: params.custom_config_base= https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Config default value correct: params.max_cpus= 16
nextflow_config - Config default value correct: params.max_memory= 128.GB
nextflow_config - Config default value correct: params.max_time= 240.h
nextflow_config - Config default value correct: params.publish_dir_mode= copy
nextflow_config - Config default value correct: params.max_multiqc_email_size= 25.MB
nextflow_config - Config default value correct: params.validate_params= true
nextflow_config - Config default value correct: params.pipelines_testdata_base_path= https://raw.githubusercontent.com/nf-core/test-datasets/
files_unchanged - .gitattributes matches the template
files_unchanged - .prettierrc.yml matches the template
files_unchanged - CODE_OF_CONDUCT.md matches the template
files_unchanged - LICENSE matches the template
files_unchanged - .github/.dockstore.yml matches the template
files_unchanged - .github/CONTRIBUTING.md matches the template
files_unchanged - .github/ISSUE_TEMPLATE/bug_report.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/config.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
files_unchanged - .github/PULL_REQUEST_TEMPLATE.md matches the template
files_unchanged - .github/workflows/branch.yml matches the template
files_unchanged - .github/workflows/linting_comment.yml matches the template
files_unchanged - .github/workflows/linting.yml matches the template
files_unchanged - assets/email_template.html matches the template
files_unchanged - assets/email_template.txt matches the template
files_unchanged - assets/sendmail_template.txt matches the template
files_unchanged - assets/nf-core-smrnaseq_logo_light.png matches the template
files_unchanged - docs/images/nf-core-smrnaseq_logo_light.png matches the template
files_unchanged - docs/images/nf-core-smrnaseq_logo_dark.png matches the template
files_unchanged - docs/README.md matches the template
files_unchanged - .gitignore matches the template
files_unchanged - .prettierignore matches the template
actions_ci - '.github/workflows/ci.yml' is triggered on expected events
actions_ci - '.github/workflows/ci.yml' checks minimum NF version
actions_awstest - '.github/workflows/awstest.yml' is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml does not use -profile test
readme - README Nextflow minimum version badge matched config. Badge: 23.04.0, Config: 23.04.0
readme - README Zenodo placeholder was replaced with DOI.
pipeline_name_conventions - Name adheres to nf-core convention
template_strings - Did not find any Jinja template strings (246 files)
schema_lint - Schema lint passed
schema_lint - Schema title + description lint passed
schema_lint - Input mimetype lint passed: 'text/csv'
schema_params - Schema matched params returned from nextflow config
system_exit - No System.exit calls found
actions_schema_validation - Workflow validation passed: awstest.yml
actions_schema_validation - Workflow validation passed: branch.yml
actions_schema_validation - Workflow validation passed: fix-linting.yml
actions_schema_validation - Workflow validation passed: linting.yml
actions_schema_validation - Workflow validation passed: clean-up.yml
actions_schema_validation - Workflow validation passed: ci.yml
actions_schema_validation - Workflow validation passed: linting_comment.yml
actions_schema_validation - Workflow validation passed: awsfulltest.yml
actions_schema_validation - Workflow validation passed: download_pipeline.yml
actions_schema_validation - Workflow validation passed: release-announcements.yml
merge_markers - No merge markers found in pipeline files
modules_json - Only installed modules found in modules.json
multiqc_config - assets/multiqc_config.yml found and not ignored.
multiqc_config - assets/multiqc_config.yml contains report_section_order
multiqc_config - assets/multiqc_config.yml contains export_plots
multiqc_config - assets/multiqc_config.yml contains report_comment
multiqc_config - assets/multiqc_config.yml follows the ordering scheme of the minimally required plugins.
multiqc_config - assets/multiqc_config.yml contains a matching 'report_comment'.
multiqc_config - assets/multiqc_config.yml contains 'export_plots: true'.
modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'
base_config - conf/base.config found and not ignored.
modules_config - conf/modules.config found and not ignored.
modules_config - NFCORE_SMRNASEQ found in conf/modules.config and Nextflow scripts.
modules_config - CAT_FASTQ found in conf/modules.config and Nextflow scripts.
modules_config - FASTP_LENGTH_FILTER found in conf/modules.config and Nextflow scripts.
modules_config - INDEX_GENOME found in conf/modules.config and Nextflow scripts.
modules_config - UMITOOLS_EXTRACT found in conf/modules.config and Nextflow scripts.
modules_config - MIRTRACE_RUN found in conf/modules.config and Nextflow scripts.
modules_config - NFCORE_SMRNASEQ found in conf/modules.config and Nextflow scripts.
modules_config - NFCORE_SMRNASEQ found in conf/modules.config and Nextflow scripts.
modules_config - NFCORE_SMRNASEQ found in conf/modules.config and Nextflow scripts.
modules_config - NFCORE_SMRNASEQ found in conf/modules.config and Nextflow scripts.
modules_config - NFCORE_SMRNASEQ found in conf/modules.config and Nextflow scripts.
modules_config - NFCORE_SMRNASEQ found in conf/modules.config and Nextflow scripts.
modules_config - NFCORE_SMRNASEQ found in conf/modules.config and Nextflow scripts.
modules_config - NFCORE_SMRNASEQ found in conf/modules.config and Nextflow scripts.
modules_config - NFCORE_SMRNASEQ found in conf/modules.config and Nextflow scripts.
modules_config - EDGER_QC found in conf/modules.config and Nextflow scripts.
modules_config - SEQCLUSTER_SEQUENCES found in conf/modules.config and Nextflow scripts.
modules_config - NFCORE_SMRNASEQ found in conf/modules.config and Nextflow scripts.
modules_config - MIRTOP_QUANT found in conf/modules.config and Nextflow scripts.
modules_config - NFCORE_SMRNASEQ found in conf/modules.config and Nextflow scripts.
modules_config - NFCORE_SMRNASEQ found in conf/modules.config and Nextflow scripts.
modules_config - NFCORE_SMRNASEQ found in conf/modules.config and Nextflow scripts.
modules_config - NFCORE_SMRNASEQ found in conf/modules.config and Nextflow scripts.
modules_config - NFCORE_SMRNASEQ found in conf/modules.config and Nextflow scripts.
modules_config - NFCORE_SMRNASEQ found in conf/modules.config and Nextflow scripts.
modules_config - MULTIQC found in conf/modules.config and Nextflow scripts.
nfcore_yml - Repository type in .nf-core.yml is valid: pipeline
nfcore_yml - nf-core version in .nf-core.yml is set to the latest version: 2.14.1

Run details

nf-core/tools version 2.14.1
Run at 2024-08-26 15:47:22

nschcolnicov · 2024-08-20T19:56:46Z

.github/workflows/ci.yml

@@ -10,6 +10,8 @@ on:

 env:
  NXF_ANSI_LOG: false
+  CAPSULE_LOG: none
+  NFTEST_VER: "0.8.4"


not using sharding or latest nf-test due to the issue with download limit from mirBase

snapshots seem to have been done with 0.9.0
maybe it is worth updating to 0.9.0 but keeping out the sharding?

…_nftests

atrigila

Looks good! Thanks for adding these tests and pointing the URLs to GitHub :)

atrigila · 2024-08-23T19:14:54Z

.github/workflows/ci.yml

@@ -10,6 +10,8 @@ on:

 env:
  NXF_ANSI_LOG: false
+  CAPSULE_LOG: none
+  NFTEST_VER: "0.8.4"


snapshots seem to have been done with 0.9.0
maybe it is worth updating to 0.9.0 but keeping out the sharding?

zxBIB Schcolnicov added 2 commits August 19, 2024 16:57

Initialized tests

82bea4d

Pulled from dev

409b1fc

zxBIB Schcolnicov and others added 4 commits August 19, 2024 19:31

Added techincalrepeats nftest and updated nf-test config

a574696

Updated tests and added snaps

70a62c6

Pulled from dev

acd4ce6

Minimal profiles for testing ci settings

f64efd0

nschcolnicov force-pushed the add_ci_nftests branch from 1fe5b29 to f64efd0 Compare August 20, 2024 19:55

nschcolnicov commented Aug 20, 2024

View reviewed changes

zxBIB Schcolnicov added 10 commits August 21, 2024 23:24

Merge branch 'dev' of https://github.com/nf-core/smrnaseq into add_ci…

de50116

…_nftests

Standardized changelog and generated nf-tests

38521e9

Updated tests and snaps

fd3405d

Updated test_index test

b598f73

Updated tests to split snapshot by subdirectories

0f8ab7f

Updated snaps

e307076

Updated test_index to work in CI

5638e24

Merge branch 'dev' of https://github.com/nf-core/smrnaseq into add_ci…

eae5055

…_nftests

Adapted params to work with ingested datasets

003eb34

Fixed pre-commit

91d18a0

nschcolnicov marked this pull request as ready for review August 23, 2024 17:08

nschcolnicov requested review from apeltzer and atrigila August 23, 2024 17:08

atrigila approved these changes Aug 23, 2024

View reviewed changes

nschcolnicov and others added 2 commits August 24, 2024 18:30

Using nf-test version 0.9.0 in ci

46ed8de

Pulled from dev

4a73a71

apeltzer approved these changes Aug 26, 2024

View reviewed changes

Added skipfastp nftest

92c289d

nschcolnicov force-pushed the add_ci_nftests branch from afa0ae8 to 92c289d Compare August 26, 2024 12:16

Updated snaps

e843703

nschcolnicov merged commit 0a39d56 into nf-core:dev Aug 26, 2024
16 checks passed

nschcolnicov mentioned this pull request Aug 26, 2024

Convert tests to nf-tests #379

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add ci nftests #381

Add ci nftests #381

nschcolnicov commented Aug 19, 2024

github-actions bot commented Aug 19, 2024 •

edited

Loading

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

nschcolnicov Aug 20, 2024

atrigila Aug 23, 2024

atrigila left a comment

atrigila Aug 23, 2024

Add ci nftests #381

Add ci nftests #381

Conversation

nschcolnicov commented Aug 19, 2024

PR checklist

github-actions bot commented Aug 19, 2024 • edited Loading

nf-core lint overall result: Passed ✅ ⚠️

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

nschcolnicov Aug 20, 2024

Choose a reason for hiding this comment

atrigila Aug 23, 2024

Choose a reason for hiding this comment

atrigila left a comment

Choose a reason for hiding this comment

atrigila Aug 23, 2024

Choose a reason for hiding this comment

github-actions bot commented Aug 19, 2024 •

edited

Loading

`nf-core lint` overall result: Passed ✅ ⚠️