Second attempt at updating all modules #455

maxulysse · 2021-11-22T13:35:25Z

Since I'm going crazy with #440
This is another attempt at it starting from scratch with tiny steps to figure out when and where and what and maybe why it is failing.

github-actions · 2021-11-22T13:50:14Z

`nf-core lint` overall result: Passed ✅ ⚠️

Posted for pipeline commit 76df2dc

+| ✅ 133 tests passed       |+
#| ❔   5 tests were ignored |#
!| ❗   5 tests had warnings |!

❗ Test warnings:

pipeline_todos - TODO string in awsfulltest.yml: You can customise AWS full pipeline tests as required
pipeline_todos - TODO string in test_full.config: Specify the paths to your full test data ( on nf-core/test-datasets or directly in repositories, e.g. SRA)
pipeline_todos - TODO string in test_full.config: Give any required params for the test so that command line flags are not needed
pipeline_todos - TODO string in test_full.config: Specify the paths to your full test data ( on nf-core/test-datasets or directly in repositories, e.g. SRA)
pipeline_todos - TODO string in test_full.config: Give any required params for the test so that command line flags are not needed

❔ Tests ignored:

files_unchanged - File ignored due to lint config: .github/workflows/linting.yml
files_unchanged - File ignored due to lint config: assets/nf-core-sarek_logo.png
files_unchanged - File ignored due to lint config: docs/images/nf-core-sarek_logo.png
files_unchanged - File ignored due to lint config: lib/NfcoreTemplate.groovy
files_unchanged - File ignored due to lint config: assets/multiqc_config.yaml

✅ Tests passed:

files_exist - File found: .gitattributes
files_exist - File found: .gitignore
files_exist - File found: .markdownlint.yml
files_exist - File found: CHANGELOG.md
files_exist - File found: CITATIONS.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_exist - File found: nextflow_schema.json
files_exist - File found: nextflow.config
files_exist - File found: README.md
files_exist - File found: .github/.dockstore.yml
files_exist - File found: .github/CONTRIBUTING.md
files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
files_exist - File found: .github/ISSUE_TEMPLATE/config.yml
files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
files_exist - File found: .github/workflows/branch.yml
files_exist - File found: .github/workflows/ci.yml
files_exist - File found: .github/workflows/linting_comment.yml
files_exist - File found: .github/workflows/linting.yml
files_exist - File found: assets/email_template.html
files_exist - File found: assets/email_template.txt
files_exist - File found: assets/sendmail_template.txt
files_exist - File found: assets/nf-core-sarek_logo.png
files_exist - File found: conf/modules.config
files_exist - File found: conf/test.config
files_exist - File found: conf/test_full.config
files_exist - File found: docs/images/nf-core-sarek_logo.png
files_exist - File found: docs/output.md
files_exist - File found: docs/README.md
files_exist - File found: docs/README.md
files_exist - File found: docs/usage.md
files_exist - File found: lib/nfcore_external_java_deps.jar
files_exist - File found: lib/NfcoreSchema.groovy
files_exist - File found: lib/NfcoreTemplate.groovy
files_exist - File found: lib/Utils.groovy
files_exist - File found: lib/WorkflowMain.groovy
files_exist - File found: main.nf
files_exist - File found: assets/multiqc_config.yaml
files_exist - File found: conf/base.config
files_exist - File found: conf/igenomes.config
files_exist - File found: .github/workflows/awstest.yml
files_exist - File found: .github/workflows/awsfulltest.yml
files_exist - File found: lib/WorkflowSarek.groovy
files_exist - File found: modules.json
files_exist - File not found check: Singularity
files_exist - File not found check: parameters.settings.json
files_exist - File not found check: bin/markdown_to_html.r
files_exist - File not found check: conf/aws.config
files_exist - File not found check: .github/workflows/push_dockerhub.yml
files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
files_exist - File not found check: .travis.yml
nextflow_config - Config variable found: manifest.name
nextflow_config - Config variable found: manifest.nextflowVersion
nextflow_config - Config variable found: manifest.description
nextflow_config - Config variable found: manifest.version
nextflow_config - Config variable found: manifest.homePage
nextflow_config - Config variable found: timeline.enabled
nextflow_config - Config variable found: trace.enabled
nextflow_config - Config variable found: report.enabled
nextflow_config - Config variable found: dag.enabled
nextflow_config - Config variable found: process.cpus
nextflow_config - Config variable found: process.memory
nextflow_config - Config variable found: process.time
nextflow_config - Config variable found: params.outdir
nextflow_config - Config variable found: params.input
nextflow_config - Config variable found: params.show_hidden_params
nextflow_config - Config variable found: params.schema_ignore_params
nextflow_config - Config variable found: manifest.mainScript
nextflow_config - Config variable found: timeline.file
nextflow_config - Config variable found: trace.file
nextflow_config - Config variable found: report.file
nextflow_config - Config variable found: dag.file
nextflow_config - Config variable (correctly) not found: params.version
nextflow_config - Config variable (correctly) not found: params.nf_required_version
nextflow_config - Config variable (correctly) not found: params.container
nextflow_config - Config variable (correctly) not found: params.singleEnd
nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
nextflow_config - Config variable (correctly) not found: params.name
nextflow_config - Config timeline.enabled had correct value: true
nextflow_config - Config report.enabled had correct value: true
nextflow_config - Config trace.enabled had correct value: true
nextflow_config - Config dag.enabled had correct value: true
nextflow_config - Config manifest.name began with nf-core/
nextflow_config - Config variable manifest.homePage began with https://github.com/nf-core/
nextflow_config - Config dag.file ended with .svg
nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
nextflow_config - Config manifest.version ends in dev: '3.0dev'
nextflow_config - Config params.custom_config_version is set to master
nextflow_config - Config params.custom_config_base is set to https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Lines for loading custom profiles found
files_unchanged - .gitattributes matches the template
files_unchanged - .markdownlint.yml matches the template
files_unchanged - CODE_OF_CONDUCT.md matches the template
files_unchanged - LICENSE matches the template
files_unchanged - .github/.dockstore.yml matches the template
files_unchanged - .github/CONTRIBUTING.md matches the template
files_unchanged - .github/ISSUE_TEMPLATE/bug_report.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/config.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
files_unchanged - .github/PULL_REQUEST_TEMPLATE.md matches the template
files_unchanged - .github/workflows/branch.yml matches the template
files_unchanged - .github/workflows/linting_comment.yml matches the template
files_unchanged - assets/email_template.html matches the template
files_unchanged - assets/email_template.txt matches the template
files_unchanged - assets/sendmail_template.txt matches the template
files_unchanged - docs/README.md matches the template
files_unchanged - lib/nfcore_external_java_deps.jar matches the template
files_unchanged - lib/NfcoreSchema.groovy matches the template
files_unchanged - .gitignore matches the template
actions_ci - '.github/workflows/ci.yml' is triggered on expected events
actions_ci - '.github/workflows/ci.yml' checks minimum NF version
actions_awstest - '.github/workflows/awstest.yml' is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml does not use -profile test
readme - README Nextflow minimum version badge matched config. Badge: 21.04.0, Config: 21.04.0
readme - README Nextflow minimum version in Quick Start section matched config. README: 21.04.0, Config: 21.04.0
pipeline_name_conventions - Name adheres to nf-core convention
template_strings - Did not find any Jinja template strings (291 files)
schema_lint - Schema lint passed
schema_lint - Schema title + description lint passed
schema_params - Schema matched params returned from nextflow config
actions_schema_validation - Workflow validation passed: branch.yml
actions_schema_validation - Workflow validation passed: local_modules.yml
actions_schema_validation - Workflow validation passed: ci.yml
actions_schema_validation - Workflow validation passed: linting_comment.yml
actions_schema_validation - Workflow validation passed: linting.yml
actions_schema_validation - Workflow validation passed: awstest.yml
actions_schema_validation - Workflow validation passed: awsfulltest.yml
merge_markers - No merge markers found in pipeline files
modules_json - Only installed modules found in modules.json

Run details

nf-core/tools version 2.2.dev0
Run at 2021-11-24 16:44:47

maxulysse · 2021-11-23T13:57:26Z

.github/workflows/linting.yml

@@ -120,7 +120,7 @@ jobs:
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
-          pip install nf-core
+          pip install nf-core git+https://github.com/nf-core/tools.git@dev


Just adding that to have the linting from dev

makes sense

not sure that @drpatelh would approve

maxulysse · 2021-11-23T13:59:07Z

conf/test.config

@@ -42,7 +42,7 @@ profiles {
        params.step                = 'annotate'
    }
    pair {
-        params.input               = 'https://raw.githubusercontent.com/nf-core/test-datasets/sarek/testdata/csv/tiny-manta-https.csv'
+        params.input               = "${baseDir}/tests/csv/3.0/fastq_pair.csv"
    }
    prepare_recalibration {
        params.input               = 'https://raw.githubusercontent.com/nf-core/test-datasets/sarek/testdata/csv/tiny-mapped-normal-https.csv'


Need to update for the tests

tests/test_save_bam_mapped.yml

FriederikeHanssen

Amazing ❤️

FriederikeHanssen · 2021-11-24T16:56:08Z

.github/workflows/linting.yml

@@ -120,7 +120,7 @@ jobs:
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
-          pip install nf-core
+          pip install nf-core git+https://github.com/nf-core/tools.git@dev


makes sense

maxulysse added 2 commits November 22, 2021 14:33

feat: update test data

76c0d89

fix: using tools dev for linting

dace03e

maxulysse changed the title ~~feat: update test data~~ Second attempt at updating all modules Nov 22, 2021

maxulysse added 3 commits November 22, 2021 14:46

fix: linting

badee21

fix lint

de70e68

sync template for lint

206c3f5

maxulysse added 3 commits November 22, 2021 15:17

fix: update tests

72e31df

fix tests

f84a294

remove part of tests for now

f8bfab0

maxulysse commented Nov 23, 2021

View reviewed changes

tests/test_save_bam_mapped.yml Outdated Show resolved Hide resolved

maxulysse added 17 commits November 23, 2021 15:40

fix: save bam and bai

4302f13

feat: update markduplicates

0024e06

feat: update modules bwa/index

67751f6

feat: update modules bwamem2/index

db42a4d

feat: update modules gatk4/createsequencedictionary

76f575a

feat: update more modules

b91d9af

fix: update syntax for samtools/merge

a105909

fix: update output for annotation subworkflows

8d37904

feat: update subworkflows

9aec3ff

Merge branch 'dev' into dev_tools_update_2

109f472

fix: input files order more consistent

df8fa7e

install modules qualimap/bamqc

e750ec2

feat: install modules samtools/stats

ad9683b

update samtools/view

8f65bc5

feat: update samtools/index

da8453e

chores: update local modules

1602c68

fix: update output for versions

b70a44f

maxulysse added 4 commits November 24, 2021 17:21

chores: updates more subworkflows

16ccc5c

chores: update more modules

a109e03

fix: nf-core lint + EClint

b36aae7

fix: output

76df2dc

maxulysse added the Ready for Review label Nov 24, 2021

maxulysse requested a review from FriederikeHanssen November 24, 2021 16:51

FriederikeHanssen approved these changes Nov 24, 2021

View reviewed changes

maxulysse merged commit d01122b into nf-core:dev Nov 25, 2021

maxulysse deleted the dev_tools_update_2 branch November 25, 2021 09:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Second attempt at updating all modules #455

Second attempt at updating all modules #455

maxulysse commented Nov 22, 2021

github-actions bot commented Nov 22, 2021 •

edited

Loading

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

maxulysse Nov 23, 2021

FriederikeHanssen Nov 24, 2021

maxulysse Nov 25, 2021

maxulysse Nov 23, 2021

FriederikeHanssen left a comment

FriederikeHanssen Nov 24, 2021

Second attempt at updating all modules #455

Second attempt at updating all modules #455

Conversation

maxulysse commented Nov 22, 2021

github-actions bot commented Nov 22, 2021 • edited Loading

nf-core lint overall result: Passed ✅ ⚠️

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

maxulysse Nov 23, 2021

Choose a reason for hiding this comment

FriederikeHanssen Nov 24, 2021

Choose a reason for hiding this comment

maxulysse Nov 25, 2021

Choose a reason for hiding this comment

maxulysse Nov 23, 2021

Choose a reason for hiding this comment

FriederikeHanssen left a comment

Choose a reason for hiding this comment

FriederikeHanssen Nov 24, 2021

Choose a reason for hiding this comment

github-actions bot commented Nov 22, 2021 •

edited

Loading

`nf-core lint` overall result: Passed ✅ ⚠️