Feature/exe 1732 update to pixelator 0.17.1 #90

johandahlberg · 2024-05-27T12:54:35Z

Updates the pipeline to use pixelator 0.17.1 and add a module for the pixelator single-cell layout command.

PR checklist

fbdtemme

Looks good! Just one thing missing:
We need to pass the layout report_json and metadata channels from PIXELATOR_LAYOUT to GENERATE_REPORTS as well similar to how we do it for all the other stages.
Right now the layout step will be missing from the qc report.

johandahlberg · 2024-05-27T13:55:07Z

Super. I have pushed a fix now.

github-actions · 2024-05-27T13:55:57Z

`nf-core lint` overall result: Failed ❌

Posted for pipeline commit 1e557b2

+| ✅ 197 tests passed       |+
#| ❔   5 tests were ignored |#
!| ❗   5 tests had warnings |!
-| ❌   2 tests failed       |-

❌ Test failures:

files_unchanged - .github/workflows/linting_comment.yml does not match the template
files_unchanged - .github/workflows/linting.yml does not match the template

❗ Test warnings:

nextflow_config - Config manifest.version should end in dev: 1.1.0
pipeline_todos - TODO string in main.nf: Optionally add in-text citation tools to this list.
pipeline_todos - TODO string in main.nf: Optionally add bibliographic entries to this list.
pipeline_todos - TODO string in main.nf: Only uncomment below if logic in toolCitationText/toolBibliographyText has been filled!
nfcore_yml - nf-core version not set in .nf-core.yml

❔ Tests ignored:

files_exist - File is ignored: assets/multiqc_config.yml
files_exist - File is ignored: conf/igenomes.config
files_unchanged - File ignored due to lint config: .github/PULL_REQUEST_TEMPLATE.md
files_unchanged - File ignored due to lint config: .gitignore or .prettierignore
multiqc_config - multiqc_config

✅ Tests passed:

files_exist - File found: .gitattributes
files_exist - File found: .gitignore
files_exist - File found: .nf-core.yml
files_exist - File found: .editorconfig
files_exist - File found: .prettierignore
files_exist - File found: .prettierrc.yml
files_exist - File found: CHANGELOG.md
files_exist - File found: CITATIONS.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_exist - File found: nextflow_schema.json
files_exist - File found: nextflow.config
files_exist - File found: README.md
files_exist - File found: .github/.dockstore.yml
files_exist - File found: .github/CONTRIBUTING.md
files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
files_exist - File found: .github/ISSUE_TEMPLATE/config.yml
files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
files_exist - File found: .github/workflows/branch.yml
files_exist - File found: .github/workflows/ci.yml
files_exist - File found: .github/workflows/linting_comment.yml
files_exist - File found: .github/workflows/linting.yml
files_exist - File found: assets/email_template.html
files_exist - File found: assets/email_template.txt
files_exist - File found: assets/sendmail_template.txt
files_exist - File found: assets/nf-core-pixelator_logo_light.png
files_exist - File found: conf/modules.config
files_exist - File found: conf/test.config
files_exist - File found: conf/test_full.config
files_exist - File found: docs/images/nf-core-pixelator_logo_light.png
files_exist - File found: docs/images/nf-core-pixelator_logo_dark.png
files_exist - File found: docs/output.md
files_exist - File found: docs/README.md
files_exist - File found: docs/README.md
files_exist - File found: docs/usage.md
files_exist - File found: main.nf
files_exist - File found: conf/base.config
files_exist - File found: .github/workflows/awstest.yml
files_exist - File found: .github/workflows/awsfulltest.yml
files_exist - File found: modules.json
files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
files_exist - File not found check: .github/workflows/push_dockerhub.yml
files_exist - File not found check: .markdownlint.yml
files_exist - File not found check: .nf-core.yaml
files_exist - File not found check: .yamllint.yml
files_exist - File not found check: bin/markdown_to_html.r
files_exist - File not found check: conf/aws.config
files_exist - File not found check: docs/images/nf-core-pixelator_logo.png
files_exist - File not found check: lib/Checks.groovy
files_exist - File not found check: lib/Completion.groovy
files_exist - File not found check: lib/NfcoreTemplate.groovy
files_exist - File not found check: lib/Utils.groovy
files_exist - File not found check: lib/Workflow.groovy
files_exist - File not found check: lib/WorkflowMain.groovy
files_exist - File not found check: lib/WorkflowPixelator.groovy
files_exist - File not found check: parameters.settings.json
files_exist - File not found check: pipeline_template.yml
files_exist - File not found check: Singularity
files_exist - File not found check: lib/nfcore_external_java_deps.jar
files_exist - File not found check: .travis.yml
nextflow_config - Config variable found: manifest.name
nextflow_config - Config variable found: manifest.nextflowVersion
nextflow_config - Config variable found: manifest.description
nextflow_config - Config variable found: manifest.version
nextflow_config - Config variable found: manifest.homePage
nextflow_config - Config variable found: timeline.enabled
nextflow_config - Config variable found: trace.enabled
nextflow_config - Config variable found: report.enabled
nextflow_config - Config variable found: dag.enabled
nextflow_config - Config variable found: process.cpus
nextflow_config - Config variable found: process.memory
nextflow_config - Config variable found: process.time
nextflow_config - Config variable found: params.outdir
nextflow_config - Config variable found: params.input
nextflow_config - Config variable found: params.validationShowHiddenParams
nextflow_config - Config variable found: params.validationSchemaIgnoreParams
nextflow_config - Config variable found: manifest.mainScript
nextflow_config - Config variable found: timeline.file
nextflow_config - Config variable found: trace.file
nextflow_config - Config variable found: report.file
nextflow_config - Config variable found: dag.file
nextflow_config - Config variable (correctly) not found: params.nf_required_version
nextflow_config - Config variable (correctly) not found: params.container
nextflow_config - Config variable (correctly) not found: params.singleEnd
nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
nextflow_config - Config variable (correctly) not found: params.name
nextflow_config - Config variable (correctly) not found: params.enable_conda
nextflow_config - Config timeline.enabled had correct value: true
nextflow_config - Config report.enabled had correct value: true
nextflow_config - Config trace.enabled had correct value: true
nextflow_config - Config dag.enabled had correct value: true
nextflow_config - Config manifest.name began with nf-core/
nextflow_config - Config variable manifest.homePage began with https://github.com/nf-core/
nextflow_config - Config dag.file ended with .html
nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
nextflow_config - Config params.custom_config_version is set to master
nextflow_config - Config params.custom_config_base is set to https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Lines for loading custom profiles found
nextflow_config - nextflow.config contains configuration profile test
nextflow_config - Config default value correct: params.trim_front= 0
nextflow_config - Config default value correct: params.trim_tail= 0
nextflow_config - Config default value correct: params.max_n_bases= 0
nextflow_config - Config default value correct: params.avg_qual= 20
nextflow_config - Config default value correct: params.adapterqc_mismatches= 0.1
nextflow_config - Config default value correct: params.demux_mismatches= 0.1
nextflow_config - Config default value correct: params.algorithm= adjacency
nextflow_config - Config default value correct: params.max_neighbours= 60
nextflow_config - Config default value correct: params.collapse_mismatches= 2
nextflow_config - Config default value correct: params.collapse_min_count= 2
nextflow_config - Config default value correct: params.multiplet_recovery= true
nextflow_config - Config default value correct: params.leiden_iterations= 10
nextflow_config - Config default value correct: params.graph_min_count= 2
nextflow_config - Config default value correct: params.dynamic_filter= min
nextflow_config - Config default value correct: params.aggregate_calling= true
nextflow_config - Config default value correct: params.compute_polarization= true
nextflow_config - Config default value correct: params.compute_colocalization= true
nextflow_config - Config default value correct: params.polarization_transformation= log1p
nextflow_config - Config default value correct: params.polarization_n_permutations= 50
nextflow_config - Config default value correct: params.polarization_min_marker_count= 5
nextflow_config - Config default value correct: params.colocalization_transformation= log1p
nextflow_config - Config default value correct: params.colocalization_neighbourhood_size= 1
nextflow_config - Config default value correct: params.colocalization_n_permutations= 50
nextflow_config - Config default value correct: params.colocalization_min_region_count= 5
nextflow_config - Config default value correct: params.no_node_marker_counts= false
nextflow_config - Config default value correct: params.layout_algorithm= pmds_3d
nextflow_config - Config default value correct: params.custom_config_version= master
nextflow_config - Config default value correct: params.custom_config_base= https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Config default value correct: params.max_cpus= 16
nextflow_config - Config default value correct: params.max_memory= 128.GB
nextflow_config - Config default value correct: params.max_time= 240.h
nextflow_config - Config default value correct: params.publish_dir_mode= copy
nextflow_config - Config default value correct: params.validate_params= true
files_unchanged - .gitattributes matches the template
files_unchanged - .prettierrc.yml matches the template
files_unchanged - CODE_OF_CONDUCT.md matches the template
files_unchanged - LICENSE matches the template
files_unchanged - .github/.dockstore.yml matches the template
files_unchanged - .github/CONTRIBUTING.md matches the template
files_unchanged - .github/ISSUE_TEMPLATE/bug_report.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/config.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
files_unchanged - .github/workflows/branch.yml matches the template
files_unchanged - assets/email_template.html matches the template
files_unchanged - assets/email_template.txt matches the template
files_unchanged - assets/sendmail_template.txt matches the template
files_unchanged - assets/nf-core-pixelator_logo_light.png matches the template
files_unchanged - docs/images/nf-core-pixelator_logo_light.png matches the template
files_unchanged - docs/images/nf-core-pixelator_logo_dark.png matches the template
files_unchanged - docs/README.md matches the template
actions_ci - '.github/workflows/ci.yml' is triggered on expected events
actions_ci - '.github/workflows/ci.yml' checks minimum NF version
actions_awstest - '.github/workflows/awstest.yml' is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml does not use -profile test
readme - README Nextflow minimum version badge matched config. Badge: 23.04.0, Config: 23.04.0
readme - README Zenodo placeholder was replaced with DOI.
pipeline_name_conventions - Name adheres to nf-core convention
template_strings - Did not find any Jinja template strings (107 files)
schema_lint - Schema lint passed
schema_lint - Schema title + description lint passed
schema_lint - Input mimetype lint passed: 'text/csv'
schema_params - Schema matched params returned from nextflow config
system_exit - No System.exit calls found
actions_schema_validation - Workflow validation passed: branch.yml
actions_schema_validation - Workflow validation passed: ci.yml
actions_schema_validation - Workflow validation passed: awsfulltest.yml
actions_schema_validation - Workflow validation passed: fix-linting.yml
actions_schema_validation - Workflow validation passed: linting.yml
actions_schema_validation - Workflow validation passed: download_pipeline.yml
actions_schema_validation - Workflow validation passed: release-announcements.yml
actions_schema_validation - Workflow validation passed: clean-up.yml
actions_schema_validation - Workflow validation passed: awstest.yml
actions_schema_validation - Workflow validation passed: linting_comment.yml
actions_schema_validation - Workflow validation passed: release-announcments.yml
merge_markers - No merge markers found in pipeline files
modules_json - Only installed modules found in modules.json
modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'
base_config - conf/base.config found and not ignored.
base_config - CUSTOM_DUMPSOFTWAREVERSIONS found in conf/base.config and Nextflow scripts.
modules_config - conf/modules.config found and not ignored.
modules_config - PIXELATOR found in conf/modules.config and Nextflow scripts.
modules_config - PIXELATOR_LIST_OPTIONS found in conf/modules.config and Nextflow scripts.
modules_config - CAT_FASTQ found in conf/modules.config and Nextflow scripts.
modules_config - RENAME_READS found in conf/modules.config and Nextflow scripts.
modules_config - PIXELATOR_AMPLICON found in conf/modules.config and Nextflow scripts.
modules_config - PIXELATOR_QC found in conf/modules.config and Nextflow scripts.
modules_config - PIXELATOR_DEMUX found in conf/modules.config and Nextflow scripts.
modules_config - PIXELATOR_COLLAPSE found in conf/modules.config and Nextflow scripts.
modules_config - PIXELATOR_GRAPH found in conf/modules.config and Nextflow scripts.
modules_config - PIXELATOR_ANNOTATE found in conf/modules.config and Nextflow scripts.
modules_config - PIXELATOR_ANALYSIS found in conf/modules.config and Nextflow scripts.
modules_config - PIXELATOR_LAYOUT found in conf/modules.config and Nextflow scripts.
modules_config - PIXELATOR_REPORT found in conf/modules.config and Nextflow scripts.
modules_config - CUSTOM_DUMPSOFTWAREVERSIONS found in conf/modules.config and Nextflow scripts.
nfcore_yml - Repository type in .nf-core.yml is valid: pipeline

Run details

nf-core/tools version 2.14.1
Run at 2024-05-27 14:32:27

workflows/pixelator.nf

fbdtemme

Ready to go!

ambarrio · 2024-05-28T04:23:40Z

CHANGELOG.md

@@ -3,6 +3,26 @@
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

+## [[1.2.0](https://github.com/nf-core/pixelator/releases/tag/1.1.0)] - 2024-05-??


I think the link is outdated on this, right?

Yes, good catch!

ambarrio · 2024-05-28T04:28:17Z

docs/output.md

@@ -168,7 +168,7 @@ newly recovered components are stored in a file (components_recovered.csv).
 - `pixelator`

  - `annotate`
-    - `<sample-id>.dataset.pxl`
+    - `<sample-id>.annotated.dataset.pxl`


annotated -> annotate

Let's use the stage name instead of a participle.

Your right, it should be annotate.

ambarrio · 2024-05-28T04:29:39Z

docs/output.md

@@ -168,7 +168,7 @@ newly recovered components are stored in a file (components_recovered.csv).
 - `pixelator`

  - `annotate`
-    - `<sample-id>.dataset.pxl`
+    - `<sample-id>.annotated.dataset.pxl`
    - `<sample-id>.meta.json`: Command invocation metadata.
    - `<sample-id>.rank_vs_size.png`


These PNG files are not produced AFAIK

ambarrio · 2024-05-28T04:30:16Z

docs/output.md

@@ -168,7 +168,7 @@ newly recovered components are stored in a file (components_recovered.csv).
 - `pixelator`

  - `annotate`
-    - `<sample-id>.dataset.pxl`
+    - `<sample-id>.annotated.dataset.pxl`
    - `<sample-id>.meta.json`: Command invocation metadata.
    - `<sample-id>.rank_vs_size.png`
    - `<sample-id>.raw_components_metrics.csv`


These are csv.gz instead

Good catch! I will update all this things in an incoming PR.

johandahlberg added 6 commits May 27, 2024 14:46

Update to use pixelator 0.17.1, add layout command

70fe195

Make graph step use process_high resource

e7d1d58

Formatting fix

99275c6

Skip layout in small test config

edab0dd

Clean-up

4a28425

Update changelog

07b0500

johandahlberg marked this pull request as ready for review May 27, 2024 13:08

johandahlberg requested a review from fbdtemme May 27, 2024 13:08

fbdtemme reviewed May 27, 2024

View reviewed changes

johandahlberg added 2 commits May 27, 2024 15:44

Fix linting issue

1d2fcde

Pass layout to the report stage

bd29d2c

fbdtemme reviewed May 27, 2024

View reviewed changes

workflows/pixelator.nf Outdated Show resolved Hide resolved

johandahlberg added 3 commits May 27, 2024 16:01

Fix copy-paste-error

1db4263

annotate step use process_high resources

ce90f8b

Clean-up the changelog a bit

1e557b2

fbdtemme approved these changes May 27, 2024

View reviewed changes

johandahlberg merged commit 7a769d9 into nf-core:dev May 27, 2024
3 of 4 checks passed

ambarrio reviewed May 28, 2024

View reviewed changes

fbdtemme deleted the feature/exe-1732-update-to-pixelator-0.17.1 branch May 29, 2024 15:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature/exe 1732 update to pixelator 0.17.1 #90

Feature/exe 1732 update to pixelator 0.17.1 #90

johandahlberg commented May 27, 2024

fbdtemme left a comment

johandahlberg commented May 27, 2024

github-actions bot commented May 27, 2024 •

edited

Loading

❌ Test failures:

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

fbdtemme left a comment

ambarrio May 28, 2024

johandahlberg May 28, 2024

ambarrio May 28, 2024

johandahlberg May 28, 2024

ambarrio May 28, 2024

johandahlberg May 28, 2024

ambarrio May 28, 2024

johandahlberg May 28, 2024

Feature/exe 1732 update to pixelator 0.17.1 #90

Feature/exe 1732 update to pixelator 0.17.1 #90

Conversation

johandahlberg commented May 27, 2024

PR checklist

fbdtemme left a comment

Choose a reason for hiding this comment

johandahlberg commented May 27, 2024

github-actions bot commented May 27, 2024 • edited Loading

nf-core lint overall result: Failed ❌

❌ Test failures:

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

fbdtemme left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented May 27, 2024 •

edited

Loading

`nf-core lint` overall result: Failed ❌