Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Joint calling: Fix meta map info for proper joining #1220

Merged
merged 7 commits into from
Sep 12, 2023
Merged

Joint calling: Fix meta map info for proper joining #1220

merged 7 commits into from
Sep 12, 2023

Conversation

FriederikeHanssen
Copy link
Contributor

PR checklist

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - have you followed the pipeline conventions in the contribution docs
  • If necessary, also make a PR on the nf-core/sarek branch on the nf-core/test-datasets repository.
  • Make sure your code lints (nf-core lint).
  • Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
  • Usage Documentation in docs/usage.md is updated.
  • Output Documentation in docs/output.md is updated.
  • CHANGELOG.md is updated.
  • README.md is updated (including new tool citations and authors/contributors).

@github-actions
Copy link

github-actions bot commented Sep 10, 2023 •
edited
Loading

nf-core lint overall result: Passed ✅ ⚠️

Posted for pipeline commit 5ea0e57

+| ✅ 140 tests passed       |+
#| ❔   9 tests were ignored |#
!| ❗   4 tests had warnings |!

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

  • nf-core/tools version 2.9
  • Run at 2023-09-12 07:57:38

@FriederikeHanssen
Copy link
Contributor Author

FriederikeHanssen commented Sep 10, 2023 •
edited
Loading 

(nf-core) [iizha01@rddsws sarek]$ pytest --tag mutect2_manual --symlink --keep-workflow-wd --git-aware
========================================================================================================================================================================== test session starts ===========================================================================================================================================================================
platform linux -- Python 3.6.13, pytest-6.2.3, py-1.11.0, pluggy-0.13.1
rootdir: /home/iizha01/sarek
plugins: workflow-1.6.0
collecting ... 
collected 132 items                                                                                                                                                                                                                                                                                                                                                      

Run variant calling on somatic sample with mutect2 without intervals:
        command:   nextflow run main.nf -profile test_cache,tools_somatic --tools mutect2 --no_intervals --outdir results
        directory: /tmp/pytest_workflow_psfie_ie/Run_variant_calling_on_somatic_sample_with_mutect2_without_intervals
        stdout:    /tmp/pytest_workflow_psfie_ie/Run_variant_calling_on_somatic_sample_with_mutect2_without_intervals/log.out
        stderr:    /tmp/pytest_workflow_psfie_ie/Run_variant_calling_on_somatic_sample_with_mutect2_without_intervals/log.err
'Run variant calling on somatic sample with mutect2 without intervals' done.

Run variant calling on somatic sample with mutect2:
        command:   nextflow run main.nf -profile test_cache,tools_somatic --tools mutect2 --outdir results
        directory: /tmp/pytest_workflow_psfie_ie/Run_variant_calling_on_somatic_sample_with_mutect2
        stdout:    /tmp/pytest_workflow_psfie_ie/Run_variant_calling_on_somatic_sample_with_mutect2/log.out
        stderr:    /tmp/pytest_workflow_psfie_ie/Run_variant_calling_on_somatic_sample_with_mutect2/log.err
'Run variant calling on somatic sample with mutect2' done.

Run joint calling on tumor only samples with mutect2:
        command:   nextflow run main.nf -profile test_cache,tools_tumoronly --input tests/csv/3.0/recalibrated_tumoronly_joint.csv --tools mutect2 --joint_mutect2 --outdir results
        directory: /tmp/pytest_workflow_psfie_ie/Run_joint_calling_on_tumor_only_samples_with_mutect2
        stdout:    /tmp/pytest_workflow_psfie_ie/Run_joint_calling_on_tumor_only_samples_with_mutect2/log.out
        stderr:    /tmp/pytest_workflow_psfie_ie/Run_joint_calling_on_tumor_only_samples_with_mutect2/log.err
'Run joint calling on tumor only samples with mutect2' done.

Run joint calling on somatic samples with mutect2:
        command:   nextflow run main.nf -profile test_cache,tools_somatic --input tests/csv/3.0/recalibrated_somatic_joint.csv --tools mutect2 --joint_mutect2 --outdir results
        directory: /tmp/pytest_workflow_psfie_ie/Run_joint_calling_on_somatic_samples_with_mutect2
        stdout:    /tmp/pytest_workflow_psfie_ie/Run_joint_calling_on_somatic_samples_with_mutect2/log.out
        stderr:    /tmp/pytest_workflow_psfie_ie/Run_joint_calling_on_somatic_samples_with_mutect2/log.err
'Run joint calling on somatic samples with mutect2' done.

tests/test_tools_manually.yml ....................................................................................................................................                                                                                                                                                                                                 [100%] Keeping temporary directories and logs. Use '--kwd' or '--keep-workflow-wd' to disable this behaviour.


==================================================================================================================================================================== 132 passed in 3521.18s (0:58:41) ====================================================================================================================================================================

@asp8200
Copy link
Contributor

asp8200 commented Sep 11, 2023

Concerning the test, are you running it with docker or singularity?

@FriederikeHanssen
Copy link
Contributor Author

Docker.

For the resulting csv for the joint calling, we have for both (somatic and tumor-only):

patient,sample,variantcaller,vcf
test,test,mutect2,results/variant_calling/mutect2/test/test.mutect2.filtered.vcf.gz

since we loose the normal_id eventually. Might be worth considering to have patient_normalID for somatic.

@FriederikeHanssen FriederikeHanssen marked this pull request as ready for review September 11, 2023 07:26
maxulysse
maxulysse reviewed Sep 11, 2023
View reviewed changes
subworkflows/local/bam_variant_calling_somatic_mutect2/main.nf Outdated
Comment on lines 46 to 50
ch_cram = input.multiMap{ meta, cram, crai ->
normal: [ meta - meta.subMap('normal_id', 'tumor_id') , cram[0], crai[0] ]
tumor: [ meta - meta.subMap('normal_id', 'tumor_id') , cram[1], crai[1] ]
}
ch_cram_normal = ch_cram.normal.unique()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

<3

@FriederikeHanssen
Copy link
Contributor Author 

=============================================================================================================================================
(nf-core) [iizha01@rddsws sarek]$ pytest --tag mutect2_manual --symlink --keep-workflow-wd --git-aware
=============================== test session starts ===============================
platform linux -- Python 3.6.13, pytest-6.2.3, py-1.11.0, pluggy-0.13.1
rootdir: /home/iizha01/sarek
plugins: workflow-1.6.0
collecting ... 
collected 132 items                                                               

Run variant calling on somatic sample with mutect2 without intervals:
        command:   nextflow run main.nf -profile test_cache,tools_somatic --tools mutect2 --no_intervals --outdir results
        directory: /tmp/pytest_workflow_kh4i1idl/Run_variant_calling_on_somatic_sample_with_mutect2_without_intervals
        stdout:    /tmp/pytest_workflow_kh4i1idl/Run_variant_calling_on_somatic_sample_with_mutect2_without_intervals/log.out
        stderr:    /tmp/pytest_workflow_kh4i1idl/Run_variant_calling_on_somatic_sample_with_mutect2_without_intervals/log.err
'Run variant calling on somatic sample with mutect2 without intervals' done.

Run variant calling on somatic sample with mutect2:
        command:   nextflow run main.nf -profile test_cache,tools_somatic --tools mutect2 --outdir results
        directory: /tmp/pytest_workflow_kh4i1idl/Run_variant_calling_on_somatic_sample_with_mutect2
        stdout:    /tmp/pytest_workflow_kh4i1idl/Run_variant_calling_on_somatic_sample_with_mutect2/log.out
        stderr:    /tmp/pytest_workflow_kh4i1idl/Run_variant_calling_on_somatic_sample_with_mutect2/log.err
'Run variant calling on somatic sample with mutect2' done.

Run joint calling on tumor only samples with mutect2:
        command:   nextflow run main.nf -profile test_cache,tools_tumoronly --input tests/csv/3.0/recalibrated_tumoronly_joint.csv --tools mutect2 --joint_mutect2 --outdir results
        directory: /tmp/pytest_workflow_kh4i1idl/Run_joint_calling_on_tumor_only_samples_with_mutect2
        stdout:    /tmp/pytest_workflow_kh4i1idl/Run_joint_calling_on_tumor_only_samples_with_mutect2/log.out
        stderr:    /tmp/pytest_workflow_kh4i1idl/Run_joint_calling_on_tumor_only_samples_with_mutect2/log.err
'Run joint calling on tumor only samples with mutect2' done.

Run joint calling on somatic samples with mutect2:
        command:   nextflow run main.nf -profile test_cache,tools_somatic --input tests/csv/3.0/recalibrated_somatic_joint.csv --tools mutect2 --joint_mutect2 --outdir results
        directory: /tmp/pytest_workflow_kh4i1idl/Run_joint_calling_on_somatic_samples_with_mutect2
        stdout:    /tmp/pytest_workflow_kh4i1idl/Run_joint_calling_on_somatic_samples_with_mutect2/log.out
        stderr:    /tmp/pytest_workflow_kh4i1idl/Run_joint_calling_on_somatic_samples_with_mutect2/log.err
'Run joint calling on somatic samples with mutect2' done.

tests/test_tools_manually.yml ............................................. [ 34%]
........................................................................... [ 90%]
............                                                                                       [100%] Keeping temporary directories and logs. Use '--kwd' or '--keep-workflow-wd' to disable this behaviour.


==================================== 132 passed in 3564.57s (0:59:24) ====================================```

@asp8200 asp8200 mentioned this pull request Sep 11, 2023
Closed
10 tasks
@FriederikeHanssen
Copy link
Contributor Author

and the latest manual tests:

=
platform linux -- Python 3.6.13, pytest-6.2.3, py-1.11.0, pluggy-0.13.1
rootdir: /home/iizha01/sarek
plugins: workflow-1.6.0
collecting ...
collected 132 items

Run variant calling on somatic sample with mutect2 without intervals:
        command:   nextflow run main.nf -profile test_cache,tools_somatic --tools mutect2 --no_intervals --outdir results
        directory: /home-link/iizha01/pytest_workflow_wafiwx2w/Run_variant_calling_on_somatic_sample_with_mutect2_without_intervals
        stdout:    /home-link/iizha01/pytest_workflow_wafiwx2w/Run_variant_calling_on_somatic_sample_with_mutect2_without_intervals/log.out
        stderr:    /home-link/iizha01/pytest_workflow_wafiwx2w/Run_variant_calling_on_somatic_sample_with_mutect2_without_intervals/log.err
'Run variant calling on somatic sample with mutect2 without intervals' done.

Run variant calling on somatic sample with mutect2:
        command:   nextflow run main.nf -profile test_cache,tools_somatic --tools mutect2 --outdir results
        directory: /home-link/iizha01/pytest_workflow_wafiwx2w/Run_variant_calling_on_somatic_sample_with_mutect2
        stdout:    /home-link/iizha01/pytest_workflow_wafiwx2w/Run_variant_calling_on_somatic_sample_with_mutect2/log.out
        stderr:    /home-link/iizha01/pytest_workflow_wafiwx2w/Run_variant_calling_on_somatic_sample_with_mutect2/log.err
'Run variant calling on somatic sample with mutect2' done.

Run joint calling on tumor only samples with mutect2:
        command:   nextflow run main.nf -profile test_cache,tools_tumoronly --input tests/csv/3.0/recalibrated_tumoronly_joint.csv --tools mutect2 --joint_mutect2 --outdir results
        directory: /home-link/iizha01/pytest_workflow_wafiwx2w/Run_joint_calling_on_tumor_only_samples_with_mutect2
        stdout:    /home-link/iizha01/pytest_workflow_wafiwx2w/Run_joint_calling_on_tumor_only_samples_with_mutect2/log.out
        stderr:    /home-link/iizha01/pytest_workflow_wafiwx2w/Run_joint_calling_on_tumor_only_samples_with_mutect2/log.err
'Run joint calling on tumor only samples with mutect2' done.

Run joint calling on somatic samples with mutect2:
        command:   nextflow run main.nf -profile test_cache,tools_somatic --input tests/csv/3.0/recalibrated_somatic_joint.csv --tools mutect2 --joint_mutect2 --outdir results
        directory: /home-link/iizha01/pytest_workflow_wafiwx2w/Run_joint_calling_on_somatic_samples_with_mutect2
        stdout:    /home-link/iizha01/pytest_workflow_wafiwx2w/Run_joint_calling_on_somatic_samples_with_mutect2/log.out
        stderr:    /home-link/iizha01/pytest_workflow_wafiwx2w/Run_joint_calling_on_somatic_samples_with_mutect2/log.err
'Run joint calling on somatic samples with mutect2' done.

tests/test_tools_manually.yml ......................................................................................................... [ 79%]
...........................                                                                                                            [100%] Keeping temporary directories and logs. Use '--kwd' or '--keep-workflow-wd' to disable this behaviour.


====================================================== 132 passed in 3544.15s (0:59:04) ======================================================

I'd say we are good to go 🚀

@FriederikeHanssen FriederikeHanssen merged commit c9c8dbf into nf-core:dev Sep 12, 2023
42 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@asp8200 asp8200 asp8200 left review comments

@maxulysse maxulysse maxulysse approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@FriederikeHanssen @asp8200 @maxulysse