migration to DSL2 #104

lpantano · 2021-09-09T22:04:32Z

PR checklist

When merging, the test config files need to point to the main branch of the datatest repository and for that this PR needs to be merged: nf-core/test-datasets#363

Note that the versions are still in TXT format because I wasn't sure on how to get it working with YML but maybe that could be another PR if it is the only thing...

…-2.1

fix docs

github-actions · 2021-09-09T22:04:54Z

Markdown linting is failing

To keep the code consistent with lots of contributors, we run automated code consistency checks.
To fix this CI test, please run:

Install markdownlint-cli
- On Mac: brew install markdownlint-cli
- Everything else: Install npm then install markdownlint-cli (npm install -g markdownlint-cli)
Fix the markdown errors
- Automatically: markdownlint . --fix
- Manually resolve anything left from markdownlint .

Once you push these changes the test should pass, and you can hide this comment 👍

We highly recommend setting up markdownlint in your code editor so that this formatting is done automatically on save. Ask about it on Slack for help!

Thanks again for your contribution!

github-actions · 2021-09-09T22:04:56Z

YAML linting is failing

To keep the code consistent with lots of contributors, we run automated code consistency checks.
To fix this CI test, please run:

Install yaml-lint
- Install npm then install yaml-lint (npm install -g yaml-lint)
Fix the markdown errors
- Run the test locally: yamllint $(find . -type f -name "*.yml" -o -name "*.yaml")
- Fix any reported errors in your YAML files

Once you push these changes the test should pass, and you can hide this comment 👍

We highly recommend setting up yaml-lint in your code editor so that this formatting is done automatically on save. Ask about it on Slack for help!

Thanks again for your contribution!

#106 #105

…skipping now defaults to false for all steps

…orced directory path type for some parameters cited as paths

… bisects JSON display problem

…ay problem

…N display problem

…sects JSON display problem

…lay problem

…re before further debugging

Co-authored-by: Kevin <klkeys@users.noreply.github.com>

JoseEspinosa

Huge work, very nice! 😄
Most of my comments are related to the new way of reporting versions, although there are some others related to more general stuff.
Also, I was wondering whether some of the local modules (e.g. bowtie/build and bowtie/align) could not be used from nf-core modules instead, but might be that I am not aware of all the details and that it is not possible to use them in this pipeline.
Almost there 🚀

.github/workflows/ci.yml

.nf-core.yml

CITATIONS.md

bin/scrape_software_versions.py

subworkflows/local/bam_sort.nf

workflows/smrnaseq.nf

nextflow.config

FriederikeHanssen

Great work!! 🚀 Just a couple of minor things, mostly regarding versioning (which I see now also Jose has reviewed already). I think adding the citations would be fairly important too.

CITATIONS.md

README.md

bin/scrape_software_versions.py

modules/local/seqcluster_collapse.nf

modules/local/trimgalore.nf

modules/nf-core/modules/samtools/view/main.nf

FriederikeHanssen · 2021-11-04T13:17:23Z

subworkflows/local/mirna_quant.nf

+params.samtools_stats_options = [:]
+params.table_merge_options = [:]
+
+include {   PARSE_FASTA_MIRNA  as PARSE_MATURE


Maybe align for readability

subworkflows/local/mirna_quant.nf

apeltzer · 2021-11-05T08:19:00Z

I will wait until Joses / Frederikes requests are accepted / discussed & then have a look afterwards - please ping me then here directly or via Slack 👍🏻

Co-authored-by: Jose Espinosa-Carrasco <kadomu@gmail.com>

Co-authored-by: FriederikeHanssen <Friederike.hanssen@qbic.uni-tuebingen.de>

Co-authored-by: Jose Espinosa-Carrasco <kadomu@gmail.com>

Co-authored-by: FriederikeHanssen <Friederike.hanssen@qbic.uni-tuebingen.de>

lpantano · 2021-11-05T20:53:07Z

Thanks @apeltzer.
@JoseEspinosa @FriederikeHanssen I tried to adapt all the comments not related to version.

Some modules are local even if nf-core has similar modules because there is something unique to smrnaseq and it would take some evaluation to do the adaptation.

I am leaving the versioning, so @JoseEspinosa can make the first change to use the YAML format, and I can then modify all the local modules to that format as well.

The plan was to merge, and the @JoseEspinosa will make another PR with the versioning and I will work on top of that. I hope that is ok for everybody else.

As well, there is a final module to add, but I need to do the docker containers, so I am skipping it here. (EDGER module in mirna_quant.nf)

Thanks!

JoseEspinosa

I think most of the changes have been addressed apart from the implementation of the new versioning. Thanks @lpantano ! 🚀
I will merge this PR and work on the versioning on top of them as we agreed.
Also, I will close #101 since we are addressing the merging of the new nf-core template on this PR.

FriederikeHanssen

Looks good :) Great work 🥳 , and as discussed remaining things can go on top of the versioning PR

FriederikeHanssen · 2021-11-08T09:38:05Z

modules/local/format_fasta_mirna.nf

+include { saveFiles; initOptions; getSoftwareName } from './functions'
+
+params.options = [:]
+options        = initOptions(params.options)


Commit this?

KevinMenden and others added 7 commits July 13, 2021 11:01

Template update for nf-core/tools version 2.0

e93246d

Template update for nf-core/tools version 2.0.1

73e2675

Template update for nf-core/tools version 2.1

3863c3d

Merge remote-tracking branch 'origin/dev' into nf-core-template-merge…

3a4da76

…-2.1

work with cat,fastq,trimmgalore

b110bf2

add mirna quant - mirtop method

e7e4470

fix docs

mimrna quant - samtools method

6206eb4

This was referenced Sep 10, 2021

DSL2 - Add mirtrace module #105

Closed

DSL2 - migrate mirdeep tool #106

Closed

DSL2 - add software version for all tools #107

Closed

DSL2 - organize output folders and update output docs #108

Closed

lpantano and others added 17 commits September 10, 2021 13:17

add example input samplesheet pointing to test-data repo

0da285d

format nextflow config

abceed3

adding the genome index and fix mirtop

f0b8fb0

ignore sam files

32c89ed

add mirtrace and mirdeep and versions

e5d0a4f

#106 #105

use temporal testdata

31b60b5

fix to schema, added missing refs, added some descriptions, pipeline …

3287bf5

…skipping now defaults to false for all steps

don't think that trimming options should have blank default, also enf…

13e126e

…orced directory path type for some parameters cited as paths

pipeline step skipping shouldn't have blank default either

e3cccbd

test: remove trimming options, pipeline skipping options, see if that…

eb1edab

… bisects JSON display problem

test: remove reference genome options, see if that bisects JSON displ…

7a75f96

…ay problem

test: remove entire JSON object descriptions, see if that bisects JSO…

fa28e54

…N display problem

test: remove reference genome JSON object description, see if that bi…

0d08924

…sects JSON display problem

test: add back JSON object description, see if that bisects JSON disp…

6daabc3

…lay problem

bisected problem to reference genome section, pinning some changes he…

e1201c9

…re before further debugging

bt_indeces --> bt_indices everywhere, minor schema JSON fixes

8294746

test: add back some of reference_genome_options, see if schema breaks

f2ba996

lpantano and others added 8 commits October 21, 2021 11:13

fix mature text in docs

1390076

Co-authored-by: Kevin <klkeys@users.noreply.github.com>

fix hairpin text in docs

74dcc48

Co-authored-by: Kevin <klkeys@users.noreply.github.com>

fix fasta text in docs

9888bda

Co-authored-by: Kevin <klkeys@users.noreply.github.com>

fix bt_indices in docs

e9d439d

Co-authored-by: Kevin <klkeys@users.noreply.github.com>

fix input sample in docs

a273b30

Co-authored-by: Kevin <klkeys@users.noreply.github.com>

Fix typo in docs

a6f6bac

Co-authored-by: Kevin <klkeys@users.noreply.github.com>

Fix profile info in docs

f3aedaf

Co-authored-by: Kevin <klkeys@users.noreply.github.com>

fix usage of config

ac90519

Co-authored-by: Kevin <klkeys@users.noreply.github.com>

lpantano mentioned this pull request Oct 26, 2021

Multiple Sample running Issue #117

Closed

klkeys requested review from ewels, ggabernet and apeltzer October 27, 2021 17:08

adding last missing output file: mirna.tsv

f4e76ea

klkeys self-requested a review November 1, 2021 16:58

JoseEspinosa requested changes Nov 4, 2021

View reviewed changes

FriederikeHanssen requested changes Nov 4, 2021

View reviewed changes

lpantano and others added 7 commits November 5, 2021 09:25

Update workflows/smrnaseq.nf

5c2d99d

Co-authored-by: Jose Espinosa-Carrasco <kadomu@gmail.com>

Update nextflow.config

b4998e1

Co-authored-by: Jose Espinosa-Carrasco <kadomu@gmail.com>

Update README.md

eade804

Co-authored-by: FriederikeHanssen <Friederike.hanssen@qbic.uni-tuebingen.de>

Update README.md

c726fe0

Co-authored-by: FriederikeHanssen <Friederike.hanssen@qbic.uni-tuebingen.de>

Update subworkflows/local/bam_sort.nf

dd63583

Co-authored-by: Jose Espinosa-Carrasco <kadomu@gmail.com>

Update README.md

6872372

Co-authored-by: FriederikeHanssen <Friederike.hanssen@qbic.uni-tuebingen.de>

update modules, add citation, fix modules structure

7ef213b

skip edger until docker container is ready

24bdb36

JoseEspinosa approved these changes Nov 8, 2021

View reviewed changes

FriederikeHanssen approved these changes Nov 8, 2021

View reviewed changes

JoseEspinosa merged commit 0444a9c into dev Nov 8, 2021

apeltzer deleted the dsl2 branch January 3, 2022 12:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

migration to DSL2 #104

migration to DSL2 #104

lpantano commented Sep 9, 2021 •

edited

Loading

github-actions bot commented Sep 9, 2021

github-actions bot commented Sep 9, 2021

JoseEspinosa left a comment

FriederikeHanssen left a comment

FriederikeHanssen Nov 4, 2021

apeltzer commented Nov 5, 2021

lpantano commented Nov 5, 2021

JoseEspinosa left a comment

FriederikeHanssen left a comment

FriederikeHanssen Nov 8, 2021

migration to DSL2 #104

migration to DSL2 #104

Conversation

lpantano commented Sep 9, 2021 • edited Loading

PR checklist

github-actions bot commented Sep 9, 2021

Markdown linting is failing

github-actions bot commented Sep 9, 2021

YAML linting is failing

JoseEspinosa left a comment

Choose a reason for hiding this comment

FriederikeHanssen left a comment

Choose a reason for hiding this comment

FriederikeHanssen Nov 4, 2021

Choose a reason for hiding this comment

apeltzer commented Nov 5, 2021

lpantano commented Nov 5, 2021

JoseEspinosa left a comment

Choose a reason for hiding this comment

FriederikeHanssen left a comment

Choose a reason for hiding this comment

FriederikeHanssen Nov 8, 2021

Choose a reason for hiding this comment

lpantano commented Sep 9, 2021 •

edited

Loading