Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update SPAdes module to official nf-core and version bump #666

Open
wants to merge 15 commits into
base: dev
Choose a base branch
from
Open

Update SPAdes module to official nf-core and version bump #666

wants to merge 15 commits into from

Conversation

jfy133
Copy link
Member

@jfy133 jfy133 commented Sep 9, 2024 •
edited
Loading

TODO:

⚠️ JAMES: MAKE SURE TO CHECK AFTER TEMPLATE MERGE TO DOUBLE CHECK THE CHEKC_MEGAHIT_CPUS ETC FUNCTIONS ⚠️

PR checklist

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - have you followed the pipeline conventions in the contribution docs
  • If necessary, also make a PR on the nf-core/mag branch on the nf-core/test-datasets repository.
  • Make sure your code lints (nf-core lint).
  • Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
  • Check for unexpected warnings in debug mode (nextflow run . -profile debug,test,docker --outdir <OUTDIR>).
  • Usage Documentation in docs/usage.md is updated.
  • Output Documentation in docs/output.md is updated.
  • CHANGELOG.md is updated.
  • README.md is updated (including new tool citations and authors/contributors).

@jfy133 jfy133 linked an issue Sep 9, 2024 that may be closed by this pull request
Update metaspades to 4.0.0 #640
Open
@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 9, 2024 •
edited
Loading

nf-core lint overall result: Passed ✅ ⚠️

Posted for pipeline commit d7e51ef

+| ✅ 305 tests passed       |+
#| ❔   2 tests were ignored |#
!| ❗   6 tests had warnings |!

❗ Test warnings:

  • nextflow_config - Config manifest.version should end in dev: 3.1.0
  • pipeline_todos - TODO string in main.nf: Remove this line if you don't need a FASTA file [TODO: try and test using for --host_fasta and --host_genome]
  • pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your preferred methods description, e.g. add publication citation for this pipeline
  • pipeline_todos - TODO string in main.nf: Optionally add in-text citation tools to this list.
  • pipeline_todos - TODO string in main.nf: Optionally add bibliographic entries to this list.
  • pipeline_todos - TODO string in main.nf: Only uncomment below if logic in toolCitationText/toolBibliographyText has been filled!

❔ Tests ignored:

✅ Tests passed:

Run details

  • nf-core/tools version 2.14.1
  • Run at 2024-10-04 07:23:46

@jfy133
Copy link
Member Author

jfy133 commented Sep 10, 2024

Need to check other downstreeam files to make sure if there is an issue, however it seems that the original SPADES(HYBIRD) modules exported the the main scaffolds as uncompressed for downstream.( even though it also exported a version as compressed).

I think I will need to add an additional GUNZIP module after the assembly for safety...

@jfy133
Copy link
Member Author

jfy133 commented Sep 12, 2024 •
edited
Loading

Added the GUNZIP module actually does somthing similar to pre-assemblies, but then local MEGAHIT module also exports uncompressed files... so I guess time to update this too

But I need to udpate the MEGAHIT module to support co-assemblies (basically reads.join()

And also test the fix cpus thing

@jfy133
Copy link
Member Author

jfy133 commented Sep 20, 2024

@nf-core-bot fix linting

@jfy133
Copy link
Member Author

jfy133 commented Sep 20, 2024

Final thing: reimplement the CPU fixing

@jfy133 jfy133 marked this pull request as draft September 20, 2024 14:00
@jfy133 jfy133 marked this pull request as ready for review October 3, 2024 14:43
skrakau
skrakau reviewed Oct 8, 2024
View reviewed changes
Copy link
Member

@skrakau skrakau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great to see :D

conf/base.config
cpus = { check_megahit_cpus (8, task.attempt ) }
memory = { check_max (40.GB * task.attempt, 'memory' ) }
time = { check_max (16.h * task.attempt, 'time' ) }
cpus = { params.megahit_fix_cpu_1 ? 1 : check_max(8 * task.attempt, 'cpus') }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does that work? I remember vaguely that it didn't work to access params here, that's why I think I shifted the corresponding functionality to nextflow.config ... but maybe that changed?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if yes, then this is course much cleaner :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should do if it's in a closure { }. That's why e.g. task.attempt works.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you test?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but I will double test again 😬

Copy link
Member Author

@jfy133 jfy133 Oct 17, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hrm wait, this can't work as is as check_max should be gone, do further testing!

@jfy133
Copy link
Member Author

jfy133 commented Oct 17, 2024

⚠️ JAMES: MAKE SURE TO CHECK AFTER TEMPLATE MERGE TO DOUBLE CHECK THE CHEKC_MEGAHIT_CPUS ETC FUNCTIONS ⚠️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@skrakau skrakau skrakau approved these changes

@d4straub d4straub Awaiting requested review from d4straub

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Update metaspades to 4.0.0
2 participants
@jfy133 @skrakau