Is there any database we need to download?

[feixiang1209]

Description of feature

I have tried to run with one sample. It seems Taxonomy classification (GTDB-Tk) didn't work. Could you please advise whether we need to download any database?

Thanks a lot

[jfy133]

Yes, there are a range of databases you may need to supply depending what options you want to run. Some may also be auto downloaded for you in some cases.

I recommend closely reviewing the parameter documentation: https://nf-co.re/mag/2.5.1/parameters

Compared to the options you set to see if you need to supply a database in advance. Note some of the tools require a specific version of databases too.

In the future I would like to improve documentation on this to make it clearer, as it is a bit hard to find at the moment.

[feixiang1209]

Thanks a lot. We are downloading the database. There is another issue, please see below error message. It seems there is a time limit for some steps. Could you please advise how to increase the time limit as our workstation is not strong enough.

RROR ~ Error executing process > 'NFCORE_MAG:MAG:BINNING_PREPARATION:BOWTIE2_ASSEMBLY_ALIGN (MEGAHIT-3-022umF-3-022umF)'

Caused by:
Process exceeded running time limit (8h)

Command executed:

INDEX=find -L ./ -name "*.rev.1.bt2l" -o -name "*.rev.1.bt2" | sed 's/.rev.1.bt2l//' | sed 's/.rev.1.bt2//'
bowtie2
-p "2"
-x $INDEX

-1 "3-022umF_run0_phix_removed.unmapped_1.fastq.gz" -2 "3-022umF_run0_phix_removed.unmapped_2.fastq.gz"
2> "MEGAHIT-3-022umF-3-022umF.bowtie2.log" | samtools view -@ "2" -bS | samtools sort -@ "2" -o "MEGAHIT-3-022umF-3-022umF.bam"
samtools index "MEGAHIT-3-022umF-3-022umF.bam"

if [ MEGAHIT-3-022umF-3-022umF = "MEGAHIT-3-022umF-3-022umF" ] ; then
mv "MEGAHIT-3-022umF-3-022umF.bowtie2.log" "MEGAHIT-3-022umF.bowtie2.log"
fi

cat <<-END_VERSIONS > versions.yml
"NFCORE_MAG:MAG:BINNING_PREPARATION:BOWTIE2_ASSEMBLY_ALIGN":
bowtie2: $(echo $(bowtie2 --version 2>&1) | sed 's/^.bowtie2-align-s version //; s/ .$//')
samtools: $(echo $(samtools --version 2>&1) | sed 's/^.samtools //; s/Using.$//')
pigz: $( pigz --version 2>&1 | sed 's/pigz //g' )
END_VERSIONS

Command exit status:

Command output:
(empty)

Work dir:
/home/diazrur/Documents/Aramco_metagenomes/work/57/22cc01980547e202b05d7c6fc05709

Tip: view the complete command output by changing to the process work dir and entering the command cat .command.out

-- Check '.nextflow.log' file for details

[jfy133]

Yes sure, in this case you just need to configure the pipeline using a config file to increase the default wall times.

Instructions to do so you can see here: https://nf-co.re/docs/usage/configuration#tuning-workflow-resources (and possibly the section above the one I've linked)

[feixiang1209]

I tried to create a config file but it seems the pipeline didn't take it( configFiles : ). Coud you please advise?

(env_nf) diazrur@KW60867:~/Documents/Aramco_metagenomes$ nextflow run nf-core/mag --input metadata.csv --outdir output -profile docker --skip_spades True --skip_spadeshybrid True --skip_concoct True --ancient_dna False --skip_metaeuk True --refine_bins_dastool True --run_gunc True --gtdbtk_pplacer_cpus 40 -c custom.conf -resume

---

                                    ,--./,-.
    ___     __   __   __   ___     /,-._.--~'

|\ | |__ __ / / \ |__) |__ } { | \| | \__, \__/ | \ |___ \-.,--, .,._,'
nf-core/mag v2.5.1-ge728900

Core Nextflow options
revision : master
runName : curious_bell
containerEngine : docker
launchDir : /home/diazrur/Documents/Aramco_metagenomes
workDir : /home/diazrur/Documents/Aramco_metagenomes/work
projectDir : /home/diazrur/.nextflow/assets/nf-core/mag
userName : diazrur
profile : docker
configFiles :

[jfy133]

That appears to be a broken error report?

[jfy133]

Could you please post the .nextflow.log file which should contain the full error message

[feixiang1209]

logs.zip
Please find attached all the log files as well as the config file we used. Thanks for your help.

[gregorysprenger]

.nextflow.log:

• This shows that you aborted the pipeline (SIGINT)

.nextflow.log.1:

• This is the previous run where I believe the error you have above is from
• At line 916, you have: Caused by: Process exceeded running time limit (8h)
• To solve this, create a custom configuration file and increase the alloted time for process BOWTIE2_ASSEMBLY_ALIGN

[feixiang1209]

Hi I used below command to resume a run. I created the "custom.txt " to increase time, memory and cpus. However, the pipeline didn't take custom config. Please find attached config file and screenshot that the pipeline didn't take it. Could you please help?
custom.txt

Thanks

nextflow run nf-core/mag --input "/home/diazrur/Documents/MAG_test/M-23-6561_3-022umB_QIA-UDI041-QIA-UDI041_L002_R{1,2}.fastq.gz" --outdir output -profile docker --skip_krona TRUE --cat_db /home/diazrur/Documents/metagenomics_DB/CAT_prepare_20210107.tar.gz --gtdb_db /home/diazrur/Documents//metagenomics_DB/gtdbtk_r214_data.tar.gz --binqc_tool checkm --skip_spades True --skip_spadeshybrid True --skip_concoct True --ancient_dna False --skip_metaeuk True --refine_bins_dastool True --run_gunc True --gtdbtk_pplacer_cpus 40 -c custom.txt -resume

[jfy133]

Can you please post the custom config?

[feixiang1209]

I already uploaded but I will upload here again. Thanks a lot
custom.txt

[jfy133]

Sorry I missed that, it was hidden above the photo...!

Did the pipeline fail when you ran it with the config? Could you supply the .nextflow.log of that run.

Just because the configFiles is empty in the summary doesn't necessarily mean it didn't get picked up...

[maxibor]

@feixiang1209 Was your problem solved ?

[jfy133]

Will assume yes for now. Please re-open if not.