multiqc_general_stats.txt not output when beeswarm plot

[cbird808]

Description of bug

First, thank you for making and maintaining multiqc, it's great!

Now the issue. I'm wondering if there's a way to change the threshold sample value that triggers the multiqc_general_stats.txt file to not be output?

I've spent some time combing over the documentation, issues, web searches, and asked chatgpt, but am coming up empty, unless I start modifying/creating a config file.

I thought by forcing interactive plots that multiqc_general_stats.txt would also be forced to be output, but that's not the case. The general stats interactive plot is produced in the html. According to the log, the module that's being run is: multiqc:modules:fastqc:fastqc. FWIW, the code below has worked as expected, producing multiqc_general_stats.txt for many data sets.

Version: 1.13

Commands:

inDIR=$1
REPORTNAME=$2
PATTERN=$3

#run fastqc in parallel 
ls ${inDIR}/*${PATTERN} | parallel --no-notice -j32 "crun fastqc {}"

# run multiqc with specific report and subdirectory names

crun multiqc -v -p -ip -f --data-dir --data-format tsv --filename $REPORTNAME --outdir $inDIR $inDIR

File that triggers the error

No response

MultiQC Error log

No response

[ewels]

I guess that this is because the general stats table becomes a beeswarm plot when you hit 500 samples: https://multiqc.info/docs/#tables--beeswarm-plots

I agree that the exported filename suddenly changing (hopefully not disappearing completely, but maybe??) is not ideal behaviour though :/

[ewels]

Short term fix is to set max_table_rows to a very high number in a config file. If this doesn't fix it then please let me know :)

[cbird808]

I'll give that a shot! Unlike most, I'm not very interested in the html output, just the text file that's then read in by our metadata processing scripts.

BTW, the there is no other text (csv/tsv) file that's output with info that's in multiqc_general_stats.txt in this data set with 1600 samples.

[cbird808]

for posterity, we successfully solved this problem by adding --cl-config "max_table_rows: 3000" to our arguments. This will, however, not result in a beeswarm plot in the html report and might result in a non-functional general stats html output.

multiqc -v -p -ip -f --data-dir --data-format tsv --cl-config "max_table_rows: 3000" --filename $REPORTNAME --outdir $DIR $DIR

[ewels]

Great! Glad it's working for you. I'll leave this issue open to look into saving that results text file even when we switch to beeswarm.

[ewels]

Yeah, beeswarm plots weren't saving any data to multiqc_data at all. Kind of surprised that I've never spotted that until now!

Added in bd976bb so should hopefully work as expected now. Filename method is consistent with tables, so the output file should be the same whether it's over the beeswarm threshold or not from now on.

Thanks for reporting!