cellranger web_summary.html with small dataset

[odoublewen]

Description of bug

We use subsampled datasets for quick integration testing of our cellranger workflow -- for example 10,000 reads. It's not enough for meaningful analysis, but cellranger will produce (what appears to be) a valid web_summary.html file. However, this same html file will cause multiqc to crash.

Using cellranger 7.1, with MultiQC v1.15.dev0 (ce261a6)

I first reported this as being associated with #1853

File that triggers the error

web_summary.html.zip

MultiQC Error log

$ multiqc .

  /// MultiQC 🔍 | v1.15.dev0 (ce261a6)

|           multiqc | Search path : /local_scratch/temp
|         searching | ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 1/1  
╭───────────────────────────────────────────────────────────────────── Oops! The 'cellranger' MultiQC module broke... ──────────────────────────────────────────────────────────────────────╮
│ Please copy this log and report it at https://github.com/ewels/MultiQC/issues                                                                                                             │
│ Please attach a file that triggers the error. The last file found was: ./web_summary.html                                                                                                 │
│                                                                                                                                                                                           │
│ Traceback (most recent call last):                                                                                                                                                        │
│   File "/local_scratch/venv/lib/python3.10/site-packages/multiqc/multiqc.py", line 65 │
│     output = mod()                                                                                                                                                                        │
│   File "/local_scratch/venv/lib/python3.10/site-packages/multiqc/modules/cellranger/c │
│     n["count"] = self.parse_count_html()                                                                                                                                                  │
│   File "/local_scratch/venv/lib/python3.10/site-packages/multiqc/modules/cellranger/c │
│     self.parse_count_report(f)                                                                                                                                                            │
│   File "/local_scratch/venv/lib/python3.10/site-packages/multiqc/modules/cellranger/c │
│     "title": f"Cell Ranger count: {summary['analysis_tab']['seq_saturation_plot']['help']['title']}",                                                                                     │
│ KeyError: 'seq_saturation_plot'                                                                                                                                                           │
│                                                                                                                                                                                           │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
|           multiqc | No analysis results found. Cleaning up..
|           multiqc | MultiQC complete

Before submitting

• I have read the troubleshooting documentation.
• I am using the latest release of MultiQC.
• I have included a full MultiQC log, not truncated.
• I have attached an input file (.zip if necessary) that triggers the error.

[odoublewen]

In case it is useful, I am also attaching the web_summary.html file for the same sample, run without subsampling reads. This one runs fine with MultiQC v1.15.dev0.

full_web_summary.html.zip

[mragsac]

Hello! I've run into this same error on version 1.14 (error log below).

Should I wait until version 1.15 up on PyPi to resolve this issue through an upgrade of the software?

Error Log

  /// MultiQC 🔍 | v1.14

|           multiqc | Search path : /dbfs/mnt/data/bioinformatics/results/scrnaseq/46a59f58-6dc1-44fe-8daf-050f106bd50e/SRR16816718
|         searching | ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 1406/1406  
╭──────────────────────────────────────────────────────── Oops! The 'cellranger' MultiQC module broke... ────────────────────────────────────────────────────────╮
│ Please copy this log and report it at https://github.com/ewels/MultiQC/issues                                                                                  │
│ Please attach a file that triggers the error. The last file found was: ./SRR16816718_cellranger/outs/web_summary.html                                          │
│                                                                                                                                                                │
│ Traceback (most recent call last):                                                                                                                             │
│   File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.9/site-packages/multiqc/multiqc.py", line 654, in run                                 │
│     output = mod()                                                                                                                                             │
│   File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.9/site-packages/multiqc/modules/cellranger/cellranger.py", line 39, in __init__       │
│     n["count"] = self.parse_count_html()                                                                                                                       │
│   File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.9/site-packages/multiqc/modules/cellranger/count.py", line 30, in parse_count_html    │
│     self.parse_count_report(f)                                                                                                                                 │
│   File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.9/site-packages/multiqc/modules/cellranger/count.py", line 245, in parse_count_report │
│     warnings[alarm["id"]] = "FAIL"                                                                                                                             │
│ KeyError: 'id'                                                                                                                                                 │
│                                                                                                                                                                │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
|            fastqc | Found 3 reports
|           multiqc | Compressing plot data
|           multiqc | Report      : multiqc_report.html
|           multiqc | Data        : multiqc_data
|           multiqc | MultiQC complete

To compile the report, I was using the following software tools to look at a dataset of ~13M reads with a hard cutoff of 50,000 cells:

• multiqc version 1.14
• fastqc version 0.12.1
• cellranger version 7.1.0

I've also attached the web_summary.html document for your reference!

web_summary.html.zip

[tamuanand]

Hi

I too ran into the same error with multiqc 1.14 and cellranger 7.1.0

 Oops! The 'cellranger' MultiQC module broke...
================================================================================
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/multiqc/multiqc.py", line 654, in run
    output = mod()
             ^^^^^
  File "/usr/local/lib/python3.11/site-packages/multiqc/modules/cellranger/cellranger.py", line 39, in __init__
    n["count"] = self.parse_count_html()
                 ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/multiqc/modules/cellranger/count.py", line 30, in parse_count_html
    self.parse_count_report(f)
  File "/usr/local/lib/python3.11/site-packages/multiqc/modules/cellranger/count.py", line 245, in parse_count_report

    warnings[alarm["id"]] = "FAIL"
             ~~~~~^^^^^^
KeyError: 'id'

Dataset for this analyses and the cellranger count run details

wget -c -N http://s3-us-west-2.amazonaws.com/10x.files/samples/cell-exp/2.1.0/pbmc8k/pbmc8k_fastqs.tar
 
OR

aws s3 cp s3://10x.files/samples/cell-exp/2.1.0/pbmc8k/pbmc8k_fastqs.tar . --no-sign-request


Reference Data: wget http://cf.10xgenomics.com/supp/cell-exp/refdata-cellranger-GRCh38-1.2.0.tar.gz

cellranger count \
            --id=pbmc8k \
            --transcriptome=<path_to>/refdata-cellranger-GRCh38-1.2.0 \
            --fastqs=fastqs/ \
            --sample=pbmc8k \
            --localcores=45 \
            --expect-cells=10000 \
            --localmem=360

pbmc8k_web_summary.html.zip

[Maj18]

I run into the same issue as mragsac as well...

[pedriniedoardo]

I got the same error from a recent dataset run using cellranger7.1 and multiqc v1.14

|         searching | ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 2665/2665  
╭─────────────────────────── Oops! The 'cellranger' MultiQC module broke... ───────────────────────────╮
│ Please copy this log and report it at https://github.com/ewels/MultiQC/issues                        │
│ Please attach a file that triggers the error. The last file found was:                               │
│ MA8483-1/outs/web_summary.html                                                                       │
│                                                                                                      │
│ Traceback (most recent call last):                                                                   │
│   File "/home/pedrini.edoardo/miniconda3/envs/bioinfo/lib/python3.8/site-packages/multiqc/multiqc.py │
│     output = mod()                                                                                   │
│   File "/home/pedrini.edoardo/miniconda3/envs/bioinfo/lib/python3.8/site-packages/multiqc/modules/ce │
│     n["count"] = self.parse_count_html()                                                             │
│   File "/home/pedrini.edoardo/miniconda3/envs/bioinfo/lib/python3.8/site-packages/multiqc/modules/ce │
│     self.parse_count_report(f)                                                                       │
│   File "/home/pedrini.edoardo/miniconda3/envs/bioinfo/lib/python3.8/site-packages/multiqc/modules/ce │
│     warnings[alarm["id"]] = "FAIL"                                                                   │
│ KeyError: 'id'                                                                                       │
│                                                                                                      │

I am attaching the file here:

web_summary.html.zip

Interestingly, If I run the same dataset with cellranger 6.1, multiQC works fine:

|         searching | ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 2481/2481  
|        cellranger | Found 1 Cell Ranger count reports
|           multiqc | Compressing plot data
|           multiqc | Previous MultiQC output found! Adjusting filenames..
|           multiqc | Use -f or --force to overwrite existing reports instead
|           multiqc | Report      : multiqc_report_1.html
|           multiqc | Data        : multiqc_data_1
|           multiqc | MultiQC complete

[ewels]

Thank you everyone! Fixed in 9d5e3ab, will be in the coming release.

I tested on all of your example reports and they all seem to work fine now on the dev version.