multipit

This repository provides a set of Python tools to perform multimodal learning with tabular data. It contains the code used in our study:

"Integration of clinical, pathological, radiological, and transcriptomic data improves the prediction of first-line immunotherapy outcome in metastatic non-small cell lung cancer"

Preprint: https://doi.org/10.1101/2024.06.27.24309583

Installation

Dependencies

• lifelines (>= 0.27.4)
• matplotlib (>= 3.5.1)
• numpy (>= 1.21.5)
• pandas (= 1.5.3)
• pyyaml (>= 6.0)
• scikit-learn (>= 1.2.0)
• scikit-survival (>= 0.21.0)
• seaborn (=0.13.0)
• shap (>= 0.41.0)
• xgboost (>= 1.7.5)

Install from source

Clone the repository:

git clone https://github.com/sysbio-curie/multipit

Key features

• Early and late fusion implementations: 4 estimators compatible with scikit-learn and scikit-surv to fuse several tabular modalities in a single multimodal model.

  • multipit.multi_model.EarlyFusionClassifier and multipit.multi_model.LateFusionClassifier for binary classification.
  • multipit.multi_model.EarlyFusionSurvival and multipit.multi_model.LateFusionSurvival for survival prediction.

• Scripts to reproduce the experiments of our study: Scripts to perform late fusion an early fusion of clinical, radiomic, pathomic and transcriptomic features with a repeated cross-validation scheme. Scripts to compute and collect the SHAP values associated with each unimodal predictive model (see scripts folder).

• Plotting functions and notebooks to reproduce the figures of our study: several functions to plot and compare the performances of different multimodal combinations as well as to display feature importance with SHAP values.

  • plot_results.ipynb
  • benchmark.ipynb
  • plot_shap.ipynb

Deep-multipit

We also provide another Github repository, named deep-multipit with a Pytorch implementation of an end-to-end integration strategy with attention weights, inspired by Vanguri et al, 2022.

Run scripts

Modify the configurations in .yaml config files (in config/ subfolder) then run the following command in your terminal:

python latefusion.py -c config/config_latefusion.yaml -s path/to/results/folder

python collect_shap_survival.py -c config/config_latefusion_survival.yaml -s path/to/results/folder

Warning: For Windows OS paths must be written with '\' or '\' separators (instead of '/').

Note: In order to modify more deeply the loading of the data or the predictive pipelines, please update the PredictionTask class in the file _init_scripts.py.

Examples

In the examples folder we provide a brief example on how to slightly modify the scripts and codes from our original experiments to perform multimodal learning for the prediction of Overall Survival from clinical and RNA-seq data extracted from TCGA (i.e., stage III and IV TCGA-LUAD and TCGA-LUSC samples).

We simply updated the PredictionTask class in a new file _init_scripts_tcga.py to load TGCA data and build predictive pipelines.

Note: clinical and transcriptomic data extracted for 201 stage III/IV TCGA patients (i.e., LUAD or LUSC) are available in the data folder.

Acknowledgements

This repository was created as part of the PhD project of Nicolas Captier in the Computational Systems Biology of Cancer group and the Laboratory of Translational Imaging in Oncology (LITO) of Institut Curie.