index.Rmd

---
title: "Visualization of highly-multiplexed imaging data with `cytomapper`"
author: "Nils Eling"
institute: 
    - "Department of Quantitative Biomedicine, University of Zurich"
    - "Institute for Molecular Health Sciences, ETH Zurich"
date: "09/07/2021"
output:
  xaringan::moon_reader:
    lib_dir: libs
    css: ["default", "custom.css", "useR-fonts"]
    nature:
      ratio: '16:9'
      highlightStyle: github
      highlightLines: true
      countIncrementalSlides: false
---
background-image: url(imgs/cytomapper_sticker.png)
background-size: 150px
background-position: 95% 10%
  
```{r setup, include=FALSE}
options(htmltools.dir.version = FALSE)
knitr::opts_chunk$set(fig.retina = 2)
```

## The `cytomapper` R/Bioconductor package

Contains three broad functionalities:

* Visualization of pixel-intensities as composites of up to six channels (`plotPixels`)

* Visualization of cell-specific features on segmentation masks (`plotCells`)

* Interactive gating of cells and visualization of gated cells on images (`cytomapperShiny`)

### Installation

Via Bioconductor (release version):

```{r, bioconductor-install, eval=FALSE, fig.alt="The release version of cytomapper can be installed using BiocManager::install"}
BiocManager::install("cytomapper")
```

or from Github (development version):

```{r, github-install, eval=FALSE, fig.alt="The development version of cytomapper can be installed using remotes::install_github"}
remotes::install_github("BodenmillerGroup/cytomapper", build_vignettes = TRUE)
```

---

## Data structure

.pull-left[

* `cytomapper` uses a `SingleCellExperiment` data container to store cell-specific metadata and intensity features

* the new class `cytomapper::CytoImageList` stores individual `EBImage::Image` objects in form of a `S4Vectors::SimpleList` container

* visualization of cell-specific metadata is possible by linking a `CytoImageList` and `SingleCellExperiment` object via the function parameters `img_id` and `cell_id`

]

.pull-right[

![cytomapper overview figure. A) The plotCells function combines a SingleCellExperiment and CytoImageList object to visualize marker expression or cell-specific metadata on segmentation masks. B) The plotPixels function requires a CytoImageList object to visualize the combined expression of up to six markers as composite images](imgs/Overview.png)

]

<div class="my-footer" style="height=100px;"><a href="https://bioconductor.org/packages/release/bioc/html/SingleCellExperiment.html">https://bioconductor.org/packages/release/bioc/html/SingleCellExperiment.html</a><br>
<a href="https://bioconductor.org/packages/release/bioc/html/EBImage.html">https://bioconductor.org/packages/release/bioc/html/EBImage.html</a><br>
<a href="https://bioconductor.org/packages/release/bioc/html/S4Vectors.html">https://bioconductor.org/packages/release/bioc/html/S4Vectors.html</a><br>
<a href="https://github.com/BodenmillerGroup/ImcSegmentationPipeline">https://github.com/BodenmillerGroup/ImcSegmentationPipeline</a></div> 

???

In the displayed framework, cells are defined as pixel sets with the same id on the segmentation mask. These masks and their quantification are done in pre-processing steps that are not mentioned in detail here put a link to a segmentation pipeline can be found in the footer.

---

## Reading in the images

The `cytomapper::loadImages` function reads in multi-channel images and segmentation masks into `CytoImageList` objects.

```{r, reading-in-data-1, message=FALSE}
library(cytomapper)

# Read in 32-bit multi-channel images
(images <- loadImages("../data/images/", pattern = ".tiff"))

# Read in 16-bit unsigned integer segmentation masks
(masks <- loadImages("../data/masks/", pattern = ".tiff", as.is = TRUE))
```

<div class="my-footer" style="height=50px;"><a href="https://github.com/BodenmillerGroup/cytomapper_demos">https://github.com/BodenmillerGroup/cytomapper_demos</a></div> 

???

The loadImages function reads in .tiff, .png or .jpg images and formats them into a CytoImageList container. The as.is parameter needs to be set to TRUE to correctly read in 16-bit unsigned integer images. The multi-channel images and segmentation masks are provided in the BodenmillerGroup/cytomapper_demos Github repository.

---

## Reading in the single-cell data

Cell-specific metadata and summarized intensity features are stored in form of a `SingleCellExperiment` object.

```{r, reading-in-data-2, message=FALSE}
# Read in single-cell data
(sce <- readRDS("../data/sce.rds"))
```

<div class="my-footer" style="height=50px;"><a href="https://github.com/BodenmillerGroup/cytomapper_demos">https://github.com/BodenmillerGroup/cytomapper_demos</a></div> 

???

In the BodenmillerGroup/cytomapper_demos Github repository, you can also find an .rds file containing a SingleCellExperiment object that holds the mean channel intensities per cell and all associated metadata (e.g. cell-type label). Cell-specific intensities are stored in the `counts(sce)` slot; cell-specific metadata are stored in the `colData(sce)` slot and marker-specific metadata are stored in `rowData(sce)`.

---

## Formatting the data

Prior to visualization, the channel names and image-specific metadata should be set.

```{r, format-the-data, message=FALSE}
# Add channel names
channelNames(images) <- rownames(sce)

# Add image name to metadata
(mcols(images) <- mcols(masks) <- DataFrame(ImageName = c("E30", "G23", "J01")))
```

???

We will need to set the channel names of the images via the `channelNames` getter/setter function. The channel order here is the same as the row order of the `SingleCellExperiment` object. We will also need to synchronise the image IDs across the multi-channel images and segmentation masks by storing a DataFrame in the elementMetadata slot of the CytoImageList object.

---

## Measuring object features

The `cytomapper::measureObjects` function computes morphological (e.g. cell shape, size and location) and intensity features (default: mean intensity per channel and object/cell).

```{r, measure-features}
sce_2 <- measureObjects(mask = masks, image = images, img_id = "ImageName")
sce_2
```

???

The pixel intensities per cell can be summarized in different ways (e.g. as quantiles). Furthermore, parallelization is possible by setting `BPPARAM = bpparam()`. Cell-specific morphological features are stored in `colData(sce)`.

---

## Visualize multi-channel images

.pull-left[

```{r plotPixels, eval=FALSE}
plotPixels(
    image = images,
    colour_by = c("PIN", "CD4", "CD8a"), 
    colour = list(PIN = c("black", "yellow"),
                  CD4 = c("black", "blue"),
                  CD8a = c("black", "red")),
    bcg = list(PIN = c(0, 10, 1),
               CD4 = c(0, 8, 1),
               CD8a = c(0, 10, 1)),
    image_title = list(
        text = c("Non-diabetic",
                 "Recent onset T1D",
                 "Long duration T1D")
        ),
    scale_bar = list(
        length = 100,
        label = expression("100 " ~ mu * "m")
        ))
```

]

.pull-right[

```{r plotPixels-2, echo=FALSE}
plotPixels(image = images,
           colour_by = c("PIN", "CD4", "CD8a"), 
           colour = list(PIN = c("black", "yellow"),
                         CD4 = c("black", "blue"),
                         CD8a = c("black", "red")),
           bcg = list(PIN = c(0, 10, 1),
                      CD4 = c(0, 8, 1),
                      CD8a = c(0, 10, 1)),
           image_title = list(text = c("Non-diabetic",
                                       "Recent onset T1D",
                                       "Long duration T1D")),
           scale_bar = list(length = 100,
                            label = expression("100 " ~ mu * "m")))
```

]

???

The `plotPixels` function visualizes pixel intensities as composites of up to six markers. It requires at least the `image` parameters that accepts a `CytoImageList` object containing the individual multi-channel images. The `colour_by` parameter defines the channel names by which to colour the composite. Per channel, a colour scale is generated by setting `colour`. The bcg parameter defines the background (b), contrast (c), gamma (g) adjustment of each channel in terms of addition (b), multiplication (c) and exponentiation (g). The attributes of the image titles can be set via the parameter `image_title` and attributes of the scale bar are set via the parameter `scale_bar`.

---

## Visualize segmented cells

.pull-left[

```{r plotCells, eval=FALSE}
cur_sce <- sce[,sce$CellType %in% 
    c("beta", "alpha", "delta", "Tc", "Th")]
plotCells(
    mask = masks,
    object = cur_sce,
    cell_id = "CellNumber",
    img_id = "ImageName",
    colour_by = "CellType",
    image_title = list(
        text = c("Non-diabetic",
                 "Recent onset T1D",
                 "Long duration T1D"),
        colour = "black"),
    scale_bar = list(
        length = 100,
        label = expression("100 " ~ mu * "m"),
        colour = "black"),
    missing_colour = "white",
    background_colour = "gray")
```

]

.pull-right[

```{r plotCells-2, echo=FALSE}
cur_sce <- sce[,sce$CellType %in% c("beta", "alpha", "delta", "Tc", "Th")]
plotCells(mask = masks,
          object = cur_sce,
          cell_id = "CellNumber",
          img_id = "ImageName",
          colour_by = "CellType",
          image_title = list(text = c("Non-diabetic",
                                      "Recent onset T1D",
                                      "Long duration T1D"),
                             colour = "black"),
          scale_bar = list(length = 100,
                           label = expression("100 " ~ mu * "m"),
                           colour = "black"),
          missing_colour = "white",
          background_colour = "gray")
```

]

To highlight the functionality of the `plotCells` function, the `SingleCellExperiment` object is first subsetted to contain only objects of a certain type. Here, these are pancreatic islet cells and T cells. The `plotCells` function requires the `mask` parameter, which takes a `CytoImageList` object storing the segmentation masks. Via the `object` parameter, the user can supply a `SingleCellExperiment` object containing expression or metadata. Via the `cell_id`, cells contained in the `SingleCellExperiment` object are linked to their ID on the segmentation mask. Setting `img_id` will link cells in the `SingleCellExperiment` object to their corresponding images. The `missing_colour` parameter defines the colour of cells, which are not contained in the SingleCellExperiment object. Finally, the `background_colour` parameter sets the colour of the segmentation mask background. 

---

## Outline cells on images

.pull-left[

```{r outlineCells, eval=FALSE}
plotPixels(
    image = images,
    object = cur_sce,
    mask = masks,
    cell_id = "CellNumber",
    img_id = "ImageName",
    colour_by = c("PIN", "CD4", "CD8a"), 
    outline_by = "CellType",
    colour = list(PIN = c("black", "yellow"),
                  CD4 = c("black", "blue"),
                  CD8a = c("black", "red")),
    bcg = list(PIN = c(0, 10, 1),
               CD4 = c(0, 8, 1),
               CD8a = c(0, 10, 1)),
    image_title = list(text = c("Non-diabetic",
                                "Recent onset T1D",
                                "Long duration T1D")),
    scale_bar = list(length = 100,
                     label = expression("100 " ~ mu * "m")),
    thick = TRUE)
```

]

.pull-right[

```{r outlineCells-2, echo=FALSE}
plotPixels(image = images,
           object = cur_sce,
           mask = masks,
           cell_id = "CellNumber",
           img_id = "ImageName",
           colour_by = c("PIN", "CD4", "CD8a"), 
           outline_by = "CellType",
           colour = list(PIN = c("black", "yellow"),
                         CD4 = c("black", "blue"),
                         CD8a = c("black", "red")),
           bcg = list(PIN = c(0, 10, 1),
                      CD4 = c(0, 8, 1),
                      CD8a = c(0, 10, 1)),
           image_title = list(text = c("Non-diabetic",
                                       "Recent onset T1D",
                                       "Long duration T1D")),
           scale_bar = list(length = 100,
                            label = expression("100 " ~ mu * "m")),
           thick = TRUE)
```

]

???

Finally, `cytomapper` can be used to outline cells on composite images while colouring the outlines based on specified metadata. For this, the `plotPixels` function combines the multi-channel images, the SingleCellExperiment object and the segmentation masks. AS before, cells and images are matched via the `cell_id` and `img_id` parameters. Here, the outline colour is determined by the cells' phenotype and outlines are expaned by setting `thick = TRUE`.

---

<img src="imgs/cytomapper_sticker.png" align="right" alt="cytomapper sticker" width="175px"/>

## Acknowledgements and useful links

**Co-authors:** 

Nicolas Damond, Tobias Hoch, Bernd Bodenmiller

**Publication:**

<a href="https://academic.oup.com/bioinformatics/article/36/24/5706/6050702">Eling _et al._, cytomapper: an R/Bioconductor package for visualization of highly multiplexed imaging data, Bioinformatics, 2020</a>

<a href="https://github.com/BodenmillerGroup/cytomapper_publication">https://github.com/BodenmillerGroup/cytomapper_publication</a>

<img src="imgs/eth_uzh_logo_kurz_pos_en.png" alt="uzh eth logo" align="right" width="520px"/>

**Vignettes:**

<a href="https://www.bioconductor.org/packages/release/bioc/vignettes/cytomapper/inst/doc/cytomapper.html">Visualization of imaging cytometry data in R</a>

<a style="color=blue;" href="https://www.bioconductor.org/packages/release/bioc/vignettes/cytomapper/inst/doc/cytomapper_ondisk.html">On disk storage of images</a>

<img src="imgs/EMBO_logo_1_640x360.jpg" align="right" alt="EMBO logo" width="250px"/>
<img src="imgs/marie_curie1.jpg" alt="MSCA logo" align="right" width="250px"/>

**Related packages:**

<a href="https://bioconductor.org/packages/release/data/experiment/html/imcdatasets.html">imcdatasets</a>

<a href="https://github.com/BodenmillerGroup/imcRtools">imcRtools</a>

???

With this, I want to thank my colleagues who helped me with the work: Nicolas Damond, Tobias Hoch and Bernd Bodenmiller and my funders EMBO and Marie Curie. If you want to check out cytomapper have a look at the publication and the package vignettes. Specifically how to read in and store images on disk. We have also been working on extending the multiplexed imaging capabilities of Bioconductor by setting up the imcdatasets package which stores publically available imaging mass cytometry datasets and imcRtools which provides functions for multiplexed imaging data analysis - but which is still under development. 

---

## Session Info

```{r zzz, echo=FALSE, message=FALSE}
library(dplyr)
library(magrittr)
```

```{r sessionInfo}
sessioninfo::package_info() %>% filter(source == "Bioconductor")
```

???

Finally, here are the main Bioconductor packages used for the displayed analysis and I'm looking forward to your questions.