TEMPLATE_fluo_analysis.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.7"
    },
    "colab": {
      "name": "TEMPLATE_fluo_analysis.ipynb",
      "provenance": [],
      "collapsed_sections": [
        "e_fiGfO33MpG",
        "eg8YfviX3MpK",
        "PTgyhnMq3MpQ",
        "6afuMBY93MpR",
        "rdVZLGbz3MpT",
        "xX05ImmN3MpZ",
        "CEen2eg13Mpb",
        "1Z7MhmXb3Mpe",
        "Qwyq6JX73Mpf",
        "2Tedm8xZ3Mpi",
        "svRV80Pc3Mpp",
        "qL8mLXul3Mpt",
        "cTyklEnB3Mpv",
        "H829dBKL3Mp3",
        "Z14pvggZ3Mp6",
        "UaUw6o2a3Mp8",
        "XaqH4vcW3MqC",
        "u9_1TGi33MqK",
        "6PTYyWN53MqN",
        "W2CK5kY93MqX",
        "mtsQJN7H3Mqg",
        "4Iw6LMDM3Mqq"
      ],
      "include_colab_link": true
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/lauraluebbert/confocal_z-stack_analysis/blob/master/TEMPLATE_fluo_analysis.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-rkQdV7l3T2U",
        "colab_type": "text"
      },
      "source": [
        "(Ignore this if not running this notebook in Google Colab.) To demonstrate how this notebook works in Google Colab, I compressed a few sample tif images into a .tar folder. This tar folder is downloaded from a public Google Drive link and unzipped. If you want to your own images using this notebook in Google Colab, compress your images into a .tar folder (click here for more information), upload this .tar folder into Google Drive (make sure to change the sharing settings such that \"everybody with the link can view\"), and replace eveything after \"id=\" with the Google Drive ID of your .tar folder (the ID is part of your Google Drive shareable link)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "i0ODKSYp3Ou3",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "### Ignore this code cell if you are not running this notebook in Google Colab\n",
        "# Install packages\n",
        "!pip install bokeh_catplot\n",
        "!pip install bebi103\n",
        "!pip install gdown\n",
        "# Download data from Google Drive using gdown\n",
        "!gdown https://drive.google.com/uc?id=18_YmrDlG5qDLImpaAOnN13JLihPPW4HF\n",
        "# Extract images from tar directory\n",
        "!tar xvf *.tar"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rr4agKyn3P6F",
        "colab_type": "text"
      },
      "source": [
        "___"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kIVooXAI3Mok",
        "colab_type": "text"
      },
      "source": [
        "# Extraction of pixel values from ROIs using the sum projection of z-stack images"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Km_OJMMD3Mol",
        "colab_type": "text"
      },
      "source": [
        "This notebook computes the sum projections of z-stack images (tif files). The user can then draw ROIs, as well as background areas, manually using the bebi103.image.record_clicks function developed by Prof. Justin Bois.\n",
        "\n",
        "The notebook computes the fold change from the mean and the median of the pixel values such that the user can decide which on to use. Both of these, mean and median, have their benefits. The median is less sensitive to outliers, therefore it will not be shifted by some outlier cells whose intensity is distorted by noise. However, if these outliers are of interest, then the median will obscure the effect. If the size of the fluorescent area is also of interest, one should use the bkg normalized Integrated Density. \n",
        "\n",
        "The notebook returns the following information in the form of a pickled dictionary, data frame and csv file:   \n",
        "\"Condition\" = Condition of sample as defined in filename (\"samplecondition\" see below)  \n",
        "\"Sample\" = Sample ID as defined in filename (\"sample\" see below)  \n",
        "\"Channel\" = Channel as defined in filename (\"channel\" see below)  \n",
        "\"Condition + Channel\" = Combination of Sample ID and channel for plotting  \n",
        "\"Mean pixel value in ROI\" = Mean pixel in ROI (not normalized)  \n",
        "\"Mean pixel value in bkg area\" = Mean pixel in background area   \n",
        "\"Median pixel value in ROI\" = Median pixel in ROI (not normalized)  \n",
        "\"Median pixel value in bkg area\" = Median pixel in background area   \n",
        "\"Norm Raw Integrated Density\" = Sum of all bkg normalized pixel values inside the ROI  \n",
        "\"Norm Integrated Density\" = ROI area (number of pixels in ROI) x mean bkg normalized pixel value inside the ROI  \n",
        "\"Mean Fold Change\" = (Mean pixel value of ROI (not normalized) - Mean bkg pixel value) / Mean bkg pixel value  \n",
        "\"Median Fold Change\" = (Median pixel value of ROI (not normalized) - Median bkg pixel value) / Median bkg pixel value\n",
        "\n",
        "Image matrices, ROI coordinates, and pixel values are saved to a dictionary (\"dicts_sum\") and continuously linked to their sample ID, channel, condition, and original filename.\n",
        "\n",
        "#### Do not click \"Run all\", but stop at \"STOP\" cells to draw the ROIs BEFORE continuing to run the notebook.\n",
        "\n",
        "#### Name tif files as follows: \n",
        "X_samplecondition_sample_channel , where \"X\" is optional and can be filled as desired by the user (do not use spaces).  \n",
        "- \"Samplecondition\" can be a string of text separated by \"-\", e.g. \"GCaMP-30C-6d_sample_channel\" (Do not use spaces or \"_\" within the condition, else part of it will be cut out in the dictionary.)  \n",
        "- \"Sample\" and \"channel\" can be a simple number or a description, e.g.: \"samplecondition_1_1\" or \"samplecondition_Brain1_DAPI\". (Do not use spaces or \"_\" within the sample or channel.)  \n",
        "\n",
        "Written by Laura Luebbert, 20th of August 2020.  \n",
        "\n",
        "Modified on: / "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wglgAu2K3Mol",
        "colab_type": "text"
      },
      "source": [
        "## Define the parameters:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BRwTQOkM3Mom",
        "colab_type": "text"
      },
      "source": [
        "### Define the directory containing the tif files:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7LfZyEMy3Mon",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "data_dir = \"/content\""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BVXlZe7C3Moq",
        "colab_type": "text"
      },
      "source": [
        "### General saving options:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MdU-kr543Moq",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Define the directory the pickled dictionaries will be saved to\n",
        "saving_dir = data_dir"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X-czweaN3Mot",
        "colab_type": "text"
      },
      "source": [
        "### Sum projection images saving options:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "eiaQfsWX3Mot",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Do you want to save the sum projection of the images? (Define as True or False.)\n",
        "save_sum_ims = True"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jOw8O-PQ3Mov",
        "colab_type": "text"
      },
      "source": [
        "#### If yes, define the following (if not you can skip the following parameters and begin running the notebook):"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FBgcyTGe3Mow",
        "colab_type": "text"
      },
      "source": [
        "Saving options:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ao9a-p3S3Mow",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Directory the sum projections of the images will be saved to (inside a new folder)\n",
        "saving_dir_sum = data_dir\n",
        "\n",
        "# Set the file format the images should be saved as\n",
        "file_format = \"tif\""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "18Ger6Mh3Moz",
        "colab_type": "text"
      },
      "source": [
        "Scale bar options:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lu-hMhW63Moz",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Turn scale bar on/off (define as True or False)\n",
        "scale_bar = True\n",
        "\n",
        "# Set scalebar color (\"white\" or \"black\")\n",
        "scale_bar_color = \"white\"\n",
        "\n",
        "# Define the desired scale bar size and width in length unit\n",
        "scale_bar_length = 100\n",
        "scale_bar_width = 5\n",
        "\n",
        "# Define position of scale bar:\n",
        "# Distance from top (Distance in % of total length of image)\n",
        "y_pos = 95\n",
        "# Distance of rightmost end of scale bar from left end of image (Distance in % of total length of image)\n",
        "x_pos = 95"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4S0AIBlh3Mo2",
        "colab_type": "text"
      },
      "source": [
        "Define microscope and magnification (to get interpixel distance) OR define interpixel distance directly if different microscope/magnification used:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wSKTVMgC3Mo2",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Define microscope used (\"Lois\", \"LSM800\" or define inter pixel distance manually below)\n",
        "microscope = \"Lois\"\n",
        "\n",
        "# Define magnification (\"20\" or define inter pixel distance manually below)\n",
        "magnification = \"20\"\n",
        "\n",
        "# Uncomment (remove the #) and define interpixel distance manually here if a different microscope/magnification was used\n",
        "# interpixel_distance = "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BFu3ot_C3Mo5",
        "colab_type": "text"
      },
      "source": [
        "___"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PQ_tAH533Mo5",
        "colab_type": "text"
      },
      "source": [
        "___"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BK_HxzbD3Mo6",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# %load_ext blackcellmagic\n",
        "%config InlineBackend.figure_format = 'retina'"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NZL0WWNN3Mo8",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Tools to read in the image files and filenames\n",
        "import glob\n",
        "import os\n",
        "import re \n",
        "\n",
        "# Calculation and data frame tools\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "\n",
        "# Image processing tools\n",
        "import bebi103\n",
        "import skimage\n",
        "import skimage.io\n",
        "import skimage.morphology\n",
        "\n",
        "# Tools to create new folders\n",
        "from pathlib import Path\n",
        "\n",
        "# Tools to save dictionaries\n",
        "import pickle\n",
        "\n",
        "# (Interactive) plotting tools\n",
        "import bokeh\n",
        "import bokeh_catplot\n",
        "import matplotlib.pyplot as plt\n",
        "# import matplotlib.patches as mpatches\n",
        "bokeh.io.output_notebook()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a_se3VAt3Mo-",
        "colab_type": "text"
      },
      "source": [
        "___"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gjoE_jg83MpA",
        "colab_type": "text"
      },
      "source": [
        "# Load in the data"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "q_6ExEe5B_d6",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Glob string for images (loads all .tif files)\n",
        "im_glob = os.path.join(data_dir, \"*.tif\")\n",
        "\n",
        "# Get list of files in directory\n",
        "im_list = sorted(glob.glob(im_glob))\n",
        "\n",
        "# Let's look at the first 10 entries\n",
        "im_list[:10]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "s-luvPnr3MpD",
        "colab_type": "text"
      },
      "source": [
        "Create a nested dictionary with information about each sample coupled to the z-stack image matrix, as well as space for matrices added later:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iPczgf-M3MpD",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "dicts_sum = {}\n",
        "\n",
        "for i in range(len(im_list)):\n",
        "    # Get sample condition\n",
        "    condition = im_list[i].split(\"/\")[-1].split(\"_\")[-3]\n",
        "    \n",
        "    # Make sure we do not overwrite a previous dictionary entry    \n",
        "    if dicts_sum.get(condition) == None:\n",
        "        dicts_sum[condition] = {}\n",
        "    \n",
        "    # Get sample number\n",
        "    sample = im_list[i].split(\"/\")[-1].split(\"_\")[-2]\n",
        "    \n",
        "    # Make sure we do not overwrite a previous dictionary entry\n",
        "    if dicts_sum.get(condition, {}).get(sample) == None:\n",
        "        dicts_sum[condition][sample] = {}\n",
        "    \n",
        "    # Get channel number    \n",
        "    channel = im_list[i].split(\"/\")[-1].split(\"_\")[-1].split(\".\")[0]\n",
        "    \n",
        "    # Add empty arrays to dictionary to populate with images later\n",
        "    dicts_sum[condition][sample][channel] = {\n",
        "        \"matrix_orig\": [],       # Original image z-stack matrix \n",
        "        \"matrix_sum\": [],        # Image matrix of sum projection image \n",
        "        \"matrix_8\": [],          # Image matrix of sum projection image as 8-bit \n",
        "        \"matrix_8_SB\": [],       # Image matrix of sum projection image as 8-bit with scale bar \n",
        "        \"filename\": [],          # The original filename, just in case\n",
        "        \"clicks\": [],            # The coordinates of \"clicks\" defining the ROI \n",
        "        \"rois\": [],              # ROI clicks converted to an ROI (polygon) \n",
        "        \"clicks_bkg\": [],        # The coordinates of \"clicks\" defining the background area to normalize against \n",
        "        \"rois_bkg\": [],          # bkg area clicks converted to a polygon\n",
        "        \"mean_int\": [],          # Mean pixel value in ROI\n",
        "        \"mean_int_bkg\": [],      # Mean pixel value in bkg area\n",
        "        \"median_int\": [],        # Median pixel value in ROI\n",
        "        \"median_int_bkg\": [],    # Median pixel value in bkg area\n",
        "        \"raw_int_den\": [],       # Raw integrated density of ROI (sum of all bkg normalized pixel values inside the ROI) \n",
        "        \"int_den\": [],           # Integrated density of ROI (product of ROI area and mean bkg normalized pixel value inside the ROI)\n",
        "        \"mean_fold_change\": [],  # (Mean pixel value of ROI (not normalized) - Mean bkg pixel value) / Mean bkg pixel value\n",
        "        \"median_fold_change\": [] # (Median pixel value of ROI (not normalized) - Median bkg pixel value) / Median bkg pixel value\n",
        "    }\n",
        "        \n",
        "    # Populate dictionary with original image matrix\n",
        "    dicts_sum[condition][sample][channel][\"matrix_orig\"] = skimage.io.imread(im_list[i])\n",
        "    \n",
        "    # Populate dictionary with original filename\n",
        "    dicts_sum[condition][sample][channel][\"filename\"] = im_list[i].split(\"/\")[-1].split(\".\")[-2]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ShmUWtxj3MpG",
        "colab_type": "text"
      },
      "source": [
        "___"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e_fiGfO33MpG",
        "colab_type": "text"
      },
      "source": [
        "# Get the interpixel distance"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AZk4CE3-3MpH",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "if microscope == \"LSM800\":\n",
        "    if magnification == \"20\":\n",
        "        interpixel_distance = 0.3119629\n",
        "        \n",
        "if microscope == \"Lois\":\n",
        "    if magnification == \"20\":\n",
        "        interpixel_distance = 0.690534"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rJZ0g6V83MpJ",
        "colab_type": "text"
      },
      "source": [
        "___"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eg8YfviX3MpK",
        "colab_type": "text"
      },
      "source": [
        "# Create sum projections for fluorescence analysis"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QdgHOU0x3MpK",
        "colab_type": "text"
      },
      "source": [
        "Compute sum of frames for each image:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "48gmDWam3MpL",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "for k1_condition, v1_sample in dicts_sum.items():\n",
        "    for k2_sample, v2_channel in v1_sample.items():\n",
        "        for k3_channel, im in v2_channel.items():\n",
        "            # Convert dict entry to array\n",
        "            image = np.array(im[\"matrix_orig\"])\n",
        "\n",
        "            # Sum z-project using numpy by definining the axis over which to sum the elements.\n",
        "            summed_image = image.sum(axis=0)\n",
        "\n",
        "            # Linearly scale down to 16-bit.\n",
        "            summed_image = (summed_image/summed_image.max())*65535\n",
        "\n",
        "            # Save summed image to dictionary\n",
        "            dicts_sum[k1_condition][k2_sample][k3_channel][\"matrix_sum\"] = summed_image"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WaWXfIF63MpN",
        "colab_type": "text"
      },
      "source": [
        "Display one sum projection image using skimage:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9O-Cdb8-3MpO",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "skimage.io.imshow(dicts_sum[k1_condition][k2_sample][k3_channel][\"matrix_sum\"])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SmGOo_br3MpQ",
        "colab_type": "text"
      },
      "source": [
        "___"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PTgyhnMq3MpQ",
        "colab_type": "text"
      },
      "source": [
        "# Save sum projection images \n",
        "(Turn this option on at the top of this notebook under \"parameters\".)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6afuMBY93MpR",
        "colab_type": "text"
      },
      "source": [
        "#### Scale down images:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "b-Al-CFn3MpR",
        "colab_type": "text"
      },
      "source": [
        "To save the images using skimage, we need to scale them down to 8 bit."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dh_nh1gG3MpR",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "if save_sum_ims == True:\n",
        "    # Scale down images to 8 bits\n",
        "    for k1_condition, v1_sample in dicts_sum.items():\n",
        "        for k2_sample, v2_channel in v1_sample.items():\n",
        "            for k3_channel, im in v2_channel.items():\n",
        "                # Linearly scale image down to 8-bit.\n",
        "                image = (im[\"matrix_sum\"] / im[\"matrix_sum\"].max()) * 255\n",
        "\n",
        "                # Change list to array and change type to 8-bit array.\n",
        "                image = np.array(image)\n",
        "                image = image.astype(np.uint8)\n",
        "\n",
        "                dicts_sum[k1_condition][k2_sample][k3_channel][\"matrix_8\"] = image"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rdVZLGbz3MpT",
        "colab_type": "text"
      },
      "source": [
        "#### Burn in scale bars:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QSnJqIYy3MpU",
        "colab_type": "text"
      },
      "source": [
        "Scale bar is burned into image by changing the pixel value to 1000 (white scale bar) or 0 (black scale bar) in scale bar area defined in the corresponding \"parameters\" cell above:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "F1zAU5nB3MpU",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "if save_sum_ims == True:\n",
        "    if scale_bar == True:\n",
        "\n",
        "        scalebar = 1 / interpixel_distance * scale_bar_length\n",
        "        scalebar_width = 1 / interpixel_distance * scale_bar_width\n",
        "\n",
        "        if scale_bar_color == \"white\":\n",
        "            for k1_condition, v1_sample in dicts_sum.items():\n",
        "                for k2_sample, v2_channel in v1_sample.items():\n",
        "                    for k3_channel, im in v2_channel.items():\n",
        "                        y_value = int((im[\"matrix_8\"].shape[0]/100)*y_pos)\n",
        "                        x_value = int((im[\"matrix_8\"].shape[1]/100)*x_pos) \n",
        "\n",
        "                        im[\"matrix_8\"][y_value : y_value + int(scalebar_width), x_value - int(scalebar) : x_value] = 255\n",
        "\n",
        "                        # Populate dict with 8-bit images containing a scale bar\n",
        "                        dicts_sum[k1_condition][k2_sample][k3_channel][\"matrix_8_SB\"] = im[\"matrix_8\"]\n",
        "\n",
        "        elif scale_bar_color == \"black\":\n",
        "            for k1_condition, v1_sample in dicts_sum.items():\n",
        "                for k2_sample, v2_channel in v1_sample.items():\n",
        "                    for k3_channel, im in v2_channel.items():\n",
        "                        y_pos = int((im[\"matrix_8\"].shape[0]/100)*y_pos)\n",
        "                        x_pos = int((im[\"matrix_8\"].shape[1]/100)*x_pos)\n",
        "\n",
        "                        im[\"matrix_8\"][y_pos : y_pos + int(scalebar_width), x_pos - int(scalebar) : x_pos] = 0\n",
        "\n",
        "                        # Populate dict with 8-bit images containing a scale bar\n",
        "                        dicts_sum[k1_condition][k2_sample][k3_channel][\"matrix_8_SB\"] = im[\"matrix_8\"]\n",
        "\n",
        "    else:\n",
        "        for k1_condition, v1_sample in dicts_sum.items():\n",
        "            for k2_sample, v2_channel in v1_sample.items():\n",
        "                for k3_channel, im in v2_channel.items():\n",
        "\n",
        "                    dicts_sum[k1_condition][k2_sample][k3_channel][\"matrix_8_SB\"] = im[\"matrix_8\"]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Jj94Ksb83MpW",
        "colab_type": "text"
      },
      "source": [
        "Display one image with scale bar:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FkKo9q8N3MpX",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "if save_sum_ims == True:\n",
        "    skimage.io.imshow(dicts_sum[k1_condition][k2_sample][k3_channel][\"matrix_8_SB\"])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xX05ImmN3MpZ",
        "colab_type": "text"
      },
      "source": [
        "### Create sum projection folder:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iB0uLT-63MpZ",
        "colab_type": "text"
      },
      "source": [
        "Create folder in saving directory to save maximum projections to:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "P-BbLsbe3Mpa",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "if save_sum_ims == True:\n",
        "    path = (\"{}/{}_sum_projections\").format(saving_dir, im_list[0].split(\"/\")[-2])\n",
        "    Path(path).mkdir(parents=True, exist_ok=True)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CEen2eg13Mpb",
        "colab_type": "text"
      },
      "source": [
        "### Save all images with a scale bar:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KCM-uI-V3Mpc",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "if save_sum_ims == True:\n",
        "    for k1_condition, v1_sample in dicts_sum.items():\n",
        "        for k2_sample, v2_channel in v1_sample.items():\n",
        "            for k3_channel, im in v2_channel.items():\n",
        "                skimage.io.imsave(\n",
        "                    (\"{}/{}_sum.{}\").format(path, im[\"filename\"], file_format),\n",
        "                    im[\"matrix_8_SB\"],\n",
        "                    plugin=None,\n",
        "                    check_contrast=True,\n",
        "                )"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cY52BPQd3Mpe",
        "colab_type": "text"
      },
      "source": [
        "___"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1Z7MhmXb3Mpe",
        "colab_type": "text"
      },
      "source": [
        "# Define ROIs "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Qwyq6JX73Mpf",
        "colab_type": "text"
      },
      "source": [
        "### Click points around area of interest to define an ROI (only draw one ROI)\n",
        "#### If you get an error using the bebi103.image.record_clicks package below, make sure to use websocket 8888. (Define localhost:8888/ in your URL.)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8pYa97DM3Mpf",
        "colab_type": "text"
      },
      "source": [
        "\"By default in bebi103.image.imshow(), the flip kwarg is set to True. This means that the image is flipped vertically so that it appears rightside up, since the indexing convention for images (2D arrays) starts in the upper left corner, but for axes, the origin is conventionally in the lower left corner. For recording clicks for use in image processing, you should be sure that the flip kwarg is False.\" - JB"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "R_SYHkaq3Mpg",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "for k1_condition, v1_sample in dicts_sum.items():\n",
        "    for k2_sample, v2_channel in v1_sample.items():\n",
        "        for k3_channel, im in v2_channel.items():\n",
        "            temp = bebi103.image.record_clicks(\n",
        "                dicts_sum[k1_condition][k2_sample][k3_channel][\"matrix_sum\"],\n",
        "                frame_height=500,\n",
        "                title=dicts_sum[k1_condition][k2_sample][k3_channel][\"filename\"],\n",
        "                flip=False\n",
        "            )\n",
        "\n",
        "            # Save clicks to dictionary\n",
        "            dicts_sum[k1_condition][k2_sample][k3_channel][\"clicks\"] = temp"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2Tedm8xZ3Mpi",
        "colab_type": "text"
      },
      "source": [
        "# STOP"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iCUo_16I3Mpj",
        "colab_type": "text"
      },
      "source": [
        "Convert clicks to a tidy data frame:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YBaJi6ZM3Mpj",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "for k1_condition, v1_sample in dicts_sum.items():\n",
        "    for k2_sample, v2_channel in v1_sample.items():\n",
        "        for k3_channel, im in v2_channel.items():\n",
        "            temp = dicts_sum[k1_condition][k2_sample][k3_channel][\"clicks\"].to_df()\n",
        "\n",
        "            # Add \"roi\" column (in this case there is just one ROI per sample with number \"0\")\n",
        "            temp[\"roi\"] = 0\n",
        "\n",
        "            # Save clicks to dictionary as tidy data frame (this will overwrite the previously saved version of the clicks!)\n",
        "            dicts_sum[k1_condition][k2_sample][k3_channel][\"clicks\"] = temp"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1xeOIt_b3Mpl",
        "colab_type": "text"
      },
      "source": [
        "Use the bebi103.image.verts_to_roi function to convert the set of vertices (clicks) that define a polygon to a region of interest (the inside of the polygon):"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ETqL9qQA3Mpl",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "for k1_condition, v1_sample in dicts_sum.items():\n",
        "    for k2_sample, v2_channel in v1_sample.items():\n",
        "        for k3_channel, im in v2_channel.items():\n",
        "            dicts_sum[k1_condition][k2_sample][k3_channel][\"rois\"] = [bebi103.image.verts_to_roi(dicts_sum[k1_condition][k2_sample][k3_channel][\"clicks\"][['x', 'y']].values, \n",
        "                                        dicts_sum[k1_condition][k2_sample][k3_channel][\"matrix_sum\"].shape[0], \n",
        "                                        dicts_sum[k1_condition][k2_sample][k3_channel][\"matrix_sum\"].shape[1])\n",
        "            for _, g in dicts_sum[k1_condition][k2_sample][k3_channel][\"clicks\"].groupby('roi')]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d36B5mEY3Mpn",
        "colab_type": "text"
      },
      "source": [
        "Check that the ROIs are correct:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TG6FhkWj3Mpn",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "plots = []\n",
        "\n",
        "for k1_condition, v1_sample in dicts_sum.items():\n",
        "    for k2_sample, v2_channel in v1_sample.items():\n",
        "        for k3_channel, im in v2_channel.items():\n",
        "            # The function above return 3 representation of our ROI:\n",
        "            # roi = Boolean matrix (mask) in the size of my original image with \"True\" values where the ROI is\n",
        "            # roi_bbox = Bounding box around ROI\n",
        "            # roi_box = Boolean matrix in the size of the bounding box (roi_bbox) with \"True\" values where the ROI is\n",
        "            roi, roi_bbox, roi_box = dicts_sum[k1_condition][k2_sample][k3_channel][\"rois\"][0]\n",
        "\n",
        "            # Make grayscale image that is now Rbkg/CMY\n",
        "            im = np.dstack(3 * [skimage.img_as_float(dicts_sum[k1_condition][k2_sample][k3_channel][\"matrix_sum\"])])\n",
        "\n",
        "            # Max out first channel\n",
        "            im[roi, 0] = im.max()\n",
        "            plots.append(\n",
        "                bebi103.image.imshow(\n",
        "                    im,\n",
        "                    title=dicts_sum[k1_condition][k2_sample][k3_channel][\"filename\"],\n",
        "                    frame_height=250,\n",
        "                    cmap=\"rgb\",\n",
        "                    flip=False,\n",
        "                )\n",
        "            )\n",
        "\n",
        "# Look at the images\n",
        "bokeh.io.show(bokeh.layouts.gridplot(plots, ncols=3))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "svRV80Pc3Mpp",
        "colab_type": "text"
      },
      "source": [
        "### Save mean fluorescence value from ROI"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MJ7u_QCH3Mpq",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "for k1_condition, v1_sample in dicts_sum.items():\n",
        "    for k2_sample, v2_channel in v1_sample.items():\n",
        "        for k3_channel, im in v2_channel.items():\n",
        "            roi, roi_bbox, roi_box = dicts_sum[k1_condition][k2_sample][k3_channel][\"rois\"][0]\n",
        "            im = dicts_sum[k1_condition][k2_sample][k3_channel][\"matrix_sum\"][roi]\n",
        "\n",
        "            # Save mean and median pixel value of ROI to dictionary\n",
        "            dicts_sum[k1_condition][k2_sample][k3_channel][\"mean_int\"] = im.mean()\n",
        "            dicts_sum[k1_condition][k2_sample][k3_channel][\"median_int\"] = np.median(im)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uofO1BHf3Mps",
        "colab_type": "text"
      },
      "source": [
        "___"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qL8mLXul3Mpt",
        "colab_type": "text"
      },
      "source": [
        "# Define background fluorescence area"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1kRJ3PIy3Mpt",
        "colab_type": "text"
      },
      "source": [
        "Define a small square outside of the ROI which is representative of the background fluorescence. Each pixel of the ROI will be divided by the average pixel value of the background before the overall fluorecence in the ROI is summed.  "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YCg6Z8863Mpt",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "for k1_condition, v1_sample in dicts_sum.items():\n",
        "    for k2_sample, v2_channel in v1_sample.items():\n",
        "        for k3_channel, im in v2_channel.items():\n",
        "            temp = bebi103.image.record_clicks(\n",
        "                dicts_sum[k1_condition][k2_sample][k3_channel][\"matrix_sum\"],\n",
        "                frame_height=500,\n",
        "                title=dicts_sum[k1_condition][k2_sample][k3_channel][\"filename\"],\n",
        "                flip=False\n",
        "            )\n",
        "\n",
        "            # Save clicks to dictionary\n",
        "            dicts_sum[k1_condition][k2_sample][k3_channel][\"clicks_bkg\"] = temp"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cTyklEnB3Mpv",
        "colab_type": "text"
      },
      "source": [
        "# STOP"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zDEmrNXO3Mpv",
        "colab_type": "text"
      },
      "source": [
        "Convert clicks to a tidy data frame:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "a7wfEJNT3Mpw",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "for k1_condition, v1_sample in dicts_sum.items():\n",
        "    for k2_sample, v2_channel in v1_sample.items():\n",
        "        for k3_channel, im in v2_channel.items():\n",
        "            temp = dicts_sum[k1_condition][k2_sample][k3_channel][\"clicks_bkg\"].to_df()\n",
        "\n",
        "            # Add \"roi\" column (in this case there is just one ROI per sample with number \"0\")\n",
        "            temp[\"roi\"] = 0\n",
        "\n",
        "            # Save clicks to dictionary as tidy data frame (this will overwrite the previously saved version of the clicks!)\n",
        "            dicts_sum[k1_condition][k2_sample][k3_channel][\"clicks_bkg\"] = temp"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "D3XG5F9N3Mpy",
        "colab_type": "text"
      },
      "source": [
        "Use the bebi103.image.verts_to_roi function to convert the set of vertices (clicks) that define a polygon to a region of interest (the inside of the polygon):"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3OyS48Fy3Mpy",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "for k1_condition, v1_sample in dicts_sum.items():\n",
        "    for k2_sample, v2_channel in v1_sample.items():\n",
        "        for k3_channel, im in v2_channel.items():\n",
        "            dicts_sum[k1_condition][k2_sample][k3_channel][\"rois_bkg\"] = [bebi103.image.verts_to_roi(dicts_sum[k1_condition][k2_sample][k3_channel][\"clicks_bkg\"][['x', 'y']].values, \n",
        "                                        dicts_sum[k1_condition][k2_sample][k3_channel][\"matrix_sum\"].shape[0], \n",
        "                                        dicts_sum[k1_condition][k2_sample][k3_channel][\"matrix_sum\"].shape[1])\n",
        "            for _, g in dicts_sum[k1_condition][k2_sample][k3_channel][\"clicks_bkg\"].groupby('roi')]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RnWh0YCv3Mp0",
        "colab_type": "text"
      },
      "source": [
        "Check that background areas were drawn correctly:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EDwxfv7q3Mp1",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "plots = []\n",
        "\n",
        "for k1_condition, v1_sample in dicts_sum.items():\n",
        "    for k2_sample, v2_channel in v1_sample.items():\n",
        "        for k3_channel, im in v2_channel.items():\n",
        "            # The function above return 3 representation of our ROI:\n",
        "            # roi = Boolean matrix (mask) in the size of my original image with \"True\" values where the ROI is\n",
        "            # roi_bbox = Bounding box around ROI\n",
        "            # roi_box = Boolean matrix in the size of the bounding box (roi_bbox) with \"True\" values where the ROI is\n",
        "            roi, roi_bbox, roi_box = dicts_sum[k1_condition][k2_sample][k3_channel][\"rois_bkg\"][0]\n",
        "\n",
        "            # Make grayscale image that is now Rbkg/CMY\n",
        "            im = np.dstack(3 * [skimage.img_as_float(dicts_sum[k1_condition][k2_sample][k3_channel][\"matrix_sum\"])])\n",
        "\n",
        "            # Max out pixel value in first channel\n",
        "            im[roi, 0] = im.max()\n",
        "\n",
        "            plots.append(\n",
        "                bebi103.image.imshow(\n",
        "                    im,\n",
        "                    title=dicts_sum[k1_condition][k2_sample][k3_channel][\"filename\"],\n",
        "                    frame_height=250,\n",
        "                    cmap=\"rgb\",\n",
        "                    flip=False,\n",
        "                )\n",
        "            )\n",
        "\n",
        "# Look at the images\n",
        "bokeh.io.show(bokeh.layouts.gridplot(plots, ncols=3))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "H829dBKL3Mp3",
        "colab_type": "text"
      },
      "source": [
        "# Save mean fluorescence value from background area"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "S0wIJ3NR3Mp3",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "for k1_condition, v1_sample in dicts_sum.items():\n",
        "    for k2_sample, v2_channel in v1_sample.items():\n",
        "        for k3_channel, im in v2_channel.items():\n",
        "            roi, roi_bbox, roi_box = dicts_sum[k1_condition][k2_sample][k3_channel][\"rois_bkg\"][0]\n",
        "            im = dicts_sum[k1_condition][k2_sample][k3_channel][\"matrix_sum\"][roi]\n",
        "\n",
        "            # Save fluorescence intensity of ROI to dictionary\n",
        "            dicts_sum[k1_condition][k2_sample][k3_channel][\"mean_int_bkg\"] = im.mean()\n",
        "            dicts_sum[k1_condition][k2_sample][k3_channel][\"median_int_bkg\"] = np.median(im)           "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EXnbKt3O3Mp5",
        "colab_type": "text"
      },
      "source": [
        "___"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Z14pvggZ3Mp6",
        "colab_type": "text"
      },
      "source": [
        "# Compute raw integrated density and integrated density of ROI from bkg normalized pixel values and compute Fold Change"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AbQWG-nG3Mp6",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "for k1_condition, v1_sample in dicts_sum.items():\n",
        "    for k2_sample, v2_channel in v1_sample.items():\n",
        "        for k3_channel, im in v2_channel.items():    \n",
        "            roi, roi_bbox, roi_box = dicts_sum[k1_condition][k2_sample][k3_channel][\"rois\"][0]\n",
        "            im = dicts_sum[k1_condition][k2_sample][k3_channel][\"matrix_sum\"][roi]\n",
        "            \n",
        "            pixel_norm = []\n",
        "            \n",
        "            # Get mean and median pixel values of ROI and bkg\n",
        "            int_mean = dicts_sum[k1_condition][k2_sample][k3_channel][\"mean_int\"]\n",
        "            bkg_mean = dicts_sum[k1_condition][k2_sample][k3_channel][\"mean_int_bkg\"]\n",
        "            int_median = dicts_sum[k1_condition][k2_sample][k3_channel][\"median_int\"]\n",
        "            bkg_median = dicts_sum[k1_condition][k2_sample][k3_channel][\"median_int_bkg\"]\n",
        "            \n",
        "            # Divide each pixel value by the average background pixel value\n",
        "            for pixel in im:\n",
        "                pixel_norm.append(pixel / bkg_mean)\n",
        "                \n",
        "            ## Compute bkg normalized raw integrated density of ROI and save to dictionary\n",
        "            dicts_sum[k1_condition][k2_sample][k3_channel][\"raw_int_den\"] = sum(pixel_norm)\n",
        "            \n",
        "            ## Compute bkg normalized integrated density of ROI and save to dictionary\n",
        "            # Calculate the number of pixels in the ROI\n",
        "            roi_area = 0\n",
        "            for i in range(len(roi)):\n",
        "                x = np.sum(roi[i])\n",
        "                roi_area = roi_area + x\n",
        "                \n",
        "            pixel_norm = np.array(pixel_norm)\n",
        "                \n",
        "            dicts_sum[k1_condition][k2_sample][k3_channel][\"int_den\"] = pixel_norm.mean() * roi_area\n",
        "            \n",
        "            ## Compute mean and median Fold Change and save to dictionary  \n",
        "            dicts_sum[k1_condition][k2_sample][k3_channel][\"mean_fold_change\"] = (int_mean - bkg_mean) / bkg_mean\n",
        "            \n",
        "            if bkg_median != 0:\n",
        "                dicts_sum[k1_condition][k2_sample][k3_channel][\"median_fold_change\"] = (int_median - bkg_median) / bkg_median \n",
        "            else: \n",
        "                dicts_sum[k1_condition][k2_sample][k3_channel][\"median_fold_change\"] = int_median"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "D5jQuflp3Mp8",
        "colab_type": "text"
      },
      "source": [
        "___"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UaUw6o2a3Mp8",
        "colab_type": "text"
      },
      "source": [
        "# Pickle sum fluorescence dictionary for later use\n",
        "This dictionary contains the sum fluorescence images, names of the images, clicks as data frame, rois and the sum of the pixel intensity values inside the roi.   "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "j1op_16-3Mp9",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "path = (\"{}\").format(saving_dir)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5KClgcZ23Mp_",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Use pickle to save dictionaries\n",
        "\n",
        "# The advantage of HIGHEST_PROTOCOL is that files get smaller. This makes unpickling sometimes much faster.\n",
        "with open(\n",
        "    (\"{}/{}_fluorescence_analysis.pickle\").format(path, im_list[0].split(\"/\")[-2]),\n",
        "    \"wb\",\n",
        ") as handle:\n",
        "    pickle.dump(dicts_sum, handle, protocol=pickle.HIGHEST_PROTOCOL)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RM4zC0L_3MqC",
        "colab_type": "text"
      },
      "source": [
        "___"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XaqH4vcW3MqC",
        "colab_type": "text"
      },
      "source": [
        "# Create a data frame and csv file for comparison of pixel values across conditions"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ooHqqYto3MqC",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "df = pd.DataFrame()\n",
        "\n",
        "for k1_condition, v1_sample in dicts_sum.items():\n",
        "    for k2_sample, v2_channel in v1_sample.items():\n",
        "        for k3_channel, im in v2_channel.items(): \n",
        "            df = df.append(\n",
        "                {\"Filename\" : dicts_sum[k1_condition][k2_sample][k3_channel][\"filename\"],\n",
        "                 \"Condition\" : k1_condition,\n",
        "                 \"Sample\" : k2_sample,\n",
        "                 \"Channel\" : k3_channel,\n",
        "                 \"Condition + Channel\" : (k1_condition + \" (\" + k3_channel + \")\"),\n",
        "                 \"Mean pixel value in ROI\" : dicts_sum[k1_condition][k2_sample][k3_channel][\"mean_int\"],\n",
        "                 \"Mean pixel value in bkg area\" : dicts_sum[k1_condition][k2_sample][k3_channel][\"mean_int_bkg\"],\n",
        "                 \"Median pixel value in ROI\" : dicts_sum[k1_condition][k2_sample][k3_channel][\"median_int\"],\n",
        "                 \"Median pixel value in bkg area\" : dicts_sum[k1_condition][k2_sample][k3_channel][\"median_int_bkg\"],\n",
        "                 \"Norm Raw Integrated Density\" : dicts_sum[k1_condition][k2_sample][k3_channel][\"raw_int_den\"],\n",
        "                 \"Norm Integrated Density\" : dicts_sum[k1_condition][k2_sample][k3_channel][\"int_den\"],\n",
        "                 \"Mean Fold Change\" : dicts_sum[k1_condition][k2_sample][k3_channel][\"mean_fold_change\"],\n",
        "                 \"Median Fold Change\" : dicts_sum[k1_condition][k2_sample][k3_channel][\"median_fold_change\"]}, \n",
        "                ignore_index=True)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Gk_m3Cn33MqF",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Sort the data frame by condition\n",
        "df = pd.DataFrame.sort_values(df, \"Condition\", ascending=False)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "70ju_RT-3MqI",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Display top of data frame\n",
        "df.head()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1cdFu5HQ3MqK",
        "colab_type": "text"
      },
      "source": [
        "___"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "u9_1TGi33MqK",
        "colab_type": "text"
      },
      "source": [
        "# Save data frame to csv"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WPCtk8WS3MqK",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "df.to_csv(\"{}/{}_fluorescence_analysis.csv\".format(saving_dir, data_dir.split(\"/\")[-1]), index=False)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nbnklwFI3MqM",
        "colab_type": "text"
      },
      "source": [
        "___"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6PTYyWN53MqN",
        "colab_type": "text"
      },
      "source": [
        "# Plot mean fold change"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QKdicXj33MqN",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Sort the data frame by channel to separate them in plots\n",
        "df = pd.DataFrame.sort_values(df, \"Channel\", ascending=False)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pcq1T-It3MqO",
        "colab_type": "text"
      },
      "source": [
        "Example scatter plot using Matplotlib:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "82c777sj3MqP",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "names = df[\"Condition + Channel\"]\n",
        "values = df[\"Mean Fold Change\"]\n",
        "\n",
        "color = \"black\"\n",
        "dotsize = 20\n",
        "\n",
        "fig, ax = plt.subplots(figsize=(20, 7))\n",
        "\n",
        "plt.scatter(names, values, c=color, s=dotsize)\n",
        "\n",
        "# Define figure title\n",
        "ax.set_title('Relative fluorescence units across conditions', weight='bold', size=17)\n",
        "\n",
        "# Set axis labels\n",
        "fontsize = 13\n",
        "fontweight = 'bold'\n",
        "fontproperties = {'weight':fontweight, 'size':fontsize}\n",
        "ax.set_xlabel('Condition', fontproperties)\n",
        "ax.set_ylabel('Fold Change', fontproperties)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "04ccPzLm3MqR",
        "colab_type": "text"
      },
      "source": [
        "Example boxplot using Matplotlib:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "b9uwCsiL3MqS",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "names = df[\"Condition\"]\n",
        "values = df[\"Mean Fold Change\"]\n",
        "\n",
        "color = \"black\"\n",
        "dotsize = 20\n",
        "\n",
        "fig, ax = plt.subplots(figsize=(20, 7))\n",
        "\n",
        "labels = names.unique()\n",
        "\n",
        "# Create boxplot\n",
        "boxplot = []\n",
        "for i in names.unique():\n",
        "    boxplot.append(df.loc[df['Condition']==i, :][\"Mean Fold Change\"].values)\n",
        "\n",
        "# ax.boxplot(boxplot) \n",
        "b_plot = ax.boxplot(boxplot, labels=labels) \n",
        "for patch in b_plot[\"boxes\"]:\n",
        "    patch.set_alpha(0.2)\n",
        "\n",
        "# Overlay scatter plot\n",
        "ax.scatter(pd.factorize(names)[0]+1, values, s=dotsize, zorder=10, c=color)\n",
        "\n",
        "# Define figure title\n",
        "ax.set_title('Relative fluorescence units across conditions',weight='bold',size=17)\n",
        "\n",
        "# Set axis labels\n",
        "fontsize = 13\n",
        "fontweight = 'bold'\n",
        "fontproperties = {'weight':fontweight, 'size':fontsize}\n",
        "ax.set_xlabel('Condition', fontproperties)\n",
        "ax.set_ylabel('Fold Change', fontproperties)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GOJ7n5N_3MqU",
        "colab_type": "text"
      },
      "source": [
        "Example barplot using Matplotlib:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Yp8vZua93MqU",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "names = df[\"Condition + Channel\"]\n",
        "values = df[\"Mean Fold Change\"]\n",
        "\n",
        "fig, ax = plt.subplots(figsize=(20, 7))\n",
        "\n",
        "plt.bar(names, values)\n",
        "\n",
        "# Define figure title\n",
        "ax.set_title('Relative fluorescence units across conditions', weight='bold', size=17)\n",
        "\n",
        "# Set axis labels\n",
        "fontsize = 13\n",
        "fontweight = 'bold'\n",
        "fontproperties = {'weight':fontweight, 'size':fontsize}\n",
        "ax.set_xlabel('Condition', fontproperties)\n",
        "ax.set_ylabel('Fold Change', fontproperties)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DWhT4C1_3MqX",
        "colab_type": "text"
      },
      "source": [
        "___"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "W2CK5kY93MqX",
        "colab_type": "text"
      },
      "source": [
        "# Plot median fold change"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MOxszHk03MqX",
        "colab_type": "text"
      },
      "source": [
        "Example scatter plot using Matplotlib:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CxDvSs3E3MqY",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "names = df[\"Condition + Channel\"]\n",
        "values = df[\"Median Fold Change\"]\n",
        "\n",
        "color = \"black\"\n",
        "dotsize = 20\n",
        "\n",
        "fig, ax = plt.subplots(figsize=(20, 7))\n",
        "\n",
        "plt.scatter(names, values, c=color, s=dotsize)\n",
        "\n",
        "# Define figure title\n",
        "ax.set_title('Relative fluorescence units across conditions', weight='bold', size=17)\n",
        "\n",
        "# Set axis labels\n",
        "fontsize = 13\n",
        "fontweight = 'bold'\n",
        "fontproperties = {'weight':fontweight, 'size':fontsize}\n",
        "ax.set_xlabel('Condition', fontproperties)\n",
        "ax.set_ylabel('Fold Change', fontproperties)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iJUEz6kt3Mqa",
        "colab_type": "text"
      },
      "source": [
        "Example boxplot using Matplotlib:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Yc_e4-Wo3Mqa",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "names = df[\"Condition\"]\n",
        "values = df[\"Median Fold Change\"]\n",
        "\n",
        "color = \"black\"\n",
        "dotsize = 20\n",
        "\n",
        "fig, ax = plt.subplots(figsize=(20, 7))\n",
        "\n",
        "labels = names.unique()\n",
        "\n",
        "# Create boxplot\n",
        "boxplot = []\n",
        "for i in names.unique():\n",
        "    boxplot.append(df.loc[df['Condition']==i, :][\"Median Fold Change\"].values)\n",
        "\n",
        "# ax.boxplot(boxplot) \n",
        "b_plot = ax.boxplot(boxplot, labels=labels) \n",
        "for patch in b_plot[\"boxes\"]:\n",
        "    patch.set_alpha(0.2)\n",
        "\n",
        "# Overlay scatter plot\n",
        "ax.scatter(pd.factorize(names)[0]+1, values, s=dotsize, zorder=10, c=color)\n",
        "\n",
        "# Define figure title\n",
        "ax.set_title('Relative fluorescence units across conditions',weight='bold',size=17)\n",
        "\n",
        "# Set axis labels\n",
        "fontsize = 13\n",
        "fontweight = 'bold'\n",
        "fontproperties = {'weight':fontweight, 'size':fontsize}\n",
        "ax.set_xlabel('Condition', fontproperties)\n",
        "ax.set_ylabel('Fold Change', fontproperties)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "L8yDPUXa3Mqd",
        "colab_type": "text"
      },
      "source": [
        "Example barplot using Matplotlib:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ObAAzd3k3Mqd",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "names = df[\"Condition + Channel\"]\n",
        "values = df[\"Median Fold Change\"]\n",
        "\n",
        "fig, ax = plt.subplots(figsize=(20, 7))\n",
        "\n",
        "plt.bar(names, values)\n",
        "\n",
        "# Define figure title\n",
        "ax.set_title('Relative fluorescence units across conditions', weight='bold', size=17)\n",
        "\n",
        "# Set axis labels\n",
        "fontsize = 13\n",
        "fontweight = 'bold'\n",
        "fontproperties = {'weight':fontweight, 'size':fontsize}\n",
        "ax.set_xlabel('Condition', fontproperties)\n",
        "ax.set_ylabel('Fold Change', fontproperties)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CmeJwdWz3Mqg",
        "colab_type": "text"
      },
      "source": [
        "___"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mtsQJN7H3Mqg",
        "colab_type": "text"
      },
      "source": [
        "# Plot normalized Integrated Density"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8qhr22u03Mqi",
        "colab_type": "text"
      },
      "source": [
        "Example scatter plot using Matplotlib:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-kpOuAKy3Mqi",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "names = df[\"Condition + Channel\"]\n",
        "values = df[\"Norm Integrated Density\"]\n",
        "\n",
        "color = \"black\"\n",
        "dotsize = 20\n",
        "\n",
        "fig, ax = plt.subplots(figsize=(20, 7))\n",
        "\n",
        "plt.scatter(names, values, c=color, s=dotsize)\n",
        "\n",
        "# Define figure title\n",
        "ax.set_title('Relative fluorescence units across conditions', weight='bold', size=17)\n",
        "\n",
        "# Set axis labels\n",
        "fontsize = 13\n",
        "fontweight = 'bold'\n",
        "fontproperties = {'weight':fontweight, 'size':fontsize}\n",
        "ax.set_xlabel('Condition', fontproperties)\n",
        "ax.set_ylabel('Relative Fluorescence Units', fontproperties)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ihhiekud3Mqj",
        "colab_type": "text"
      },
      "source": [
        "Example boxplot using Matplotlib:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ek9_2V3R3Mqk",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "names = df[\"Condition\"]\n",
        "values = df[\"Norm Integrated Density\"]\n",
        "\n",
        "color = \"black\"\n",
        "dotsize = 20\n",
        "\n",
        "fig, ax = plt.subplots(figsize=(20, 7))\n",
        "\n",
        "labels = names.unique()\n",
        "\n",
        "# Create boxplot\n",
        "boxplot = []\n",
        "for i in names.unique():\n",
        "    boxplot.append(df.loc[df['Condition']==i, :][\"Norm Integrated Density\"].values)\n",
        "\n",
        "# ax.boxplot(boxplot) \n",
        "b_plot = ax.boxplot(boxplot, labels=labels) \n",
        "for patch in b_plot[\"boxes\"]:\n",
        "    patch.set_alpha(0.2)\n",
        "\n",
        "# Overlay scatter plot\n",
        "ax.scatter(pd.factorize(names)[0]+1, values, s=dotsize, zorder=10, c=color)\n",
        "\n",
        "# Define figure title\n",
        "ax.set_title('Relative fluorescence units across conditions',weight='bold',size=17)\n",
        "\n",
        "# Set axis labels\n",
        "fontsize = 13\n",
        "fontweight = 'bold'\n",
        "fontproperties = {'weight':fontweight, 'size':fontsize}\n",
        "ax.set_xlabel('Condition', fontproperties)\n",
        "ax.set_ylabel('Relative Fluorescence Units', fontproperties)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZNNgb8BD3Mql",
        "colab_type": "text"
      },
      "source": [
        "Example barplot using Matplotlib:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pYoPv25D3Mql",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "names = df[\"Condition + Channel\"]\n",
        "values = df[\"Norm Integrated Density\"]\n",
        "\n",
        "fig, ax = plt.subplots(figsize=(20, 7))\n",
        "\n",
        "plt.bar(names, values)\n",
        "\n",
        "# Define figure title\n",
        "ax.set_title('Relative fluorescence units across conditions', weight='bold', size=17)\n",
        "\n",
        "# Set axis labels\n",
        "fontsize = 13\n",
        "fontweight = 'bold'\n",
        "fontproperties = {'weight':fontweight, 'size':fontsize}\n",
        "ax.set_xlabel('Condition', fontproperties)\n",
        "ax.set_ylabel('Relative Fluorescence Units', fontproperties)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uxJuwDNE3Mqp",
        "colab_type": "text"
      },
      "source": [
        "___"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4Iw6LMDM3Mqq",
        "colab_type": "text"
      },
      "source": [
        "# Computing environment"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Rd7dvB8A3Mqq",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%load_ext watermark\n",
        "\n",
        "%watermark -v -p re,numpy,pandas,bebi103,skimage,matplotlib,panel,bokeh,bokeh_catplot,jupyterlab"
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}