Alternate_POSIX.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "8369f35a-17cc-46ff-b1f0-dc8c5439100c",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "# EMBO Practical Course <br/>\"Advanced methods in bioimage analysis\"\n",
    "\n",
    "***\n",
    "\n",
    "Homepage: https://www.embl.org/about/info/course-and-conference-office/events/bia23-01/\n",
    "\n",
    "***\n",
    "\n",
    "## Day 2 - Session 1: Image Data Management - 10:15 to 11:15 \"Get set\"\n",
    "\n",
    "Note: This is an alternative and slightly earlier version of 3_POSIX and 4_OMETiff combined in a single notebook for illustration purposes."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dda5d895-3805-439a-909a-2e4f95c17756",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "## 3a. Bash basics\n",
    "\n",
    "If you are managing files on a server, it will often be easier to use the command-line once you learn it. And if you are using a command-line, it's likely going to be Bash.\n",
    "\n",
    "There are a large number of useful commands. Often, the shorter the name the more useful it is. Let's take `ls` for \"list\".\n",
    "\n",
    "To learn more about a tool, you can use the `man` tool."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "id": "9f7bd03e-464a-42c3-88a4-556318571e54",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LS(1)                       General Commands Manual                      LS(1)\n",
      "\n",
      "NAME\n",
      "     ls – list directory contents\n",
      "\n",
      "SYNOPSIS\n",
      "     ls [-@ABCFGHILOPRSTUWabcdefghiklmnopqrstuvwxy1%,] [--color=when]\n",
      "        [-D format] [file ...]\n",
      "\n",
      "DESCRIPTION\n",
      "cat: stdout: Broken pipe\n"
     ]
    }
   ],
   "source": [
    "!man ls | head"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e1371242-8a28-448e-9c7b-e3e8bc9cb97d",
   "metadata": {},
   "source": [
    "<br/>\n",
    "<br/>\n",
    "\n",
    "I bring up `ls` first because it can be used to find other commands:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "ccebbfb1-275c-4391-a8af-5da4162fe861",
   "metadata": {
    "editable": true,
    "scrolled": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "drwxr-xr-x   936 root     wheel  29952 Jul 11 10:56 \u001b[1m\u001b[36m/usr/bin/\u001b[m\u001b[m\n",
      "drwxr-xr-x@   39 root     wheel   1248 Jul 11 10:56 \u001b[1m\u001b[36m/bin\u001b[m\u001b[m\n",
      "drwxr-xr-x  2086 jamoore  admin  66752 Aug 20 18:35 \u001b[1m\u001b[36m/usr/local/bin\u001b[m\u001b[m\n"
     ]
    }
   ],
   "source": [
    "!ls -ltrad /bin /usr/bin/ /usr/local/bin"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bc5f8b7c-8ebd-420d-91a4-f27c21c38608",
   "metadata": {},
   "source": [
    "These are the directories where you will find many of them. For example, the command `bash` is one of them:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "03a47295-9aee-4bc3-8400-2b6c0ccd3da6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/bin/bash\n"
     ]
    }
   ],
   "source": [
    "!which bash"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d0835bce-d3df-48d8-99f7-e4e21ae0c51c",
   "metadata": {},
   "source": [
    "`which` is a command that tells you where a command lives. `/bin/bash` is the command that gets run when you login. It's your \"shell\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "f94217ec-c93d-459d-9e59-34d4ef87fd14",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/bin/bash\n"
     ]
    }
   ],
   "source": [
    "!echo $SHELL"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c63b096-1333-4721-9944-6cecfe7a8893",
   "metadata": {},
   "source": [
    "So it's the context that everything you are doing is happening in:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "07b69ad4-297a-4eb4-bbc4-0fb720717765",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-+= 00001 root /sbin/launchd\n",
      " \\-+= 00560 jamoore /Applications/iTerm.app/Contents/MacOS/iTerm2\n",
      "   \\-+- 02144 jamoore /Users/jamoore/Library/Application Support/iTerm2/iTermSe\n",
      "     \\-+= 12058 root login -fp jamoore\n",
      "       \\-+= 12059 jamoore -bash\n",
      "         \\-+= 12310 jamoore /Users/jamoore/micromamba/envs/embo/bin/python3.9 /\n",
      "           \\-+= 18104 jamoore /Users/jamoore/micromamba/envs/embo/bin/python -m\n",
      "             \\-+= 18150 jamoore pstree -p 18150\n",
      "               \\--- 18151 root ps -axwwo user,pid,ppid,pgid,command\n"
     ]
    }
   ],
   "source": [
    "!pstree -p $$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b0730c3-68ef-4f37-b15d-1c6e658a8937",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "## 3b. Mixing BASH & Jupyter\n",
    "\n",
    "Each Jupyter notebook runs within a Python interpreter, but that Python interpreter runs within a Bash shell. Each cell starting with \"!\" starts a **new** Bash shell. Therefore the first time you run the cells below you won't be able to just set a variable.\n",
    "\n",
    "This is important because you will need to know the state of the different shells/interpreters, or at least, if you don't it can be confusing. (⚠ Jupyter notebooks can already be confusing because you can change the order of things that you are doing.)\n",
    "\n",
    "For example, we will use the `whoami` command to set a variable for creating directories for everyone:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "1d0dd013-239c-4049-8805-16c3c9640c03",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "jamoore\n"
     ]
    }
   ],
   "source": [
    "!whoami"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "69cda76a-0a22-4c9c-9b4e-e026f80dab8f",
   "metadata": {},
   "source": [
    "You can try to set this variable in Bash like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "9aff5364-ac01-4dbc-91dd-25ab299b81df",
   "metadata": {},
   "outputs": [],
   "source": [
    "!YOURNAME=$(whoami)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "67d7483e-d75a-4fbc-b9ce-906658166e71",
   "metadata": {},
   "source": [
    "But when you try to use it, it will be empty:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "2765d3be-cad0-4b01-9112-cefbf2153ab7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "!echo $YOURNAME"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a18bfe31-3958-459d-a727-581f5510c6d5",
   "metadata": {},
   "source": [
    "What's happened is that the `!` here created a new Bash shell, ran your command, and then that shell exited. Changes made have disappeared.\n",
    "\n",
    "Instead, for this action we will need to make use of Python to update the *parent shell*:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "a7e6da5c-ca7a-4bb5-b4e9-0e8fec0a3f7a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "env: YOURNAME=jamoore\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "YOURNAME = os.getlogin()\n",
    "%env YOURNAME=$YOURNAME"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "e1de7ba4-de1d-4199-8c1c-4ae81693ef6e",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "jamoore\n"
     ]
    }
   ],
   "source": [
    "!echo $YOURNAME"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20863384-d2d0-489c-a78e-e269be279389",
   "metadata": {},
   "source": [
    "Now we have the variable we want set."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "43d03269-a7bf-4147-ab25-f8d5ddc0709a",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "## 3c. Our data directory"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d70a7bf9-fcd6-4f8a-8286-79c9c7c7a263",
   "metadata": {},
   "source": [
    "Now with this out of the way, we can start to work with the data directory \n",
    "\n",
    "`pwd` prints the directory that the current process is in:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "4b67eee0-1222-4b54-999a-7f0ebedd0426",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'/opt/EMBO-Workshop-2023'"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pwd"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2a91c8a-d402-4a3f-b107-7327c4c72c73",
   "metadata": {},
   "source": [
    "We want to move to another directory, but again we can't just use `cd MYDIR` because that will only change the subshell. Instead we use Jupyter magic (commands starting with `%`):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "2ff1acb9-82c9-450f-a561-d62e68282350",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/System/Volumes/Data/scratch/bioimagecourse2023/session1\n"
     ]
    }
   ],
   "source": [
    "%cd /scratch/bioimagecourse2023/session1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b833eadd-c28d-4d33-8f60-c635d6887fb3",
   "metadata": {},
   "source": [
    "Now we are in our working directory. Feel free to look around using `ls`. For example the flags `-ltra` mean: \"show me a long listing of the files in reverse order by time and even show me the weird files starting with `.`\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "27ee9b79-ee85-41e5-b42a-75cb4e727e59",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "total 248\n",
      "-rw-r--r--   1 jamoore  wheel    529 Aug 23 18:13 2a.sh\n",
      "-rw-r--r--   1 jamoore  wheel  42000 Aug 23 18:13 .2a.sh.un~\n",
      "-rw-r--r--   1 jamoore  wheel     69 Aug 23 18:16 notes\n",
      "-rw-r--r--   1 jamoore  wheel   3123 Aug 23 18:16 .notes.un~\n",
      "drwxr-xr-x   5 jamoore  wheel    160 Aug 23 18:31 \u001b[1m\u001b[36mdata\u001b[m\u001b[m\n",
      "-rw-r--r--   1 jamoore  wheel    469 Aug 23 18:50 2b.sh\n",
      "-rw-r--r--   1 jamoore  wheel  27560 Aug 23 18:50 .2b.sh.un~\n",
      "-rw-r--r--   1 jamoore  wheel    638 Aug 23 18:58 3a.sh\n",
      "-rw-r--r--   1 jamoore  wheel  31925 Aug 23 18:58 .3a.sh.un~\n",
      "drwxr-xr-x  11 jamoore  wheel    352 Aug 23 18:58 \u001b[1m\u001b[36m.\u001b[m\u001b[m\n",
      "drwxr-xr-x   3 jamoore  wheel     96 Aug 24 16:39 \u001b[1m\u001b[36m..\u001b[m\u001b[m\n"
     ]
    }
   ],
   "source": [
    "!ls -ltra"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "992717a4-4e74-40b6-8069-71c4ec380eb5",
   "metadata": {},
   "source": [
    "The \"-S\" flag means sort by size instead of time:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "8a2c1979-2527-4ec5-bc79-97e766a85302",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "total 0\n",
      "-rw-r--r--  1 jamoore  wheel   0 Aug 23 18:01 a.fake\n",
      "lrwxr-xr-x  1 jamoore  wheel  33 Aug 23 18:12 \u001b[1m\u001b[35mdata\u001b[m\u001b[m@ -> /scratch/josh_openmicroscopy/data\n",
      "drwxr-xr-x  3 jamoore  wheel  96 Aug 23 18:31 \u001b[1m\u001b[36mcellprofiler\u001b[m\u001b[m/\n"
     ]
    }
   ],
   "source": [
    "ls -lSr data/"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bdd06ad2-4054-4e52-aad0-82d4ec73545c",
   "metadata": {},
   "source": [
    "Another important tool for understanding the size of your data is `du`, for \"disk usage\". The `-sh` flags \"say show me only a (s)ummary of the data and put it in (h)uman-readable form\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "2f9aa050-0d2d-4270-bb67-e14531b0705d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "8.7M\tdata/\n"
     ]
    }
   ],
   "source": [
    "! du -sh data/"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "826687e7-cca8-4d5f-9a06-305b1f91d5f0",
   "metadata": {},
   "source": [
    "Now we want to make a directory for you to do all of your work in. The \"-p\" flag means \"create all parents\" (but also: \"don't fail if it also exists\" which is useful for Jupyter notebooks!)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "5fc481cd-dae9-493c-ae7c-c3fb6b49b457",
   "metadata": {},
   "outputs": [],
   "source": [
    "!mkdir -p /scratch/${YOURNAME}/session1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "f9833e87-1658-41c2-886a-c7f6e6a3fbfe",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/System/Volumes/Data/scratch/jamoore/session1\n"
     ]
    }
   ],
   "source": [
    "%cd /scratch/{YOURNAME}/session1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c2e57915-42ff-439d-a96a-95531ce44edb",
   "metadata": {
    "editable": true,
    "hide_input": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "hide_code"
    ]
   },
   "outputs": [],
   "source": [
    "!test -e data || ln -s /scratch/josh_openmicroscopy/data data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f425797-0a67-4c0f-9901-f4770ef7d209",
   "metadata": {},
   "source": [
    "## TODO\n",
    "https://downloads.openmicroscopy.org/images/HCS/Operetta/59548/\n",
    "https://downloads.openmicroscopy.org/presentations/2013/fs-workshop-paris/#/7/1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2210a230-edd5-4a11-acb3-3d31d3850798",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "`find` is a very powerful tool."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2bd5e2ab-7182-45da-bb23-e24d610f4a3c",
   "metadata": {
    "editable": true,
    "scrolled": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "!find . | tail -n 10\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "e304cc36-6277-4ca8-a11c-fcc373e6c420",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "       2       2      31\n"
     ]
    }
   ],
   "source": [
    "!find . -name \"*.tiff\" | wc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "94248236-38e6-41b0-8ddb-654dfcac4cce",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "./a.ome.zarr/.zattrs: JSON data\n",
      "./a.ome.zarr/.zgroup: JSON data\n",
      "./a.ome.zarr/0/.zattrs: JSON data\n",
      "./a.ome.zarr/0/.zgroup: JSON data\n",
      "./a.ome.zarr/0/0/.zarray: JSON data\n",
      "./a.ome.zarr/0/0/0/0/0/0/0: Targa image data - Color (1-1024) 1604 x 65536 x 24 +2 \"\"\n",
      "./a.ome.zarr/0/1/.zarray: JSON data\n",
      "./a.ome.zarr/0/1/0/0/0/0/0: data\n",
      "./a.ome.zarr/OME/METADATA.ome.xml: XML 1.0 document text, ASCII text, with very long lines (678), with no line terminators\n",
      "./a.ome.zarr/OME/.zattrs: JSON data\n",
      "./a.ome.zarr/OME/.zgroup: JSON data\n",
      "./pos002.ome.tiff: TIFF image data, little-endian, direntries=17, height=1006, bps=8, compression=none, PhotometricIntepretation=RGB Palette, width=1000\n",
      "./a.ome.tiff: TIFF image data, little-endian, direntries=16, height=512, bps=8, compression=none, PhotometricIntepretation=BlackIsZero, description=<?xml version=\"1.0\" encoding=\"UTF-8\"?><!-- Warning: this comment is an OME-XML metadata block, which contains crucial dimension, width=512\n"
     ]
    }
   ],
   "source": [
    "!find . -type f -exec file {} \\;"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eb78591a-35d8-48ff-9132-921cb07c197e",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "## Excercise: use the `ln` tool to symlink data into your own folder."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "f709974e-a12d-46e5-86bb-324e8ba00642",
   "metadata": {},
   "outputs": [],
   "source": [
    "## Do something here or on the command-line"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1d13e667-2cc6-41f0-92db-c04329649fc2",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "## Your work here"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9a4f211-93bb-4425-ac3b-b5ce0d5c3b8e",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "## 3d. Extras (time permitting)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0cf807f9-3a11-4889-b0b1-344d599c183c",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "### Permissions"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ee603707-6f54-45b4-9708-b4e15c798dff",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "In the directory listings above, e.g.:\n",
    "\n",
    "```\n",
    "drwxr-xr-x   936 root     wheel  29952 Jul 11 10:56 /usr/bin/\n",
    "drwxr-xr-x@   39 root     wheel   1248 Jul 11 10:56 /bin\n",
    "drwxr-xr-x  2086 jamoore  admin  66752 Aug 20 18:35 /usr/local/bin\n",
    "```\n",
    "\n",
    "the info here is **critical**:\n",
    "\n",
    "```\n",
    "PERMISSIONS ---- USER     GROUP   SIZE MODIFIED     NAME\n",
    "```\n",
    "\n",
    "Figuring out your user and group is pretty easy. `whoami` from above is your user. `id` can tell you your groups:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "be3d8817-16c7-446f-876a-8b92c015777b",
   "metadata": {
    "editable": true,
    "scrolled": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "uid=501(jamoore) gid=20(staff) groups=20(staff),12(everyone),61(localaccounts),79(_appserverusr),80(admin),81(_appserveradm),98(_lpadmin),399(com.apple.access_ssh),33(_appstore),100(_lpoperator),204(_developer),250(_analyticsusers),395(com.apple.access_ftp),398(com.apple.access_screensharing),400(com.apple.access_remote_ae)\n"
     ]
    }
   ],
   "source": [
    "!id"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c298211-2c3f-47a1-a2c0-1f71591b35e3",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "Together we can start to piece together whether or not you can read or edit a file. From https://www.grymoire.com/Unix/Permissions.html :\n",
    "\n",
    "```\n",
    "Three sections after the \"d\" for directory marker:\n",
    "+------------+------+-------+\n",
    "| Permission | Octal| Field |\n",
    "+------------+------+-------+\n",
    "| rwx------  | 700  | User  |\n",
    "| ---rwx---  | 070  | Group |\n",
    "| ------rwx  | 007  | Other |\n",
    "+------------+------+-------+\n",
    "\n",
    "Each section can be in one of 8 states:\n",
    "+-----+---+--------------------------+\n",
    "| rwx | 7 | Read, write and execute  |\n",
    "| rw- | 6 | Read, write              |\n",
    "| r-x | 5 | Read, and execute        |\n",
    "| r-- | 4 | Read,                    |\n",
    "| -wx | 3 | Write and execute        |\n",
    "| -w- | 2 | Write                    |\n",
    "| --x | 1 | Execute                  |\n",
    "| --- | 0 | no permissions           |\n",
    "+------------------------------------+\n",
    "\n",
    "Examples:\n",
    "+------------------------+-----------+\n",
    "| chmod u=rwx,g=rwx,o=rx | chmod 775 | For world executables files\n",
    "| chmod u=rwx,g=rx,o=    | chmod 750 | For executables by group only\n",
    "| chmod u=rw,g=r,o=r     | chmod 644 | For world readable files\n",
    "| chmod u=rw,g=r,o=      | chmod 640 | For group readable files\n",
    "| chmod u=rw,go=         | chmod 600 | For private readable files\n",
    "| chmod u=rwx,go=        | chmod 700 | For private executables\n",
    "+------------------------+-----------+\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f91bb175-e40f-4803-8023-4203ee56009a",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "### Simple scripting\n",
    "\n",
    "For loops can be fairly useful when working with many files:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "c4ecfd46-25ed-470b-b099-f138673853a0",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "for x in $(find . -name \"*.fake\");\n",
    "do\n",
    "    sha1sum ${x}\n",
    "done"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "75c76b95-c65c-4c27-9f2f-33e769819128",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "But this is obviously just a starting point. Perhaps check out a [Carpentry lesson](https://software-carpentry.org/lessons/) like [The Unix Shell](https://swcarpentry.github.io/shell-novice/) for more information. I can highly recommend them."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b9b40fde-eb96-4151-8f0f-a50d00c773b9",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "### shasum"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "0a30aed4",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1a45bdb742869f84fd71ae0fd67f79f1b26923fe  ./a.ome.zarr/.zattrs\n",
      "63de336a45370c236af207996ffd1bca2d7ae2f4  ./a.ome.zarr/.zgroup\n",
      "74075dab6c0712c1d9ae80053ec1410e7f3099cf  ./a.ome.zarr/0/.zattrs\n",
      "63de336a45370c236af207996ffd1bca2d7ae2f4  ./a.ome.zarr/0/.zgroup\n",
      "965f764929f2da5e2691cb8049035975c2ccb5a0  ./a.ome.zarr/0/0/.zarray\n",
      "7c2485fb91905ebd879e745d2e80f097584de271  ./a.ome.zarr/0/0/0/0/0/0/0\n",
      "e6f199c7ed96964069fce5faa5e1cca8513a2320  ./a.ome.zarr/0/1/.zarray\n",
      "dbc22354672642df9dcc2dcb5fffcced6a18b83a  ./a.ome.zarr/0/1/0/0/0/0/0\n",
      "eda28681265146f6d153f53baaee09c030453123  ./a.ome.zarr/OME/METADATA.ome.xml\n",
      "156e48269827cb4611d5a3899d862c60c8f483f4  ./a.ome.zarr/OME/.zattrs\n",
      "63de336a45370c236af207996ffd1bca2d7ae2f4  ./a.ome.zarr/OME/.zgroup\n",
      "722c2d9b3e9a7596389f03ffd5994be8be9c438f  ./pos002.ome.tiff\n",
      "6b7350beeed3e54806cf46ff110e1a8c77c0fe80  ./a.ome.tiff\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "for x in $(find . -type f);\n",
    "do\n",
    "    sha1sum ${x}\n",
    "done"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9fc149ef-c19c-4da8-b389-5c00a28e28e3",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "### h5tools"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bbaa5b56-f724-4912-a233-5d721909af4b",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "!h5ls -h"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6a3dbd74-9a86-4401-b22d-9bde9e809b9b",
   "metadata": {},
   "source": [
    "<hr/>\n",
    "\n",
    "## Half-time\n",
    "\n",
    "<hr/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "030e9b93-2683-4848-8ee1-e8da0e8360b9",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "\n",
    "##\n",
    "## Setup & Sanity checks\n",
    "##\n",
    "\n",
    "YOURNAME=$(whoami)\n",
    "WORKDIR=/scratch/${YOURNAME}/session1\n",
    "test -e ${WORKDIR} || {\n",
    "    echo Please run the steps above first.\n",
    "    exit 1\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "8662a965-5294-46fb-899c-0e5b3b933d9f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/System/Volumes/Data/scratch/jamoore/session1\n"
     ]
    }
   ],
   "source": [
    "%cd /scratch/{YOURNAME}/session1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "72cc3bba-2bbe-4874-848d-73679a6102cb",
   "metadata": {},
   "source": [
    "## 4a. bftools: showinf, bfconvert, etc.\n",
    "\n",
    "bftools is a package of tools that can be downloaded from https://downloads.openmicroscopy.org/bio-formats/latest/artifacts/ or [conda](https://anaconda.org/ome/bftools) for working with bioimaging files from the command-line.\n",
    "\n",
    "One of the tools, `formatlist` simply lists all the supported file formats with their associated file endings:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "f3a0f17d-c1f6-4e0d-96b7-159112cf2023",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "File pattern: can read (pattern)\n",
      "Zip: can read (zip)\n",
      "Animated PNG: can read, can write, can write multiple (png)\n",
      "JPEG: can read, can write (jpg, jpeg, jpe)\n",
      "SlideBook 7 SLD (native): can read (sldy)\n",
      "Portable Any Map: can read (pbm, pgm, ppm)\n",
      "Flexible Image Transport System: can read (fits, fts)\n",
      "PCX: can read (pcx)\n",
      "Graphics Interchange Format: can read (gif)\n",
      "Windows Bitmap: can read (bmp)\n",
      "IPLab: can read (ipl)\n",
      "IVision: can read (ipm)\n",
      "RCPNL: can read (rcpnl)\n",
      "Deltavision: can read (dv, r3d, r3d_d3d, dv.log, r3d.log)\n",
      "Medical Research Council: can read (mrc, st, ali, map, rec, mrcs)\n",
      "Gatan Digital Micrograph: can read (dm3, dm4)\n",
      "Gatan DM2: can read (dm2)\n",
      "Bitplane Imaris: can read (ims)\n",
      "Openlab RAW: can read (raw)\n",
      "OME-XML: can read, can write, can write multiple (ome, ome.xml)\n",
      "Leica Image File Format: can read (lif)\n",
      "Audio Video Interleave: can read, can write, can write multiple (avi)\n",
      "PICT: can read (pict, pct)\n",
      "SPCImage Data: can read (sdt)\n",
      "SPC FIFO Data: can read (spc, set)\n",
      "Encapsulated PostScript: can read, can write (eps, epsi, ps)\n",
      "Olympus Slidebook: can read (sld, spl)\n",
      "Alicona AL3D: can read (al3d)\n",
      "Multiple-image Network Graphics: can read (mng)\n",
      "Khoros XV: can read (xv)\n",
      "Visitech XYS: can read (xys, html)\n",
      "Laboratory Imaging: can read (lim)\n",
      "Adobe Photoshop: can read (psd)\n",
      "InCell 1000/2000: can read (xdce, xml, tiff, tif, xlog)\n",
      "Li-Cor L2D: can read (l2d, scn, tif)\n",
      "FEI/Philips: can read (img)\n",
      "Hamamatsu Aquacosmos: can read (naf)\n",
      "MINC MRI: can read (mnc)\n",
      "QuickTime: can read, can write, can write multiple (mov)\n",
      "Minolta MRW: can read (mrw)\n",
      "TillVision: can read (vws, pst, inf)\n",
      "ARF: can read (arf)\n",
      "Cellomics C01: can read (c01, dib)\n",
      "LI-FLIM: can read (fli)\n",
      "Truevision Targa: can read (tga)\n",
      "Oxford Instruments: can read (top)\n",
      "VG SAM: can read (dti)\n",
      "Hamamatsu HIS: can read (his)\n",
      "WA Technology TOP: can read (wat)\n",
      "Seiko: can read (xqd, xqf)\n",
      "TopoMetrix: can read (tfr, ffr, zfr, zfp, 2fl)\n",
      "UBM: can read (pr3)\n",
      "Quesant AFM: can read (afm)\n",
      "Bio-Rad GEL: can read (1sc)\n",
      "RHK Technologies: can read (sm2, sm3)\n",
      "Molecular Imaging: can read (stp)\n",
      "CellWorx: can read (pnl, htd, log)\n",
      "MetaXpress TIFF: can read (htd, tif)\n",
      "ECAT7: can read (v)\n",
      "Varian FDF: can read (fdf)\n",
      "AIM: can read (aim)\n",
      "InCell 3000: can read (frm)\n",
      "SPIDER: can read (spi)\n",
      "Volocity Library: can read (mvd2, aisf, aiix, dat, atsf)\n",
      "IMAGIC: can read (hed, img)\n",
      "Hamamatsu VMS: can read (vms)\n",
      "CellSens VSI: can read (vsi, ets)\n",
      "INR: can read (inr)\n",
      "Kodak Molecular Imaging: can read (bip)\n",
      "Volocity Library Clipping: can read (acff)\n",
      "Zeiss CZI: can read (czi)\n",
      "Andor SIF: can read (sif)\n",
      "Hamamatsu NDPIS: can read (ndpis)\n",
      "POV-Ray: can read (df3)\n",
      "IMOD: can read (mod)\n",
      "Simulated data: can read (fake)\n",
      "Aperio AFI: can read (afi)\n",
      "Lavision Imspector: can read (msr)\n",
      "Bio-Rad SCN: can read (scn)\n",
      "Zeiss LMS: can read (lms)\n",
      "PicoQuant Bin: can read (bin)\n",
      "FlowSight: can read (cif)\n",
      "Perkin-Elmer Nuance IM3: can read (im3)\n",
      "I2I: can read (i2i)\n",
      "Princeton Instruments SPE: can read (spe)\n",
      "Olympus OIR: can read (oir)\n",
      "KLB: can read (klb)\n",
      "MicroCT: can read (vff)\n",
      "Leica Object Format: can read (lof)\n",
      "Extended leica file: can read (xlef)\n",
      "Olympus .omp2info: can read (omp2info)\n",
      "JEOL: can read (dat, img, par)\n",
      "NIfTI: can read (nii, img, hdr, nii.gz)\n",
      "Analyze 7.5: can read (img, hdr)\n",
      "Olympus APL: can read (apl, tnb, mtb, tif)\n",
      "NRRD: can read (nrrd, nhdr)\n",
      "Image Cytometry Standard: can read, can write, can write multiple (ics, ids)\n",
      "PerkinElmer: can read (ano, cfg, csv, htm, rec, tim, zpo, tif)\n",
      "Amira: can read (am, amiramesh, grey, hx, labels)\n",
      "Olympus ScanR: can read (dat, xml, tif)\n",
      "BD Pathway: can read (exp, tif)\n",
      "Unisoku STM: can read (hdr, dat)\n",
      "Perkin Elmer Densitometer: can read (hdr, img)\n",
      "Fuji LAS 3000: can read (img, inf)\n",
      "PerkinElmer Operetta: can read (tif, tiff, xml)\n",
      "Inveon: can read (hdr)\n",
      "CellVoyager: can read (tif, xml)\n",
      "PerkinElmer Columbus: can read (xml)\n",
      "Yokogawa CV7000: can read (wpi)\n",
      "Bio-Rad PIC: can read (pic, xml, raw)\n",
      "Olympus FV1000: can read (oib, oif, pty, lut)\n",
      "Zeiss Vision Image (ZVI): can read (zvi)\n",
      "Image-Pro Workspace: can read (ipw)\n",
      "JPEG-2000: can read, can write (jp2, j2k, jpf)\n",
      "JPX: can read (jpx)\n",
      "Nikon ND2: can read (nd2, jp2)\n",
      "Compix Simple-PCI: can read (cxd)\n",
      "Bitplane Imaris 5.5 (HDF): can read (ims)\n",
      "CellH5 (HDF): can read (ch5)\n",
      "Veeco: can read (hdf)\n",
      "Tecan Spark Cyto: can read (db)\n",
      "Zeiss Laser-Scanning Microscopy: can read (lsm, mdb)\n",
      "Image-Pro Sequence: can read (seq, ips)\n",
      "Amersham Biosciences GEL: can read (gel)\n",
      "Bitplane Imaris 3 (TIFF): can read (ims)\n",
      "Evotec Flex: can read (flex, mea, res)\n",
      "Aperio SVS: can read (svs)\n",
      "Imacon: can read (fff)\n",
      "LEO: can read (sxm, tif, tiff)\n",
      "JPK Instruments: can read (jpk)\n",
      "Hamamatsu NDPI: can read (ndpi)\n",
      "PCO-RAW: can read (pcoraw, rec)\n",
      "Ventana .bif: can read (bif)\n",
      "OME-TIFF: can read, can write, can write multiple (ome.tiff, ome.tif, ome.tf2, ome.tf8, ome.btf, companion.ome)\n",
      "Pyramid TIFF: can read (tif, tiff)\n",
      "MIAS: can read (tif, tiff, txt)\n",
      "Leica TCS TIFF: can read (tif, tiff, xml)\n",
      "Leica: can read (lei, tif, tiff, raw)\n",
      "Nikon NEF: can read (nef, tif, tiff)\n",
      "Olympus Fluoview/ABD TIFF: can read (tif, tiff)\n",
      "Prairie TIFF: can read (tif, tiff, cfg, env, xml)\n",
      "Metamorph STK: can read (stk, nd, scan, tif, tiff)\n",
      "Micro-Manager: can read (tif, tiff, txt, xml)\n",
      "Improvision TIFF: can read (tif, tiff)\n",
      "Metamorph TIFF: can read (tif, tiff)\n",
      "Nikon TIFF: can read (tif, tiff)\n",
      "Mikroscan TIFF: can read (tif, tiff)\n",
      "Adobe Photoshop TIFF: can read (tif, tiff)\n",
      "FEI TIFF: can read (tif, tiff)\n",
      "SimplePCI TIFF: can read (tif, tiff)\n",
      "Nikon Elements TIFF: can read (tif, tiff)\n",
      "Trestle: can read (tif)\n",
      "Olympus SIS TIFF: can read (tif, tiff)\n",
      "DNG: can read (cr2, crw, jpg, thm, wav, tif, tiff)\n",
      "Zeiss AxioVision TIFF: can read (tif, xml)\n",
      "Leica SCN: can read (scn)\n",
      "PerkinElmer Vectra/QPTIFF: can read (tiff, tif, qptiff)\n",
      "Slidebook TIFF: can read (tif, tiff)\n",
      "Ionpath MIBI: can read (tif, tiff)\n",
      "DICOM: can read, can write, can write multiple (dic, dcm, dicom, jp2, j2ki, j2kr, raw, ima)\n",
      "Hitachi: can read (txt)\n",
      "Tagged Image File Format: can read, can write, can write multiple (tif, tiff, tf2, tf8, btf)\n",
      "Text: can read (txt, csv)\n",
      "Burleigh: can read (img)\n",
      "Openlab LIFF: can read (liff)\n",
      "SM Camera: can read ()\n",
      "SBIG: can read ()\n",
      "NOAA-HRD Gridded Data Format: can read ()\n",
      "Bruker: can read ()\n",
      "Canon RAW: can read (cr2, crw, jpg, thm, wav)\n",
      "OBF: can read (obf, msr)\n",
      "BDV: can read (xml, h5)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!formatlist"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "edf83384-f79e-4244-9999-7eec2f182eba",
   "metadata": {},
   "source": [
    "We'll look at a few of these types."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "8d0f09e9-22f0-490d-a8e5-2c7e72af4396",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Initializing reader\n",
      "FakeReader initializing data/a.fake\n",
      "Initialization took 0.082s\n",
      "\n",
      "Reading core metadata\n",
      "filename = data/a.fake\n",
      "Used files = [/System/Volumes/Data/scratch/jamoore/session1/data/a.fake]\n",
      "Series count = 1\n",
      "Series #0 :\n",
      "\tImage count = 1\n",
      "\tRGB = false (1) \n",
      "\tInterleaved = false\n",
      "\tIndexed = false (true color)\n",
      "\tWidth = 512\n",
      "\tHeight = 512\n",
      "\tSizeZ = 1\n",
      "\tSizeT = 1\n",
      "\tSizeC = 1\n",
      "\tTile size = 512 x 512\n",
      "\tThumbnail size = 128 x 128\n",
      "\tEndianness = intel (little)\n",
      "\tDimension order = XYZCT (certain)\n",
      "\tPixel type = uint8\n",
      "\tValid bits per pixel = 8\n",
      "\tMetadata complete = true\n",
      "\tThumbnail series = false\n",
      "\t-----\n",
      "\tPlane #0 <=> Z 0, C 0, T 0\n",
      "\n",
      "\n",
      "Reading global metadata\n",
      "\n",
      "Reading metadata\n"
     ]
    }
   ],
   "source": [
    "!showinf -nopix data/a.fake"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "fd520130-b8d6-4612-8535-26f91caddf4c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n",
      "<OME xmlns=\"http://www.openmicroscopy.org/Schemas/OME/2016-06\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xsi:schemaLocation=\"http://www.openmicroscopy.org/Schemas/OME/2016-06 http://www.openmicroscopy.org/Schemas/OME/2016-06/ome.xsd\">\n",
      "   <Image ID=\"Image:0\" Name=\"data/a\">\n",
      "      <Pixels BigEndian=\"false\" DimensionOrder=\"XYZCT\" ID=\"Pixels:0\" Interleaved=\"false\" SignificantBits=\"8\" SizeC=\"1\" SizeT=\"1\" SizeX=\"512\" SizeY=\"512\" SizeZ=\"1\" Type=\"uint8\">\n",
      "         <Channel ID=\"Channel:0:0\" SamplesPerPixel=\"1\">\n",
      "            <LightPath/>\n",
      "         </Channel>\n",
      "         <MetadataOnly/>\n",
      "      </Pixels>\n",
      "   </Image>\n",
      "</OME>\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!showinf -nopix -omexml-only data/a.fake"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "a8b579ae-54cb-46af-bfb5-f505c303432d",
   "metadata": {},
   "outputs": [],
   "source": [
    "! test -e a.ome.tiff && rm -rf a.ome.tiff"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "d698eaae-a24d-4bbb-85cb-df8b9895bce9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "data/a.fake\n",
      "FakeReader initializing data/a.fake\n",
      "[Simulated data] -> a.ome.tiff [OME-TIFF]\n",
      "\tConverted 1/1 planes (100%)\n",
      "[done]\n",
      "0.993s elapsed (17.0+149.0ms per plane, 786ms overhead)\n"
     ]
    }
   ],
   "source": [
    "! bfconvert data/a.fake a.ome.tiff"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "79671812-717c-43ba-b2ff-2d827a04288b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Initializing reader\n",
      "Exception in thread \"main\" java.io.FileNotFoundException: cellprofiler/fruit-fly-cells/POS218.pattern/POS218.pattern (No such file or directory)\n",
      "\tat java.base/java.io.RandomAccessFile.open0(Native Method)\n",
      "\tat java.base/java.io.RandomAccessFile.open(RandomAccessFile.java:345)\n",
      "\tat java.base/java.io.RandomAccessFile.<init>(RandomAccessFile.java:259)\n",
      "\tat java.base/java.io.RandomAccessFile.<init>(RandomAccessFile.java:214)\n",
      "\tat loci.common.NIOFileHandle.<init>(NIOFileHandle.java:130)\n",
      "\tat loci.common.NIOFileHandle.<init>(NIOFileHandle.java:151)\n",
      "\tat loci.common.NIOFileHandle.<init>(NIOFileHandle.java:165)\n",
      "\tat loci.common.Location.getHandle(Location.java:522)\n",
      "\tat loci.common.Location.getHandle(Location.java:462)\n",
      "\tat loci.common.Location.getHandle(Location.java:443)\n",
      "\tat loci.common.Location.getHandle(Location.java:426)\n",
      "\tat loci.common.Location.checkValidId(Location.java:551)\n",
      "\tat loci.formats.ImageReader.getReader(ImageReader.java:182)\n",
      "\tat loci.formats.ImageReader.setId(ImageReader.java:858)\n",
      "\tat loci.formats.ReaderWrapper.setId(ReaderWrapper.java:660)\n",
      "\tat loci.formats.tools.ImageInfo.testRead(ImageInfo.java:1043)\n",
      "\tat loci.formats.tools.ImageInfo.main(ImageInfo.java:1129)\n"
     ]
    }
   ],
   "source": [
    "! showinf cellprofiler/fruit-fly-cells/POS218.pattern/POS218.pattern"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "79ded011-475a-4320-993d-2227b620681c",
   "metadata": {},
   "outputs": [],
   "source": [
    "from tifffile import imread, tiffcomment\n",
    "\n",
    "data = imread(\"a.ome.tiff\")\n",
    "metadata = tiffcomment(\"a.ome.tiff\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "8a8d96dc-e6d3-4e9d-944c-b66160811305",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "id='Image:0' name='data/a' pixels={'channels': [{'annotation_refs': [], 'light_path': {'excitation_filters': [], 'emission_filters': [], 'annotation_refs': []}, 'id': 'Channel:0:0', 'samples_per_pixel': 1, 'color': Color('white', rgb=(255, 255, 255))}], 'bin_data_blocks': [], 'tiff_data_blocks': [{'uuid': {'value': 'urn:uuid:0cc95cd0-950f-40bb-bc97-77a3fe7f7b64', 'file_name': 'a.ome.tiff'}, 'plane_count': 1}], 'planes': [], 'id': 'Pixels:0', 'dimension_order': <Pixels_DimensionOrder.XYZCT: 'XYZCT'>, 'type': <PixelType.UINT8: 'uint8'>, 'significant_bits': 8, 'interleaved': False, 'big_endian': False, 'size_x': 512, 'size_y': 512, 'size_z': 1, 'size_c': 1, 'size_t': 1}\n"
     ]
    }
   ],
   "source": [
    "from ome_types import from_xml, from_tiff\n",
    "\n",
    "obj = from_xml(metadata)\n",
    "print(obj.images[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "289f0d8b-a10a-4bf4-8f8f-909117954b5f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "id='Image:0' name='data/a' pixels={'channels': [{'annotation_refs': [], 'light_path': {'excitation_filters': [], 'emission_filters': [], 'annotation_refs': []}, 'id': 'Channel:0:0', 'samples_per_pixel': 1, 'color': Color('white', rgb=(255, 255, 255))}], 'bin_data_blocks': [], 'tiff_data_blocks': [{'uuid': {'value': 'urn:uuid:0cc95cd0-950f-40bb-bc97-77a3fe7f7b64', 'file_name': 'a.ome.tiff'}, 'plane_count': 1}], 'planes': [], 'id': 'Pixels:0', 'dimension_order': <Pixels_DimensionOrder.XYZCT: 'XYZCT'>, 'type': <PixelType.UINT8: 'uint8'>, 'significant_bits': 8, 'interleaved': False, 'big_endian': False, 'size_x': 512, 'size_y': 512, 'size_z': 1, 'size_c': 1, 'size_t': 1}\n"
     ]
    }
   ],
   "source": [
    "obj = from_tiff(\"a.ome.tiff\")\n",
    "print(obj.images[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "id": "b9ec37a2-5bd0-42a2-89a7-07bc03a614db",
   "metadata": {},
   "outputs": [],
   "source": [
    "from aicsimageio import AICSImage\n",
    "\n",
    "img = AICSImage(\"a.ome.tiff\") "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "id": "3fa37cf6-60c5-4258-a370-acea417edad1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "    <tr>\n",
       "        <td>\n",
       "            <table style=\"border-collapse: collapse;\">\n",
       "                <thead>\n",
       "                    <tr>\n",
       "                        <td> </td>\n",
       "                        <th> Array </th>\n",
       "                        <th> Chunk </th>\n",
       "                    </tr>\n",
       "                </thead>\n",
       "                <tbody>\n",
       "                    \n",
       "                    <tr>\n",
       "                        <th> Bytes </th>\n",
       "                        <td> 256.00 kiB </td>\n",
       "                        <td> 256.00 kiB </td>\n",
       "                    </tr>\n",
       "                    \n",
       "                    <tr>\n",
       "                        <th> Shape </th>\n",
       "                        <td> (1, 1, 1, 512, 512) </td>\n",
       "                        <td> (1, 1, 1, 512, 512) </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <th> Dask graph </th>\n",
       "                        <td colspan=\"2\"> 1 chunks in 6 graph layers </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <th> Data type </th>\n",
       "                        <td colspan=\"2\"> uint8 numpy.ndarray </td>\n",
       "                    </tr>\n",
       "                </tbody>\n",
       "            </table>\n",
       "        </td>\n",
       "        <td>\n",
       "        <svg width=\"374\" height=\"184\" style=\"stroke:rgb(0,0,0);stroke-width:1\" >\n",
       "\n",
       "  <!-- Horizontal lines -->\n",
       "  <line x1=\"0\" y1=\"0\" x2=\"25\" y2=\"0\" style=\"stroke-width:2\" />\n",
       "  <line x1=\"0\" y1=\"25\" x2=\"25\" y2=\"25\" style=\"stroke-width:2\" />\n",
       "\n",
       "  <!-- Vertical lines -->\n",
       "  <line x1=\"0\" y1=\"0\" x2=\"0\" y2=\"25\" style=\"stroke-width:2\" />\n",
       "  <line x1=\"25\" y1=\"0\" x2=\"25\" y2=\"25\" style=\"stroke-width:2\" />\n",
       "\n",
       "  <!-- Colored Rectangle -->\n",
       "  <polygon points=\"0.0,0.0 25.412616514582485,0.0 25.412616514582485,25.412616514582485 0.0,25.412616514582485\" style=\"fill:#ECB172A0;stroke-width:0\"/>\n",
       "\n",
       "  <!-- Text -->\n",
       "  <text x=\"12.706308\" y=\"45.412617\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" >1</text>\n",
       "  <text x=\"45.412617\" y=\"12.706308\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(0,45.412617,12.706308)\">1</text>\n",
       "\n",
       "\n",
       "  <!-- Horizontal lines -->\n",
       "  <line x1=\"95\" y1=\"0\" x2=\"109\" y2=\"14\" style=\"stroke-width:2\" />\n",
       "  <line x1=\"95\" y1=\"120\" x2=\"109\" y2=\"134\" style=\"stroke-width:2\" />\n",
       "\n",
       "  <!-- Vertical lines -->\n",
       "  <line x1=\"95\" y1=\"0\" x2=\"95\" y2=\"120\" style=\"stroke-width:2\" />\n",
       "  <line x1=\"109\" y1=\"14\" x2=\"109\" y2=\"134\" style=\"stroke-width:2\" />\n",
       "\n",
       "  <!-- Colored Rectangle -->\n",
       "  <polygon points=\"95.0,0.0 109.9485979497544,14.948597949754403 109.9485979497544,134.9485979497544 95.0,120.0\" style=\"fill:#ECB172A0;stroke-width:0\"/>\n",
       "\n",
       "  <!-- Horizontal lines -->\n",
       "  <line x1=\"95\" y1=\"0\" x2=\"215\" y2=\"0\" style=\"stroke-width:2\" />\n",
       "  <line x1=\"109\" y1=\"14\" x2=\"229\" y2=\"14\" style=\"stroke-width:2\" />\n",
       "\n",
       "  <!-- Vertical lines -->\n",
       "  <line x1=\"95\" y1=\"0\" x2=\"109\" y2=\"14\" style=\"stroke-width:2\" />\n",
       "  <line x1=\"215\" y1=\"0\" x2=\"229\" y2=\"14\" style=\"stroke-width:2\" />\n",
       "\n",
       "  <!-- Colored Rectangle -->\n",
       "  <polygon points=\"95.0,0.0 215.0,0.0 229.9485979497544,14.948597949754403 109.9485979497544,14.948597949754403\" style=\"fill:#ECB172A0;stroke-width:0\"/>\n",
       "\n",
       "  <!-- Horizontal lines -->\n",
       "  <line x1=\"109\" y1=\"14\" x2=\"229\" y2=\"14\" style=\"stroke-width:2\" />\n",
       "  <line x1=\"109\" y1=\"134\" x2=\"229\" y2=\"134\" style=\"stroke-width:2\" />\n",
       "\n",
       "  <!-- Vertical lines -->\n",
       "  <line x1=\"109\" y1=\"14\" x2=\"109\" y2=\"134\" style=\"stroke-width:2\" />\n",
       "  <line x1=\"229\" y1=\"14\" x2=\"229\" y2=\"134\" style=\"stroke-width:2\" />\n",
       "\n",
       "  <!-- Colored Rectangle -->\n",
       "  <polygon points=\"109.9485979497544,14.948597949754403 229.9485979497544,14.948597949754403 229.9485979497544,134.9485979497544 109.9485979497544,134.9485979497544\" style=\"fill:#ECB172A0;stroke-width:0\"/>\n",
       "\n",
       "  <!-- Text -->\n",
       "  <text x=\"169.948598\" y=\"154.948598\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" >512</text>\n",
       "  <text x=\"249.948598\" y=\"74.948598\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(-90,249.948598,74.948598)\">512</text>\n",
       "  <text x=\"92.474299\" y=\"147.474299\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(45,92.474299,147.474299)\">1</text>\n",
       "</svg>\n",
       "        </td>\n",
       "    </tr>\n",
       "</table>"
      ],
      "text/plain": [
       "dask.array<transpose, shape=(1, 1, 1, 512, 512), dtype=uint8, chunksize=(1, 1, 1, 512, 512), chunktype=numpy.ndarray>"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "img.dask_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "id": "81cce918-9345-4a06-9070-8e361fca1677",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Initializing reader\n",
      "FakeReader initializing a&series=2.fake\n",
      "Initialization took 0.071s\n",
      "\n",
      "Reading core metadata\n",
      "filename = a&series=2.fake\n",
      "Used files = [/System/Volumes/Data/scratch/jamoore/session1/a&series=2.fake]\n",
      "Series count = 2\n",
      "Series #0 :\n",
      "\tImage count = 1\n",
      "\tRGB = false (1) \n",
      "\tInterleaved = false\n",
      "\tIndexed = false (true color)\n",
      "\tWidth = 512\n",
      "\tHeight = 512\n",
      "\tSizeZ = 1\n",
      "\tSizeT = 1\n",
      "\tSizeC = 1\n",
      "\tTile size = 512 x 512\n",
      "\tThumbnail size = 128 x 128\n",
      "\tEndianness = intel (little)\n",
      "\tDimension order = XYZCT (certain)\n",
      "\tPixel type = uint8\n",
      "\tValid bits per pixel = 8\n",
      "\tMetadata complete = true\n",
      "\tThumbnail series = false\n",
      "\t-----\n",
      "\tPlane #0 <=> Z 0, C 0, T 0\n",
      "\n",
      "Series #1 :\n",
      "\tImage count = 1\n",
      "\tRGB = false (1) \n",
      "\tInterleaved = false\n",
      "\tIndexed = false (true color)\n",
      "\tWidth = 512\n",
      "\tHeight = 512\n",
      "\tSizeZ = 1\n",
      "\tSizeT = 1\n",
      "\tSizeC = 1\n",
      "\tTile size = 512 x 512\n",
      "\tThumbnail size = 128 x 128\n",
      "\tEndianness = intel (little)\n",
      "\tDimension order = XYZCT (certain)\n",
      "\tPixel type = uint8\n",
      "\tValid bits per pixel = 8\n",
      "\tMetadata complete = true\n",
      "\tThumbnail series = false\n",
      "\t-----\n",
      "\tPlane #0 <=> Z 0, C 0, T 0\n",
      "\n",
      "\n",
      "Reading global metadata\n",
      "\n",
      "Reading series #0 metadata\n"
     ]
    }
   ],
   "source": [
    "! showinf -nopix 'a&series=2.fake'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c1a3d89a-e7f3-4180-b506-69865c12db9b",
   "metadata": {},
   "source": [
    "## License\n",
    "Copyright (C) 2023 German BioImaging. All Rights Reserved.\n",
    "This program is free software; you can redistribute it and/or modify it\n",
    "under the terms of the GNU General Public License as published by the\n",
    "Free Software Foundation; either version 2 of the License, or\n",
    "(at your option) any later version.\n",
    "This program is distributed in the hope that it will be useful, but\n",
    "WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY\n",
    "or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for\n",
    "more details. You should have received a copy of the GNU General\n",
    "Public License along with this program; if not, write to the\n",
    "Free Software Foundation,\n",
    "Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.15"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}