examples/3-observation-for-condition.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "Let us consider a third problem: We want to find a set of patients having a specific condition,\n",
    "and then find some examination belonging to these patients. To be more practical, let us consider\n",
    " all patients that have a history of seizures, and that we are interested in finding out their\n",
    " blood pressure.\n",
    "\n",
    "The process:\n",
    "1. Get all Conditions that have the SNOMED code for seizures and store the Patients.\n",
    "2. Drop the Patient duplicates.\n",
    "3. Get all Observations for these Patients that have LOINC code for blood pressure."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "First, we initialize FHIR-PYrate."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "outputs": [],
   "source": [
    "from fhir_pyrate import Pirate\n",
    "from fhir_pyrate.util import FHIRObj\n",
    "from typing import List, Dict\n",
    "\n",
    "search = Pirate(\n",
    "    auth=None,\n",
    "    base_url=\"http://hapi.fhir.org/baseDstu2\",\n",
    "    print_request_url=False,\n",
    "    num_processes=1,\n",
    ")"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "To find out all patients with seizures, we need to have a look at Condition.\n",
    "\n",
    "Remember to install [fhirpath-py](https://github.com/beda-software/fhirpath-py) to use the\n",
    "`fhir_paths` parameter."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Query & Build DF (Condition): 100%|██████████| 1/1 [00:00<00:00, 392.17it/s]\n"
     ]
    },
    {
     "data": {
      "text/plain": "        id      patient_id verificationStatus\n0     1839    Patient/1834          confirmed\n1    14316   Patient/14311          confirmed\n2    34354   Patient/34346          confirmed\n3    43629   Patient/43625          confirmed\n4    46711   Patient/46706          confirmed\n5    49356   Patient/49351          confirmed\n6    57176   Patient/57171          confirmed\n7    62556   Patient/62550          confirmed\n8    65046   Patient/64991          confirmed\n9    69808   Patient/69804          confirmed\n10   86323   Patient/86272          confirmed\n11   95632   Patient/95620          confirmed\n12   96660   Patient/96655          confirmed\n13   97272   Patient/97231          confirmed\n14   99287   Patient/99229          confirmed\n15  101164  Patient/101157          confirmed\n16  110749  Patient/110743          confirmed\n17  113272  Patient/113267          confirmed\n18  118736  Patient/118732          confirmed\n19  120324  Patient/120317          confirmed\n20  122070  Patient/122065          confirmed\n21  126009  Patient/126004          confirmed\n22  126939  Patient/126935          confirmed\n23  139585  Patient/139581          confirmed\n24  140767  Patient/140763          confirmed\n25  142918  Patient/142913          confirmed\n26  145356  Patient/145351          confirmed\n27  155254  Patient/155249          confirmed\n28  162626  Patient/162620          confirmed\n29  195429  Patient/195425          confirmed\n30  198328  Patient/198323          confirmed",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>id</th>\n      <th>patient_id</th>\n      <th>verificationStatus</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1839</td>\n      <td>Patient/1834</td>\n      <td>confirmed</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>14316</td>\n      <td>Patient/14311</td>\n      <td>confirmed</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>34354</td>\n      <td>Patient/34346</td>\n      <td>confirmed</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>43629</td>\n      <td>Patient/43625</td>\n      <td>confirmed</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>46711</td>\n      <td>Patient/46706</td>\n      <td>confirmed</td>\n    </tr>\n    <tr>\n      <th>5</th>\n      <td>49356</td>\n      <td>Patient/49351</td>\n      <td>confirmed</td>\n    </tr>\n    <tr>\n      <th>6</th>\n      <td>57176</td>\n      <td>Patient/57171</td>\n      <td>confirmed</td>\n    </tr>\n    <tr>\n      <th>7</th>\n      <td>62556</td>\n      <td>Patient/62550</td>\n      <td>confirmed</td>\n    </tr>\n    <tr>\n      <th>8</th>\n      <td>65046</td>\n      <td>Patient/64991</td>\n      <td>confirmed</td>\n    </tr>\n    <tr>\n      <th>9</th>\n      <td>69808</td>\n      <td>Patient/69804</td>\n      <td>confirmed</td>\n    </tr>\n    <tr>\n      <th>10</th>\n      <td>86323</td>\n      <td>Patient/86272</td>\n      <td>confirmed</td>\n    </tr>\n    <tr>\n      <th>11</th>\n      <td>95632</td>\n      <td>Patient/95620</td>\n      <td>confirmed</td>\n    </tr>\n    <tr>\n      <th>12</th>\n      <td>96660</td>\n      <td>Patient/96655</td>\n      <td>confirmed</td>\n    </tr>\n    <tr>\n      <th>13</th>\n      <td>97272</td>\n      <td>Patient/97231</td>\n      <td>confirmed</td>\n    </tr>\n    <tr>\n      <th>14</th>\n      <td>99287</td>\n      <td>Patient/99229</td>\n      <td>confirmed</td>\n    </tr>\n    <tr>\n      <th>15</th>\n      <td>101164</td>\n      <td>Patient/101157</td>\n      <td>confirmed</td>\n    </tr>\n    <tr>\n      <th>16</th>\n      <td>110749</td>\n      <td>Patient/110743</td>\n      <td>confirmed</td>\n    </tr>\n    <tr>\n      <th>17</th>\n      <td>113272</td>\n      <td>Patient/113267</td>\n      <td>confirmed</td>\n    </tr>\n    <tr>\n      <th>18</th>\n      <td>118736</td>\n      <td>Patient/118732</td>\n      <td>confirmed</td>\n    </tr>\n    <tr>\n      <th>19</th>\n      <td>120324</td>\n      <td>Patient/120317</td>\n      <td>confirmed</td>\n    </tr>\n    <tr>\n      <th>20</th>\n      <td>122070</td>\n      <td>Patient/122065</td>\n      <td>confirmed</td>\n    </tr>\n    <tr>\n      <th>21</th>\n      <td>126009</td>\n      <td>Patient/126004</td>\n      <td>confirmed</td>\n    </tr>\n    <tr>\n      <th>22</th>\n      <td>126939</td>\n      <td>Patient/126935</td>\n      <td>confirmed</td>\n    </tr>\n    <tr>\n      <th>23</th>\n      <td>139585</td>\n      <td>Patient/139581</td>\n      <td>confirmed</td>\n    </tr>\n    <tr>\n      <th>24</th>\n      <td>140767</td>\n      <td>Patient/140763</td>\n      <td>confirmed</td>\n    </tr>\n    <tr>\n      <th>25</th>\n      <td>142918</td>\n      <td>Patient/142913</td>\n      <td>confirmed</td>\n    </tr>\n    <tr>\n      <th>26</th>\n      <td>145356</td>\n      <td>Patient/145351</td>\n      <td>confirmed</td>\n    </tr>\n    <tr>\n      <th>27</th>\n      <td>155254</td>\n      <td>Patient/155249</td>\n      <td>confirmed</td>\n    </tr>\n    <tr>\n      <th>28</th>\n      <td>162626</td>\n      <td>Patient/162620</td>\n      <td>confirmed</td>\n    </tr>\n    <tr>\n      <th>29</th>\n      <td>195429</td>\n      <td>Patient/195425</td>\n      <td>confirmed</td>\n    </tr>\n    <tr>\n      <th>30</th>\n      <td>198328</td>\n      <td>Patient/198323</td>\n      <td>confirmed</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "condition_df = search.steal_bundles_to_dataframe(  # search.sail_through_search_space can also be used in case we have a lot of data. But we would need to specify a time frame for the query.\n",
    "    resource_type=\"Condition\",\n",
    "    request_params={\n",
    "        \"_count\": 100,\n",
    "        \"code\": \"http://snomed.info/sct%7C84757009\",  # Code for seizures\n",
    "        \"_sort\": \"_id\",\n",
    "    },\n",
    "    fhir_paths=[\"id\", (\"patient_id\", \"patient.reference\"), \"verificationStatus\"],\n",
    ")\n",
    "condition_df"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "This query could also be run in parallel with the `search.sail_through_search_space` function, which splits the desired period into smaller time frames and runs on query for each period. In this case, this is not needed."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Now we get the patients, and we make sure that there are no duplicates."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "outputs": [
    {
     "data": {
      "text/plain": "31"
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "patient_df = condition_df[\"patient_id\"].drop_duplicates(keep=\"first\").to_frame()\n",
    "len(patient_df)"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Now we have our patients, and we need to get their blood pressure Observations, and decide which\n",
    "fields are relevant for us."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Query & Build DF (Observation): 100%|██████████| 31/31 [00:07<00:00,  4.03it/s]\n"
     ]
    },
    {
     "data": {
      "text/plain": "         id          effectiveDateTime  \\\n0      1860  2008-07-10T10:38:49-04:00   \n1      1879  2009-04-17T10:51:37-04:00   \n2      1904  2010-05-04T11:15:19-04:00   \n3      1929  2011-04-08T07:41:18-04:00   \n4      1947  2012-02-17T17:43:51-05:00   \n..      ...                        ...   \n115  145436  2012-12-11T03:13:58-05:00   \n116  145444  2014-01-26T21:33:05-05:00   \n117  145454  2015-01-16T07:30:36-05:00   \n118  145468  2016-02-02T03:33:14-05:00   \n119  145479  2017-02-15T23:19:16-05:00   \n\n                                                  test      value  \\\n0    [Systolic Blood Pressure, Diastolic Blood Pres...  [111, 72]   \n1    [Systolic Blood Pressure, Diastolic Blood Pres...  [120, 81]   \n2    [Systolic Blood Pressure, Diastolic Blood Pres...  [102, 85]   \n3    [Systolic Blood Pressure, Diastolic Blood Pres...  [130, 77]   \n4    [Systolic Blood Pressure, Diastolic Blood Pres...  [127, 74]   \n..                                                 ...        ...   \n115  [Systolic Blood Pressure, Diastolic Blood Pres...  [124, 87]   \n116  [Systolic Blood Pressure, Diastolic Blood Pres...  [119, 82]   \n117  [Systolic Blood Pressure, Diastolic Blood Pres...  [107, 75]   \n118  [Systolic Blood Pressure, Diastolic Blood Pres...  [112, 81]   \n119  [Systolic Blood Pressure, Diastolic Blood Pres...  [122, 81]   \n\n             unit      patient_id  \n0    [mmHg, mmHg]    Patient/1834  \n1    [mmHg, mmHg]    Patient/1834  \n2    [mmHg, mmHg]    Patient/1834  \n3    [mmHg, mmHg]    Patient/1834  \n4    [mmHg, mmHg]    Patient/1834  \n..            ...             ...  \n115  [mmHg, mmHg]  Patient/145351  \n116  [mmHg, mmHg]  Patient/145351  \n117  [mmHg, mmHg]  Patient/145351  \n118  [mmHg, mmHg]  Patient/145351  \n119  [mmHg, mmHg]  Patient/145351  \n\n[120 rows x 6 columns]",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>id</th>\n      <th>effectiveDateTime</th>\n      <th>test</th>\n      <th>value</th>\n      <th>unit</th>\n      <th>patient_id</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1860</td>\n      <td>2008-07-10T10:38:49-04:00</td>\n      <td>[Systolic Blood Pressure, Diastolic Blood Pres...</td>\n      <td>[111, 72]</td>\n      <td>[mmHg, mmHg]</td>\n      <td>Patient/1834</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1879</td>\n      <td>2009-04-17T10:51:37-04:00</td>\n      <td>[Systolic Blood Pressure, Diastolic Blood Pres...</td>\n      <td>[120, 81]</td>\n      <td>[mmHg, mmHg]</td>\n      <td>Patient/1834</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1904</td>\n      <td>2010-05-04T11:15:19-04:00</td>\n      <td>[Systolic Blood Pressure, Diastolic Blood Pres...</td>\n      <td>[102, 85]</td>\n      <td>[mmHg, mmHg]</td>\n      <td>Patient/1834</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1929</td>\n      <td>2011-04-08T07:41:18-04:00</td>\n      <td>[Systolic Blood Pressure, Diastolic Blood Pres...</td>\n      <td>[130, 77]</td>\n      <td>[mmHg, mmHg]</td>\n      <td>Patient/1834</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1947</td>\n      <td>2012-02-17T17:43:51-05:00</td>\n      <td>[Systolic Blood Pressure, Diastolic Blood Pres...</td>\n      <td>[127, 74]</td>\n      <td>[mmHg, mmHg]</td>\n      <td>Patient/1834</td>\n    </tr>\n    <tr>\n      <th>...</th>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n    </tr>\n    <tr>\n      <th>115</th>\n      <td>145436</td>\n      <td>2012-12-11T03:13:58-05:00</td>\n      <td>[Systolic Blood Pressure, Diastolic Blood Pres...</td>\n      <td>[124, 87]</td>\n      <td>[mmHg, mmHg]</td>\n      <td>Patient/145351</td>\n    </tr>\n    <tr>\n      <th>116</th>\n      <td>145444</td>\n      <td>2014-01-26T21:33:05-05:00</td>\n      <td>[Systolic Blood Pressure, Diastolic Blood Pres...</td>\n      <td>[119, 82]</td>\n      <td>[mmHg, mmHg]</td>\n      <td>Patient/145351</td>\n    </tr>\n    <tr>\n      <th>117</th>\n      <td>145454</td>\n      <td>2015-01-16T07:30:36-05:00</td>\n      <td>[Systolic Blood Pressure, Diastolic Blood Pres...</td>\n      <td>[107, 75]</td>\n      <td>[mmHg, mmHg]</td>\n      <td>Patient/145351</td>\n    </tr>\n    <tr>\n      <th>118</th>\n      <td>145468</td>\n      <td>2016-02-02T03:33:14-05:00</td>\n      <td>[Systolic Blood Pressure, Diastolic Blood Pres...</td>\n      <td>[112, 81]</td>\n      <td>[mmHg, mmHg]</td>\n      <td>Patient/145351</td>\n    </tr>\n    <tr>\n      <th>119</th>\n      <td>145479</td>\n      <td>2017-02-15T23:19:16-05:00</td>\n      <td>[Systolic Blood Pressure, Diastolic Blood Pres...</td>\n      <td>[122, 81]</td>\n      <td>[mmHg, mmHg]</td>\n      <td>Patient/145351</td>\n    </tr>\n  </tbody>\n</table>\n<p>120 rows × 6 columns</p>\n</div>"
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "observation_df = search.trade_rows_for_dataframe(\n",
    "    df=patient_df,\n",
    "    resource_type=\"Observation\",\n",
    "    request_params={\n",
    "        \"_count\": 100,\n",
    "        \"code\": \"http://loinc.org%7C55284-4\",  # Blood pressure code\n",
    "        \"_sort\": \"_id\",\n",
    "    },\n",
    "    df_constraints={\"subject\": \"patient_id\"},\n",
    "    fhir_paths=[\n",
    "        \"id\",\n",
    "        \"effectiveDateTime\",\n",
    "        (\"test\", \"component.code.coding.display\"),\n",
    "        (\"value\", \"component.valueQuantity.value\"),\n",
    "        (\"unit\", \"component.valueQuantity.unit\"),\n",
    "    ],\n",
    "    with_ref=True,\n",
    ")\n",
    "observation_df"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "As you may see, the DataFrame now contains all the information we need, but it requires a tiny of\n",
    "post-processing to be able to be a bit nicer. We have another option though, which is to use\n",
    "processing functions instead of FHIR paths."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "We can first use `explode` to get the values in single rows."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "outputs": [
    {
     "data": {
      "text/plain": "         id          effectiveDateTime                      test value  unit  \\\n0      1860  2008-07-10T10:38:49-04:00   Systolic Blood Pressure   111  mmHg   \n0      1860  2008-07-10T10:38:49-04:00  Diastolic Blood Pressure    72  mmHg   \n1      1879  2009-04-17T10:51:37-04:00   Systolic Blood Pressure   120  mmHg   \n1      1879  2009-04-17T10:51:37-04:00  Diastolic Blood Pressure    81  mmHg   \n2      1904  2010-05-04T11:15:19-04:00   Systolic Blood Pressure   102  mmHg   \n..      ...                        ...                       ...   ...   ...   \n117  145454  2015-01-16T07:30:36-05:00  Diastolic Blood Pressure    75  mmHg   \n118  145468  2016-02-02T03:33:14-05:00   Systolic Blood Pressure   112  mmHg   \n118  145468  2016-02-02T03:33:14-05:00  Diastolic Blood Pressure    81  mmHg   \n119  145479  2017-02-15T23:19:16-05:00   Systolic Blood Pressure   122  mmHg   \n119  145479  2017-02-15T23:19:16-05:00  Diastolic Blood Pressure    81  mmHg   \n\n         patient_id  \n0      Patient/1834  \n0      Patient/1834  \n1      Patient/1834  \n1      Patient/1834  \n2      Patient/1834  \n..              ...  \n117  Patient/145351  \n118  Patient/145351  \n118  Patient/145351  \n119  Patient/145351  \n119  Patient/145351  \n\n[240 rows x 6 columns]",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>id</th>\n      <th>effectiveDateTime</th>\n      <th>test</th>\n      <th>value</th>\n      <th>unit</th>\n      <th>patient_id</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1860</td>\n      <td>2008-07-10T10:38:49-04:00</td>\n      <td>Systolic Blood Pressure</td>\n      <td>111</td>\n      <td>mmHg</td>\n      <td>Patient/1834</td>\n    </tr>\n    <tr>\n      <th>0</th>\n      <td>1860</td>\n      <td>2008-07-10T10:38:49-04:00</td>\n      <td>Diastolic Blood Pressure</td>\n      <td>72</td>\n      <td>mmHg</td>\n      <td>Patient/1834</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1879</td>\n      <td>2009-04-17T10:51:37-04:00</td>\n      <td>Systolic Blood Pressure</td>\n      <td>120</td>\n      <td>mmHg</td>\n      <td>Patient/1834</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1879</td>\n      <td>2009-04-17T10:51:37-04:00</td>\n      <td>Diastolic Blood Pressure</td>\n      <td>81</td>\n      <td>mmHg</td>\n      <td>Patient/1834</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1904</td>\n      <td>2010-05-04T11:15:19-04:00</td>\n      <td>Systolic Blood Pressure</td>\n      <td>102</td>\n      <td>mmHg</td>\n      <td>Patient/1834</td>\n    </tr>\n    <tr>\n      <th>...</th>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n    </tr>\n    <tr>\n      <th>117</th>\n      <td>145454</td>\n      <td>2015-01-16T07:30:36-05:00</td>\n      <td>Diastolic Blood Pressure</td>\n      <td>75</td>\n      <td>mmHg</td>\n      <td>Patient/145351</td>\n    </tr>\n    <tr>\n      <th>118</th>\n      <td>145468</td>\n      <td>2016-02-02T03:33:14-05:00</td>\n      <td>Systolic Blood Pressure</td>\n      <td>112</td>\n      <td>mmHg</td>\n      <td>Patient/145351</td>\n    </tr>\n    <tr>\n      <th>118</th>\n      <td>145468</td>\n      <td>2016-02-02T03:33:14-05:00</td>\n      <td>Diastolic Blood Pressure</td>\n      <td>81</td>\n      <td>mmHg</td>\n      <td>Patient/145351</td>\n    </tr>\n    <tr>\n      <th>119</th>\n      <td>145479</td>\n      <td>2017-02-15T23:19:16-05:00</td>\n      <td>Systolic Blood Pressure</td>\n      <td>122</td>\n      <td>mmHg</td>\n      <td>Patient/145351</td>\n    </tr>\n    <tr>\n      <th>119</th>\n      <td>145479</td>\n      <td>2017-02-15T23:19:16-05:00</td>\n      <td>Diastolic Blood Pressure</td>\n      <td>81</td>\n      <td>mmHg</td>\n      <td>Patient/145351</td>\n    </tr>\n  </tbody>\n</table>\n<p>240 rows × 6 columns</p>\n</div>"
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "observation_df.explode(\n",
    "    [\n",
    "        \"test\",\n",
    "        \"value\",\n",
    "        \"unit\",\n",
    "    ]\n",
    ")"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "Or we could build a processing function, which would give us a nicer naming scheme and a better\n",
    "relationship between the patients and the columns."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Query & Build DF (Observation): 100%|██████████| 31/31 [00:07<00:00,  3.97it/s]\n"
     ]
    },
    {
     "data": {
      "text/plain": "    observation_id  Systolic Blood Pressure Systolic Blood Pressure Unit  \\\n0             1860                    111.0                         mmHg   \n1             1879                    120.0                         mmHg   \n2             1904                    102.0                         mmHg   \n3             1929                    130.0                         mmHg   \n4             1947                    127.0                         mmHg   \n..             ...                      ...                          ...   \n115         145436                    124.0                         mmHg   \n116         145444                    119.0                         mmHg   \n117         145454                    107.0                         mmHg   \n118         145468                    112.0                         mmHg   \n119         145479                    122.0                         mmHg   \n\n     Diastolic Blood Pressure Diastolic Blood Pressure Unit      patient_id  \n0                        72.0                          mmHg    Patient/1834  \n1                        81.0                          mmHg    Patient/1834  \n2                        85.0                          mmHg    Patient/1834  \n3                        77.0                          mmHg    Patient/1834  \n4                        74.0                          mmHg    Patient/1834  \n..                        ...                           ...             ...  \n115                      87.0                          mmHg  Patient/145351  \n116                      82.0                          mmHg  Patient/145351  \n117                      75.0                          mmHg  Patient/145351  \n118                      81.0                          mmHg  Patient/145351  \n119                      81.0                          mmHg  Patient/145351  \n\n[120 rows x 6 columns]",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>observation_id</th>\n      <th>Systolic Blood Pressure</th>\n      <th>Systolic Blood Pressure Unit</th>\n      <th>Diastolic Blood Pressure</th>\n      <th>Diastolic Blood Pressure Unit</th>\n      <th>patient_id</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1860</td>\n      <td>111.0</td>\n      <td>mmHg</td>\n      <td>72.0</td>\n      <td>mmHg</td>\n      <td>Patient/1834</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1879</td>\n      <td>120.0</td>\n      <td>mmHg</td>\n      <td>81.0</td>\n      <td>mmHg</td>\n      <td>Patient/1834</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1904</td>\n      <td>102.0</td>\n      <td>mmHg</td>\n      <td>85.0</td>\n      <td>mmHg</td>\n      <td>Patient/1834</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1929</td>\n      <td>130.0</td>\n      <td>mmHg</td>\n      <td>77.0</td>\n      <td>mmHg</td>\n      <td>Patient/1834</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1947</td>\n      <td>127.0</td>\n      <td>mmHg</td>\n      <td>74.0</td>\n      <td>mmHg</td>\n      <td>Patient/1834</td>\n    </tr>\n    <tr>\n      <th>...</th>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n    </tr>\n    <tr>\n      <th>115</th>\n      <td>145436</td>\n      <td>124.0</td>\n      <td>mmHg</td>\n      <td>87.0</td>\n      <td>mmHg</td>\n      <td>Patient/145351</td>\n    </tr>\n    <tr>\n      <th>116</th>\n      <td>145444</td>\n      <td>119.0</td>\n      <td>mmHg</td>\n      <td>82.0</td>\n      <td>mmHg</td>\n      <td>Patient/145351</td>\n    </tr>\n    <tr>\n      <th>117</th>\n      <td>145454</td>\n      <td>107.0</td>\n      <td>mmHg</td>\n      <td>75.0</td>\n      <td>mmHg</td>\n      <td>Patient/145351</td>\n    </tr>\n    <tr>\n      <th>118</th>\n      <td>145468</td>\n      <td>112.0</td>\n      <td>mmHg</td>\n      <td>81.0</td>\n      <td>mmHg</td>\n      <td>Patient/145351</td>\n    </tr>\n    <tr>\n      <th>119</th>\n      <td>145479</td>\n      <td>122.0</td>\n      <td>mmHg</td>\n      <td>81.0</td>\n      <td>mmHg</td>\n      <td>Patient/145351</td>\n    </tr>\n  </tbody>\n</table>\n<p>120 rows × 6 columns</p>\n</div>"
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def get_observation_info(bundle: FHIRObj) -> List[Dict]:\n",
    "    records = []\n",
    "    for entry in bundle.entry or []:\n",
    "        resource = entry.resource\n",
    "        # Store the ID\n",
    "        base_dict = {\"observation_id\": resource.id}\n",
    "        for component in resource.component or []:\n",
    "            # Go through the code.codings of the current components to get a name for our value\n",
    "            # and store the display value\n",
    "            resource_name = next(\n",
    "                iter([coding.display for coding in component.code.coding or []]), None\n",
    "            )\n",
    "            if component.valueQuantity is not None:\n",
    "                # If the component is a valueQuantity, get the value\n",
    "                base_dict[resource_name] = component.valueQuantity.value\n",
    "                base_dict[resource_name + \" Unit\"] = component.valueQuantity.unit\n",
    "        records.append(base_dict)\n",
    "    return records\n",
    "\n",
    "observation_df = search.trade_rows_for_dataframe(\n",
    "    df=patient_df,\n",
    "    resource_type=\"Observation\",\n",
    "    request_params={\n",
    "        \"_count\": 100,\n",
    "        \"code\": \"http://loinc.org%7C55284-4\",\n",
    "        \"_sort\": \"_id\",\n",
    "    },\n",
    "    with_ref=True,\n",
    "    df_constraints={\"subject\": \"patient_id\"},\n",
    "    process_function=get_observation_info,  # Use processing function instead of FHIRPath\n",
    ")\n",
    "observation_df"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "And here they are, our results neatly organized in separate rows for each patient!"
   ],
   "metadata": {
    "collapsed": false
   }
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}