US-Core-R4-Patient.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create Argonaut Patient Resource Using the Python FHIRClient Module\n",
    "\n",
    "### Prerequisites:\n",
    "\n",
    "- Python 3.6 or greater\n",
    "    for instruction how to install Python and modules and optionally setting up atom as a python IDE and creating virtual environments, If you are new to Python or need help getting starged I recommend these YouYube videos by Corey Schaefer:\n",
    "    \n",
    "[![image.jpg](http://img.youtube.com/vi/YYXdXT2l-Gg/sddefault.jpg)](https://www.youtube.com/watch?v=YYXdXT2l-Gg)\n",
    "\n",
    "[![image.jpg](http://img.youtube.com/vi/CqvZ3vGoGs0/sddefault.jpg)](https://www.youtube.com/watch?v=CqvZ3vGoGs0)\n",
    "\n",
    "[![image.jpg](http://img.youtube.com/vi/DjEuROpsvp4/sddefault.jpg)](https://www.youtube.com/watch?v=DjEuROpsvp4)\n",
    "\n",
    "[![image.jpg](http://img.youtube.com/vi/N5vscPTWKOk/sddefault.jpg)](https://www.youtube.com/watch?v=N5vscPTWKOk)\n",
    "\n",
    "\n",
    "- installed modules:\n",
    "   - [fhirclient](https://github.com/smart-on-fhir/client-py)  ( FHIR version STU3 )\n",
    "   - [requests](docs.python-requests.org/en/master/)\n",
    "   - [csv — CSV File Reading and Writing](https://docs.python.org/3/library/csv.html)\n",
    "   \n",
    "### Introduction:\n",
    " \n",
    "IN the code below, I walk through the steps to create an Argonaut (FHIR DSTU2) Patient resource, validate it, Post it to a FHIR server and then fetch it in xml format.  The data is based off of a simple excel file which serves as a data entry form.  To simplify things the worksheet is saved as a CSV file which is used directly in the code below.  \n",
    "\n",
    "See this nice [FHIRClient introduction documentation](https://github.com/bmilius-nmdp/pytestfhir/blob/master/MakePatSpeObs.ipynb) by Bob Milius for a quick overview how to use the Python fhirclient models.  The models are instantiated with a Python dictionary structure or primitives types ( str, int, float, or bool ).   Repeating elements are represented at lists.  Also the FHIRClient models perform some data validation as well and provide for a more `pythonic` way to represent FHIR resources.\n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### First import all the necessary modules\n",
    "\n",
    "We will be using all these FHIRClient models ( classes ) for this examples and they include the patient resource models and all the datatype classes used by the resource.  The `IPython` is used to make this Jupyter Notebook display pretty.\n",
    "\n",
    "The version of the fhirclient.models used for R4 Resource is not yet available on pip install  ( see the Fhirclient docs ) \n",
    "\n",
    "[Here](https://github.com/Healthedata1/MyNotebooks/blob/master/r4models.zip) are my  personal fhirclient r4models which I generated  from the fhir-parser.  They run all the models I used so far except for Bundles.\n",
    "\n",
    "Add the folder in your virtual env library\n",
    "like so:\n",
    "\n",
    "`...[our python environment path]/lib/python3.7/site-packages/fhirclient/r4models`\n",
    "e.g.\n",
    "\n",
    "`.../Python/Venv/venv37/lib/python3.7/site-packages/fhirclient/r4models`\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ******************** modules **************************\n",
    "\n",
    "import fhirclient.r4models.patient as P\n",
    "import fhirclient.r4models.address as A\n",
    "import fhirclient.r4models.identifier as I\n",
    "import fhirclient.r4models.coding as C\n",
    "import fhirclient.r4models.codeableconcept as CC\n",
    "import fhirclient.r4models.fhirdate as D\n",
    "import fhirclient.r4models.extension as X\n",
    "import fhirclient.r4models.humanname as H\n",
    "import fhirclient.r4models.meta as M\n",
    "import fhirclient.r4models.valueset as VS\n",
    "from json import dumps\n",
    "from requests import get, post, put\n",
    "import os\n",
    "from csv import DictReader\n",
    "from IPython.display import display, Markdown, HTML\n",
    "from pprint import pprint\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "####  Assign Global Variables\n",
    "\n",
    "Here is where we assign all the global varaibles for this example such as the local paths for file input and output which need to\n",
    "be updated for your local environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "out_path = '/Users/ehaas/Documents/FHIR/pyfhir/test/'\n",
    "#out_path = \"C:/Users/Eric/Documents/Jan_2019_FHIR_Experience\"\n",
    "#fhir_term_server = 'http://fhirtest.uhn.ca/baseDstu2'\n",
    "#fhir_test_server = 'http://fhirtest.uhn.ca/baseDstu2'\n",
    "fhir_term_server = 'http://fhirtest.uhn.ca/R4'\n",
    "fhir_test_server = 'http://fhirtest.uhn.ca/R4'\n",
    "fhir_test_server= 'http://test.fhir.org/r4'\n",
    "\n",
    "headers = {\n",
    "'Accept':'application/fhir+json',\n",
    "'Content-Type':'application/fhir+json'\n",
    "}\n",
    "\n",
    "profile = 'http://hl7.org/fhir/us/core/StructureDefinition/us-core-patient|4.0.0' # The official canonical URL for this profile is: http://hl7.org/fhir/us/core/StructureDefinition/us-core-patient\n",
    "# profile = 'http://fhir.org/guides/argonaut/StructureDefinition/argo-patient'\n",
    "\n",
    "code_map = {  # assign o local map to store displays since unable to locate open test terminology server that support the argo valuesets lookups :-(\n",
    "'ar': 'Arabic',\n",
    "'bn': 'Bengali',\n",
    "'cs': 'Czech',\n",
    "'da': 'Danish',\n",
    "'de': 'German',\n",
    "'de-AT': 'German (Austria)',\n",
    "'de-CH': 'German (Switzerland)',\n",
    "'de-DE': 'German (Germany)',\n",
    "'el': 'Greek',\n",
    "'en': 'English',\n",
    "'en-AU': 'English (Australia)',\n",
    "'en-CA': 'English (Canada)',\n",
    "'en-GB': 'English (Great Britain)',\n",
    "'en-IN': 'English (India)',\n",
    "'en-NZ': 'English (New Zeland)',\n",
    "'en-SG': 'English (Singapore)',\n",
    "'en-US': 'English (United States)',\n",
    "'es': 'Spanish',\n",
    "'es-AR': 'Spanish (Argentina)',\n",
    "'es-ES': 'Spanish (Spain)',\n",
    "'es-UY': 'Spanish (Uruguay)',\n",
    "'fi': 'Finnish',\n",
    "'fr': 'French',\n",
    "'fr-BE': 'French (Belgium)',\n",
    "'fr-CH': 'French (Switzerland)',\n",
    "'fr-FR': 'French (France)',\n",
    "'fy': 'Frysian',\n",
    "'fy-NL': 'Frysian (Netherlands)',\n",
    "'hi': 'Hindi',\n",
    "'hr': 'Croatian',\n",
    "'it': 'Italian',\n",
    "'it-CH': 'Italian (Switzerland)',\n",
    "'it-IT': 'Italian (Italy)',\n",
    "'ja': 'Japanese',\n",
    "'ko': 'Korean',\n",
    "'nl': 'Dutch',\n",
    "'nl-BE': 'Dutch (Belgium)',\n",
    "'nl-NL': 'Dutch (Netherlands)',\n",
    "'no': 'Norwegian',\n",
    "'no-NO': 'Norwegian (Norway)',\n",
    "'pa': 'Punjabi',\n",
    "'pl': 'Polish',\n",
    "'pt': 'Portuguese',\n",
    "'pt-BR': 'Portuguese (Brazil)',\n",
    "'ru': 'Russian',\n",
    "'ru-RU': 'Russian (Russia)',\n",
    "'sr': 'Serbian',\n",
    "'sr-RS': 'Serbian (Serbia)',\n",
    "'sv': 'Swedish',\n",
    "'sv-SE': 'Swedish (Sweden)',\n",
    "'te': 'Telegu',\n",
    "'zh': 'Chinese',\n",
    "'zh-CN': 'Chinese (China)',\n",
    "'zh-HK': 'Chinese (Hong Kong)',\n",
    "'zh-SG': 'Chinese (Singapore)',\n",
    "'zh-TW': 'Chinese (Taiwan)',\n",
    "'1002-5': 'American Indian or Alaska Native',\n",
    "'2028-9': 'Asian',\n",
    "'2054-5': 'Black or African American',\n",
    "'2076-8': 'Native Hawaiian or Other Pacific Islander',\n",
    "'2106-3': 'White',\n",
    "'2135-2': 'Hispanic or Latino',\n",
    "'2186-5': 'Not Hispanic or Latino'\n",
    "}\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "####  Instanatiating All the Classes\n",
    "\n",
    "Before we start assigning patient data to the Patient Class, The classes are instantatiated.  See [this video](https://www.youtube.com/watch?v=ZDa-Z5JzLYM) for more\n",
    "information about Python classes and instances.  For the FHIRClient Python models, the classes are represented as Python dictionary structures using the method `as_json()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "**Instantiated Python FHIR Patient Class as a Python dictionary object:** `{'resourceType': 'Patient'}`"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#********************* instantiate Patient classes **********************\n",
    "\n",
    "patient = P.Patient()\n",
    "\n",
    "display(Markdown(f'**Instantiated Python FHIR Patient Class as a Python dictionary object:** `{patient.as_json()}`'))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Get patient data\n",
    "\n",
    "To populate the patient resource, the data is entered from some source.  The datasource may be a database, spreadsheet, manual entry\n",
    "(e.g. from a form). For this example, a simple Python dictionary file of name:value pairs is used.  See the US Core Patient example for how data from a csv file is imported into the resource."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = dict(\n",
    "mrn = '1234',\n",
    "f_name = 'John',\n",
    "l_name = \"Doe\",\n",
    "mi = 'M',\n",
    "b_date = '1964-06-19',\n",
    "gender = 'male',\n",
    "b_sex = 'M',\n",
    "lang = 'en',\n",
    "race = '2106-3', # code for 'White'\n",
    "ethn = '2186-5', # Not hispanic or latino\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Create Patient resource\n",
    "\n",
    "After getting the source data, the patient resource can be populated.   The following code cells shows step by step how the different data types are assigned using the FHIRClient model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#####  Assign values to the simple id and active elements"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'active': True, 'id': 'subject1', 'resourceType': 'Patient'}\n"
     ]
    }
   ],
   "source": [
    "# create simple id and active elements\n",
    "patient.id = 'subject1'\n",
    "patient.active = True\n",
    "pprint(patient.as_json())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#####  Assign values to complex `meta` element \n",
    " \n",
    " The meta class contains a repeating `profile` element"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'active': True,\n",
      " 'id': 'subject1',\n",
      " 'meta': {'profile': ['http://hl7.org/fhir/us/core/StructureDefinition/us-core-patient|4.0.0']},\n",
      " 'resourceType': 'Patient'}\n"
     ]
    }
   ],
   "source": [
    "meta = M.Meta() # instantiates meta class\n",
    "meta.profile = []\n",
    "meta.profile.append(profile)\n",
    "patient.meta = meta\n",
    "pprint(patient.as_json())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#####  Assign values to the complex identifier element\n",
    "\n",
    "Note in this example only a identifier is provided."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'active': True,\n",
      " 'id': 'subject1',\n",
      " 'identifier': [{'system': 'http://example.org/pids', 'value': '1234'}],\n",
      " 'meta': {'profile': ['http://hl7.org/fhir/us/core/StructureDefinition/us-core-patient|4.0.0']},\n",
      " 'resourceType': 'Patient'}\n"
     ]
    }
   ],
   "source": [
    "# create complex identifier element\n",
    "identifier = I.Identifier()\n",
    "identifier.system = 'http://example.org/pids'\n",
    "identifier.value = data['mrn']\n",
    "patient.identifier=[]\n",
    "patient.identifier.append(identifier)\n",
    "pprint(patient.as_json())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#####  Assign values to the complex repeating name element\n",
    "\n",
    "Note in this example only a single name is provided."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'active': True,\n",
      " 'id': 'subject1',\n",
      " 'identifier': [{'system': 'http://example.org/pids', 'value': '1234'}],\n",
      " 'meta': {'profile': ['http://hl7.org/fhir/us/core/StructureDefinition/us-core-patient|4.0.0']},\n",
      " 'name': [{'family': 'Doe', 'given': ['John', 'M']}],\n",
      " 'resourceType': 'Patient'}\n"
     ]
    }
   ],
   "source": [
    "# create complex repeating name element\n",
    "name = H.HumanName()\n",
    "name.family = data['l_name']\n",
    "name.given = []\n",
    "name.given.append(data['f_name'])\n",
    "if data['mi']: name.given.append(data['mi'])\n",
    "patient.name=[]\n",
    "patient.name.append(name)\n",
    "pprint(patient.as_json())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#####   Assign simple `gender` and `birthdate` elements\n",
    "\n",
    "Birthdate is treated as an iso string in Python."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'active': True,\n",
      " 'birthDate': '1964-06-19',\n",
      " 'gender': 'male',\n",
      " 'id': 'subject1',\n",
      " 'identifier': [{'system': 'http://example.org/pids', 'value': '1234'}],\n",
      " 'meta': {'profile': ['http://hl7.org/fhir/us/core/StructureDefinition/us-core-patient|4.0.0']},\n",
      " 'name': [{'family': 'Doe', 'given': ['John', 'M']}],\n",
      " 'resourceType': 'Patient'}\n"
     ]
    }
   ],
   "source": [
    "#create simple gender and birthdate element ( birthdate is treated as iso string in Python)\n",
    "patient.gender = data['gender']\n",
    "b_date = D.FHIRDate(data['b_date'])\n",
    "patient.birthDate = b_date\n",
    "pprint(patient.as_json())\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### add address"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'id': 'subject1',\n",
       " 'meta': {'profile': ['http://hl7.org/fhir/us/core/StructureDefinition/us-core-patient|4.0.0']},\n",
       " 'active': True,\n",
       " 'address': [{'city': 'Anytown',\n",
       "   'country': 'USA',\n",
       "   'line': ['100 Main St'],\n",
       "   'postalCode': '99999',\n",
       "   'state': 'CA',\n",
       "   'text': '100 Main St\\nAnytown, CA 99999\\nUSA'}],\n",
       " 'birthDate': '1964-06-19',\n",
       " 'gender': 'male',\n",
       " 'identifier': [{'system': 'http://example.org/pids', 'value': '1234'}],\n",
       " 'name': [{'family': 'Doe', 'given': ['John', 'M']}],\n",
       " 'resourceType': 'Patient'}"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "address = A.Address()\n",
    "address.line = ['100 Main St']\n",
    "address.city = 'Anytown'\n",
    "address.state = 'CA'\n",
    "address.postalCode = '99999'\n",
    "address.country = 'USA'\n",
    "address.text = f'''{address.line[0]}\n",
    "{address.city}, {address.state} {address.postalCode}\n",
    "{address.country}'''\n",
    "patient.address = [address]\n",
    "patient.as_json()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#####  Assign backbone `communication` element \n",
    "\n",
    "`CodeableConcept` and `Coding` datatypes are used in this element.  This pattern is repeated for other coded elements below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'active': True,\n",
      " 'address': [{'city': 'Anytown',\n",
      "              'country': 'USA',\n",
      "              'line': ['100 Main St'],\n",
      "              'postalCode': '99999',\n",
      "              'state': 'CA',\n",
      "              'text': '100 Main St\\nAnytown, CA 99999\\nUSA'}],\n",
      " 'birthDate': '1964-06-19',\n",
      " 'communication': [{'language': {'coding': [{'code': 'en',\n",
      "                                             'display': 'English',\n",
      "                                             'system': 'urn:ietf:bcp:47'}],\n",
      "                                 'text': 'English'}}],\n",
      " 'gender': 'male',\n",
      " 'id': 'subject1',\n",
      " 'identifier': [{'system': 'http://example.org/pids', 'value': '1234'}],\n",
      " 'meta': {'profile': ['http://hl7.org/fhir/us/core/StructureDefinition/us-core-patient|4.0.0']},\n",
      " 'name': [{'family': 'Doe', 'given': ['John', 'M']}],\n",
      " 'resourceType': 'Patient'}\n"
     ]
    }
   ],
   "source": [
    "# create backbone communication element with codeableconcept\n",
    "patientcommunication = P.PatientCommunication()\n",
    "language = CC.CodeableConcept()\n",
    "coding = C.Coding()\n",
    "patient.communication = []\n",
    "dis = code_map[data['lang']]\n",
    "sys = 'urn:ietf:bcp:47'\n",
    "language.coding = []\n",
    "coding.code = data['lang']\n",
    "coding.system = sys\n",
    "coding.display = dis\n",
    "language.coding.append(coding)\n",
    "language.text = dis\n",
    "patientcommunication.language = language\n",
    "patient.communication.append(patientcommunication)\n",
    "pprint(patient.as_json())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#####  Assign values to the extension\n",
    "\n",
    "There are three extensions defined for the Argonaut Patient.  The \"Birth Sex\" extension is a simple extension and the \"Race\" and \"Ethnicity\" extensions are complex extensions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "######  Assign values to the simple \"Birth Sex\" extension\n",
    "\n",
    "For each simple extension, a url and a value ( in this case a code ) are assigned."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'active': True,\n",
      " 'address': [{'city': 'Anytown',\n",
      "              'country': 'USA',\n",
      "              'line': ['100 Main St'],\n",
      "              'postalCode': '99999',\n",
      "              'state': 'CA',\n",
      "              'text': '100 Main St\\nAnytown, CA 99999\\nUSA'}],\n",
      " 'birthDate': '1964-06-19',\n",
      " 'communication': [{'language': {'coding': [{'code': 'en',\n",
      "                                             'display': 'English',\n",
      "                                             'system': 'urn:ietf:bcp:47'}],\n",
      "                                 'text': 'English'}}],\n",
      " 'extension': [{'url': 'http://hl7.org/fhir/us/core/StructureDefinition/us-core-birthsex',\n",
      "                'valueCode': 'M'}],\n",
      " 'gender': 'male',\n",
      " 'id': 'subject1',\n",
      " 'identifier': [{'system': 'http://example.org/pids', 'value': '1234'}],\n",
      " 'meta': {'profile': ['http://hl7.org/fhir/us/core/StructureDefinition/us-core-patient|4.0.0']},\n",
      " 'name': [{'family': 'Doe', 'given': ['John', 'M']}],\n",
      " 'resourceType': 'Patient'}\n"
     ]
    }
   ],
   "source": [
    "# create simple birth sex (repeating) extension\n",
    "b_sex_extension = X.Extension()\n",
    "patient.extension = []\n",
    "b_sex_extension.url = 'http://hl7.org/fhir/us/core/StructureDefinition/us-core-birthsex'\n",
    "#b_sex_extension.url = 'http://fhir.org/guides/argonaut/StructureDefinition/argo-birthsex'  # canonical url for birthsex is 'http://hl7.org/fhir/us/core/StructureDefinition/us-core-birthsex'\n",
    "b_sex_extension.valueCode = data['b_sex']\n",
    "patient.extension.append(b_sex_extension)\n",
    "pprint(patient.as_json())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###### Assign values to the complex \"race\" extension\n",
    "\n",
    "For each complex extension, a url and nested extensions and a values ( in this case a CodeableConcepts and strings ) are assigned.\n",
    "\n",
    "For this example, a simple hash map is used to fetch a code display for the race code.  For an example of using a Terminology Server and the $lookup operation, see the US Core Patient example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'active': True,\n",
      " 'address': [{'city': 'Anytown',\n",
      "              'country': 'USA',\n",
      "              'line': ['100 Main St'],\n",
      "              'postalCode': '99999',\n",
      "              'state': 'CA',\n",
      "              'text': '100 Main St\\nAnytown, CA 99999\\nUSA'}],\n",
      " 'birthDate': '1964-06-19',\n",
      " 'communication': [{'language': {'coding': [{'code': 'en',\n",
      "                                             'display': 'English',\n",
      "                                             'system': 'urn:ietf:bcp:47'}],\n",
      "                                 'text': 'English'}}],\n",
      " 'extension': [{'url': 'http://hl7.org/fhir/us/core/StructureDefinition/us-core-birthsex',\n",
      "                'valueCode': 'M'},\n",
      "               {'extension': [{'url': 'text', 'valueString': 'White'},\n",
      "                              {'url': 'ombCategory',\n",
      "                               'valueCoding': {'code': '2106-3',\n",
      "                                               'display': 'White',\n",
      "                                               'system': 'urn:oid:2.16.840.1.113883.6.238'}}],\n",
      "                'url': 'http://hl7.org/fhir/us/core/StructureDefinition/us-core-race'}],\n",
      " 'gender': 'male',\n",
      " 'id': 'subject1',\n",
      " 'identifier': [{'system': 'http://example.org/pids', 'value': '1234'}],\n",
      " 'meta': {'profile': ['http://hl7.org/fhir/us/core/StructureDefinition/us-core-patient|4.0.0']},\n",
      " 'name': [{'family': 'Doe', 'given': ['John', 'M']}],\n",
      " 'resourceType': 'Patient'}\n"
     ]
    }
   ],
   "source": [
    "# create complex race  (repeating) extension using codeableconcept\n",
    "race_extension = X.Extension()\n",
    "race_ombcat_extension = X.Extension()\n",
    "race_ombdet_extension = X.Extension()\n",
    "race_text_extension = X.Extension()\n",
    "race = C.Coding()\n",
    "\n",
    "sys = 'urn:oid:2.16.840.1.113883.6.238'  if data['race'] not in ['UNK','ASKU'] else 'http://hl7.org/fhir/v3/NullFlavor'\n",
    "dis = code_map[data['race']]\n",
    "#race_extension.url ='http://fhir.org/guides/argonaut/StructureDefinition/argo-race'\n",
    "race_extension.url = 'http://hl7.org/fhir/us/core/StructureDefinition/us-core-race'  # canonical url for race\n",
    "race_extension.extension = []\n",
    "race_text_extension.url = 'text'  # url for \"subextension\"\n",
    "race_text_extension.valueString = dis  # use display\n",
    "race_extension.extension.append(race_text_extension)\n",
    "\n",
    "race_ombcat_extension.url = 'ombCategory'  # url for \"subextension\"\n",
    "race.system = sys  # look up using terminology server\n",
    "race.code = data['race']\n",
    "race.display = dis  # look up using terminology server\n",
    "race_ombcat_extension.valueCoding = race\n",
    "race_extension.extension.append(race_ombcat_extension)\n",
    "\n",
    "patient.extension.append(race_extension)\n",
    "pprint(patient.as_json())\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally to complete this example , the ehtnicity extension is populated with a similar pattern to the race extension above"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'active': True,\n",
      " 'address': [{'city': 'Anytown',\n",
      "              'country': 'USA',\n",
      "              'line': ['100 Main St'],\n",
      "              'postalCode': '99999',\n",
      "              'state': 'CA',\n",
      "              'text': '100 Main St\\nAnytown, CA 99999\\nUSA'}],\n",
      " 'birthDate': '1964-06-19',\n",
      " 'communication': [{'language': {'coding': [{'code': 'en',\n",
      "                                             'display': 'English',\n",
      "                                             'system': 'urn:ietf:bcp:47'}],\n",
      "                                 'text': 'English'}}],\n",
      " 'extension': [{'url': 'http://hl7.org/fhir/us/core/StructureDefinition/us-core-birthsex',\n",
      "                'valueCode': 'M'},\n",
      "               {'extension': [{'url': 'text', 'valueString': 'White'},\n",
      "                              {'url': 'ombCategory',\n",
      "                               'valueCoding': {'code': '2106-3',\n",
      "                                               'display': 'White',\n",
      "                                               'system': 'urn:oid:2.16.840.1.113883.6.238'}}],\n",
      "                'url': 'http://hl7.org/fhir/us/core/StructureDefinition/us-core-race'},\n",
      "               {'extension': [{'url': 'text',\n",
      "                               'valueString': 'Not Hispanic or Latino'},\n",
      "                              {'url': 'ombCategory',\n",
      "                               'valueCoding': {'code': '2186-5',\n",
      "                                               'display': 'Not Hispanic or '\n",
      "                                                          'Latino',\n",
      "                                               'system': 'urn:oid:2.16.840.1.113883.6.238'}}],\n",
      "                'url': 'http://hl7.org/fhir/us/core/StructureDefinition/us-core-ethnicity'}],\n",
      " 'gender': 'male',\n",
      " 'id': 'subject1',\n",
      " 'identifier': [{'system': 'http://example.org/pids', 'value': '1234'}],\n",
      " 'meta': {'profile': ['http://hl7.org/fhir/us/core/StructureDefinition/us-core-patient|4.0.0']},\n",
      " 'name': [{'family': 'Doe', 'given': ['John', 'M']}],\n",
      " 'resourceType': 'Patient'}\n"
     ]
    }
   ],
   "source": [
    "# create complex ethnicity (repeating) extension using codeableconcept\n",
    "\n",
    "ethn_extension = X.Extension()\n",
    "ethn_ombcat_extension = X.Extension()\n",
    "ethn_ombdet_extension = X.Extension()\n",
    "ethn_text_extension = X.Extension()\n",
    "ethn = C.Coding()\n",
    "\n",
    "sys = 'urn:oid:2.16.840.1.113883.6.238'\n",
    "dis = code_map[data['ethn']]\n",
    "\n",
    "ethn_extension.url = 'http://hl7.org/fhir/us/core/StructureDefinition/us-core-ethnicity'  # canonical url for ethn\n",
    "ethn_extension.extension = []\n",
    "\n",
    "ethn_text_extension.url = 'text'  # url for \"subextension\"\n",
    "ethn_text_extension.valueString = dis  # use display\n",
    "ethn_extension.extension.append(ethn_text_extension)\n",
    "\n",
    "ethn_ombcat_extension.url = 'ombCategory'  # url for \"subextension\"\n",
    "ethn.system = sys  # look up using terminology server\n",
    "ethn.code = data['ethn']\n",
    "ethn.display = dis  # look up using terminology server\n",
    "ethn_ombcat_extension.valueCoding = ethn\n",
    "ethn_extension.extension.append(ethn_ombcat_extension)\n",
    "\n",
    "patient.extension.append(ethn_extension)\n",
    "pprint(patient.as_json())\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Pretty Print the resource as a json string\n",
    "\n",
    "In Python the resource is represented as a 'dict' type.  to share this data it needs to be converted to JSON format using the Python standard library methods from the \"json\" module"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " {\n",
      "   \"id\": \"subject1\",\n",
      "   \"meta\": {\n",
      "      \"profile\": [\n",
      "         \"http://hl7.org/fhir/us/core/StructureDefinition/us-core-patient|4.0.0\"\n",
      "      ]\n",
      "   },\n",
      "   \"extension\": [\n",
      "      {\n",
      "         \"url\": \"http://hl7.org/fhir/us/core/StructureDefinition/us-core-birthsex\",\n",
      "         \"valueCode\": \"M\"\n",
      "      },\n",
      "      {\n",
      "         \"extension\": [\n",
      "            {\n",
      "               \"url\": \"text\",\n",
      "               \"valueString\": \"White\"\n",
      "            },\n",
      "            {\n",
      "               \"url\": \"ombCategory\",\n",
      "               \"valueCoding\": {\n",
      "                  \"code\": \"2106-3\",\n",
      "                  \"display\": \"White\",\n",
      "                  \"system\": \"urn:oid:2.16.840.1.113883.6.238\"\n",
      "               }\n",
      "            }\n",
      "         ],\n",
      "         \"url\": \"http://hl7.org/fhir/us/core/StructureDefinition/us-core-race\"\n",
      "      },\n",
      "      {\n",
      "         \"extension\": [\n",
      "            {\n",
      "               \"url\": \"text\",\n",
      "               \"valueString\": \"Not Hispanic or Latino\"\n",
      "            },\n",
      "            {\n",
      "               \"url\": \"ombCategory\",\n",
      "               \"valueCoding\": {\n",
      "                  \"code\": \"2186-5\",\n",
      "                  \"display\": \"Not Hispanic or Latino\",\n",
      "                  \"system\": \"urn:oid:2.16.840.1.113883.6.238\"\n",
      "               }\n",
      "            }\n",
      "         ],\n",
      "         \"url\": \"http://hl7.org/fhir/us/core/StructureDefinition/us-core-ethnicity\"\n",
      "      }\n",
      "   ],\n",
      "   \"active\": true,\n",
      "   \"address\": [\n",
      "      {\n",
      "         \"city\": \"Anytown\",\n",
      "         \"country\": \"USA\",\n",
      "         \"line\": [\n",
      "            \"100 Main St\"\n",
      "         ],\n",
      "         \"postalCode\": \"99999\",\n",
      "         \"state\": \"CA\",\n",
      "         \"text\": \"100 Main St\\nAnytown, CA 99999\\nUSA\"\n",
      "      }\n",
      "   ],\n",
      "   \"birthDate\": \"1964-06-19\",\n",
      "   \"communication\": [\n",
      "      {\n",
      "         \"language\": {\n",
      "            \"coding\": [\n",
      "               {\n",
      "                  \"code\": \"en\",\n",
      "                  \"display\": \"English\",\n",
      "                  \"system\": \"urn:ietf:bcp:47\"\n",
      "               }\n",
      "            ],\n",
      "            \"text\": \"English\"\n",
      "         }\n",
      "      }\n",
      "   ],\n",
      "   \"gender\": \"male\",\n",
      "   \"identifier\": [\n",
      "      {\n",
      "         \"system\": \"http://example.org/pids\",\n",
      "         \"value\": \"1234\"\n",
      "      }\n",
      "   ],\n",
      "   \"name\": [\n",
      "      {\n",
      "         \"family\": \"Doe\",\n",
      "         \"given\": [\n",
      "            \"John\",\n",
      "            \"M\"\n",
      "         ]\n",
      "      }\n",
      "   ],\n",
      "   \"resourceType\": \"Patient\"\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "# pretty print the resource as a json string  ( in Python is a 'dict' type)\n",
    "\n",
    "print('\\n',dumps(patient.as_json(),indent =3 ))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "####  Validate the Resource\n",
    "\n",
    "Using the [$validate](http://build.fhir.org/resource-operation-validate.html) operation, the example is validated \n",
    "by a FHIR Reference Server.  The results are displayed below in the human readable text as xhtml.\n",
    "\n",
    "**NOTE** Unable to locate a Server that performs the \\$validate operation on the Argonaut Profiles.  Therefore this operation fails.  See the US Core Patient example for a successful outcome using this operation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<h1>Validation output</h1><div xmlns=\"http://www.w3.org/1999/xhtml\"><p>Unknown message Unknown StructureDefinition url: http://hl7.org/fhir/us/core/StructureDefinition/us-core-patient|4.0.0</p></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# *********************** validate Resource ********************************\n",
    "\n",
    "params = {\n",
    "'profile': profile\n",
    "}\n",
    "#   r = requests.post('https://httpbin.org/post', data = {'key':'value'})\n",
    "r = post(f'{fhir_test_server}/Patient/$validate', params = params, headers = headers, data = dumps(patient.as_json()))\n",
    "# print(r.status_code)\n",
    "# view  output\n",
    "display(HTML(f'<h1>Validation output</h1>{r.json()[\"text\"][\"div\"]}'))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### PUT Resource on FHIR Server\n",
    "\n",
    "Using the FHIR RESTful `PUT` transaction, the resource is loaded onto the FHIR test server. The results are displayed below in the human readable text as xhtml."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "<h1>Put Response</h1><p><strong>Transaction Status</strong> = 200</p><div xmlns=\"http://www.w3.org/1999/xhtml\"><p><b>Generated Narrative with Details</b></p><p><b style=\"color: maroon\">Exception generating Narrative: unknown extension http://hl7.org/fhir/us/core/StructureDefinition/us-core-birthsex</b></p></div><p>raw output :</p><code>{\n",
       "   \"resourceType\": \"Patient\",\n",
       "   \"id\": \"subject1\",\n",
       "   \"meta\": {\n",
       "      \"versionId\": \"5\",\n",
       "      \"lastUpdated\": \"2019-05-08T16:50:12.531Z\",\n",
       "      \"profile\": [\n",
       "         \"http://hl7.org/fhir/us/core/StructureDefinition/us-core-patient|4.0.0\",\n",
       "         \"http://hl7.org/fhir/us/core/StructureDefinition/us-core-patient\"\n",
       "      ]\n",
       "   },\n",
       "   \"text\": {\n",
       "      \"status\": \"generated\",\n",
       "      \"div\": \"<div xmlns=\\\"http://www.w3.org/1999/xhtml\\\"><p><b>Generated Narrative with Details</b></p><p><b style=\\\"color: maroon\\\">Exception generating Narrative: unknown extension http://hl7.org/fhir/us/core/StructureDefinition/us-core-birthsex</b></p></div>\"\n",
       "   },\n",
       "   \"extension\": [\n",
       "      {\n",
       "         \"url\": \"http://hl7.org/fhir/us/core/StructureDefinition/us-core-birthsex\",\n",
       "         \"valueCode\": \"M\"\n",
       "      },\n",
       "      {\n",
       "         \"extension\": [\n",
       "            {\n",
       "               \"url\": \"text\",\n",
       "               \"valueString\": \"White\"\n",
       "            },\n",
       "            {\n",
       "               \"url\": \"ombCategory\",\n",
       "               \"valueCoding\": {\n",
       "                  \"system\": \"urn:oid:2.16.840.1.113883.6.238\",\n",
       "                  \"code\": \"2106-3\",\n",
       "                  \"display\": \"White\"\n",
       "               }\n",
       "            }\n",
       "         ],\n",
       "         \"url\": \"http://hl7.org/fhir/us/core/StructureDefinition/us-core-race\"\n",
       "      },\n",
       "      {\n",
       "         \"extension\": [\n",
       "            {\n",
       "               \"url\": \"text\",\n",
       "               \"valueString\": \"Not Hispanic or Latino\"\n",
       "            },\n",
       "            {\n",
       "               \"url\": \"ombCategory\",\n",
       "               \"valueCoding\": {\n",
       "                  \"system\": \"urn:oid:2.16.840.1.113883.6.238\",\n",
       "                  \"code\": \"2186-5\",\n",
       "                  \"display\": \"Not Hispanic or Latino\"\n",
       "               }\n",
       "            }\n",
       "         ],\n",
       "         \"url\": \"http://hl7.org/fhir/us/core/StructureDefinition/us-core-ethnicity\"\n",
       "      }\n",
       "   ],\n",
       "   \"identifier\": [\n",
       "      {\n",
       "         \"system\": \"http://example.org/pids\",\n",
       "         \"value\": \"1234\"\n",
       "      }\n",
       "   ],\n",
       "   \"active\": true,\n",
       "   \"name\": [\n",
       "      {\n",
       "         \"family\": \"Doe\",\n",
       "         \"given\": [\n",
       "            \"John\",\n",
       "            \"M\"\n",
       "         ]\n",
       "      }\n",
       "   ],\n",
       "   \"gender\": \"male\",\n",
       "   \"birthDate\": \"1964-06-19\",\n",
       "   \"address\": [\n",
       "      {\n",
       "         \"text\": \"100 Main St\\nAnytown, CA 99999\\nUSA\",\n",
       "         \"line\": [\n",
       "            \"100 Main St\"\n",
       "         ],\n",
       "         \"city\": \"Anytown\",\n",
       "         \"state\": \"CA\",\n",
       "         \"postalCode\": \"99999\",\n",
       "         \"country\": \"USA\"\n",
       "      }\n",
       "   ],\n",
       "   \"communication\": [\n",
       "      {\n",
       "         \"language\": {\n",
       "            \"coding\": [\n",
       "               {\n",
       "                  \"system\": \"urn:ietf:bcp:47\",\n",
       "                  \"code\": \"en\",\n",
       "                  \"display\": \"English\"\n",
       "               }\n",
       "            ],\n",
       "            \"text\": \"English\"\n",
       "         }\n",
       "      }\n",
       "   ]\n",
       "}</code>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# *********************** PUT Resource on server********************************\n",
    "\n",
    "#   r = requests.post('https://httpbin.org/post', data = {'key':'value'})\n",
    "r = put(f'{fhir_test_server}/Patient/{patient.id}', headers = headers, data = dumps(patient.as_json()))\n",
    "\n",
    "display(Markdown(f'<h1>Put Response</h1><p><strong>Transaction Status</strong> = {r.status_code}</p>{r.json()[\"text\"][\"div\"]}<p>raw output :</p><code>{dumps(r.json(),indent=3)}</code>'))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### GET Resource from FHIR Server in XML format\n",
    "\n",
    "Using the FHIR RESTful `GET` transaction, fetch the resource from the FHIR test server. The results are displayed below  human  xml."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "<h1>Get Resource as XML</h1><p><strong>Transaction Status</strong> = 200</p>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n",
      "<Patient xmlns=\"http://hl7.org/fhir\">\n",
      "  <id value=\"subject1\"/>\n",
      "  <meta>\n",
      "    <versionId value=\"5\"/>\n",
      "    <lastUpdated value=\"2019-05-08T16:50:12.531Z\"/>\n",
      "    <profile value=\"http://hl7.org/fhir/us/core/StructureDefinition/us-core-patient|4.0.0\"/>\n",
      "    <profile value=\"http://hl7.org/fhir/us/core/StructureDefinition/us-core-patient\"/>\n",
      "  </meta>\n",
      "  <text>\n",
      "    <status value=\"generated\"/>\n",
      "    <div xmlns=\"http://www.w3.org/1999/xhtml\">\n",
      "      <p>\n",
      "        <b>Generated Narrative with Details</b>\n",
      "      </p>\n",
      "      <p>\n",
      "        <b style=\"color: maroon\">Exception generating Narrative: unknown extension http://hl7.org/fhir/us/core/StructureDefinition/us-core-birthsex</b>\n",
      "      </p>\n",
      "    </div>\n",
      "  </text>\n",
      "  <extension url=\"http://hl7.org/fhir/us/core/StructureDefinition/us-core-birthsex\">\n",
      "    <valueCode value=\"M\"/>\n",
      "  </extension>\n",
      "  <extension url=\"http://hl7.org/fhir/us/core/StructureDefinition/us-core-race\">\n",
      "    <extension url=\"text\">\n",
      "      <valueString value=\"White\"/>\n",
      "    </extension>\n",
      "    <extension url=\"ombCategory\">\n",
      "      <valueCoding>\n",
      "        <system value=\"urn:oid:2.16.840.1.113883.6.238\"/>\n",
      "        <code value=\"2106-3\"/>\n",
      "        <display value=\"White\"/>\n",
      "      </valueCoding>\n",
      "    </extension>\n",
      "  </extension>\n",
      "  <extension url=\"http://hl7.org/fhir/us/core/StructureDefinition/us-core-ethnicity\">\n",
      "    <extension url=\"text\">\n",
      "      <valueString value=\"Not Hispanic or Latino\"/>\n",
      "    </extension>\n",
      "    <extension url=\"ombCategory\">\n",
      "      <valueCoding>\n",
      "        <system value=\"urn:oid:2.16.840.1.113883.6.238\"/>\n",
      "        <code value=\"2186-5\"/>\n",
      "        <display value=\"Not Hispanic or Latino\"/>\n",
      "      </valueCoding>\n",
      "    </extension>\n",
      "  </extension>\n",
      "  <identifier>\n",
      "    <system value=\"http://example.org/pids\"/>\n",
      "    <value value=\"1234\"/>\n",
      "  </identifier>\n",
      "  <active value=\"true\"/>\n",
      "  <name>\n",
      "    <family value=\"Doe\"/>\n",
      "    <given value=\"John\"/>\n",
      "    <given value=\"M\"/>\n",
      "  </name>\n",
      "  <gender value=\"male\"/>\n",
      "  <birthDate value=\"1964-06-19\"/>\n",
      "  <address>\n",
      "    <text value=\"100 Main St&#10;Anytown, CA 99999&#10;USA\"/>\n",
      "    <line value=\"100 Main St\"/>\n",
      "    <city value=\"Anytown\"/>\n",
      "    <state value=\"CA\"/>\n",
      "    <postalCode value=\"99999\"/>\n",
      "    <country value=\"USA\"/>\n",
      "  </address>\n",
      "  <communication>\n",
      "    <language>\n",
      "      <coding>\n",
      "        <system value=\"urn:ietf:bcp:47\"/>\n",
      "        <code value=\"en\"/>\n",
      "        <display value=\"English\"/>\n",
      "      </coding>\n",
      "      <text value=\"English\"/>\n",
      "    </language>\n",
      "  </communication>\n",
      "</Patient>\n"
     ]
    }
   ],
   "source": [
    "# *********************** GET Resource as xml from server********************************\n",
    "headers_xml = {\n",
    "'Accept':'application/fhir+xml',\n",
    "'Content-Type':'application/fhir+xml'\n",
    "}\n",
    "params_xml =  {\n",
    "'_pretty':'true'    \n",
    "}\n",
    "#   r = requests.post('https://httpbin.org/post', data = {'key':'value'})\n",
    "r = get(f'{fhir_test_server}/Patient/{patient.id}', headers = headers_xml, params = params_xml)\n",
    "\n",
    "display(Markdown(f'<h1>Get Resource as XML</h1><p><strong>Transaction Status</strong> = {r.status_code}</p>'))\n",
    "print(r.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Extending this example\n",
    "\n",
    "There are several ways to improve the above code:\n",
    "\n",
    "For example\n",
    "\n",
    "- Getting data from remote source\n",
    "- Using a terminology server for code display and lookup\n",
    "- Checking for missing data\n",
    "- adding capability for repeating elements\n",
    "- refactoring code to make more compact and reusable"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## FIN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}