examples/getting_started/getting_started_classify_taxonomy_api.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Example notebook to get started with the taxonomy API\n",
    "## Overview\n",
    "The taxonomy API can be used to classify and categorize any text into a predefined taxonomy or categories.  \n",
    "It is a powerful API that helps you to resolve any classification problem in which you need to match an textual item to a category.\n",
    "\n",
    "## Objective\n",
    "In this tutorial, you will learn how to use the taxonomy API of <a href=\"https://howsustainabledataservices.com/\" target=\"_blank\">How Sustainable Data Services</a>.\n",
    "\n",
    "## Dataset\n",
    "The dataset you will use in this notebook consists out of dummy data that is available in the code cells.\n",
    "\n",
    "## API documentation\n",
    "API documentation can be found at: <a href=\"https://api.howsustainabledataservices.com/docs\" target=\"_blank\">How Sustainable Data Services - API docs</a>.\n",
    "\n",
    "## Questions?\n",
    "Contact us at: <a href=\"https://howsustainabledataservices.com/contact/\" target=\"_blank\">How Sustainable Data Services - Contact</a>"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Installation\n",
    "Install the packages required for executing this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Defaulting to user installation because normal site-packages is not writeable\n",
      "Requirement already satisfied: pandas in c:\\users\\freek\\appdata\\roaming\\python\\python311\\site-packages (1.5.3)\n",
      "Requirement already satisfied: python-dateutil>=2.8.1 in c:\\users\\freek\\appdata\\roaming\\python\\python311\\site-packages (from pandas) (2.8.2)\n",
      "Requirement already satisfied: pytz>=2020.1 in c:\\users\\freek\\appdata\\roaming\\python\\python311\\site-packages (from pandas) (2023.3)\n",
      "Requirement already satisfied: numpy>=1.21.0 in c:\\users\\freek\\appdata\\roaming\\python\\python311\\site-packages (from pandas) (1.24.3)\n",
      "Requirement already satisfied: six>=1.5 in c:\\users\\freek\\appdata\\roaming\\python\\python311\\site-packages (from python-dateutil>=2.8.1->pandas) (1.16.0)\n",
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "%pip install pandas"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import dependencies\n",
    "Import the dependencies and define the API server URL"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "import pandas as pd\n",
    "from requests.exceptions import HTTPError, JSONDecodeError\n",
    "import json\n",
    "\n",
    "API_SERVER = \"https://api.howsustainabledataservices.com/latest\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Obtain API token\n",
    "Before we can use the API, you need to obtain a free API token at: <a href=\"https://howsustainabledataservices.com/auth/register/\" target=\"_blank\">How Sustainable Data Services - Register for API</a>.  \n",
    "You can simply fill in the form on the website and you will directly receive an API token.  \n",
    "Once you have received this token, you can paste it in the cell below. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "API_TOKEN = \"INSERT_YOUR_API_TOKEN_HERE\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define helper functions\n",
    "To call the API we need to define a helper function that makes a POST requests to the API endpoint that is authorized using the API token "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Helper function to call the API\n",
    "def make_api_call(endpoint, json_body):\n",
    "    url = API_SERVER + endpoint\n",
    "    headers = {\"Authorization\": f\"Bearer {API_TOKEN}\"}\n",
    "\n",
    "    try:\n",
    "        response = requests.post(url, json=json_body, headers=headers)\n",
    "        response.raise_for_status()  # Raise an exception for non-2xx status codes\n",
    "        result = response.json()\n",
    "        return result\n",
    "    except HTTPError as http_err:\n",
    "        print(f\"HTTP error occurred: {http_err}\")\n",
    "    except JSONDecodeError as json_err:\n",
    "        print(f\"JSON decoding error occurred: {json_err}\")\n",
    "    except Exception as err:\n",
    "        print(f\"An error occurred: {err}\")\n",
    "\n",
    "    return None"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Upload your categories to the API\n",
    "The simplest form of a taxonomy is a flat taxonomy, all the items or categories are listed at the same hierarchical level without any further subcategories or levels of nesting.  \n",
    "When you have a list of categories this is a flat taxonomy, which can be uploaded as a list to the API.  \n",
    "In our example we have two categories that are uploaded to the API as a list.  \n",
    "When uploaded successfully the API will return an taxonomy identifier, this identifier will be used in the rest of the notebook to reference back to our uploaded taxonomy.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Identifier for your uploaded taxonomy: NQBFzDvigBSg\n"
     ]
    }
   ],
   "source": [
    "your_category_list = [\"Renewable energy\", \"Non-renewable energy\"] # replace with a list of your categories\n",
    "\n",
    "def upload_category_list(category_list):\n",
    "    endpoint = \"/taxonomies/upload_list\"\n",
    "    response = make_api_call(endpoint, category_list)\n",
    "    taxonomy_id = response[\"taxonomy_id\"]\n",
    "\n",
    "    return taxonomy_id\n",
    "\n",
    "taxonomy_id = upload_category_list(your_category_list)\n",
    "print(\"Identifier for your uploaded taxonomy:\", taxonomy_id)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Classify items using the API\n",
    "Now that the API knows our taxonomy, we can ask it to classify our items.  \n",
    "In this example we want to classify each item (in our case: each energy source) into our flat taxonomy.  \n",
    "The API will return for each item a dictionary with the classification results.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'gpt_product_name': 'Solar panels',\n",
       " 'gpt_product_category': 'Renewable energy',\n",
       " 'gpt_confidence': 1.0,\n",
       " 'gpt_notebook_version': 'v1'}"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "your_item_list = [\"Solar panels\", \"Wind turbines\", \"Hydropower plant\", \"Coal power plant\", \"Natural gas power plant\"] # replace with a list of items that you want to classify into categories, in this case we use products\n",
    "\n",
    "def classify_items(taxonomy_id, product_list):\n",
    "    endpoint = f\"/taxonomies/{taxonomy_id}/classify_items_list\"\n",
    "    response = make_api_call(endpoint, product_list)\n",
    "\n",
    "    return response\n",
    "\n",
    "results = classify_items(taxonomy_id, your_item_list)\n",
    "results[0]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Check results\n",
    "These dictionaries can easily be transformed into a Pandas DataFrame, to check the results.  \n",
    "The column 'product_category' contains the predicted result of the API"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>gpt_product_name</th>\n",
       "      <th>gpt_product_category</th>\n",
       "      <th>gpt_confidence</th>\n",
       "      <th>gpt_notebook_version</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Solar panels</td>\n",
       "      <td>Renewable energy</td>\n",
       "      <td>1.0</td>\n",
       "      <td>v1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Wind turbines</td>\n",
       "      <td>Renewable energy</td>\n",
       "      <td>1.0</td>\n",
       "      <td>v1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Hydropower plant</td>\n",
       "      <td>Renewable energy</td>\n",
       "      <td>1.0</td>\n",
       "      <td>v1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Coal power plant</td>\n",
       "      <td>Non-renewable energy</td>\n",
       "      <td>1.0</td>\n",
       "      <td>v1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Natural gas power plant</td>\n",
       "      <td>Non-renewable energy</td>\n",
       "      <td>1.0</td>\n",
       "      <td>v1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          gpt_product_name  gpt_product_category  gpt_confidence  \\\n",
       "0             Solar panels      Renewable energy             1.0   \n",
       "1            Wind turbines      Renewable energy             1.0   \n",
       "2         Hydropower plant      Renewable energy             1.0   \n",
       "3         Coal power plant  Non-renewable energy             1.0   \n",
       "4  Natural gas power plant  Non-renewable energy             1.0   \n",
       "\n",
       "  gpt_notebook_version  \n",
       "0                   v1  \n",
       "1                   v1  \n",
       "2                   v1  \n",
       "3                   v1  \n",
       "4                   v1  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame(results)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Upload a hierarchical taxonomy to the API\n",
    "If you have a hierarchical taxonomy that consists out of multiple levels, this is supported by the API as well.  \n",
    "You can upload this taxonomy as a JSON to the API, which should adhere to same schema as the example below.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Identifier for your uploaded taxonomy: 7EhOSwN9V19S\n"
     ]
    }
   ],
   "source": [
    "def upload_taxonomy_json(taxonomy_json):\n",
    "    endpoint = \"/taxonomy/taxonomies/upload_json\"\n",
    "    response = make_api_call(endpoint, taxonomy_json)\n",
    "    taxonomy_id = response[\"taxonomy_id\"]\n",
    "\n",
    "    return taxonomy_id\n",
    "\n",
    "your_taxonomy_json = {\n",
    "  \"categories\": [\n",
    "    {\n",
    "      \"category_name\": \"Renewable energy\",\n",
    "      \"category_description\": \"Power generation based on renewable energy sources\",\n",
    "      \"subcategories\": [\n",
    "        {\n",
    "          \"category_name\": \"Wind energy\",\n",
    "          \"category_description\": \"Power generation based on wind\",\n",
    "          \"subcategories\": []\n",
    "        },\n",
    "        {\n",
    "          \"category_name\": \"Solar energy\",\n",
    "          \"category_description\": \"Power generation based on sun\",\n",
    "          \"subcategories\": []\n",
    "        }\n",
    "      ]\n",
    "    },\n",
    "    {\n",
    "      \"category_name\": \"Fossil fuel based energy\t\",\n",
    "      \"category_description\": \"Power generation based on fossil fuel energy sources\",\n",
    "    }\n",
    "  ]\n",
    "}\n",
    "\n",
    "\n",
    "taxonomy_id = upload_category_list(your_category_list)\n",
    "print(\"Identifier for your uploaded taxonomy:\", taxonomy_id)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Check results of hierarchical taxonomy\n",
    "As you can see the \"product_category\" column now also contains categories which where subcategories in the taxonomy.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>gpt_product_name</th>\n",
       "      <th>gpt_product_category</th>\n",
       "      <th>gpt_confidence</th>\n",
       "      <th>gpt_notebook_version</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Solar panels</td>\n",
       "      <td>Solar energy</td>\n",
       "      <td>1.0</td>\n",
       "      <td>v1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Wind turbines</td>\n",
       "      <td>Wind energy</td>\n",
       "      <td>1.0</td>\n",
       "      <td>v1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Hydropower plant</td>\n",
       "      <td>Renewable energy</td>\n",
       "      <td>1.0</td>\n",
       "      <td>v1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Coal power plant</td>\n",
       "      <td>Non-renewable energy</td>\n",
       "      <td>1.0</td>\n",
       "      <td>v1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Natural gas power plant</td>\n",
       "      <td>Non-renewable energy</td>\n",
       "      <td>1.0</td>\n",
       "      <td>v1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          gpt_product_name  gpt_product_category  gpt_confidence  \\\n",
       "0             Solar panels      Renewable energy             1.0   \n",
       "1            Wind turbines      Renewable energy             1.0   \n",
       "2         Hydropower plant      Renewable energy             1.0   \n",
       "3         Coal power plant  Non-renewable energy             1.0   \n",
       "4  Natural gas power plant  Non-renewable energy             1.0   \n",
       "\n",
       "  gpt_notebook_version  \n",
       "0                   v1  \n",
       "1                   v1  \n",
       "2                   v1  \n",
       "3                   v1  \n",
       "4                   v1  "
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results = classify_items(taxonomy_id, your_item_list) # same item list as before, based on new taxonomy\n",
    "pd.DataFrame(results)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.3"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}