examples/load-data-via-graph-construction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KhH0v_bwUrXp"
   },
   "source": [
    "# Graph construct: Import from Pandas"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "view-in-github"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/neo4j/graph-data-science-client/blob/main/examples/load-data-via-graph-construction.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This Jupyter notebook is hosted [here](https://github.com/neo4j/graph-data-science-client/blob/main/examples/load-data-via-graph-construction.ipynb) in the Neo4j Graph Data Science Client Github repository.\n",
    "\n",
    "The notebook shows the usage of the `gds.graph.construct` method (available only in GDS 2.1+) to build a graph directly in memory.\n",
    "\n",
    "NOTE: If you are using AuraDS, it is currently not possible to write the projected graph back to Neo4j."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "X5ttqidnWPTC"
   },
   "source": [
    "## Setup\n",
    "\n",
    "We need an environment where Neo4j and GDS are available, for example AuraDS (which comes with GDS preinstalled) or Neo4j Desktop. \n",
    "\n",
    "Once the credentials to this environment are available, we can install the `graphdatascience` package and import the client class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install graphdatascience"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from graphdatascience import GraphDataScience"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When using a local Neo4j setup, the default connection URI is `bolt://localhost:7687`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get Neo4j DB URI, credentials and name from environment if applicable\n",
    "NEO4J_URI = os.environ.get(\"NEO4J_URI\", \"bolt://localhost:7687\")\n",
    "NEO4J_AUTH = None\n",
    "NEO4J_DB = os.environ.get(\"NEO4J_DB\", \"neo4j\")\n",
    "if os.environ.get(\"NEO4J_USER\") and os.environ.get(\"NEO4J_PASSWORD\"):\n",
    "    NEO4J_AUTH = (\n",
    "        os.environ.get(\"NEO4J_USER\"),\n",
    "        os.environ.get(\"NEO4J_PASSWORD\"),\n",
    "    )\n",
    "gds = GraphDataScience(NEO4J_URI, auth=NEO4J_AUTH, database=NEO4J_DB)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When using AuraDS, the connection URI is slightly different as it uses the `neo4j+s` protocol. The client should also include the `aura_ds=True` flag to enable AuraDS-recommended settings.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# On AuraDS:\n",
    "#\n",
    "# gds = GraphDataScience(NEO4J_URI, auth=NEO4J_AUTH, database=NEO4J_DB, aura_ds=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "verify-version"
    ]
   },
   "outputs": [],
   "source": [
    "from graphdatascience.server_version.server_version import ServerVersion\n",
    "\n",
    "assert gds.server_version() >= ServerVersion(2, 1, 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zOK08QpoWFKv"
   },
   "source": [
    "We also import `pandas` to create a Pandas `DataFrame` from the original data source."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "oDE2LqTXAmy8"
   },
   "source": [
    "## Load the Cora dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "CORA_CONTENT = \"https://data.neo4j.com/cora/cora.content\"\n",
    "CORA_CITES = \"https://data.neo4j.com/cora/cora.cites\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can load each CSV locally as a Pandas `DataFrame`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "content = pd.read_csv(CORA_CONTENT, header=None)\n",
    "cites = pd.read_csv(CORA_CITES, header=None)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4Y6DVaoDNNdq"
   },
   "source": [
    "We need to perform an additional preprocessing step to convert the `subject` field (which is a string in the dataset) into an integer, because node properties have to be numerical in order to be projected into a graph. We can use a map for this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "SUBJECT_TO_ID = {\n",
    "    \"Neural_Networks\": 0,\n",
    "    \"Rule_Learning\": 1,\n",
    "    \"Reinforcement_Learning\": 2,\n",
    "    \"Probabilistic_Methods\": 3,\n",
    "    \"Theory\": 4,\n",
    "    \"Genetic_Algorithms\": 5,\n",
    "    \"Case_Based\": 6,\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now create a new `DataFrame` with a `nodeId` field, a list of node labels,\n",
    "and the additional node properties `subject` (using the `SUBJECT_TO_ID` \n",
    "mapping) and `features` (converting all the feature columns to a single\n",
    "array column)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "nodes = pd.DataFrame().assign(\n",
    "    nodeId=content[0],\n",
    "    labels=\"Paper\",\n",
    "    subject=content[1].replace(SUBJECT_TO_ID),\n",
    "    features=content.iloc[:, 2:].apply(list, axis=1),\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's check the first 5 rows of the new `DataFrame`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "nodes.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we create a new `DataFrame` containing the relationships between the nodes.\n",
    "To create the equivalent of an undirected graph, we need to add direct\n",
    "and inverse relationships explicitly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dir_relationships = pd.DataFrame().assign(sourceNodeId=cites[0], targetNodeId=cites[1], relationshipType=\"CITES\")\n",
    "inv_relationships = pd.DataFrame().assign(sourceNodeId=cites[1], targetNodeId=cites[0], relationshipType=\"CITES\")\n",
    "\n",
    "relationships = pd.concat([dir_relationships, inv_relationships]).drop_duplicates()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Again, let's check the first 5 rows of the new `DataFrame`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "relationships.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we can create the in-memory graph."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "G = gds.graph.construct(\"cora-graph\", nodes, relationships)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Use the graph"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's check that the new graph has been created:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gds.graph.list()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's also count the nodes in the graph:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "G.node_count()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The count matches with the number of rows in the Pandas dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "len(content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can stream the value of the `subject` node property for\n",
    "each node in the graph, printing only the first 10."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gds.graph.nodeProperties.stream(G, [\"subject\"]).head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "plRCiikGOofd"
   },
   "source": [
    "## Cleanup\n",
    "\n",
    "When the graph is no longer needed, it should be dropped to free up memory:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "G.drop()"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}