Merge pull request #11 from wey-gu/algo_verification

feat: algo verification and docs
wey-gu · Feb 28, 2023 · def52f4 · def52f4
2 parents 217fd2e + 63fc7cb
commit def52f4
Show file tree

Hide file tree

Showing 5 changed files with 197 additions and 133 deletions.
diff --git a/README.md b/README.md
@@ -49,7 +49,7 @@ pip install ngdi
 - Spark 2.4, 3.0(not yet tested)
 - [NebulaGraph 3.4+](https://github.com/vesoft-inc/nebula)
 - [NebulaGraph Spark Connector 3.4+](https://repo1.maven.org/maven2/com/vesoft/nebula-spark-connector/)
-- [NebulaGraph Algorithm 3.4+](https://repo1.maven.org/maven2/com/vesoft/nebula-algorithm/)
+- [NebulaGraph Algorithm 3.1+](https://repo1.maven.org/maven2/com/vesoft/nebula-algorithm/)
 
 ### NebulaGraph Engine Prerequisites
 - [NebulaGraph 3.4+](https://github.com/vesoft-inc/nebula)

diff --git a/examples/spark_engine.ipynb b/examples/spark_engine.ipynb
@@ -1,8 +1,19 @@
 {
  "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "f46fdd40",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# install ngdi\n",
+    "!pip install ngdi"
+   ]
+  },
   {
    "cell_type": "markdown",
-   "id": "8d81af21",
+   "id": "5b4e4143",
    "metadata": {},
    "source": [
     "## Spark Engine Examples\n",
@@ -11,7 +22,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "8d0a7751",
+   "id": "f17abcf8",
    "metadata": {},
    "source": [
     "In this example, we are leveraging the Spark Engine of NebulaGraph DI Suite, with the Storage Scan mode.\n",
@@ -69,7 +80,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "d29ebd47",
+   "id": "3617de5f",
    "metadata": {},
    "source": [
     "#### Step 2, run Pagerank Algorithm"
@@ -78,7 +89,7 @@
   {
    "cell_type": "code",
    "execution_count": 2,
-   "id": "adeeacbe",
+   "id": "90069aaf",
    "metadata": {},
    "outputs": [
     {
@@ -138,6 +149,13 @@
       "only showing top 20 rows\n",
       "\n"
      ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "23/02/28 12:13:38 WARN BlockManager: Block rdd_75_4 already exists on this machine; not re-adding it\n"
+     ]
     }
    ],
    "source": [
@@ -146,7 +164,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "ef66c9e2",
+   "id": "66e70ca0",
    "metadata": {},
    "source": [
     "#### Step 3, check results of the algorithm\n"
@@ -165,11 +183,11 @@
       "+---------+-------------------+\n",
       "|      _id|           pagerank|\n",
       "+---------+-------------------+\n",
-      "|player133|0.18601069183310506|\n",
-      "|player126|0.18601069183310506|\n",
-      "|player130| 1.2400712788873671|\n",
-      "|player108|0.18601069183310506|\n",
-      "|player102| 1.6602373739502538|\n",
+      "|player133|0.18601069183310504|\n",
+      "|player126|0.18601069183310504|\n",
+      "|player130|  1.240071278887367|\n",
+      "|player108|0.18601069183310504|\n",
+      "|player102| 1.6602373739502536|\n",
       "+---------+-------------------+\n",
       "only showing top 5 rows\n",
       "\n"
@@ -182,7 +200,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "4b9cf5fe",
+   "id": "3eb228f8",
    "metadata": {},
    "source": [
     "------------------\n",
@@ -198,7 +216,7 @@
   {
    "cell_type": "code",
    "execution_count": 4,
-   "id": "b6542994",
+   "id": "e44ac3e9",
    "metadata": {},
    "outputs": [
     {
@@ -231,7 +249,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "d24e5903",
+   "id": "49becbdb",
    "metadata": {},
    "source": [
     "#### Step 2, run Conncted Components Algorithm"
@@ -240,7 +258,7 @@
   {
    "cell_type": "code",
    "execution_count": 5,
-   "id": "fc5dcc2c",
+   "id": "cfbcda82",
    "metadata": {},
    "outputs": [
     {
@@ -300,13 +318,6 @@
       "only showing top 20 rows\n",
       "\n"
      ]
-    },
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "23/02/27 11:29:58 WARN BlockManager: Block rdd_308_3 already exists on this machine; not re-adding it\n"
-     ]
     }
    ],
    "source": [
@@ -315,7 +326,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "88a13923",
+   "id": "38181d45",
    "metadata": {},
    "source": [
     "#### Step 3, check results of the algorithm\n"
@@ -324,7 +335,7 @@
   {
    "cell_type": "code",
    "execution_count": 6,
-   "id": "843b9662",
+   "id": "bed14375",
    "metadata": {},
    "outputs": [
     {
@@ -349,13 +360,33 @@
     "cc_result.show(5)"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "5bcb02e2",
+   "metadata": {},
+   "source": [
+    "## Other algorithm examples"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "a8d97f96",
+   "id": "ff5a866d",
    "metadata": {},
    "outputs": [],
-   "source": []
+   "source": [
+    "# lpa_result  = df.algo.label_propagation()\n",
+    "# louvain_result = df.algo.louvain()\n",
+    "# k_core_result = df.algo.k_core()\n",
+    "# degree_statics_result = df.algo.degree_statics()\n",
+    "# betweenness_centrality_result = df.algo.betweenness_centrality()\n",
+    "# coefficient_centrality_result = df.algo.coefficient_centrality()\n",
+    "# bfs_result = df.algo.bfs()\n",
+    "# hanp_result = df.algo.hanp()\n",
+    "# jaccard_result = df.algo.jaccard()\n",
+    "# strong_connected_components_result = df.algo.strong_connected_components()\n",
+    "# triangle_count_result = df.algo.triangle_count()"
+   ]
   }
  ],
  "metadata": {