OpenAlex4NodeXL.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "gpuType": "T4",
      "authorship_tag": "ABX9TyOr6+1A0YttlfyOh/znVYgI",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "ir",
      "display_name": "R"
    },
    "language_info": {
      "name": "R"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/Ifeanyi55/OpenAlex4NodeXL/blob/main/OpenAlex4NodeXL.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Introduction to OpenAlex4NodeXL**\n",
        "\n",
        "The OpenAlex4NodeXL project is a project that allows a user to fetch network data from openalex.org via its API in a format that can be consumed by NodeXL. The network data is a bimodal network of **authors to publications**. The program outputs a csv flat file that can be downloaded to your local machine and then imported into NodeXL for visualization and analysis."
      ],
      "metadata": {
        "id": "Z0PB5dutnhqo"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## **Install Package**"
      ],
      "metadata": {
        "id": "ty7iZwV1osbd"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Install openalexR package by running this code cell. Click the play button to run the cell or CTRL + Enter\n",
        "install.packages(\"openalexR\")"
      ],
      "metadata": {
        "id": "O5mgOwzoniej"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## **Keywords**\n",
        "If you are entering more than one keyword in the search parameter below, make sure to separate them by a comma.\n",
        "\n",
        "## **Dates**\n",
        "Dates inputs are always in the **yyyy-mm-dd** format.\n",
        "\n",
        "## **Run Code**\n",
        "To run the code after entering your search parameters, press the play button on the left of the code cell.\n"
      ],
      "metadata": {
        "id": "F2qOJqQko3pw"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## **Enter Search Parameters & Run Cell to Generate Author-To-Publication Network Data As CSV Flat File**"
      ],
      "metadata": {
        "id": "O2BMTt5XpYWL"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# software code\n",
        "OpenAlex4NodeXL <- function(keywords, pub_start_date, pub_end_date) {\n",
        "\n",
        "  keywords <- keywords\n",
        "  pub_start_date <- pub_start_date\n",
        "  pub_end_date <- pub_end_date\n",
        "\n",
        "  # create search engine function\n",
        "  search_engine <- function(keywords, pub_start_date, pub_end_date) {\n",
        "    # load software libraries\n",
        "    suppressPackageStartupMessages(library(openalexR))\n",
        "    suppressPackageStartupMessages(library(tidyverse))\n",
        "\n",
        "    # set options\n",
        "    options(openalexR.mailto = \"youremail@email.com\") # replace with your email address\n",
        "\n",
        "    # search engine\n",
        "    works_search <- oa_fetch(\n",
        "      entity = \"works\",\n",
        "      title.search = c(keywords),\n",
        "      cited_by_count = \">50\",\n",
        "      from_publication_date = pub_start_date,\n",
        "      to_publication_date = pub_end_date,\n",
        "      options = list(sort = \"cited_by_count:desc\"),\n",
        "      verbose = FALSE\n",
        "    )\n",
        "\n",
        "    return(works_search)\n",
        "  }\n",
        "\n",
        "  # fetch data from openalex.org api\n",
        "  search_data <- search_engine(keywords, pub_start_date, pub_end_date)\n",
        "\n",
        "  # grab authors and group them according to collaboration\n",
        "  authors_collaboration_groups <- list()\n",
        "  for (i in 1:nrow(search_data)) {\n",
        "    authors_collaboration_groups[[i]] <- search_data$author[[i]][2]\n",
        "  }\n",
        "\n",
        "  all_authors <- c()\n",
        "  for (i in 1:length(authors_collaboration_groups)) {\n",
        "    all_authors <- c(all_authors, authors_collaboration_groups[[i]][[1]])\n",
        "  }\n",
        "\n",
        "  # grab author position\n",
        "  authors_position <- list()\n",
        "  for (i in 1:nrow(search_data)) {\n",
        "    authors_position[[i]] <- search_data$author[[i]][4]\n",
        "  }\n",
        "\n",
        "  all_authors_positions <- c() # grab all authors positions\n",
        "  for (i in 1:length(authors_position)) {\n",
        "    all_authors_positions <- c(all_authors_positions, authors_position[[i]][[1]])\n",
        "  }\n",
        "\n",
        "  # grab author affiliation\n",
        "  authors_affiliation <- list()\n",
        "  for (i in 1:nrow(search_data)) {\n",
        "    authors_affiliation[[i]] <- search_data$author[[i]][7]\n",
        "  }\n",
        "\n",
        "  all_authors_affiliation <- c() # grab all authors affiliations\n",
        "  for (i in 1:length(authors_affiliation)) {\n",
        "    all_authors_affiliation <- c(all_authors_affiliation, authors_affiliation[[i]][[1]])\n",
        "  }\n",
        "\n",
        "  # grab authors institution country code\n",
        "  authors_institution_country_code <- list()\n",
        "  for (i in 1:nrow(search_data)) {\n",
        "    authors_institution_country_code[[i]] <- search_data$author[[i]][9]\n",
        "  }\n",
        "\n",
        "\n",
        "  all_authors_institution_country_code <- c() # grab all authors institution country code\n",
        "  for (i in 1:length(authors_institution_country_code)) {\n",
        "    all_authors_institution_country_code <- c(all_authors_institution_country_code, authors_institution_country_code[[i]][[1]])\n",
        "  }\n",
        "\n",
        "  # grab author institution type\n",
        "  authors_institution_type <- list()\n",
        "  for (i in 1:nrow(search_data)) {\n",
        "    authors_institution_type[[i]] <- search_data$author[[i]][10]\n",
        "  }\n",
        "\n",
        "\n",
        "  all_authors_institution_type <- c() # grab all authors institution type\n",
        "  for (i in 1:length(authors_institution_type)) {\n",
        "    all_authors_institution_type <- c(all_authors_institution_type, authors_institution_type[[i]][[1]])\n",
        "  }\n",
        "\n",
        "  # get length of each authors collaboration\n",
        "  authors_length <- c()\n",
        "  for (authors in 1:length(authors_collaboration_groups)) {\n",
        "    authors_length <- c(authors_length, authors_collaboration_groups[[authors]] |> nrow())\n",
        "  }\n",
        "\n",
        "\n",
        "  # create authors data frame\n",
        "  authorAtt_df <- data.frame(\n",
        "    Authors = all_authors,\n",
        "    Position = all_authors_positions,\n",
        "    Affiliation = all_authors_affiliation,\n",
        "    Institution = all_authors_institution_type\n",
        "  )\n",
        "\n",
        "  # I did not want to have to use underscore to separate\n",
        "  # the two words (Institution_Country). That is why I\n",
        "  # created that column in the data frame using back ticks\n",
        "  # instead as shown below\n",
        "  authorAtt_df$`Institution Country` <- all_authors_institution_country_code\n",
        "\n",
        "  # publication attributes\n",
        "  # grab all publications\n",
        "\n",
        "  publications <- list()\n",
        "  for (i in 1:nrow(search_data)) {\n",
        "    publications[[i]] <- rep(search_data$display_name[i], each = authors_length[i])\n",
        "  }\n",
        "\n",
        "  all_publications <- c()\n",
        "  for (i in 1:length(publications)) {\n",
        "    all_publications <- c(all_publications, publications[[i]])\n",
        "  }\n",
        "\n",
        "  # grab all so\n",
        "  pub_so <- list()\n",
        "  for (i in 1:nrow(search_data)) {\n",
        "    pub_so[[i]] <- rep(search_data$so[i], each = authors_length[i])\n",
        "  }\n",
        "\n",
        "  all_so <- c()\n",
        "  for (i in 1:length(pub_so)) {\n",
        "    all_so <- c(all_so, pub_so[[i]])\n",
        "  }\n",
        "\n",
        "  # grab all host organization\n",
        "  hostOrg <- list()\n",
        "  for (i in 1:nrow(search_data)) {\n",
        "    hostOrg[[i]] <- rep(search_data$host_organization[i], each = authors_length[i])\n",
        "  }\n",
        "\n",
        "  all_hostOrg <- c()\n",
        "  for (i in 1:length(hostOrg)) {\n",
        "    all_hostOrg <- c(all_hostOrg, hostOrg[[i]])\n",
        "  }\n",
        "\n",
        "  # grab all cited by count\n",
        "  citedby_count <- list()\n",
        "  for (i in 1:nrow(search_data)) {\n",
        "    citedby_count[[i]] <- rep(search_data$cited_by_count[i], each = authors_length[i])\n",
        "  }\n",
        "\n",
        "  all_citedby_count <- c()\n",
        "  for (i in 1:length(citedby_count)) {\n",
        "    all_citedby_count <- c(all_citedby_count, citedby_count[[i]])\n",
        "  }\n",
        "\n",
        "  # grab all publication year\n",
        "  pub_year <- list()\n",
        "  for (i in 1:nrow(search_data)) {\n",
        "    pub_year[[i]] <- rep(search_data$publication_year[i], each = authors_length[i])\n",
        "  }\n",
        "\n",
        "  all_pub_year <- c()\n",
        "  for (i in 1:length(citedby_count)) {\n",
        "    all_pub_year <- c(all_pub_year, pub_year[[i]])\n",
        "  }\n",
        "\n",
        "  # grab all type\n",
        "  type <- list()\n",
        "  for (i in 1:nrow(search_data)) {\n",
        "    type[[i]] <- rep(search_data$type[i], each = authors_length[i])\n",
        "  }\n",
        "\n",
        "  all_type <- c()\n",
        "  for (i in 1:length(type)) {\n",
        "    all_type <- c(all_type, type[[i]])\n",
        "  }\n",
        "\n",
        "  # grab all abstract\n",
        "  abstract <- list()\n",
        "  for (i in 1:nrow(search_data)) {\n",
        "    abstract[[i]] <- rep(search_data$ab[i], each = authors_length[i])\n",
        "  }\n",
        "\n",
        "  all_abstracts <- c()\n",
        "  for (i in 1:length(abstract)) {\n",
        "    all_abstracts <- c(all_abstracts, abstract[[i]])\n",
        "  }\n",
        "\n",
        "  # grab all referenced works\n",
        "  referenced <- list()\n",
        "  for (i in 1:nrow(search_data)) {\n",
        "    referenced[[i]] <- rep(search_data$referenced_works[i], each = authors_length[i])\n",
        "  }\n",
        "\n",
        "  all_referenced <- c()\n",
        "  for (i in 1:length(referenced)) {\n",
        "    all_referenced <- c(all_referenced, referenced[[i]])\n",
        "  }\n",
        "\n",
        "  # update the authors data frame\n",
        "  {\n",
        "    authorAtt_df$Publication <- all_publications\n",
        "    authorAtt_df$`Abstract` <- all_abstracts\n",
        "    authorAtt_df$`Publication Type` <- all_type\n",
        "    authorAtt_df$`Publication Year` <- all_pub_year\n",
        "    authorAtt_df$`Cited By Count` <- all_citedby_count\n",
        "    authorAtt_df$`Referenced Works` <- all_referenced\n",
        "    authorAtt_df$`Host Organization` <- all_hostOrg\n",
        "    authorAtt_df$SO <- all_so\n",
        "  }\n",
        "\n",
        "\n",
        "  # filter out missing values from the data frame\n",
        "  authorAtt_df <- authorAtt_df |>\n",
        "    na.omit()\n",
        "\n",
        "  # move abstract column to behind Publication\n",
        "  authorAtt_df <- authorAtt_df |>\n",
        "    relocate(Abstract, .after = Publication)\n",
        "\n",
        "  # rearrange columns for NodeXL flat file csv format\n",
        "  authorAtt_df <- authorAtt_df |>\n",
        "    relocate(Publication, .after = Authors)\n",
        "\n",
        "\n",
        "  # rename columns\n",
        "  colnames(authorAtt_df)[c(1:13)] <- c(\n",
        "    \"Vertex1\",\n",
        "    \"Vertex2\",\n",
        "    \"Vertex1 Position\",\n",
        "    \"Vertex1 Affiliation\",\n",
        "    \"Vertex1 Institution\",\n",
        "    \"Vertex1 Institution Country\",\n",
        "    \"Vertex2 Abstract\",\n",
        "    \"Vertex2 Type\",\n",
        "    \"Vertex2 Publication Year\",\n",
        "    \"Vertex2 Cited By Count\",\n",
        "    \"Vertex2 Referenced Works\",\n",
        "    \"Vertex2 Host Organization\",\n",
        "    \"Vertex2 SO\"\n",
        "  )\n",
        "\n",
        "  list2vec <- function(x){\n",
        "    paste(x,collapse = \" \")\n",
        "  }\n",
        "  # convert list column into character column\n",
        "  authorAtt_df$`Vertex2 Referenced Works` <- sapply(authorAtt_df$`Vertex2 Referenced Works`,list2vec)\n",
        "\n",
        "\n",
        "  return(authorAtt_df)\n",
        "\n",
        "\n",
        "\n",
        "}\n",
        "\n",
        "\n",
        "Keywords = \"medicine, health, grants\" # @param {type:\"string\"}\n",
        "Pub_start_date = \"2020-02-01\" # @param {type:\"date\"}\n",
        "Pub_end_date = \"2022-02-28\" # @param {type:\"date\"}\n",
        "\n",
        "Keywords <- c(unlist(strsplit(Keywords,split = \",\")))\n",
        "\n",
        "network_data <- openAlex4NodeXL(\n",
        "  keywords = Keywords,\n",
        "  pub_start_date = Pub_start_date,\n",
        "  pub_end_date = Pub_end_date\n",
        ")\n",
        "\n",
        "# export csv flat file\n",
        "write.csv(network_data,file = \"author2pub.csv\",row.names = F)\n",
        "\n"
      ],
      "metadata": {
        "cellView": "form",
        "id": "FW_XxPQvpYwB"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "To download the file, click the **Files** folder icon in the sidebar on the left of the notebook. There, you will see the generated csv file. Hover over the file and click on the 3 dots to download the file."
      ],
      "metadata": {
        "id": "AZUDwqmMrd-0"
      }
    }
  ]
}