net_art_revolution.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# NET ART REVOLUTION?\n",
    "\n",
    "**Abstract**\n",
    "\n",
    "The advent of the internet and personal computers as tools of communication and creation radically transformed the world we live in over the past 40 years. The impacts of both technologies across various fields and topics have been well documented, including on the art world. However, discussion in the latter field is often focused on how the internet and computers have transformed art creation or the ways in which memory institutions function and interact with their audiences. Seemingly slightly less focus has been given to how these technologies may have impacted which artists get to be included in art collections. Net Art Revolution? aims to look at this particular aspect of the digital revolution by analyzing data from two different NYC-based institutions that represent the art world before and after the internet and personal computers came into our lives: MoMA, representing the traditional art world; and Rhizome, a smaller and younger institution dedicated to digital-born art and culture, often referred to as net art. Both institutions make data about their collections publicly available -- MoMA in csv and json formats, Rhizome as Linked Open Data -- thus allowing us to peer into their potential differences and what they might mean, with the caveat that both institutions are located in one of the primary art and media centers of the Global North and thus represent a distinctive perspective on art (regardless of type). Despite the potential limitations of the datasets, we believe they are a useful starting point from which to investigate our research question: **Did the internet and personal computers have an impact on the make up of art collections?** \n",
    "\n",
    "This project was undertaken by Margherita Donelli, Chiara Catizone, and Laurent Fintoni for the Electronic Publishing and Digital Storytelling (a.y. 2021-2022) final examination, as part of the *Digital Humanities and Digital Knowledge Master's Degree* at the University of Bologna.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "#IMPORTS + PATH\n",
    "path = './'\n",
    "\n",
    "#GENERIC\n",
    "import sys\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import re\n",
    "import ssl\n",
    "ssl._create_default_https_context = ssl._create_unverified_context\n",
    "#PLOTLY\n",
    "import plotly.express as px\n",
    "import plotly.graph_objects as go\n",
    "import plotly.io as pio\n",
    "import pycountry\n",
    "import chart_studio.tools as tls\n",
    "import ipywidgets as ipw\n",
    "from statsmodels.graphics.mosaicplot import mosaic\n",
    "from plotly.subplots import make_subplots\n",
    "#PREPPING/CLEANING\n",
    "from SPARQLWrapper import SPARQLWrapper, JSON\n",
    "from functools import reduce\n",
    "from collections import defaultdict\n",
    "pio.templates.default = \"simple_white\"\n",
    "#SCRAPING\n",
    "from __future__ import print_function\n",
    "from bs4 import BeautifulSoup\n",
    "import requests\n",
    "#KEYWORDS\n",
    "import nltk\n",
    "from nltk.corpus import stopwords\n",
    "from nltk import word_tokenize \n",
    "from nltk.util import ngrams"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "#FUNCTIONS\n",
    "\n",
    "#CLEAN DATA\n",
    "def normalizeDate(year):\n",
    "    if  re.match('.*\\d\\d\\d\\d.*', str(year)):\n",
    "        newDate = re.findall(r'\\d\\d\\d\\d', year)\n",
    "        #modify the date in order to normalize it using the first year in the list \n",
    "        return int(newDate[0])\n",
    "        \n",
    "    else: \n",
    "        return 0\n",
    "\n",
    "def getGender(ids):\n",
    "    listIds = str(ids).split(', ')\n",
    "    listGenders = list()\n",
    "    for id in listIds:\n",
    "        gender = artists[artists['ID']== id]\n",
    "        if len(gender)>0:\n",
    "            gender = gender['Gender'].values[0]\n",
    "            listGenders.append(gender)\n",
    "        else:\n",
    "            listGenders.append('missing')\n",
    "\n",
    "    \n",
    "    return \", \".join(listGenders)\n",
    "\n",
    "def getNationality(ids):\n",
    "    listIds = str(ids).split(', ')\n",
    "    listNationality = list()\n",
    "    for id in listIds:\n",
    "        nationality = artists[artists['ID']== id]\n",
    "        if len(nationality)>0:\n",
    "            nationality = nationality['Nationality'].values[0]\n",
    "            listNationality.append(nationality)\n",
    "        else:\n",
    "            listNationality.append('missing')\n",
    "\n",
    "    \n",
    "    return \", \".join(listNationality)\n",
    "\n",
    "#SPARQL QUERY\n",
    "def get_results(endpoint_url, query):\n",
    "    user_agent = \"WDQS-example Python/%s.%s\" % (sys.version_info[0], sys.version_info[1])\n",
    "    # TODO adjust user agent; see https://w.wiki/CX6\n",
    "    sparql = SPARQLWrapper(endpoint_url, agent=user_agent)\n",
    "    sparql.setQuery(query)\n",
    "    sparql.setReturnFormat(JSON)\n",
    "    return sparql.query().convert()\n",
    "\n",
    "#Scrape summaries, descriptions and artists statements from Rhizome website \n",
    "def url_to_text_rhizome(url):\n",
    "    page = requests.get(url).text\n",
    "    soup = BeautifulSoup(page, \"html.parser\")\n",
    "    accordion = [p.text.strip() for p in soup.find(id=\"AccordionDescriptionBody\").find_all('div')]\n",
    "    print(url)\n",
    "    return accordion\n",
    "\n",
    "#Extract text from MoMA website w/ exclusions for 404 status, missing URLs, and pages w/ no despcription\n",
    "def url_to_text_moma(url):\n",
    "    if url != 'missing':\n",
    "        page = requests.get(url)\n",
    "        status = page.status_code\n",
    "        if status != 404:\n",
    "            soup = BeautifulSoup(page.content, \"html.parser\")\n",
    "            try:\n",
    "                if soup.find(class_=\"uneven-columns--work\").find(class_='main-content') is not None:\n",
    "                    text = soup.find(class_=\"uneven-columns--work\").find(class_='main-content').find_all('p')\n",
    "                else:\n",
    "                    text = ''\n",
    "            except AttributeError:\n",
    "                text = ''\n",
    "                pass\n",
    "        else:\n",
    "            text = '404'\n",
    "    else:\n",
    "        text = 'missing'     \n",
    "    print(url)\n",
    "    return text\n",
    "\n",
    "#Extract keywords Rhizome and pass them to a new column\n",
    "def get_keywords(row):\n",
    "    punct_tokenizer = nltk.RegexpTokenizer(r\"\\w+\")\n",
    "    some_text = row['Text']\n",
    "    tokens = punct_tokenizer.tokenize(some_text)\n",
    "    keywords = [keyword for keyword in tokens if keyword.isalpha() and not keyword in stop_words]\n",
    "    freqdist = nltk.FreqDist(keywords)\n",
    "    most_common = freqdist.most_common(20)\n",
    "    most_common_list = []\n",
    "    for t in most_common:\n",
    "        most_common_list.append(t[0])\n",
    "    keywords_string = ', '.join(most_common_list)\n",
    "    return keywords_string\n",
    "\n",
    "#get keywords MoMA \n",
    "def get_keywords_2(row):\n",
    "    punct_tokenizer = nltk.RegexpTokenizer(r\"\\w+\")\n",
    "    some_text = row['text']\n",
    "    tokens = punct_tokenizer.tokenize(some_text)\n",
    "    keywords = [keyword for keyword in tokens if keyword.isalpha() and not keyword in stop_words]\n",
    "    freqdist = nltk.FreqDist(keywords)\n",
    "    most_common = freqdist.most_common(20)\n",
    "    most_common_list = []\n",
    "    for t in most_common:\n",
    "        most_common_list.append(t[0])\n",
    "    keywords_string = ', '.join(most_common_list)\n",
    "    return keywords_string\n",
    "\n",
    "#group genders \n",
    "def group_genders(df):\n",
    "    collectives = ['M', 'F', 'missing', 'NB']\n",
    "    df = df['Gender'].value_counts().rename_axis('Gender').reset_index(name='Counts')\n",
    "    df_collectives = ['Collectives & Collabs', df[df.Gender.str.contains(',')]['Counts'].sum()]\n",
    "    df = df.loc[~df.Gender.isin(collectives) == False].reset_index(drop=True)\n",
    "    df.loc[5] = df_collectives\n",
    "    df = df.sort_values(by='Counts', ascending=False).replace('M', 'Male').replace('F', 'Female').replace('missing', 'Missing').replace('NB', 'Non-Binary')\n",
    "    return df\n",
    "\n",
    "#group genders by department and source\n",
    "def group_genders_dept(df, period, department):\n",
    "    collectives = ['M', 'F', 'missing', 'NB']\n",
    "    df = df['Gender'].value_counts().rename_axis('Gender').reset_index(name='Counts')\n",
    "    df_collectives = ['Collectives & Collabs', df[df.Gender.str.contains(',')]['Counts'].sum()]\n",
    "    df = df.loc[~df.Gender.isin(collectives) == False].reset_index(drop=True)\n",
    "    df.loc[5] = df_collectives\n",
    "    df = df.sort_values(by='Counts', ascending=False).replace('M', 'Male').replace('F', 'Female').replace('missing', 'Missing').replace('NB', 'Non-Binary')\n",
    "    df['Period'] = period\n",
    "    df['Department'] = department\n",
    "    #if period == 'Before 1983':\n",
    "        #df['Counts'] = df['Counts'].div(100).round(2)\n",
    "        #df['Counts'] = df['Counts'].multiply(55).round(2)\n",
    "        #df['Counts'] = df['Counts'].astype(int)\n",
    "    df = df.sort_values(by='Counts', ascending=False).replace('M', 'Male').replace('F', 'Female').replace('missing', 'Missing').replace('NB', 'Non-Binary')\n",
    "    return df\n",
    "\n",
    "#Turn nationalities into north/south divide\n",
    "def getDivide(nation):\n",
    "    if nation in global_north:\n",
    "        return 'Global North'\n",
    "    elif nation in global_south:\n",
    "        return 'Global South'\n",
    "    else: \n",
    "        return 'missing'\n",
    "\n",
    "#Turn nationalities into north/south divide\n",
    "def getRegion(nation):\n",
    "    if nation in africa:\n",
    "        return 'Africa'\n",
    "    elif nation in arab_states:\n",
    "        return 'Arab States'\n",
    "    elif nation in asia_pacific:\n",
    "        return 'Asia & Pacific'\n",
    "    elif nation in europe:\n",
    "        return 'Europe'\n",
    "    elif nation in middle_east:\n",
    "        return 'Middle east'\n",
    "    elif nation in north_america:\n",
    "        return 'North America'\n",
    "    elif nation in south_america:\n",
    "        return 'South/Latin America'\n",
    "    else: \n",
    "        return 'missing'\n",
    "\n",
    "#Get country codes\n",
    "def do_fuzzy_search(country):\n",
    "    try:\n",
    "        result = pycountry.countries.search_fuzzy(country)\n",
    "        return result[0].alpha_3\n",
    "    except:\n",
    "        return np.nan\n",
    "\n",
    "#assign country codes/nations to nationalities \n",
    "def assign_codes(df):\n",
    "    df_nats = df.drop_duplicates(subset =\"Artist\", keep = 'last')\n",
    "    df_nats_set = set(df_nats.Nationality)\n",
    "    missing = pd.DataFrame({'Aalborgenser': ['Korean', 'Native American', 'Canadian Inuit'], 'Aalborg': ['Korea', 'United States', 'Canada']})\n",
    "    parser = pd.read_csv('https://raw.githubusercontent.com/knowitall/chunkedextractor/master/src/main/resources/edu/knowitall/chunkedextractor/demonyms.csv')\n",
    "    corrections = pd.concat([missing, parser])\n",
    "    df_nats_2 = df_nats['Nationality'].value_counts().rename_axis('Nationality').reset_index(name='Count')\n",
    "    for item in df_nats_set:\n",
    "        my = corrections[corrections['Aalborgenser'] == item]\n",
    "        country = my[:1]['Aalborg'].values\n",
    "        if len(country)>0:\n",
    "            country_str = my[:1]['Aalborg'].values[0]\n",
    "            df_nats_2.loc[df_nats_2[\"Nationality\"] == item, \"Nationality\"] = country_str\n",
    "    df_countries_count = pd.DataFrame(columns= ['Nation', 'Count'])\n",
    "    new_set = set(df_nats_2.Nationality)\n",
    "    for item in new_set:\n",
    "        subCountry = df_nats_2[df_nats_2['Nationality'] == item]\n",
    "        sum_acquisitions = subCountry['Count'].sum()\n",
    "        df_countries_count.loc[len(df_countries_count.index)] = [item, sum_acquisitions]\n",
    "    df_countries_count[\"country_code\"] = df_countries_count[\"Nation\"].apply(lambda country: do_fuzzy_search(country))\n",
    "    df_countries_count[\"country_code\"] = df_countries_count[\"country_code\"].astype(str)\n",
    "    df_countries_count = df_countries_count.sort_values(by='Nation').reset_index(drop=True)\n",
    "    return df_countries_count"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "#LOAD ALL NEEDED PICKLED/EXTERNAL DATA \n",
    "\n",
    "#complete DFs\n",
    "rhz_artworks = pd.read_pickle(path+'Rhizome_data/pickles/rhizome_artworks_extra.pkl')\n",
    "rhz_artworks = rhz_artworks.replace('N/B', 'NB')\n",
    "moma_artworks = pd.read_pickle(path+'MOMA_data/pickle/MoMAartworks.pkl')\n",
    "moma_artworks_old =  pd.read_pickle(path+'MOMA_data/pickle/old_artworks.pkl')\n",
    "moma_artworks_new = pd.read_pickle(path+'MOMA_data/pickle/new_artworks.pkl')\n",
    "moma_rhz_compare = moma_artworks_new.loc[moma_artworks_new['dateAcquired'] >= 2000]\n",
    "\n",
    "#MoMA department DFs\n",
    "moma_draw = moma_artworks.loc[moma_artworks['Department'] == 'Drawings & Prints']\n",
    "moma_draw_before = moma_draw.loc[moma_draw['dateCreated'] <= 1982]\n",
    "moma_draw_after = moma_draw.loc[moma_draw['dateCreated'] >= 1983]\n",
    "moma_films = moma_artworks.loc[moma_artworks['Department'] == 'Film']\n",
    "moma_films_before = moma_films.loc[moma_films['dateCreated'] <= 1982]\n",
    "moma_films_after = moma_films.loc[moma_films['dateCreated'] >= 1983]\n",
    "moma_media = moma_artworks.loc[moma_artworks['Department'] == 'Media and Performance']\n",
    "moma_media_before = moma_media.loc[moma_media['dateCreated'] <= 1982]\n",
    "moma_media_after = moma_media.loc[moma_media['dateCreated'] >= 1983]\n",
    "moma_photo = moma_artworks.loc[moma_artworks['Department'] == 'Photography']\n",
    "moma_photo_before = moma_photo.loc[moma_photo['dateCreated'] <= 1982]\n",
    "moma_photo_after = moma_photo.loc[moma_photo['dateCreated'] >= 1983]\n",
    "\n",
    "#Rhizome with text\n",
    "rhizome_txt_clean = pd.read_pickle(path+'Rhizome_data/pickles/rhizome_artworks_extra_text_clean.pkl')\n",
    "rhizome_txt_stop_kw = pd.read_pickle(path+'Rhizome_data/pickles/rhizome_artworks_extra_text_clean_stop_keywords.pkl')\n",
    "\n",
    "#MoMA with text\n",
    "moma_arch_cont_text_stop = pd.read_pickle(path+'MOMA_data/pickle/departments/architecture_design_cont_text_final_stop.pkl')\n",
    "moma_arch_mod_text_stop = pd.read_pickle(path+'MOMA_data/pickle/departments/architecture_design_mod_text_only_final_stop.pkl')\n",
    "moma_draw_cont_text_stop = pd.read_pickle(path+'MOMA_data/pickle/departments/draws_prints_cont_text_final_stop.pkl')\n",
    "moma_draw_mod_text_stop = pd.read_pickle(path+'MOMA_data/pickle/departments/draws_prints_mod_text_final_stop.pkl')\n",
    "moma_films_cont_text_stop = pd.read_pickle(path+'MOMA_data/pickle/departments/films_cont_text_final_stop.pkl')\n",
    "moma_films_mod_text_stop = pd.read_pickle(path+'MOMA_data/pickle/departments/films_mod_text_final_stop.pkl')\n",
    "moma_media_cont_text_stop = pd.read_pickle(path+'MOMA_data/pickle/departments/media_perf_cont_text_final_stop.pkl')\n",
    "moma_media_mod_text_stop = pd.read_pickle(path+'MOMA_data/pickle/departments/media_perf_mod_text_final_stop.pkl')\n",
    "moma_paint_cont_text_stop = pd.read_pickle(path+'MOMA_data/pickle/departments/paint_sculp_cont_text_final_stop.pkl')\n",
    "moma_paint_mod_text_stop = pd.read_pickle(path+'MOMA_data/pickle/departments/paint_sculp_mod_text_final_stop.pkl')\n",
    "moma_photo_cont_text_stop = pd.read_pickle(path+'MOMA_data/pickle/departments/photo_cont_text_final_stop.pkl')\n",
    "moma_photo_mod_text_stop = pd.read_pickle(path+'MOMA_data/pickle/departments/photo_mod_text_final_stop.pkl')\n",
    "moma_full_text_stop = pd.concat([moma_arch_cont_text_stop, moma_draw_cont_text_stop, moma_draw_mod_text_stop, moma_films_cont_text_stop, moma_films_mod_text_stop, moma_media_cont_text_stop, moma_media_mod_text_stop, moma_paint_cont_text_stop, moma_paint_mod_text_stop, moma_photo_cont_text_stop, moma_photo_mod_text_stop])\n",
    "\n",
    "#create global north/south filters + regions\n",
    "north_south_divide = pd.read_excel(path+'divide_list.xlsx')\n",
    "global_north = north_south_divide.loc[north_south_divide['Divide'] == 'Global North']\n",
    "global_north = global_north['Country'].to_list()\n",
    "global_south = north_south_divide.loc[north_south_divide['Divide'] == 'Global South']\n",
    "global_south = global_south['Country'].to_list()\n",
    "africa = north_south_divide.loc[north_south_divide['Region'] == 'Africa']\n",
    "africa = africa['Country'].to_list()\n",
    "arab_states = north_south_divide.loc[north_south_divide['Region'] == 'Arab States']\n",
    "arab_states = arab_states['Country'].to_list()\n",
    "asia_pacific = north_south_divide.loc[north_south_divide['Region'] == 'Asia & Pacific']\n",
    "asia_pacific = asia_pacific['Country'].to_list()\n",
    "europe = north_south_divide.loc[north_south_divide['Region'] == 'Europe']\n",
    "europe = europe['Country'].to_list()\n",
    "middle_east = north_south_divide.loc[north_south_divide['Region'] == 'Middle east']\n",
    "middle_east = middle_east['Country'].to_list()\n",
    "north_america = north_south_divide.loc[north_south_divide['Region'] == 'North America']\n",
    "north_america = north_america['Country'].to_list()\n",
    "south_america = north_south_divide.loc[north_south_divide['Region'] == 'South/Latin America']\n",
    "south_america = south_america['Country'].to_list()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Assessing the Available Data (Everyone)\n",
    "\n",
    "Our research began by looking at the available datasets. We downloaded [MoMA's publicly available dumps](https://github.com/MuseumofModernArt/collection) and looked at their formatting, and we then explored Rhizome's dataset (AKA the ArtBase) via their [SPARQL endpoint](https://query.artbase.rhizome.org/). The first key observations were as follows:\n",
    "- MoMA's artworks and artists datasets are linked via an ID system, which did not exist in Rhizome\n",
    "- Rhizome's dataset has more missing information, such as biographical information of artists \n",
    "- The sizes of the datasets are drastically different with Rhizome's artists set being 10% of MoMA's and its artworks set being 1.6% percent of MoMA's \n",
    "\n",
    "We contacted Rhizome with some questions about how their data was formatted and they did eventually supply us with a copy of their RDF dump as of February 2022. We used this RDF file to primarily check our queries and extracted all the data we needed directly via the endpoint. We would like to acknowledge and thank Dragan Espenschied, Preservation Director at Rhizome, for his willingness to answer some of our questions about the formatting of Rhizome's data and supplying us with a copy of the RDF dump. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Cleaning and Preparing MoMA Data (Chiara)\n",
    "\n",
    "For the artworks dataset we selected the columns in MoMA's datasets most relevant to our research, as well as most aligned with existing Rhizome data formatting. These were: Title; Artist; ConstituentID (linking system); Date (renamed to Date Created); Medium; Department; DateAcquired; URL (for web scraping); ThumbnailURL (for potential use of images). We also decided to split the artworks dataset by Departments, created subsequent filtered sets for potential more meaningful analysis with Rhizome's collection.<br><br>\n",
    "For the artists dataset all columns were kept as they were deemed potentially useful and we then used the Constituent ID mechanism to populate the artworks dataset with two key values for our analysis: gender and nationality. This way we would be able to work from the artworks dataset only. <br><br>\n",
    "Finally we also selected 1983 as a pivot date for the MoMA artworks dataset that could help us see changes between acquisitions before and after the internet. 1983 was the year that the first artwork was created in Rhizome as well as the year in which TCP/IP was standardized, in effect a turning point towards the ubiquity of the worldwide web. The MoMA datasets were thus also duplicated into a before and after version.<br><br>\n",
    "Finally, considering that we knew there would be missing data relevant to our research (such as gender) we decided to normalize all missing gender values using the string 'missing' and all numerical values (years) using 0 and transformed all dates to 4-digit years as we did not want to go more granular than the year level when looking at time. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 236,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\" #COMMENTED OUT TO AVOID BINDER RUNNING IT - select and clean artist columns \n",
    "original_artists = pd.read_csv(path+'MOMA_data/csv/Artists.csv')\n",
    "artists= original_artists[['ConstituentID','DisplayName','Nationality','Gender','BeginDate','EndDate','Wiki QID','ULAN']]\n",
    "artists.rename(columns={'ConstituentID': 'ID', 'DisplayName': 'Artist','BeginDate': 'Birth', 'EndDate': 'Death'}, inplace=True)\n",
    "artists['Gender'] = artists['Gender'].fillna('missing').astype('str')\n",
    "artists['Wiki QID'] = artists['Wiki QID'].fillna('missing').astype('str')\n",
    "artists['ULAN'] = artists['ULAN'].fillna('0').astype('int')\n",
    "artists['ID'] = artists['ID'].fillna('0').astype('str')\n",
    "artists['Artist'] = artists['Artist'].fillna('Unknown').astype('str')\n",
    "artists['Nationality'] = artists['Nationality'].fillna('missing').astype('str')\n",
    "artists['Gender'] = artists['Gender'].fillna('missing').astype('str')\n",
    "artists['Birth'] = artists['Birth'].fillna('0').astype('int')\n",
    "artists['Death'] = artists['Death'].fillna('0').astype('int')\n",
    "\n",
    "#normalize gender labels to M, F, NB for ease of comparison w/ Rhizome\n",
    "for x, row in artists.iterrows():\n",
    "    if re.match(r'[M|m]ale', row.Gender):\n",
    "        artists.at[x, 'Gender'] = 'M'\n",
    "    elif re.match(r'[F|f]emale',row.Gender):\n",
    "        artists.at[x, 'Gender'] = 'F'\n",
    "    elif re.match(r'Non-[B|b]inary', row.Gender):\n",
    "        artists.at[x, 'Gender'] = 'NB'\n",
    "\n",
    "#clean original artworks df\n",
    "original_artworks = pd.read_csv(path+'MOMA_data/csv/Artworks.csv')\n",
    "original_artworks =  original_artworks[['Title','Artist','ConstituentID','Date','Medium','Department','DateAcquired','URL','ThumbnailURL']]\n",
    "original_artworks['Date'] = original_artworks['Date'].apply(lambda x: normalizeDate(x))\n",
    "original_artworks['DateAcquired']=original_artworks['DateAcquired'].where((original_artworks['DateAcquired'].str.len() <= 4), original_artworks['DateAcquired'].str[0:4])\n",
    "original_artworks['Gender'] = original_artworks['ConstituentID'].apply(lambda x: getGender(x))\n",
    "original_artworks['Nationality'] = original_artworks['ConstituentID'].apply(lambda x: getNationality(x))\n",
    "\n",
    "original_artworks = original_artworks.rename(columns={\"Date\": \"dateCreated\", \"DateAcquired\": \"dateAcquired\"})\n",
    "original_artworks['Title'] = original_artworks['Title'].fillna('Unknown')\n",
    "original_artworks['dateAcquired'] = original_artworks['dateAcquired'].astype('str')\n",
    "original_artworks['dateAcquired'] = original_artworks['dateAcquired'].replace('nan', str('0'))\n",
    "original_artworks['dateAcquired']=original_artworks['dateAcquired'].astype('int')\n",
    "original_artworks['dateCreated'] = original_artworks['dateCreated'].astype('str')\n",
    "original_artworks['dateCreated'] = original_artworks['dateCreated'].replace('nan', str('0'))\n",
    "original_artworks['dateCreated']=original_artworks['dateCreated'].astype('int')\n",
    "\n",
    "#pickle for reuse\n",
    "artists.to_pickle(path+'MoMA_data/pickle/MoMAartists.pkl')\n",
    "original_artworks.to_pickle(path+'MoMA_data/pickle/MoMAartworks.pkl')\n",
    "\n",
    "#create time-based subsets\n",
    "before83 = original_artworks[(original_artworks['dateAcquired'] < 1983) & (original_artworks['dateCreated'] < 1983)]\n",
    "after83 = original_artworks[(original_artworks['dateAcquired'] >= 1983) & (original_artworks['dateCreated'] >= 1983)]\n",
    "before83.to_pickle(path+'MoMA_data/pickle/old_artworks.pkl')\n",
    "after83.to_pickle(path+'MoMA_data/pickle/new_artworks.pkl')\n",
    "\n",
    "#split artwork datasets further based on department for ease of comparison\n",
    "architecture_design = before83[before83['Department'] == \"Architecture & Design\"]\n",
    "architecture_design.to_pickle(path+'MoMA_data/pickle/departments/architecture_design_mod.pkl')\n",
    "architecture_design_img  = before83[before83['Department'] == \"Architecture & Design - Image Archive\"]\n",
    "architecture_design_img.to_pickle(path+'MoMA_data/pickle/departments/architecture_design_img_mod.pkl')\n",
    "draws_prints= before83[before83['Department'] == \"Drawings & Prints\"]\n",
    "draws_prints.to_pickle(path+'MoMA_data/pickle/departments/draws_prints_mod.pkl')\n",
    "films= before83[before83['Department'] == \"Film\"]\n",
    "films.to_pickle(path+'MoMA_data/pickle/departments/films_mod.pkl')\n",
    "fluxus= before83[before83['Department'] == \"Fluxus Collection\"]\n",
    "fluxus.to_pickle(path+'MoMA_data/pickle/departments/fluxus_mod.pkl')\n",
    "media_perf= before83[before83['Department'] == \"Media and Performance\"]\n",
    "media_perf.to_pickle(path+'MoMA_data/pickle/departments/media_perf_mod.pkl')\n",
    "painting_sculp= before83[before83['Department'] == \"Painting & Sculpture\"]\n",
    "painting_sculp.to_pickle(path+'MoMA_data/pickle/departments/paint_sculp_mod.pkl')\n",
    "photo= before83[before83['Department'] == \"Photography\"]\n",
    "photo.to_pickle(path+'MoMA_data/pickle/departments/photo_mod.pkl')\n",
    "architecture_design = after83[after83['Department'] == \"Architecture & Design\"]\n",
    "architecture_design.to_pickle(path+'MoMA_data/pickle/departments/architecture_design_cont.pkl')\n",
    "architecture_design_img  = after83[after83['Department'] == \"Architecture & Design - Image Archive\"]\n",
    "architecture_design_img.to_pickle(path+'MoMA_data/pickle/departments/architecture_design_img_cont.pkl')\n",
    "draws_prints= after83[after83['Department'] == \"Drawings & Prints\"]\n",
    "draws_prints.to_pickle(path+'MoMA_data/pickle/departments/draws_prints_cont.pkl')\n",
    "films= after83[after83['Department'] == \"Film\"]\n",
    "films.to_pickle(path+'MoMA_data/pickle/departments/films_cont.pkl')\n",
    "fluxus= after83[after83['Department'] == \"Fluxus Collection\"]\n",
    "fluxus.to_pickle(path+'MoMA_data/pickle/departments/fluxus_cont.pkl')\"\"\"  \"\"\"\n",
    "media_perf= after83[after83['Department'] == \"Media and Performance\"]\n",
    "media_perf.to_pickle(path+'MoMA_data/pickle/departments/media_perf_cont.pkl')\n",
    "painting_sculp= after83[after83['Department'] == \"Painting & Sculpture\"]\n",
    "painting_sculp.to_pickle(path+'MoMA_data/pickle/departments/paint_sculp_cont.pkl')\n",
    "photo= after83[after83['Department'] == \"Photography\"]\n",
    "photo.to_pickle(path+'MoMA_data/pickle/departments/photo_cont.pkl') \"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Title</th>\n",
       "      <th>Artist</th>\n",
       "      <th>ConstituentID</th>\n",
       "      <th>ArtistBio</th>\n",
       "      <th>Nationality</th>\n",
       "      <th>BeginDate</th>\n",
       "      <th>EndDate</th>\n",
       "      <th>Gender</th>\n",
       "      <th>Date</th>\n",
       "      <th>Medium</th>\n",
       "      <th>...</th>\n",
       "      <th>ThumbnailURL</th>\n",
       "      <th>Circumference (cm)</th>\n",
       "      <th>Depth (cm)</th>\n",
       "      <th>Diameter (cm)</th>\n",
       "      <th>Height (cm)</th>\n",
       "      <th>Length (cm)</th>\n",
       "      <th>Weight (kg)</th>\n",
       "      <th>Width (cm)</th>\n",
       "      <th>Seat Height (cm)</th>\n",
       "      <th>Duration (sec.)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Ferdinandsbrücke Project, Vienna, Austria (Ele...</td>\n",
       "      <td>Otto Wagner</td>\n",
       "      <td>6210</td>\n",
       "      <td>(Austrian, 1841–1918)</td>\n",
       "      <td>(Austrian)</td>\n",
       "      <td>(1841)</td>\n",
       "      <td>(1918)</td>\n",
       "      <td>(Male)</td>\n",
       "      <td>1896</td>\n",
       "      <td>Ink and cut-and-pasted painted pages on paper</td>\n",
       "      <td>...</td>\n",
       "      <td>http://www.moma.org/media/W1siZiIsIjU5NDA1Il0s...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>48.6000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>168.9000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>City of Music, National Superior Conservatory ...</td>\n",
       "      <td>Christian de Portzamparc</td>\n",
       "      <td>7470</td>\n",
       "      <td>(French, born 1944)</td>\n",
       "      <td>(French)</td>\n",
       "      <td>(1944)</td>\n",
       "      <td>(0)</td>\n",
       "      <td>(Male)</td>\n",
       "      <td>1987</td>\n",
       "      <td>Paint and colored pencil on print</td>\n",
       "      <td>...</td>\n",
       "      <td>http://www.moma.org/media/W1siZiIsIjk3Il0sWyJw...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40.6401</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>29.8451</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Villa near Vienna Project, Outside Vienna, Aus...</td>\n",
       "      <td>Emil Hoppe</td>\n",
       "      <td>7605</td>\n",
       "      <td>(Austrian, 1876–1957)</td>\n",
       "      <td>(Austrian)</td>\n",
       "      <td>(1876)</td>\n",
       "      <td>(1957)</td>\n",
       "      <td>(Male)</td>\n",
       "      <td>1903</td>\n",
       "      <td>Graphite, pen, color pencil, ink, and gouache ...</td>\n",
       "      <td>...</td>\n",
       "      <td>http://www.moma.org/media/W1siZiIsIjk4Il0sWyJw...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>34.3000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>31.8000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>The Manhattan Transcripts Project, New York, N...</td>\n",
       "      <td>Bernard Tschumi</td>\n",
       "      <td>7056</td>\n",
       "      <td>(French and Swiss, born Switzerland 1944)</td>\n",
       "      <td>()</td>\n",
       "      <td>(1944)</td>\n",
       "      <td>(0)</td>\n",
       "      <td>(Male)</td>\n",
       "      <td>1980</td>\n",
       "      <td>Photographic reproduction with colored synthet...</td>\n",
       "      <td>...</td>\n",
       "      <td>http://www.moma.org/media/W1siZiIsIjEyNCJdLFsi...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>50.8000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>50.8000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Villa, project, outside Vienna, Austria, Exter...</td>\n",
       "      <td>Emil Hoppe</td>\n",
       "      <td>7605</td>\n",
       "      <td>(Austrian, 1876–1957)</td>\n",
       "      <td>(Austrian)</td>\n",
       "      <td>(1876)</td>\n",
       "      <td>(1957)</td>\n",
       "      <td>(Male)</td>\n",
       "      <td>1903</td>\n",
       "      <td>Graphite, color pencil, ink, and gouache on tr...</td>\n",
       "      <td>...</td>\n",
       "      <td>http://www.moma.org/media/W1siZiIsIjEyNiJdLFsi...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>38.4000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>19.1000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 29 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                               Title  \\\n",
       "0  Ferdinandsbrücke Project, Vienna, Austria (Ele...   \n",
       "1  City of Music, National Superior Conservatory ...   \n",
       "2  Villa near Vienna Project, Outside Vienna, Aus...   \n",
       "3  The Manhattan Transcripts Project, New York, N...   \n",
       "4  Villa, project, outside Vienna, Austria, Exter...   \n",
       "\n",
       "                     Artist ConstituentID  \\\n",
       "0               Otto Wagner          6210   \n",
       "1  Christian de Portzamparc          7470   \n",
       "2                Emil Hoppe          7605   \n",
       "3           Bernard Tschumi          7056   \n",
       "4                Emil Hoppe          7605   \n",
       "\n",
       "                                   ArtistBio Nationality BeginDate EndDate  \\\n",
       "0                      (Austrian, 1841–1918)  (Austrian)    (1841)  (1918)   \n",
       "1                        (French, born 1944)    (French)    (1944)     (0)   \n",
       "2                      (Austrian, 1876–1957)  (Austrian)    (1876)  (1957)   \n",
       "3  (French and Swiss, born Switzerland 1944)          ()    (1944)     (0)   \n",
       "4                      (Austrian, 1876–1957)  (Austrian)    (1876)  (1957)   \n",
       "\n",
       "   Gender  Date                                             Medium  ...  \\\n",
       "0  (Male)  1896      Ink and cut-and-pasted painted pages on paper  ...   \n",
       "1  (Male)  1987                  Paint and colored pencil on print  ...   \n",
       "2  (Male)  1903  Graphite, pen, color pencil, ink, and gouache ...  ...   \n",
       "3  (Male)  1980  Photographic reproduction with colored synthet...  ...   \n",
       "4  (Male)  1903  Graphite, color pencil, ink, and gouache on tr...  ...   \n",
       "\n",
       "                                        ThumbnailURL Circumference (cm)  \\\n",
       "0  http://www.moma.org/media/W1siZiIsIjU5NDA1Il0s...                NaN   \n",
       "1  http://www.moma.org/media/W1siZiIsIjk3Il0sWyJw...                NaN   \n",
       "2  http://www.moma.org/media/W1siZiIsIjk4Il0sWyJw...                NaN   \n",
       "3  http://www.moma.org/media/W1siZiIsIjEyNCJdLFsi...                NaN   \n",
       "4  http://www.moma.org/media/W1siZiIsIjEyNiJdLFsi...                NaN   \n",
       "\n",
       "  Depth (cm) Diameter (cm) Height (cm) Length (cm) Weight (kg)  Width (cm)  \\\n",
       "0        NaN           NaN     48.6000         NaN         NaN    168.9000   \n",
       "1        NaN           NaN     40.6401         NaN         NaN     29.8451   \n",
       "2        NaN           NaN     34.3000         NaN         NaN     31.8000   \n",
       "3        NaN           NaN     50.8000         NaN         NaN     50.8000   \n",
       "4        NaN           NaN     38.4000         NaN         NaN     19.1000   \n",
       "\n",
       "  Seat Height (cm) Duration (sec.)  \n",
       "0              NaN             NaN  \n",
       "1              NaN             NaN  \n",
       "2              NaN             NaN  \n",
       "3              NaN             NaN  \n",
       "4              NaN             NaN  \n",
       "\n",
       "[5 rows x 29 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#raw data\n",
    "moma_raw = pd.read_csv(path+'MOMA_data/csv/Artworks.csv')\n",
    "moma_raw.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 237,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Title</th>\n",
       "      <th>Artist</th>\n",
       "      <th>ConstituentID</th>\n",
       "      <th>dateCreated</th>\n",
       "      <th>Medium</th>\n",
       "      <th>Department</th>\n",
       "      <th>dateAcquired</th>\n",
       "      <th>URL</th>\n",
       "      <th>ThumbnailURL</th>\n",
       "      <th>Gender</th>\n",
       "      <th>Nationality</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Ferdinandsbrücke Project, Vienna, Austria (Ele...</td>\n",
       "      <td>Otto Wagner</td>\n",
       "      <td>6210</td>\n",
       "      <td>1896</td>\n",
       "      <td>Ink and cut-and-pasted painted pages on paper</td>\n",
       "      <td>Architecture &amp; Design</td>\n",
       "      <td>1996</td>\n",
       "      <td>http://www.moma.org/collection/works/2</td>\n",
       "      <td>http://www.moma.org/media/W1siZiIsIjU5NDA1Il0s...</td>\n",
       "      <td>M</td>\n",
       "      <td>Austrian</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>City of Music, National Superior Conservatory ...</td>\n",
       "      <td>Christian de Portzamparc</td>\n",
       "      <td>7470</td>\n",
       "      <td>1987</td>\n",
       "      <td>Paint and colored pencil on print</td>\n",
       "      <td>Architecture &amp; Design</td>\n",
       "      <td>1995</td>\n",
       "      <td>http://www.moma.org/collection/works/3</td>\n",
       "      <td>http://www.moma.org/media/W1siZiIsIjk3Il0sWyJw...</td>\n",
       "      <td>M</td>\n",
       "      <td>French</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Villa near Vienna Project, Outside Vienna, Aus...</td>\n",
       "      <td>Emil Hoppe</td>\n",
       "      <td>7605</td>\n",
       "      <td>1903</td>\n",
       "      <td>Graphite, pen, color pencil, ink, and gouache ...</td>\n",
       "      <td>Architecture &amp; Design</td>\n",
       "      <td>1997</td>\n",
       "      <td>http://www.moma.org/collection/works/4</td>\n",
       "      <td>http://www.moma.org/media/W1siZiIsIjk4Il0sWyJw...</td>\n",
       "      <td>M</td>\n",
       "      <td>Austrian</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>The Manhattan Transcripts Project, New York, N...</td>\n",
       "      <td>Bernard Tschumi</td>\n",
       "      <td>7056</td>\n",
       "      <td>1980</td>\n",
       "      <td>Photographic reproduction with colored synthet...</td>\n",
       "      <td>Architecture &amp; Design</td>\n",
       "      <td>1995</td>\n",
       "      <td>http://www.moma.org/collection/works/5</td>\n",
       "      <td>http://www.moma.org/media/W1siZiIsIjEyNCJdLFsi...</td>\n",
       "      <td>M</td>\n",
       "      <td>missing</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Villa, project, outside Vienna, Austria, Exter...</td>\n",
       "      <td>Emil Hoppe</td>\n",
       "      <td>7605</td>\n",
       "      <td>1903</td>\n",
       "      <td>Graphite, color pencil, ink, and gouache on tr...</td>\n",
       "      <td>Architecture &amp; Design</td>\n",
       "      <td>1997</td>\n",
       "      <td>http://www.moma.org/collection/works/6</td>\n",
       "      <td>http://www.moma.org/media/W1siZiIsIjEyNiJdLFsi...</td>\n",
       "      <td>M</td>\n",
       "      <td>Austrian</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                               Title  \\\n",
       "0  Ferdinandsbrücke Project, Vienna, Austria (Ele...   \n",
       "1  City of Music, National Superior Conservatory ...   \n",
       "2  Villa near Vienna Project, Outside Vienna, Aus...   \n",
       "3  The Manhattan Transcripts Project, New York, N...   \n",
       "4  Villa, project, outside Vienna, Austria, Exter...   \n",
       "\n",
       "                     Artist ConstituentID  dateCreated  \\\n",
       "0               Otto Wagner          6210         1896   \n",
       "1  Christian de Portzamparc          7470         1987   \n",
       "2                Emil Hoppe          7605         1903   \n",
       "3           Bernard Tschumi          7056         1980   \n",
       "4                Emil Hoppe          7605         1903   \n",
       "\n",
       "                                              Medium             Department  \\\n",
       "0      Ink and cut-and-pasted painted pages on paper  Architecture & Design   \n",
       "1                  Paint and colored pencil on print  Architecture & Design   \n",
       "2  Graphite, pen, color pencil, ink, and gouache ...  Architecture & Design   \n",
       "3  Photographic reproduction with colored synthet...  Architecture & Design   \n",
       "4  Graphite, color pencil, ink, and gouache on tr...  Architecture & Design   \n",
       "\n",
       "   dateAcquired                                     URL  \\\n",
       "0          1996  http://www.moma.org/collection/works/2   \n",
       "1          1995  http://www.moma.org/collection/works/3   \n",
       "2          1997  http://www.moma.org/collection/works/4   \n",
       "3          1995  http://www.moma.org/collection/works/5   \n",
       "4          1997  http://www.moma.org/collection/works/6   \n",
       "\n",
       "                                        ThumbnailURL Gender Nationality  \n",
       "0  http://www.moma.org/media/W1siZiIsIjU5NDA1Il0s...      M    Austrian  \n",
       "1  http://www.moma.org/media/W1siZiIsIjk3Il0sWyJw...      M      French  \n",
       "2  http://www.moma.org/media/W1siZiIsIjk4Il0sWyJw...      M    Austrian  \n",
       "3  http://www.moma.org/media/W1siZiIsIjEyNCJdLFsi...      M     missing  \n",
       "4  http://www.moma.org/media/W1siZiIsIjEyNiJdLFsi...      M    Austrian  "
      ]
     },
     "execution_count": 237,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#cleaned data\n",
    "original_artworks = pd.read_pickle(path+'MoMA_data/pickle/MoMAartworks.pkl')\n",
    "original_artworks.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Cleaning and Preparing Rhizome Data (Laurent & Margherita)\n",
    "\n",
    "We began our exploration of Rhizome's dataset with a variety of SPARQL queries before settling on the following three to help us build artworks and artists datasets that could be equivalent to MoMA's: \n",
    "- All artists with available biographic information\n",
    "- All artworks with available metadata (title, dates, associated artist, website URLs) \n",
    "- Artists who are/aren't part of a collective \n",
    "\n",
    "These queries were conducted directly on the endpoint (and checked against the local RDF file for consistency) to retrieve them as csv files and are replicated below formatted using the SPARQLwrapper library.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#set endpoint\n",
    "\n",
    "endpoint_url = \"https://query.artbase.rhizome.org/proxy/wdqs/bigdata/namespace/wdq/sparql\"\n",
    "\n",
    "#query to get all artworks and available information \n",
    "query_all_artworks = \"\"\"SELECT ?artwork_page ?artwork_label ?artist_label ?accession ?inception ?summary_url ?description_url ?statement_url\n",
    "{ \n",
    "  ?artwork rdfs:label ?artwork_label ;\n",
    "   rt:P3 r:Q5 ;\n",
    "   rt:P29 ?artist .\n",
    "   ?artist rdfs:label ?artist_label . \n",
    "   ?artwork_page schema:about ?artwork .\n",
    "  OPTIONAL {\n",
    "   ?artwork rt:P85 ?accession ;\n",
    "           rt:P26 ?inception . }\n",
    "  OPTIONAL {\n",
    "  ?artwork rt:P123 ?summary_description .\n",
    "  ?summary_description rt:P3 r:Q4985 ;\n",
    "          rdfs:label ?summary_label .\n",
    "  ?summary_url schema:about ?summary_description .}\n",
    "  OPTIONAL {\n",
    "  ?artwork rt:P123 ?description .\n",
    "  ?description rt:P3 r:Q9759 ;\n",
    "          rdfs:label ?description_label .\n",
    "  ?description_url schema:about ?description . }\n",
    "  OPTIONAL {\n",
    "    ?artwork rt:P123 ?statement .\n",
    "  ?statement rt:P3 r:Q11838 ;\n",
    "          rdfs:label ?statement_label .\n",
    "  ?statement_url schema:about ?statement .\n",
    "}}\n",
    "\n",
    "\"\"\"\n",
    "#collectives and their individual members\n",
    "query_artists = '''SELECT ?subject ?member ?collectiveLabel ?memberLabel\n",
    "WHERE { \n",
    "  ?subject rt:P3 r:Q7;\n",
    "           rdfs:label ?collectiveLabel;\n",
    "           rt:P43 ?member.\n",
    "  ?member rdfs:label ?memberLabel.\n",
    "  \n",
    "SERVICE wikibase:label { bd:serviceParam wikibase:language \"en\" }}'''\n",
    "\n",
    "#artists who aren't in a collective \n",
    "query_collective_filter = '''SELECT DISTINCT ?artistLabel ?artistPage\n",
    "WHERE {\n",
    "?artwork rt:P3 r:Q5.\n",
    "?artwork rt:P29 ?artist.\n",
    "?artist rt:P3 r:Q6.\n",
    "?artistPage schema:about ?artist ;\n",
    "schema:isPartOf <https://artbase.rhizome.org/> .\n",
    "SERVICE wikibase:label { bd:serviceParam wikibase:language \"en\" }\n",
    "FILTER (?artist not in (r:Q7) )\n",
    "}\n",
    "ORDER BY ?artistLabel\n",
    "'''\n",
    "\n",
    "#results = get_results(endpoint_url, query_all_artworks)\n",
    "\n",
    "#for result in results[\"results\"][\"bindings\"]:\n",
    "#    print(result)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The resulting datasets were however not complete enough for us to conduct a meaningful analysis. A major issue was that Rhizome's ArtBase had no meaningful biographical information for artists, and with gender and nationality being key variables for our exploration we needed to complete this information. We tried a variety of automated ways -- including using Google's [Graph Search API](https://developers.google.com/knowledge-graph) -- however they proved limiting as the majority of artists in Rhizome's ArtBase are not popular enough to appear in Google's graph or if they did they were likely to have a name similar to someone else. <br><br>\n",
    "In the end we decided to complete our dataset manually by conducting searches for nationality, gender, birth and death dates as well as WikiData and ULAN IDs. We used a simple common procedure: take the artist name and input it into Google and [Getty's ULAN database](https://www.getty.edu/research/tools/vocabularies/ulan/); from there try to quickly assert if information is available and input it. We also gave ourselves the following leeway: *if an artist's gender or nationality was not clearly stated we inferred it from available text (such as the use of pronouns and primary places of work/study), and we added this into a separate gender/nationality column marked as guessed to differentiate it from the confirmed information*. In total we were able to confirm or infer nationality information for 70% of artists and gender information for over 90% of artists (we inferred a total of 224 nationalities and 233 genders).<br><br>\n",
    "The final results of this work can be seen in the artists_complete.csv file held in the Rhizome_data folder. <br><br>\n",
    "Following this we ran the Rhizome datasets through some scripts to align them with the MoMA datasets as well as perform the same linking exercise using an ID system that would allow us to have an artwork dataset containing gender and nationality information for our analysis. At this point inferred gender and nationality were combined with the confirmed information. We also decided that artists who were part of a collective would be treated as a collective only unless they also appeared in the ArtBase as a solo artist or unless the collective was larger than 2 artists. This decision was made to allow us to increase the Rhizome artist dataset as much as possible to help with our analysis, and was refined over various iterations including some discoveries during the scraping stage which revealed that Rhizome contained artworks on their site with artists not yet included in the ArtBase. <br><br>\n",
    "As a result the total artists dataset increased from 1,251 entries to 1,272, while the artworks dataset was decreased from a raw search of 2,743 down to 2,270, mainly due to the removal of duplicates or entries with missing information (Rhizome publicly claims the ArtBase to contain more than 2,200 artworks). \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\" #COMMENTED OUT TO AVOID BINDER RUNNING IT - create a df for artists, pickle \n",
    "rhz_artists= pd.read_csv('./Rhizome_data/csv/artists_complete.csv', dtype='string')\n",
    "#set empty fields w/ missing\n",
    "rhz_artists['artistLabel'] = rhz_artists['artistLabel'].fillna('')\n",
    "rhz_artists['artistPage'] = rhz_artists['artistPage'].fillna('')\n",
    "rhz_artists['Nationality'] = rhz_artists['Nationality'].fillna('')\n",
    "rhz_artists['Nationality Guessed'] = rhz_artists['Nationality Guessed'].fillna('')\n",
    "rhz_artists['Gender'] = rhz_artists['Gender'].fillna('')\n",
    "rhz_artists['Gender Guessed'] = rhz_artists['Gender Guessed'].fillna('')\n",
    "rhz_artists['Birth'] = rhz_artists['Birth'].fillna('0').astype('int')\n",
    "rhz_artists['Death'] = rhz_artists['Death'].fillna('0').astype('int')\n",
    "rhz_artists['Wiki QID'] = rhz_artists['Wiki QID'].fillna('')\n",
    "rhz_artists['ULAN'] = rhz_artists['ULAN'].fillna('')\n",
    "#set collective fields to empty not missing \n",
    "rhz_artists['collectiveLabel'] = rhz_artists['collectiveLabel'].fillna('')\n",
    "rhz_artists['collectivePage'] = rhz_artists['collectivePage'].fillna('')\n",
    "#create and populate ID field to cross reference with artworks \n",
    "rhz_artists['ID'] = range(1, 1+len(rhz_artists))\n",
    "rhz_artists['ID'] = rhz_artists['ID'].astype('string')\n",
    "#drop the notes column \n",
    "rhz_artists = rhz_artists.drop('Notes', axis=1)\n",
    "#rename columns \n",
    "rhz_artists.rename(columns={'artistLabel': 'Artist', 'artistPage': 'Artist URL', 'collectiveLabel': 'Collective', 'collectivePage': 'Collective URL', 'Wiki QID': 'WikiQID', 'Nationality Guessed': 'nationalityGuessed', 'Gender Guessed': 'genderGuessed', 'Wiki QID': 'wikiQID'}, inplace=True)\n",
    "\n",
    "#rhz_artists.to_pickle(path+'Rhizome_data/pickles/rhizome_artists.pkl')\n",
    "\n",
    "#create a second artists DF w/ concataned gender and nationality columns \n",
    "rhz_artists_extra = rhz_artists.copy()\n",
    "rhz_artists_extra['Gender'] = rhz_artists_extra['Gender'].str.cat(rhz_artists_extra['genderGuessed'], join='outer', na_rep='missing')\n",
    "rhz_artists_extra = rhz_artists_extra.drop('genderGuessed', axis=1)\n",
    "rhz_artists_extra.loc[rhz_artists_extra['Gender'] == '', 'Gender'] = 'missing'\n",
    "rhz_artists_extra['Nationality'] = rhz_artists_extra['Nationality'].str.cat(rhz_artists_extra['nationalityGuessed'], join='outer', na_rep='missing')\n",
    "rhz_artists_extra = rhz_artists_extra.drop('nationalityGuessed', axis=1)\n",
    "rhz_artists_extra.loc[rhz_artists_extra['Nationality'] == '', 'Nationality'] = 'missing'\n",
    "\n",
    "#rhz_artists_extra.to_csv(path+'Rhizome_data/pickles/rhizome_artists_extra.csv')\n",
    "#rhz_artists_extra.to_pickle(path+'Rhizome_data/pickles/rhizome_artists_extra.pkl')\n",
    "\n",
    "#create a df for artworks, pickle \n",
    "rhz_artworks= pd.read_csv('./Rhizome_data/csv/artwork_complete.csv', dtype='string')\n",
    "#set empty fields w/ missing or empty strings \n",
    "rhz_artworks['accession'] = rhz_artworks['accession'].fillna('0')\n",
    "rhz_artworks.loc[:,'accession'] = rhz_artworks['accession'].where((rhz_artworks['accession'].str.len() <= 4), rhz_artworks['accession'].str[0:4])\n",
    "rhz_artworks['inception'] = rhz_artworks['inception'].fillna('0')\n",
    "rhz_artworks.loc[:,'inception'] = rhz_artworks['inception'].where((rhz_artworks['inception'].str.len() <= 4), rhz_artworks['inception'].str[0:4])\n",
    "url_columns = ['summary_url', 'summary_url_2', 'description_url', 'description_url_2', 'statement_url']\n",
    "rhz_artworks[url_columns] = rhz_artworks[url_columns].fillna('')\n",
    "#rename columns \n",
    "rhz_artworks.rename(columns={'artwork_label': 'Title', 'artist_label': 'Artist', 'accession': 'dateAcquired', 'inception': 'dateCreated', 'artwork_page': 'URL'}, inplace=True)\n",
    "#auto populate ID based on artist df \n",
    "for index, row in rhz_artworks.iterrows():\n",
    "    ids = list()\n",
    "    artist = getattr(row, 'Artist')\n",
    "    artist = artist.split(', ')\n",
    "    artists = rhz_artists_extra[['Artist', 'ID']]\n",
    "    artists = dict(list(zip(artists.Artist, artists.ID)))\n",
    "    collective_ids = defaultdict(list)\n",
    "    for i, j in zip(rhz_artists_extra.Collective,rhz_artists_extra.ID):\n",
    "        collective_ids[i].append(j)\n",
    "    collective_ids = dict(collective_ids)\n",
    "    del collective_ids['']\n",
    "    for k, v in collective_ids.items():\n",
    "        collective_ids.update({k: ', '.join([n for n in v])})\n",
    "\n",
    "    for person in artist:\n",
    "        if person in artists:\n",
    "            ids.append(str(artists[person]))\n",
    "        elif person in collective_ids:\n",
    "            ids.append(collective_ids[person])\n",
    "        else:\n",
    "            ids.append('missing')\n",
    "    string = ', '.join(ids)\n",
    "\n",
    "    rhz_artworks.at[index, 'ID'] = string\n",
    "\n",
    "#rhz_artworks.to_pickle(path+'Rhizome_data/pickles/rhizome_artworks.pkl')\n",
    "\n",
    "#create a second df for artworks w/ additional artist info for visualisation \n",
    "rhz_artworks_extra = rhz_artworks.copy()\n",
    "rhz_artworks_extra.drop(['summary_url', 'summary_url_2', 'description_url', 'description_url_2', 'statement_url'], inplace=True, axis=1)\n",
    "\n",
    "#add nationality \n",
    "for index, row in rhz_artworks_extra.iterrows():\n",
    "    nationality = list()\n",
    "    artist = getattr(row, 'Artist')\n",
    "    artist = artist.split(', ')\n",
    "    artists = rhz_artists_extra[['Artist', 'Nationality']]\n",
    "    artists = dict(list(zip(artists.Artist, artists.Nationality)))\n",
    "    collectives_nat = defaultdict(list)\n",
    "    for i, j in zip(rhz_artists_extra.Collective,rhz_artists_extra.Nationality):\n",
    "        collectives_nat[i].append(j)\n",
    "    collectives_nat = dict(collectives_nat)\n",
    "    del collectives_nat['']\n",
    "    for k, v in collectives_nat.items():\n",
    "        collectives_nat.update({k: ', '.join([n for n in v])})\n",
    "\n",
    "    for person in artist:\n",
    "        if person in artists:\n",
    "            nationality.append(str(artists[person]))\n",
    "        elif person in collectives_nat:\n",
    "            nationality.append(collectives_nat[person])\n",
    "        else:\n",
    "            nationality.append('missing')\n",
    "    string_nationality = ', '.join(nationality)\n",
    "\n",
    "    rhz_artworks_extra.at[index, 'Nationality'] = string_nationality\n",
    "    \n",
    "#add gender \n",
    "for index, row in rhz_artworks_extra.iterrows():\n",
    "    gender = list()\n",
    "    artist = getattr(row, 'Artist')\n",
    "    artist = artist.split(', ')\n",
    "    artists = rhz_artists_extra[['Artist', 'Gender']]\n",
    "    artists = dict(list(zip(artists.Artist, artists.Gender)))\n",
    "    collectives_gen = defaultdict(list)\n",
    "    for i, j in zip(rhz_artists_extra.Collective,rhz_artists_extra.Gender):\n",
    "        collectives_gen[i].append(j)\n",
    "    collectives_gen = dict(collectives_gen)\n",
    "    del collectives_gen['']\n",
    "    for k, v in collectives_gen.items():\n",
    "        collectives_gen.update({k: ', '.join([n for n in v])})\n",
    "\n",
    "    for person in artist:\n",
    "        if person in artists:\n",
    "            gender.append(str(artists[person]))\n",
    "        elif person in collectives_gen:\n",
    "            gender.append(collectives_gen[person])\n",
    "        else:\n",
    "            gender.append('missing')\n",
    "    string_gender = ', '.join(gender)\n",
    "\n",
    "    rhz_artworks_extra.at[index, 'Gender'] = string_gender\n",
    "\n",
    "#rhz_artworks_extra.to_pickle(path+'Rhizome_data/pickles/rhizome_artworks_extra.pkl')\n",
    "#rhz_artworks_extra.to_csv(path+'Rhizome_data/pickles/rhizome_artworks_extra.csv') \"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>artistLabel</th>\n",
       "      <th>artistPage</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0100101110101101</td>\n",
       "      <td>https://artbase.rhizome.org/wiki/Q1167</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1010 1010</td>\n",
       "      <td>https://artbase.rhizome.org/wiki/Q1005</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>11811</td>\n",
       "      <td>https://artbase.rhizome.org/wiki/Q4757</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>220hex</td>\n",
       "      <td>https://artbase.rhizome.org/wiki/Q139</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6</td>\n",
       "      <td>https://artbase.rhizome.org/wiki/Q340</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        artistLabel                              artistPage\n",
       "0  0100101110101101  https://artbase.rhizome.org/wiki/Q1167\n",
       "1         1010 1010  https://artbase.rhizome.org/wiki/Q1005\n",
       "2             11811  https://artbase.rhizome.org/wiki/Q4757\n",
       "3            220hex   https://artbase.rhizome.org/wiki/Q139\n",
       "4                 6   https://artbase.rhizome.org/wiki/Q340"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#raw data\n",
    "rhz_raw = pd.read_csv(path+'Rhizome_data/csv/artists_unique.csv')\n",
    "rhz_raw.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>URL</th>\n",
       "      <th>Title</th>\n",
       "      <th>Artist</th>\n",
       "      <th>dateAcquired</th>\n",
       "      <th>dateCreated</th>\n",
       "      <th>Nationality</th>\n",
       "      <th>Gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>879</td>\n",
       "      <td>https://artbase.rhizome.org/wiki/Q2423</td>\n",
       "      <td>ZUR FARBENLEHRE (THEORY OF COLOURS)</td>\n",
       "      <td>Steven Jones</td>\n",
       "      <td>2007</td>\n",
       "      <td>2007</td>\n",
       "      <td>British</td>\n",
       "      <td>M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1020</td>\n",
       "      <td>https://artbase.rhizome.org/wiki/Q4089</td>\n",
       "      <td>Zones de Convergence</td>\n",
       "      <td>cicero</td>\n",
       "      <td>2005</td>\n",
       "      <td>2005</td>\n",
       "      <td>missing</td>\n",
       "      <td>missing</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>243, 701</td>\n",
       "      <td>https://artbase.rhizome.org/wiki/Q1475</td>\n",
       "      <td>Zombie and Mummy</td>\n",
       "      <td>Dragan Espenschied, Olia Lialina</td>\n",
       "      <td>2004</td>\n",
       "      <td>2002</td>\n",
       "      <td>German, Russian</td>\n",
       "      <td>M, F</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>312</td>\n",
       "      <td>https://artbase.rhizome.org/wiki/Q4374</td>\n",
       "      <td>Zaira, City of Memories</td>\n",
       "      <td>Gokcen Erguven</td>\n",
       "      <td>2004</td>\n",
       "      <td>2004</td>\n",
       "      <td>Turkish</td>\n",
       "      <td>F</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>920</td>\n",
       "      <td>https://artbase.rhizome.org/wiki/Q3972</td>\n",
       "      <td>Z_G [zeitgeist gestalten]</td>\n",
       "      <td>Tiago Borges</td>\n",
       "      <td>2008</td>\n",
       "      <td>2007</td>\n",
       "      <td>Angolan</td>\n",
       "      <td>M</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         ID                                     URL  \\\n",
       "0       879  https://artbase.rhizome.org/wiki/Q2423   \n",
       "1      1020  https://artbase.rhizome.org/wiki/Q4089   \n",
       "2  243, 701  https://artbase.rhizome.org/wiki/Q1475   \n",
       "3       312  https://artbase.rhizome.org/wiki/Q4374   \n",
       "4       920  https://artbase.rhizome.org/wiki/Q3972   \n",
       "\n",
       "                                 Title                            Artist  \\\n",
       "0  ZUR FARBENLEHRE (THEORY OF COLOURS)                      Steven Jones   \n",
       "1                 Zones de Convergence                            cicero   \n",
       "2                     Zombie and Mummy  Dragan Espenschied, Olia Lialina   \n",
       "3              Zaira, City of Memories                    Gokcen Erguven   \n",
       "4            Z_G [zeitgeist gestalten]                      Tiago Borges   \n",
       "\n",
       "  dateAcquired dateCreated      Nationality   Gender  \n",
       "0         2007        2007          British        M  \n",
       "1         2005        2005          missing  missing  \n",
       "2         2004        2002  German, Russian     M, F  \n",
       "3         2004        2004         Turkish         F  \n",
       "4         2008        2007          Angolan        M  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#cleaned data\n",
    "rhz_artworks_extra = pd.read_pickle(path+'Rhizome_data/pickles/rhizome_artworks_extra.pkl')\n",
    "rhz_artworks_extra.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Scraping Text (Laurent)\n",
    "\n",
    "One last data preparation step involved scraping text about artworks from both MoMA and Rhizome's websites. This was suggested early on in our exploration process and as we came to better understand the differences between the available datasets we decided that grabbing the text might prove useful as an additional variable for our analysis. In addition it was decided that by doing this we might be able to create department/medium information for Rhizome somewhat equivalent to MoMA's.<br><br>\n",
    "After some trials with the html construction of the Rhizome artwork pages we decided to grab the `<div>` that includes all three possible descriptions (summary, artist statement, description).  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\" #COMMENTED OUT TO AVOID BINDER RUNNING IT - grab URLs from artworks DF, scrape them and return them back to the DF\n",
    "rhz_artworks_extra = pd.read_pickle(path+'Rhizome_data/pickles/rhizome_artworks_extra.pkl')\n",
    "urls = rhz_artworks_extra['URL'].to_list()\n",
    "scrapes = [url_to_text_rhizome(u) for u in urls]\n",
    "rhz_artworks_extra_text = rhz_artworks_extra.copy()\n",
    "rhz_artworks_extra_text['Text'] = pd.Series(scrapes)\n",
    "#fix an erroneous ID in original first round of scraping \n",
    "rhz_artworks_extra_text.loc[777, 'ID'] = '926, 1268'\n",
    "rhz_artworks_extra_text = rhz_artworks_extra_text.astype(str)\n",
    "rhz_artworks_extra_text.to_pickle(path+'Rhizome_data/pickles/rhizome_artworks_extra_text.pkl') \"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>URL</th>\n",
       "      <th>Title</th>\n",
       "      <th>Artist</th>\n",
       "      <th>dateAcquired</th>\n",
       "      <th>dateCreated</th>\n",
       "      <th>Nationality</th>\n",
       "      <th>Gender</th>\n",
       "      <th>Text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>879</td>\n",
       "      <td>https://artbase.rhizome.org/wiki/Q2423</td>\n",
       "      <td>ZUR FARBENLEHRE (THEORY OF COLOURS)</td>\n",
       "      <td>Steven Jones</td>\n",
       "      <td>2007</td>\n",
       "      <td>2007</td>\n",
       "      <td>British</td>\n",
       "      <td>M</td>\n",
       "      <td>['summary edit\\n\\n\\t\\t\\t\\tA short film.\\n\"When...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1020</td>\n",
       "      <td>https://artbase.rhizome.org/wiki/Q4089</td>\n",
       "      <td>Zones de Convergence</td>\n",
       "      <td>cicero</td>\n",
       "      <td>2005</td>\n",
       "      <td>2005</td>\n",
       "      <td>missing</td>\n",
       "      <td>missing</td>\n",
       "      <td>['description edit\\n\\n\\t\\t\\t\\t‘Zones de Conver...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>243, 701</td>\n",
       "      <td>https://artbase.rhizome.org/wiki/Q1475</td>\n",
       "      <td>Zombie and Mummy</td>\n",
       "      <td>Dragan Espenschied, Olia Lialina</td>\n",
       "      <td>2004</td>\n",
       "      <td>2002</td>\n",
       "      <td>German, Russian</td>\n",
       "      <td>M, F</td>\n",
       "      <td>['summary edit\\n\\n Zombie and Mummy is a websi...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>312</td>\n",
       "      <td>https://artbase.rhizome.org/wiki/Q4374</td>\n",
       "      <td>Zaira, City of Memories</td>\n",
       "      <td>Gokcen Erguven</td>\n",
       "      <td>2004</td>\n",
       "      <td>2004</td>\n",
       "      <td>Turkish</td>\n",
       "      <td>F</td>\n",
       "      <td>['description edit\\n\\n\\t\\t\\t\\tThe project is b...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>920</td>\n",
       "      <td>https://artbase.rhizome.org/wiki/Q3972</td>\n",
       "      <td>Z_G [zeitgeist gestalten]</td>\n",
       "      <td>Tiago Borges</td>\n",
       "      <td>2008</td>\n",
       "      <td>2007</td>\n",
       "      <td>Angolan</td>\n",
       "      <td>M</td>\n",
       "      <td>['description edit\\n\\n\\t\\t\\t\\tfrom looking at ...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         ID                                     URL  \\\n",
       "0       879  https://artbase.rhizome.org/wiki/Q2423   \n",
       "1      1020  https://artbase.rhizome.org/wiki/Q4089   \n",
       "2  243, 701  https://artbase.rhizome.org/wiki/Q1475   \n",
       "3       312  https://artbase.rhizome.org/wiki/Q4374   \n",
       "4       920  https://artbase.rhizome.org/wiki/Q3972   \n",
       "\n",
       "                                 Title                            Artist  \\\n",
       "0  ZUR FARBENLEHRE (THEORY OF COLOURS)                      Steven Jones   \n",
       "1                 Zones de Convergence                            cicero   \n",
       "2                     Zombie and Mummy  Dragan Espenschied, Olia Lialina   \n",
       "3              Zaira, City of Memories                    Gokcen Erguven   \n",
       "4            Z_G [zeitgeist gestalten]                      Tiago Borges   \n",
       "\n",
       "  dateAcquired dateCreated      Nationality   Gender  \\\n",
       "0         2007        2007          British        M   \n",
       "1         2005        2005          missing  missing   \n",
       "2         2004        2002  German, Russian     M, F   \n",
       "3         2004        2004         Turkish         F   \n",
       "4         2008        2007          Angolan        M   \n",
       "\n",
       "                                                Text  \n",
       "0  ['summary edit\\n\\n\\t\\t\\t\\tA short film.\\n\"When...  \n",
       "1  ['description edit\\n\\n\\t\\t\\t\\t‘Zones de Conver...  \n",
       "2  ['summary edit\\n\\n Zombie and Mummy is a websi...  \n",
       "3  ['description edit\\n\\n\\t\\t\\t\\tThe project is b...  \n",
       "4  ['description edit\\n\\n\\t\\t\\t\\tfrom looking at ...  "
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#show result\n",
    "rhz_artworks_extra_text = pd.read_pickle(path+'Rhizome_data/pickles/rhizome_artworks_extra_text.pkl')\n",
    "rhz_artworks_extra_text.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For MOMA URLs there was only one possible description on the page but the containing `<div>` is repeated elsewhere so we used its parent container to only extract what we needed. Considering the size of the MoMA dataset we used the department splits created earlier to go through it in chunks. We show the process for one dept only, but this was repeated for all of them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\" # COMMENTED OUT TO AVOID BINDER RUNNING IT - load department DFs\n",
    "moma_arch_cont = pd.read_pickle(path+'MOMA_data/pickle/departments/architecture_design_cont.pkl')\n",
    "moma_arch_mod = pd.read_pickle(path+'MOMA_data/pickle/departments/architecture_design_mod.pkl')\n",
    "moma_design_cont = pd.read_pickle(path+'MOMA_data/pickle/departments/architecture_design_img_cont.pkl')\n",
    "moma_design_mod = pd.read_pickle(path+'MOMA_data/pickle/departments/architecture_design_img_mod.pkl')\n",
    "moma_draw_cont = pd.read_pickle(path+'MOMA_data/pickle/departments/draws_prints_cont.pkl')\n",
    "moma_draw_mod = pd.read_pickle(path+'MOMA_data/pickle/departments/draws_prints_mod.pkl')\n",
    "moma_films_cont = pd.read_pickle(path+'MOMA_data/pickle/departments/films_cont.pkl')\n",
    "moma_films_mod = pd.read_pickle(path+'MOMA_data/pickle/departments/films_mod.pkl')\n",
    "moma_fluxus_cont = pd.read_pickle(path+'MOMA_data/pickle/departments/fluxus_cont.pkl')\n",
    "moma_fluxus_mod = pd.read_pickle(path+'MOMA_data/pickle/departments/fluxus_mod.pkl')\n",
    "moma_media_cont = pd.read_pickle(path+'MOMA_data/pickle/departments/media_perf_cont.pkl')\n",
    "moma_media_mod = pd.read_pickle(path+'MOMA_data/pickle/departments/media_perf_mod.pkl')\"\"\"  \"\"\"\n",
    "moma_paint_cont = pd.read_pickle(path+'MOMA_data/pickle/departments/paint_sculp_cont.pkl')\n",
    "moma_paint_mod = pd.read_pickle(path+'MOMA_data/pickle/departments/paint_sculp_mod.pkl')\n",
    "moma_photo_cont = pd.read_pickle(path+'MOMA_data/pickle/departments/photo_cont.pkl')\n",
    "moma_photo_mod = pd.read_pickle(path+'MOMA_data/pickle/departments/photo_mod.pkl')\n",
    "\n",
    "#grab all links from a dept as a list \n",
    "links = moma_photo_mod['URL'].to_list()\n",
    "#process links w/ function \n",
    "moma_photo_to_add = [url_to_text_moma(u) for u in links]\n",
    "#add results back to a copy of the original DF\n",
    "moma_photo_mod_text = moma_photo_mod.copy()\n",
    "moma_photo_mod_text['Text'] = moma_photo_to_add\n",
    "moma_photo_mod_text['Text'] = moma_photo_mod_text['Text'].astype(str)\n",
    "moma_photo_mod_text.to_pickle(path+'MOMA_data/pickle/departments/photo_mod_text.pkl') \"\"\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Title</th>\n",
       "      <th>Artist</th>\n",
       "      <th>ID</th>\n",
       "      <th>DateCreated</th>\n",
       "      <th>Medium</th>\n",
       "      <th>Department</th>\n",
       "      <th>DateAcquired</th>\n",
       "      <th>URL</th>\n",
       "      <th>ThumbnailURL</th>\n",
       "      <th>Nationality</th>\n",
       "      <th>Gender</th>\n",
       "      <th>Text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>City of Music, National Superior Conservatory ...</td>\n",
       "      <td>Christian de Portzamparc</td>\n",
       "      <td>7470</td>\n",
       "      <td>1987</td>\n",
       "      <td>Paint and colored pencil on print</td>\n",
       "      <td>Architecture &amp; Design</td>\n",
       "      <td>1995</td>\n",
       "      <td>http://www.moma.org/collection/works/3</td>\n",
       "      <td>http://www.moma.org/media/W1siZiIsIjk3Il0sWyJw...</td>\n",
       "      <td>French</td>\n",
       "      <td>M</td>\n",
       "      <td>missing</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>The Manhattan Transcripts Project, New York, N...</td>\n",
       "      <td>Bernard Tschumi</td>\n",
       "      <td>7056</td>\n",
       "      <td>1980</td>\n",
       "      <td>Photographic reproduction with colored synthet...</td>\n",
       "      <td>Architecture &amp; Design</td>\n",
       "      <td>1995</td>\n",
       "      <td>http://www.moma.org/collection/works/5</td>\n",
       "      <td>http://www.moma.org/media/W1siZiIsIjEyNCJdLFsi...</td>\n",
       "      <td>missing</td>\n",
       "      <td>M</td>\n",
       "      <td>The Manhattan Transcripts are theoretical prop...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>The Manhattan Transcripts Project, New York, N...</td>\n",
       "      <td>Bernard Tschumi</td>\n",
       "      <td>7056</td>\n",
       "      <td>1980</td>\n",
       "      <td>Photographic reproduction with colored synthet...</td>\n",
       "      <td>Architecture &amp; Design</td>\n",
       "      <td>1995</td>\n",
       "      <td>http://www.moma.org/collection/works/33</td>\n",
       "      <td>http://www.moma.org/media/W1siZiIsIjIwMCJdLFsi...</td>\n",
       "      <td>missing</td>\n",
       "      <td>M</td>\n",
       "      <td>The Manhattan Transcripts are theoretical prop...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>The Manhattan Transcripts Project, New York, N...</td>\n",
       "      <td>Bernard Tschumi</td>\n",
       "      <td>7056</td>\n",
       "      <td>1980</td>\n",
       "      <td>Photographic reproduction with colored synthet...</td>\n",
       "      <td>Architecture &amp; Design</td>\n",
       "      <td>1995</td>\n",
       "      <td>http://www.moma.org/collection/works/38</td>\n",
       "      <td>http://www.moma.org/media/W1siZiIsIjI2NyJdLFsi...</td>\n",
       "      <td>missing</td>\n",
       "      <td>M</td>\n",
       "      <td>The Manhattan Transcripts are theoretical prop...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>The Manhattan Transcripts Project, New York, N...</td>\n",
       "      <td>Bernard Tschumi</td>\n",
       "      <td>7056</td>\n",
       "      <td>1980</td>\n",
       "      <td>Ink on tracing paper</td>\n",
       "      <td>Architecture &amp; Design</td>\n",
       "      <td>1995</td>\n",
       "      <td>http://www.moma.org/collection/works/44</td>\n",
       "      <td>http://www.moma.org/media/W1siZiIsIjI5NiJdLFsi...</td>\n",
       "      <td>missing</td>\n",
       "      <td>M</td>\n",
       "      <td>The Manhattan Transcripts are theoretical prop...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                Title  \\\n",
       "1   City of Music, National Superior Conservatory ...   \n",
       "3   The Manhattan Transcripts Project, New York, N...   \n",
       "31  The Manhattan Transcripts Project, New York, N...   \n",
       "35  The Manhattan Transcripts Project, New York, N...   \n",
       "40  The Manhattan Transcripts Project, New York, N...   \n",
       "\n",
       "                      Artist    ID  DateCreated  \\\n",
       "1   Christian de Portzamparc  7470         1987   \n",
       "3            Bernard Tschumi  7056         1980   \n",
       "31           Bernard Tschumi  7056         1980   \n",
       "35           Bernard Tschumi  7056         1980   \n",
       "40           Bernard Tschumi  7056         1980   \n",
       "\n",
       "                                               Medium             Department  \\\n",
       "1                   Paint and colored pencil on print  Architecture & Design   \n",
       "3   Photographic reproduction with colored synthet...  Architecture & Design   \n",
       "31  Photographic reproduction with colored synthet...  Architecture & Design   \n",
       "35  Photographic reproduction with colored synthet...  Architecture & Design   \n",
       "40                               Ink on tracing paper  Architecture & Design   \n",
       "\n",
       "   DateAcquired                                      URL  \\\n",
       "1          1995   http://www.moma.org/collection/works/3   \n",
       "3          1995   http://www.moma.org/collection/works/5   \n",
       "31         1995  http://www.moma.org/collection/works/33   \n",
       "35         1995  http://www.moma.org/collection/works/38   \n",
       "40         1995  http://www.moma.org/collection/works/44   \n",
       "\n",
       "                                         ThumbnailURL Nationality Gender  \\\n",
       "1   http://www.moma.org/media/W1siZiIsIjk3Il0sWyJw...      French      M   \n",
       "3   http://www.moma.org/media/W1siZiIsIjEyNCJdLFsi...     missing      M   \n",
       "31  http://www.moma.org/media/W1siZiIsIjIwMCJdLFsi...     missing      M   \n",
       "35  http://www.moma.org/media/W1siZiIsIjI2NyJdLFsi...     missing      M   \n",
       "40  http://www.moma.org/media/W1siZiIsIjI5NiJdLFsi...     missing      M   \n",
       "\n",
       "                                                 Text  \n",
       "1                                             missing  \n",
       "3   The Manhattan Transcripts are theoretical prop...  \n",
       "31  The Manhattan Transcripts are theoretical prop...  \n",
       "35  The Manhattan Transcripts are theoretical prop...  \n",
       "40  The Manhattan Transcripts are theoretical prop...  "
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#show results\n",
    "moma_arch_cont_text = pd.read_pickle(path+'MOMA_data/pickle/departments/architecture_design_cont_text.pkl')\n",
    "moma_arch_cont_text.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Cleaning & Keywords\n",
    "\n",
    "Scraping rhizome resulted in text heavy with encoding noise. We cleaned it as best we could without damaging the underlying text and then analyzed the text for the most common words usng NLTK's stopwords and our own custom list based on analysis on the scraped results and some early tests. We then extracted 20 keywords from available text and assigned those to their own column. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\" #COMMENTED TO AVOID BINDER RUNNING IT - load the DF w/ scraped text\n",
    "rhz_artworks_extra_text = pd.read_pickle(path+'Rhizome_data/pickles/rhizome_artworks_extra_text.pkl')\n",
    "rhz_artworks_extra_text = rhz_artworks_extra_text.astype('string')\n",
    "\n",
    "#remove all punctuation\n",
    "rhz_artworks_extra_text['Text'] = rhz_artworks_extra_text['Text'].replace(regex=r'[^\\w\\s]', value='')\n",
    "\n",
    "#create a list of strings to remove from all scrapes\n",
    "remove = ['description editnntttt', '[', ']', 'nnn', 'description edit', 'editnn', 'summary edit', 'tttt', 'nn']\n",
    "for char in remove:\n",
    "    rhz_artworks_extra_text['Text'] = rhz_artworks_extra_text['Text'].str.replace(char, '')\n",
    "\n",
    "#export cleaned version to pickle\n",
    "rhz_artworks_extra_text_clean = rhz_artworks_extra_text.copy()\n",
    "rhz_artworks_extra_text_clean.to_pickle(path+'Rhizome_data/pickles/rhizome_artworks_extra_text_clean.pkl')\n",
    "\n",
    "# create a list of stop words and add custom stopwords\n",
    "stop_words = set(stopwords.words(\"english\"))\n",
    "new_words =  ['rhizome', 'attributed', 'summary', 'inception', 'staff', 'attribution', 'summary', 'legacy', 'nattributed']\n",
    "stop_words = stop_words.union(new_words)\n",
    "\n",
    "#remove stopwords from cleanedtext \n",
    "rhz_artworks_extra_text_clean['Text'] = rhz_artworks_extra_text_clean['Text'].str.lower()\n",
    "rhz_artworks_extra_text_clean['Text'] = rhz_artworks_extra_text_clean['Text'].apply(lambda x: ' '.join([item for item in x.split() if item not in stop_words]))\n",
    "#remove errand string\n",
    "rhz_artworks_extra_text_clean['Text'] = rhz_artworks_extra_text_clean['Text'].replace(regex={r'.attributed.': '', r'.legacy descriptive tags.': '', r'.attribution: staff.': '', r'.attribution.': '', r'.inception.': '', r'http[a-z]*\\s': ' '})\n",
    "\n",
    "#extract kws and assign them to new column\n",
    "rhz_artworks_extra_text_clean['Keywords'] = rhz_artworks_extra_text_clean.apply(lambda row:get_keywords(row), axis=1) \n",
    "\n",
    "#export new pickle for reuse\n",
    "rhz_artworks_extra_text_clean.to_pickle(path+'Rhizome_data/rhizome_artworks_extra_text_clean_stop_keywords.pkl')\n",
    " \"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>URL</th>\n",
       "      <th>Title</th>\n",
       "      <th>Artist</th>\n",
       "      <th>dateAcquired</th>\n",
       "      <th>dateCreated</th>\n",
       "      <th>Nationality</th>\n",
       "      <th>Gender</th>\n",
       "      <th>Text</th>\n",
       "      <th>Keywords</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>879</td>\n",
       "      <td>https://artbase.rhizome.org/wiki/Q2423</td>\n",
       "      <td>ZUR FARBENLEHRE (THEORY OF COLOURS)</td>\n",
       "      <td>Steven Jones</td>\n",
       "      <td>2007</td>\n",
       "      <td>2007</td>\n",
       "      <td>British</td>\n",
       "      <td>M</td>\n",
       "      <td>short filmnwhen eye sees colour immediately ex...</td>\n",
       "      <td>colour, march, eye, sees, immediately, excited...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1020</td>\n",
       "      <td>https://artbase.rhizome.org/wiki/Q4089</td>\n",
       "      <td>Zones de Convergence</td>\n",
       "      <td>cicero</td>\n",
       "      <td>2005</td>\n",
       "      <td>2005</td>\n",
       "      <td>missing</td>\n",
       "      <td>missing</td>\n",
       "      <td>zones de convergence treats days summit g8 evi...</td>\n",
       "      <td>summit, art, august, zones, de, convergence, t...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>243, 701</td>\n",
       "      <td>https://artbase.rhizome.org/wiki/Q1475</td>\n",
       "      <td>Zombie and Mummy</td>\n",
       "      <td>Dragan Espenschied, Olia Lialina</td>\n",
       "      <td>2004</td>\n",
       "      <td>2002</td>\n",
       "      <td>German, Russian</td>\n",
       "      <td>M, F</td>\n",
       "      <td>zombie mummy website hosted episodic online co...</td>\n",
       "      <td>zombie, site, olia, espenschied, mummy, advent...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>312</td>\n",
       "      <td>https://artbase.rhizome.org/wiki/Q4374</td>\n",
       "      <td>Zaira, City of Memories</td>\n",
       "      <td>Gokcen Erguven</td>\n",
       "      <td>2004</td>\n",
       "      <td>2004</td>\n",
       "      <td>Turkish</td>\n",
       "      <td>F</td>\n",
       "      <td>project based solely italo calvinos novel call...</td>\n",
       "      <td>cities, content, city, zaira, project, based, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>920</td>\n",
       "      <td>https://artbase.rhizome.org/wiki/Q3972</td>\n",
       "      <td>Z_G [zeitgeist gestalten]</td>\n",
       "      <td>Tiago Borges</td>\n",
       "      <td>2008</td>\n",
       "      <td>2007</td>\n",
       "      <td>Angolan</td>\n",
       "      <td>M</td>\n",
       "      <td>looking den zeitgeist gestalten design spirit ...</td>\n",
       "      <td>experience, web, page, den, zeitgeist, gestalt...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         ID                                     URL  \\\n",
       "0       879  https://artbase.rhizome.org/wiki/Q2423   \n",
       "1      1020  https://artbase.rhizome.org/wiki/Q4089   \n",
       "2  243, 701  https://artbase.rhizome.org/wiki/Q1475   \n",
       "3       312  https://artbase.rhizome.org/wiki/Q4374   \n",
       "4       920  https://artbase.rhizome.org/wiki/Q3972   \n",
       "\n",
       "                                 Title                            Artist  \\\n",
       "0  ZUR FARBENLEHRE (THEORY OF COLOURS)                      Steven Jones   \n",
       "1                 Zones de Convergence                            cicero   \n",
       "2                     Zombie and Mummy  Dragan Espenschied, Olia Lialina   \n",
       "3              Zaira, City of Memories                    Gokcen Erguven   \n",
       "4            Z_G [zeitgeist gestalten]                      Tiago Borges   \n",
       "\n",
       "  dateAcquired dateCreated      Nationality   Gender  \\\n",
       "0         2007        2007          British        M   \n",
       "1         2005        2005          missing  missing   \n",
       "2         2004        2002  German, Russian     M, F   \n",
       "3         2004        2004         Turkish         F   \n",
       "4         2008        2007          Angolan        M   \n",
       "\n",
       "                                                Text  \\\n",
       "0  short filmnwhen eye sees colour immediately ex...   \n",
       "1  zones de convergence treats days summit g8 evi...   \n",
       "2  zombie mummy website hosted episodic online co...   \n",
       "3  project based solely italo calvinos novel call...   \n",
       "4  looking den zeitgeist gestalten design spirit ...   \n",
       "\n",
       "                                            Keywords  \n",
       "0  colour, march, eye, sees, immediately, excited...  \n",
       "1  summit, art, august, zones, de, convergence, t...  \n",
       "2  zombie, site, olia, espenschied, mummy, advent...  \n",
       "3  cities, content, city, zaira, project, based, ...  \n",
       "4  experience, web, page, den, zeitgeist, gestalt...  "
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#show results\n",
    "rhz_artworks_extra_text_clean = pd.read_pickle(path+'Rhizome_data/pickles/rhizome_artworks_extra_text_clean_stop_keywords.pkl')\n",
    "rhz_artworks_extra_text_clean.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For MoMA the scrapes were cleaner of encoding noise so we simply ran everything through the same stopwords (and a different set of custom stopwords) and then removed empty entries. Out of over 138K artworks in MoMA's dataset we found text for 2,880 items.<br><br>\n",
    "**NB**: while scraping one of the larger departments we made an error that resulted in the dataframe including only the extracted text (with the other original information dropped). Rather than rescrape everything we kept this DF but subjected it to slightly different processing as seen below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\" #COMMENTED TO AVOID BINDER RUNNING IT - load DFs with text\n",
    "moma_arch_cont_text = pd.read_pickle(path+'MOMA_data/pickle/departments/architecture_design_cont_text.pkl')\n",
    "moma_arch_mod_text = pd.read_pickle(path+'MOMA_data/pickle/departments/architecture_design_mod_text_only.pkl')\n",
    "moma_draw_cont_text = pd.read_pickle(path+'MOMA_data/pickle/departments/draws_prints_cont_text.pkl')\n",
    "moma_draw_mod_text = pd.read_pickle(path+'MOMA_data/pickle/departments/draws_prints_mod_text.pkl')\n",
    "moma_films_cont_text = pd.read_pickle(path+'MOMA_data/pickle/departments/films_cont_text.pkl')\n",
    "moma_films_mod_text = pd.read_pickle(path+'MOMA_data/pickle/departments/films_mod_text.pkl')\n",
    "moma_media_cont_text = pd.read_pickle(path+'MOMA_data/pickle/departments/media_perf_cont_text.pkl')\n",
    "moma_media_mod_text = pd.read_pickle(path+'MOMA_data/pickle/departments/media_perf_mod_text.pkl')\n",
    "moma_paint_cont_text = pd.read_pickle(path+'MOMA_data/pickle/departments/paint_sculp_cont_text.pkl')\n",
    "moma_paint_mod_text = pd.read_pickle(path+'MOMA_data/pickle/departments/paint_sculp_mod_text.pkl')\n",
    "moma_photo_cont_text = pd.read_pickle(path+'MOMA_data/pickle/departments/photo_cont_text.pkl')\n",
    "moma_photo_mod_text = pd.read_pickle(path+'MOMA_data/pickle/departments/photo_mod_text.pkl')\n",
    "\n",
    "#DFs to list then clean them \n",
    "moma_texts = [moma_arch_cont_text, moma_draw_cont_text, moma_draw_mod_text, moma_films_cont_text, moma_films_mod_text, moma_media_cont_text, moma_media_mod_text, moma_paint_cont_text, moma_paint_mod_text, moma_photo_cont_text, moma_photo_mod_text]\n",
    "\n",
    "remove = ['[', ']', '</p>', '<p>', '<strong>', '</strong>', '<em>', '</em>', '</br>']\n",
    "for text in moma_texts:\n",
    "    for char in remove:\n",
    "        text['Text'] = text['Text'].str.replace(char, '')\n",
    "\n",
    "#exception for DF that was formatted differently due to scraping error (it has one column only w/ scraped text)\n",
    "for char in remove:\n",
    "    moma_arch_mod_text['text'] = moma_arch_mod_text['text'].str.replace(char, '')\n",
    "\n",
    "#replace empty strings w/ missing value to filter useful results only\n",
    "moma_arch_cont_text['Text'] = moma_arch_cont_text['Text'].replace([''], 'missing')\n",
    "moma_arch_mod_text['text'] = moma_arch_mod_text['text'].replace([''], 'missing')\n",
    "moma_draw_cont_text['Text'] = moma_draw_cont_text['Text'].replace([''], 'missing')\n",
    "moma_draw_mod_text['Text'] = moma_draw_mod_text['Text'].replace([''], 'missing')\n",
    "moma_films_cont_text['Text'] = moma_films_cont_text['Text'].replace([''], 'missing')\n",
    "moma_films_mod_text['Text'] = moma_films_mod_text['Text'].replace([''], 'missing')\n",
    "moma_media_cont_text['Text'] = moma_media_cont_text['Text'].replace([''], 'missing')\n",
    "moma_media_mod_text['Text'] = moma_media_mod_text['Text'].replace([''], 'missing')\n",
    "moma_paint_cont_text['Text'] = moma_paint_cont_text['Text'].replace([''], 'missing')\n",
    "moma_paint_mod_text['Text'] = moma_paint_mod_text['Text'].replace([''], 'missing')\n",
    "moma_photo_cont_text['Text'] = moma_photo_cont_text['Text'].replace([''], 'missing')\n",
    "moma_photo_mod_text['Text'] = moma_photo_mod_text['Text'].replace([''], 'missing')\n",
    "\n",
    "#save cleaned DFs back \n",
    "moma_arch_cont_text.to_pickle(path+'MOMA_data/pickle/departments/architecture_design_cont_text.pkl')\n",
    "moma_arch_mod_text.to_pickle(path+'MOMA_data/pickle/departments/architecture_design_mod_text_only.pkl')\n",
    "moma_draw_cont_text.to_pickle(path+'MOMA_data/pickle/departments/draws_prints_cont_text.pkl')\n",
    "moma_draw_mod_text.to_pickle(path+'MOMA_data/pickle/departments/draws_prints_mod_text.pkl')\n",
    "moma_films_cont_text.to_pickle(path+'MOMA_data/pickle/departments/films_cont_text.pkl')\n",
    "moma_films_mod_text.to_pickle(path+'MOMA_data/pickle/departments/films_mod_text.pkl')\n",
    "moma_media_cont_text.to_pickle(path+'MOMA_data/pickle/departments/media_perf_cont_text.pkl')\n",
    "moma_media_mod_text.to_pickle(path+'MOMA_data/pickle/departments/media_perf_mod_text.pkl')\n",
    "moma_paint_cont_text.to_pickle(path+'MOMA_data/pickle/departments/paint_sculp_cont_text.pkl')\n",
    "moma_paint_mod_text.to_pickle(path+'MOMA_data/pickle/departments/paint_sculp_mod_text.pkl')\n",
    "moma_photo_cont_text.to_pickle(path+'MOMA_data/pickle/departments/photo_cont_text.pkl')\n",
    "moma_photo_mod_text.to_pickle(path+'MOMA_data/pickle/departments/photo_mod_text.pkl')\n",
    "\n",
    "#reduce DFs down to entries w/ only useful text and remove duplicates \n",
    "moma_arch_cont_text_final = moma_arch_cont_text.loc[moma_arch_cont_text['Text'] != 'missing']\n",
    "moma_arch_cont_text_final = moma_arch_cont_text_final.drop_duplicates(subset=['Text'])\n",
    "moma_arch_mod_text_final = moma_arch_mod_text.loc[moma_arch_mod_text['text'] != 'missing']\n",
    "moma_arch_mod_text_final = moma_arch_mod_text_final.drop_duplicates(subset=['text'])\n",
    "moma_draw_cont_text_final = moma_draw_cont_text.loc[moma_draw_cont_text['Text'] != 'missing']\n",
    "moma_draw_cont_text_final = moma_draw_cont_text_final.drop_duplicates(subset=['Text'])\n",
    "moma_draw_mod_text_final = moma_draw_mod_text.loc[moma_draw_mod_text['Text'] != 'missing']\n",
    "moma_draw_mod_text_final = moma_draw_mod_text_final.drop_duplicates(subset=['Text'])\n",
    "moma_films_cont_text_final = moma_films_cont_text.loc[moma_films_cont_text['Text'] != 'missing']\n",
    "moma_films_cont_text_final = moma_films_cont_text_final.drop_duplicates(subset=['Text'])\n",
    "moma_films_mod_text_final = moma_films_mod_text.loc[moma_films_mod_text['Text'] != 'missing']\n",
    "moma_films_mod_text_final = moma_films_mod_text_final.drop_duplicates(subset=['Text'])\n",
    "moma_media_cont_text_final = moma_media_cont_text.loc[moma_media_cont_text['Text'] != 'missing']\n",
    "moma_media_cont_text_final = moma_media_cont_text_final.drop_duplicates(subset=['Text'])\n",
    "moma_media_mod_text_final = moma_media_mod_text.loc[moma_media_mod_text['Text'] != 'missing']\n",
    "moma_media_mod_text_final = moma_media_mod_text_final.drop_duplicates(subset=['Text'])\n",
    "moma_paint_cont_text_final = moma_paint_cont_text.loc[moma_paint_cont_text['Text'] != 'missing']\n",
    "moma_paint_cont_text_final = moma_paint_cont_text_final.drop_duplicates(subset=['Text'])\n",
    "moma_paint_mod_text_final = moma_paint_mod_text.loc[moma_paint_mod_text['Text'] != 'missing']\n",
    "moma_paint_mod_text_final = moma_paint_mod_text_final.drop_duplicates(subset=['Text'])\n",
    "moma_photo_cont_text_final = moma_photo_cont_text.loc[moma_photo_cont_text['Text'] != 'missing']\n",
    "moma_photo_cont_text_final = moma_photo_cont_text_final.drop_duplicates(subset=['Text'])\n",
    "moma_photo_mod_text_final = moma_photo_mod_text.loc[moma_photo_mod_text['Text'] != 'missing']\n",
    "moma_photo_mod_text_final = moma_photo_mod_text_final.drop_duplicates(subset=['Text'])\n",
    "\n",
    "#save DFs\n",
    "moma_arch_cont_text_final.to_pickle(path+'MOMA_data/pickle/departments/architecture_design_cont_text_final.pkl')\n",
    "moma_arch_mod_text_final.to_pickle(path+'MOMA_data/pickle/departments/architecture_design_mod_text_only_final.pkl')\n",
    "moma_draw_cont_text_final.to_pickle(path+'MOMA_data/pickle/departments/draws_prints_cont_text_final.pkl')\n",
    "moma_draw_mod_text_final.to_pickle(path+'MOMA_data/pickle/departments/draws_prints_mod_text_final.pkl')\n",
    "moma_films_cont_text_final.to_pickle(path+'MOMA_data/pickle/departments/films_cont_text_final.pkl')\n",
    "moma_films_mod_text_final.to_pickle(path+'MOMA_data/pickle/departments/films_mod_text_final.pkl')\n",
    "moma_media_cont_text_final.to_pickle(path+'MOMA_data/pickle/departments/media_perf_cont_text_final.pkl')\n",
    "moma_media_mod_text_final.to_pickle(path+'MOMA_data/pickle/departments/media_perf_mod_text_final.pkl')\n",
    "moma_paint_cont_text_final.to_pickle(path+'MOMA_data/pickle/departments/paint_sculp_cont_text_final.pkl')\n",
    "moma_paint_mod_text_final.to_pickle(path+'MOMA_data/pickle/departments/paint_sculp_mod_text_final.pkl')\n",
    "moma_photo_cont_text_final.to_pickle(path+'MOMA_data/pickle/departments/photo_cont_text_final.pkl')\n",
    "moma_photo_mod_text_final.to_pickle(path+'MOMA_data/pickle/departments/photo_mod_text_final.pkl')\n",
    "\n",
    "#load stopwords and remove them\n",
    "stop_words = set(stopwords.words(\"english\"))\n",
    "new_words =  ['.', 'one', 'two', 'also']\n",
    "stop_words = stop_words.union(new_words)\n",
    "\n",
    "moma_texts_final = [moma_arch_cont_text_final, moma_draw_cont_text_final, moma_draw_mod_text_final, moma_films_cont_text_final, moma_films_mod_text_final, moma_media_cont_text_final, moma_media_mod_text_final, moma_paint_cont_text_final, moma_paint_mod_text_final, moma_photo_cont_text_final, moma_photo_mod_text_final]\n",
    "\n",
    "for text in moma_texts_final:\n",
    "    text['Text'] = text['Text'].str.lower()\n",
    "    text['Text'] = text['Text'].apply(lambda x: ' '.join([item for item in x.split() if item not in stop_words]))\n",
    "\n",
    "#exception \n",
    "moma_arch_mod_text_final['text'] = moma_arch_mod_text_final['text'].str.lower()\n",
    "moma_arch_mod_text_final['text'] = moma_arch_mod_text_final['text'].apply(lambda x: ' '.join([item for item in x.split() if item not in stop_words]))\n",
    "\n",
    "#get keywords\n",
    "moma_arch_cont_text_final['Keywords'] = moma_arch_cont_text_final.apply(lambda row:get_keywords(row), axis=1)\n",
    "moma_draw_cont_text_final['Keywords'] = moma_draw_cont_text_final.apply(lambda row:get_keywords(row), axis=1) \n",
    "moma_draw_mod_text_final['Keywords'] = moma_draw_mod_text_final.apply(lambda row:get_keywords(row), axis=1) \n",
    "moma_films_cont_text_final['Keywords'] = moma_films_cont_text_final.apply(lambda row:get_keywords(row), axis=1)\n",
    "moma_films_mod_text_final['Keywords'] = moma_films_mod_text_final.apply(lambda row:get_keywords(row), axis=1) \n",
    "moma_media_cont_text_final['Keywords'] = moma_media_cont_text_final.apply(lambda row:get_keywords(row), axis=1)\n",
    "moma_media_mod_text_final['Keywords'] = moma_media_mod_text_final.apply(lambda row:get_keywords(row), axis=1)\n",
    "moma_paint_cont_text_final['Keywords'] = moma_paint_cont_text_final.apply(lambda row:get_keywords(row), axis=1)\n",
    "moma_paint_mod_text_final['Keywords'] = moma_paint_mod_text_final.apply(lambda row:get_keywords(row), axis=1)\n",
    "moma_photo_cont_text_final['Keywords'] = moma_photo_cont_text_final.apply(lambda row:get_keywords(row), axis=1)\n",
    "moma_photo_mod_text_final['Keywords'] = moma_photo_mod_text_final.apply(lambda row:get_keywords(row), axis=1)\n",
    "moma_arch_mod_text_final['Keywords'] = moma_arch_mod_text_final.apply(lambda row:get_keywords_2(row), axis=1) \n",
    "\n",
    "#pickle version w/ stopwords removed and keywords added\n",
    "moma_arch_cont_text_final.to_pickle(path+'MOMA_data/pickle/departments/architecture_design_cont_text_final_stop.pkl')\n",
    "moma_arch_mod_text_final.to_pickle(path+'MOMA_data/pickle/departments/architecture_design_mod_text_only_final_stop.pkl')\n",
    "moma_draw_cont_text_final.to_pickle(path+'MOMA_data/pickle/departments/draws_prints_cont_text_final_stop.pkl')\n",
    "moma_draw_mod_text_final.to_pickle(path+'MOMA_data/pickle/departments/draws_prints_mod_text_final_stop.pkl')\n",
    "moma_films_cont_text_final.to_pickle(path+'MOMA_data/pickle/departments/films_cont_text_final_stop.pkl')\n",
    "moma_films_mod_text_final.to_pickle(path+'MOMA_data/pickle/departments/films_mod_text_final_stop.pkl')\n",
    "moma_media_cont_text_final.to_pickle(path+'MOMA_data/pickle/departments/media_perf_cont_text_final_stop.pkl')\n",
    "moma_media_mod_text_final.to_pickle(path+'MOMA_data/pickle/departments/media_perf_mod_text_final_stop.pkl')\n",
    "moma_paint_cont_text_final.to_pickle(path+'MOMA_data/pickle/departments/paint_sculp_cont_text_final_stop.pkl')\n",
    "moma_paint_mod_text_final.to_pickle(path+'MOMA_data/pickle/departments/paint_sculp_mod_text_final_stop.pkl')\n",
    "moma_photo_cont_text_final.to_pickle(path+'MOMA_data/pickle/departments/photo_cont_text_final_stop.pkl')\n",
    "moma_photo_mod_text_final.to_pickle(path+'MOMA_data/pickle/departments/photo_mod_text_final_stop.pkl')\n",
    " \"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Title</th>\n",
       "      <th>Artist</th>\n",
       "      <th>ID</th>\n",
       "      <th>DateCreated</th>\n",
       "      <th>Medium</th>\n",
       "      <th>Department</th>\n",
       "      <th>DateAcquired</th>\n",
       "      <th>URL</th>\n",
       "      <th>ThumbnailURL</th>\n",
       "      <th>Nationality</th>\n",
       "      <th>Gender</th>\n",
       "      <th>Text</th>\n",
       "      <th>Keywords</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>The Manhattan Transcripts Project, New York, N...</td>\n",
       "      <td>Bernard Tschumi</td>\n",
       "      <td>7056</td>\n",
       "      <td>1980</td>\n",
       "      <td>Photographic reproduction with colored synthet...</td>\n",
       "      <td>Architecture &amp; Design</td>\n",
       "      <td>1995</td>\n",
       "      <td>http://www.moma.org/collection/works/5</td>\n",
       "      <td>http://www.moma.org/media/W1siZiIsIjEyNCJdLFsi...</td>\n",
       "      <td>missing</td>\n",
       "      <td>M</td>\n",
       "      <td>manhattan transcripts theoretical propositions...</td>\n",
       "      <td>events, architecture, manhattan, transcripts, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>The Manhattan Transcripts Project, New York, N...</td>\n",
       "      <td>Bernard Tschumi</td>\n",
       "      <td>7056</td>\n",
       "      <td>1980</td>\n",
       "      <td>Photographic reproduction with colored synthet...</td>\n",
       "      <td>Architecture &amp; Design</td>\n",
       "      <td>1995</td>\n",
       "      <td>http://www.moma.org/collection/works/38</td>\n",
       "      <td>http://www.moma.org/media/W1siZiIsIjI2NyJdLFsi...</td>\n",
       "      <td>missing</td>\n",
       "      <td>M</td>\n",
       "      <td>manhattan transcripts theoretical propositions...</td>\n",
       "      <td>events, architecture, manhattan, movement, tra...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>151</th>\n",
       "      <td>Slow House Project, North Haven, New York, Pla...</td>\n",
       "      <td>Diller + Scofidio, Elizabeth Diller, Ricardo S...</td>\n",
       "      <td>8707, 6951, 6952</td>\n",
       "      <td>1989</td>\n",
       "      <td>Computer-generated print on frosted polymer sh...</td>\n",
       "      <td>Architecture &amp; Design</td>\n",
       "      <td>1992</td>\n",
       "      <td>http://www.moma.org/collection/works/201</td>\n",
       "      <td>http://www.moma.org/media/W1siZiIsIjE5MjQiXSxb...</td>\n",
       "      <td>American, American, American</td>\n",
       "      <td>missing, F, M</td>\n",
       "      <td>elizabeth diller ricardo scofidio's slow house...</td>\n",
       "      <td>house, drawing, view, diller, scofidio, city, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>152</th>\n",
       "      <td>The Peak Project, Hong Kong, China (Exterior p...</td>\n",
       "      <td>Zaha Hadid</td>\n",
       "      <td>6953</td>\n",
       "      <td>1991</td>\n",
       "      <td>Synthetic polymer on paper mounted on canvas</td>\n",
       "      <td>Architecture &amp; Design</td>\n",
       "      <td>1992</td>\n",
       "      <td>http://www.moma.org/collection/works/202</td>\n",
       "      <td>http://www.moma.org/media/W1siZiIsIjE5MzgiXSxb...</td>\n",
       "      <td>British</td>\n",
       "      <td>F</td>\n",
       "      <td>painting depicts hadid’s winning entry archite...</td>\n",
       "      <td>hadid, painting, hills, structure, forms, winn...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>170</th>\n",
       "      <td>Chikatsu-Asuka Historical Museum, Minami-Kawac...</td>\n",
       "      <td>Tadao Ando</td>\n",
       "      <td>7055</td>\n",
       "      <td>1989</td>\n",
       "      <td>Graphite and crayon with scoring on paper</td>\n",
       "      <td>Architecture &amp; Design</td>\n",
       "      <td>1993</td>\n",
       "      <td>http://www.moma.org/collection/works/224</td>\n",
       "      <td>http://www.moma.org/media/W1siZiIsIjI0MjEiXSxb...</td>\n",
       "      <td>Japanese</td>\n",
       "      <td>M</td>\n",
       "      <td>tadao ando's drawings like hieroglyphic map re...</td>\n",
       "      <td>ando, museum, building, landscape, tadao, like...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                 Title  \\\n",
       "3    The Manhattan Transcripts Project, New York, N...   \n",
       "35   The Manhattan Transcripts Project, New York, N...   \n",
       "151  Slow House Project, North Haven, New York, Pla...   \n",
       "152  The Peak Project, Hong Kong, China (Exterior p...   \n",
       "170  Chikatsu-Asuka Historical Museum, Minami-Kawac...   \n",
       "\n",
       "                                                Artist                ID  \\\n",
       "3                                      Bernard Tschumi              7056   \n",
       "35                                     Bernard Tschumi              7056   \n",
       "151  Diller + Scofidio, Elizabeth Diller, Ricardo S...  8707, 6951, 6952   \n",
       "152                                         Zaha Hadid              6953   \n",
       "170                                         Tadao Ando              7055   \n",
       "\n",
       "     DateCreated                                             Medium  \\\n",
       "3           1980  Photographic reproduction with colored synthet...   \n",
       "35          1980  Photographic reproduction with colored synthet...   \n",
       "151         1989  Computer-generated print on frosted polymer sh...   \n",
       "152         1991       Synthetic polymer on paper mounted on canvas   \n",
       "170         1989          Graphite and crayon with scoring on paper   \n",
       "\n",
       "                Department DateAcquired  \\\n",
       "3    Architecture & Design         1995   \n",
       "35   Architecture & Design         1995   \n",
       "151  Architecture & Design         1992   \n",
       "152  Architecture & Design         1992   \n",
       "170  Architecture & Design         1993   \n",
       "\n",
       "                                          URL  \\\n",
       "3      http://www.moma.org/collection/works/5   \n",
       "35    http://www.moma.org/collection/works/38   \n",
       "151  http://www.moma.org/collection/works/201   \n",
       "152  http://www.moma.org/collection/works/202   \n",
       "170  http://www.moma.org/collection/works/224   \n",
       "\n",
       "                                          ThumbnailURL  \\\n",
       "3    http://www.moma.org/media/W1siZiIsIjEyNCJdLFsi...   \n",
       "35   http://www.moma.org/media/W1siZiIsIjI2NyJdLFsi...   \n",
       "151  http://www.moma.org/media/W1siZiIsIjE5MjQiXSxb...   \n",
       "152  http://www.moma.org/media/W1siZiIsIjE5MzgiXSxb...   \n",
       "170  http://www.moma.org/media/W1siZiIsIjI0MjEiXSxb...   \n",
       "\n",
       "                      Nationality         Gender  \\\n",
       "3                         missing              M   \n",
       "35                        missing              M   \n",
       "151  American, American, American  missing, F, M   \n",
       "152                       British              F   \n",
       "170                      Japanese              M   \n",
       "\n",
       "                                                  Text  \\\n",
       "3    manhattan transcripts theoretical propositions...   \n",
       "35   manhattan transcripts theoretical propositions...   \n",
       "151  elizabeth diller ricardo scofidio's slow house...   \n",
       "152  painting depicts hadid’s winning entry archite...   \n",
       "170  tadao ando's drawings like hieroglyphic map re...   \n",
       "\n",
       "                                              Keywords  \n",
       "3    events, architecture, manhattan, transcripts, ...  \n",
       "35   events, architecture, manhattan, movement, tra...  \n",
       "151  house, drawing, view, diller, scofidio, city, ...  \n",
       "152  hadid, painting, hills, structure, forms, winn...  \n",
       "170  ando, museum, building, landscape, tadao, like...  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#show results\n",
    "moma_arch_cont_text_final = pd.read_pickle(path+'MOMA_data/pickle/departments/architecture_design_cont_text_final_stop.pkl')\n",
    "moma_arch_cont_text_final.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Manipulating and Visualising the Data (Everyone)\n",
    "\n",
    "For the manipulation and visualisation of the data we decided on a workflow inspired by what we learnt during the class. Starting from our initial research question -- Did the internet and personal computers have an impact on the make up of art collections? -- we looked at the types of variables we had to work with, their potential relationships, and the ways in which this could help us explore and visualize the data. <br><br>\n",
    "We decided on the following variables as essential to our exploration and storytelling: \n",
    "- Gender (categorical + tranformed into nominal)\n",
    "- Nationality (categorical + tranformed into nominal)\n",
    "- Date Created and Acquired (ordinal)\n",
    "- Departments (categorical)\n",
    "- Keywords (categorical + tranformed into numerical)\n",
    "\n",
    "We tried various permutations and combinations of these variables and in the end settled on using gender, nationalities, and keywords as the three lenses through which we would try to answer our research question and tell our story, while the time and department variables would be used as additional filters during the process. <br><br>\n",
    "Considering the difference in size between our datasets we spent some time discussing the best approach and settled on the following to help our manipulation, visualization, and storytelling: **the Rhizome dataset would be used in full, MoMA would only be used in full untouched for the first visualization and subsequently would only be used as either full with the 1983 pivot date (to see if the internet and personal computers might have made a difference in acquisitions) or as a trimmed-down version for comparison with Rhizome, aligned to Rhizome by using the same creation date for the first artwork (1983) and the same first acquisition date (2000)**.<br><br> \n",
    "Therefore we would in effect have three versions of the MoMA artworks dataset: the full version (138K items, 15K artists); a version split at 1983 as a pivot date (69,310 items vs 30,227, 7,053 artists vs 3,910); and a version trimmed alongside Rhizome's key dates (21,580 items, 3,492 artists). With this approach we felt we could more confidently look at the data and make potential observations.<br><br>\n",
    "As part of the cleaning work we decided that it would be useful to represent collectives and collaborations as a type of gender as they are relevant to Rhizome and help expand gender beyond the male/female binary. Similarly we found many artists in Rhizome with dual nationalities. In MoMA, collectives were not clearly marked but once we connected artworks to genders and nationalities we could see that some of them also had multiple artists, often due to collaborations or collected works. As such we decided to represent artworks with multiple creators as belonging to collectives and collaborations in terms of gender. Dual nationalities and those of collectives and collaborations were represented as best as possible with manual clean ups and precedence given to nationalities from the Global South if present or to the first nationality in the collaboration/collective.<br><br>\n",
    "From there we discussed what types of visualizations we might want -- comparisons, relationships, distribution, composition etc... -- and ended up with a list of questions expanding from the initial research question which became a guide for both creating visualizations and building the story. For the latter we also made use of Miro to create a storyboard of the website and help decide which sub question/visualization would prove useful. <br><br>\n",
    "The final list of questions and their associated viusalizations and datasets are:\n",
    "1. **What is the gender representation in each dataset?**\n",
    "    - Count artists. \n",
    "    - Rhizome vs MoMA full + MoMA before and after 1983.\n",
    "    - Pie charts -> gender as categorical and numerical. \n",
    "2. **How does gender representation compare between both datasets?** \n",
    "    - Count artists.\n",
    "    - Rhizome vs MoMA (sampled).\n",
    "    - Pie charts -> gender as categorical and numerical.\n",
    "3. **How many collectives & collaborations are there and what is their gender breakdown?** \n",
    "    - Count artists.\n",
    "    - Rhizome vs MoMA (sampled).\n",
    "    - Sunburst charts -> gender as categorical and numerical.\n",
    "4. **How does gender representation compare between datasets alongside departments?** \n",
    "    - Count artists. \n",
    "    - Rhizome vs MoMA (sampled) + MoMA before and after 1983.\n",
    "    - Barcharts -> gender as categorical and numerical + departments as categorical.\n",
    "5. **What is the nationality representation in each dataset?**\n",
    "    - Count occurences of nationalities. \n",
    "    - Rhizome vs MoMA (sampled). \n",
    "    - Map -> nationality as categorical and numerical.\n",
    "6. **How does nationality representation compare between datasets?** \n",
    "    - Count occurences of nationalities. \n",
    "    - Rhizome vs MoMA (sampled).\n",
    "    - Bar charts -> nationality as numerical + dataset as categorical (with additional filtering using the concept of Global North/South divide).\n",
    "7. **How does nationality relate to gender?** \n",
    "    - Count occurences of nationalities and gender.\n",
    "    - Rhizome vs MoMA (sampled).\n",
    "    - Parallel chart -> gender as categorical + department as categorical + nationality as categorical and numerical (with some additional filtering using the concept of Global North/South divide).\n",
    "8. **Which years have the highest acquisition date in relation to gender and/or nationality?**\n",
    "    - Count artworks. \n",
    "    - Rhizome vs MoMA (sampled). \n",
    "    - Scatter plot -> date created/acquired as ordinal + gender/nationality as categorical and numerical (with some additional filtering using the concept of Global North/South divide)\n",
    "9. **Which keywords are most prominent in artwork descriptions?** \n",
    "    - Count artworks. \n",
    "    - Rhizome vs MoMA (sampled).\n",
    "    - Tree maps -> keywords as ordinal + gender/nationality as categorical. \n",
    "10. **How do keywords relate to gender and/or nationality?**\n",
    "    - Same as above.\n",
    "\n",
    "Let's do this! 💪"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Representing Gender by Dataset, Comparing them, Visualizing Collectives Make Up (Margherita & Laurent)\n",
    "\n",
    "We decided to use pie and sunburst charts to give a simple overall perspective of the gender breakdown of each collection. For the sunburst charts we tried to automate the counting process but kept hitting some walls with regular expressions so in the end we exported the needed data to csv and did some regex and counting in Excel as the Plotly figure could be easily manuallly populated. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b62ee2e6107a4aad89ade7cd61c8eb58",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FigureWidget({\n",
       "    'data': [{'domain': {'x': [0.0, 1.0], 'y': [0.0, 1.0]},\n",
       "              'hover…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Rhizome vs MoMA full\n",
    "rhz_artists = rhz_artworks.drop_duplicates(subset='Artist', keep='last')\n",
    "rhz_gender_rep = group_genders(rhz_artists)\n",
    "moma_artists = moma_artworks.drop_duplicates(subset='Artist', keep='last')\n",
    "moma_gender_rep = group_genders(moma_artists)\n",
    "\n",
    "rhz_gender_pie = px.pie(rhz_gender_rep, values='Counts', names='Gender',\n",
    "             title='Gender Composition of Rhizome Collection', color_discrete_sequence = ['#007c91', \"#ff5252\", \"#8c7b75\", \"#ce93d8\", \"#ffd54f\",])\n",
    "\n",
    "rhz_gender_pie.update_traces(textposition='auto', textinfo='label+percent', hoverinfo='label+value+percent', hovertemplate=None, textfont=dict(size=13))\n",
    "rhz_gender_pie.update_layout(title_x=0.5)\n",
    "\n",
    "moma_gender_pie = px.pie(moma_gender_rep, values='Counts', names='Gender',\n",
    "             title='Gender Composition of MoMA Collection (Full)', color_discrete_sequence = ['#007c91', \"#ce93d8\", \"#ff5252\", \"#8c7b75\", \"#ffd54f\",])\n",
    "moma_gender_pie.update_traces(textposition='auto', textinfo='label+percent', hoverinfo='label+value+percent', hovertemplate=None, textfont=dict(size=13))\n",
    "moma_gender_pie.update_layout(title_x=0.5)\n",
    "\n",
    "#create local copies w/o titles \n",
    "#rhz_gender_pie.write_html(path+\"Plotly_embeds/rhz_full_pie.html\", include_plotlyjs='directory', full_html=False, default_width='100%')\n",
    "#moma_gender_pie.write_html(path+\"Plotly_embeds/moma_full_pie.html\", include_plotlyjs='directory', full_html=False, default_width='100%')\n",
    "\n",
    "rhz_gender_pie = go.FigureWidget(rhz_gender_pie)\n",
    "moma_gender_pie = go.FigureWidget(moma_gender_pie)\n",
    "gender_pies = ipw.HBox([rhz_gender_pie, moma_gender_pie])\n",
    "gender_pies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8e95d4c99b014259a4ed2f8768089deb",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FigureWidget({\n",
       "    'data': [{'domain': {'x': [0.0, 1.0], 'y': [0.0, 1.0]},\n",
       "              'hover…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Rhizome vs MoMA sampled\n",
    "moma_artists_compare = moma_rhz_compare.drop_duplicates(subset='Artist', keep='last')\n",
    "moma_rhz_compare_gender_rep = group_genders(moma_artists_compare)\n",
    "\n",
    "moma_gender_pie_2 = px.pie(moma_rhz_compare_gender_rep, values='Counts', names='Gender',\n",
    "             title='Gender Composition of MoMA Collection (Sampled)',\n",
    "             color_discrete_sequence = ['#007c91', \"#ff5252\", \"#ce93d8\", \"#8c7b75\", \"#ffd54f\"])\n",
    "moma_gender_pie_2.update_traces(textposition='auto', textinfo='label+percent', hoverinfo='label+value+percent', hovertemplate=None, sort=False, textfont=dict(size=13))\n",
    "moma_gender_pie_2.update_layout(title_x=0.5)\n",
    "\n",
    "#create local copies\n",
    "#moma_gender_pie_2.write_html(path+\"Plotly_embeds/moma_sampled_pie.html\", include_plotlyjs='directory',  full_html=False, default_width='100%')\n",
    "\n",
    "moma_gender_pie_2 = go.FigureWidget(moma_gender_pie_2)\n",
    "gender_pies_2 = ipw.HBox([rhz_gender_pie, moma_gender_pie_2])\n",
    "gender_pies_2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1d9d541b1df44513b20d7dc8c7d4a73f",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FigureWidget({\n",
       "    'data': [{'domain': {'x': [0.0, 1.0], 'y': [0.0, 1.0]},\n",
       "              'hover…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#create a DF w/ genders and counts, group collectives together\n",
    "moma_artists_old = moma_artworks_old.drop_duplicates(subset='Artist', keep='last')\n",
    "moma_b4_gender_rep = group_genders(moma_artists_old)\n",
    "moma_artists_new = moma_artworks_new.drop_duplicates(subset='Artist', keep='last')\n",
    "moma_after_gender_rep = group_genders(moma_artists_new)\n",
    "\n",
    "moma_gender_pie_b4 = px.pie(moma_b4_gender_rep, values='Counts', names='Gender',\n",
    "             title='Gender Composition of MoMA Collection Before 1983',\n",
    "            color_discrete_sequence = ['#007c91', \"#ff5252\",  \"#ce93d8\", \"#8c7b75\"])\n",
    "moma_gender_pie_b4.update_traces(textposition='auto', textinfo='label+percent', hoverinfo='label+value+percent', hovertemplate=None, textfont=dict(size=13))\n",
    "moma_gender_pie_b4.update_layout(title_x=0.5)\n",
    "\n",
    "moma_gender_pie_after = px.pie(moma_after_gender_rep, values='Counts', names='Gender',\n",
    "             title='Gender Composition of MoMA Collection After 1983',\n",
    "            color_discrete_sequence = ['#007c91', \"#ff5252\", \"#ce93d8\", \"#8c7b75\", \"#ffd54f\"]\n",
    ")\n",
    "moma_gender_pie_after.update_traces(textposition='auto', textinfo='label+percent', hoverinfo='label+value+percent', hovertemplate=None, textfont=dict(size=13))\n",
    "moma_gender_pie_after.update_layout(title_x=0.5)\n",
    "\n",
    "#create local copies\n",
    "#moma_gender_pie_b4.write_html(path+\"Plotly_embeds/moma_b4_pie.html\", include_plotlyjs='directory',  full_html=False, default_width='100%')\n",
    "#moma_gender_pie_after.write_html(path+\"Plotly_embeds/moma_after_pie.html\", include_plotlyjs='directory',  full_html=False, default_width='100%')\n",
    "\n",
    "moma_gender_pie_3 = go.FigureWidget(moma_gender_pie_b4)\n",
    "moma_gender_pie_4 = go.FigureWidget(moma_gender_pie_after)\n",
    "gender_pies_3 = ipw.HBox([moma_gender_pie_3, moma_gender_pie_4])\n",
    "gender_pies_3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "#count genders, remove M and F only, send to csv for counting\n",
    "moma_gender_counts = moma_artists_compare['Gender'].value_counts()\n",
    "rhz_gender_counts = rhz_artists['Gender'].value_counts()\n",
    "#choose only collectives\n",
    "moma_gender_collectives = moma_gender_counts[3:].reset_index()\n",
    "#moma_gender_collectives.to_csv(path+'MOMA_data/csv/moma_collectives_gender.csv')\n",
    "rhz_gender_collectives = rhz_gender_counts[3:].reset_index()\n",
    "#rhz_gender_collectives.to_csv(path+'Rhizome_data/csv/rhz_collectives_gender.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2cc3d5776b6c4ca1bf64fb35041b74e4",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FigureWidget({\n",
       "    'data': [{'hoverinfo': 'label+value',\n",
       "              'labels': [Collectives &…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Rhizome vs MoMA sampled - collectives gender rep \n",
    "#set color sequence\n",
    "scale1=['#ffffff', '#007c91','#007c91', '#007c91', '#ff5252' ,  \"#ff5252\",  \"#ff5252\",  \"#ce93d8\", \"#ce93d8\", \"#ffd54f\", \"#ffd54f\"]\n",
    "\n",
    "rhz_sun =go.Figure(go.Sunburst(\n",
    "    labels=[\"Collectives & Collabs\", \"Males\", \"Male Only\", \"Male & Missing\", \"Females\", \"Female Only\", \"Female & Missing\", \"Mixed Gender\", 'Incl. Missing Gender'],\n",
    "    parents=[\"\", \"Collectives & Collabs\",\"Males\", \"Males\", \"Collectives & Collabs\", \"Females\", \"Females\", \"Collectives & Collabs\", 'Mixed Gender'],\n",
    "    values=[71, 22, 20, 2, 7, 5, 2, 42, 2], \n",
    "),layout=dict(title=dict(text=\"Who Takes Part in Collectives & Collaborations? - Rhizome\")))\n",
    "rhz_sun.update_traces(textinfo='label+value', hoverinfo='label+value', hovertemplate=None, \n",
    "    marker=dict(colors = scale1))\n",
    "rhz_sun.update_layout(title_x=0.5, uniformtext=dict(minsize=8, mode='hide'))\n",
    "\n",
    "moma_sun =go.Figure(go.Sunburst(\n",
    "    labels=[\"Collectives & Collabs\", \"Males\", \"Male Only\", \"Male & Missing\", \"Females\", \"Female Only\", \"Female & Missing\", \"Mixed Gender\", 'Incl. Missing Gender', 'Incl. Non-Binary'],\n",
    "    parents=[\"\", \"Collectives & Collabs\",\"Males\", \"Males\", \"Collectives & Collabs\", \"Females\", \"Females\", \"Collectives & Collabs\", 'Mixed Gender', 'Mixed Gender'],\n",
    "    values=[744, 434, 195, 239, 62, 28, 34, 248, 117, 1]\n",
    "),layout=dict(title=dict(text=\"Who Takes Part in Collectives & Collaborations? - MoMA (sampled)\")))\n",
    "moma_sun.update_traces(textinfo='label+value', hoverinfo='label+value', hovertemplate=None,\n",
    "       marker=dict(colors = scale1))\n",
    "moma_sun.update_layout(title_x=0.5, uniformtext=dict(minsize=8, mode='hide'))\n",
    "\n",
    "#create local copies\n",
    "#rhz_sun.write_html(path+\"Plotly_embeds/rhz_sunburst.html\", include_plotlyjs='directory',  full_html=False, default_width='100%')\n",
    "#moma_sun.write_html(path+\"Plotly_embeds/moma_sunburst.html\", include_plotlyjs='directory',  full_html=False, default_width='100%')\n",
    "\n",
    "rhz_sun = go.FigureWidget(rhz_sun)\n",
    "moma_sun = go.FigureWidget(moma_sun)\n",
    "rhz_moma_sun = ipw.HBox([rhz_sun, moma_sun])\n",
    "rhz_moma_sun"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Representing Gender by Departments (Everyone)\n",
    "\n",
    "When it came to gender and departments we decided to use barcharts and facet the results to allow the user to see things more clearly: in the first chart the facet is department because they're even, and this way you can easily see if gender representation changes before and after 1983; in the second chart we used the databases as the facet because the departments have been reduced to three and this way you can more easily see differences in acquisitions between segments of the collections.<br><br>\n",
    "We also decided to use a shared logarithmic representation of the y axis for the first and last chart as we had some outliers in the Drawings & Prints department that made a standard barchart representation more difficult to read. <br><br>\n",
    "For the departments we started with the four MoMA departments we felt were most aligned to the types of art also found in Rhizome: film, photography, media and performance, and drawing and prints. The latter might seem odd but there are over 300 items in the Rhizome collection that make reference to print and drawing. To align them with Rhizome we first created four equivalent departments for Rhizome, using the description text to assign them, and then we combined photography and drawings & prints into still images for MoMA and for Rhizome we combined installations and net art into media and performance. **The result is three overall departments: still images (covers photography and print/drawings), moving images, and media and performance (covers net art)**.<br><br>\n",
    "A note on the Rhizome department assignment: the process is far from perfect as it looks for a specific keyword in the text and uses the first one it finds to assign the department. Therefore it misses out on artworks that could potentially belong to multiple departments but considering MoMA is strict with its department assignment we feel taking the same approach for Rhizome is appropriate. <br><br>\n",
    "Lastly, 7% of Rhizome artists (9% of artworks) do not have a department assigned after filtering (due to the keywords not appearing in the text or there not being any text associated with the artwork)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "#get gender counts, source, department \n",
    "moma_draw_before_artists = moma_draw_before.drop_duplicates(subset='Artist', keep='last')\n",
    "moma_draw_before_gender_rep = group_genders_dept(moma_draw_before_artists, 'Before 1983', 'Drawings & Prints')\n",
    "moma_draw_after_artists = moma_draw_after.drop_duplicates(subset='Artist', keep='last')\n",
    "moma_draw_after_gender_rep = group_genders_dept(moma_draw_after_artists, 'After 1983', 'Drawings & Prints')\n",
    "moma_films_before_artists = moma_films_before.drop_duplicates(subset='Artist', keep='last')\n",
    "moma_films_before_gender_rep = group_genders_dept(moma_films_before_artists, 'Before 1983', 'Film')\n",
    "moma_films_after_artists = moma_films_after.drop_duplicates(subset='Artist', keep='last')\n",
    "moma_films_after_gender_rep = group_genders_dept(moma_films_after_artists, 'After 1983', 'Film')\n",
    "moma_media_before_artists = moma_media_before.drop_duplicates(subset='Artist', keep='last')\n",
    "moma_media_before_gender_rep = group_genders_dept(moma_media_before_artists, 'Before 1983', 'Media and Performance')\n",
    "moma_media_after_artists = moma_media_after.drop_duplicates(subset='Artist', keep='last')\n",
    "moma_media_after_gender_rep = group_genders_dept(moma_media_after_artists, 'After 1983', 'Media and Performance')\n",
    "moma_photo_before_artists = moma_photo_before.drop_duplicates(subset='Artist', keep='last')\n",
    "moma_photo_before_gender_rep = group_genders_dept(moma_photo_before_artists, 'Before 1983', 'Photography')\n",
    "moma_photo_after_artists = moma_photo_after.drop_duplicates(subset='Artist', keep='last')\n",
    "moma_photo_after_gender_rep = group_genders_dept(moma_photo_after_artists, 'After 1983', 'Photography')\n",
    "\n",
    "#group all together\n",
    "moma_draw_gender_rep_all = pd.concat([moma_draw_before_gender_rep, moma_draw_after_gender_rep])\n",
    "moma_films_gender_rep_all = pd.concat([moma_films_before_gender_rep, moma_films_after_gender_rep])\n",
    "moma_media_gender_rep_all = pd.concat([moma_media_before_gender_rep, moma_media_after_gender_rep])\n",
    "moma_photo_gender_rep_all = pd.concat([moma_photo_before_gender_rep, moma_photo_after_gender_rep])\n",
    "moma_depts_gender_rep_all = pd.concat([moma_photo_gender_rep_all, moma_media_gender_rep_all, moma_films_gender_rep_all, moma_draw_gender_rep_all])\n",
    "moma_depts_gender_rep_all = moma_depts_gender_rep_all.reset_index(drop='True')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "alignmentgroup": "True",
         "hoverinfo": "x+y+name",
         "legendgroup": "Before 1983",
         "marker": {
          "color": "#49599a",
          "pattern": {
           "shape": ""
          }
         },
         "name": "Before 1983",
         "offsetgroup": "Before 1983",
         "orientation": "v",
         "showlegend": true,
         "text": [
          1319,
          246,
          139,
          107
         ],
         "textposition": "auto",
         "type": "bar",
         "x": [
          "Male",
          "Female",
          "Missing",
          "Collectives & Collabs"
         ],
         "xaxis": "x3",
         "y": [
          1319,
          246,
          139,
          107
         ],
         "yaxis": "y3"
        },
        {
         "alignmentgroup": "True",
         "hoverinfo": "x+y+name",
         "legendgroup": "Before 1983",
         "marker": {
          "color": "#49599a",
          "pattern": {
           "shape": ""
          }
         },
         "name": "Before 1983",
         "offsetgroup": "Before 1983",
         "orientation": "v",
         "showlegend": false,
         "text": [
          116,
          62,
          59,
          8
         ],
         "textposition": "auto",
         "type": "bar",
         "x": [
          "Male",
          "Female",
          "Collectives & Collabs",
          "Missing"
         ],
         "xaxis": "x4",
         "y": [
          116,
          62,
          59,
          8
         ],
         "yaxis": "y4"
        },
        {
         "alignmentgroup": "True",
         "hoverinfo": "x+y+name",
         "legendgroup": "Before 1983",
         "marker": {
          "color": "#49599a",
          "pattern": {
           "shape": ""
          }
         },
         "name": "Before 1983",
         "offsetgroup": "Before 1983",
         "orientation": "v",
         "showlegend": false,
         "text": [
          167,
          26,
          21,
          14
         ],
         "textposition": "auto",
         "type": "bar",
         "x": [
          "Male",
          "Collectives & Collabs",
          "Female",
          "Missing"
         ],
         "xaxis": "x",
         "y": [
          167,
          26,
          21,
          14
         ],
         "yaxis": "y"
        },
        {
         "alignmentgroup": "True",
         "hoverinfo": "x+y+name",
         "legendgroup": "Before 1983",
         "marker": {
          "color": "#49599a",
          "pattern": {
           "shape": ""
          }
         },
         "name": "Before 1983",
         "offsetgroup": "Before 1983",
         "orientation": "v",
         "showlegend": false,
         "text": [
          3224,
          788,
          543,
          214
         ],
         "textposition": "auto",
         "type": "bar",
         "x": [
          "Male",
          "Collectives & Collabs",
          "Female",
          "Missing"
         ],
         "xaxis": "x2",
         "y": [
          3224,
          788,
          543,
          214
         ],
         "yaxis": "y2"
        },
        {
         "alignmentgroup": "True",
         "hoverinfo": "x+y+name",
         "legendgroup": "After 1983",
         "marker": {
          "color": "#aab6fe",
          "pattern": {
           "shape": ""
          }
         },
         "name": "After 1983",
         "offsetgroup": "After 1983",
         "orientation": "v",
         "showlegend": true,
         "text": [
          410,
          222,
          14,
          13
         ],
         "textposition": "auto",
         "type": "bar",
         "x": [
          "Male",
          "Female",
          "Missing",
          "Collectives & Collabs"
         ],
         "xaxis": "x3",
         "y": [
          410,
          222,
          14,
          13
         ],
         "yaxis": "y3"
        },
        {
         "alignmentgroup": "True",
         "hoverinfo": "x+y+name",
         "legendgroup": "After 1983",
         "marker": {
          "color": "#aab6fe",
          "pattern": {
           "shape": ""
          }
         },
         "name": "After 1983",
         "offsetgroup": "After 1983",
         "orientation": "v",
         "showlegend": false,
         "text": [
          276,
          168,
          124,
          8,
          3
         ],
         "textposition": "auto",
         "type": "bar",
         "x": [
          "Male",
          "Female",
          "Collectives & Collabs",
          "Missing",
          "Non-Binary"
         ],
         "xaxis": "x4",
         "y": [
          276,
          168,
          124,
          8,
          3
         ],
         "yaxis": "y4"
        },
        {
         "alignmentgroup": "True",
         "hoverinfo": "x+y+name",
         "legendgroup": "After 1983",
         "marker": {
          "color": "#aab6fe",
          "pattern": {
           "shape": ""
          }
         },
         "name": "After 1983",
         "offsetgroup": "After 1983",
         "orientation": "v",
         "showlegend": false,
         "text": [
          111,
          35,
          25,
          12
         ],
         "textposition": "auto",
         "type": "bar",
         "x": [
          "Male",
          "Female",
          "Collectives & Collabs",
          "Missing"
         ],
         "xaxis": "x",
         "y": [
          111,
          35,
          25,
          12
         ],
         "yaxis": "y"
        },
        {
         "alignmentgroup": "True",
         "hoverinfo": "x+y+name",
         "legendgroup": "After 1983",
         "marker": {
          "color": "#aab6fe",
          "pattern": {
           "shape": ""
          }
         },
         "name": "After 1983",
         "offsetgroup": "After 1983",
         "orientation": "v",
         "showlegend": false,
         "text": [
          1387,
          598,
          391,
          113
         ],
         "textposition": "auto",
         "type": "bar",
         "x": [
          "Male",
          "Female",
          "Collectives & Collabs",
          "Missing"
         ],
         "xaxis": "x2",
         "y": [
          1387,
          598,
          391,
          113
         ],
         "yaxis": "y2"
        }
       ],
       "layout": {
        "annotations": [
         {
          "font": {},
          "showarrow": false,
          "text": "Film",
          "x": 0.225,
          "xanchor": "center",
          "xref": "paper",
          "y": 0.45,
          "yanchor": "bottom",
          "yref": "paper"
         },
         {
          "font": {},
          "showarrow": false,
          "text": "Drawings & Prints",
          "x": 0.775,
          "xanchor": "center",
          "xref": "paper",
          "y": 0.45,
          "yanchor": "bottom",
          "yref": "paper"
         },
         {
          "font": {},
          "showarrow": false,
          "text": "Photography",
          "x": 0.225,
          "xanchor": "center",
          "xref": "paper",
          "y": 1,
          "yanchor": "bottom",
          "yref": "paper"
         },
         {
          "font": {},
          "showarrow": false,
          "text": "Media and Performance",
          "x": 0.775,
          "xanchor": "center",
          "xref": "paper",
          "y": 1,
          "yanchor": "bottom",
          "yref": "paper"
         }
        ],
        "barmode": "group",
        "height": 800,
        "legend": {
         "title": {
          "text": "Period"
         },
         "tracegroupgap": 0
        },
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "rgb(36,36,36)"
            },
            "error_y": {
             "color": "rgb(36,36,36)"
            },
            "marker": {
             "line": {
              "color": "white",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "white",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "rgb(36,36,36)",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "rgb(36,36,36)"
            },
            "baxis": {
             "endlinecolor": "rgb(36,36,36)",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "rgb(36,36,36)"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "line": {
              "color": "white",
              "width": 0.6
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "fillpattern": {
             "fillmode": "overlay",
             "size": 10,
             "solidity": 0.2
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "rgb(237,237,237)"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "rgb(217,217,217)"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 1,
            "tickcolor": "rgb(36,36,36)",
            "ticks": "outside"
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "rgb(103,0,31)"
            ],
            [
             0.1,
             "rgb(178,24,43)"
            ],
            [
             0.2,
             "rgb(214,96,77)"
            ],
            [
             0.3,
             "rgb(244,165,130)"
            ],
            [
             0.4,
             "rgb(253,219,199)"
            ],
            [
             0.5,
             "rgb(247,247,247)"
            ],
            [
             0.6,
             "rgb(209,229,240)"
            ],
            [
             0.7,
             "rgb(146,197,222)"
            ],
            [
             0.8,
             "rgb(67,147,195)"
            ],
            [
             0.9,
             "rgb(33,102,172)"
            ],
            [
             1,
             "rgb(5,48,97)"
            ]
           ],
           "sequential": [
            [
             0,
             "#440154"
            ],
            [
             0.1111111111111111,
             "#482878"
            ],
            [
             0.2222222222222222,
             "#3e4989"
            ],
            [
             0.3333333333333333,
             "#31688e"
            ],
            [
             0.4444444444444444,
             "#26828e"
            ],
            [
             0.5555555555555556,
             "#1f9e89"
            ],
            [
             0.6666666666666666,
             "#35b779"
            ],
            [
             0.7777777777777778,
             "#6ece58"
            ],
            [
             0.8888888888888888,
             "#b5de2b"
            ],
            [
             1,
             "#fde725"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#440154"
            ],
            [
             0.1111111111111111,
             "#482878"
            ],
            [
             0.2222222222222222,
             "#3e4989"
            ],
            [
             0.3333333333333333,
             "#31688e"
            ],
            [
             0.4444444444444444,
             "#26828e"
            ],
            [
             0.5555555555555556,
             "#1f9e89"
            ],
            [
             0.6666666666666666,
             "#35b779"
            ],
            [
             0.7777777777777778,
             "#6ece58"
            ],
            [
             0.8888888888888888,
             "#b5de2b"
            ],
            [
             1,
             "#fde725"
            ]
           ]
          },
          "colorway": [
           "#1F77B4",
           "#FF7F0E",
           "#2CA02C",
           "#D62728",
           "#9467BD",
           "#8C564B",
           "#E377C2",
           "#7F7F7F",
           "#BCBD22",
           "#17BECF"
          ],
          "font": {
           "color": "rgb(36,36,36)"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "white",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "white",
          "polar": {
           "angularaxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           },
           "bgcolor": "white",
           "radialaxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "white",
            "gridcolor": "rgb(232,232,232)",
            "gridwidth": 2,
            "linecolor": "rgb(36,36,36)",
            "showbackground": true,
            "showgrid": false,
            "showline": true,
            "ticks": "outside",
            "zeroline": false,
            "zerolinecolor": "rgb(36,36,36)"
           },
           "yaxis": {
            "backgroundcolor": "white",
            "gridcolor": "rgb(232,232,232)",
            "gridwidth": 2,
            "linecolor": "rgb(36,36,36)",
            "showbackground": true,
            "showgrid": false,
            "showline": true,
            "ticks": "outside",
            "zeroline": false,
            "zerolinecolor": "rgb(36,36,36)"
           },
           "zaxis": {
            "backgroundcolor": "white",
            "gridcolor": "rgb(232,232,232)",
            "gridwidth": 2,
            "linecolor": "rgb(36,36,36)",
            "showbackground": true,
            "showgrid": false,
            "showline": true,
            "ticks": "outside",
            "zeroline": false,
            "zerolinecolor": "rgb(36,36,36)"
           }
          },
          "shapedefaults": {
           "fillcolor": "black",
           "line": {
            "width": 0
           },
           "opacity": 0.3
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           },
           "baxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           },
           "bgcolor": "white",
           "caxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "rgb(232,232,232)",
           "linecolor": "rgb(36,36,36)",
           "showgrid": false,
           "showline": true,
           "ticks": "outside",
           "title": {
            "standoff": 15
           },
           "zeroline": false,
           "zerolinecolor": "rgb(36,36,36)"
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "rgb(232,232,232)",
           "linecolor": "rgb(36,36,36)",
           "showgrid": false,
           "showline": true,
           "ticks": "outside",
           "title": {
            "standoff": 15
           },
           "zeroline": false,
           "zerolinecolor": "rgb(36,36,36)"
          }
         }
        },
        "title": {
         "text": "Gender Representation in MoMA Departments before and after 1983",
         "x": 0.5
        },
        "width": 1600,
        "xaxis": {
         "anchor": "y",
         "categoryarray": [
          "Male",
          "Female",
          "Collectives & Collabs",
          "Missing",
          "Non-Binary"
         ],
         "categoryorder": "array",
         "domain": [
          0,
          0.45
         ],
         "showticklabels": true,
         "title": {
          "text": "Artist Gender"
         }
        },
        "xaxis2": {
         "anchor": "y2",
         "categoryarray": [
          "Male",
          "Female",
          "Collectives & Collabs",
          "Missing",
          "Non-Binary"
         ],
         "categoryorder": "array",
         "domain": [
          0.55,
          1
         ],
         "matches": "x",
         "showticklabels": true,
         "title": {
          "text": "Artist Gender"
         }
        },
        "xaxis3": {
         "anchor": "y3",
         "categoryarray": [
          "Male",
          "Female",
          "Collectives & Collabs",
          "Missing",
          "Non-Binary"
         ],
         "categoryorder": "array",
         "domain": [
          0,
          0.45
         ],
         "matches": "x",
         "showticklabels": true
        },
        "xaxis4": {
         "anchor": "y4",
         "categoryarray": [
          "Male",
          "Female",
          "Collectives & Collabs",
          "Missing",
          "Non-Binary"
         ],
         "categoryorder": "array",
         "domain": [
          0.55,
          1
         ],
         "matches": "x",
         "showticklabels": true
        },
        "yaxis": {
         "anchor": "x",
         "domain": [
          0,
          0.45
         ],
         "showticklabels": true,
         "title": {
          "text": "Artwork Count"
         },
         "type": "log"
        },
        "yaxis2": {
         "anchor": "x2",
         "domain": [
          0,
          0.45
         ],
         "matches": "y",
         "showticklabels": true,
         "type": "log"
        },
        "yaxis3": {
         "anchor": "x3",
         "domain": [
          0.55,
          1
         ],
         "matches": "y",
         "showticklabels": true,
         "title": {
          "text": "Artwork Count"
         },
         "type": "log"
        },
        "yaxis4": {
         "anchor": "x4",
         "domain": [
          0.55,
          1
         ],
         "matches": "y",
         "showticklabels": true,
         "type": "log"
        }
       }
      }
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "moma_gender_depts_full = px.bar(moma_depts_gender_rep_all, x=\"Gender\", y=\"Counts\", color=\"Period\", facet_col=\"Department\",\n",
    "    facet_col_spacing=0.10, facet_row_spacing=0.10,  facet_col_wrap=2, title=\"Gender Representation in MoMA Departments before and after 1983\",\n",
    "    barmode='group', log_y=True, color_discrete_map={\"Before 1983\": \"#49599a\", \"After 1983\": \"#aab6fe\"},\n",
    "    category_orders={\"Period\": [\"Before 1983\", 'After 1983']}, text='Counts', labels={\"Counts\": \"Artwork Count\", \"Gender\": \"Artist Gender\"})\n",
    "moma_gender_depts_full.update_xaxes(showticklabels=True, categoryorder='array', categoryarray=['Male', 'Female', 'Collectives & Collabs', 'Missing', 'Non-Binary'])\n",
    "moma_gender_depts_full.update_yaxes(showticklabels=True)\n",
    "moma_gender_depts_full.update_traces(textposition=\"auto\", hoverinfo='x+y+name', hovertemplate=None)\n",
    "moma_gender_depts_full.update_layout(height=800, width=1600, title_x=0.5)\n",
    "moma_gender_depts_full.for_each_annotation(lambda a: a.update(text=a.text.split(\"=\")[-1]))\n",
    "#create local copies\n",
    "#moma_gender_depts_full.write_html(path+\"Plotly_embeds/moma_full_gender_dept_bars.html\", include_plotlyjs='directory',  full_html=False, default_width='100%')\n",
    "moma_gender_depts_full.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "#create a list of rhizome mediums based on keywords research \n",
    "rhz_medium = rhizome_txt_stop_kw.copy()\n",
    "rhz_medium = rhz_medium.replace('N/B', 'NB')\n",
    "# create a list of our conditionsP\n",
    "conditions = [\n",
    "    (rhz_medium['Text'].str.contains('video|film|animation|movie|tv', regex=True)),\n",
    "    (rhz_medium['Text'].str.contains('images|image|photographs|photography|photo|photos|painting|drawing', regex=True)),\n",
    "    (rhz_medium['Text'].str.contains('installation|performance|sound|audio|song|music', regex=True)),\n",
    "    (rhz_medium['Text'].str.contains('website|flash|java|data|database|webpage|net|internet|site|sites|online|webbased|augmented|virtual|reality|game|interaction|interactive|interface|project|xml|mashup|map|generative|wifi|network|email|computer', regex=True)),\n",
    "    ]\n",
    "\n",
    "# create a list of the values we want to assign for each condition\n",
    "values = ['Moving Images', 'Still Images', 'Installation & Performance', 'Net Art']\n",
    "\n",
    "# create a new column and use np.select to assign values to it using our lists as arguments\n",
    "rhz_medium['Medium'] = np.select(conditions, values)\n",
    "\n",
    "# display updated DataFrame\n",
    "rhz_medium['Medium'] = rhz_medium['Medium'].replace('0', 'missing')\n",
    "rhz_medium['Medium'] = rhz_medium['Medium'].astype(str)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "#count Rhizome genders by department\n",
    "rhz_artists_medium = rhz_medium.drop_duplicates(subset='Artist', keep='last')\n",
    "rhz_gender_rep_mi = group_genders_dept(rhz_artists_medium.loc[rhz_artists_medium['Medium'] == 'Moving Images'], '', 'Moving Images')\n",
    "rhz_gender_rep_img = group_genders_dept(rhz_artists_medium.loc[rhz_artists_medium['Medium'] == 'Still Images'], '', 'Still Images')\n",
    "rhz_gender_rep_intall = group_genders_dept(rhz_artists_medium.loc[rhz_artists_medium['Medium'] == 'Installation & Performance'], '', 'Installation')\n",
    "rhz_gender_rep_netart = group_genders_dept(rhz_artists_medium.loc[rhz_artists_medium['Medium'] == 'Net Art'], '', 'Net Art')\n",
    "rhz_rep_all = pd.concat([rhz_gender_rep_mi, rhz_gender_rep_img, rhz_gender_rep_intall, rhz_gender_rep_netart])\n",
    "rhz_rep_all['Source'] = 'Rhizome'\n",
    "\n",
    "#count MoMA genders by department\n",
    "moma_draw_artists = moma_rhz_compare.loc[moma_rhz_compare['Department'] == 'Drawings & Prints'].drop_duplicates(subset='Artist', keep='last')\n",
    "moma_draw_gender_rep = group_genders_dept(moma_draw_artists, 'After 1983', 'Drawings & Prints')\n",
    "moma_films_artists = moma_rhz_compare.loc[moma_rhz_compare['Department'] == 'Film'].drop_duplicates(subset='Artist', keep='last')\n",
    "moma_films_gender_rep = group_genders_dept(moma_films_artists, 'After 1983', 'Film')\n",
    "moma_media_artists = moma_rhz_compare.loc[moma_rhz_compare['Department'] == 'Media and Performance'].drop_duplicates(subset='Artist', keep='last')\n",
    "moma_media_gender_rep = group_genders_dept(moma_media_artists, 'After 1983', 'Media and Performance')\n",
    "moma_photo_artists = moma_rhz_compare.loc[moma_rhz_compare['Department'] == 'Photography'].drop_duplicates(subset='Artist', keep='last')\n",
    "moma_photo_gender_rep = group_genders_dept(moma_photo_artists, 'After 1983', 'Photography')\n",
    "\n",
    "#group all together\n",
    "moma_depts_gender_rep_all_compare = pd.concat([moma_draw_gender_rep, moma_films_gender_rep, moma_media_gender_rep, moma_photo_gender_rep])\n",
    "moma_depts_gender_rep_all_compare = moma_depts_gender_rep_all_compare.reset_index(drop='True')\n",
    "\n",
    "\n",
    "#align departments alongside new 3x3 grouping\n",
    "rhz_rep_all = rhz_rep_all.replace(['Installation', 'Net Art'], 'Media and Performance').groupby(['Department', 'Source', 'Gender'],as_index=False).agg({'Counts': 'sum'})\n",
    "moma_depts_gender_rep_all_compare_2 = moma_depts_gender_rep_all_compare.replace(['Photography', 'Drawings & Prints'], 'Still Images').replace(['Film'], 'Moving Images').groupby(['Department', 'Period', 'Gender'],as_index=False).agg({'Counts': 'sum'})\n",
    "\n",
    "#create a combined dataframe\n",
    "#moma_depts_gender_rep_all_compare_2['Counts'] = moma_depts_gender_rep_all_compare_2['Counts'].div(100).round(2)\n",
    "#moma_depts_gender_rep_all_compare_2['Counts'] = moma_depts_gender_rep_all_compare_2['Counts'].multiply(40).round(2)\n",
    "#moma_depts_gender_rep_all_compare_2['Counts'] = moma_depts_gender_rep_all_compare_2['Counts'].astype(int)\n",
    "moma_depts_gender_rep_all_compare_2['Source'] = 'MoMA'\n",
    "#moma_depts_gender_rep_all_compare_2.loc[4, 'Counts'] = 3\n",
    "#rhz_rep_all['Counts'] = rhz_rep_all['Counts'].multiply(1.07)\n",
    "#rhz_rep_all['Counts'] = rhz_rep_all['Counts'].apply(np.floor)\n",
    "#rhz_rep_all['Counts'] = rhz_rep_all['Counts'].astype(int)\n",
    "moma_rhz_gender_compare = pd.concat([rhz_rep_all, moma_depts_gender_rep_all_compare_2])\n",
    "moma_rhz_gender_compare = moma_rhz_gender_compare.drop('Period', axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "alignmentgroup": "True",
         "hoverinfo": "x+y+name",
         "legendgroup": "Still Images",
         "marker": {
          "color": "#90caf9",
          "pattern": {
           "shape": ""
          }
         },
         "name": "Still Images",
         "offsetgroup": "Still Images",
         "orientation": "v",
         "showlegend": true,
         "textfont": {
          "family": "Lato",
          "size": 12
         },
         "textposition": "outside",
         "texttemplate": "%{y}",
         "type": "bar",
         "x": [
          "Missing",
          "Female",
          "Collectives & Collabs",
          "Male"
         ],
         "xaxis": "x",
         "y": [
          121,
          641,
          301,
          1254
         ],
         "yaxis": "y"
        },
        {
         "alignmentgroup": "True",
         "hoverinfo": "x+y+name",
         "legendgroup": "Still Images",
         "marker": {
          "color": "#90caf9",
          "pattern": {
           "shape": ""
          }
         },
         "name": "Still Images",
         "offsetgroup": "Still Images",
         "orientation": "v",
         "showlegend": false,
         "textfont": {
          "family": "Lato",
          "size": 12
         },
         "textposition": "outside",
         "texttemplate": "%{y}",
         "type": "bar",
         "x": [
          "Female",
          "Collectives & Collabs",
          "Male",
          "Missing"
         ],
         "xaxis": "x2",
         "y": [
          68,
          11,
          141,
          13
         ],
         "yaxis": "y2"
        },
        {
         "alignmentgroup": "True",
         "hoverinfo": "x+y+name",
         "legendgroup": "Media and Performance",
         "marker": {
          "color": "#ffe082",
          "pattern": {
           "shape": ""
          }
         },
         "name": "Media and Performance",
         "offsetgroup": "Media and Performance",
         "orientation": "v",
         "showlegend": true,
         "textfont": {
          "family": "Lato",
          "size": 12
         },
         "textposition": "outside",
         "texttemplate": "%{y}",
         "type": "bar",
         "x": [
          "Non-Binary",
          "Missing",
          "Male",
          "Female",
          "Collectives & Collabs"
         ],
         "xaxis": "x",
         "y": [
          3,
          6,
          186,
          118,
          77
         ],
         "yaxis": "y"
        },
        {
         "alignmentgroup": "True",
         "hoverinfo": "x+y+name",
         "legendgroup": "Media and Performance",
         "marker": {
          "color": "#ffe082",
          "pattern": {
           "shape": ""
          }
         },
         "name": "Media and Performance",
         "offsetgroup": "Media and Performance",
         "orientation": "v",
         "showlegend": false,
         "textfont": {
          "family": "Lato",
          "size": 12
         },
         "textposition": "outside",
         "texttemplate": "%{y}",
         "type": "bar",
         "x": [
          "Collectives & Collabs",
          "Missing",
          "Male",
          "Female"
         ],
         "xaxis": "x2",
         "y": [
          23,
          40,
          256,
          108
         ],
         "yaxis": "y2"
        },
        {
         "alignmentgroup": "True",
         "hoverinfo": "x+y+name",
         "legendgroup": "Moving Images",
         "marker": {
          "color": "#b94e52",
          "pattern": {
           "shape": ""
          }
         },
         "name": "Moving Images",
         "offsetgroup": "Moving Images",
         "orientation": "v",
         "showlegend": true,
         "textfont": {
          "family": "Lato",
          "size": 12
         },
         "textposition": "outside",
         "texttemplate": "%{y}",
         "type": "bar",
         "x": [
          "Missing",
          "Male",
          "Female",
          "Collectives & Collabs"
         ],
         "xaxis": "x",
         "y": [
          7,
          103,
          34,
          23
         ],
         "yaxis": "y"
        },
        {
         "alignmentgroup": "True",
         "hoverinfo": "x+y+name",
         "legendgroup": "Moving Images",
         "marker": {
          "color": "#b94e52",
          "pattern": {
           "shape": ""
          }
         },
         "name": "Moving Images",
         "offsetgroup": "Moving Images",
         "orientation": "v",
         "showlegend": false,
         "textfont": {
          "family": "Lato",
          "size": 12
         },
         "textposition": "outside",
         "texttemplate": "%{y}",
         "type": "bar",
         "x": [
          "Non-Binary",
          "Missing",
          "Male",
          "Female",
          "Collectives & Collabs"
         ],
         "xaxis": "x2",
         "y": [
          1,
          40,
          273,
          138,
          32
         ],
         "yaxis": "y2"
        }
       ],
       "layout": {
        "annotations": [
         {
          "font": {},
          "showarrow": false,
          "text": "MoMA (sampled)",
          "x": 0.225,
          "xanchor": "center",
          "xref": "paper",
          "y": 1,
          "yanchor": "bottom",
          "yref": "paper"
         },
         {
          "font": {},
          "showarrow": false,
          "text": "Rhizome",
          "x": 0.775,
          "xanchor": "center",
          "xref": "paper",
          "y": 1,
          "yanchor": "bottom",
          "yref": "paper"
         }
        ],
        "barmode": "group",
        "legend": {
         "title": {
          "text": "Department"
         },
         "tracegroupgap": 0
        },
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "rgb(36,36,36)"
            },
            "error_y": {
             "color": "rgb(36,36,36)"
            },
            "marker": {
             "line": {
              "color": "white",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "white",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "rgb(36,36,36)",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "rgb(36,36,36)"
            },
            "baxis": {
             "endlinecolor": "rgb(36,36,36)",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "rgb(36,36,36)"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "line": {
              "color": "white",
              "width": 0.6
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "fillpattern": {
             "fillmode": "overlay",
             "size": 10,
             "solidity": 0.2
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "rgb(237,237,237)"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "rgb(217,217,217)"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 1,
            "tickcolor": "rgb(36,36,36)",
            "ticks": "outside"
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "rgb(103,0,31)"
            ],
            [
             0.1,
             "rgb(178,24,43)"
            ],
            [
             0.2,
             "rgb(214,96,77)"
            ],
            [
             0.3,
             "rgb(244,165,130)"
            ],
            [
             0.4,
             "rgb(253,219,199)"
            ],
            [
             0.5,
             "rgb(247,247,247)"
            ],
            [
             0.6,
             "rgb(209,229,240)"
            ],
            [
             0.7,
             "rgb(146,197,222)"
            ],
            [
             0.8,
             "rgb(67,147,195)"
            ],
            [
             0.9,
             "rgb(33,102,172)"
            ],
            [
             1,
             "rgb(5,48,97)"
            ]
           ],
           "sequential": [
            [
             0,
             "#440154"
            ],
            [
             0.1111111111111111,
             "#482878"
            ],
            [
             0.2222222222222222,
             "#3e4989"
            ],
            [
             0.3333333333333333,
             "#31688e"
            ],
            [
             0.4444444444444444,
             "#26828e"
            ],
            [
             0.5555555555555556,
             "#1f9e89"
            ],
            [
             0.6666666666666666,
             "#35b779"
            ],
            [
             0.7777777777777778,
             "#6ece58"
            ],
            [
             0.8888888888888888,
             "#b5de2b"
            ],
            [
             1,
             "#fde725"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#440154"
            ],
            [
             0.1111111111111111,
             "#482878"
            ],
            [
             0.2222222222222222,
             "#3e4989"
            ],
            [
             0.3333333333333333,
             "#31688e"
            ],
            [
             0.4444444444444444,
             "#26828e"
            ],
            [
             0.5555555555555556,
             "#1f9e89"
            ],
            [
             0.6666666666666666,
             "#35b779"
            ],
            [
             0.7777777777777778,
             "#6ece58"
            ],
            [
             0.8888888888888888,
             "#b5de2b"
            ],
            [
             1,
             "#fde725"
            ]
           ]
          },
          "colorway": [
           "#1F77B4",
           "#FF7F0E",
           "#2CA02C",
           "#D62728",
           "#9467BD",
           "#8C564B",
           "#E377C2",
           "#7F7F7F",
           "#BCBD22",
           "#17BECF"
          ],
          "font": {
           "color": "rgb(36,36,36)"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "white",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "white",
          "polar": {
           "angularaxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           },
           "bgcolor": "white",
           "radialaxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "white",
            "gridcolor": "rgb(232,232,232)",
            "gridwidth": 2,
            "linecolor": "rgb(36,36,36)",
            "showbackground": true,
            "showgrid": false,
            "showline": true,
            "ticks": "outside",
            "zeroline": false,
            "zerolinecolor": "rgb(36,36,36)"
           },
           "yaxis": {
            "backgroundcolor": "white",
            "gridcolor": "rgb(232,232,232)",
            "gridwidth": 2,
            "linecolor": "rgb(36,36,36)",
            "showbackground": true,
            "showgrid": false,
            "showline": true,
            "ticks": "outside",
            "zeroline": false,
            "zerolinecolor": "rgb(36,36,36)"
           },
           "zaxis": {
            "backgroundcolor": "white",
            "gridcolor": "rgb(232,232,232)",
            "gridwidth": 2,
            "linecolor": "rgb(36,36,36)",
            "showbackground": true,
            "showgrid": false,
            "showline": true,
            "ticks": "outside",
            "zeroline": false,
            "zerolinecolor": "rgb(36,36,36)"
           }
          },
          "shapedefaults": {
           "fillcolor": "black",
           "line": {
            "width": 0
           },
           "opacity": 0.3
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           },
           "baxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           },
           "bgcolor": "white",
           "caxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "rgb(232,232,232)",
           "linecolor": "rgb(36,36,36)",
           "showgrid": false,
           "showline": true,
           "ticks": "outside",
           "title": {
            "standoff": 15
           },
           "zeroline": false,
           "zerolinecolor": "rgb(36,36,36)"
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "rgb(232,232,232)",
           "linecolor": "rgb(36,36,36)",
           "showgrid": false,
           "showline": true,
           "ticks": "outside",
           "title": {
            "standoff": 15
           },
           "zeroline": false,
           "zerolinecolor": "rgb(36,36,36)"
          }
         }
        },
        "title": {
         "text": "Gender Representation Between Rhizome and MoMA (sampled) by Departments",
         "x": 0.5
        },
        "xaxis": {
         "anchor": "y",
         "categoryarray": [
          "Male",
          "Female",
          "Collectives & Collabs",
          "Missing",
          "Non-Binary"
         ],
         "categoryorder": "array",
         "domain": [
          0,
          0.45
         ],
         "showticklabels": true,
         "title": {
          "text": "Artist Gender"
         }
        },
        "xaxis2": {
         "anchor": "y2",
         "categoryarray": [
          "Male",
          "Female",
          "Collectives & Collabs",
          "Missing",
          "Non-Binary"
         ],
         "categoryorder": "array",
         "domain": [
          0.55,
          1
         ],
         "showticklabels": true,
         "title": {
          "text": "Artist Gender"
         }
        },
        "yaxis": {
         "anchor": "x",
         "domain": [
          0,
          1
         ],
         "showticklabels": true,
         "title": {
          "text": "Artwork Count"
         },
         "type": "log"
        },
        "yaxis2": {
         "anchor": "x2",
         "domain": [
          0,
          1
         ],
         "matches": "y",
         "showticklabels": true,
         "type": "log"
        }
       }
      }
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "moma_rhz_gender_compare = moma_rhz_gender_compare.replace('MoMA', 'MoMA (sampled)')\n",
    "moma_rhz_depts_compare = px.bar(moma_rhz_gender_compare.sort_values(by='Source'), x=\"Gender\", y=\"Counts\", color=\"Department\", facet_col=\"Source\",\n",
    "             facet_col_spacing=0.10,  facet_col_wrap=2,\n",
    "             title=\"Gender Representation Between Rhizome and MoMA (sampled) by Departments\", barmode='group', \n",
    "             text_auto=True, color_discrete_map ={\"Still Images\":\"#90caf9\", \"Media and Performance\":\"#ffe082\", \"Moving Images\":\"#b94e52\"} , category_orders={\"Department\": [\"Still Images\", 'Media and Performance', \"Moving Images\"]}, log_y=True, labels={\"Counts\": \"Artwork Count\", \"Gender\": \"Artist Gender\"})\n",
    "moma_rhz_depts_compare.update_xaxes(matches=None, showticklabels=True,  categoryorder='array', categoryarray=['Male', 'Female', 'Collectives & Collabs', 'Missing', 'Non-Binary'])\n",
    "moma_rhz_depts_compare.update_yaxes(showticklabels=True)\n",
    "moma_rhz_depts_compare.update_traces(textposition=\"outside\", hoverinfo='x+y+name', hovertemplate=None, textfont=dict(family='Lato', size=12)), \n",
    "moma_rhz_depts_compare.update_layout(title_x=0.5)\n",
    "moma_rhz_depts_compare.for_each_annotation(lambda a: a.update(text=a.text.split(\"=\")[-1]))\n",
    "#create local copies\n",
    "#moma_rhz_depts_compare.write_html(path+\"Plotly_embeds/moma_rhz_depts_compare.html\", include_plotlyjs='directory', full_html=False, default_width='100%')\n",
    "moma_rhz_depts_compare.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Representing Nationality by Dataset (Everyone)\n",
    "\n",
    "For nationalities we decided to use maps first for a simple and interactive visualisation of how the collections are spread across the globe. From there we decided to use the concept of the Global North/South divide to go deeper into what the maps showed us. We used [Wikimedia's list of country classications](https://meta.wikimedia.org/wiki/List_of_countries_by_regional_classification) to assign north/south divide and regions and displayed the results using bar charts, first comparing both datasets by the north/south divide, then going into nationality and regional details for the Global South.<br><br>\n",
    "Having populated the Rhizome dataset manually we were able to take into account dual nationalities of artists in these vizualisations, by doing some additional manual cleaning of countries. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Nation</th>\n",
       "      <th>Count</th>\n",
       "      <th>country_code</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>90</th>\n",
       "      <td>Beninese</td>\n",
       "      <td>1</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>254</th>\n",
       "      <td>Ivory Coast</td>\n",
       "      <td>1</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>282</th>\n",
       "      <td>Nationality unknown</td>\n",
       "      <td>3</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>313</th>\n",
       "      <td>Scotland</td>\n",
       "      <td>14</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>374</th>\n",
       "      <td>Yugoslavia</td>\n",
       "      <td>2</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>376</th>\n",
       "      <td>missing</td>\n",
       "      <td>196</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                  Nation  Count country_code\n",
       "90              Beninese      1          nan\n",
       "254          Ivory Coast      1          nan\n",
       "282  Nationality unknown      3          nan\n",
       "313             Scotland     14          nan\n",
       "374           Yugoslavia      2          nan\n",
       "376              missing    196          nan"
      ]
     },
     "execution_count": 117,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Create a DF of MoMA nationalities transformed into countries and country code w/ counts\n",
    "moma_countries_count = assign_codes(moma_rhz_compare)\n",
    "#check for countries w/ missing codes\n",
    "missing_codes = moma_countries_count.loc[moma_countries_count['country_code'] == 'nan']\n",
    "missing_codes[~missing_codes.Nation.str.contains(',')]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Nation</th>\n",
       "      <th>Count</th>\n",
       "      <th>country_code</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>1</td>\n",
       "      <td>AFG</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Albania</td>\n",
       "      <td>2</td>\n",
       "      <td>ALB</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Algeria</td>\n",
       "      <td>3</td>\n",
       "      <td>DZA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Argentina</td>\n",
       "      <td>20</td>\n",
       "      <td>ARG</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Australia</td>\n",
       "      <td>10</td>\n",
       "      <td>AUS</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        Nation  Count country_code\n",
       "0  Afghanistan      1          AFG\n",
       "1      Albania      2          ALB\n",
       "2      Algeria      3          DZA\n",
       "3    Argentina     20          ARG\n",
       "4    Australia     10          AUS"
      ]
     },
     "execution_count": 118,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#fix them\n",
    "moma_countries_count.loc[254, 'country_code'] = 'CIV'\n",
    "moma_countries_count.loc[90, 'country_code'] = 'BEN'\n",
    "moma_countries_count.loc[90, 'Nation'] = 'Benin'\n",
    "moma_countries_count.loc[266, 'Nation'] = 'South Korea'\n",
    "moma_countries_count.loc[266, 'country_code'] = 'KOR'\n",
    "#add Scots to GBR\n",
    "moma_countries_count.loc[366, 'Count'] = 194\n",
    "#add Serbs to SRB\n",
    "moma_countries_count.loc[315, 'Count'] = 44\n",
    "#add Croats to HRV\n",
    "moma_countries_count.loc[152, 'Count'] = 8\n",
    "\n",
    "#reset types and filter for final use\n",
    "moma_countries_count['Count'] = moma_countries_count['Count'].astype(int)\n",
    "moma_countries_count = moma_countries_count.loc[~moma_countries_count.Nation.str.contains(',')]\n",
    "moma_countries_count = moma_countries_count.loc[~moma_countries_count.country_code.str.contains('nan')]\n",
    "moma_countries_count = moma_countries_count.sort_values(by='Nation').reset_index(drop=True)\n",
    "moma_countries_count.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Nation</th>\n",
       "      <th>Count</th>\n",
       "      <th>country_code</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Afghan/American</td>\n",
       "      <td>1</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>American, Russian, British, Swiss/American, Du...</td>\n",
       "      <td>1</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>American/British</td>\n",
       "      <td>2</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>American/Canadian</td>\n",
       "      <td>1</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>American/Israeli</td>\n",
       "      <td>1</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>American/Mexican</td>\n",
       "      <td>1</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>American/South African</td>\n",
       "      <td>1</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>Brasilian/American</td>\n",
       "      <td>2</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>British/American</td>\n",
       "      <td>1</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>British/Canadian</td>\n",
       "      <td>1</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>British/Irish</td>\n",
       "      <td>1</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>Canadian/American</td>\n",
       "      <td>1</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>Cherokee/American</td>\n",
       "      <td>1</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>52</th>\n",
       "      <td>Dutch/Brazilian</td>\n",
       "      <td>1</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>68</th>\n",
       "      <td>Greek/Australian</td>\n",
       "      <td>1</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>72</th>\n",
       "      <td>Icelandic/American</td>\n",
       "      <td>1</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>76</th>\n",
       "      <td>Iranian/Canadian</td>\n",
       "      <td>1</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79</th>\n",
       "      <td>Israeli/American</td>\n",
       "      <td>1</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>81</th>\n",
       "      <td>Italian, Austrian, Swiss/American</td>\n",
       "      <td>1</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>83</th>\n",
       "      <td>Italian, Spanish/French, Lybian</td>\n",
       "      <td>1</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>84</th>\n",
       "      <td>Italian/Danish</td>\n",
       "      <td>1</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>90</th>\n",
       "      <td>Mexican/American</td>\n",
       "      <td>2</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>91</th>\n",
       "      <td>Mexican/Canadian</td>\n",
       "      <td>1</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>98</th>\n",
       "      <td>Polish/American</td>\n",
       "      <td>1</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99</th>\n",
       "      <td>Polish/German</td>\n",
       "      <td>1</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>115</th>\n",
       "      <td>Spanish/French</td>\n",
       "      <td>1</td>\n",
       "      <td>nan</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                Nation  Count country_code\n",
       "0                                      Afghan/American      1          nan\n",
       "7    American, Russian, British, Swiss/American, Du...      1          nan\n",
       "9                                     American/British      2          nan\n",
       "10                                   American/Canadian      1          nan\n",
       "11                                    American/Israeli      1          nan\n",
       "12                                    American/Mexican      1          nan\n",
       "13                              American/South African      1          nan\n",
       "26                                  Brasilian/American      2          nan\n",
       "31                                    British/American      1          nan\n",
       "32                                    British/Canadian      1          nan\n",
       "33                                       British/Irish      1          nan\n",
       "38                                   Canadian/American      1          nan\n",
       "39                                   Cherokee/American      1          nan\n",
       "52                                     Dutch/Brazilian      1          nan\n",
       "68                                    Greek/Australian      1          nan\n",
       "72                                  Icelandic/American      1          nan\n",
       "76                                    Iranian/Canadian      1          nan\n",
       "79                                    Israeli/American      1          nan\n",
       "81                   Italian, Austrian, Swiss/American      1          nan\n",
       "83                     Italian, Spanish/French, Lybian      1          nan\n",
       "84                                      Italian/Danish      1          nan\n",
       "90                                    Mexican/American      2          nan\n",
       "91                                    Mexican/Canadian      1          nan\n",
       "98                                     Polish/American      1          nan\n",
       "99                                       Polish/German      1          nan\n",
       "115                                     Spanish/French      1          nan"
      ]
     },
     "execution_count": 119,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Create a DF of Rhizome nationalities transformed into countries and country code w/ counts\n",
    "rhz_countries_count = assign_codes(rhz_artworks)\n",
    "#check for countries w/ missing codes\n",
    "missing_codes_2 = rhz_countries_count.loc[rhz_countries_count['country_code'] == 'nan']\n",
    "missing_codes_2[missing_codes_2.Nation.str.contains('/')]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Nation</th>\n",
       "      <th>Count</th>\n",
       "      <th>country_code</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>1</td>\n",
       "      <td>AFG</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>American</td>\n",
       "      <td>11</td>\n",
       "      <td>ASM</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Angola</td>\n",
       "      <td>1</td>\n",
       "      <td>AGO</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Argentina</td>\n",
       "      <td>12</td>\n",
       "      <td>ARG</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Argentine</td>\n",
       "      <td>1</td>\n",
       "      <td>ARG</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        Nation  Count country_code\n",
       "0  Afghanistan      1          AFG\n",
       "1    American      11          ASM\n",
       "2       Angola      1          AGO\n",
       "3    Argentina     12          ARG\n",
       "4   Argentine       1          ARG"
      ]
     },
     "execution_count": 120,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#fix them\n",
    "#Add Swiss \n",
    "rhz_countries_count.loc[121, 'Count'] = 7\n",
    "#Add Iranians \n",
    "rhz_countries_count.loc[74, 'Count'] = 3\n",
    "#Add Danes \n",
    "rhz_countries_count.loc[49, 'Count'] = 2\n",
    "#Add Mexicans \n",
    "rhz_countries_count.loc[92, 'Count'] = 11\n",
    "#Add SAs \n",
    "rhz_countries_count.loc[109, 'Count'] = 3\n",
    "#Add Poles \n",
    "rhz_countries_count.loc[97, 'Count'] = 12\n",
    "#Add Chileans \n",
    "rhz_countries_count.loc[40, 'Count'] = 6\n",
    "#Add Turks\n",
    "rhz_countries_count.loc[124, 'Count'] = 6\n",
    "#Add Canadians\n",
    "rhz_countries_count.loc[35, 'Count'] = 51\n",
    "#Add Serbs\n",
    "rhz_countries_count.loc[106, 'Count'] = 4\n",
    "#Add Fins\n",
    "rhz_countries_count.loc[55, 'Count'] = 3\n",
    "#Add Japanese\n",
    "rhz_countries_count.loc[86, 'Count'] = 11\n",
    "#Add Americans\n",
    "rhz_countries_count.loc[127, 'Count'] = 287\n",
    "#Add Columbians\n",
    "rhz_countries_count.loc[44, 'Count'] = 3\n",
    "#Add Germans\n",
    "rhz_countries_count.loc[66, 'Count'] = 34\n",
    "#Add Brits\n",
    "rhz_countries_count.loc[126, 'Count'] = 58\n",
    "#Add Irish\n",
    "rhz_countries_count.loc[77, 'Count'] = 3\n",
    "#Add Greeks\n",
    "rhz_countries_count.loc[67, 'Count'] = 5\n",
    "#Add Australians\n",
    "rhz_countries_count.loc[18, 'Count'] = 22\n",
    "#Add Spanish\n",
    "rhz_countries_count.loc[114, 'Count'] = 15\n",
    "#Add French\n",
    "rhz_countries_count.loc[57, 'Count'] = 30\n",
    "#Add Israeli \n",
    "rhz_countries_count.loc[78, 'Count'] = 15\n",
    "#Add Dutch \n",
    "rhz_countries_count.loc[93, 'Count'] = 15\n",
    "#Add Italians \n",
    "rhz_countries_count.loc[85, 'Count'] = 22\n",
    "#Create Brazilians\n",
    "rhz_countries_count.loc[140, 'Count'] = 16\n",
    "rhz_countries_count.loc[140, 'Nation'] = 'Brazil'\n",
    "rhz_countries_count.loc[140, 'country_code'] = 'BRA'\n",
    "#Create Afghanis\n",
    "rhz_countries_count.loc[141, 'Count'] = 1\n",
    "rhz_countries_count.loc[141, 'Nation'] = 'Afghanistan'\n",
    "rhz_countries_count.loc[141, 'country_code'] = 'AFG'\n",
    "#Create Icelandic\n",
    "rhz_countries_count.loc[142, 'Count'] = 1\n",
    "rhz_countries_count.loc[142, 'Nation'] = 'Iceland'\n",
    "rhz_countries_count.loc[142, 'country_code'] = 'ISL'\n",
    "\n",
    "#reset types and filter for final use\n",
    "rhz_countries_count['Count'] = rhz_countries_count['Count'].astype(int)\n",
    "rhz_countries_count = rhz_countries_count.loc[~rhz_countries_count.Nation.str.contains(',')]\n",
    "rhz_countries_count = rhz_countries_count.loc[~rhz_countries_count.country_code.str.contains('nan')]\n",
    "rhz_countries_count = rhz_countries_count.sort_values(by='Nation').reset_index(drop=True)\n",
    "rhz_countries_count.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Nation</th>\n",
       "      <th>Count</th>\n",
       "      <th>country_code</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>1</td>\n",
       "      <td>AFG</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Angola</td>\n",
       "      <td>1</td>\n",
       "      <td>AGO</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Argentina</td>\n",
       "      <td>13</td>\n",
       "      <td>ARG</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Australia</td>\n",
       "      <td>22</td>\n",
       "      <td>AUS</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Austria</td>\n",
       "      <td>16</td>\n",
       "      <td>AUT</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        Nation  Count country_code\n",
       "0  Afghanistan      1          AFG\n",
       "1       Angola      1          AGO\n",
       "2    Argentina     13          ARG\n",
       "3    Australia     22          AUS\n",
       "4      Austria     16          AUT"
      ]
     },
     "execution_count": 121,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#add stragglers based on north/south divide\n",
    "rhz_countries_count.loc[56, 'Count'] = 298\n",
    "rhz_countries_count.loc[20, 'Count'] = 31\n",
    "rhz_countries_count.loc[33, 'Count'] = 24\n",
    "rhz_countries_count.loc[3, 'Count'] = 13\n",
    "#remove them\n",
    "rhz_countries_count = rhz_countries_count.drop([1, 4, 21, 32], axis=0)\n",
    "rhz_countries_count['Count'] = rhz_countries_count['Count'].astype(int)\n",
    "rhz_countries_count = rhz_countries_count.sort_values(by='Nation').reset_index(drop=True)\n",
    "rhz_countries_count.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "cb5f002a963e445784ed2d880c68ee9c",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(FigureWidget({\n",
       "    'data': [{'autocolorscale': False,\n",
       "              'colorbar': {'title': {'tex…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#MAP IT! \n",
    "\n",
    "moma_map = go.Figure(data=go.Choropleth(\n",
    "    locations = moma_countries_count['country_code'],\n",
    "    z = moma_countries_count['Count'].apply(np.floor),\n",
    "    text = moma_countries_count['Nation'],\n",
    "    colorscale=\n",
    "            [[0,\"#0e1123\"],[0.25,\"#242e5c\"],[0.5,\"#2f3b70\"], [0.75,\"#465493\"],[1,\"#9facf1\"]],\n",
    "    autocolorscale=False,\n",
    "    reversescale=True,\n",
    "    marker_line_color='black',\n",
    "    marker_line_width=0.2,\n",
    "    colorbar_title = 'Occurences',\n",
    "))\n",
    "\n",
    "moma_map.update_traces(hoverinfo='z+text')\n",
    "\n",
    "moma_map.update_layout(\n",
    "    title_text='Where Do Artists Come From? And How Many Are There? - MoMA (sampled)',\n",
    "    geo=dict(\n",
    "        showframe=True,\n",
    "        showcoastlines=False,\n",
    "        projection_type='natural earth',\n",
    "        showocean=True, oceancolor=\"LightBlue\"\n",
    "    ),\n",
    "    title_x = 0.5\n",
    ")\n",
    "\n",
    "rhz_map = go.Figure(data=go.Choropleth(\n",
    "    locations = rhz_countries_count['country_code'],\n",
    "    z = rhz_countries_count['Count'].apply(np.floor),\n",
    "    text = rhz_countries_count['Nation'],\n",
    "    colorscale=[[0,\"#2a4858\"],[0.25,\"#106775\"],[0.5,\"#55b378\"],[0.75,\"#9ccc65\"],[1,\"#d9f2bc\"]],\n",
    "    autocolorscale=False,\n",
    "    reversescale=True,\n",
    "    marker_line_color='black',\n",
    "    marker_line_width=0.2,\n",
    "    colorbar_title = 'Occurences',\n",
    "))\n",
    "\n",
    "rhz_map.update_traces(hoverinfo='z+text')\n",
    "\n",
    "rhz_map.update_layout(\n",
    "    title_text='Where Do Artists Come From? And How Many Are There? - Rhizome',\n",
    "    geo=dict(\n",
    "        showframe=True,\n",
    "        showcoastlines=False,\n",
    "        projection_type='natural earth',\n",
    "        showocean=True, oceancolor=\"LightBlue\"\n",
    "    ),\n",
    "     title_x=0.5\n",
    ")\n",
    "\n",
    "#create local copies\n",
    "rhz_map.write_html(path+\"Plotly_embeds/rhz_map.html\", include_plotlyjs='directory',  full_html=False, default_width='100%')\n",
    "moma_map.write_html(path+\"Plotly_embeds/moma_map.html\", include_plotlyjs='directory', full_html=False, default_width='100%')\n",
    "\n",
    "rhz_map = go.FigureWidget(rhz_map)\n",
    "moma_map = go.FigureWidget(moma_map)\n",
    "maps = ipw.VBox([rhz_map, moma_map])\n",
    "maps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Nation</th>\n",
       "      <th>Count</th>\n",
       "      <th>country_code</th>\n",
       "      <th>Divide</th>\n",
       "      <th>Region</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>Iran</td>\n",
       "      <td>3</td>\n",
       "      <td>IRN</td>\n",
       "      <td>missing</td>\n",
       "      <td>missing</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>Russia</td>\n",
       "      <td>8</td>\n",
       "      <td>RUS</td>\n",
       "      <td>missing</td>\n",
       "      <td>missing</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>South Korea</td>\n",
       "      <td>6</td>\n",
       "      <td>KOR</td>\n",
       "      <td>missing</td>\n",
       "      <td>missing</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49</th>\n",
       "      <td>Tanzania</td>\n",
       "      <td>1</td>\n",
       "      <td>TZA</td>\n",
       "      <td>missing</td>\n",
       "      <td>missing</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         Nation  Count country_code   Divide   Region\n",
       "26         Iran      3          IRN  missing  missing\n",
       "40       Russia      8          RUS  missing  missing\n",
       "44  South Korea      6          KOR  missing  missing\n",
       "49     Tanzania      1          TZA  missing  missing"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#populate Rhizome country count DF w/ divide information \n",
    "rhz_north_south = rhz_countries_count.copy()\n",
    "rhz_north_south['Divide'] = rhz_north_south['Nation'].apply(lambda nation: getDivide(nation))\n",
    "rhz_north_south['Region'] = rhz_north_south['Nation'].apply(lambda nation: getRegion(nation))\n",
    "rhz_north_south = rhz_north_south.sort_values(by='Nation').reset_index(drop=True)\n",
    "\n",
    "#check for missing info\n",
    "rhz_north_south.loc[rhz_north_south['Divide'] == 'missing']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "#fix them \n",
    "rhz_north_south.loc[26, 'Divide'] = 'Global South'\n",
    "rhz_north_south.loc[26, 'Region'] = 'Middle east'\n",
    "rhz_north_south.loc[40, 'Divide'] = 'Global North'\n",
    "rhz_north_south.loc[40, 'Region'] = 'Europe'\n",
    "rhz_north_south.loc[44, 'Divide'] = 'Global North'\n",
    "rhz_north_south.loc[44, 'Region'] = 'Asia & Pacific'\n",
    "rhz_north_south.loc[49, 'Divide'] = 'Global South'\n",
    "rhz_north_south.loc[49, 'Region'] = 'Africa'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Nation</th>\n",
       "      <th>Count</th>\n",
       "      <th>country_code</th>\n",
       "      <th>Divide</th>\n",
       "      <th>Region</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Bosnia And Herzegovina</td>\n",
       "      <td>5</td>\n",
       "      <td>BIH</td>\n",
       "      <td>missing</td>\n",
       "      <td>missing</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>Iran</td>\n",
       "      <td>4</td>\n",
       "      <td>IRN</td>\n",
       "      <td>missing</td>\n",
       "      <td>missing</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>Ivory Coast</td>\n",
       "      <td>1</td>\n",
       "      <td>CIV</td>\n",
       "      <td>missing</td>\n",
       "      <td>missing</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>55</th>\n",
       "      <td>Palestine</td>\n",
       "      <td>2</td>\n",
       "      <td>PSE</td>\n",
       "      <td>missing</td>\n",
       "      <td>missing</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>60</th>\n",
       "      <td>Republic Of The Congo</td>\n",
       "      <td>3</td>\n",
       "      <td>COG</td>\n",
       "      <td>missing</td>\n",
       "      <td>missing</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>62</th>\n",
       "      <td>Russia</td>\n",
       "      <td>25</td>\n",
       "      <td>RUS</td>\n",
       "      <td>missing</td>\n",
       "      <td>missing</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>69</th>\n",
       "      <td>South Korea</td>\n",
       "      <td>22</td>\n",
       "      <td>KOR</td>\n",
       "      <td>missing</td>\n",
       "      <td>missing</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>76</th>\n",
       "      <td>The Bahamas</td>\n",
       "      <td>1</td>\n",
       "      <td>BHS</td>\n",
       "      <td>missing</td>\n",
       "      <td>missing</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                    Nation  Count country_code   Divide   Region\n",
       "10  Bosnia And Herzegovina      5          BIH  missing  missing\n",
       "35                    Iran      4          IRN  missing  missing\n",
       "39             Ivory Coast      1          CIV  missing  missing\n",
       "55               Palestine      2          PSE  missing  missing\n",
       "60   Republic Of The Congo      3          COG  missing  missing\n",
       "62                  Russia     25          RUS  missing  missing\n",
       "69             South Korea     22          KOR  missing  missing\n",
       "76             The Bahamas      1          BHS  missing  missing"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#same for MoMA\n",
    "moma_north_south = moma_countries_count.copy()\n",
    "moma_north_south['Divide'] = moma_north_south['Nation'].apply(lambda nation: getDivide(nation))\n",
    "moma_north_south['Region'] = moma_north_south['Nation'].apply(lambda nation: getRegion(nation))\n",
    "moma_north_south = moma_north_south.sort_values(by='Nation').reset_index(drop=True)\n",
    "\n",
    "#check for missing info\n",
    "moma_north_south.loc[moma_north_south['Divide'] == 'missing']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "#fix them \n",
    "moma_north_south.loc[10, 'Divide'] = 'Global North'\n",
    "moma_north_south.loc[10, 'Region'] = 'Europe'\n",
    "moma_north_south.loc[35, 'Divide'] = 'Global South'\n",
    "moma_north_south.loc[35, 'Region'] = 'Middle east'\n",
    "moma_north_south.loc[39, 'Divide'] = 'Global South'\n",
    "moma_north_south.loc[39, 'Region'] = 'Africa'\n",
    "moma_north_south.loc[55, 'Divide'] = 'Global South'\n",
    "moma_north_south.loc[55, 'Region'] = 'Arab States'\n",
    "moma_north_south.loc[60, 'Divide'] = 'Global South'\n",
    "moma_north_south.loc[60, 'Region'] = 'Africa'\n",
    "moma_north_south.loc[62, 'Divide'] = 'Global North'\n",
    "moma_north_south.loc[62, 'Region'] = 'Europe'\n",
    "moma_north_south.loc[69, 'Divide'] = 'Global North'\n",
    "moma_north_south.loc[69, 'Region'] = 'Asia & Pacific'\n",
    "moma_north_south.loc[76, 'Divide'] = 'Global South'\n",
    "moma_north_south.loc[76, 'Region'] = 'South/Latin America'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Nation</th>\n",
       "      <th>Count</th>\n",
       "      <th>country_code</th>\n",
       "      <th>Divide</th>\n",
       "      <th>Region</th>\n",
       "      <th>Source</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Afghanistan</td>\n",
       "      <td>1</td>\n",
       "      <td>AFG</td>\n",
       "      <td>Global South</td>\n",
       "      <td>Asia &amp; Pacific</td>\n",
       "      <td>MoMA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Albania</td>\n",
       "      <td>2</td>\n",
       "      <td>ALB</td>\n",
       "      <td>Global North</td>\n",
       "      <td>Europe</td>\n",
       "      <td>MoMA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Algeria</td>\n",
       "      <td>3</td>\n",
       "      <td>DZA</td>\n",
       "      <td>Global South</td>\n",
       "      <td>Arab States</td>\n",
       "      <td>MoMA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Argentina</td>\n",
       "      <td>20</td>\n",
       "      <td>ARG</td>\n",
       "      <td>Global South</td>\n",
       "      <td>South/Latin America</td>\n",
       "      <td>MoMA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Australia</td>\n",
       "      <td>10</td>\n",
       "      <td>AUS</td>\n",
       "      <td>Global North</td>\n",
       "      <td>Asia &amp; Pacific</td>\n",
       "      <td>MoMA</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        Nation  Count country_code        Divide               Region Source\n",
       "0  Afghanistan      1          AFG  Global South       Asia & Pacific   MoMA\n",
       "1      Albania      2          ALB  Global North               Europe   MoMA\n",
       "2      Algeria      3          DZA  Global South          Arab States   MoMA\n",
       "3    Argentina     20          ARG  Global South  South/Latin America   MoMA\n",
       "4    Australia     10          AUS  Global North       Asia & Pacific   MoMA"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#combine the two\n",
    "moma_north_south['Source'] = 'MoMA'\n",
    "rhz_north_south['Source'] = 'Rhizome'\n",
    "north_v_south = pd.concat([moma_north_south, rhz_north_south])\n",
    "north_v_south.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 150,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "alignmentgroup": "True",
         "hoverinfo": "x+y+name",
         "legendgroup": "MoMA (sampled)",
         "marker": {
          "color": "#aab6fe",
          "pattern": {
           "shape": ""
          }
         },
         "name": "MoMA (sampled)",
         "offsetgroup": "MoMA (sampled)",
         "orientation": "v",
         "showlegend": true,
         "textposition": "inside",
         "texttemplate": "%{y}",
         "type": "bar",
         "x": [
          "Global North",
          "Global South"
         ],
         "xaxis": "x",
         "y": [
          2252,
          337
         ],
         "yaxis": "y"
        },
        {
         "alignmentgroup": "True",
         "hoverinfo": "x+y+name",
         "legendgroup": "Rhizome",
         "marker": {
          "color": "#9ccc65",
          "pattern": {
           "shape": ""
          }
         },
         "name": "Rhizome",
         "offsetgroup": "Rhizome",
         "orientation": "v",
         "showlegend": true,
         "textposition": "inside",
         "texttemplate": "%{y}",
         "type": "bar",
         "x": [
          "Global North",
          "Global South"
         ],
         "xaxis": "x",
         "y": [
          709,
          67
         ],
         "yaxis": "y"
        }
       ],
       "layout": {
        "barmode": "group",
        "legend": {
         "title": {
          "text": "Source"
         },
         "tracegroupgap": 0
        },
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "rgb(36,36,36)"
            },
            "error_y": {
             "color": "rgb(36,36,36)"
            },
            "marker": {
             "line": {
              "color": "white",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "white",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "rgb(36,36,36)",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "rgb(36,36,36)"
            },
            "baxis": {
             "endlinecolor": "rgb(36,36,36)",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "rgb(36,36,36)"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "line": {
              "color": "white",
              "width": 0.6
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "fillpattern": {
             "fillmode": "overlay",
             "size": 10,
             "solidity": 0.2
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "rgb(237,237,237)"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "rgb(217,217,217)"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 1,
            "tickcolor": "rgb(36,36,36)",
            "ticks": "outside"
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "rgb(103,0,31)"
            ],
            [
             0.1,
             "rgb(178,24,43)"
            ],
            [
             0.2,
             "rgb(214,96,77)"
            ],
            [
             0.3,
             "rgb(244,165,130)"
            ],
            [
             0.4,
             "rgb(253,219,199)"
            ],
            [
             0.5,
             "rgb(247,247,247)"
            ],
            [
             0.6,
             "rgb(209,229,240)"
            ],
            [
             0.7,
             "rgb(146,197,222)"
            ],
            [
             0.8,
             "rgb(67,147,195)"
            ],
            [
             0.9,
             "rgb(33,102,172)"
            ],
            [
             1,
             "rgb(5,48,97)"
            ]
           ],
           "sequential": [
            [
             0,
             "#440154"
            ],
            [
             0.1111111111111111,
             "#482878"
            ],
            [
             0.2222222222222222,
             "#3e4989"
            ],
            [
             0.3333333333333333,
             "#31688e"
            ],
            [
             0.4444444444444444,
             "#26828e"
            ],
            [
             0.5555555555555556,
             "#1f9e89"
            ],
            [
             0.6666666666666666,
             "#35b779"
            ],
            [
             0.7777777777777778,
             "#6ece58"
            ],
            [
             0.8888888888888888,
             "#b5de2b"
            ],
            [
             1,
             "#fde725"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#440154"
            ],
            [
             0.1111111111111111,
             "#482878"
            ],
            [
             0.2222222222222222,
             "#3e4989"
            ],
            [
             0.3333333333333333,
             "#31688e"
            ],
            [
             0.4444444444444444,
             "#26828e"
            ],
            [
             0.5555555555555556,
             "#1f9e89"
            ],
            [
             0.6666666666666666,
             "#35b779"
            ],
            [
             0.7777777777777778,
             "#6ece58"
            ],
            [
             0.8888888888888888,
             "#b5de2b"
            ],
            [
             1,
             "#fde725"
            ]
           ]
          },
          "colorway": [
           "#1F77B4",
           "#FF7F0E",
           "#2CA02C",
           "#D62728",
           "#9467BD",
           "#8C564B",
           "#E377C2",
           "#7F7F7F",
           "#BCBD22",
           "#17BECF"
          ],
          "font": {
           "color": "rgb(36,36,36)"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "white",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "white",
          "polar": {
           "angularaxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           },
           "bgcolor": "white",
           "radialaxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "white",
            "gridcolor": "rgb(232,232,232)",
            "gridwidth": 2,
            "linecolor": "rgb(36,36,36)",
            "showbackground": true,
            "showgrid": false,
            "showline": true,
            "ticks": "outside",
            "zeroline": false,
            "zerolinecolor": "rgb(36,36,36)"
           },
           "yaxis": {
            "backgroundcolor": "white",
            "gridcolor": "rgb(232,232,232)",
            "gridwidth": 2,
            "linecolor": "rgb(36,36,36)",
            "showbackground": true,
            "showgrid": false,
            "showline": true,
            "ticks": "outside",
            "zeroline": false,
            "zerolinecolor": "rgb(36,36,36)"
           },
           "zaxis": {
            "backgroundcolor": "white",
            "gridcolor": "rgb(232,232,232)",
            "gridwidth": 2,
            "linecolor": "rgb(36,36,36)",
            "showbackground": true,
            "showgrid": false,
            "showline": true,
            "ticks": "outside",
            "zeroline": false,
            "zerolinecolor": "rgb(36,36,36)"
           }
          },
          "shapedefaults": {
           "fillcolor": "black",
           "line": {
            "width": 0
           },
           "opacity": 0.3
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           },
           "baxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           },
           "bgcolor": "white",
           "caxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "rgb(232,232,232)",
           "linecolor": "rgb(36,36,36)",
           "showgrid": false,
           "showline": true,
           "ticks": "outside",
           "title": {
            "standoff": 15
           },
           "zeroline": false,
           "zerolinecolor": "rgb(36,36,36)"
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "rgb(232,232,232)",
           "linecolor": "rgb(36,36,36)",
           "showgrid": false,
           "showline": true,
           "ticks": "outside",
           "title": {
            "standoff": 15
           },
           "zeroline": false,
           "zerolinecolor": "rgb(36,36,36)"
          }
         }
        },
        "title": {
         "text": "The Global North/South Divide across Rhizome and MoMA (sampled)",
         "x": 0.5
        },
        "xaxis": {
         "anchor": "y",
         "domain": [
          0,
          1
         ],
         "showticklabels": true,
         "title": {
          "text": "Divide"
         }
        },
        "yaxis": {
         "anchor": "x",
         "domain": [
          0,
          1
         ],
         "showticklabels": true,
         "title": {
          "text": "Artwork Count"
         }
        }
       }
      }
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "north_south_viz = north_v_south.groupby(['Source', 'Divide'],as_index=False).agg({'Count': 'sum'})\n",
    "north_south_viz = north_south_viz.replace(['MoMA'], 'MoMA (sampled)')\n",
    "north_south_bars = px.bar(north_south_viz, x=\"Divide\", y=\"Count\", color='Source', title=\"The Global North/South Divide across Rhizome and MoMA (sampled)\", barmode='group', text_auto=True, color_discrete_map ={\"MoMA (sampled)\":\"#aab6fe\", \"Rhizome\":\"#9ccc65\"}, category_orders={\"Source\": [\"MoMA (sampled)\", 'Rhizome']}, labels={\"Count\": \"Artwork Count\"})\n",
    "north_south_bars.update_xaxes(showticklabels=True)\n",
    "north_south_bars.update_yaxes(showticklabels=True)\n",
    "north_south_bars.update_traces(textposition=\"inside\", hoverinfo='x+y+name', hovertemplate=None)\n",
    "north_south_bars.update_layout(height=800, width=800, title_x=0.5)\n",
    "#create local copies\n",
    "#north_south_bars.write_html(path+\"Plotly_embeds/rhz_moma_north_south_bars.html\", include_plotlyjs='directory',   full_html=False, default_width='100%')\n",
    "north_south_bars.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "#focus only on Global South\n",
    "rhz_moma_south = north_v_south.loc[north_v_south['Divide'] == 'Global South'] \n",
    "rhz_moma_south_nats = rhz_moma_south.groupby(['Source', 'Nation', 'Region'],as_index=False).agg({'Count': 'sum'})\n",
    "rhz_moma_south_regions = rhz_moma_south.groupby(['Source', 'Region'],as_index=False).agg({'Count': 'sum'})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "cb0ec1a6386a4af6a51df6b34dea2ccd",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(FigureWidget({\n",
       "    'data': [{'alignmentgroup': 'True',\n",
       "              'hoverinfo': 'x+y+name',\n",
       " …"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#BAR IT! \n",
    "rhz_moma_south_nats = rhz_moma_south_nats.replace(['MoMA'], 'MoMA (sampled)')\n",
    "north_south_nats = px.bar(rhz_moma_south_nats, x=\"Nation\", y=\"Count\", color='Source', title=\"Artists from the Global South by Nationality - Rhizome vs MoMA (sampled)\", color_discrete_map ={\"MoMA (sampled)\":\"#aab6fe\", \"Rhizome\":\"#9ccc65\"},\n",
    "category_orders={\"Source\": [\"MoMA (sampled)\", 'Rhizome']}, text_auto=True, barmode='stack', labels={\"Count\": \"Artist Count\"})\n",
    "north_south_nats.update_xaxes(showticklabels=True, categoryorder='sum descending', tickangle=45)\n",
    "north_south_nats.update_yaxes(showticklabels=True)\n",
    "north_south_nats.update_traces(textposition=\"outside\", hoverinfo='x+y+name', hovertemplate=None)\n",
    "north_south_nats.update_layout(title_x=0.5)\n",
    "\n",
    "rhz_moma_south_regions = rhz_moma_south_regions.replace(['MoMA'], 'MoMA (sampled)')\n",
    "north_south_regs = px.bar(rhz_moma_south_regions, x=\"Region\", y=\"Count\", color='Source', title=\"Artists from the Global South by Regions - Rhizome vs MoMA (sampled)\", color_discrete_map ={\"MoMA (sampled)\":\"#aab6fe\", \"Rhizome\":\"#9ccc65\"},\n",
    "category_orders={\"Source\": [\"MoMA (sampled)\", 'Rhizome']}, text_auto=True, barmode='group', labels={\"Count\": \"Artist Count\", 'Region': 'Global South Region'})\n",
    "north_south_regs.update_xaxes(showticklabels=True, categoryorder='sum descending')\n",
    "north_south_regs.update_yaxes(showticklabels=True)\n",
    "north_south_regs.update_traces(textposition=\"outside\", hoverinfo='x+y+name', hovertemplate=None)\n",
    "north_south_regs.update_layout(title_x=0.5)\n",
    "\n",
    "#create local copies\n",
    "#north_south_nats.write_html(path+\"Plotly_embeds/rhz_moma_north_south_bars_nats.html\", include_plotlyjs='directory',  full_html=False,  default_width='100%')\n",
    "#north_south_regs.write_html(path+\"Plotly_embeds/rhz_moma_north_south_bars_regions.html\", include_plotlyjs='directory',  full_html=False,  default_width='100%')\n",
    "\n",
    "north_south_nats = go.FigureWidget(north_south_nats)\n",
    "north_south_regs = go.FigureWidget(north_south_regs)\n",
    "north_south = ipw.VBox([north_south_nats, north_south_regs])\n",
    "north_south"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Relationship between Gender and Nationality (Everyone)\n",
    "\n",
    "For the combination of gender and nationality variables we decided to try parallel charts in order to give the user an insight into how gender and nationality related with the additional categorical variable of departments and an overall filter on the data using the Global North/South narrative, which allowed us to ensure that the parallel charts remained legible by reducing nationalities to their regional grouping. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "#create a new Rhizome DF w/ gender + department info\n",
    "missing = pd.DataFrame({'Aalborgenser': ['Korean', 'Native American', 'Canadian Inuit'], 'Aalborg': ['Korea', 'United States', 'Canada']})\n",
    "parser = pd.read_csv('https://raw.githubusercontent.com/knowitall/chunkedextractor/master/src/main/resources/edu/knowitall/chunkedextractor/demonyms.csv')\n",
    "corrections = pd.concat([missing, parser])\n",
    "rhz_parallel = rhz_medium.copy()\n",
    "rhz_parallel['Nation'] = ''\n",
    "nats = set(rhz_parallel.Nationality)\n",
    "for item in nats:\n",
    "    my = corrections[corrections['Aalborgenser'] == item]\n",
    "    country = my[:1]['Aalborg'].values\n",
    "    if len(country)>0:\n",
    "        country_str = my[:1]['Aalborg'].values[0]\n",
    "        rhz_parallel.loc[rhz_parallel[\"Nationality\"] == item, \"Nation\"] = country_str\n",
    "rhz_parallel['Nation'] = rhz_parallel['Nation'].replace('', 'missing')\n",
    "rhz_parallel['country_code'] = rhz_parallel['Nation'].apply(lambda country: do_fuzzy_search(country))\n",
    "rhz_parallel['Divide'] = rhz_parallel['Nation'].apply(lambda country: getDivide(country))\n",
    "rhz_parallel['Region'] = rhz_parallel['Nation'].apply(lambda country: getRegion(country))\n",
    "rhz_parallel = rhz_parallel.drop(['ID'], axis=1)\n",
    "rhz_parallel = rhz_parallel.rename(columns={\"Medium\": \"Department\"})\n",
    "rhz_parallel['country_code'] = rhz_parallel['country_code'].fillna('missing')\n",
    "rhz_parallel['Source'] = 'Rhizome'\n",
    "rhz_parallel = rhz_parallel.replace(['Installation & Performance', 'Net Art'], 'Media and Performance') \n",
    "rhz_parallel = rhz_parallel.loc[rhz_parallel['Department'] != 'missing'] \n",
    "rhz_parallel.to_pickle(path+'Rhizome_data/pickles/rhizome_parallel.pkl')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "#export to excel to manually clean up missing country values\n",
    "#rhz_parallel.to_excel(path+'Rhizome_data/csv/rhz_parallel.xlsx')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "rhz_parallel = pd.read_excel(path+'Rhizome_data/csv/rhz_parallel_clean.xlsx')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "#same for MoMA \n",
    "moma_parallel = moma_rhz_compare.copy()\n",
    "moma_parallel['Nation'] = ''\n",
    "nats_moma = set(moma_parallel.Nationality)\n",
    "for item in nats_moma:\n",
    "    my = corrections[corrections['Aalborgenser'] == item]\n",
    "    country = my[:1]['Aalborg'].values\n",
    "    if len(country)>0:\n",
    "        country_str = my[:1]['Aalborg'].values[0]\n",
    "        moma_parallel.loc[moma_parallel[\"Nationality\"] == item, \"Nation\"] = country_str\n",
    "moma_parallel['Nation'] = moma_parallel['Nation'].replace('', 'missing')\n",
    "moma_parallel['country_code'] = moma_parallel['Nation'].apply(lambda country: do_fuzzy_search(country))\n",
    "moma_parallel['Divide'] = moma_parallel['Nation'].apply(lambda country: getDivide(country))\n",
    "moma_parallel['Region'] = moma_parallel['Nation'].apply(lambda country: getRegion(country))\n",
    "moma_parallel = moma_parallel.drop(['ThumbnailURL', 'ConstituentID', 'Medium'], axis=1)\n",
    "moma_parallel['country_code'] = moma_parallel['country_code'].fillna('missing')\n",
    "moma_parallel['Source'] = 'MoMA (sampled)'\n",
    "moma_parallel = moma_parallel.replace(['Photography', 'Drawings & Prints'], 'Still Images').replace(['Film'], 'Moving Images')\n",
    "moma_parallel = moma_parallel.loc[moma_parallel['Department'] != 'Architecture & Design']\n",
    "moma_parallel = moma_parallel.loc[moma_parallel['Department'] != 'Fluxus Collection'] \n",
    "moma_parallel = moma_parallel.loc[moma_parallel['Department'] != 'Painting & Sculpture'] \n",
    "moma_parallel.to_pickle(path+'MOMA_data/pickle/moma_parallel.pkl')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "#export to excel to manually clean up missing country values\n",
    "#moma_parallel.to_csv(path+'MOMA_data/csv/moma_parallel.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "moma_parallel = pd.read_excel(path+'MOMA_data/csv/moma_parallel_clean.xlsx')\n",
    "moma_parallel.columns = moma_parallel.iloc[0]\n",
    "moma_parallel = moma_parallel.drop(0, axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [],
   "source": [
    "#combine them, remove entries with missing gender/divide info, add a numerical version of gender for parallel viz\n",
    "rhz_parallel_both = rhz_parallel.copy()\n",
    "rhz_parallel_both = rhz_parallel_both.loc[(rhz_parallel_both['Gender'] != 'missing') & (rhz_parallel_both['Divide'] != 'missing')]\n",
    "moma_parallel_both = moma_parallel.copy()\n",
    "moma_parallel_both = moma_parallel_both.loc[(moma_parallel_both['Gender'] != 'missing')]\n",
    "moma_rhz_parallel_both = pd.concat([rhz_parallel_both, moma_parallel_both])\n",
    "moma_rhz_parallel_both['Gender_ID'] = moma_rhz_parallel_both['Gender']\n",
    "moma_rhz_parallel_both['Gender_ID'] = moma_rhz_parallel_both['Gender_ID'].replace('M', 0)\n",
    "moma_rhz_parallel_both['Gender_ID'] = moma_rhz_parallel_both['Gender_ID'].replace('F', 1)\n",
    "moma_rhz_parallel_both['Gender_ID'] = moma_rhz_parallel_both['Gender_ID'].replace('NB', 3)\n",
    "moma_rhz_parallel_both['Gender_ID'] = moma_rhz_parallel_both['Gender_ID'].astype(str)\n",
    "moma_rhz_parallel_both.loc[moma_rhz_parallel_both.Gender_ID.str.contains(','), 'Gender_ID'] = '2'\n",
    "moma_rhz_parallel_both['Gender_ID'] = moma_rhz_parallel_both['Gender_ID'].astype(int)\n",
    "moma_rhz_parallel_both_artists = moma_rhz_parallel_both.drop_duplicates(subset='Artist', keep='last').reset_index(drop=True)\n",
    "#fix a cleaning error \n",
    "moma_rhz_parallel_both_artists.loc[767, 'Divide'] = 'Global North'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 152,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "4743acf2d71c498fabd783fdc88b2215",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(FigureWidget({\n",
       "    'data': [{'arrangement': 'freeform',\n",
       "              'dimensions': [{'category…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#RHZ // Sets - by region, global south\n",
    "\n",
    "rhz_only = moma_rhz_parallel_both_artists.loc[moma_rhz_parallel_both_artists['Source'] == 'Rhizome']\n",
    "rhz_only = rhz_only.loc[rhz_only['Department'] != 'missing']\n",
    "rhz_only = rhz_only.loc[rhz_only['Divide'] == 'Global South']\n",
    "\n",
    "#Create dimensions\n",
    "\n",
    "#Regions\n",
    "region_dim_rhz = go.parcats.Dimension(\n",
    "    values=rhz_only.Region,\n",
    "    categoryorder='category ascending', label=\"Region\"\n",
    ")\n",
    "\n",
    "#departments\n",
    "dept_dim_rhz = go.parcats.Dimension(values=rhz_only.Department, label=\"Department\")\n",
    "\n",
    "\n",
    "#genders\n",
    "gender_dim_rhz = go.parcats.Dimension(\n",
    "    values=rhz_only.Gender_ID, label=\"Gender\", categoryarray=[0, 1, 2],\n",
    "    ticktext=['Male', 'Female', 'Collective']\n",
    ")\n",
    "\n",
    "# Create parcats trace\n",
    "rhz_color = rhz_only.Gender_ID;\n",
    "rhz_colorscale = ['#007c91','#ff5252', '#ce93d8'];\n",
    "\n",
    "\n",
    "rhz_parallel_region_south = go.Figure(data = [go.Parcats(dimensions=[region_dim_rhz, gender_dim_rhz, dept_dim_rhz], \n",
    "        line={'color': rhz_color, 'colorscale': rhz_colorscale},\n",
    "        hoveron='color', hoverinfo='count',\n",
    "        labelfont={'size': 12},\n",
    "        tickfont={'size': 12},\n",
    "        arrangement='freeform')])\n",
    "rhz_parallel_region_south.update_traces(dimensions=[{'categoryorder':'array'}])\n",
    "rhz_parallel_region_south.update_layout(title=\"Relationship between Region, Gender, and Department of Global South Artists - Rhizome\", title_x=0.5)\n",
    "\n",
    "#MoMA // Sets - by region, global south\n",
    "\n",
    "moma_only = moma_rhz_parallel_both_artists.loc[moma_rhz_parallel_both_artists['Source'] == 'MoMA (sampled)']\n",
    "moma_only = moma_only.loc[moma_only['Divide'] == 'Global South']\n",
    "\n",
    "#Create dimensions\n",
    "\n",
    "#Regions\n",
    "region_dim_moma = go.parcats.Dimension(\n",
    "    values=moma_only.Region,\n",
    "    categoryorder='category ascending', label=\"Region\"\n",
    ")\n",
    "\n",
    "#departments\n",
    "dept_dim_moma = go.parcats.Dimension(values=moma_only.Department, label=\"Department\")\n",
    "\n",
    "#genders\n",
    "gender_dim_moma = go.parcats.Dimension(\n",
    "    values=moma_only.Gender_ID, label=\"Gender\", categoryarray=[0, 1, 2],\n",
    "    ticktext=['Male', 'Female', 'Collective']\n",
    ")\n",
    "\n",
    "# Create parcats trace\n",
    "moma_color = moma_only.Gender_ID;\n",
    "moma_colorscale = ['#007c91','#ff5252', '#ce93d8'];\n",
    "\n",
    "moma_parallel_region_south = go.Figure(data = [go.Parcats(dimensions=[region_dim_moma, gender_dim_moma, dept_dim_moma], \n",
    "        line={'color': moma_color, 'colorscale': moma_colorscale},\n",
    "        hoveron='color', hoverinfo='count',\n",
    "        labelfont={'size': 12},\n",
    "        tickfont={'size': 12},\n",
    "        arrangement='freeform')])\n",
    "moma_parallel_region_south.update_traces(dimensions=[{'categoryorder':'array'}])\n",
    "moma_parallel_region_south.update_layout(title=\"Relationship between Region, Gender, and Department of Global South Artists - MoMA (sampled)\", title_x=0.5)\n",
    "\n",
    "#create local copies\n",
    "#moma_parallel_region_south.write_html(path+\"Plotly_embeds/moma_paralle_regions.html\", include_plotlyjs='directory',  full_html=False, default_width='100%')\n",
    "#rhz_parallel_region_south.write_html(path+\"Plotly_embeds/rhz_paralle_regions.html\", include_plotlyjs='directory',  full_html=False, default_width='100%')\n",
    "\n",
    "moma_parallel_region_south = go.FigureWidget(moma_parallel_region_south)\n",
    "rhz_parallel_region_south = go.FigureWidget(rhz_parallel_region_south)\n",
    "parallel_both = ipw.VBox([rhz_parallel_region_south, moma_parallel_region_south])\n",
    "parallel_both\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Relationship between Time, Gender, and Nationality (Everyone)\n",
    "\n",
    "Combining time, gender, and nationality proved challenging and we ultimately settled on bubble scatter charts as the best way to see if there were patterns of acquisition at play.<br><br>\n",
    "The first set of scatter plots follows through from the parallel charts by looking at the relationship between the creation and acquisition date of an artwork for artists from the Global South, with gender as a facet to avoid confusing the plot.<br><br>\n",
    "The second scatter plot is a more simple look at the relationship between creation and acquisition date by gender, which reveals 2007 as a key intersectional date for Rhizome as well as show how they stopped acquiring artworks for most of the 2010s following this. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>URL</th>\n",
       "      <th>Title</th>\n",
       "      <th>Artist</th>\n",
       "      <th>dateAcquired</th>\n",
       "      <th>dateCreated</th>\n",
       "      <th>Nationality</th>\n",
       "      <th>Gender</th>\n",
       "      <th>Text</th>\n",
       "      <th>Keywords</th>\n",
       "      <th>Department</th>\n",
       "      <th>Nation</th>\n",
       "      <th>country_code</th>\n",
       "      <th>Divide</th>\n",
       "      <th>Region</th>\n",
       "      <th>Source</th>\n",
       "      <th>Gender_ID</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1351</th>\n",
       "      <td>https://artbase.rhizome.org/wiki/Q2093</td>\n",
       "      <td>GLOBAL PRESIDENTIAL ELECTION</td>\n",
       "      <td>Kenneth Hung</td>\n",
       "      <td>2004</td>\n",
       "      <td>2004</td>\n",
       "      <td>Chinese</td>\n",
       "      <td>M</td>\n",
       "      <td>global presidential election  world one love o...</td>\n",
       "      <td>global, democracy, presidential, election, vot...</td>\n",
       "      <td>Media and Performance</td>\n",
       "      <td>China</td>\n",
       "      <td>CHN</td>\n",
       "      <td>Global South</td>\n",
       "      <td>Asia &amp; Pacific</td>\n",
       "      <td>Rhizome</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1987</th>\n",
       "      <td>https://artbase.rhizome.org/wiki/Q1654</td>\n",
       "      <td>60X1.COM</td>\n",
       "      <td>Kenneth Hung</td>\n",
       "      <td>2002</td>\n",
       "      <td>2001</td>\n",
       "      <td>Chinese</td>\n",
       "      <td>M</td>\n",
       "      <td>artist statementnhttpwww1111111111111111111111...</td>\n",
       "      <td>internet, asian, prince, identity, one, conten...</td>\n",
       "      <td>Still Images</td>\n",
       "      <td>China</td>\n",
       "      <td>CHN</td>\n",
       "      <td>Global South</td>\n",
       "      <td>Asia &amp; Pacific</td>\n",
       "      <td>Rhizome</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1988</th>\n",
       "      <td>https://artbase.rhizome.org/wiki/Q2635</td>\n",
       "      <td>60X1.CAM(tm) Ultimate Interactive Webcam Surve...</td>\n",
       "      <td>Kenneth Hung</td>\n",
       "      <td>2002</td>\n",
       "      <td>2002</td>\n",
       "      <td>Chinese</td>\n",
       "      <td>M</td>\n",
       "      <td>start war terrorism home todaynnow easily trac...</td>\n",
       "      <td>war, home, tactical, december, optimator, yerb...</td>\n",
       "      <td>Media and Performance</td>\n",
       "      <td>China</td>\n",
       "      <td>CHN</td>\n",
       "      <td>Global South</td>\n",
       "      <td>Asia &amp; Pacific</td>\n",
       "      <td>Rhizome</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>751</th>\n",
       "      <td>http://www.moma.org/collection/works/83748</td>\n",
       "      <td>Stone Stele #14</td>\n",
       "      <td>Wenda Gu</td>\n",
       "      <td>2002</td>\n",
       "      <td>2001</td>\n",
       "      <td>Chinese</td>\n",
       "      <td>M</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Still Images</td>\n",
       "      <td>China</td>\n",
       "      <td>CHN</td>\n",
       "      <td>Global South</td>\n",
       "      <td>Asia &amp; Pacific</td>\n",
       "      <td>MoMA (sampled)</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1449</th>\n",
       "      <td>http://www.moma.org/collection/works/88021</td>\n",
       "      <td>The Elegant Gathering</td>\n",
       "      <td>Yun-Fei Ji</td>\n",
       "      <td>2006</td>\n",
       "      <td>2002</td>\n",
       "      <td>Chinese</td>\n",
       "      <td>M</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Still Images</td>\n",
       "      <td>China</td>\n",
       "      <td>CHN</td>\n",
       "      <td>Global South</td>\n",
       "      <td>Asia &amp; Pacific</td>\n",
       "      <td>MoMA (sampled)</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18174</th>\n",
       "      <td>http://www.moma.org/collection/works/404971</td>\n",
       "      <td>Untitled from the series 1999</td>\n",
       "      <td>Zhang Dali</td>\n",
       "      <td>2019</td>\n",
       "      <td>1998</td>\n",
       "      <td>Chinese</td>\n",
       "      <td>M</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Still Images</td>\n",
       "      <td>China</td>\n",
       "      <td>CHN</td>\n",
       "      <td>Global South</td>\n",
       "      <td>Asia &amp; Pacific</td>\n",
       "      <td>MoMA (sampled)</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18175</th>\n",
       "      <td>http://www.moma.org/collection/works/404972</td>\n",
       "      <td>Untitled from the series 1999</td>\n",
       "      <td>Zhang Dali</td>\n",
       "      <td>2019</td>\n",
       "      <td>1998</td>\n",
       "      <td>Chinese</td>\n",
       "      <td>M</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Still Images</td>\n",
       "      <td>China</td>\n",
       "      <td>CHN</td>\n",
       "      <td>Global South</td>\n",
       "      <td>Asia &amp; Pacific</td>\n",
       "      <td>MoMA (sampled)</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18176</th>\n",
       "      <td>http://www.moma.org/collection/works/404973</td>\n",
       "      <td>Untitled from the series 1999</td>\n",
       "      <td>Zhang Dali</td>\n",
       "      <td>2019</td>\n",
       "      <td>1998</td>\n",
       "      <td>Chinese</td>\n",
       "      <td>M</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Still Images</td>\n",
       "      <td>China</td>\n",
       "      <td>CHN</td>\n",
       "      <td>Global South</td>\n",
       "      <td>Asia &amp; Pacific</td>\n",
       "      <td>MoMA (sampled)</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18177</th>\n",
       "      <td>http://www.moma.org/collection/works/404974</td>\n",
       "      <td>Untitled from the series 1999</td>\n",
       "      <td>Zhang Dali</td>\n",
       "      <td>2019</td>\n",
       "      <td>1998</td>\n",
       "      <td>Chinese</td>\n",
       "      <td>M</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Still Images</td>\n",
       "      <td>China</td>\n",
       "      <td>CHN</td>\n",
       "      <td>Global South</td>\n",
       "      <td>Asia &amp; Pacific</td>\n",
       "      <td>MoMA (sampled)</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18178</th>\n",
       "      <td>http://www.moma.org/collection/works/404975</td>\n",
       "      <td>Untitled from the series 1999</td>\n",
       "      <td>Zhang Dali</td>\n",
       "      <td>2019</td>\n",
       "      <td>1998</td>\n",
       "      <td>Chinese</td>\n",
       "      <td>M</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Still Images</td>\n",
       "      <td>China</td>\n",
       "      <td>CHN</td>\n",
       "      <td>Global South</td>\n",
       "      <td>Asia &amp; Pacific</td>\n",
       "      <td>MoMA (sampled)</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>236 rows × 16 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                               URL  \\\n",
       "1351        https://artbase.rhizome.org/wiki/Q2093   \n",
       "1987        https://artbase.rhizome.org/wiki/Q1654   \n",
       "1988        https://artbase.rhizome.org/wiki/Q2635   \n",
       "751     http://www.moma.org/collection/works/83748   \n",
       "1449    http://www.moma.org/collection/works/88021   \n",
       "...                                            ...   \n",
       "18174  http://www.moma.org/collection/works/404971   \n",
       "18175  http://www.moma.org/collection/works/404972   \n",
       "18176  http://www.moma.org/collection/works/404973   \n",
       "18177  http://www.moma.org/collection/works/404974   \n",
       "18178  http://www.moma.org/collection/works/404975   \n",
       "\n",
       "                                                   Title        Artist  \\\n",
       "1351                        GLOBAL PRESIDENTIAL ELECTION  Kenneth Hung   \n",
       "1987                                            60X1.COM  Kenneth Hung   \n",
       "1988   60X1.CAM(tm) Ultimate Interactive Webcam Surve...  Kenneth Hung   \n",
       "751                                      Stone Stele #14      Wenda Gu   \n",
       "1449                               The Elegant Gathering    Yun-Fei Ji   \n",
       "...                                                  ...           ...   \n",
       "18174                      Untitled from the series 1999    Zhang Dali   \n",
       "18175                      Untitled from the series 1999    Zhang Dali   \n",
       "18176                      Untitled from the series 1999    Zhang Dali   \n",
       "18177                      Untitled from the series 1999    Zhang Dali   \n",
       "18178                      Untitled from the series 1999    Zhang Dali   \n",
       "\n",
       "      dateAcquired dateCreated Nationality Gender  \\\n",
       "1351          2004        2004     Chinese      M   \n",
       "1987          2002        2001     Chinese      M   \n",
       "1988          2002        2002     Chinese      M   \n",
       "751           2002        2001     Chinese      M   \n",
       "1449          2006        2002     Chinese      M   \n",
       "...            ...         ...         ...    ...   \n",
       "18174         2019        1998     Chinese      M   \n",
       "18175         2019        1998     Chinese      M   \n",
       "18176         2019        1998     Chinese      M   \n",
       "18177         2019        1998     Chinese      M   \n",
       "18178         2019        1998     Chinese      M   \n",
       "\n",
       "                                                    Text  \\\n",
       "1351   global presidential election  world one love o...   \n",
       "1987   artist statementnhttpwww1111111111111111111111...   \n",
       "1988   start war terrorism home todaynnow easily trac...   \n",
       "751                                                  NaN   \n",
       "1449                                                 NaN   \n",
       "...                                                  ...   \n",
       "18174                                                NaN   \n",
       "18175                                                NaN   \n",
       "18176                                                NaN   \n",
       "18177                                                NaN   \n",
       "18178                                                NaN   \n",
       "\n",
       "                                                Keywords  \\\n",
       "1351   global, democracy, presidential, election, vot...   \n",
       "1987   internet, asian, prince, identity, one, conten...   \n",
       "1988   war, home, tactical, december, optimator, yerb...   \n",
       "751                                                  NaN   \n",
       "1449                                                 NaN   \n",
       "...                                                  ...   \n",
       "18174                                                NaN   \n",
       "18175                                                NaN   \n",
       "18176                                                NaN   \n",
       "18177                                                NaN   \n",
       "18178                                                NaN   \n",
       "\n",
       "                  Department Nation country_code        Divide  \\\n",
       "1351   Media and Performance  China          CHN  Global South   \n",
       "1987            Still Images  China          CHN  Global South   \n",
       "1988   Media and Performance  China          CHN  Global South   \n",
       "751             Still Images  China          CHN  Global South   \n",
       "1449            Still Images  China          CHN  Global South   \n",
       "...                      ...    ...          ...           ...   \n",
       "18174           Still Images  China          CHN  Global South   \n",
       "18175           Still Images  China          CHN  Global South   \n",
       "18176           Still Images  China          CHN  Global South   \n",
       "18177           Still Images  China          CHN  Global South   \n",
       "18178           Still Images  China          CHN  Global South   \n",
       "\n",
       "               Region          Source  Gender_ID  \n",
       "1351   Asia & Pacific         Rhizome          0  \n",
       "1987   Asia & Pacific         Rhizome          0  \n",
       "1988   Asia & Pacific         Rhizome          0  \n",
       "751    Asia & Pacific  MoMA (sampled)          0  \n",
       "1449   Asia & Pacific  MoMA (sampled)          0  \n",
       "...               ...             ...        ...  \n",
       "18174  Asia & Pacific  MoMA (sampled)          0  \n",
       "18175  Asia & Pacific  MoMA (sampled)          0  \n",
       "18176  Asia & Pacific  MoMA (sampled)          0  \n",
       "18177  Asia & Pacific  MoMA (sampled)          0  \n",
       "18178  Asia & Pacific  MoMA (sampled)          0  \n",
       "\n",
       "[236 rows x 16 columns]"
      ]
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "moma_rhz_parallel_both.loc[moma_rhz_parallel_both['Nation'] == 'China']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "moma_rhz_parallel_artworks = moma_rhz_parallel_both.copy()\n",
    "moma_rhz_parallel_artworks = moma_rhz_parallel_artworks.loc[moma_rhz_parallel_artworks['dateCreated'] >= 2000]\n",
    "moma_rhz_parallel_artworks = moma_rhz_parallel_artworks.loc[moma_rhz_parallel_artworks['dateAcquired'] != 0]\n",
    "#filter genders\n",
    "filter_out = ['M', 'F', 'NB', 'missing']\n",
    "compare_collectives = moma_rhz_parallel_artworks[moma_rhz_parallel_artworks.Gender.isin(filter_out) == False]\n",
    "compare_collective_names = set(compare_collectives['Gender'].to_list())\n",
    "moma_rhz_parallel_artworks.loc[moma_rhz_parallel_artworks.Gender.isin(compare_collective_names), 'Gender'] = 'Collectives'\n",
    "\n",
    "#group by\n",
    "moma_rhz_parallel_artworks_counts= moma_rhz_parallel_artworks.groupby(['Source', 'Gender','dateCreated', 'dateAcquired', 'Nationality', 'Nation', 'Divide', 'Region']).size()\n",
    "moma_rhz_parallel_artworks_counts_df = moma_rhz_parallel_artworks_counts.to_frame(name = 'size').reset_index()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [],
   "source": [
    "#compare_to_divide = moma_rhz_parallel_artworks_counts_df.loc[moma_rhz_parallel_artworks_counts_df['Source'] == 'MoMA (sampled)']\n",
    "#compare_to_divide['size'] = compare_to_divide['size'].div(100).round(2)\n",
    "#compare_to_divide['size'] = compare_to_divide['size'].multiply(8).round(2)\n",
    "#compare_to_divide['size'] = compare_to_divide['size'].astype(int)\n",
    "#fix non binary count\n",
    "#compare_to_divide.loc[1829, 'size'] = 1\n",
    "#compare_to_divide.loc[1830, 'size'] = 1\n",
    "moma = moma_rhz_parallel_artworks_counts_df.loc[moma_rhz_parallel_artworks_counts_df['Source'] == 'MoMA (sampled)']\n",
    "rhizome = moma_rhz_parallel_artworks_counts_df.loc[moma_rhz_parallel_artworks_counts_df['Source'] == 'Rhizome']\n",
    "moma_rhz_parallel_artworks_counts_df_final = pd.concat([rhizome, moma])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Colombia<br>Gender=Collectives & Collabs<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Colombia",
         "marker": {
          "color": "#FCD116",
          "size": [
           1
          ],
          "sizemode": "area",
          "sizeref": 0.0125,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Colombia",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2001
         ],
         "xaxis": "x",
         "y": [
          2001
         ],
         "yaxis": "y"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Colombia<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Colombia",
         "marker": {
          "color": "#FCD116",
          "size": [
           2,
           1,
           3,
           3
          ],
          "sizemode": "area",
          "sizeref": 0.0125,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Colombia",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2005,
          2007,
          2007,
          2008
         ],
         "xaxis": "x3",
         "y": [
          2005,
          2006,
          2007,
          2008
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Iran<br>Gender=Collectives & Collabs<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Iran",
         "marker": {
          "color": "#239f40",
          "size": [
           1
          ],
          "sizemode": "area",
          "sizeref": 0.0125,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Iran",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2002
         ],
         "xaxis": "x",
         "y": [
          2001
         ],
         "yaxis": "y"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Iran<br>Gender=Female Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Iran",
         "marker": {
          "color": "#239f40",
          "size": [
           1
          ],
          "sizemode": "area",
          "sizeref": 0.0125,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Iran",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2001
         ],
         "xaxis": "x2",
         "y": [
          2001
         ],
         "yaxis": "y2"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Mexico<br>Gender=Collectives & Collabs<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Mexico",
         "marker": {
          "color": "#CE1126",
          "size": [
           2
          ],
          "sizemode": "area",
          "sizeref": 0.0125,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Mexico",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2007
         ],
         "xaxis": "x",
         "y": [
          2007
         ],
         "yaxis": "y"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Mexico<br>Gender=Female Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Mexico",
         "marker": {
          "color": "#CE1126",
          "size": [
           1,
           1,
           1,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.0125,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Mexico",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2002,
          2005,
          2006,
          2016
         ],
         "xaxis": "x2",
         "y": [
          2001,
          2005,
          2006,
          2015
         ],
         "yaxis": "y2"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Mexico<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Mexico",
         "marker": {
          "color": "#CE1126",
          "size": [
           1,
           1,
           1,
           2,
           1,
           1,
           1,
           2,
           1,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.0125,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Mexico",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2002,
          2005,
          2001,
          2001,
          2003,
          2002,
          2004,
          2005,
          2008,
          2007
         ],
         "xaxis": "x3",
         "y": [
          2000,
          2000,
          2001,
          2001,
          2001,
          2002,
          2004,
          2005,
          2005,
          2006
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Brazil<br>Gender=Female Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Brazil",
         "marker": {
          "color": "#009c3b",
          "size": [
           1,
           2,
           1,
           1,
           1,
           5
          ],
          "sizemode": "area",
          "sizeref": 0.0125,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Brazil",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2001,
          2002,
          2005,
          2003,
          2007,
          2007
         ],
         "xaxis": "x2",
         "y": [
          2001,
          2002,
          2002,
          2003,
          2004,
          2007
         ],
         "yaxis": "y2"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Brazil<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Brazil",
         "marker": {
          "color": "#009c3b",
          "size": [
           1,
           1,
           2,
           1,
           1,
           3,
           1,
           2,
           1,
           1,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.0125,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Brazil",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2001,
          2001,
          2002,
          2003,
          2003,
          2004,
          2004,
          2007,
          2008,
          2008,
          2008
         ],
         "xaxis": "x3",
         "y": [
          2000,
          2001,
          2002,
          2002,
          2003,
          2004,
          2004,
          2007,
          2007,
          2008,
          2008
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Argentina<br>Gender=Female Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Argentina",
         "marker": {
          "color": "#6CACE4",
          "size": [
           1,
           4,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.0125,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Argentina",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2002,
          2004,
          2006
         ],
         "xaxis": "x2",
         "y": [
          2002,
          2004,
          2006
         ],
         "yaxis": "y2"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Argentina<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Argentina",
         "marker": {
          "color": "#6CACE4",
          "size": [
           2,
           1,
           1,
           1,
           1,
           1,
           1,
           2,
           2,
           1,
           1,
           1,
           1,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.0125,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Argentina",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2001,
          2005,
          2002,
          2002,
          2002,
          2003,
          2003,
          2003,
          2004,
          2005,
          2005,
          2007,
          2008,
          2009
         ],
         "xaxis": "x3",
         "y": [
          2000,
          2000,
          2001,
          2001,
          2002,
          2002,
          2003,
          2003,
          2004,
          2004,
          2005,
          2007,
          2007,
          2009
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Chile<br>Gender=Female Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Chile",
         "marker": {
          "color": "#DA291C",
          "size": [
           1,
           1,
           1,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.0125,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Chile",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2004,
          2007,
          2006,
          2006
         ],
         "xaxis": "x2",
         "y": [
          2004,
          2004,
          2005,
          2006
         ],
         "yaxis": "y2"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Chile<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Chile",
         "marker": {
          "color": "#DA291C",
          "size": [
           1,
           1,
           1,
           1,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.0125,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Chile",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2002,
          2003,
          2004,
          2005,
          2006
         ],
         "xaxis": "x3",
         "y": [
          2002,
          2003,
          2004,
          2005,
          2006
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Puerto Rico<br>Gender=Female Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Puerto Rico",
         "marker": {
          "color": "#E92228",
          "size": [
           2
          ],
          "sizemode": "area",
          "sizeref": 0.0125,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Puerto Rico",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2008
         ],
         "xaxis": "x2",
         "y": [
          2008
         ],
         "yaxis": "y2"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Afghanistan<br>Gender=Female Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Afghanistan",
         "marker": {
          "color": "#000000",
          "size": [
           1,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.0125,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Afghanistan",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2011,
          2012
         ],
         "xaxis": "x2",
         "y": [
          2011,
          2012
         ],
         "yaxis": "y2"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Egypt<br>Gender=Female Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Egypt",
         "marker": {
          "color": "#ce1126",
          "size": [
           1
          ],
          "sizemode": "area",
          "sizeref": 0.0125,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Egypt",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2012
         ],
         "xaxis": "x2",
         "y": [
          2012
         ],
         "yaxis": "y2"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=South Africa<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "South Africa",
         "marker": {
          "color": "#FFB81C",
          "size": [
           1,
           2,
           1,
           1,
           1,
           1,
           2,
           1,
           1,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.0125,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "South Africa",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2001,
          2001,
          2002,
          2002,
          2004,
          2004,
          2005,
          2009,
          2009,
          2010
         ],
         "xaxis": "x3",
         "y": [
          2000,
          2001,
          2001,
          2001,
          2003,
          2004,
          2005,
          2007,
          2009,
          2010
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Venezuela<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Venezuela",
         "marker": {
          "color": "#003DA5",
          "size": [
           1
          ],
          "sizemode": "area",
          "sizeref": 0.0125,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Venezuela",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2001
         ],
         "xaxis": "x3",
         "y": [
          2001
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=China<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "China",
         "marker": {
          "color": "#EE1C25",
          "size": [
           1,
           1,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.0125,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "China",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2002,
          2002,
          2004
         ],
         "xaxis": "x3",
         "y": [
          2001,
          2002,
          2004
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=India<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "India",
         "marker": {
          "color": "#FF9933",
          "size": [
           1,
           1,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.0125,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "India",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2002,
          2007,
          2008
         ],
         "xaxis": "x3",
         "y": [
          2002,
          2007,
          2008
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Guatemala<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Guatemala",
         "marker": {
          "color": "#4997D0",
          "size": [
           1
          ],
          "sizemode": "area",
          "sizeref": 0.0125,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Guatemala",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2003
         ],
         "xaxis": "x3",
         "y": [
          2003
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Tanzania<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Tanzania",
         "marker": {
          "color": "#1EB53A",
          "size": [
           1
          ],
          "sizemode": "area",
          "sizeref": 0.0125,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Tanzania",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2007
         ],
         "xaxis": "x3",
         "y": [
          2007
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Angola<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Angola",
         "marker": {
          "color": "#FFCB00",
          "size": [
           1
          ],
          "sizemode": "area",
          "sizeref": 0.0125,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Angola",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2008
         ],
         "xaxis": "x3",
         "y": [
          2007
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Haiti<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Haiti",
         "marker": {
          "color": "#F1B517",
          "size": [
           1
          ],
          "sizemode": "area",
          "sizeref": 0.0125,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Haiti",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2021
         ],
         "xaxis": "x3",
         "y": [
          2017
         ],
         "yaxis": "y3"
        }
       ],
       "layout": {
        "annotations": [
         {
          "font": {},
          "showarrow": false,
          "text": "Collectives & Collabs",
          "x": 0.15999999999999998,
          "xanchor": "center",
          "xref": "paper",
          "y": 1,
          "yanchor": "bottom",
          "yref": "paper"
         },
         {
          "font": {},
          "showarrow": false,
          "text": "Female Artists",
          "x": 0.49999999999999994,
          "xanchor": "center",
          "xref": "paper",
          "y": 1,
          "yanchor": "bottom",
          "yref": "paper"
         },
         {
          "font": {},
          "showarrow": false,
          "text": "Male Artists",
          "x": 0.8399999999999999,
          "xanchor": "center",
          "xref": "paper",
          "y": 1,
          "yanchor": "bottom",
          "yref": "paper"
         }
        ],
        "legend": {
         "itemsizing": "constant",
         "title": {
          "text": "Nation"
         },
         "tracegroupgap": 0
        },
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "rgb(36,36,36)"
            },
            "error_y": {
             "color": "rgb(36,36,36)"
            },
            "marker": {
             "line": {
              "color": "white",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "white",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "rgb(36,36,36)",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "rgb(36,36,36)"
            },
            "baxis": {
             "endlinecolor": "rgb(36,36,36)",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "rgb(36,36,36)"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "line": {
              "color": "white",
              "width": 0.6
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "fillpattern": {
             "fillmode": "overlay",
             "size": 10,
             "solidity": 0.2
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "rgb(237,237,237)"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "rgb(217,217,217)"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 1,
            "tickcolor": "rgb(36,36,36)",
            "ticks": "outside"
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "rgb(103,0,31)"
            ],
            [
             0.1,
             "rgb(178,24,43)"
            ],
            [
             0.2,
             "rgb(214,96,77)"
            ],
            [
             0.3,
             "rgb(244,165,130)"
            ],
            [
             0.4,
             "rgb(253,219,199)"
            ],
            [
             0.5,
             "rgb(247,247,247)"
            ],
            [
             0.6,
             "rgb(209,229,240)"
            ],
            [
             0.7,
             "rgb(146,197,222)"
            ],
            [
             0.8,
             "rgb(67,147,195)"
            ],
            [
             0.9,
             "rgb(33,102,172)"
            ],
            [
             1,
             "rgb(5,48,97)"
            ]
           ],
           "sequential": [
            [
             0,
             "#440154"
            ],
            [
             0.1111111111111111,
             "#482878"
            ],
            [
             0.2222222222222222,
             "#3e4989"
            ],
            [
             0.3333333333333333,
             "#31688e"
            ],
            [
             0.4444444444444444,
             "#26828e"
            ],
            [
             0.5555555555555556,
             "#1f9e89"
            ],
            [
             0.6666666666666666,
             "#35b779"
            ],
            [
             0.7777777777777778,
             "#6ece58"
            ],
            [
             0.8888888888888888,
             "#b5de2b"
            ],
            [
             1,
             "#fde725"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#440154"
            ],
            [
             0.1111111111111111,
             "#482878"
            ],
            [
             0.2222222222222222,
             "#3e4989"
            ],
            [
             0.3333333333333333,
             "#31688e"
            ],
            [
             0.4444444444444444,
             "#26828e"
            ],
            [
             0.5555555555555556,
             "#1f9e89"
            ],
            [
             0.6666666666666666,
             "#35b779"
            ],
            [
             0.7777777777777778,
             "#6ece58"
            ],
            [
             0.8888888888888888,
             "#b5de2b"
            ],
            [
             1,
             "#fde725"
            ]
           ]
          },
          "colorway": [
           "#1F77B4",
           "#FF7F0E",
           "#2CA02C",
           "#D62728",
           "#9467BD",
           "#8C564B",
           "#E377C2",
           "#7F7F7F",
           "#BCBD22",
           "#17BECF"
          ],
          "font": {
           "color": "rgb(36,36,36)"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "white",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "white",
          "polar": {
           "angularaxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           },
           "bgcolor": "white",
           "radialaxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "white",
            "gridcolor": "rgb(232,232,232)",
            "gridwidth": 2,
            "linecolor": "rgb(36,36,36)",
            "showbackground": true,
            "showgrid": false,
            "showline": true,
            "ticks": "outside",
            "zeroline": false,
            "zerolinecolor": "rgb(36,36,36)"
           },
           "yaxis": {
            "backgroundcolor": "white",
            "gridcolor": "rgb(232,232,232)",
            "gridwidth": 2,
            "linecolor": "rgb(36,36,36)",
            "showbackground": true,
            "showgrid": false,
            "showline": true,
            "ticks": "outside",
            "zeroline": false,
            "zerolinecolor": "rgb(36,36,36)"
           },
           "zaxis": {
            "backgroundcolor": "white",
            "gridcolor": "rgb(232,232,232)",
            "gridwidth": 2,
            "linecolor": "rgb(36,36,36)",
            "showbackground": true,
            "showgrid": false,
            "showline": true,
            "ticks": "outside",
            "zeroline": false,
            "zerolinecolor": "rgb(36,36,36)"
           }
          },
          "shapedefaults": {
           "fillcolor": "black",
           "line": {
            "width": 0
           },
           "opacity": 0.3
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           },
           "baxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           },
           "bgcolor": "white",
           "caxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "rgb(232,232,232)",
           "linecolor": "rgb(36,36,36)",
           "showgrid": false,
           "showline": true,
           "ticks": "outside",
           "title": {
            "standoff": 15
           },
           "zeroline": false,
           "zerolinecolor": "rgb(36,36,36)"
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "rgb(232,232,232)",
           "linecolor": "rgb(36,36,36)",
           "showgrid": false,
           "showline": true,
           "ticks": "outside",
           "title": {
            "standoff": 15
           },
           "zeroline": false,
           "zerolinecolor": "rgb(36,36,36)"
          }
         }
        },
        "title": {
         "text": "When Do Artworks from the Global South Become Acquired? - Rhizome",
         "x": 0.5
        },
        "xaxis": {
         "anchor": "y",
         "domain": [
          0,
          0.31999999999999995
         ],
         "dtick": 2,
         "nticks": 10,
         "tickformat": "%Y",
         "title": {
          "text": "Year Acquired"
         }
        },
        "xaxis2": {
         "anchor": "y2",
         "domain": [
          0.33999999999999997,
          0.6599999999999999
         ],
         "dtick": 2,
         "matches": "x",
         "nticks": 10,
         "tickformat": "%Y",
         "title": {
          "text": "Year Acquired"
         }
        },
        "xaxis3": {
         "anchor": "y3",
         "domain": [
          0.6799999999999999,
          0.9999999999999999
         ],
         "dtick": 2,
         "matches": "x",
         "nticks": 10,
         "tickformat": "%Y",
         "title": {
          "text": "Year Acquired"
         }
        },
        "yaxis": {
         "anchor": "x",
         "domain": [
          0,
          1
         ],
         "dtick": 2,
         "nticks": 10,
         "tickformat": "%Y",
         "title": {
          "text": "Year Created"
         }
        },
        "yaxis2": {
         "anchor": "x2",
         "domain": [
          0,
          1
         ],
         "dtick": 2,
         "matches": "y",
         "nticks": 10,
         "showticklabels": false,
         "tickformat": "%Y"
        },
        "yaxis3": {
         "anchor": "x3",
         "domain": [
          0,
          1
         ],
         "dtick": 2,
         "matches": "y",
         "nticks": 10,
         "showticklabels": false,
         "tickformat": "%Y"
        }
       }
      }
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#create bubble chart w/facet\n",
    "\n",
    "rhz_south = moma_rhz_parallel_artworks_counts_df_final.loc[moma_rhz_parallel_artworks_counts_df_final['Divide'] == 'Global South']\n",
    "rhz_south = rhz_south.loc[rhz_south['Source'] == 'Rhizome']\n",
    "rhz_south = rhz_south.replace('M', 'Male Artists')\n",
    "rhz_south = rhz_south.replace('F', 'Female Artists')\n",
    "rhz_south = rhz_south.replace('Collectives', 'Collectives & Collabs')\n",
    "\n",
    "fig = px.scatter(rhz_south, y='dateCreated', x='dateAcquired', color='Nation', size='size', facet_col='Gender', title= \"When Do Artworks from the Global South Become Acquired? - Rhizome\", color_discrete_sequence = ['#FCD116', \"#239f40\", \"#CE1126\", \"#009c3b\", \"#6CACE4\", '#DA291C', '#E92228', '#000000', '#ce1126', '#FFB81C', '#003DA5', '#EE1C25', '#FF9933', '#4997D0', '#1EB53A', '#FFCB00', '#F1B517'], labels={\"dateCreated\": \"Year Created\", \"dateAcquired\": \"Year Acquired\"})\n",
    "fig.for_each_annotation(lambda a: a.update(text=a.text.split(\"=\")[-1]))\n",
    "fig.update_yaxes(nticks=10, tickformat=\"%Y\",  dtick=2)\n",
    "fig.update_xaxes(nticks=10, tickformat=\"%Y\",  dtick=2)\n",
    "fig.update_traces(hoverinfo='all')\n",
    "fig.update_layout(title_x=0.5)\n",
    "fig.show()\n",
    "#create local copies\n",
    "#fig.write_html(path+\"Plotly_embeds/scatter_rhz_south_artworks.html\", include_plotlyjs='directory', full_html=False, default_width='100%')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Egypt<br>Gender=Collectives & Collabs<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Egypt",
         "marker": {
          "color": "#ce1126",
          "size": [
           1,
           1,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Egypt",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2006,
          2006,
          2015
         ],
         "xaxis": "x",
         "y": [
          2000,
          2002,
          2014
         ],
         "yaxis": "y"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Egypt<br>Gender=Female Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Egypt",
         "marker": {
          "color": "#ce1126",
          "size": [
           1,
           12
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Egypt",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2007,
          2016
         ],
         "xaxis": "x2",
         "y": [
          2000,
          2011
         ],
         "yaxis": "y2"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Egypt<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Egypt",
         "marker": {
          "color": "#ce1126",
          "size": [
           1,
           7,
           1,
           1,
           1,
           7,
           19,
           1,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Egypt",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2006,
          2012,
          2015,
          2015,
          2014,
          2015,
          2015,
          2016,
          2016
         ],
         "xaxis": "x3",
         "y": [
          2000,
          2010,
          2010,
          2011,
          2012,
          2012,
          2014,
          2014,
          2015
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=India<br>Gender=Collectives & Collabs<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "India",
         "marker": {
          "color": "#FF9933",
          "size": [
           10,
           2
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "India",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2011,
          2017
         ],
         "xaxis": "x",
         "y": [
          2000,
          2016
         ],
         "yaxis": "y"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=India<br>Gender=Female Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "India",
         "marker": {
          "color": "#FF9933",
          "size": [
           1,
           1,
           10,
           1,
           1,
           2,
           1,
           17,
           1,
           1,
           12,
           4
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "India",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2018,
          2007,
          2018,
          2018,
          2015,
          2018,
          2012,
          2013,
          2017,
          2015,
          2018,
          2018
         ],
         "xaxis": "x2",
         "y": [
          2002,
          2003,
          2003,
          2004,
          2005,
          2008,
          2011,
          2012,
          2013,
          2014,
          2015,
          2016
         ],
         "yaxis": "y2"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=India<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "India",
         "marker": {
          "color": "#FF9933",
          "size": [
           1,
           3,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "India",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2007,
          2005,
          2016
         ],
         "xaxis": "x3",
         "y": [
          2000,
          2004,
          2007
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Cuba<br>Gender=Collectives & Collabs<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Cuba",
         "marker": {
          "color": "#004B87",
          "size": [
           1,
           1,
           2,
           1,
           1,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Cuba",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2004,
          2005,
          2005,
          2005,
          2008,
          2008
         ],
         "xaxis": "x",
         "y": [
          2001,
          2002,
          2003,
          2004,
          2007,
          2007
         ],
         "yaxis": "y"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Cuba<br>Gender=Female Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Cuba",
         "marker": {
          "color": "#004B87",
          "size": [
           1,
           9
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Cuba",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2015,
          2018
         ],
         "xaxis": "x2",
         "y": [
          2000,
          2017
         ],
         "yaxis": "y2"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Cuba<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Cuba",
         "marker": {
          "color": "#004B87",
          "size": [
           1,
           1,
           1,
           1,
           1,
           2,
           1,
           1,
           2,
           1,
           1,
           1,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Cuba",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2001,
          2001,
          2003,
          2003,
          2018,
          2004,
          2005,
          2008,
          2008,
          2017,
          2007,
          2008,
          2018
         ],
         "xaxis": "x3",
         "y": [
          2000,
          2001,
          2001,
          2002,
          2002,
          2003,
          2004,
          2005,
          2006,
          2006,
          2007,
          2008,
          2016
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Venezuela<br>Gender=Collectives & Collabs<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Venezuela",
         "marker": {
          "color": "#003DA5",
          "size": [
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Venezuela",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2017
         ],
         "xaxis": "x",
         "y": [
          2001
         ],
         "yaxis": "y"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Venezuela<br>Gender=Female Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Venezuela",
         "marker": {
          "color": "#003DA5",
          "size": [
           2
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Venezuela",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2005
         ],
         "xaxis": "x2",
         "y": [
          2004
         ],
         "yaxis": "y2"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Venezuela<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Venezuela",
         "marker": {
          "color": "#003DA5",
          "size": [
           1,
           1,
           1,
           2,
           1,
           10,
           4
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Venezuela",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2002,
          2017,
          2005,
          2010,
          2010,
          2015,
          2017
         ],
         "xaxis": "x3",
         "y": [
          2000,
          2000,
          2003,
          2007,
          2008,
          2009,
          2016
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Mexico<br>Gender=Collectives & Collabs<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Mexico",
         "marker": {
          "color": "#CE1126",
          "size": [
           1,
           1,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Mexico",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2013,
          2013,
          2016
         ],
         "xaxis": "x",
         "y": [
          2003,
          2013,
          2015
         ],
         "yaxis": "y"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Mexico<br>Gender=Female Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Mexico",
         "marker": {
          "color": "#CE1126",
          "size": [
           2,
           1,
           2,
           1,
           1,
           1,
           1,
           16
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Mexico",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2006,
          2013,
          2006,
          2006,
          2016,
          2013,
          2011,
          2018
         ],
         "xaxis": "x2",
         "y": [
          2001,
          2001,
          2002,
          2005,
          2005,
          2008,
          2009,
          2016
         ],
         "yaxis": "y2"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Mexico<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Mexico",
         "marker": {
          "color": "#CE1126",
          "size": [
           5,
           2,
           7,
           1,
           1,
           1,
           1,
           2,
           15,
           1,
           7,
           1,
           2,
           1,
           1,
           1,
           1,
           1,
           1,
           1,
           11,
           1,
           1,
           1,
           1,
           2,
           1,
           1,
           3,
           1,
           1,
           9,
           1,
           1,
           1,
           7,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Mexico",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2004,
          2006,
          2005,
          2006,
          2002,
          2004,
          2005,
          2006,
          2015,
          2004,
          2005,
          2006,
          2005,
          2013,
          2006,
          2008,
          2017,
          2007,
          2008,
          2014,
          2009,
          2013,
          2014,
          2015,
          2011,
          2009,
          2012,
          2017,
          2011,
          2017,
          2017,
          2013,
          2014,
          2017,
          2013,
          2015,
          2016
         ],
         "xaxis": "x3",
         "y": [
          2000,
          2000,
          2001,
          2001,
          2002,
          2002,
          2002,
          2002,
          2002,
          2003,
          2003,
          2003,
          2004,
          2004,
          2005,
          2005,
          2005,
          2006,
          2006,
          2006,
          2007,
          2007,
          2007,
          2007,
          2008,
          2009,
          2009,
          2009,
          2010,
          2010,
          2011,
          2012,
          2012,
          2012,
          2013,
          2015,
          2015
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Kyrgyzstan<br>Gender=Collectives & Collabs<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Kyrgyzstan",
         "marker": {
          "color": "#FFFF00",
          "size": [
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Kyrgyzstan",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2006
         ],
         "xaxis": "x",
         "y": [
          2004
         ],
         "yaxis": "y"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=China<br>Gender=Collectives & Collabs<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "China",
         "marker": {
          "color": "#EE1C25",
          "size": [
           12,
           16,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "China",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2016,
          2016,
          2017
         ],
         "xaxis": "x",
         "y": [
          2005,
          2012,
          2017
         ],
         "yaxis": "y"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=China<br>Gender=Female Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "China",
         "marker": {
          "color": "#EE1C25",
          "size": [
           1,
           1,
           2,
           4,
           1,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "China",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2003,
          2004,
          2004,
          2007,
          2008,
          2017
         ],
         "xaxis": "x2",
         "y": [
          2000,
          2001,
          2002,
          2006,
          2007,
          2017
         ],
         "yaxis": "y2"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=China<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "China",
         "marker": {
          "color": "#EE1C25",
          "size": [
           3,
           1,
           1,
           2,
           1,
           1,
           2,
           1,
           2,
           3,
           8,
           6,
           2,
           5,
           1,
           1,
           1,
           8,
           2,
           3,
           1,
           1,
           1,
           13,
           2,
           1,
           5,
           1,
           1,
           1,
           1,
           1,
           2,
           1,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "China",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2004,
          2006,
          2008,
          2009,
          2011,
          2002,
          2018,
          2004,
          2006,
          2008,
          2004,
          2005,
          2005,
          2007,
          2008,
          2008,
          2006,
          2007,
          2008,
          2008,
          2010,
          2017,
          2008,
          2010,
          2012,
          2009,
          2010,
          2012,
          2011,
          2013,
          2017,
          2018,
          2016,
          2017,
          2017
         ],
         "xaxis": "x3",
         "y": [
          2000,
          2000,
          2000,
          2000,
          2000,
          2001,
          2001,
          2002,
          2002,
          2002,
          2003,
          2003,
          2004,
          2004,
          2004,
          2005,
          2006,
          2006,
          2006,
          2007,
          2007,
          2007,
          2008,
          2008,
          2008,
          2009,
          2009,
          2009,
          2011,
          2012,
          2014,
          2014,
          2015,
          2015,
          2016
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Brazil<br>Gender=Collectives & Collabs<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Brazil",
         "marker": {
          "color": "#009c3b",
          "size": [
           1,
           1,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Brazil",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2007,
          2009,
          2018
         ],
         "xaxis": "x",
         "y": [
          2006,
          2009,
          2011
         ],
         "yaxis": "y"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Brazil<br>Gender=Female Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Brazil",
         "marker": {
          "color": "#009c3b",
          "size": [
           3,
           2,
           1,
           1,
           1,
           1,
           1,
           3,
           1,
           2,
           1,
           1,
           2,
           4,
           2
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Brazil",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2001,
          2006,
          2002,
          2016,
          2005,
          2015,
          2008,
          2009,
          2010,
          2017,
          2017,
          2015,
          2018,
          2017,
          2018
         ],
         "xaxis": "x2",
         "y": [
          2000,
          2001,
          2002,
          2002,
          2004,
          2005,
          2007,
          2008,
          2009,
          2011,
          2013,
          2014,
          2014,
          2016,
          2017
         ],
         "yaxis": "y2"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Brazil<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Brazil",
         "marker": {
          "color": "#009c3b",
          "size": [
           7,
           1,
           1,
           1,
           1,
           1,
           4,
           1,
           2,
           1,
           1,
           2,
           9,
           1,
           1,
           1,
           2,
           1,
           5,
           3,
           1,
           1,
           1,
           2,
           1,
           1,
           1,
           1,
           2,
           1,
           1,
           10,
           1,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Brazil",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2000,
          2008,
          2009,
          2016,
          2007,
          2011,
          2005,
          2006,
          2005,
          2008,
          2010,
          2017,
          2005,
          2009,
          2006,
          2009,
          2014,
          2007,
          2008,
          2008,
          2014,
          2017,
          2008,
          2014,
          2014,
          2012,
          2013,
          2014,
          2013,
          2014,
          2015,
          2017,
          2015,
          2018
         ],
         "xaxis": "x3",
         "y": [
          2000,
          2000,
          2000,
          2000,
          2001,
          2001,
          2002,
          2002,
          2003,
          2003,
          2003,
          2003,
          2004,
          2004,
          2005,
          2005,
          2005,
          2006,
          2006,
          2007,
          2007,
          2007,
          2008,
          2008,
          2009,
          2010,
          2011,
          2011,
          2012,
          2012,
          2013,
          2013,
          2015,
          2017
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Colombia<br>Gender=Collectives & Collabs<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Colombia",
         "marker": {
          "color": "#FCD116",
          "size": [
           1,
           2
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Colombia",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2017,
          2017
         ],
         "xaxis": "x",
         "y": [
          2008,
          2015
         ],
         "yaxis": "y"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Colombia<br>Gender=Female Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Colombia",
         "marker": {
          "color": "#FCD116",
          "size": [
           1,
           1,
           2,
           3,
           4
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Colombia",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2004,
          2012,
          2012,
          2012,
          2017
         ],
         "xaxis": "x2",
         "y": [
          2000,
          2005,
          2007,
          2011,
          2016
         ],
         "yaxis": "y2"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Colombia<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Colombia",
         "marker": {
          "color": "#FCD116",
          "size": [
           1,
           14,
           1,
           11,
           13,
           1,
           9,
           1,
           1,
           5,
           15,
           1,
           5,
           9,
           3,
           12,
           3,
           3,
           1,
           2,
           106,
           12,
           3,
           16,
           1,
           5,
           3,
           1,
           6,
           2,
           7,
           17,
           2,
           14,
           2
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Colombia",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2005,
          2009,
          2012,
          2009,
          2009,
          2012,
          2009,
          2017,
          2005,
          2009,
          2009,
          2006,
          2007,
          2011,
          2008,
          2011,
          2012,
          2014,
          2008,
          2009,
          2010,
          2011,
          2012,
          2014,
          2017,
          2011,
          2012,
          2011,
          2012,
          2017,
          2012,
          2014,
          2019,
          2017,
          2017
         ],
         "xaxis": "x3",
         "y": [
          2000,
          2000,
          2000,
          2001,
          2002,
          2002,
          2003,
          2003,
          2004,
          2004,
          2005,
          2006,
          2006,
          2006,
          2007,
          2007,
          2007,
          2007,
          2008,
          2008,
          2008,
          2008,
          2008,
          2008,
          2008,
          2009,
          2009,
          2010,
          2010,
          2010,
          2011,
          2011,
          2012,
          2016,
          2017
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=South Africa<br>Gender=Collectives & Collabs<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "South Africa",
         "marker": {
          "color": "#FFB81C",
          "size": [
           1,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "South Africa",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2013,
          2018
         ],
         "xaxis": "x",
         "y": [
          2011,
          2017
         ],
         "yaxis": "y"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=South Africa<br>Gender=Female Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "South Africa",
         "marker": {
          "color": "#FFB81C",
          "size": [
           1,
           6,
           12,
           3,
           1,
           17,
           6,
           1,
           1,
           4,
           1,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "South Africa",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2005,
          2007,
          2005,
          2007,
          2005,
          2005,
          2007,
          2010,
          2012,
          2013,
          2018,
          2017
         ],
         "xaxis": "x2",
         "y": [
          2000,
          2000,
          2001,
          2001,
          2002,
          2003,
          2005,
          2010,
          2011,
          2011,
          2016,
          2017
         ],
         "yaxis": "y2"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=South Africa<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "South Africa",
         "marker": {
          "color": "#FFB81C",
          "size": [
           1,
           3,
           1,
           12,
           2,
           10,
           1,
           4,
           1,
           16,
           1,
           7,
           1,
           2,
           1,
           2,
           9,
           2,
           1,
           1,
           5,
           7,
           14,
           1,
           28,
           1,
           5,
           3,
           1,
           1,
           13,
           10,
           7,
           3,
           2,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "South Africa",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2002,
          2005,
          2006,
          2007,
          2008,
          2001,
          2002,
          2006,
          2007,
          2005,
          2006,
          2004,
          2005,
          2006,
          2004,
          2005,
          2006,
          2007,
          2009,
          2005,
          2006,
          2007,
          2010,
          2016,
          2006,
          2007,
          2008,
          2010,
          2011,
          2007,
          2009,
          2010,
          2009,
          2010,
          2015,
          2018
         ],
         "xaxis": "x3",
         "y": [
          2000,
          2000,
          2000,
          2000,
          2000,
          2001,
          2001,
          2001,
          2001,
          2002,
          2002,
          2003,
          2003,
          2003,
          2004,
          2004,
          2004,
          2004,
          2004,
          2005,
          2005,
          2005,
          2005,
          2005,
          2006,
          2006,
          2006,
          2006,
          2006,
          2007,
          2007,
          2007,
          2008,
          2009,
          2012,
          2014
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Peru<br>Gender=Collectives & Collabs<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Peru",
         "marker": {
          "color": "#cd853f",
          "size": [
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Peru",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2017
         ],
         "xaxis": "x",
         "y": [
          2011
         ],
         "yaxis": "y"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Peru<br>Gender=Female Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Peru",
         "marker": {
          "color": "#cd853f",
          "size": [
           1,
           2,
           1,
           5,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Peru",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2007,
          2007,
          2008,
          2008,
          2017
         ],
         "xaxis": "x2",
         "y": [
          2000,
          2007,
          2007,
          2008,
          2012
         ],
         "yaxis": "y2"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Peru<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Peru",
         "marker": {
          "color": "#cd853f",
          "size": [
           1,
           1,
           1,
           1,
           1,
           4,
           1,
           2
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Peru",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2007,
          2003,
          2005,
          2011,
          2017,
          2014,
          2017,
          2017
         ],
         "xaxis": "x3",
         "y": [
          2000,
          2002,
          2003,
          2003,
          2012,
          2013,
          2013,
          2014
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Ghana<br>Gender=Collectives & Collabs<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Ghana",
         "marker": {
          "color": "#006B3D",
          "size": [
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Ghana",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2013
         ],
         "xaxis": "x",
         "y": [
          2012
         ],
         "yaxis": "y"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Ghana<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Ghana",
         "marker": {
          "color": "#006B3D",
          "size": [
           1,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Ghana",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2007,
          2012
         ],
         "xaxis": "x3",
         "y": [
          2000,
          2012
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Morocco<br>Gender=Collectives & Collabs<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Morocco",
         "marker": {
          "color": "#963846",
          "size": [
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Morocco",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2013
         ],
         "xaxis": "x",
         "y": [
          2012
         ],
         "yaxis": "y"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Morocco<br>Gender=Female Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Morocco",
         "marker": {
          "color": "#963846",
          "size": [
           1,
           1,
           1,
           1,
           4,
           1,
           1,
           1,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Morocco",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2018,
          2012,
          2018,
          2013,
          2018,
          2014,
          2018,
          2013,
          2018
         ],
         "xaxis": "x2",
         "y": [
          2001,
          2002,
          2002,
          2004,
          2006,
          2008,
          2009,
          2012,
          2013
         ],
         "yaxis": "y2"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Palestine<br>Gender=Collectives & Collabs<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Palestine",
         "marker": {
          "color": "#E4312b",
          "size": [
           6
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Palestine",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2017
         ],
         "xaxis": "x",
         "y": [
          2012
         ],
         "yaxis": "y"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Palestine<br>Gender=Female Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Palestine",
         "marker": {
          "color": "#E4312b",
          "size": [
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Palestine",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2016
         ],
         "xaxis": "x2",
         "y": [
          2015
         ],
         "yaxis": "y2"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Philippines<br>Gender=Collectives & Collabs<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Philippines",
         "marker": {
          "color": "#0033A0",
          "size": [
           25
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Philippines",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2019
         ],
         "xaxis": "x",
         "y": [
          2013
         ],
         "yaxis": "y"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Philippines<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Philippines",
         "marker": {
          "color": "#0033A0",
          "size": [
           1,
           1,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Philippines",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2016,
          2016,
          2016
         ],
         "xaxis": "x3",
         "y": [
          2012,
          2014,
          2016
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Iran<br>Gender=Female Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Iran",
         "marker": {
          "color": "#239f40",
          "size": [
           1,
           1,
           1,
           1,
           1,
           1,
           2
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Iran",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2011,
          2007,
          2007,
          2015,
          2015,
          2015,
          2015
         ],
         "xaxis": "x2",
         "y": [
          2000,
          2001,
          2006,
          2007,
          2012,
          2013,
          2014
         ],
         "yaxis": "y2"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Iran<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Iran",
         "marker": {
          "color": "#239f40",
          "size": [
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Iran",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2013
         ],
         "xaxis": "x3",
         "y": [
          2012
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Pakistan<br>Gender=Female Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Pakistan",
         "marker": {
          "color": "#006600",
          "size": [
           1,
           10,
           2,
           1,
           5,
           1,
           1,
           1,
           13,
           13
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Pakistan",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2001,
          2002,
          2005,
          2005,
          2016,
          2005,
          2018,
          2008,
          2008,
          2008
         ],
         "xaxis": "x2",
         "y": [
          2001,
          2001,
          2002,
          2003,
          2004,
          2005,
          2005,
          2006,
          2007,
          2008
         ],
         "yaxis": "y2"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Argentina<br>Gender=Female Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Argentina",
         "marker": {
          "color": "#6CACE4",
          "size": [
           1,
           2
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Argentina",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2005,
          2017
         ],
         "xaxis": "x2",
         "y": [
          2003,
          2011
         ],
         "yaxis": "y2"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Argentina<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Argentina",
         "marker": {
          "color": "#6CACE4",
          "size": [
           1,
           44,
           1,
           1,
           1,
           4,
           1,
           13,
           14,
           1,
           2,
           1,
           1,
           1,
           1,
           4,
           1,
           6,
           2,
           6,
           1,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Argentina",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2006,
          2007,
          2018,
          2002,
          2003,
          2005,
          2017,
          2003,
          2006,
          2009,
          2008,
          2009,
          2018,
          2014,
          2018,
          2013,
          2014,
          2016,
          2017,
          2017,
          2018,
          2018
         ],
         "xaxis": "x3",
         "y": [
          2000,
          2000,
          2001,
          2002,
          2002,
          2002,
          2002,
          2003,
          2006,
          2006,
          2007,
          2007,
          2008,
          2011,
          2011,
          2012,
          2013,
          2015,
          2015,
          2016,
          2016,
          2018
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Kenya<br>Gender=Female Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Kenya",
         "marker": {
          "color": "#BB0000",
          "size": [
           1,
           1,
           9
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Kenya",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2005,
          2005,
          2007
         ],
         "xaxis": "x2",
         "y": [
          2003,
          2004,
          2006
         ],
         "yaxis": "y2"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Kenya<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Kenya",
         "marker": {
          "color": "#BB0000",
          "size": [
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Kenya",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2008
         ],
         "xaxis": "x3",
         "y": [
          2007
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Afghanistan<br>Gender=Female Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Afghanistan",
         "marker": {
          "color": "#000000",
          "size": [
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Afghanistan",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2008
         ],
         "xaxis": "x2",
         "y": [
          2005
         ],
         "yaxis": "y2"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Algeria<br>Gender=Female Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Algeria",
         "marker": {
          "color": "#006633",
          "size": [
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Algeria",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2013
         ],
         "xaxis": "x2",
         "y": [
          2006
         ],
         "yaxis": "y2"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Algeria<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Algeria",
         "marker": {
          "color": "#006633",
          "size": [
           1,
           1,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Algeria",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2016,
          2013,
          2016
         ],
         "xaxis": "x3",
         "y": [
          2007,
          2008,
          2016
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Guatemala<br>Gender=Female Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Guatemala",
         "marker": {
          "color": "#4997D0",
          "size": [
           1,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Guatemala",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2017,
          2017
         ],
         "xaxis": "x2",
         "y": [
          2008,
          2010
         ],
         "yaxis": "y2"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Guatemala<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Guatemala",
         "marker": {
          "color": "#4997D0",
          "size": [
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Guatemala",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2017
         ],
         "xaxis": "x3",
         "y": [
          2002
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Chile<br>Gender=Female Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Chile",
         "marker": {
          "color": "#EE1C25",
          "size": [
           1,
           2,
           1,
           1,
           1,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Chile",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2018,
          2018,
          2018,
          2018,
          2017,
          2017
         ],
         "xaxis": "x2",
         "y": [
          2008,
          2009,
          2010,
          2012,
          2015,
          2016
         ],
         "yaxis": "y2"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Chile<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Chile",
         "marker": {
          "color": "#EE1C25",
          "size": [
           10,
           1,
           1,
           1,
           1,
           1,
           53
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Chile",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2013,
          2011,
          2013,
          2014,
          2013,
          2014,
          2018
         ],
         "xaxis": "x3",
         "y": [
          2008,
          2010,
          2010,
          2010,
          2012,
          2012,
          2013
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Malaysia<br>Gender=Female Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Malaysia",
         "marker": {
          "color": "#FFD100",
          "size": [
           1,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Malaysia",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2012,
          2014
         ],
         "xaxis": "x2",
         "y": [
          2011,
          2013
         ],
         "yaxis": "y2"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Malaysia<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Malaysia",
         "marker": {
          "color": "#FFD100",
          "size": [
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Malaysia",
         "orientation": "v",
         "showlegend": false,
         "type": "scatter",
         "x": [
          2015
         ],
         "xaxis": "x3",
         "y": [
          2013
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=United Arab Emirates<br>Gender=Female Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "United Arab Emirates",
         "marker": {
          "color": "#00843D",
          "size": [
           3,
           2,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "United Arab Emirates",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2019,
          2019,
          2020
         ],
         "xaxis": "x2",
         "y": [
          2016,
          2017,
          2020
         ],
         "yaxis": "y2"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Namibia<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Namibia",
         "marker": {
          "color": "#001489",
          "size": [
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Namibia",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2007
         ],
         "xaxis": "x3",
         "y": [
          2000
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Uganda<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Uganda",
         "marker": {
          "color": "#D90000",
          "size": [
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Uganda",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2007
         ],
         "xaxis": "x3",
         "y": [
          2000
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Lebanon<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Lebanon",
         "marker": {
          "color": "#00A850",
          "size": [
           1,
           1,
           32,
           1,
           1,
           1,
           1,
           3,
           1,
           1,
           1,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Lebanon",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2003,
          2012,
          2008,
          2012,
          2013,
          2012,
          2016,
          2013,
          2013,
          2018,
          2014,
          2017
         ],
         "xaxis": "x3",
         "y": [
          2001,
          2003,
          2004,
          2005,
          2009,
          2010,
          2011,
          2012,
          2013,
          2013,
          2014,
          2017
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Nigeria<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Nigeria",
         "marker": {
          "color": "#008751",
          "size": [
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Nigeria",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2005
         ],
         "xaxis": "x3",
         "y": [
          2002
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Uruguay<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Uruguay",
         "marker": {
          "color": "#001489",
          "size": [
           1,
           3,
           1,
           1,
           2,
           1,
           1,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Uruguay",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2017,
          2003,
          2005,
          2017,
          2005,
          2010,
          2013,
          2017
         ],
         "xaxis": "x3",
         "y": [
          2002,
          2003,
          2003,
          2003,
          2004,
          2009,
          2009,
          2009
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Thailand<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Thailand",
         "marker": {
          "color": "#EF3340",
          "size": [
           2,
           2,
           3,
           1,
           2,
           1,
           1,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Thailand",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2006,
          2008,
          2011,
          2012,
          2014,
          2015,
          2017,
          2018
         ],
         "xaxis": "x3",
         "y": [
          2003,
          2007,
          2008,
          2011,
          2014,
          2015,
          2017,
          2018
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Sierra Leone<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Sierra Leone",
         "marker": {
          "color": "#1EB53A",
          "size": [
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Sierra Leone",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2019
         ],
         "xaxis": "x3",
         "y": [
          2006
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Ivory Coast<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Ivory Coast",
         "marker": {
          "color": "#F77F00",
          "size": [
           2,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Ivory Coast",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2019,
          2019
         ],
         "xaxis": "x3",
         "y": [
          2007,
          2008
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Zimbabwe<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Zimbabwe",
         "marker": {
          "color": "#EF3340",
          "size": [
           5,
           3,
           1
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Zimbabwe",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2010,
          2010,
          2018
         ],
         "xaxis": "x3",
         "y": [
          2008,
          2009,
          2018
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Cameroon<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Cameroon",
         "marker": {
          "color": "#007A5E",
          "size": [
           13
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Cameroon",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2010
         ],
         "xaxis": "x3",
         "y": [
          2009
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Sudan<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Sudan",
         "marker": {
          "color": "#D21034",
          "size": [
           13
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Sudan",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2018
         ],
         "xaxis": "x3",
         "y": [
          2016
         ],
         "yaxis": "y3"
        },
        {
         "hoverinfo": "all",
         "hovertemplate": "Nation=Bangladesh<br>Gender=Male Artists<br>Year Acquired=%{x}<br>Year Created=%{y}<br>size=%{marker.size}<extra></extra>",
         "legendgroup": "Bangladesh",
         "marker": {
          "color": "#006A4E",
          "size": [
           2
          ],
          "sizemode": "area",
          "sizeref": 0.265,
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Bangladesh",
         "orientation": "v",
         "showlegend": true,
         "type": "scatter",
         "x": [
          2018
         ],
         "xaxis": "x3",
         "y": [
          2017
         ],
         "yaxis": "y3"
        }
       ],
       "layout": {
        "annotations": [
         {
          "font": {},
          "showarrow": false,
          "text": "Collectives & Collabs",
          "x": 0.15999999999999998,
          "xanchor": "center",
          "xref": "paper",
          "y": 1,
          "yanchor": "bottom",
          "yref": "paper"
         },
         {
          "font": {},
          "showarrow": false,
          "text": "Female Artists",
          "x": 0.49999999999999994,
          "xanchor": "center",
          "xref": "paper",
          "y": 1,
          "yanchor": "bottom",
          "yref": "paper"
         },
         {
          "font": {},
          "showarrow": false,
          "text": "Male Artists",
          "x": 0.8399999999999999,
          "xanchor": "center",
          "xref": "paper",
          "y": 1,
          "yanchor": "bottom",
          "yref": "paper"
         }
        ],
        "legend": {
         "itemsizing": "constant",
         "title": {
          "text": "Nation"
         },
         "tracegroupgap": 0
        },
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "rgb(36,36,36)"
            },
            "error_y": {
             "color": "rgb(36,36,36)"
            },
            "marker": {
             "line": {
              "color": "white",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "white",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "rgb(36,36,36)",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "rgb(36,36,36)"
            },
            "baxis": {
             "endlinecolor": "rgb(36,36,36)",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "rgb(36,36,36)"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "line": {
              "color": "white",
              "width": 0.6
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "fillpattern": {
             "fillmode": "overlay",
             "size": 10,
             "solidity": 0.2
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "rgb(237,237,237)"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "rgb(217,217,217)"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 1,
            "tickcolor": "rgb(36,36,36)",
            "ticks": "outside"
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "rgb(103,0,31)"
            ],
            [
             0.1,
             "rgb(178,24,43)"
            ],
            [
             0.2,
             "rgb(214,96,77)"
            ],
            [
             0.3,
             "rgb(244,165,130)"
            ],
            [
             0.4,
             "rgb(253,219,199)"
            ],
            [
             0.5,
             "rgb(247,247,247)"
            ],
            [
             0.6,
             "rgb(209,229,240)"
            ],
            [
             0.7,
             "rgb(146,197,222)"
            ],
            [
             0.8,
             "rgb(67,147,195)"
            ],
            [
             0.9,
             "rgb(33,102,172)"
            ],
            [
             1,
             "rgb(5,48,97)"
            ]
           ],
           "sequential": [
            [
             0,
             "#440154"
            ],
            [
             0.1111111111111111,
             "#482878"
            ],
            [
             0.2222222222222222,
             "#3e4989"
            ],
            [
             0.3333333333333333,
             "#31688e"
            ],
            [
             0.4444444444444444,
             "#26828e"
            ],
            [
             0.5555555555555556,
             "#1f9e89"
            ],
            [
             0.6666666666666666,
             "#35b779"
            ],
            [
             0.7777777777777778,
             "#6ece58"
            ],
            [
             0.8888888888888888,
             "#b5de2b"
            ],
            [
             1,
             "#fde725"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#440154"
            ],
            [
             0.1111111111111111,
             "#482878"
            ],
            [
             0.2222222222222222,
             "#3e4989"
            ],
            [
             0.3333333333333333,
             "#31688e"
            ],
            [
             0.4444444444444444,
             "#26828e"
            ],
            [
             0.5555555555555556,
             "#1f9e89"
            ],
            [
             0.6666666666666666,
             "#35b779"
            ],
            [
             0.7777777777777778,
             "#6ece58"
            ],
            [
             0.8888888888888888,
             "#b5de2b"
            ],
            [
             1,
             "#fde725"
            ]
           ]
          },
          "colorway": [
           "#1F77B4",
           "#FF7F0E",
           "#2CA02C",
           "#D62728",
           "#9467BD",
           "#8C564B",
           "#E377C2",
           "#7F7F7F",
           "#BCBD22",
           "#17BECF"
          ],
          "font": {
           "color": "rgb(36,36,36)"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "white",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "white",
          "polar": {
           "angularaxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           },
           "bgcolor": "white",
           "radialaxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "white",
            "gridcolor": "rgb(232,232,232)",
            "gridwidth": 2,
            "linecolor": "rgb(36,36,36)",
            "showbackground": true,
            "showgrid": false,
            "showline": true,
            "ticks": "outside",
            "zeroline": false,
            "zerolinecolor": "rgb(36,36,36)"
           },
           "yaxis": {
            "backgroundcolor": "white",
            "gridcolor": "rgb(232,232,232)",
            "gridwidth": 2,
            "linecolor": "rgb(36,36,36)",
            "showbackground": true,
            "showgrid": false,
            "showline": true,
            "ticks": "outside",
            "zeroline": false,
            "zerolinecolor": "rgb(36,36,36)"
           },
           "zaxis": {
            "backgroundcolor": "white",
            "gridcolor": "rgb(232,232,232)",
            "gridwidth": 2,
            "linecolor": "rgb(36,36,36)",
            "showbackground": true,
            "showgrid": false,
            "showline": true,
            "ticks": "outside",
            "zeroline": false,
            "zerolinecolor": "rgb(36,36,36)"
           }
          },
          "shapedefaults": {
           "fillcolor": "black",
           "line": {
            "width": 0
           },
           "opacity": 0.3
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           },
           "baxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           },
           "bgcolor": "white",
           "caxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "rgb(232,232,232)",
           "linecolor": "rgb(36,36,36)",
           "showgrid": false,
           "showline": true,
           "ticks": "outside",
           "title": {
            "standoff": 15
           },
           "zeroline": false,
           "zerolinecolor": "rgb(36,36,36)"
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "rgb(232,232,232)",
           "linecolor": "rgb(36,36,36)",
           "showgrid": false,
           "showline": true,
           "ticks": "outside",
           "title": {
            "standoff": 15
           },
           "zeroline": false,
           "zerolinecolor": "rgb(36,36,36)"
          }
         }
        },
        "title": {
         "text": "Artworks from the Global South by Date Created and Acquired, Gender, and Nationality - MoMA (sampled)",
         "x": 0.5
        },
        "xaxis": {
         "anchor": "y",
         "domain": [
          0,
          0.31999999999999995
         ],
         "dtick": 2,
         "nticks": 10,
         "tickformat": "%Y",
         "title": {
          "text": "Year Acquired"
         }
        },
        "xaxis2": {
         "anchor": "y2",
         "domain": [
          0.33999999999999997,
          0.6599999999999999
         ],
         "dtick": 2,
         "matches": "x",
         "nticks": 10,
         "tickformat": "%Y",
         "title": {
          "text": "Year Acquired"
         }
        },
        "xaxis3": {
         "anchor": "y3",
         "domain": [
          0.6799999999999999,
          0.9999999999999999
         ],
         "dtick": 2,
         "matches": "x",
         "nticks": 10,
         "tickformat": "%Y",
         "title": {
          "text": "Year Acquired"
         }
        },
        "yaxis": {
         "anchor": "x",
         "domain": [
          0,
          1
         ],
         "dtick": 2,
         "nticks": 10,
         "tickformat": "%Y",
         "title": {
          "text": "Year Created"
         }
        },
        "yaxis2": {
         "anchor": "x2",
         "domain": [
          0,
          1
         ],
         "dtick": 2,
         "matches": "y",
         "nticks": 10,
         "showticklabels": false,
         "tickformat": "%Y"
        },
        "yaxis3": {
         "anchor": "x3",
         "domain": [
          0,
          1
         ],
         "dtick": 2,
         "matches": "y",
         "nticks": 10,
         "showticklabels": false,
         "tickformat": "%Y"
        }
       }
      }
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#create bubble chart w/facet\n",
    "\n",
    "moma_south = moma_rhz_parallel_artworks_counts_df_final.loc[moma_rhz_parallel_artworks_counts_df_final['Divide'] == 'Global South']\n",
    "moma_south = moma_south.loc[moma_south['Source'] == 'MoMA (sampled)']\n",
    "moma_south = moma_south.loc[moma_south['Nation'] != 'missing']\n",
    "moma_south[['Nation', 'Gender']] = moma_south[['Nation', 'Gender']].astype(str)\n",
    "moma_south = moma_south.replace('M', 'Male Artists')\n",
    "moma_south = moma_south.replace('F', 'Female Artists')\n",
    "moma_south = moma_south.replace('Collectives', 'Collectives & Collabs')\n",
    "\n",
    "fig = px.scatter(moma_south, y='dateCreated', x='dateAcquired', color='Nation', size='size', facet_col='Gender', title= \"Artworks from the Global South by Date Created and Acquired, Gender, and Nationality - MoMA (sampled)\", labels={\"dateCreated\": \"Year Created\", \"dateAcquired\": \"Year Acquired\"}, color_discrete_sequence = ['#ce1126', '#FF9933', '#004B87', '#003DA5', '#CE1126', '#FFFF00', '#EE1C25', '#009c3b', '#FCD116', '#FFB81C', '#cd853f', '#006B3D', '#963846', '#E4312b', '#0033A0', '#239f40', '#006600', '#6CACE4', '#BB0000', '#000000', '#006633', '#4997D0', '#EE1C25', '#FFD100', '#00843D', '#001489', '#D90000', '#00A850', '#008751', '#001489', '#EF3340', '#1EB53A', '#F77F00', '#EF3340', '#007A5E', '#D21034', '#006A4E'])\n",
    "fig.for_each_annotation(lambda a: a.update(text=a.text.split(\"=\")[-1]))\n",
    "fig.update_yaxes(nticks=10, tickformat=\"%Y\",  dtick=2)\n",
    "fig.update_xaxes(nticks=10, tickformat=\"%Y\",  dtick=2)\n",
    "fig.update_traces(hoverinfo='all')\n",
    "fig.update_layout(title_x=0.5)\n",
    "fig.show()\n",
    "#fig.write_html(path+\"Plotly_embeds/scatter_moma_south_artworks.html\", include_plotlyjs='directory',  full_html=False, default_width='100%')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "563fc8bfc07341d58ab56eb7fe1f898f",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FigureWidget({\n",
       "    'data': [{'hoverinfo': 'all',\n",
       "              'hovertemplate': ('Gender=Collec…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#create bubble chart w/facet\n",
    "moma_rhz_parallel_artworks_counts_df_final = moma_rhz_parallel_artworks_counts_df_final.replace('M', 'Male Artists')\n",
    "moma_rhz_parallel_artworks_counts_df_final = moma_rhz_parallel_artworks_counts_df_final.replace('F', 'Female Artists')\n",
    "moma_rhz_parallel_artworks_counts_df_final = moma_rhz_parallel_artworks_counts_df_final.replace('Collectives', 'Collectives & Collabs')\n",
    "moma_rhz_parallel_artworks_counts_df_final = moma_rhz_parallel_artworks_counts_df_final.replace('NB', 'Non-Binary Artists')\n",
    "\n",
    "rhz_scatter = px.scatter(moma_rhz_parallel_artworks_counts_df_final.loc[moma_rhz_parallel_artworks_counts_df_final['Source']=='Rhizome'], y='dateCreated', x='dateAcquired', color='Gender', size='size', title= \"Artworks Acquired by Creation Date - Rhizome\", trendline=\"ols\", color_discrete_sequence = ['#ce93d8', \"#ff5252\", \"#007c91\", \"#ffd54f\"], labels={\"dateCreated\": \"Year Created\", \"dateAcquired\": \"Year Acquired\"})\n",
    "#green line before and after\n",
    "rhz_scatter.add_hline(y=2007, line_dash=\"dot\", line_width=3, \n",
    "              annotation_text=\"2007\", \n",
    "              annotation_position=\"bottom left\",\n",
    "              annotation_font_size=10,\n",
    "              annotation_font_color=\"blue\"\n",
    "             )\n",
    "rhz_scatter.add_vline(x=2007, line_dash=\"dot\", line_width=3\n",
    "             )\n",
    "rhz_scatter.update_yaxes(nticks=10, tickformat=\"%Y\",  dtick=2)\n",
    "rhz_scatter.update_xaxes(nticks=10, tickformat=\"%Y\",  dtick=2)\n",
    "rhz_scatter.update_traces(hoverinfo='all')\n",
    "rhz_scatter.update_layout(title_x=0.5)\n",
    "trendline=\"ols\"\n",
    "\n",
    "moma_scatter = px.scatter(moma_rhz_parallel_artworks_counts_df_final.loc[moma_rhz_parallel_artworks_counts_df_final['Source']=='MoMA (sampled)'], y='dateCreated', x='dateAcquired', color='Gender', size='size', title= \"Artworks Acquired by Creation Date - MoMA (sampled)\", trendline=\"ols\", color_discrete_sequence = ['#ce93d8', \"#ff5252\", \"#007c91\", \"#ffd54f\"], labels={\"dateCreated\": \"Year Created\", \"dateAcquired\": \"Year Acquired\"})\n",
    "#green line before and after\n",
    "moma_scatter.add_hline(y=2007, line_dash=\"dot\", line_width=3, \n",
    "              annotation_text=\"2007\", \n",
    "              annotation_position=\"bottom left\",\n",
    "              annotation_font_size=10,\n",
    "              annotation_font_color=\"blue\"\n",
    "             )\n",
    "moma_scatter.add_vline(x=2007, line_dash=\"dot\", line_width=3\n",
    "             )\n",
    "moma_scatter.update_yaxes(nticks=10, tickformat=\"%Y\",  dtick=2)\n",
    "moma_scatter.update_xaxes(nticks=10, tickformat=\"%Y\",  dtick=2)\n",
    "moma_scatter.update_traces(hoverinfo='all')\n",
    "moma_scatter.update_layout(title_x=0.5)\n",
    "trendline=\"ols\"\n",
    "\n",
    "#create local copies\n",
    "rhz_scatter.write_html(path+\"Plotly_embeds/scatter_rhz_acquisitions.html\", include_plotlyjs='directory',  full_html=False, default_width='100%')\n",
    "moma_scatter.write_html(path+\"Plotly_embeds/scatter_moma_acquisitions.html\", include_plotlyjs='directory',  full_html=False, default_width='100%')\n",
    "\n",
    "rhz_scatter = go.FigureWidget(rhz_scatter)\n",
    "moma_scatter = go.FigureWidget(moma_scatter)\n",
    "scatter_both = ipw.HBox([rhz_scatter, moma_scatter])\n",
    "scatter_both"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Keywords\n",
    "\n",
    "For our last variable, keywords, we played around with bar charts and word clouds but ultimately decided to settle on tree maps as a more meaningful way to display the data to the user as well as tap into some of the themes and filters we had been using elsewhere as the keywords would be the last thing we show in the story - a palate cleanser and conclusion of sorts. <br><br>\n",
    "The first treep map is a simple visualization of keywords appearing more than 100 times in both datasets. The second takes this and other themes to create a nested tree map that shows more information about which keywords are used most in describing who. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Keywords</th>\n",
       "      <th>Counts</th>\n",
       "      <th>Source</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>work</td>\n",
       "      <td>614</td>\n",
       "      <td>MoMA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>art</td>\n",
       "      <td>469</td>\n",
       "      <td>MoMA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>artist</td>\n",
       "      <td>449</td>\n",
       "      <td>MoMA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>made</td>\n",
       "      <td>334</td>\n",
       "      <td>MoMA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>new</td>\n",
       "      <td>295</td>\n",
       "      <td>MoMA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13104</th>\n",
       "      <td>reinfurt</td>\n",
       "      <td>1</td>\n",
       "      <td>Rhizome</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13105</th>\n",
       "      <td>tisch</td>\n",
       "      <td>1</td>\n",
       "      <td>Rhizome</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13106</th>\n",
       "      <td>telecommunications</td>\n",
       "      <td>1</td>\n",
       "      <td>Rhizome</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13107</th>\n",
       "      <td>participatory two</td>\n",
       "      <td>1</td>\n",
       "      <td>Rhizome</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13108</th>\n",
       "      <td>residing</td>\n",
       "      <td>1</td>\n",
       "      <td>Rhizome</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>27503 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                 Keywords  Counts   Source\n",
       "0                    work     614     MoMA\n",
       "1                     art     469     MoMA\n",
       "2                  artist     449     MoMA\n",
       "3                    made     334     MoMA\n",
       "4                     new     295     MoMA\n",
       "...                   ...     ...      ...\n",
       "13104            reinfurt       1  Rhizome\n",
       "13105               tisch       1  Rhizome\n",
       "13106  telecommunications       1  Rhizome\n",
       "13107   participatory two       1  Rhizome\n",
       "13108            residing       1  Rhizome\n",
       "\n",
       "[27503 rows x 3 columns]"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rhz_tree = pd.DataFrame(columns=['Keywords'])\n",
    "rhz_kw_2 = rhizome_txt_stop_kw['Keywords'].values.tolist()\n",
    "rhz_kw_2 = ' '.join(rhz_kw_2)\n",
    "rhz_kw_2 = list(rhz_kw_2.split(', '))\n",
    "rhz_tree['Keywords'] = rhz_kw_2\n",
    "rhz_tree = rhz_tree['Keywords'].value_counts().rename_axis('Keywords').reset_index(name='Counts')\n",
    "rhz_tree['Source'] = 'Rhizome'\n",
    "moma_tree = pd.DataFrame(columns=['Keywords'])\n",
    "moma_kw_2 = moma_full_text_stop['Keywords'].values.tolist()\n",
    "moma_kw_2 = ' '.join(moma_kw_2)\n",
    "moma_kw_2 = list(moma_kw_2.split(', '))\n",
    "moma_tree['Keywords'] = moma_kw_2\n",
    "moma_tree = moma_tree['Keywords'].value_counts().rename_axis('Keywords').reset_index(name='Counts')\n",
    "moma_tree['Source'] = 'MoMA'\n",
    "kw_tree = pd.concat([moma_tree, rhz_tree])\n",
    "kw_tree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [],
   "source": [
    "stopwords=['september', 'december', 'august', 'april', 'july', 'march', 'january', 'february', 'may', 'june', 'october', 'november', 'like', 'use', 'two', 'one', 'u', 'br', 'three', 'The', 'like', 'year', 'also', 'well', 'twenty', 'forms', 'used', 'using', 'part', 'york', ]\n",
    "\n",
    "kw_tree_clean = kw_tree[~kw_tree['Keywords'].isin(stopwords)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/plotly/express/_core.py:1637: FutureWarning:\n",
      "\n",
      "The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.\n",
      "\n",
      "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/plotly/express/_core.py:1637: FutureWarning:\n",
      "\n",
      "The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.\n",
      "\n",
      "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/plotly/express/_core.py:1637: FutureWarning:\n",
      "\n",
      "The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.\n",
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "branchvalues": "total",
         "domain": {
          "x": [
           0,
           1
          ],
          "y": [
           0,
           1
          ]
         },
         "hoverinfo": "label+text+name+percent root+percent entry+percent parent",
         "ids": [
          "Keywords/MoMA/american",
          "Keywords/Rhizome/animation",
          "Keywords/MoMA/art",
          "Keywords/Rhizome/art",
          "Keywords/MoMA/artist",
          "Keywords/Rhizome/artist",
          "Keywords/MoMA/artists",
          "Keywords/Rhizome/audio",
          "Keywords/MoMA/based",
          "Keywords/MoMA/began",
          "Keywords/MoMA/black",
          "Keywords/MoMA/color",
          "Keywords/Rhizome/computer",
          "Keywords/Rhizome/conceptual",
          "Keywords/MoMA/created",
          "Keywords/Rhizome/created",
          "Keywords/Rhizome/digital",
          "Keywords/MoMA/drawing",
          "Keywords/MoMA/drawings",
          "Keywords/MoMA/early",
          "Keywords/MoMA/figure",
          "Keywords/MoMA/first",
          "Keywords/Rhizome/flash",
          "Keywords/MoMA/form",
          "Keywords/Rhizome/html",
          "Keywords/MoMA/images",
          "Keywords/Rhizome/images",
          "Keywords/Rhizome/interactive",
          "Keywords/Rhizome/internet",
          "Keywords/Rhizome/language",
          "Keywords/MoMA/life",
          "Keywords/MoMA/made",
          "Keywords/MoMA/many",
          "Keywords/Rhizome/media",
          "Keywords/Rhizome/narrative",
          "Keywords/MoMA/new",
          "Keywords/Rhizome/new",
          "Keywords/Rhizome/online",
          "Keywords/MoMA/painting",
          "Keywords/MoMA/paintings",
          "Keywords/MoMA/paper",
          "Keywords/MoMA/photographs",
          "Keywords/Rhizome/piece",
          "Keywords/MoMA/print",
          "Keywords/MoMA/printmaking",
          "Keywords/MoMA/prints",
          "Keywords/Rhizome/project",
          "Keywords/MoMA/sculpture",
          "Keywords/MoMA/series",
          "Keywords/Rhizome/series",
          "Keywords/MoMA/space",
          "Keywords/Rhizome/space",
          "Keywords/Rhizome/text",
          "Keywords/MoMA/time",
          "Keywords/Rhizome/time",
          "Keywords/Rhizome/video",
          "Keywords/Rhizome/visual",
          "Keywords/MoMA/war",
          "Keywords/Rhizome/web",
          "Keywords/MoMA/work",
          "Keywords/Rhizome/work",
          "Keywords/MoMA/works",
          "Keywords/MoMA/world",
          "Keywords/Rhizome/world",
          "Keywords/MoMA/years",
          "Keywords/MoMA",
          "Keywords/Rhizome",
          "Keywords"
         ],
         "labels": [
          "american",
          "animation",
          "art",
          "art",
          "artist",
          "artist",
          "artists",
          "audio",
          "based",
          "began",
          "black",
          "color",
          "computer",
          "conceptual",
          "created",
          "created",
          "digital",
          "drawing",
          "drawings",
          "early",
          "figure",
          "first",
          "flash",
          "form",
          "html",
          "images",
          "images",
          "interactive",
          "internet",
          "language",
          "life",
          "made",
          "many",
          "media",
          "narrative",
          "new",
          "new",
          "online",
          "painting",
          "paintings",
          "paper",
          "photographs",
          "piece",
          "print",
          "printmaking",
          "prints",
          "project",
          "sculpture",
          "series",
          "series",
          "space",
          "space",
          "text",
          "time",
          "time",
          "video",
          "visual",
          "war",
          "web",
          "work",
          "work",
          "works",
          "world",
          "world",
          "years",
          "MoMA",
          "Rhizome",
          "Keywords"
         ],
         "maxdepth": 2,
         "name": "",
         "parents": [
          "Keywords/MoMA",
          "Keywords/Rhizome",
          "Keywords/MoMA",
          "Keywords/Rhizome",
          "Keywords/MoMA",
          "Keywords/Rhizome",
          "Keywords/MoMA",
          "Keywords/Rhizome",
          "Keywords/MoMA",
          "Keywords/MoMA",
          "Keywords/MoMA",
          "Keywords/MoMA",
          "Keywords/Rhizome",
          "Keywords/Rhizome",
          "Keywords/MoMA",
          "Keywords/Rhizome",
          "Keywords/Rhizome",
          "Keywords/MoMA",
          "Keywords/MoMA",
          "Keywords/MoMA",
          "Keywords/MoMA",
          "Keywords/MoMA",
          "Keywords/Rhizome",
          "Keywords/MoMA",
          "Keywords/Rhizome",
          "Keywords/MoMA",
          "Keywords/Rhizome",
          "Keywords/Rhizome",
          "Keywords/Rhizome",
          "Keywords/Rhizome",
          "Keywords/MoMA",
          "Keywords/MoMA",
          "Keywords/MoMA",
          "Keywords/Rhizome",
          "Keywords/Rhizome",
          "Keywords/MoMA",
          "Keywords/Rhizome",
          "Keywords/Rhizome",
          "Keywords/MoMA",
          "Keywords/MoMA",
          "Keywords/MoMA",
          "Keywords/MoMA",
          "Keywords/Rhizome",
          "Keywords/MoMA",
          "Keywords/MoMA",
          "Keywords/MoMA",
          "Keywords/Rhizome",
          "Keywords/MoMA",
          "Keywords/MoMA",
          "Keywords/Rhizome",
          "Keywords/MoMA",
          "Keywords/Rhizome",
          "Keywords/Rhizome",
          "Keywords/MoMA",
          "Keywords/Rhizome",
          "Keywords/Rhizome",
          "Keywords/Rhizome",
          "Keywords/MoMA",
          "Keywords/Rhizome",
          "Keywords/MoMA",
          "Keywords/Rhizome",
          "Keywords/MoMA",
          "Keywords/MoMA",
          "Keywords/Rhizome",
          "Keywords/MoMA",
          "Keywords",
          "Keywords",
          ""
         ],
         "root": {
          "color": "lightgrey"
         },
         "textfont": {
          "size": 14
         },
         "textinfo": "label+value+percent parent",
         "textposition": "middle center",
         "type": "treemap",
         "values": [
          110,
          119,
          469,
          277,
          449,
          104,
          186,
          106,
          105,
          148,
          115,
          116,
          101,
          149,
          135,
          106,
          164,
          139,
          101,
          158,
          106,
          240,
          139,
          103,
          165,
          147,
          151,
          148,
          148,
          125,
          131,
          334,
          113,
          109,
          102,
          295,
          214,
          104,
          271,
          169,
          122,
          107,
          113,
          103,
          107,
          202,
          266,
          133,
          260,
          115,
          104,
          190,
          114,
          156,
          126,
          167,
          142,
          111,
          139,
          614,
          276,
          225,
          163,
          124,
          115,
          6662,
          4303,
          10965
         ]
        }
       ],
       "layout": {
        "legend": {
         "tracegroupgap": 0
        },
        "margin": {
         "b": 25,
         "l": 25,
         "r": 25,
         "t": 50
        },
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "rgb(36,36,36)"
            },
            "error_y": {
             "color": "rgb(36,36,36)"
            },
            "marker": {
             "line": {
              "color": "white",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "white",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "rgb(36,36,36)",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "rgb(36,36,36)"
            },
            "baxis": {
             "endlinecolor": "rgb(36,36,36)",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "rgb(36,36,36)"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "line": {
              "color": "white",
              "width": 0.6
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "fillpattern": {
             "fillmode": "overlay",
             "size": 10,
             "solidity": 0.2
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 1,
              "tickcolor": "rgb(36,36,36)",
              "ticks": "outside"
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 1,
             "tickcolor": "rgb(36,36,36)",
             "ticks": "outside"
            },
            "colorscale": [
             [
              0,
              "#440154"
             ],
             [
              0.1111111111111111,
              "#482878"
             ],
             [
              0.2222222222222222,
              "#3e4989"
             ],
             [
              0.3333333333333333,
              "#31688e"
             ],
             [
              0.4444444444444444,
              "#26828e"
             ],
             [
              0.5555555555555556,
              "#1f9e89"
             ],
             [
              0.6666666666666666,
              "#35b779"
             ],
             [
              0.7777777777777778,
              "#6ece58"
             ],
             [
              0.8888888888888888,
              "#b5de2b"
             ],
             [
              1,
              "#fde725"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "rgb(237,237,237)"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "rgb(217,217,217)"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 1,
            "tickcolor": "rgb(36,36,36)",
            "ticks": "outside"
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "rgb(103,0,31)"
            ],
            [
             0.1,
             "rgb(178,24,43)"
            ],
            [
             0.2,
             "rgb(214,96,77)"
            ],
            [
             0.3,
             "rgb(244,165,130)"
            ],
            [
             0.4,
             "rgb(253,219,199)"
            ],
            [
             0.5,
             "rgb(247,247,247)"
            ],
            [
             0.6,
             "rgb(209,229,240)"
            ],
            [
             0.7,
             "rgb(146,197,222)"
            ],
            [
             0.8,
             "rgb(67,147,195)"
            ],
            [
             0.9,
             "rgb(33,102,172)"
            ],
            [
             1,
             "rgb(5,48,97)"
            ]
           ],
           "sequential": [
            [
             0,
             "#440154"
            ],
            [
             0.1111111111111111,
             "#482878"
            ],
            [
             0.2222222222222222,
             "#3e4989"
            ],
            [
             0.3333333333333333,
             "#31688e"
            ],
            [
             0.4444444444444444,
             "#26828e"
            ],
            [
             0.5555555555555556,
             "#1f9e89"
            ],
            [
             0.6666666666666666,
             "#35b779"
            ],
            [
             0.7777777777777778,
             "#6ece58"
            ],
            [
             0.8888888888888888,
             "#b5de2b"
            ],
            [
             1,
             "#fde725"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#440154"
            ],
            [
             0.1111111111111111,
             "#482878"
            ],
            [
             0.2222222222222222,
             "#3e4989"
            ],
            [
             0.3333333333333333,
             "#31688e"
            ],
            [
             0.4444444444444444,
             "#26828e"
            ],
            [
             0.5555555555555556,
             "#1f9e89"
            ],
            [
             0.6666666666666666,
             "#35b779"
            ],
            [
             0.7777777777777778,
             "#6ece58"
            ],
            [
             0.8888888888888888,
             "#b5de2b"
            ],
            [
             1,
             "#fde725"
            ]
           ]
          },
          "colorway": [
           "#1F77B4",
           "#FF7F0E",
           "#2CA02C",
           "#D62728",
           "#9467BD",
           "#8C564B",
           "#E377C2",
           "#7F7F7F",
           "#BCBD22",
           "#17BECF"
          ],
          "font": {
           "color": "rgb(36,36,36)"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "white",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "white",
          "polar": {
           "angularaxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           },
           "bgcolor": "white",
           "radialaxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "white",
            "gridcolor": "rgb(232,232,232)",
            "gridwidth": 2,
            "linecolor": "rgb(36,36,36)",
            "showbackground": true,
            "showgrid": false,
            "showline": true,
            "ticks": "outside",
            "zeroline": false,
            "zerolinecolor": "rgb(36,36,36)"
           },
           "yaxis": {
            "backgroundcolor": "white",
            "gridcolor": "rgb(232,232,232)",
            "gridwidth": 2,
            "linecolor": "rgb(36,36,36)",
            "showbackground": true,
            "showgrid": false,
            "showline": true,
            "ticks": "outside",
            "zeroline": false,
            "zerolinecolor": "rgb(36,36,36)"
           },
           "zaxis": {
            "backgroundcolor": "white",
            "gridcolor": "rgb(232,232,232)",
            "gridwidth": 2,
            "linecolor": "rgb(36,36,36)",
            "showbackground": true,
            "showgrid": false,
            "showline": true,
            "ticks": "outside",
            "zeroline": false,
            "zerolinecolor": "rgb(36,36,36)"
           }
          },
          "shapedefaults": {
           "fillcolor": "black",
           "line": {
            "width": 0
           },
           "opacity": 0.3
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           },
           "baxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           },
           "bgcolor": "white",
           "caxis": {
            "gridcolor": "rgb(232,232,232)",
            "linecolor": "rgb(36,36,36)",
            "showgrid": false,
            "showline": true,
            "ticks": "outside"
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "rgb(232,232,232)",
           "linecolor": "rgb(36,36,36)",
           "showgrid": false,
           "showline": true,
           "ticks": "outside",
           "title": {
            "standoff": 15
           },
           "zeroline": false,
           "zerolinecolor": "rgb(36,36,36)"
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "rgb(232,232,232)",
           "linecolor": "rgb(36,36,36)",
           "showgrid": false,
           "showline": true,
           "ticks": "outside",
           "title": {
            "standoff": 15
           },
           "zeroline": false,
           "zerolinecolor": "rgb(36,36,36)"
          }
         }
        },
        "title": {
         "text": "Keywords Appearning More Than 50 Times Across Collections",
         "x": 0.5
        },
        "treemapcolorway": [
         "#7986cb",
         "#9ccc65"
        ]
       }
      }
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "kw_tree_clean = kw_tree_clean.loc[kw_tree_clean['Counts'] >= 100]\n",
    "fig = px.treemap(kw_tree_clean, path=[px.Constant(\"Keywords\"), 'Source', 'Keywords'], values='Counts', maxdepth=2, title= \"Keywords Appearning More Than 50 Times Across Collections\", color_discrete_sequence = ['#7986cb', \"#9ccc65\"])\n",
    "fig.update_traces(hovertemplate=None, textinfo='label+value+percent parent', hoverinfo='label+text+name+percent root+percent entry+percent parent', root_color=\"lightgrey\", textposition='middle center', textfont=dict(size=14))\n",
    "fig.update_layout(margin = dict(t=50, l=25, r=25, b=25), title_x=0.5)\n",
    "fig.show()\n",
    "#create local copies\n",
    "#fig.write_html(path+\"Plotly_embeds/keywords_tree_map.html\", include_plotlyjs='directory',  full_html=False, default_width='100%')"
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "aee8b7b246df8f9039afb4144a1f6fd8d2ca17a180786b69acc140d282b71a49"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}