Background_masking_labels_showcase.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "08536f17",
   "metadata": {},
   "source": [
    "# This notebook shows off Background options in views_mapper2\n",
    "Ensure that the views_mapper2 is up to date with the latest update as documented in main read me file\n",
    "Things covered in this notebook are background options, transparency, masking, and views_experimental labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cf3a60bd",
   "metadata": {},
   "outputs": [],
   "source": [
    "#mapper version verification\n",
    "!pip show views_mapper2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b751b9fd",
   "metadata": {},
   "outputs": [],
   "source": [
    "#obtain data, country month\n",
    "from viewser import Queryset, Column\n",
    "\n",
    "queryset = (Queryset(\"Mapper_tester_cm\", \"country_month\")\n",
    "            # CONFLICT TYPES NOT LOGGED and NOT time lagged\n",
    "            .with_column(Column(\"ged_best_sb\", from_table=\"ged2_cm\", from_column=\"ged_sb_best_sum_nokgi\")\n",
    "                         .transform.missing.fill()\n",
    "                         )\n",
    "            # .with_column(Column('name', from_table='country', from_column= 'name')\n",
    "            # the above does not seem to want to work\n",
    "\n",
    "            .with_theme(\"fatalities\")\n",
    "\n",
    "            .describe(\"\"\"Fatalities conflict history, cm level\n",
    "            Mapper testing\n",
    "\n",
    "            \"\"\")\n",
    "\n",
    "            )\n",
    "\n",
    "cm_master = queryset.publish().fetch()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bd22fc1d",
   "metadata": {},
   "outputs": [],
   "source": [
    "#fetch the multipolygons for mapping\n",
    "import geopandas as gpd\n",
    "import sqlalchemy as sa\n",
    "from ingester3.config import source_db_path\n",
    "\n",
    "engine = sa.create_engine(source_db_path)\n",
    "gdf_ci_master = gpd.GeoDataFrame.from_postgis(\n",
    "    \"SELECT id as country_id, name, in_africa, in_me, geom FROM prod.country\",\n",
    "    engine,\n",
    "    geom_col='geom'\n",
    ")\n",
    "gdf_ci_master = gdf_ci_master.to_crs(4326)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7f135e0f",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Mapper2 \n",
    "from views_mapper2.mapper2 import *\n",
    "from views_mapper2.label_writer import *\n",
    "from views_mapper2.dictionary_writer import *\n",
    "from views_mapper2.BBoxWriter import*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "688ad617",
   "metadata": {},
   "source": [
    "# showing off the various backgrounds\n",
    "\n",
    "background tiles are essentially jpeg like images that vary depending on the location where we focus. More information on regional information with closer look and less further out. As we are using them in their non-native forms through geopandas, a lot of the original html/web modifiability is lost. So not possible to ask only for country names and large cities. \n",
    "\n",
    "To see examples, please substitute with the names below in the name variable, e.g. name = 'OpenTopoMap'\n",
    "\n",
    "Some of the maps look better as background in masked versions. \n",
    "\n",
    "\n",
    "1. OpenTopoMap - includes both country names and topography, colorful, no built in layered labels so 100% opacity will prevent the view of the country names. Additionally colorful background interferes with the 1:1 use of the colorbar when violence layer is transparent as one must account for underlying color and how it interacts with the transparent overlay. But the general gist of the color is still understandble. Works well for masked versions of specific countries.\n",
    "2. OpenStreetMap - includes names of countries, regions, etc. No topographic data. No build in layered labels so 100% opacity will prevent the view of country names. Monochromatic colorpalette allows for easier interpretation of transparent layer overlayers. \n",
    "3. StamenLite - includes names of countries, regions, etc. No topographic data. Greyscale. Built layer. Does not look good for 100% opacity but this version will still make it possible to see the labels. Due to nature of geopandas, labels occasionally collide with the viewtextbox as they are layered on top of everything. \n",
    "4. StamenWatercolor - same as above but with colorful theme, no country borders build in.\n",
    "5. StamenTerrain - tiles with both country names and terrain, no built in labels so 100% opacity will interfere.\n",
    "6. StamenTerrainBackground - only terrain information of the above\n",
    "7. CartoDBPositron - similar to StamenLite but with slightly less information\n",
    "8. CartoDBVoyager - similar to above, more recent release\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6d3dd4a4",
   "metadata": {},
   "outputs": [],
   "source": [
    "data= cm_master.copy()\n",
    "gdf = gdf_ci_master.copy()\n",
    "\n",
    "data = data.join(gdf.set_index(\"country_id\"))\n",
    "gdf = gpd.GeoDataFrame(data, geometry=\"geom\")\n",
    "\n",
    "#adjust the background here\n",
    "name = 'CartoDBPositron'\n",
    "\n",
    "\n",
    "a = Mapper2(\n",
    "    width=10,\n",
    "    height=10,\n",
    "    frame_on=True,\n",
    "    title=name, \n",
    "    bbox = bbox_from_cid('ame')\n",
    ").add_layer(\n",
    "    gdf=gdf.loc[450],\n",
    "    map_dictionary = dictionary_stand,\n",
    "    cmap = 'rainbow',\n",
    "    transparency = 1,\n",
    "    edgecolor=\"black\",\n",
    "    linewidth=0.7,\n",
    "    column=\"ged_best_sb\", \n",
    "    background = name\n",
    ").add_views_textbox(\n",
    "    text='hello world, hello world, hello world',\n",
    "    textsize=10)\n",
    "\n",
    "a2 = Mapper2(\n",
    "    width=10,\n",
    "    height=10,\n",
    "    frame_on=True,\n",
    "    title=f'{name}_country', \n",
    "    bbox = bbox_from_cid(79)\n",
    ").add_layer(\n",
    "    gdf=gdf.loc[450],\n",
    "    map_dictionary = dictionary_stand,\n",
    "    cmap = 'rainbow',\n",
    "    transparency = 1,\n",
    "    edgecolor=\"black\",\n",
    "    linewidth=0.7,\n",
    "    column=\"ged_best_sb\", \n",
    "    background = name\n",
    ").add_views_textbox(\n",
    "    text='hello world, hello world, hello world',\n",
    "    textsize=10)\n",
    "\n",
    "b = Mapper2(\n",
    "    width=10,\n",
    "    height=10,\n",
    "    frame_on=True,\n",
    "    title=f'{name}_with_transparency', \n",
    "    bbox = bbox_from_cid('ame')\n",
    ").add_layer(\n",
    "    gdf=gdf.loc[450],\n",
    "    map_dictionary = dictionary_stand,\n",
    "    cmap = 'rainbow',\n",
    "    transparency = 0.5,\n",
    "    edgecolor=\"black\",\n",
    "    linewidth=0.7,\n",
    "    column=\"ged_best_sb\", \n",
    "    background = name\n",
    ").add_views_textbox(\n",
    "    text='hello world, hello world, hello world',\n",
    "    textsize=10)\n",
    "\n",
    "c = Mapper2(\n",
    "    width=10,\n",
    "    height=10,\n",
    "    frame_on=True,\n",
    "    title= f'{name}_only', \n",
    "    bbox = bbox_from_cid('ame')\n",
    ").add_layer(\n",
    "    gdf=gdf.loc[450],\n",
    "    map_dictionary = dictionary_stand,\n",
    "    cmap = 'rainbow',\n",
    "    transparency = 0.0,\n",
    "    edgecolor=\"black\",\n",
    "    linewidth=0.7,\n",
    "    column=\"ged_best_sb\", \n",
    "    background = name\n",
    ").add_views_textbox(\n",
    "    text='hello world, hello world, hello world',\n",
    "    textsize=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f8c6a489",
   "metadata": {},
   "source": [
    "# Experimental labels\n",
    "\n",
    "experimental labels are my own creation. It creates and centers a name in the country geometry\n",
    "works at variable success rate\n",
    "adjustment of the font size can result in improved results\n",
    "recommend with zoom in on specific countries\n",
    "\n",
    "adjust font to note how sometimes labels work and other times they start to collide"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "95de35d6",
   "metadata": {},
   "outputs": [],
   "source": [
    "data= cm_master.copy()\n",
    "gdf = gdf_ci_master.copy()\n",
    "\n",
    "data = data.join(gdf.set_index(\"country_id\"))\n",
    "gdf = gpd.GeoDataFrame(data, geometry=\"geom\")\n",
    "\n",
    "font = 10\n",
    "\n",
    "\n",
    "a = Mapper2(\n",
    "    width=10,\n",
    "    height=10,\n",
    "    frame_on=True,\n",
    "    title='views_experimental1', \n",
    "    bbox = bbox_from_cid('ame')\n",
    ").add_layer(\n",
    "    gdf=gdf.loc[450],\n",
    "    map_dictionary = dictionary_stand,\n",
    "    cmap = 'rainbow',\n",
    "    transparency = 1,\n",
    "    edgecolor=\"black\",\n",
    "    linewidth=0.7,\n",
    "    column=\"ged_best_sb\",\n",
    "    views_experimental_labels = 10\n",
    ").add_views_textbox(\n",
    "    text='hello world, hello world, hello world',\n",
    "    textsize=10)\n",
    "\n",
    "a2 = Mapper2(\n",
    "    width=10,\n",
    "    height=10,\n",
    "    frame_on=True,\n",
    "    title= 'views_experimental2', \n",
    "    bbox = bbox_from_cid(79)\n",
    ").add_layer(\n",
    "    gdf=gdf.loc[450],\n",
    "    map_dictionary = dictionary_stand,\n",
    "    cmap = 'rainbow',\n",
    "    transparency = 1,\n",
    "    edgecolor=\"black\",\n",
    "    linewidth=0.7,\n",
    "    column=\"ged_best_sb\",\n",
    "    views_experimental_labels = font\n",
    ").add_views_textbox(\n",
    "    text='hello world, hello world, hello world',\n",
    "    textsize=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3922309e",
   "metadata": {},
   "source": [
    "## Masks\n",
    "\n",
    "Masks allow for greying out of the regions not of interest\n",
    "The available sections are africa, africa+middle east, and country\n",
    "All must be written as strings and require a column with country_names\n",
    "I have defaulted it to look for names as written in the name of the country table (some variation from iso_names)\n",
    "Ensure appropriate spelling of the country before use\n",
    "\n",
    "Masks can be combined with the backgrounds as underlying layers or used on their own\n",
    "Below are two cells that allow you to use either method\n",
    "Additionally a third cell provides a demonstration for pg level maps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "53ce0ffb",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Masking using layers, this version of masking is required if you wish to use no background\n",
    "#essentially you create a layer where all values are grey-scaled and placing a mask over the top\n",
    "#changing transparency of the base layer can help achieve a more or less washed out appearance\n",
    "#WARNING: very northern or southern countries start to behave strangely using 2 layers\n",
    "#use the add_mask on its own via second method for those countries as globe bbox\n",
    "\n",
    "data= cm_master.copy()\n",
    "gdf = gdf_ci_master.copy()\n",
    "\n",
    "data = data.join(gdf.set_index(\"country_id\"))\n",
    "gdf = gpd.GeoDataFrame(data, geometry=\"geom\")\n",
    "\n",
    "#you can adjust these to see various effects and combinations\n",
    "background2 = None #adjust this and note no difference, as the layer1 obscurs the background at transparency = 1\n",
    "views_label_font = 7\n",
    "textbox_label_font = 20\n",
    "masking_location_choice = \"Syria\" #choices allowed are 'africa', 'ame', or country_name\n",
    "\n",
    "\n",
    "ame=Mapper2(\n",
    "    width=20,\n",
    "    height=20,\n",
    "    frame_on=True,\n",
    "    title='mask with layers',\n",
    "    bbox=bbox_from_cid('ame')\n",
    ").add_layer(\n",
    "    gdf=gdf.loc[450],\n",
    "    map_dictionary = dictionary_stand,\n",
    "    cmap = 'binary',\n",
    "    transparency = 1.0,\n",
    "    edgecolor=\"black\",\n",
    "    linewidth=0.5,\n",
    "    column=\"ged_best_sb\"\n",
    ").add_mask(\n",
    "    gdf = gdf.loc[450],\n",
    "    map_dictionary = dictionary_stand,\n",
    "    cmap = 'rainbow',\n",
    "    transparency = 1,\n",
    "    masking_location = masking_location_choice,\n",
    "    views_experimental_labels = views_label_font,\n",
    "    column='ged_best_sb',\n",
    "    background = background2,\n",
    "    edgecolor=\"black\",\n",
    "    linewidth=1\n",
    ").add_views_textbox(\n",
    "    text='hello world, hello world',\n",
    "    textsize=textbox_label_font)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "274174a9",
   "metadata": {},
   "outputs": [],
   "source": [
    "gdf.loc[450]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6dcb8e95",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Masking by using .add_mask requires a background\n",
    "#this version of masking requires the use of background, otherwise you will create floating regions\n",
    "\n",
    "data= cm_master.copy()\n",
    "gdf = gdf_ci_master.copy()\n",
    "\n",
    "data = data.join(gdf.set_index(\"country_id\"))\n",
    "gdf = gpd.GeoDataFrame(data, geometry=\"geom\")\n",
    "\n",
    "#you can adjust these to see various effects and combinations\n",
    "#you can also delete the views_experimental_labels to see appearance without them\n",
    "#avoid using the views_labels with globe level\n",
    "background2 = 'StamenTerrainBackground' #use None to see the need for background use\n",
    "views_label_font = 7\n",
    "textbox_label_font = 20\n",
    "masking_location_choice = 'Congo' #choices allowed are 'africa', 'ame', or country_id number\n",
    "\n",
    "\n",
    "ame=Mapper2(\n",
    "    width=20,\n",
    "    height=20,\n",
    "    frame_on=True,\n",
    "    title='mask with layers',\n",
    "    bbox=bbox_from_cid('ame') ##you can change this to ame, africa or country bbox\n",
    ").add_mask(\n",
    "    gdf = gdf.loc[450],\n",
    "    map_dictionary = dictionary_stand,\n",
    "    cmap = 'rainbow',\n",
    "    transparency = 1,\n",
    "    masking_location = masking_location_choice,\n",
    "    column='ged_best_sb',\n",
    "    background = background2,\n",
    "    views_experimental_labels = views_label_font,\n",
    "    edgecolor=\"black\",\n",
    "    linewidth=1\n",
    ").add_views_textbox(\n",
    "    text='hello world, hello world',\n",
    "    textsize=textbox_label_font)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90aaa508",
   "metadata": {},
   "source": [
    "### PG level masking, note the slightly more complicated geometry creation\n",
    "- also note that the information for pg is not available for the entire globe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5ccfe1af",
   "metadata": {},
   "outputs": [],
   "source": [
    "#PG level mapping example with masking\n",
    "\n",
    "\n",
    "#Data gathering\n",
    "queryset = (Queryset(\"m_fatalities_conflict_pgm\", \"priogrid_month\")\n",
    "              #CONFLICT TYPES NOT LOGGED and NOT time lagged \n",
    "            .with_column(Column(\"ged_best_sb\", from_table = \"ged2_pgm\", from_column = \"ged_sb_best_sum_nokgi\")\n",
    "                         .transform.missing.fill()\n",
    "                        )\n",
    "                \n",
    "            #.with_column(Column('name', from_table='country', from_column= 'name')\n",
    "            #the above does not seem to want to work \n",
    "                         \n",
    "            .with_theme(\"fatalities\")\n",
    "                         \n",
    "            .describe(\"\"\"Fatalities conflict history, pgm level\n",
    "            Fatalities description \n",
    "                \n",
    "            \"\"\")\n",
    "           )\n",
    "\n",
    "pg_master = queryset.publish().fetch()\n",
    "\n",
    "\n",
    "#fetching geometries\n",
    "engine = sa.create_engine(source_db_path)\n",
    "gdf_pid = gpd.GeoDataFrame.from_postgis(\n",
    "    \"SELECT id as priogrid_gid, in_africa, in_me, geom FROM prod.priogrid\", \n",
    "    engine, \n",
    "    geom_col='geom'\n",
    ")\n",
    "gdf_pid = gdf_pid.to_crs(4326)\n",
    "\n",
    "#now grab the country names\n",
    "pg_country_queryset = (Queryset(\"m_associate_country_2_priogrid\", \"priogrid_month\")\n",
    "              #CONFLICT TYPES NOT LOGGED and NOT time lagged \n",
    "            #.with_column(Column(\"month_id\", from_table = \"priogrid_month\", from_column = \"month_id\"))\n",
    "            .with_column(Column('name', from_table = 'country', from_column = 'name'))\n",
    "                             \n",
    "            .with_theme(\"Mapping\")\n",
    "                         \n",
    "            .describe(\"\"\"Mapping, allows for association of priogrid with country \n",
    "                \n",
    "            \"\"\")\n",
    "            \n",
    "           )\n",
    "pg_country = pg_country_queryset.publish().fetch().reset_index()\n",
    "#pg_country_cleaned = pg_country.reset_index().drop(['month_id'], axis = 1).drop_duplicates(subset=['priogrid_gid'])\n",
    "\n",
    "gdf_pid_master = pd.merge(left = gdf_pid, right = pg_country, left_on = 'priogrid_gid', \n",
    "                          right_on = 'priogrid_gid', how = 'right').set_index(['month_id', 'priogrid_gid'])\n",
    "\n",
    "\n",
    "#grab country geometries for borders creation\n",
    "gdf_ci_master = gpd.GeoDataFrame.from_postgis(\n",
    "    \"SELECT id as country_id, in_africa, in_me, geom FROM prod.country\",\n",
    "    engine,\n",
    "    geom_col='geom'\n",
    ")\n",
    "gdf_ci_master = gdf_ci_master.to_crs(4326)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b0f6a908",
   "metadata": {},
   "outputs": [],
   "source": [
    "#mapping code for the above\n",
    "\n",
    "data = pg_master.copy()\n",
    "gdf = gdf_pid_master.copy()\n",
    "data_pid = pd.merge(left = data, right = gdf, left_index = True, right_index = True, how = 'left')\n",
    "gdf = gpd.GeoDataFrame(data_pid, geometry=\"geom\")\n",
    "gdf_c = gdf_ci_master.copy()\n",
    "\n",
    "#you can adjust these to see various effects and combinations\n",
    "#please do not use the experimental labels as the name will be the name of priogrid_cell\n",
    "background2 = 'StamenTerrainBackground' #use None to see the need for background use\n",
    "textbox_label_font = 20\n",
    "masking_location_choice = 'Somalia' #choices allowed are 'africa', 'ame', or country_id number\n",
    "\n",
    "\n",
    "masked=Mapper2(\n",
    "    width=20,\n",
    "    height=20,\n",
    "    frame_on=True,\n",
    "    title='mask with layers',\n",
    "    bbox=bbox_from_cid('ame')\n",
    ").add_mask(\n",
    "    gdf = gdf.loc[450],\n",
    "    map_dictionary = dictionary_stand,\n",
    "    cmap = 'rainbow',\n",
    "    transparency = 1,\n",
    "    masking_location = masking_location_choice,\n",
    "    column='ged_best_sb',\n",
    "    background = background2,\n",
    "    edgecolor=\"black\",\n",
    "    linewidth=0.5\n",
    ").add_views_textbox(\n",
    "    text='hello world, hello world',\n",
    "    textsize=textbox_label_font)\n",
    "\n",
    "#Jim procedure for country border creation using the geometries inherent to views3\n",
    "ax = masked.ax\n",
    "masked_with_borders=gdf_c.plot(ax=ax,edgecolor='black',linewidth=2.0,facecolor='None')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}