CONUS404_Panel_App.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# CONUS404 Explorer Panel App\n",
    "Using hvplot and Panel from the [holoviz](https://holoviz.org) suite"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import xarray as xr\n",
    "import cf_xarray \n",
    "import metpy\n",
    "import intake\n",
    "import numpy as np\n",
    "import hvplot.xarray\n",
    "from geoviews import tile_sources as gvts\n",
    "import panel as pn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "url = 'https://raw.githubusercontent.com/hytest-org/hytest/main/dataset_catalog/hytest_intake_catalog.yml'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "hytest_cat = intake.open_catalog(url)\n",
    "list(hytest_cat)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# open the conus404 sub-catalog\n",
    "cat = hytest_cat['conus404-catalog']\n",
    "list(cat)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "dataset = 'conus404-hourly-osn'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This app views the entire spatial domain, so tell Dasks to read in bigger chunks to speed up loading. The native chunk size in the Zarr dataset is `{'x':175, 'y':175} ` so pick multiples of that.  This reduces the time to load the entire domain on a 2 core machine on AWS us-west-2 from 15s to 5s. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "cat[dataset].kwargs.update({'chunks':{'x':2*175, 'y':2*175}})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "ds = cat[dataset].to_dask()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Add `standard_name='time'` so `cf_xarray` can find the time coordinate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "ds.time.attrs['standard_name'] = 'time'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Find data variables that have a time dimension (but are not time `bounds` variables)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "time_vars = []\n",
    "for var in ds.data_vars:\n",
    "    if len(ds[var].dims) > 0:\n",
    "        if 'time' in ds[var].dims[0] and not 'bounds' in var:\n",
    "            time_vars.append(var)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "print(len(time_vars))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "init_var = 'T2'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "ds[init_var]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use Metpy to parse the CRS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "ds = ds.metpy.parse_cf()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "crs = ds.T2.metpy.cartopy_crs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create widget for variable selection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "var_select = pn.widgets.Select(name='CONUS404 Variables:', options=time_vars, value=init_var)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create widget for basemap selection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "base_map_select = pn.widgets.Select(name='Basemap:', options=gvts.tile_sources, value=gvts.OSM)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "The `plot` function below creates the `hvplot` panel layout object.   ROMS is on a C-Grid and variables have different coordinates depending on where they are on the grid.   So we use `cf-xarray` to determine the time, depth, longitude and latitude coordinates for each variable.  \n",
    "\n",
    "We specify a basemap, specify the CRS for the selected variable, and indicate we want to `rasterize` the plot so that we can render massive meshes in the browser. \n",
    "\n",
    "We also specify the `groupby` parameter as the list of dimensions that remains after we remove Y and X: `ds[var].dims[:-2]`, which automatically handles variables with either dimensions [T, Y, X] or [T, Z, Y, X].  \n",
    "\n",
    "We also specify which `bokeh` controls we want to be active by default:  the `wheel_zoom` and `pan` controls.\n",
    "\n",
    "We also change the default slider to a selection widget for the time dimension (and vertical dimension if it exists) so that specific values are easy to select.  See https://stackoverflow.com/a/54912917/2005869"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "time_vals = 48"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One might think that it would be faster to make an image map using the projected coordinates instead of quadmesh with 2D lon,lat coordinates, but it seems the same.   And QuadMesh is a nicer looking map. "
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "@pn.depends(var_select, base_map_select)\n",
    "def plot(var, base_map):\n",
    "    extra_dims = list(ds[var].dims[:-2])\n",
    "    da = ds[var].cf.isel(T=slice(-time_vals,-1)).unify_chunks()\n",
    "\n",
    "    mesh = da.hvplot(x='x', y='y', rasterize=True, crs=crs, title=var, attr_labels=False, \n",
    "                    cmap='turbo', width=600, grid=True, xlabel='Longitude', ylabel='Latitude',\n",
    "                          height=600).opts(alpha=0.7)\n",
    "\n",
    "    return pn.panel(mesh * base_map, widgets={extra_dims[0]: pn.widgets.Select})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "@pn.depends(var_select, base_map_select)\n",
    "def plot(var, base_map):\n",
    "    extra_dims = list(ds[var].dims[:-2])\n",
    "    da = ds[var].cf.isel(T=slice(-time_vals,-1)).unify_chunks().load()\n",
    "\n",
    "    mesh = da.hvplot.quadmesh(x=da.cf['longitude'].name, y=da.cf['latitude'].name,\n",
    "                    rasterize=True, geo=True, title=var, attr_labels=False, \n",
    "                    cmap='turbo', grid=True, xlabel='Longitude', ylabel='Latitude',\n",
    "                          frame_height=400).opts(alpha=0.7)\n",
    "\n",
    "    return pn.panel(mesh * base_map, widgets={extra_dims[0]: pn.widgets.Select})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "col = pn.Column(base_map_select, var_select, plot)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "We use `.servable()` below not only to display the panel object, but to make the panel servable outside the notebook via:\n",
    "\n",
    "`panel serve CONUS404_Panel_App.ipynb`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "col.servable('CONUS404 Dashboard')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "global-global-pangeo",
   "language": "python",
   "name": "conda-env-global-global-pangeo-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.6"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {},
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}