Config.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>Setting Stuff Up</h2>\n",
    "\n",
    "Here we import some packages that we'll need in various places."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/home/jovyan/agave\n",
      "Requirement already up-to-date: setvar in /opt/conda/lib/python3.6/site-packages\n"
     ]
    }
   ],
   "source": [
    "!mkdir -p ~/agave\n",
    "\n",
    "%cd ~/agave\n",
    "\n",
    "!pip3 install --upgrade setvar\n",
    "\n",
    "import re\n",
    "import os\n",
    "import sys\n",
    "from setvar import *\n",
    "from time import sleep\n",
    "\n",
    "# This cell enables inline plotting in the notebook\n",
    "%matplotlib inline\n",
    "\n",
    "import matplotlib\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>Set Variables</h2>\n",
    "\n",
    "The setvar() package gives us a bash-like way of setting environment variables in Python. It's a lot easier to type `PORT=10022` than it is to type `os.environ[\"PORT\"]=\"10022\"`. The setvar package also supports comments and running a command in the background and getting its output."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MACHINE_IP=204.90.47.30\n",
      "MACHINE_NAME=nectar\n",
      "DOMAIN=nectar.org\n",
      "EMAIL=deardooley@gmail.com\n",
      "AGAVE_USERNAME=dooley\n",
      "MACHINE_USERNAME=jovyan\n",
      "PORT=10022\n",
      "DOCKERHUB_NAME=dooley\n",
      "WORK_DIR=/home/jovyan\n",
      "HOME_DIR=/home/jovyan\n",
      "SCRATCH_DIR=/home/jovyan\n",
      "DEPLOYMENT_PATH=agave-deployment\n",
      "AGAVE_JSON_PARSER=jq\n",
      "AGAVE_TENANTS_API_BASEURL=https://agave-auth.solveij.com/tenants\n",
      "APP_NAME=funwave-tvd-nectar-dooley\n",
      "STORAGE_MACHINE=nectar-storage-dooley\n",
      "EXEC_MACHINE=nectar-exec-dooley\n",
      "REQUESTBIN_URL=https://requestbin.agaveapi.co/13s48n91\n"
     ]
    }
   ],
   "source": [
    "setvar(\"\"\"\n",
    "MACHINE_IP=$VM_IPADDRESS\n",
    "MACHINE_NAME=nectar\n",
    "DOMAIN=nectar.org\n",
    "EMAIL=deardooley@gmail.com\n",
    "AGAVE_USERNAME=dooley\n",
    "MACHINE_USERNAME=jovyan\n",
    "PORT=10022\n",
    "\n",
    "DOCKERHUB_NAME=dooley\n",
    "WORK_DIR=/home/${MACHINE_USERNAME}\n",
    "HOME_DIR=/home/${MACHINE_USERNAME}\n",
    "SCRATCH_DIR=/home/${MACHINE_USERNAME}\n",
    "DEPLOYMENT_PATH=agave-deployment\n",
    "AGAVE_JSON_PARSER=jq\n",
    "AGAVE_TENANTS_API_BASEURL=https://agave-auth.solveij.com/tenants\n",
    "APP_NAME=funwave-tvd-${MACHINE_NAME}-${AGAVE_USERNAME}\n",
    "STORAGE_MACHINE=${MACHINE_NAME}-storage-${AGAVE_USERNAME}\n",
    "EXEC_MACHINE=${MACHINE_NAME}-exec-${AGAVE_USERNAME}\n",
    "REQUESTBIN_URL=$(requestbin-create)\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you are running locally using Docker Compose, you will need to pull the ip and port of your reverse tunnel from the sandbox. Uncomment the following command, and enter below"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning: Permanently added 'sandbox,172.20.0.2' (ECDSA) to the list of known hosts.\n",
      "MACHINE_IP=0.tcp.ngrok.io\n",
      "PORT=11397\n"
     ]
    }
   ],
   "source": [
    "if os.environ.get('USE_TUNNEL') == \"False\" :\n",
    "    !ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null sandbox 'curl -s  http://localhost:4040/api/tunnels | jq -r '.tunnels[0].public_url'' > ngrok_url.txt\n",
    "    !cat ngrok_url.txt | sed 's|^tcp://||' | sed -r 's#(.*):(.*)#\\1#' > ngrok_host.txt\n",
    "    !cat ngrok_url.txt | sed 's|^tcp://||' | sed -r 's#(.*):(.*)#\\2#'  > ngrok_port.txt\n",
    "\n",
    "    setvar(\"\"\"\n",
    "    MACHINE_IP=$(cat ngrok_host.txt)\n",
    "    PORT=$(cat ngrok_port.txt)\n",
    "    \"\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>PushBullet</h2>\n",
    "If you wish to use pushbullet to receive status updates, then run the cell below and enter your pushbullet token."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Password or secret: PBTOK\n",
      "········\n",
      "Writing file `PBTOK.txt'\n"
     ]
    }
   ],
   "source": [
    "readpass(\"PBTOK\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>Agave Password</h2>\n",
    "\n",
    "We'll be running several commands that require the Agave password. We'll store it here for convenience."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Password or secret: AGAVE_PASSWD\n",
      "········\n",
      "Writing file `AGAVE_PASSWD.txt'\n"
     ]
    }
   ],
   "source": [
    "readpass(\"AGAVE_PASSWD\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>Setting up Agave</h2>\n",
    "Agave uses machines called \"tenants\" to manage user login and authentication. In this step, we tell Agave we are using the standard proction tenant."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1;0mYou are now configured to interact with the APIs at https://agave-auth.solveij.com/\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "!tenants-init -t sandbox"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>Create the Client</h3>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this next step we delete the client if it exists. Chances are, yours doesn't yet. We put this command here in case, for some reason, you want to re-create your client later on. If you delete the client you intend to create before you create it, no harm is done."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1;0mSuccessfully deleted client funwave-tvd-nectar-dooley\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "!clients-delete -u $AGAVE_USERNAME -p \"$AGAVE_PASSWD\" $APP_NAME"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this step we create the client. Clients provide a way of encapsulating resources connected to a single project. Through the client, you will receive a token which you can use to run most of the Agave commands."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1;0m\u001b[1;0mSuccessfully created client funwave-tvd-nectar-dooley\r\n",
      "key: 91nTFFj_rW2a9AVMH9_la5lwjXga \r\n",
      "secret: isDl5PgdyZfBxXhuzGyUw99FX1ca\u001b[0m\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "!clients-create -p \"$AGAVE_PASSWD\" -S -N $APP_NAME -u $AGAVE_USERNAME"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create the token for your client. You will, from this point on, use this token to run the remainder of the Agave commands in this tutorial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1;0mToken for sandbox:dooley successfully refreshed and cached for 14400 seconds\r\n",
      "ee9db41fb614148518836e5a4b908d\u001b[0m\r\n"
     ]
    }
   ],
   "source": [
    "!auth-tokens-create -u $AGAVE_USERNAME -p \"$AGAVE_PASSWD\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}