notebooks/official/pipelines/get_started_with_hpt_pipeline_components.ipynb

{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "copyright"
      },
      "outputs": [],
      "source": [
        "# Copyright 2022 Google LLC\n",
        "#\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "#     https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "title:generic,gcp"
      },
      "source": [
        "# Get started with Vertex AI hyperparameter tuning pipeline components\n",
        "\n",
        "<table align=\"left\">\n",
        "    <td style=\"text-align: center\">\n",
        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/get_started_with_hpt_pipeline_components.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Google Colaboratory logo\"><br> Open in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/colab/import/https:%2F%2Fraw.luolix.top%2FGoogleCloudPlatform%2Fvertex-ai-samples%2Fmain%2Fnotebooks%2Fofficial%2Fpipelines%2Fget_started_with_hpt_pipeline_components.ipynb\">\n",
        "      <img width=\"32px\" src=\"https://cloud.google.com/ml-engine/images/colab-enterprise-logo-32px.png\" alt=\"Google Cloud Colab Enterprise logo\"> <br> Open in Colab Enterprise\n",
        "    </a>\n",
        "  </td> \n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/pipelines/get_started_with_hpt_pipeline_components.ipynb\">\n",
        "        <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\"><br> Open in Workbench\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/pipelines/get_started_with_hpt_pipeline_components.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\"><br> View on GitHub\n",
        "    </a>\n",
        "  </td>\n",
        "</table>\n",
        "<br><br><br><br>\n",
        "*Note: This notebook uses KFP 1.x and GCPC 1.x. We recommend using 2.x verions*"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "overview:mlops"
      },
      "source": [
        "## Overview\n",
        "\n",
        "\n",
        "This tutorial demonstrates how to use Vertex AI hyperparameter tuning pipeline components.\n",
        "\n",
        "Learn more about [Vertex AI Pipelines](https://cloud.google.com/vertex-ai/docs/pipelines/introduction) and [Vertex AI hyperparameter tuning](https://cloud.google.com/vertex-ai/docs/training/hyperparameter-tuning-overview)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "objective:mlops,stage3,get_started_hpt_pipeline_components"
      },
      "source": [
        "### Objective\n",
        "\n",
        "In this tutorial, you learn how to use prebuilt Google Cloud Pipeline Components for Vertex AI hyperparameter tuning.\n",
        "\n",
        "This tutorial uses the following Google Cloud ML services:\n",
        "\n",
        "- Google Cloud Pipeline Components\n",
        "- Vertex AI dataset, model and endpoint resources\n",
        "- Vertex AI hyperparameter tuning\n",
        "\n",
        "The steps performed include:\n",
        "\n",
        "- Construct a pipeline for:\n",
        "    - Hyperparameter tune/train a custom model.\n",
        "    - Retrieve the tuned hyperparameter values and metrics to optimize.\n",
        "    - If the metrics exceed a specified threshold.\n",
        "      - Get the location of the model artifacts for the best tuned model.\n",
        "      - Upload the model artifacts to a Vertex AI model resource.\n",
        "- Execute a Vertex AI pipeline."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dataset:horses_or_humans,icn"
      },
      "source": [
        "### Dataset\n",
        "\n",
        "The dataset used for this tutorial is the [Horses or Humans](https://www.tensorflow.org/datasets/catalog/horses_or_humans) from [TensorFlow Datasets](https://www.tensorflow.org/datasets/catalog/overview). The trained model predicts whether an image is a horse or human being."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5e2eba58ad71"
      },
      "source": [
        "### Costs \n",
        "\n",
        "This tutorial uses billable components of Google Cloud:\n",
        "\n",
        "* Vertex AI\n",
        "* Cloud Storage\n",
        "\n",
        "Learn about [Vertex AI\n",
        "pricing](https://cloud.google.com/vertex-ai/pricing) and [Cloud Storage\n",
        "pricing](https://cloud.google.com/storage/pricing), and use the [Pricing\n",
        "Calculator](https://cloud.google.com/products/calculator/)\n",
        "to generate a cost estimate based on your projected usage."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f0316df526f8"
      },
      "source": [
        "## Get started"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a2c2cb2109a0"
      },
      "source": [
        "### Install Vertex AI SDK for Python and other required packages\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "LR9HQnyiMoT5"
      },
      "outputs": [],
      "source": [
        "! pip3 install -U tensorflow -q\n",
        "! pip3 install -U tensorflow-data-validation==1.14.0  -q\n",
        "! pip3 install -U tensorflow-transform==1.14.0  -q\n",
        "! pip3 install -U tensorflow-io==0.36.0  -q\n",
        "! pip3 install --upgrade google-cloud-aiplatform[tensorboard] -q\n",
        "! pip3 install --upgrade 'google-cloud-pipeline-components<2'  -q\n",
        "! pip3 install --upgrade 'kfp<2' -q"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ff555b32bab8"
      },
      "source": [
        "### Restart runtime (Colab only)\n",
        "\n",
        "To use the newly installed packages, you must restart the runtime on Google Colab."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "f09b4dff629a"
      },
      "outputs": [],
      "source": [
        "import sys\n",
        "\n",
        "if \"google.colab\" in sys.modules:\n",
        "\n",
        "    import IPython\n",
        "\n",
        "    app = IPython.Application.instance()\n",
        "    app.kernel.do_shutdown(True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ee775571c2b5"
      },
      "source": [
        "<div class=\"alert alert-block alert-warning\">\n",
        "<b>⚠️ The kernel is going to restart. Wait until it's finished before continuing to the next step. ⚠️</b>\n",
        "</div>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "92e68cfc3a90"
      },
      "source": [
        "### Authenticate your notebook environment (Colab only)\n",
        "\n",
        "Authenticate your environment on Google Colab.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "46604f70e831"
      },
      "outputs": [],
      "source": [
        "import sys\n",
        "\n",
        "if \"google.colab\" in sys.modules:\n",
        "\n",
        "    from google.colab import auth\n",
        "\n",
        "    auth.authenticate_user()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4f872cd812d0"
      },
      "source": [
        "### Set Google Cloud project information and initialize Vertex AI SDK for Python\n",
        "\n",
        "To get started using Vertex AI, you must have an existing Google Cloud project and [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com). Learn more about [setting up a project and a development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "be175254a715"
      },
      "outputs": [],
      "source": [
        "PROJECT_ID = \"[your-project-id]\"  # @param {type:\"string\"}\n",
        "\n",
        "# set the project id\n",
        "! gcloud config set project $PROJECT_ID\n",
        "\n",
        "LOCATION = \"us-central1\"  # @param {type: \"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e5755d1a554f"
      },
      "source": [
        "### Create a Cloud Storage bucket\n",
        "\n",
        "Create a storage bucket to store intermediate artifacts such as datasets."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "d2de92accb67"
      },
      "outputs": [],
      "source": [
        "BUCKET_URI = f\"gs://your-bucket-name-{PROJECT_ID}-unique\"  # @param {type:\"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_bucket"
      },
      "source": [
        "**If your bucket doesn't already exist**: Run the following cell to create your Cloud Storage bucket."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2smRgc53MoT-"
      },
      "outputs": [],
      "source": [
        "! gsutil mb -l $LOCATION $BUCKET_URI"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "set_service_account"
      },
      "source": [
        "#### Service Account\n",
        "\n",
        "**If you don't know your service account**, try to get your service account using `gcloud` command by executing the second cell below."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "EIivrR-3MoT-"
      },
      "outputs": [],
      "source": [
        "SERVICE_ACCOUNT = \"[your-service-account]\"  # @param {type:\"string\"}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "autoset_service_account"
      },
      "outputs": [],
      "source": [
        "import sys\n",
        "\n",
        "IS_COLAB = \"google.colab\" in sys.modules\n",
        "\n",
        "if (\n",
        "    SERVICE_ACCOUNT == \"\"\n",
        "    or SERVICE_ACCOUNT is None\n",
        "    or SERVICE_ACCOUNT == \"[your-service-account]\"\n",
        "):\n",
        "    # Get your service account from gcloud\n",
        "    if not IS_COLAB:\n",
        "        shell_output = !gcloud auth list 2>/dev/null\n",
        "        SERVICE_ACCOUNT = shell_output[2].replace(\"*\", \"\").strip()\n",
        "\n",
        "    if IS_COLAB:\n",
        "        shell_output = ! gcloud projects describe  $PROJECT_ID\n",
        "        project_number = shell_output[-1].split(\":\")[1].strip().replace(\"'\", \"\")\n",
        "        SERVICE_ACCOUNT = f\"{project_number}-compute@developer.gserviceaccount.com\"\n",
        "\n",
        "    print(\"Service Account:\", SERVICE_ACCOUNT)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "set_service_account:pipelines"
      },
      "source": [
        "#### Set service account access for Vertex AI Pipelines\n",
        "\n",
        "Run the following commands to grant your service account access to read and write pipeline artifacts in the bucket that you created in the previous step -- you only need to run these once per service account."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "mtwsjYnIMoT_"
      },
      "outputs": [],
      "source": [
        "! gsutil iam ch serviceAccount:{SERVICE_ACCOUNT}:roles/storage.objectCreator $BUCKET_URI\n",
        "\n",
        "! gsutil iam ch serviceAccount:{SERVICE_ACCOUNT}:roles/storage.objectViewer $BUCKET_URI"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "setup_vars"
      },
      "source": [
        "### Set up variables\n",
        "\n",
        "Next, set up some variables used throughout the tutorial.\n",
        "### Import libraries and define constants"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "import_aip:mbsdk"
      },
      "outputs": [],
      "source": [
        "import google.cloud.aiplatform as aiplatform"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "import_kfp"
      },
      "outputs": [],
      "source": [
        "import json\n",
        "\n",
        "from kfp import dsl\n",
        "from kfp.v2 import compiler\n",
        "from kfp.v2.dsl import component"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "import_tf"
      },
      "source": [
        "#### Import TensorFlow\n",
        "\n",
        "Import the TensorFlow package into your Python environment."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "DtTIHh_KMoUA"
      },
      "outputs": [],
      "source": [
        "import tensorflow as tf"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "init_aip:mbsdk"
      },
      "source": [
        "### Initialize Vertex AI SDK for Python\n",
        "\n",
        "Initialize the Vertex AI SDK for Python for your project and corresponding bucket."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "sUFJPDW0MoUA"
      },
      "outputs": [],
      "source": [
        "aiplatform.init(project=PROJECT_ID, staging_bucket=BUCKET_URI)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "accelerators:training,prediction,ngpu,mbsdk"
      },
      "source": [
        "#### Set hardware accelerators\n",
        "\n",
        "You can set hardware accelerators for training and prediction.\n",
        "\n",
        "Set the variables `TRAIN_GPU/TRAIN_NGPU` and `DEPLOY_GPU/DEPLOY_NGPU` to use a container image supporting a GPU and the number of GPUs allocated to the virtual machine (VM) instance. For example, to use a GPU container image with 4 Nvidia Telsa T4 GPUs allocated to each VM, you would specify:\n",
        "\n",
        "    (aiplatform.gapic.AcceleratorType.NVIDIA_TESLA_T4, 4)\n",
        "\n",
        "\n",
        "Otherwise specify `(None, None)` to use a container image to run on a CPU.\n",
        "\n",
        "Learn more about [hardware accelerator support for your region](https://cloud.google.com/vertex-ai/docs/general/locations#accelerators).\n",
        "\n",
        "*Note*: TF releases before 2.3 for GPU support fails to load the custom model in this tutorial. It's a known issue and fixed in TF 2.3. This is caused by static graph ops that are generated in the serving function. If you encounter this issue on your own custom models, use a container image for TF 2.3 with GPU support."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "A6dzi4cXMoUA"
      },
      "outputs": [],
      "source": [
        "TRAIN_GPU, TRAIN_NGPU = (None, None)\n",
        "DEPLOY_GPU, DEPLOY_NGPU = (None, None)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "container:prediction"
      },
      "source": [
        "#### Set pre-built containers\n",
        "\n",
        "Set the pre-built Docker container image for prediction.\n",
        "\n",
        "- Set the variable `TF` to the TensorFlow version of the container image. For example, `2-1` would be version 2.1, and `1-15` would be version 1.15. The following list shows some of the pre-built images available:\n",
        "\n",
        "\n",
        "For the latest list, see [Pre-built containers for prediction](https://cloud.google.com/ai-platform-unified/docs/predictions/pre-built-containers)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "gxai072KMoUB"
      },
      "outputs": [],
      "source": [
        "TF = \"2.13\".replace(\".\", \"-\")\n",
        "\n",
        "if TF[0] == \"2\":\n",
        "    if DEPLOY_GPU:\n",
        "        DEPLOY_VERSION = \"tf2-gpu.{}\".format(TF)\n",
        "    else:\n",
        "        DEPLOY_VERSION = \"tf2-cpu.{}\".format(TF)\n",
        "else:\n",
        "    if DEPLOY_GPU:\n",
        "        DEPLOY_VERSION = \"tf-gpu.{}\".format(TF)\n",
        "    else:\n",
        "        DEPLOY_VERSION = \"tf-cpu.{}\".format(TF)\n",
        "\n",
        "DEPLOY_IMAGE = \"{}-docker.pkg.dev/vertex-ai/prediction/{}:latest\".format(\n",
        "    LOCATION.split(\"-\")[0], DEPLOY_VERSION\n",
        ")\n",
        "\n",
        "print(\"Deployment:\", DEPLOY_IMAGE, DEPLOY_GPU)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "machine:training,prediction"
      },
      "source": [
        "#### Set machine type\n",
        "\n",
        "Next, set the machine type to use for training and prediction.\n",
        "\n",
        "- Set the variables `TRAIN_COMPUTE` and `DEPLOY_COMPUTE` to configure  the compute resources for the VMs you use for for training and prediction.\n",
        " - `machine type`\n",
        "     - `n1-standard`: 3.75GB of memory per vCPU.\n",
        "     - `n1-highmem`: 6.5GB of memory per vCPU\n",
        "     - `n1-highcpu`: 0.9 GB of memory per vCPU\n",
        " - `vCPUs`: number of \\[2, 4, 8, 16, 32, 64, 96 \\]\n",
        "\n",
        "*Note: The following is not supported for training:*\n",
        "\n",
        " - `standard`: 2 vCPUs\n",
        " - `highcpu`: 2, 4 and 8 vCPUs\n",
        "\n",
        "*Note: You may also use n2 and e2 machine types for training and deployment, but they don't support GPUs*."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "LEYjL1ojMoUB"
      },
      "outputs": [],
      "source": [
        "TRAIN_COMPUTE = \"n1-standard-4\"\n",
        "print(\"Train machine type\", TRAIN_COMPUTE)\n",
        "\n",
        "DEPLOY_COMPUTE = \"n1-standard-4\"\n",
        "print(\"Deploy machine type\", DEPLOY_COMPUTE)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "examine_training_package"
      },
      "source": [
        "### Examine the tuning package\n",
        "\n",
        "#### Package layout\n",
        "\n",
        "Before you start the tuning, you look at how a Python package is assembled for a custom tuning job. When unarchived, the package contains the following directory/file layout.\n",
        "\n",
        "- PKG-INFO\n",
        "- README.md\n",
        "- setup.cfg\n",
        "- setup.py\n",
        "- trainer\n",
        "  - \\_\\_init\\_\\_.py\n",
        "  - task.py\n",
        "\n",
        "The files `setup.cfg` and `setup.py` are the instructions for installing the package into the operating environment of the Docker image.\n",
        "\n",
        "The file `trainer/task.py` is the Python script for executing the custom tuning job. \"Note*: When trainer/task.py is referred to in the worker pool specification, the directory slash is replaced with a dot and the file suffix (`.py`) is dropped (`trainer.task`).\n",
        "\n",
        "#### Package Assembly\n",
        "\n",
        "In the following cells, you assemble the training package."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "UIoZdxPqMoUC"
      },
      "outputs": [],
      "source": [
        "# Make folder for Python tuning script\n",
        "! rm -rf custom\n",
        "! mkdir custom\n",
        "\n",
        "# Add package information\n",
        "! touch custom/README.md\n",
        "\n",
        "setup_cfg = \"[egg_info]\\n\\ntag_build =\\n\\ntag_date = 0\"\n",
        "! echo \"$setup_cfg\" > custom/setup.cfg\n",
        "\n",
        "setup_py = \"import setuptools\\n\\nsetuptools.setup(\\n\\n    install_requires=[\\n\\n        'tensorflow==2.15.1',\\n\\n        'tensorflow_datasets==4.9.6',\\n\\n    ],\\n\\n    packages=setuptools.find_packages())\"\n",
        "! echo \"$setup_py\" > custom/setup.py\n",
        "\n",
        "pkg_info = \"Metadata-Version: 1.0\\n\\nName: Horses or Humans image classification\\n\\nVersion: 0.0.0\\n\\nSummary: Demostration tuning script\\n\\nHome-page: www.google.com\\n\\nAuthor: Google\\n\\nAuthor-email: aferlitsch@google.com\\n\\nLicense: Public\\n\\nDescription: Demo\\n\\nPlatform: Vertex\"\n",
        "! echo \"$pkg_info\" > custom/PKG-INFO\n",
        "\n",
        "# Make the training subfolder\n",
        "! mkdir custom/trainer\n",
        "! touch custom/trainer/__init__.py"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "taskpy_contents:dataset,horses_or_humans"
      },
      "source": [
        "### Create the task script for the Python training package\n",
        "\n",
        "\n",
        "Next, you create the `task.py` script for driving the training package. Some noteable steps include:\n",
        "\n",
        "- Command-line arguments:\n",
        "    - `model-dir`: The location to save the trained model. When using Vertex AI custom training, the location is specified in the environment variable: `AIP_MODEL_DIR`,\n",
        "    - `epochs`: The number of epochs to train for.\n",
        "    - `learning_rate`: Hyperparameter for learning rate.\n",
        "    - `batch_size`: Hyperparameter for batch size.\n",
        "\n",
        "\n",
        "- Data preprocessing (`get_data()`)\n",
        "    - Loads and preprocesses the dataset as a `tf.data.Dataset` generator.\n",
        "\n",
        "\n",
        "- Model architecture (`get_model()`):\n",
        "    - Builds the corresponding model architecture.\n",
        "\n",
        "\n",
        "- Training (`train_model()`):\n",
        "    - Trains the model\n",
        "\n",
        "\n",
        "- Model artifact saving\n",
        "    - Saves the model artifacts where the Cloud Storage location is specified."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "uk4MAGErMoUC"
      },
      "outputs": [],
      "source": [
        "%%writefile custom/trainer/task.py\n",
        "import os\n",
        "os.system('pip install cloudml-hypertune')  # alternaterly, this can be added to the Dockerfile\n",
        "\n",
        "import tensorflow as tf\n",
        "import tensorflow_datasets as tfds\n",
        "import argparse\n",
        "import hypertune\n",
        "\n",
        "\n",
        "def get_args():\n",
        "  '''Parses args. Must include all hyperparameters you want to tune.'''\n",
        "\n",
        "  parser = argparse.ArgumentParser()\n",
        "  parser.add_argument(\n",
        "      '--epochs',\n",
        "      required=True,\n",
        "      type=int,\n",
        "      help='number of epochs')\n",
        "  parser.add_argument(\n",
        "      '--learning_rate',\n",
        "      required=True,\n",
        "      type=float,\n",
        "      help='learning rate')\n",
        "  parser.add_argument(\n",
        "      '--momentum',\n",
        "      required=False,\n",
        "      type=float,\n",
        "      default=0.5,\n",
        "      help='SGD momentum value')\n",
        "  parser.add_argument(\n",
        "      '--batch_size',\n",
        "      required=True,\n",
        "      type=int,\n",
        "      help='the batch size')\n",
        "  parser.add_argument(\n",
        "      '--model-dir',\n",
        "      dest='model_dir',\n",
        "      default=os.getenv('AIP_MODEL_DIR'),\n",
        "      type=str, help='Model dir.')\n",
        "  args = parser.parse_args()\n",
        "  return args\n",
        "\n",
        "def preprocess_data(image, label):\n",
        "  '''Resizes and scales images.'''\n",
        "\n",
        "  image = tf.image.resize(image, (150,150))\n",
        "  return tf.cast(image, tf.float32) / 255., label\n",
        "\n",
        "\n",
        "def get_data():\n",
        "  '''Loads Horses Or Humans dataset and preprocesses data.'''\n",
        "\n",
        "  data, info = tfds.load(name='horses_or_humans', as_supervised=True, with_info=True)\n",
        "\n",
        "  # Create train dataset\n",
        "  train_data = data['train'].map(preprocess_data)\n",
        "  train_data  = train_data.shuffle(1000)\n",
        "  train_data  = train_data.batch(64)\n",
        "\n",
        "  # Create validation dataset\n",
        "  validation_data = data['test'].map(preprocess_data)\n",
        "  validation_data  = validation_data.batch(64)\n",
        "\n",
        "  return train_data, validation_data\n",
        "\n",
        "\n",
        "def get_model(learning_rate, momentum):\n",
        "  '''Defines and complies model.'''\n",
        "\n",
        "  inputs = tf.keras.Input(shape=(150, 150, 3))\n",
        "  x = tf.keras.layers.Conv2D(16, (3, 3), activation='relu')(inputs)\n",
        "  x = tf.keras.layers.MaxPooling2D((2, 2))(x)\n",
        "  x = tf.keras.layers.Conv2D(32, (3, 3), activation='relu')(x)\n",
        "  x = tf.keras.layers.MaxPooling2D((2, 2))(x)\n",
        "  x = tf.keras.layers.Conv2D(64, (3, 3), activation='relu')(x)\n",
        "  x = tf.keras.layers.MaxPooling2D((2, 2))(x)\n",
        "  x = tf.keras.layers.Flatten()(x)\n",
        "  outputs = tf.keras.layers.Dense(1, activation='sigmoid')(x)\n",
        "  model = tf.keras.Model(inputs, outputs)\n",
        "  model.compile(\n",
        "      loss='binary_crossentropy',\n",
        "      optimizer=tf.keras.optimizers.SGD(learning_rate=learning_rate, momentum=momentum),\n",
        "      metrics=['accuracy'])\n",
        "  return model\n",
        "\n",
        "def train_model(model, train_data, validation_data, epochs, batch_size):\n",
        "\n",
        "  history = model.fit(train_data, epochs=epochs, batch_size=batch_size, validation_data=validation_data)\n",
        "\n",
        "  # DEFINE METRIC\n",
        "  hp_metric = history.history['val_accuracy'][-1]\n",
        "\n",
        "  hpt = hypertune.HyperTune()\n",
        "  hpt.report_hyperparameter_tuning_metric(\n",
        "      hyperparameter_metric_tag='accuracy',\n",
        "      metric_value=hp_metric,\n",
        "      global_step=epochs\n",
        "  )\n",
        "\n",
        "  return model\n",
        "\n",
        "\n",
        "def main():\n",
        "  args = get_args()\n",
        "  train_data, validation_data = get_data()\n",
        "\n",
        "  model = get_model(args.learning_rate, args.momentum)\n",
        "\n",
        "  model = train_model(model, train_data, validation_data, args.epochs, args.batch_size)\n",
        "\n",
        "  model.save(args.model_dir)\n",
        "\n",
        "\n",
        "if __name__ == \"__main__\":\n",
        "    main()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tarball_training_script"
      },
      "source": [
        "#### Store tuning script on your Cloud Storage bucket\n",
        "\n",
        "Next, package the tuning folder into a compressed tar ball, and then store it in your Cloud Storage bucket."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bHcLQLGkMoUD"
      },
      "outputs": [],
      "source": [
        "! rm -f custom.tar custom.tar.gz\n",
        "! tar cvf custom.tar custom\n",
        "! gzip custom.tar\n",
        "! gsutil cp custom.tar.gz $BUCKET_URI/trainer_horses_or_humans.tar.gz"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_docker_container:training"
      },
      "source": [
        "### Create a Docker file\n",
        "\n",
        "To use your own custom training container, build a Docker file and embed it into the container with your training scripts."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "write_docker_file:training,tf-dlvm"
      },
      "source": [
        "#### Write the Docker file contents\n",
        "\n",
        "Your first step in containerizing your code is to create a Docker file. In your Docker you’ll include all the commands needed to run your container image. It’ll install all the libraries you’re using and set up the entry point for your training code.\n",
        "\n",
        "1. Install a pre-defined container image from TensorFlow repository for deep learning images.\n",
        "2. Copies in the Python training code, to be shown subsequently.\n",
        "3. Sets the entry into the Python training script as `trainer/task.py`. Note, the `.py` is dropped in the ENTRYPOINT command, as it's implied."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "MaMK4AoBMoUE"
      },
      "outputs": [],
      "source": [
        "%%writefile custom/Dockerfile\n",
        "\n",
        "FROM gcr.io/deeplearning-platform-release/tf2-cpu.2-3\n",
        "WORKDIR /root\n",
        "\n",
        "WORKDIR /\n",
        "\n",
        "# Copies the trainer code to the docker image.\n",
        "COPY trainer /trainer\n",
        "\n",
        "# Sets up the entry point to invoke the trainer.\n",
        "ENTRYPOINT [\"python\", \"-m\", \"trainer.task\"]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "228fc5ade2b6"
      },
      "source": [
        "### Enable Artifact Registry API\n",
        "\n",
        "First, you must enable the Artifact Registry API service for your project.\n",
        "\n",
        "Learn more about [Enabling service](https://cloud.google.com/artifact-registry/docs/enable-service).\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "05f2a52093dd"
      },
      "outputs": [],
      "source": [
        "! gcloud services enable artifactregistry.googleapis.com"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "740aec59bf63"
      },
      "source": [
        "### Build the container\n",
        "\n",
        "You store the Docker image in Artifact Registry. First, create a Docker repository in Artifact Registry"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "e498d34129b8"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "\n",
        "REPO_NAME='horses-app'\n",
        "\n",
        "if os.getenv(\"IS_TESTING\"):\n",
        "    ! sudo apt-get update --yes && sudo apt-get --only-upgrade --yes install google-cloud-sdk-cloud-run-proxy google-cloud-sdk-harbourbridge google-cloud-sdk-cbt google-cloud-sdk-gke-gcloud-auth-plugin google-cloud-sdk-kpt google-cloud-sdk-local-extract google-cloud-sdk-minikube google-cloud-sdk-app-engine-java google-cloud-sdk-app-engine-go google-cloud-sdk-app-engine-python google-cloud-sdk-spanner-emulator google-cloud-sdk-bigtable-emulator google-cloud-sdk-nomos google-cloud-sdk-package-go-module google-cloud-sdk-firestore-emulator kubectl google-cloud-sdk-datastore-emulator google-cloud-sdk-app-engine-python-extras google-cloud-sdk-cloud-build-local google-cloud-sdk-kubectl-oidc google-cloud-sdk-anthos-auth google-cloud-sdk-app-engine-grpc google-cloud-sdk-pubsub-emulator google-cloud-sdk-datalab google-cloud-sdk-skaffold google-cloud-sdk google-cloud-sdk-terraform-tools google-cloud-sdk-config-connector\n",
        "    ! gcloud components update --quiet\n",
        "\n",
        "!gcloud artifacts repositories create $REPO_NAME --repository-format=docker \\\n",
        "--location=$LOCATION --description=\"Docker repository\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a6c368666793"
      },
      "source": [
        "### Configure authentication to your private repo\n",
        "\n",
        "Before you push or pull container images, configure Docker to use the `gcloud` command-line tool to authenticate requests to Artifact Registry for your region."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7a6f43afca3c"
      },
      "outputs": [],
      "source": [
        "! gcloud auth configure-docker {LOCATION}-docker.pkg.dev --quiet"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "name_container:training"
      },
      "source": [
        "Define a variable with the URI of your Docker image in Artifact Registry:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9qVJGXT2MoUE"
      },
      "outputs": [],
      "source": [
        "TRAIN_IMAGE = (\n",
        "    f\"{LOCATION}-docker.pkg.dev/{PROJECT_ID}/{REPO_NAME}/horse_human_hptune:latest\"\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a6ad28220418"
      },
      "source": [
        "Then, build the container and push it to an Artifact Registry repository by running the following:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "35b3f0bf1318"
      },
      "outputs": [],
      "source": [
        "cd custom"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ae8d05824499"
      },
      "source": [
        "Push the image to the created artifact repository using Cloud-Build.\n",
        "\n",
        "**Note**: The following command automatically uses the Dockerfile from the directory it's being run from."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "71cc6522ae97"
      },
      "outputs": [],
      "source": [
        "! gcloud builds submit --region={LOCATION} --tag={TRAIN_IMAGE}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3b1d7d702f20"
      },
      "outputs": [],
      "source": [
        "cd ../"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_hpt_pipeline:icn"
      },
      "source": [
        "### Construct hyperparameter tuning pipeline\n",
        "\n",
        "Next, construct the pipeline with the following tasks:\n",
        "\n",
        "- Create/Execute a hyperparameter tuning job\n",
        "- Get all trial results.\n",
        "- Get the best trial results.\n",
        "- Determine if the best trial results exceed a threshold:\n",
        "    - Retrieve the hyperparameter values.\n",
        "    - Determine Cloud Storage location of the best model.\n",
        "    - Upload the best model as a Vertex AI Model resource."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Epzlh8M-MoUF"
      },
      "outputs": [],
      "source": [
        "PIPELINE_ROOT = \"{}/pipeline_root/custom_icn_tuning\".format(BUCKET_URI)\n",
        "\n",
        "\n",
        "@component(packages_to_install=[\"google-cloud-aiplatform\"])\n",
        "def model_dir(base_output_directory: str, best_trial: str) -> str:\n",
        "    from google.cloud.aiplatform_v1.types import study\n",
        "\n",
        "    trial_proto = study.Trial.from_json(best_trial)\n",
        "    model_id = trial_proto.id\n",
        "    return f\"{base_output_directory}/{model_id}/model\"\n",
        "\n",
        "\n",
        "@dsl.pipeline(\n",
        "    name=\"hp-tuning\", description=\"Custom image classification hyperparameter tuning\"\n",
        ")\n",
        "def pipeline(\n",
        "    display_name: str,\n",
        "    worker_pool_specs: list,\n",
        "    study_spec_metrics: list,\n",
        "    study_spec_parameters: list,\n",
        "    threshold: float,\n",
        "    deploy_image: str,\n",
        "    max_trial_count: int = 5,\n",
        "    parallel_trial_count: int = 1,\n",
        "    base_output_directory: str = PIPELINE_ROOT,\n",
        "    labels: dict = {},\n",
        "    project: str = PROJECT_ID,\n",
        "    region: str = LOCATION,\n",
        "):\n",
        "    from google_cloud_pipeline_components.experimental import \\\n",
        "        hyperparameter_tuning_job\n",
        "    from google_cloud_pipeline_components.types import artifact_types\n",
        "    from google_cloud_pipeline_components.v1.hyperparameter_tuning_job import \\\n",
        "        HyperparameterTuningJobRunOp\n",
        "    from google_cloud_pipeline_components.v1.model import ModelUploadOp\n",
        "    from kfp.v2.components import importer_node\n",
        "\n",
        "    tuning_op = HyperparameterTuningJobRunOp(\n",
        "        display_name=display_name,\n",
        "        project=project,\n",
        "        location=region,\n",
        "        worker_pool_specs=worker_pool_specs,\n",
        "        study_spec_metrics=study_spec_metrics,\n",
        "        study_spec_parameters=study_spec_parameters,\n",
        "        max_trial_count=max_trial_count,\n",
        "        parallel_trial_count=parallel_trial_count,\n",
        "        base_output_directory=base_output_directory,\n",
        "    )\n",
        "\n",
        "    trials_op = hyperparameter_tuning_job.GetTrialsOp(\n",
        "        gcp_resources=tuning_op.outputs[\"gcp_resources\"]\n",
        "    )\n",
        "\n",
        "    best_trial_op = hyperparameter_tuning_job.GetBestTrialOp(\n",
        "        trials=trials_op.output, study_spec_metrics=study_spec_metrics\n",
        "    )\n",
        "\n",
        "    threshold_op = hyperparameter_tuning_job.IsMetricBeyondThresholdOp(\n",
        "        trial=best_trial_op.output,\n",
        "        study_spec_metrics=study_spec_metrics,\n",
        "        threshold=threshold,\n",
        "    )\n",
        "\n",
        "    with dsl.Condition(\n",
        "        threshold_op.output == \"true\",\n",
        "        name=\"deploy_decision\",\n",
        "    ):\n",
        "        _ = hyperparameter_tuning_job.GetHyperparametersOp(trial=best_trial_op.output)\n",
        "\n",
        "        model_dir_op = model_dir(base_output_directory, best_trial_op.output)\n",
        "\n",
        "        import_unmanaged_model_op = importer_node.importer(\n",
        "            artifact_uri=model_dir_op.output,\n",
        "            artifact_class=artifact_types.UnmanagedContainerModel,\n",
        "            metadata={\n",
        "                \"containerSpec\": {\n",
        "                    \"imageUri\": DEPLOY_IMAGE,\n",
        "                },\n",
        "            },\n",
        "        ).after(model_dir_op)\n",
        "\n",
        "        _ = ModelUploadOp(\n",
        "            project=project,\n",
        "            display_name=display_name,\n",
        "            unmanaged_container_model=import_unmanaged_model_op.outputs[\"artifact\"],\n",
        "        ).after(import_unmanaged_model_op)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_hpt_specs"
      },
      "source": [
        "### Create hyperparameter tuning specifications\n",
        "\n",
        "Next, you construct the worker pool specification, and the study's metric and parameter specifications, as follows:\n",
        "\n",
        "**Worker pool specification**\n",
        "\n",
        "This specification describes the machine and container requirements, and scaling for executing the hyperparameter study. Since the training module is embedded in the docker image, you use the `args` field to specify any command-lime arguments, which are not part of the study, to the training module. In this example, you pass the number of epochs.\n",
        "\n",
        "**Parameter specification**\n",
        "\n",
        "This specification describes the hyperparameters to tune, and the range of values to tune then for. For each study, the trial values for these parameters are passed as command-line arguments to the training module, as `--<parameter_name>=<trial_value>`.\n",
        "\n",
        "**Metric specification**\n",
        "\n",
        "This specification describes the metric or metrics to be evaluated in the study, and whether or not to minize or maximize the metric."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "aEm7RuwMMoUG"
      },
      "outputs": [],
      "source": [
        "from google_cloud_pipeline_components.experimental import \\\n",
        "    hyperparameter_tuning_job\n",
        "\n",
        "if TRAIN_GPU:\n",
        "    gpu = TRAIN_GPU.name\n",
        "    accelerator_count = 1\n",
        "\n",
        "else:\n",
        "    gpu = \"ACCELERATOR_TYPE_UNSPECIFIED\"\n",
        "    accelerator_count = 0\n",
        "\n",
        "CMDARGS = [\n",
        "    \"--epochs=10\",\n",
        "]\n",
        "\n",
        "# The spec of the worker pools including machine type and Docker image\n",
        "worker_pool_specs = [\n",
        "    {\n",
        "        \"machine_spec\": {\n",
        "            \"machine_type\": TRAIN_COMPUTE,\n",
        "            \"accelerator_type\": gpu,\n",
        "            \"accelerator_count\": accelerator_count,\n",
        "        },\n",
        "        \"replica_count\": 1,\n",
        "        \"container_spec\": {\"image_uri\": TRAIN_IMAGE, \"args\": CMDARGS},\n",
        "    }\n",
        "]\n",
        "\n",
        "# List serialized from the dictionary representing metrics to optimize.\n",
        "# The dictionary key is the metric_id, which is reported by your training job,\n",
        "# and the dictionary value is the optimization goal of the metric.\n",
        "metric_spec = hyperparameter_tuning_job.serialize_metrics({\"accuracy\": \"maximize\"})\n",
        "\n",
        "# List serialized from the parameter dictionary. The dictionary\n",
        "# represents parameters to optimize. The dictionary key is the parameter_id,\n",
        "# which is passed into your training job as a command line key word argument, and the\n",
        "# dictionary value is the parameter specification of the metric.\n",
        "parameter_spec = hyperparameter_tuning_job.serialize_parameters(\n",
        "    {\n",
        "        \"learning_rate\": aiplatform.hyperparameter_tuning.DoubleParameterSpec(\n",
        "            min=0.001, max=1, scale=\"log\"\n",
        "        ),\n",
        "        \"batch_size\": aiplatform.hyperparameter_tuning.DiscreteParameterSpec(\n",
        "            values=[16, 32, 64], scale=None\n",
        "        ),\n",
        "    }\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "run_pipeline:hpt"
      },
      "source": [
        "### Compile and execute hyperparameter tuning pipeline\n",
        "\n",
        "Next, you compile the pipeline and then exeute it. The pipeline takes the following parameters, which are passed as the dictionary `parameter_values`:\n",
        "\n",
        "- `display_name`: A human readable name for the pipeline job.\n",
        "- `worker_pool_specs`: The the machine and container, and auto-scaling requirements, as well as command line arguments.\n",
        "- `study_spec_metrics`: The metrics to optimize in the study trials.\n",
        "- `study_spec_parameters`: The parameters to tune."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "42YNp9Y9MoUG"
      },
      "outputs": [],
      "source": [
        "compiler.Compiler().compile(\n",
        "    pipeline_func=pipeline, package_path=\"hp_tune_pipeline_job.json\"\n",
        ")\n",
        "\n",
        "pipeline = aiplatform.PipelineJob(\n",
        "    display_name=\"hp_tuning\",\n",
        "    template_path=\"hp_tune_pipeline_job.json\",\n",
        "    pipeline_root=PIPELINE_ROOT,\n",
        "    parameter_values={\n",
        "        \"display_name\": \"hp_tuning\",\n",
        "        \"worker_pool_specs\": worker_pool_specs,\n",
        "        \"study_spec_metrics\": metric_spec,\n",
        "        \"study_spec_parameters\": parameter_spec,\n",
        "        \"threshold\": 0.7,\n",
        "        \"deploy_image\": DEPLOY_IMAGE,\n",
        "    },\n",
        "    enable_caching=False,\n",
        ")\n",
        "\n",
        "pipeline.run()\n",
        "\n",
        "! rm -rf hp_tune_pipeline_job.json custom custom.tar.gz"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view_pipleline_results:hpt,horses_or_humans"
      },
      "source": [
        "### View the data pipeline execution results"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4X3jrdX1MoUH"
      },
      "outputs": [],
      "source": [
        "PROJECT_NUMBER = pipeline.gca_resource.name.split(\"/\")[1]\n",
        "print(PROJECT_NUMBER)\n",
        "\n",
        "\n",
        "def print_pipeline_output(job, output_task_name):\n",
        "    JOB_ID = job.name\n",
        "    print(JOB_ID)\n",
        "    for _ in range(len(job.gca_resource.job_detail.task_details)):\n",
        "        TASK_ID = job.gca_resource.job_detail.task_details[_].task_id\n",
        "        EXECUTE_OUTPUT = (\n",
        "            PIPELINE_ROOT\n",
        "            + \"/\"\n",
        "            + PROJECT_NUMBER\n",
        "            + \"/\"\n",
        "            + JOB_ID\n",
        "            + \"/\"\n",
        "            + output_task_name\n",
        "            + \"_\"\n",
        "            + str(TASK_ID)\n",
        "            + \"/executor_output.json\"\n",
        "        )\n",
        "        GCP_RESOURCES = (\n",
        "            PIPELINE_ROOT\n",
        "            + \"/\"\n",
        "            + PROJECT_NUMBER\n",
        "            + \"/\"\n",
        "            + JOB_ID\n",
        "            + \"/\"\n",
        "            + output_task_name\n",
        "            + \"_\"\n",
        "            + str(TASK_ID)\n",
        "            + \"/gcp_resources\"\n",
        "        )\n",
        "        EVAL_METRICS = (\n",
        "            PIPELINE_ROOT\n",
        "            + \"/\"\n",
        "            + PROJECT_NUMBER\n",
        "            + \"/\"\n",
        "            + JOB_ID\n",
        "            + \"/\"\n",
        "            + output_task_name\n",
        "            + \"_\"\n",
        "            + str(TASK_ID)\n",
        "            + \"/evaluation_metrics\"\n",
        "        )\n",
        "        if tf.io.gfile.exists(EXECUTE_OUTPUT):\n",
        "            ! gsutil cat $EXECUTE_OUTPUT\n",
        "            return EXECUTE_OUTPUT\n",
        "        elif tf.io.gfile.exists(GCP_RESOURCES):\n",
        "            ! gsutil cat $GCP_RESOURCES\n",
        "            return GCP_RESOURCES\n",
        "        elif tf.io.gfile.exists(EVAL_METRICS):\n",
        "            ! gsutil cat $EVAL_METRICS\n",
        "            return EVAL_METRICS\n",
        "\n",
        "    return None\n",
        "\n",
        "\n",
        "print(\"hyperparameter-tuning-job\")\n",
        "artifacts = print_pipeline_output(pipeline, \"hyperparameter-tuning-job\")\n",
        "print(\"\\n\\n\")\n",
        "print(\"gettrialsop\")\n",
        "artifacts = print_pipeline_output(pipeline, \"gettrialsop\")\n",
        "print(\"\\n\\n\")\n",
        "print(\"getbesttrialop\")\n",
        "artifacts = print_pipeline_output(pipeline, \"getbesttrialop\")\n",
        "print(\"\\n\\n\")\n",
        "output = !gsutil cat $artifacts\n",
        "output = json.loads(output[0])\n",
        "best_trial = json.loads(output[\"parameters\"][\"Output\"][\"stringValue\"])\n",
        "model_id = best_trial[\"id\"]\n",
        "print(\"BEST MODEL\", model_id)\n",
        "parameters = best_trial[\"parameters\"]\n",
        "batch_size = parameters[0][\"value\"]\n",
        "print(\"BATCH SIZE\", batch_size)\n",
        "learning_rate = parameters[1][\"value\"]\n",
        "print(\"LR\", learning_rate)\n",
        "MODEL_DIR = f\"{PIPELINE_ROOT}/{model_id}/model\"\n",
        "\n",
        "print(\"ismetricbeyondthresholdop\")\n",
        "artifacts = print_pipeline_output(pipeline, \"ismetricbeyondthresholdop\")\n",
        "print(\"\\n\\n\")\n",
        "print(\"deploy-decision\")\n",
        "artifacts = print_pipeline_output(pipeline, \"deploy-decision\")\n",
        "print(\"\\n\\n\")\n",
        "print(\"model-dir\")\n",
        "artifacts = print_pipeline_output(pipeline, \"model-dir\")\n",
        "print(\"\\n\\n\")\n",
        "print(\"model-upload\")\n",
        "artifacts = print_pipeline_output(pipeline, \"model-upload\")\n",
        "print(\"\\n\\n\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "delete_pipeline"
      },
      "source": [
        "### Delete a pipeline job\n",
        "\n",
        "After a pipeline job is completed, you can delete the pipeline job with the `delete()` method.  Prior to completion, a pipeline job can be canceled with the `cancel()` method."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "hS53o3FcMoUH"
      },
      "outputs": [],
      "source": [
        "pipeline.delete()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cleanup:mbsdk"
      },
      "source": [
        "# Cleaning up\n",
        "\n",
        "To clean up all Google Cloud resources used in this project, you can [delete the Google Cloud\n",
        "project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial.\n",
        "\n",
        "Otherwise, you can delete the individual resources you created in this tutorial:\n",
        "\n",
        "- Cloud Storage Bucket"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "laAQFM4aoBm3"
      },
      "outputs": [],
      "source": [
        "delete_bucket = False\n",
        "\n",
        "if delete_bucket:\n",
        "    ! gsutil rm -r $BUCKET_URI\n",
        "\n",
        "# Delete artifact registry repo\n",
        "! gcloud artifacts repositories delete $REPO_NAME --location $LOCATION --quiet"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [],
      "name": "get_started_with_hpt_pipeline_components.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}