python-sdk/tutorials/deploy-local/1.deploy-local.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "fd969e25",
   "metadata": {
    "name": "deploy-models-to-azure"
   },
   "source": [
    "# Deploy machine learning models to Azure\n",
    "\n",
    "description: (preview) deploy your machine learning or deep learning model as a web service in the Azure cloud."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "996082fd",
   "metadata": {
    "name": "connect-to-workspace"
   },
   "source": [
    "## Connect to your workspace"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9a9f14ab",
   "metadata": {
    "name": "connect-to-workspace-code"
   },
   "outputs": [],
   "source": [
    "from azureml.core import Workspace\n",
    "\n",
    "ws = Workspace.from_config()\n",
    "ws"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9057aaf6",
   "metadata": {
    "name": "register-the-model"
   },
   "source": [
    "## Register your model\n",
    "\n",
    "A registered model is a logical container stored in the cloud, containing all files located at `model_path`, which is associated with a version number and other metadata.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cfe3cd0e",
   "metadata": {
    "name": "register-model-from-local-file"
   },
   "source": [
    "## Register a model from a local file\n",
    "\n",
    "You can register a model by providing the local path of the model. You can provide the path of either a folder or a single file on your local machine."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fbf6e9da",
   "metadata": {
    "name": "register-model-from-local-file-code"
   },
   "outputs": [],
   "source": [
    "import urllib.request\n",
    "from azureml.core.model import Model\n",
    "\n",
    "# Download model\n",
    "urllib.request.urlretrieve(\"https://aka.ms/bidaf-9-model\", \"model.onnx\")\n",
    "\n",
    "# Register model\n",
    "model = Model.register(ws, model_name=\"bidaf_onnx\", model_path=\"./model.onnx\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fc200fe9",
   "metadata": {
    "name": "inference-configuration"
   },
   "source": [
    "## Define an inference configuration\n",
    "\n",
    "The inference configuration below specifies that the machine learning deployment will use the file echo_score.py in the ./source_dir directory to process incoming requests and that it will use the Docker image with the Python packages specified in the project_environment environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c9e5efc8",
   "metadata": {
    "name": "inference-configuration-code"
   },
   "outputs": [],
   "source": [
    "from azureml.core import Environment\n",
    "from azureml.core.model import InferenceConfig\n",
    "\n",
    "env = Environment(name=\"project_environment\")\n",
    "dummy_inference_config = InferenceConfig(\n",
    "    environment=env,\n",
    "    source_directory=\"./source_dir\",\n",
    "    entry_script=\"./echo_score.py\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c850e5f4",
   "metadata": {
    "name": "deployment-configuration"
   },
   "source": [
    "## Define a deployment configuration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "87a6fb85",
   "metadata": {
    "name": "deployment-configuration-code"
   },
   "outputs": [],
   "source": [
    "from azureml.core.webservice import LocalWebservice\n",
    "\n",
    "deployment_config = LocalWebservice.deploy_configuration(port=6789)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f5d34207",
   "metadata": {
    "name": "deploy-model"
   },
   "source": [
    "## Deploy your machine learning model\n",
    "\n",
    "A deployment configuration specifies the amount of memory and cores to reserve for your webservice will require in order to run, as well as configuration details of the underlying webservice. For example, a deployment configuration lets you specify that your service needs 2 gigabytes of memory, 2 CPU cores, 1 GPU core, and that you want to enable autoscaling.\n",
    "\n",
    "The options available for a deployment configuration differ depending on the compute target you choose. In a local deployment, all you can specify is which port your webservice will be served on."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9762e02b",
   "metadata": {
    "name": "deploy-model-code",
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "service = Model.deploy(\n",
    "    ws,\n",
    "    \"myservice\",\n",
    "    [model],\n",
    "    dummy_inference_config,\n",
    "    deployment_config,\n",
    "    overwrite=True,\n",
    ")\n",
    "service.wait_for_deployment(show_output=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ede851f8",
   "metadata": {
    "name": "deploy-model-print-logs"
   },
   "outputs": [],
   "source": [
    "print(service.get_logs())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1b9d47b4",
   "metadata": {
    "name": "call-into-model"
   },
   "source": [
    "## Call into your model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "45e92e91",
   "metadata": {
    "name": "call-into-model-code"
   },
   "outputs": [],
   "source": [
    "import requests\n",
    "import json\n",
    "\n",
    "uri = service.scoring_uri\n",
    "requests.get(\"http://localhost:6789\")\n",
    "headers = {\"Content-Type\": \"application/json\"}\n",
    "data = {\n",
    "    \"query\": \"What color is the fox\",\n",
    "    \"context\": \"The quick brown fox jumped over the lazy dog.\",\n",
    "}\n",
    "data = json.dumps(data)\n",
    "response = requests.post(uri, data=data, headers=headers)\n",
    "print(response.json())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "stable-devices",
   "metadata": {},
   "source": [
    "## Define a new inference configuration\n",
    "\n",
    "Now that we've deployed with a dummy entry script, let's try using a real entry script."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34c9e82c",
   "metadata": {
    "name": "notice"
   },
   "source": [
    "Notice the use of the AZUREML_MODEL_DIR environment variable to locate your registered model. Now that you've added some pip packages, you also need to update your inference configuration to add in those additional packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "71b9d8a2",
   "metadata": {
    "name": "add-pip-package"
   },
   "outputs": [],
   "source": [
    "env = Environment(name=\"myenv\")\n",
    "python_packages = [\"nltk\", \"numpy\", \"onnxruntime\"]\n",
    "for package in python_packages:\n",
    "    env.python.conda_dependencies.add_pip_package(package)\n",
    "\n",
    "inference_config = InferenceConfig(\n",
    "    environment=env, source_directory=\"./source_dir\", entry_script=\"./score.py\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "08c0c387",
   "metadata": {
    "name": "re-deploy-model"
   },
   "source": [
    "## Deploy again and call your service"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fd084966",
   "metadata": {
    "name": "re-deploy-model-code"
   },
   "outputs": [],
   "source": [
    "service = Model.deploy(\n",
    "    ws,\n",
    "    \"myservice\",\n",
    "    [model],\n",
    "    inference_config,\n",
    "    deployment_config,\n",
    "    overwrite=True,\n",
    ")\n",
    "service.wait_for_deployment(show_output=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "72f79356",
   "metadata": {
    "name": "re-deploy-model-print-logs"
   },
   "outputs": [],
   "source": [
    "print(service.get_logs())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7861af8f",
   "metadata": {
    "name": "send-post-request"
   },
   "source": [
    "Then ensure you can send a post request to the service:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c0dac0e2",
   "metadata": {
    "name": "send-post-request-code"
   },
   "outputs": [],
   "source": [
    "import requests\n",
    "\n",
    "uri = service.scoring_uri\n",
    "\n",
    "headers = {\"Content-Type\": \"application/json\"}\n",
    "data = {\n",
    "    \"query\": \"What color is the fox\",\n",
    "    \"context\": \"The quick brown fox jumped over the lazy dog.\",\n",
    "}\n",
    "data = json.dumps(data)\n",
    "response = requests.post(uri, data=data, headers=headers)\n",
    "print(response.json())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fc9aab6b",
   "metadata": {
    "name": "deploy-model-on-cloud"
   },
   "source": [
    "## Re-deploy to cloud\n",
    "\n",
    "Once you've confirmed your service works locally and chosen a remote compute target, you are ready to deploy to the cloud.\n",
    "\n",
    "Change your deploy configuration to correspond to the compute target you've chosen, in this case Azure Container Instances."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e0a4ce66",
   "metadata": {
    "name": "deploy-model-on-cloud-code"
   },
   "outputs": [],
   "source": [
    "from azureml.core.webservice import AciWebservice\n",
    "\n",
    "deployment_config = AciWebservice.deploy_configuration(\n",
    "    cpu_cores=0.5, memory_gb=1, auth_enabled=True\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4828de68",
   "metadata": {
    "name": "re-deploy-service"
   },
   "source": [
    "Deploy your service again"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "51029f65",
   "metadata": {
    "name": "re-deploy-service-code"
   },
   "outputs": [],
   "source": [
    "service = Model.deploy(\n",
    "    ws,\n",
    "    \"myservice\",\n",
    "    [model],\n",
    "    inference_config,\n",
    "    deployment_config,\n",
    "    overwrite=True,\n",
    ")\n",
    "service.wait_for_deployment(show_output=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d141a247",
   "metadata": {
    "name": "re-deploy-service-print-logs"
   },
   "outputs": [],
   "source": [
    "print(service.get_logs())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b5448133",
   "metadata": {
    "name": "call-remote-web-service"
   },
   "source": [
    "## Call your remote webservice\n",
    "\n",
    "When you deploy remotely, you may have key authentication enabled. The example below shows how to get your service key with Python in order to make an inference request."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fd48987d",
   "metadata": {
    "name": "call-remote-web-service-code",
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "import requests\n",
    "import json\n",
    "from azureml.core import Webservice\n",
    "\n",
    "service = Webservice(workspace=ws, name=\"myservice\")\n",
    "scoring_uri = service.scoring_uri\n",
    "\n",
    "# If the service is authenticated, set the key or token\n",
    "key, _ = service.get_keys()\n",
    "\n",
    "# Set the appropriate headers\n",
    "headers = {\"Content-Type\": \"application/json\"}\n",
    "headers[\"Authorization\"] = f\"Bearer {key}\"\n",
    "\n",
    "# Make the request and display the response and logs\n",
    "data = {\n",
    "    \"query\": \"What color is the fox\",\n",
    "    \"context\": \"The quick brown fox jumped over the lazy dog.\",\n",
    "}\n",
    "data = json.dumps(data)\n",
    "resp = requests.post(scoring_uri, data=data, headers=headers)\n",
    "print(resp.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "51a6b357",
   "metadata": {
    "name": "call-remote-webservice-print-logs"
   },
   "outputs": [],
   "source": [
    "print(service.get_logs())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4c5ca0cb",
   "metadata": {
    "name": "delete-resource"
   },
   "source": [
    "## Delete resources"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7a5a03fd",
   "metadata": {
    "name": "delete-resource-code"
   },
   "outputs": [],
   "source": [
    "service.delete()\n",
    "model.delete()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f3af1a4",
   "metadata": {
    "name": "next-steps"
   },
   "source": [
    "## Next Steps"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8f530f08",
   "metadata": {
    "name": "next-steps-link"
   },
   "source": [
    "Try reading [our documentation](https://docs.microsoft.com/en-us/azure/machine-learning/how-to-deploy-and-where?tabs=python)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.8 - AzureML",
   "language": "python",
   "name": "python38-azureml"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}