Simplified build with Azure CLI; various bugfixes #121
Conversation
algattik
commented
Nov 28, 2019
•
algattik
commented
- Use Azure CLI tasks to remove need for variables SP_APP_ID, SP_APP_SECRET, SUBSCRIPTION_ID, TENANT_ID
- Updated getting_started.md to point out variable path to artifact (closes Deployment to QA failed #117)
- Made random hyperparameter (ridge regression alpha) a pipeline parameter, setting its value in the DevOps pipeline (closes Hyperparameter a pipeline parameter, not random value #107)
- Changed unit test to test actual training code (partially solves Include realistic test scenarios #74)
- Fixed: attach_compute should set exit code after exception (closes attach_compute should set exit code after exception #113)
- Fixed scoring endpoint HTTP behavior (closes Fix scoring endpoint HTTP behavior #110)
- Fixed PipelineParameters format in call to Azure ML Extension (closes PipelineParameters format in call to Azure ML Extension #118)
- Fixed Model build environment for Azure Web App for containers (closes Model build fails for Azure Web App for containers #119)
bump pip requirements versions (microsoft#104)
* Use Azure CLI tasks to remove need for variables SP_APP_ID, SP_APP_SECRET, SUBSCRIPTION_ID, TENANT_ID * Updated gettin_started.md to point out variable path to artifact (microsoft#117) * Made random hyperparameter (ridge regression alpha) a pipeline parameter, setting its value in the DevOps pipeline (microsoft#107) * Changed unit test to test actual training code (partially solves microsoft#74) * Fixed: attach_compute should set exit code after exception (microsoft#113) * Fixed scoring endpoint HTTP behavior (microsoft#110) * Fixed PipelineParameters format in call to Azure ML Extension (microsoft#118) * Fixed Model build environment for Azure Web App for containers (microsoft#119)
Builds on top of PR microsoft#121, and: - Replaces separate build + release pipelines with a single multi-stage pipeline with stages for Build, Train, ACI, AKS and Webapp deployment. - ACI, AKS and Webapp deployment are optionally enabled (through Variables). - Added example of custom traces (logging) and correlating traces with requests with Kusto queries. - Added example of distributed tracing (client passes a trace header, logged by the scoring service). 
|
|
||
|  | ||
|
|
||
| In the Variables tab, link the pipeline to your variable group (`devopsforai-aml-vg`). In the variable group definition, add the following variables: |
There was a problem hiding this comment.
These should already be picked up by the variables template file. If you want to add instructions for people to modify that file, then no problem else remove this section.
There was a problem hiding this comment.
This is the doc for the release pipeline, which if I'm not mistaken the user must create by hand from scratch. (This is removed in PR #122 that replace the release pipeline with a multi-stage build pipeline)
| set -euo pipefail # strict mode, fail on error | ||
| set -x # verbose | ||
|
|
||
| docker run \ |
There was a problem hiding this comment.
Eugene already has a pipeline file to do this and not the best idea to do docker-in-docker. I'd prefer you remove this file and we can work into a different PR.
There was a problem hiding this comment.
This is used in the release pipeline (created by hand) which per getting_started.md runs directly on the Ubuntu 16 build agent, so there is no use of docker-in-docker.
I agree this script is pretty ugly-looking. It is removed anyway with PR #122, which does away with release pipelines.
If I just remove the file, the getting_started file will not have complete instructions to release a webapp...
|
Thanks a lot for the detailed review! Comments have been addressed with code updates or answers above. Because of changes to requirements.txt, running the build from the PR now requires the image at mcr.microsoft.com/mlops/python:latest to have been updated accordingly. This is why the unit tests are now failing on the PR check. I've verified a modified version of the PR where the pipeline runs on a container deployed in a private registry, built from environment_setup/Dockerfile. |
