azure-cli 2.30.0 throws ERROR: {'Azure-cli-ml Version': '1.33.0', 'Error': TypeError("init() got an unexpected keyword argument 'async_persist'",)} #20297

Cam-Borrowell · 2021-11-11T15:01:19Z

The latests version of az cli on the Ubuntu agents (2.30.0) breaks when running az ml commands.

I'm running the following

  - task: AzureCLI@1
    displayName: "Azure CLI: Install AZ ML Extension"
    inputs:
      azureSubscription: "${{ parameters.AZURE_SUBSCRIPTION }}"
      scriptLocation: inlineScript
      inlineScript: "az extension add --name azure-cli-ml --yes"
      useGlobalConfig: true

  - task: AzureCLI@1
    displayName: "Azure CLI: Update Service (${{ parameters.SERVICE_NAME }})"
    inputs:
      azureSubscription: "${{ parameters.AZURE_SUBSCRIPTION }}"
      scriptLocation: inlineScript
      useGlobalConfig: true
      inlineScript: |
        az ml service update \
        --name ${{ parameters.SERVICE_NAME }} \
        --resource-group ${{ parameters.SERVICE_RESOURCE_GROUP }} \
        --scoring-timeout-ms ${{ parameters.SERVICE_SCORING_TIMEOUT}} \
        --workspace-name ${{ parameters.SERVICE_WORKSPACE_NAME }}

This was running fine with az cli 2.29.2 and as a workaround I've configured my pipeline to downgrade with this

  - task: Bash@3
    displayName: "Bash: Downgrade AZ CLI to [2.29.2]"
    inputs:
      targetType: "inline"
      workingDirectory: "$(Build.SourcesDirectory)"
      script: |
        sudo add-apt-repository https://packages.microsoft.com/repos/azure-cli
        sudo apt update
        apt-cache policy azure-cli
        sudo apt install -y --allow-downgrades azure-cli=2.29.2-1~focal

Logs from the error

==============================================================================
Task         : Azure CLI
Description  : Run Azure CLI commands against an Azure subscription in a Shell script when running on Linux agent or Batch script when running on Windows agent.
Version      : 1.164.0
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/deploy/azure-cli
==============================================================================
/usr/bin/az --version
azure-cli                         2.30.0

core                              2.30.0
telemetry                          1.0.6

Extensions:
azure-cli-ml                      1.33.0
azure-devops                      0.21.0

Python location '/opt/az/bin/python3'
Extensions directory '/opt/az/azcliextensions'

Python (Linux) 3.6.10 (default, Oct 29 2021, 10:11:58) 
[GCC 7.5.0]

Legal docs and information: aka.ms/AzureCliLegal


Your CLI is up-to-date.

Please let us know how we are doing: https://aka.ms/azureclihats
and let us know if you're interested in trying out our newest features: https://aka.ms/CLIUXstudy
Setting active cloud to: AzureCloud
/usr/bin/az cloud set -n AzureCloud
/usr/bin/az login --service-principal -u *** --password=*** --tenant ***
[
  {
   ***** Redacted ******
    }
  }
]
/usr/bin/az account set --subscription ***** Redacted ******
/bin/bash /home/vsts/work/_temp/azureclitaskscript1636529461981.sh
ERROR: {'Azure-cli-ml Version': '1.33.0', 'Error': TypeError("__init__() got an unexpected keyword argument 'async_persist'",)}

Logs when things were working

==============================================================================
Task         : Azure CLI
Description  : Run Azure CLI commands against an Azure subscription in a Shell script when running on Linux agent or Batch script when running on Windows agent.
Version      : 1.164.0
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/deploy/azure-cli
==============================================================================
WARNING: You have 2 updates available. Consider updating your CLI installation with 'az upgrade'

Please let us know how we are doing: https://aka.ms/azureclihats
and let us know if you're interested in trying out our newest features: https://aka.ms/CLIUXstudy
/usr/bin/az --version
azure-cli                         2.29.2 *

core                              2.29.2 *
telemetry                          1.0.6

Extensions:
azure-cli-ml                      1.33.0
azure-devops                      0.21.0

Python location '/opt/az/bin/python3'
Extensions directory '/opt/az/azcliextensions'

Python (Linux) 3.6.10 (default, Oct 29 2021, 08:10:01) 
[GCC 7.5.0]

Legal docs and information: aka.ms/AzureCliLegal


Setting active cloud to: AzureCloud
/usr/bin/az cloud set -n AzureCloud
/usr/bin/az login --service-principal -u *** --password=*** --tenant ***
[
  {
    ***** Redacted ******
       }
  }
]
/usr/bin/az account set --subscription ***** Redacted ******
/bin/bash /home/vsts/work/_temp/azureclitaskscript1636443057305.sh
AKS service creation operation finished, operation "Succeeded"

The text was updated successfully, but these errors were encountered:

ghost · 2021-11-11T15:38:03Z

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @azureml-github.

Issue Details

The latests version of az cli on the Ubuntu agents (2.30.0) breaks when running az ml commands.

I'm running the following

  - task: AzureCLI@1
    displayName: "Azure CLI: Install AZ ML Extension"
    inputs:
      azureSubscription: "${{ parameters.AZURE_SUBSCRIPTION }}"
      scriptLocation: inlineScript
      inlineScript: "az extension add --name azure-cli-ml --yes"
      useGlobalConfig: true

  - task: AzureCLI@1
    displayName: "Azure CLI: Update Service (${{ parameters.SERVICE_NAME }})"
    inputs:
      azureSubscription: "${{ parameters.AZURE_SUBSCRIPTION }}"
      scriptLocation: inlineScript
      useGlobalConfig: true
      inlineScript: |
        az ml service update \
        --name ${{ parameters.SERVICE_NAME }} \
        --resource-group ${{ parameters.SERVICE_RESOURCE_GROUP }} \
        --scoring-timeout-ms ${{ parameters.SERVICE_SCORING_TIMEOUT}} \
        --workspace-name ${{ parameters.SERVICE_WORKSPACE_NAME }}

This was running fine with az cli 2.29.2 and as a workaround I've configured my pipeline to downgrade with this

  - task: Bash@3
    displayName: "Bash: Downgrade AZ CLI to [2.29.2]"
    inputs:
      targetType: "inline"
      workingDirectory: "$(Build.SourcesDirectory)"
      script: |
        sudo add-apt-repository https://packages.microsoft.com/repos/azure-cli
        sudo apt update
        apt-cache policy azure-cli
        sudo apt install -y --allow-downgrades azure-cli=2.29.2-1~focal

Logs from the error

==============================================================================
Task         : Azure CLI
Description  : Run Azure CLI commands against an Azure subscription in a Shell script when running on Linux agent or Batch script when running on Windows agent.
Version      : 1.164.0
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/deploy/azure-cli
==============================================================================
/usr/bin/az --version
azure-cli                         2.30.0

core                              2.30.0
telemetry                          1.0.6

Extensions:
azure-cli-ml                      1.33.0
azure-devops                      0.21.0

Python location '/opt/az/bin/python3'
Extensions directory '/opt/az/azcliextensions'

Python (Linux) 3.6.10 (default, Oct 29 2021, 10:11:58) 
[GCC 7.5.0]

Legal docs and information: aka.ms/AzureCliLegal


Your CLI is up-to-date.

Please let us know how we are doing: https://aka.ms/azureclihats
and let us know if you're interested in trying out our newest features: https://aka.ms/CLIUXstudy
Setting active cloud to: AzureCloud
/usr/bin/az cloud set -n AzureCloud
/usr/bin/az login --service-principal -u *** --password=*** --tenant ***
[
  {
   ***** Redacted ******
    }
  }
]
/usr/bin/az account set --subscription ***** Redacted ******
/bin/bash /home/vsts/work/_temp/azureclitaskscript1636529461981.sh
ERROR: {'Azure-cli-ml Version': '1.33.0', 'Error': TypeError("__init__() got an unexpected keyword argument 'async_persist'",)}

Logs when things were working

==============================================================================
Task         : Azure CLI
Description  : Run Azure CLI commands against an Azure subscription in a Shell script when running on Linux agent or Batch script when running on Windows agent.
Version      : 1.164.0
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/deploy/azure-cli
==============================================================================
WARNING: You have 2 updates available. Consider updating your CLI installation with 'az upgrade'

Please let us know how we are doing: https://aka.ms/azureclihats
and let us know if you're interested in trying out our newest features: https://aka.ms/CLIUXstudy
/usr/bin/az --version
azure-cli                         2.29.2 *

core                              2.29.2 *
telemetry                          1.0.6

Extensions:
azure-cli-ml                      1.33.0
azure-devops                      0.21.0

Python location '/opt/az/bin/python3'
Extensions directory '/opt/az/azcliextensions'

Python (Linux) 3.6.10 (default, Oct 29 2021, 08:10:01) 
[GCC 7.5.0]

Legal docs and information: aka.ms/AzureCliLegal


Setting active cloud to: AzureCloud
/usr/bin/az cloud set -n AzureCloud
/usr/bin/az login --service-principal -u *** --password=*** --tenant ***
[
  {
    ***** Redacted ******
       }
  }
]
/usr/bin/az account set --subscription ***** Redacted ******
/bin/bash /home/vsts/work/_temp/azureclitaskscript1636443057305.sh
AKS service creation operation finished, operation "Succeeded"

Author:	Cam-Borrowell
Assignees:	-
Labels:	`Service Attention`, `Machine Learning`, `customer-reported`
Milestone:	-

yonzhan · 2021-11-11T15:38:10Z

route to service team

jiasli · 2021-11-12T07:30:13Z

async_persist is dropped during MSAL migration (#19853). azure-cli-ml should adopt to the new azure-cli-core 2.30.0.

sigeisler · 2021-11-16T09:35:55Z

When can we expect a fix? Is there a workaround right now?

Cam-Borrowell · 2021-11-16T14:02:28Z

When can we expect a fix? Is there a workaround right now?

As a workaround I've downgraded the azure-cli with this

  - task: Bash@3
    displayName: "Bash: Downgrade AZ CLI to [2.29.2]"
    inputs:
      targetType: "inline"
      workingDirectory: "$(Build.SourcesDirectory)"
      script: |
        sudo add-apt-repository https://packages.microsoft.com/repos/azure-cli
        sudo apt update
        apt-cache policy azure-cli
        sudo apt install -y --allow-downgrades azure-cli=2.29.2-1~focal

However there seems to be a new (unrelated) issue with a new azure-cli-ml version, so I've also downgraded/locked it with this

  - task: AzureCLI@1
    displayName: "Azure CLI: [az extension add --name azure-cli-ml --version 1.33.1]"
    inputs:
      azureSubscription: "${{ parameters.AZURE_SUBSCRIPTION }}"
      scriptLocation: inlineScript
      # 1.5.0 throws some kind of auth error.
      inlineScript: "az extension add --name azure-cli-ml --version 1.33.1"
      useGlobalConfig: true

Mossaka · 2021-11-16T18:38:21Z

@Cam-Borrowell what was the issue with the new azure-cli-ml version 1.33.1?

Cam-Borrowell · 2021-11-16T19:10:49Z

@Cam-Borrowell what was the issue with the new azure-cli-ml version 1.33.1?

With these versions

azure-cli                         2.29.2 *

core                              2.29.2 *
telemetry                          1.0.6

Extensions:
azure-devops                      0.21.0
azure-cli-ml                       1.5.0

Error

ERROR: {'Azure-cli-ml Version': '1.5.0', 'Error': AuthenticationException:
	Message: Could not retrieve user token. Please run 'az login'
	InnerException _retrieve_token() missing 1 required positional argument: 'token_resource'
	ErrorResponse 
{
    "error": {
        "code": "UserError",
        "inner_error": {
            "code": "Authentication"
        },
        "message": "Could not retrieve user token. Please run 'az login'"
    }
}}

Mossaka · 2021-11-16T19:32:43Z

azure-cli-ml 1.5.0 is an old package. Do you see any issues when you upgrade it to 1.33.1?

@kairu-ms When azure cli parses the package version, does 1.33.1 come newer than 1.5.0?

Cam-Borrowell · 2021-11-16T19:37:53Z

azure-cli-ml 1.5.0 is an old package. Do you see any issues when you upgrade it to 1.33.1?

@kairu-ms When azure cli parses the package version, does 1.33.1 come newer than 1.5.0?

@Mossaka After my initial workaround the agents were automatically using 1.5.0 when I was running az extension add --name azure-cli-ml so that's why I had to go back and lock in version 1.33.1. No issues with azure-cli-ml 1.33.1 along with azure-cli 2.29.2

Mossaka · 2021-11-16T19:39:35Z

Thanks for your inputs!

jiasli · 2021-11-17T06:44:01Z

Though I don't have the full pipeline YAML, I would like to mention that Azure CLI 2.29.2 and 2.30.0's login contexts aren't compatible.

Make sure to downgrade Azure CLI first before calling an Azure CLI task.

tranguyen221 · 2021-11-18T09:14:13Z

When can we expect a fix? Is there a workaround right now?

As a workaround I've downgraded the azure-cli with this

  - task: Bash@3
    displayName: "Bash: Downgrade AZ CLI to [2.29.2]"
    inputs:
      targetType: "inline"
      workingDirectory: "$(Build.SourcesDirectory)"
      script: |
        sudo add-apt-repository https://packages.microsoft.com/repos/azure-cli
        sudo apt update
        apt-cache policy azure-cli
        sudo apt install -y --allow-downgrades azure-cli=2.29.2-1~focal

However there seems to be a new (unrelated) issue with a new azure-cli-ml version, so I've also downgraded/locked it with this

  - task: AzureCLI@1
    displayName: "Azure CLI: [az extension add --name azure-cli-ml --version 1.33.1]"
    inputs:
      azureSubscription: "${{ parameters.AZURE_SUBSCRIPTION }}"
      scriptLocation: inlineScript
      # 1.5.0 throws some kind of auth error.
      inlineScript: "az extension add --name azure-cli-ml --version 1.33.1"
      useGlobalConfig: true

Nice. It worked for me. just FYI this command works if you are using image "ubuntu-latest (20.04)", if you are using "ubuntu-18.04", change "focal" to "bionic"

navba-MSFT · 2022-04-01T12:18:48Z

@Cam-Borrowell Apologies for the late reply. Could you please let us know if you are still facing this issue ? Awaiting your reply.

ghost · 2022-04-08T14:00:55Z

Hi, we're sending this friendly reminder because we haven't heard back from you in a while. We need more information about this issue to help address it. Please be sure to give us your input within the next 7 days. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!

tranguyen221 · 2022-04-08T14:07:56Z

Hello, I tested this on Feb 10th, 2022 and this error has still occurred. So I have to keep the quick fix by using ubuntu-version is 18:04, then downgrade azure-devops version to 0.21.0 and the azure-cli to 2.29.2. We also referred to this error and the quick fix in microsoft/dstoolkit-mlops-base#47

ghost · 2022-04-15T20:00:47Z

Hi, we're sending this friendly reminder because we haven't heard back from you in a while. We need more information about this issue to help address it. Please be sure to give us your input within the next 7 days. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!

yonzhan added the Machine Learning az ml label Nov 11, 2021

ghost removed the needs-triage This is a new issue that needs to be triaged to the appropriate team. label Nov 11, 2021

yonzhan added Service Attention This issue is responsible by Azure service team. and removed question The issue doesn't require a change to the product in order to be resolved. Most issues start as that labels Nov 11, 2021

lokijota mentioned this issue Nov 15, 2021

az ml model list throws error in AzureCLI@1 task microsoft/MLOpsPython#384

Open

mariamedp mentioned this issue Nov 17, 2021

The latest version of az cli (2.30.0) break while running az commands microsoft/dstoolkit-mlops-base#47

Closed

tranguyen221 mentioned this issue Nov 18, 2021

Azure CLI (azure-devops) 0.22.0 installed on Ubuntu1804 actions/runner-images#4539

Closed

7 tasks

mariamedp mentioned this issue Nov 18, 2021

Change default auth method from pipelines to Service Principal microsoft/dstoolkit-mlops-base#52

Closed

jiasli mentioned this issue Dec 8, 2021

TypeError: __init__() got an unexpected keyword argument 'user_agent' in CI pipeline command-az extension add -n azure-cli-ml #18626

Closed

jiasli mentioned this issue Dec 23, 2021

Profile __init__() missing keyword argument async_persist when calling az login through default cli invoke() #20796

Closed

navba-MSFT added the needs-author-feedback More information is needed from author to address the issue. label Apr 1, 2022

ghost added the no-recent-activity There has been no recent activity on this issue. label Apr 8, 2022

ghost removed the no-recent-activity There has been no recent activity on this issue. label Apr 8, 2022

ghost added the no-recent-activity There has been no recent activity on this issue. label Apr 15, 2022

ghost closed this as completed Apr 30, 2022

navba-MSFT self-assigned this May 15, 2022

ghost removed the no-recent-activity There has been no recent activity on this issue. label May 15, 2022

navba-MSFT added the question The issue doesn't require a change to the product in order to be resolved. Most issues start as that label Sep 29, 2022

This issue was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

azure-cli 2.30.0 throws ERROR: {'Azure-cli-ml Version': '1.33.0', 'Error': TypeError("init() got an unexpected keyword argument 'async_persist'",)} #20297

azure-cli 2.30.0 throws ERROR: {'Azure-cli-ml Version': '1.33.0', 'Error': TypeError("init() got an unexpected keyword argument 'async_persist'",)} #20297

Cam-Borrowell commented Nov 11, 2021 •

edited

Loading

ghost commented Nov 11, 2021

yonzhan commented Nov 11, 2021

jiasli commented Nov 12, 2021

sigeisler commented Nov 16, 2021

Cam-Borrowell commented Nov 16, 2021

Mossaka commented Nov 16, 2021

Cam-Borrowell commented Nov 16, 2021 •

edited

Loading

Mossaka commented Nov 16, 2021

Cam-Borrowell commented Nov 16, 2021

Mossaka commented Nov 16, 2021

jiasli commented Nov 17, 2021

tranguyen221 commented Nov 18, 2021

navba-MSFT commented Apr 1, 2022

ghost commented Apr 8, 2022

tranguyen221 commented Apr 8, 2022 via email •

edited

Loading

ghost commented Apr 15, 2022

azure-cli 2.30.0 throws ERROR: {'Azure-cli-ml Version': '1.33.0', 'Error': TypeError("__init__() got an unexpected keyword argument 'async_persist'",)} #20297

azure-cli 2.30.0 throws ERROR: {'Azure-cli-ml Version': '1.33.0', 'Error': TypeError("__init__() got an unexpected keyword argument 'async_persist'",)} #20297

Comments

Cam-Borrowell commented Nov 11, 2021 • edited Loading

ghost commented Nov 11, 2021

yonzhan commented Nov 11, 2021

jiasli commented Nov 12, 2021

sigeisler commented Nov 16, 2021

Cam-Borrowell commented Nov 16, 2021

Mossaka commented Nov 16, 2021

Cam-Borrowell commented Nov 16, 2021 • edited Loading

Mossaka commented Nov 16, 2021

Cam-Borrowell commented Nov 16, 2021

Mossaka commented Nov 16, 2021

jiasli commented Nov 17, 2021

tranguyen221 commented Nov 18, 2021

navba-MSFT commented Apr 1, 2022

ghost commented Apr 8, 2022

tranguyen221 commented Apr 8, 2022 via email • edited Loading

ghost commented Apr 15, 2022

azure-cli 2.30.0 throws ERROR: {'Azure-cli-ml Version': '1.33.0', 'Error': TypeError("init() got an unexpected keyword argument 'async_persist'",)} #20297

azure-cli 2.30.0 throws ERROR: {'Azure-cli-ml Version': '1.33.0', 'Error': TypeError("init() got an unexpected keyword argument 'async_persist'",)} #20297

Cam-Borrowell commented Nov 11, 2021 •

edited

Loading

Cam-Borrowell commented Nov 16, 2021 •

edited

Loading

tranguyen221 commented Apr 8, 2022 via email •

edited

Loading