Failed to refresh token - Error details: Manually created ServicePrincipalToken does not contain secret material to retrieve a new access token

[jtharm]

Which version of the AzCopy was used? 10.2.1

Which platform are you using? Linux - CentOS 7.6

What command did you run?

./azcopy copy /tmp/hi https://adlsdb2woc.dfs.core.windows.net/datalake/hi --overwrite=false --from-to=LocalBlobFS --cap-mbps 1

What problem was encountered?

[root@mythos1 azcopy_linux_amd64_10.2.1]# ./azcopy copy /tmp/hi https://adlsdb2woc.dfs.core.windows.net/datalake/hi --overwrite=false --from-to=LocalBlobFS --cap-mbps 1
INFO: Scanning...
INFO: Using OAuth token for authentication.
INFO: failed to refresh token, please check error details and try to log in with azcopy login command again. (Error details: Manually created ServicePrincipalToken does not contain secret material to retrieve a new access token)

Job 9bf81cbf-045f-5440-6166-8353a218755b has started
Log file is located at: /root/.azcopy/9bf81cbf-045f-5440-6166-8353a218755b.log

0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 


Job 9bf81cbf-045f-5440-6166-8353a218755b summary
Elapsed Time (Minutes): 0.0335
Total Number Of Transfers: 1
Number of Transfers Completed: 0
Number of Transfers Failed: 1
Number of Transfers Skipped: 0
TotalBytesTransferred: 0
Final Job Status: Failed

azcopy.log

How can we reproduce the problem in the simplest way?

See above

Have you found a mitigation/solution?

No

[JohnRusk]

That's a weird error. @adreed-msft Can you please take a look at this on Friday or next week?

[adreed-msft]

That does seem odd. @jeeva86 Are you authenticating via a certificate or via a secret? That should help me figure out what causes this.

[adreed-msft]

Furthermore, could you show me the login command/env vars you used (with secret details redacted)?

[jtharm]

I logged in via:

[root@mythos1 azcopy_linux_amd64_10.2.1]# ./azcopy login
To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code A27JZRQ3N to authenticate.

INFO: Logging in under the "Common" tenant. This will log the account in under its home tenant.
INFO: If you plan to use AzCopy with a B2B account (where the account's home tenant is separate from the tenant of the target storage account), please sign in under the target tenant with --tenant-id
INFO: Login succeeded.

[adreed-msft]

The plot thickens-- You are using OAuth and not service principal auth.

[adreed-msft]

I'll have to investigate what could cause this kind of error to be generated from a typical OAuth token.

[jtharm]

Note that I don't receive this error with azcopy v8, so I've always resorted to using that v. but I can't this time because I need to pull/push into ADLS Gen2 :).

[adreed-msft]

Doubly odd, You're getting this error on ADLSG2? This error occurs when we're trying to create a blob pipeline, not a blobFS pipeline.

[jtharm]

I get the error there and in Blob(https://fiftypct.blob.core.windows.net). v10 never liked me...

[ameyaagashe]

`azurerm_virtual_machine.tf-vm-test[0]: Still creating... [52m40s elapsed]
azurerm_virtual_machine.tf-vm-test[1]: Still creating... [52m40s elapsed]

Error: azure.BearerAuthorizer#WithAuthorization: Failed to refresh the Token for the request to https://management.azure.com/subscriptions/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/providers/Microsoft.Compute/locations/australiaeast/operations/da4d39bb-b6c5-4f59-9916-60a8edf9c50f?api-version=2019-03-01: StatusCode=0 -- Original Error: Manually created ServicePrincipalToken does not contain secret material to retrieve a new access token

on test_tpi_standalone.tf line 117, in resource "azurerm_virtual_machine" "tf-vm-test":
117: resource "azurerm_virtual_machine" "tf-vm-test" {

Error: azure.BearerAuthorizer#WithAuthorization: Failed to refresh the Token for the request to https://management.azure.com/subscriptions/xxxxxxxxxxxxxxxxxxxxxxxxxxx/providers/Microsoft.Compute/locations/australiaeast/operations/298d85c7-0fee-4ce9-95f4-4b76e89c4cfa?api-version=2019-03-01: StatusCode=0 -- Original Error: Manually created ServicePrincipalToken does not contain secret material to retrieve a new access token

on test_tpi_standalone.tf line 117, in resource "azurerm_virtual_machine" "tf-vm-test":
117: resource "azurerm_virtual_machine" "tf-vm-test" {
`

[ameyaagashe]

Took an hour to provision two VM's which manually can take by 5 minutes.

[jtharm]

@adreed-msft any update on this?

[adreed-msft]

I'm a bit befuddled, honestly. The latter bit of the error is returned from a SetAuthenticationValues call in ADAL. We attempt to create a service principal token from an OAuth token, which returns a secret type of ServicePrincipalNoSecret. Which means this call could only be triggered during a refresh if your OAuth token somehow contained no refresh information.

If the scope parameter of the device code login contained offline_access, a refresh token is issued. Lucky for us, According to this article, offline_access is enabled by default.

Is it possible you may have disallowed some permissions?

[jtharm]

I used azcopy login to authenticate. What permissions should I be looking for?

[adreed-msft]

I've just tried to log in under a new account and I didn't see any special prompts... I'm honestly unsure of what might trigger a scenario like this other than the inferred offline_access scope not kicking in for some reason.

@JohnRusk and @zezha-msft, We can't currently explicitly add any scope parameters in the autorest ADAL Go SDK. Perhaps we should send out a PR to fix that, and explicitly place the two inferred permissions (user.read and offline_access) into that scope. That would probably resolve this issue.

For now though, we can at least error out on azcopy login if no refresh token is available for OAuth. It doesn't FIX this issue, but it'd save us some similar confusion in the future.

[zezha-msft]

@amishra-dev thoughts on this issue?

[hamedhsn]

we have the same problem any update?
Every-time we want to run azcopy copy we have to do azcopy login and use Oath Token to login.

[AnuradhaAshavathaRao]

Encountered same issue when running terraform to create resources. Any updates on this issue?

[landro]

Have you seen this? hashicorp/go-azure-helpers#43

…

4. nov. 2019 kl. 22:33 skrev AnuradhaAshavathaRao ***@***.***>:  Encountered same issue when running terraform to create resources. Any updates on this issue? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[zezha-msft]

@AnuradhaAshavathaRao could you please clarify how is that related to AzCopy?

[velvia]

I'm still getting this problem btw:

INFO: Scanning...
INFO: Authenticating to source using Azure AD
INFO: failed to refresh token, please check error details and try to log in with azcopy login command again. (Error details: Manually created ServicePrincipalToken does not contain secret material to retrieve a new access token)
INFO: azcopy: A newer version 10.8.0 is available to download

INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support

failed to perform copy command due to error: cannot start job due to error: cannot list files due to reason -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /Users/runner/go/pkg/mod/github.com/!azure/azure-storage-blob-go@v0.10.0/azblob/zc_storage_error.go:42

Version 10.5.0

[adreed-msft]

For users experiencing this issue: We now have the ability to do a one-shot login with environment variables, including the standard interactive login. This unfortunately isn't a full fix, but it should unblock users. Please view azcopy env for more info.