Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New Authorizers for Azure Storage #416

Merged
merged 8 commits into from
Oct 22, 2019

Conversation

tombuildsstuff
Copy link
Contributor

👋

This PR introduces a couple of new Authorizers for Azure Storage - the logic for which is lifted/amended from the existing (deprecated) Azure Storage SDK for Go - which allows SDK's based on this library to talk to Azure Storage.

The first Authorizer allows authenticating to the Blob, File and Queue API's using a Shared Key Lite (SharedKeyLiteAuthorizer) and the other (SharedKeyLiteTableAuthorizer) allows authenticating to the Table Storage API's using a Shared Key Lite - both of which are based off the Account Name and Account Key.

We've been using these in the Storage SDK we're building - and I can confirm these work:

$ go build . && envchain azurerm ./example
2019/07/01 19:10:28 Testing if Service Principal / Client Certificate is applicable for Authentication..
2019/07/01 19:10:28 Testing if Service Principal / Client Secret is applicable for Authentication..
2019/07/01 19:10:28 Using Service Principal / Client Secret for Authentication
2019/07/01 19:10:28 [DEBUG] AzureRM Request:
POST /Tables HTTP/1.1
Host: tomdevsa29.table.core.windows.net
User-Agent: Go/go1.12.6 (amd64-darwin) go-autorest/v11.7.0 tombuildsstuff/giovanni/v0.0.1 storage/2018-11-09
Content-Length: 23
Accept: application/json;odata=nometadata
Content-Type: application/json
Date: Mon, 01 Jul 2019 17:10:28 GMT
Prefer: return-no-content
X-Ms-Date: Mon, 01 Jul 2019 17:10:28 GMT
X-Ms-Version: 2018-11-09
Accept-Encoding: gzip

{"TableName":"sdktest"}
2019/07/01 19:10:29 [DEBUG] AzureRM Response for https://tomdevsa29.table.core.windows.net/Tables:
HTTP/1.1 204 No Content
Content-Length: 0
Cache-Control: no-cache
Dataserviceid: https://tomdevsa29.table.core.windows.net/Tables('sdktest')
Date: Mon, 01 Jul 2019 17:10:29 GMT
Location: https://tomdevsa29.table.core.windows.net/Tables('sdktest')
Preference-Applied: return-no-content
Server: Windows-Azure-Table/1.0 Microsoft-HTTPAPI/2.0
X-Content-Type-Options: nosniff
X-Ms-Request-Id: 54dbae8a-7002-007a-2f2f-30250d000000
X-Ms-Version: 2018-11-09

using a client based on Azure/go-autorest

storageKeyLiteAuthorizer := auth.NewSharedKeyLiteTableAuthorizer(accountName, accountKey)
client := tables.New()
client.Client = prepareClient(client.Client, storageKeyLiteAuthorizer)

_, err := blobClient.Create(ctx, "tomdevsa29", "sdktest")
if err != nil {
	return err
}

Thank you for your contribution to Go-AutoRest! We will triage and review it as soon as we can.

As part of submitting, please make sure you can make the following assertions:

  • I've tested my changes, adding unit tests if applicable.
  • I've added Apache 2.0 Headers to the top of any new source files.
  • I'm submitting this PR to the dev branch, except in the case of urgent bug fixes warranting their own release.
  • If I'm targeting master, I've updated CHANGELOG.md to address the changes I'm making.

@tombuildsstuff
Copy link
Contributor Author

@jhendrixMSFT I've split the storage authorizers out into their own files since there's a /lot/ in the existing ones - perhaps it's worth splitting the authorizers out into their own package/files?

@tombuildsstuff tombuildsstuff force-pushed the f/storage-authorizers branch 2 times, most recently from b165525 to a634f73 Compare July 1, 2019 19:27
@tombuildsstuff
Copy link
Contributor Author

Heads up that the SDK we've been using to build/test this is now public - if it's helpful to test this works as expected: https://github.com/tombuildsstuff/giovanni

@jhendrixMSFT
Copy link
Member

@tombuildsstuff is the guts of the implementation a copy of what's in the storage package?

@tombuildsstuff
Copy link
Contributor Author

@jhendrixMSFT that was the starting point for this, yeah - I'm not sure if that wanted a different license on it or something to state that (sorry!)?

@kahing
Copy link

kahing commented Aug 16, 2019

From a quick glance this doesn't sign the entire query string? That's not going to work for things like datalake gen 2. Also want to fix bugs like Azure/azure-storage-blob-go#146 while you port that over

@tombuildsstuff
Copy link
Contributor Author

@kahing at the moment this authorizer only supports SharedKeyLite rather than SharedKey authorization - since that's what we needed when building a Storage SDK based on go-autorest.

That said - I believe it should be possible to extend this to a SharedKey authorizer if that's needed, since the Storage API's should work with either (I guess really both should be supported?)

@invidian
Copy link

It looks like Premium Tier for storage accounts does not work with SharedKeyLite (see hashicorp/terraform-provider-azurerm#3939 (comment)), so I also suggest adding SharedKey support here, especially that the basic implementation is not much different from SharedKeyLite.

@tombuildsstuff
Copy link
Contributor Author

@invidian thanks for the heads up - so that we can get both the SharedKey and the SharedKeyLite Authorizers into this PR I'm going to pull that patch into AzureRM, run the tests and then update this PR with you as a co-author in both cases, I hope you don't mind?

@invidian
Copy link

invidian commented Sep 2, 2019

@invidian thanks for the heads up - so that we can get both the SharedKey and the SharedKeyLite Authorizers into this PR I'm going to pull that patch into AzureRM, run the tests and then update this PR with you as a co-author in both cases, I hope you don't mind?

I don't mind at all!

I have a testing patch for that, available here: https://github.com/kinvolk/terraform-provider-azurerm/commit/f9676819c28b236967b191d8858c04f118135095, where I tested that Terraform provider behaves as expected. It requires some rework of course, which I was going to do today and push it to AzureRM provider. Like:

  • move shared code between SharedKeyLite and SharedKey to helpers
  • clarify on buildCanonicalizedResource interface.

@tombuildsstuff what would be your timeline on that? I'm asking, because if you work on that next week, then I will do it today, so we can move things forward faster, as this is important for me that this issue is resolved.

Also feel free to pull me for any related code-reviews if that's helpful.

@tombuildsstuff
Copy link
Contributor Author

@invidian

what would be your timeline on that?

I'm looking into this at the moment, so once I've done some testing to determine if we also need a SharedKey authorizer for Table storage we should be good to move forward :)

autorest/authorization_storage.go Outdated Show resolved Hide resolved
autorest/authorization_storage.go Outdated Show resolved Hide resolved
autorest/authorization_storage.go Outdated Show resolved Hide resolved
@jhendrixMSFT
Copy link
Member

@tombuildsstuff I refactored this to pretty much be a copy of what's in the /storage package in the azure-sdk-for-go repo. There's just one SharedKeyAuthorizer type now that takes an enum value for the shared key type. PTAL

Copy link
Contributor Author

@tombuildsstuff tombuildsstuff left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few minor things I spotted but these otherwise look good to me 👍

autorest/authorization_storage.go Outdated Show resolved Hide resolved
autorest/authorization_storage_test.go Show resolved Hide resolved
autorest/authorization_storage_test.go Show resolved Hide resolved
autorest/authorization_storage_test.go Show resolved Hide resolved
autorest/authorization_storage_test.go Show resolved Hide resolved
@jhendrixMSFT jhendrixMSFT merged commit a5c6556 into Azure:dev Oct 22, 2019
@tombuildsstuff tombuildsstuff deleted the f/storage-authorizers branch October 23, 2019 11:23
jhendrixMSFT added a commit that referenced this pull request Oct 28, 2019
* Deserialize additionalInfo in ARM error

* Allow a new authorizer to be created from a configuration file by specifying a resource instead of a base url.

This enables resource like KeyVault and Container Registry to use an authorizer configured from a configuration file.

* [WIP] Using the Context from the timeout if provided (#315)

* Using the timeout from the context if available

- Makes PollingDuration optional

* Renaming the registration start time

* Making PollingDuration not a pointer

* fixing a broken reference

* Add NewAuthorizerFromCli method which uses Azure CLI to obtain a token for the currently logged in user, for  local development scenarios. (#316)

* Adding User assigned identity support for the MSIConfig authorizor (#332)

* Adding ByteSlicePtr (#399)

* Adding a new `WithXML` method (#402)

* Add HTTP status code response helpers (#403)

Added IsHTTPStatus() and HasHTTPStatus() methods to autorest.Response

* adding a new preparer for `MERGE` used in the Storage API's (#406)

* New Preparer/Responder for `Unmarshalling Bytes` (#407)

* New Preparer: WithBytes

* New Responder: `ByUnmarshallingBytes`

* Reusing the bytes, rather than copying them

* Fixing the broken test / switching to read the bytes directly

* Support HTTP-Date in Retry-After header (#410)

RFC specifies Retry-After header can be integer value expressing seconds
or an HTTP-Date indicating when to try again.
Removed superfluous check for HTTP status code.

* Add support for multi-tenant authentication (#412)

* Add support for multi-tenant authentication

Support for multi-tenant via x-ms-authorization-auxiliary header has
been added for client credentials with secret scenario; this basically
bundles multiple OAuthConfig and ServicePrincipalToken types into
corresponding MultiTenant* types along with a new authorizer that adds
the primary and auxiliary token headers to the reqest.
The authenticaion helpers have been updated to support this scenario; if
environment var AZURE_AUXILIARY_TENANT_IDS is set with a semicolon
delimited list of tenants the multi-tenant codepath will kick in to
create the appropriate authorizer.

* feedback

* rename Options to OAuthOptions (#415)

* Support custom SendDecorator chains via context (#417)

* Support custom SendDecorator chains via context

Added `autorest.WithSendDecorators` and `autorest.GetSendDecorators` for
adding and retrieving a custom chain of SendDecorators to the provided
context.
Added `autorest.DoRetryForStatusCodesWithCap` and
`autorest.DelayForBackoffWithCap` to enforce an upper bound on the
duration between retries.
Fixed up some code comments.

* small refactor based on PR feedback

* remove some changes for dev branch

* merge master into dev (#427)

* v12.3.0 (#418)

* Deserialize additionalInfo in ARM error

* Allow a new authorizer to be created from a configuration file by specifying a resource instead of a base url.

This enables resource like KeyVault and Container Registry to use an authorizer configured from a configuration file.

* [WIP] Using the Context from the timeout if provided (#315)

* Using the timeout from the context if available

- Makes PollingDuration optional

* Renaming the registration start time

* Making PollingDuration not a pointer

* fixing a broken reference

* Add NewAuthorizerFromCli method which uses Azure CLI to obtain a token for the currently logged in user, for  local development scenarios. (#316)

* Adding User assigned identity support for the MSIConfig authorizor (#332)

* Adding ByteSlicePtr (#399)

* Adding a new `WithXML` method (#402)

* Add HTTP status code response helpers (#403)

Added IsHTTPStatus() and HasHTTPStatus() methods to autorest.Response

* adding a new preparer for `MERGE` used in the Storage API's (#406)

* New Preparer/Responder for `Unmarshalling Bytes` (#407)

* New Preparer: WithBytes

* New Responder: `ByUnmarshallingBytes`

* Reusing the bytes, rather than copying them

* Fixing the broken test / switching to read the bytes directly

* Support HTTP-Date in Retry-After header (#410)

RFC specifies Retry-After header can be integer value expressing seconds
or an HTTP-Date indicating when to try again.
Removed superfluous check for HTTP status code.

* Add support for multi-tenant authentication (#412)

* Add support for multi-tenant authentication

Support for multi-tenant via x-ms-authorization-auxiliary header has
been added for client credentials with secret scenario; this basically
bundles multiple OAuthConfig and ServicePrincipalToken types into
corresponding MultiTenant* types along with a new authorizer that adds
the primary and auxiliary token headers to the reqest.
The authenticaion helpers have been updated to support this scenario; if
environment var AZURE_AUXILIARY_TENANT_IDS is set with a semicolon
delimited list of tenants the multi-tenant codepath will kick in to
create the appropriate authorizer.

* feedback

* rename Options to OAuthOptions (#415)

* Support custom SendDecorator chains via context (#417)

* Support custom SendDecorator chains via context

Added `autorest.WithSendDecorators` and `autorest.GetSendDecorators` for
adding and retrieving a custom chain of SendDecorators to the provided
context.
Added `autorest.DoRetryForStatusCodesWithCap` and
`autorest.DelayForBackoffWithCap` to enforce an upper bound on the
duration between retries.
Fixed up some code comments.

* small refactor based on PR feedback

* remove some changes for dev branch

* v12.3.0

* add yaml file for azure devops CI (#419)

* add status badge for azure devops CI (#420)

* enable build and test on linux (#421)

* enable build and test on linux

* fail on first error and use portable std*

* update test to run on devops

* Refactor azure devops pipeline (#422)

Break monolithic script into separate scripts with useful names.
Moved formatting checks to the end with succeededOrFailed conditions.

* remove travis artifacts (#423)

* remove unnecessary trigger section from devops (#424)

* Use accessTokens.json from AZURE_CONFIG_DIR if AZURE_ACCESS_TOKEN_FILE is not set before falling back on ~/.azure/ (#471)

* support for parsing error messages from xml responses (#465)

* support for parsing error messages from xml responses

* fixing the linting

* removed some duplicate code

* fix bug introduced in refactoring

* added XML test and fixed bug it uncovered

* fix godoc comment for methods that are safe for concurrent use (#475)

* New Authorizers for Azure Storage (#416)

* Authorizers for Blob, File, Queue and Table Storage

* Adding a SharedKey authorizer

* refactor based on existing storage implementation

* add missing storage emulator account name

* replace hard-coded strings with constants

* changed to by-ref

* Adding a new Authorizer for SAS Token Authentication (#478)

* Adding a new Authorizer for SAS Token Authentication

This commit introduces a new Authorizer for authenticating with
Blob Storage using a SAS Token

```
$ go test -v ./autorest/ -run="TestSas"
=== RUN   TestSasNewSasAuthorizerEmptyToken
--- PASS: TestSasNewSasAuthorizerEmptyToken (0.00s)
=== RUN   TestSasNewSasAuthorizerEmptyTokenWithWhitespace
--- PASS: TestSasNewSasAuthorizerEmptyTokenWithWhitespace (0.00s)
=== RUN   TestSasNewSasAuthorizerValidToken
--- PASS: TestSasNewSasAuthorizerValidToken (0.00s)
=== RUN   TestSasAuthorizerRequest
--- PASS: TestSasAuthorizerRequest (0.00s)
    authorization_sas_test.go:76: [DEBUG] Testing Case "empty querystring without a prefix"..
    authorization_sas_test.go:76: [DEBUG] Testing Case "empty querystring with a prefix"..
    authorization_sas_test.go:76: [DEBUG] Testing Case "existing querystring without a prefix"..
    authorization_sas_test.go:76: [DEBUG] Testing Case "existing querystring with a prefix"..
PASS
ok  	github.com/Azure/go-autorest/autorest	0.011s
```

* minor clean-up

* token: support for a custom refresh func (#476)

* token: support for a custom refresh func

* pass closures by value

* minor clean-up

* v13.3.0
jhendrixMSFT added a commit that referenced this pull request Feb 7, 2020
* Deserialize additionalInfo in ARM error

* Allow a new authorizer to be created from a configuration file by specifying a resource instead of a base url.

This enables resource like KeyVault and Container Registry to use an authorizer configured from a configuration file.

* [WIP] Using the Context from the timeout if provided (#315)

* Using the timeout from the context if available

- Makes PollingDuration optional

* Renaming the registration start time

* Making PollingDuration not a pointer

* fixing a broken reference

* Add NewAuthorizerFromCli method which uses Azure CLI to obtain a token for the currently logged in user, for  local development scenarios. (#316)

* Adding User assigned identity support for the MSIConfig authorizor (#332)

* Adding ByteSlicePtr (#399)

* Adding a new `WithXML` method (#402)

* Add HTTP status code response helpers (#403)

Added IsHTTPStatus() and HasHTTPStatus() methods to autorest.Response

* adding a new preparer for `MERGE` used in the Storage API's (#406)

* New Preparer/Responder for `Unmarshalling Bytes` (#407)

* New Preparer: WithBytes

* New Responder: `ByUnmarshallingBytes`

* Reusing the bytes, rather than copying them

* Fixing the broken test / switching to read the bytes directly

* Support HTTP-Date in Retry-After header (#410)

RFC specifies Retry-After header can be integer value expressing seconds
or an HTTP-Date indicating when to try again.
Removed superfluous check for HTTP status code.

* Add support for multi-tenant authentication (#412)

* Add support for multi-tenant authentication

Support for multi-tenant via x-ms-authorization-auxiliary header has
been added for client credentials with secret scenario; this basically
bundles multiple OAuthConfig and ServicePrincipalToken types into
corresponding MultiTenant* types along with a new authorizer that adds
the primary and auxiliary token headers to the reqest.
The authenticaion helpers have been updated to support this scenario; if
environment var AZURE_AUXILIARY_TENANT_IDS is set with a semicolon
delimited list of tenants the multi-tenant codepath will kick in to
create the appropriate authorizer.

* feedback

* rename Options to OAuthOptions (#415)

* Support custom SendDecorator chains via context (#417)

* Support custom SendDecorator chains via context

Added `autorest.WithSendDecorators` and `autorest.GetSendDecorators` for
adding and retrieving a custom chain of SendDecorators to the provided
context.
Added `autorest.DoRetryForStatusCodesWithCap` and
`autorest.DelayForBackoffWithCap` to enforce an upper bound on the
duration between retries.
Fixed up some code comments.

* small refactor based on PR feedback

* remove some changes for dev branch

* merge master into dev (#427)

* v12.3.0 (#418)

* Deserialize additionalInfo in ARM error

* Allow a new authorizer to be created from a configuration file by specifying a resource instead of a base url.

This enables resource like KeyVault and Container Registry to use an authorizer configured from a configuration file.

* [WIP] Using the Context from the timeout if provided (#315)

* Using the timeout from the context if available

- Makes PollingDuration optional

* Renaming the registration start time

* Making PollingDuration not a pointer

* fixing a broken reference

* Add NewAuthorizerFromCli method which uses Azure CLI to obtain a token for the currently logged in user, for  local development scenarios. (#316)

* Adding User assigned identity support for the MSIConfig authorizor (#332)

* Adding ByteSlicePtr (#399)

* Adding a new `WithXML` method (#402)

* Add HTTP status code response helpers (#403)

Added IsHTTPStatus() and HasHTTPStatus() methods to autorest.Response

* adding a new preparer for `MERGE` used in the Storage API's (#406)

* New Preparer/Responder for `Unmarshalling Bytes` (#407)

* New Preparer: WithBytes

* New Responder: `ByUnmarshallingBytes`

* Reusing the bytes, rather than copying them

* Fixing the broken test / switching to read the bytes directly

* Support HTTP-Date in Retry-After header (#410)

RFC specifies Retry-After header can be integer value expressing seconds
or an HTTP-Date indicating when to try again.
Removed superfluous check for HTTP status code.

* Add support for multi-tenant authentication (#412)

* Add support for multi-tenant authentication

Support for multi-tenant via x-ms-authorization-auxiliary header has
been added for client credentials with secret scenario; this basically
bundles multiple OAuthConfig and ServicePrincipalToken types into
corresponding MultiTenant* types along with a new authorizer that adds
the primary and auxiliary token headers to the reqest.
The authenticaion helpers have been updated to support this scenario; if
environment var AZURE_AUXILIARY_TENANT_IDS is set with a semicolon
delimited list of tenants the multi-tenant codepath will kick in to
create the appropriate authorizer.

* feedback

* rename Options to OAuthOptions (#415)

* Support custom SendDecorator chains via context (#417)

* Support custom SendDecorator chains via context

Added `autorest.WithSendDecorators` and `autorest.GetSendDecorators` for
adding and retrieving a custom chain of SendDecorators to the provided
context.
Added `autorest.DoRetryForStatusCodesWithCap` and
`autorest.DelayForBackoffWithCap` to enforce an upper bound on the
duration between retries.
Fixed up some code comments.

* small refactor based on PR feedback

* remove some changes for dev branch

* v12.3.0

* add yaml file for azure devops CI (#419)

* add status badge for azure devops CI (#420)

* enable build and test on linux (#421)

* enable build and test on linux

* fail on first error and use portable std*

* update test to run on devops

* Refactor azure devops pipeline (#422)

Break monolithic script into separate scripts with useful names.
Moved formatting checks to the end with succeededOrFailed conditions.

* remove travis artifacts (#423)

* remove unnecessary trigger section from devops (#424)

* Use accessTokens.json from AZURE_CONFIG_DIR if AZURE_ACCESS_TOKEN_FILE is not set before falling back on ~/.azure/ (#471)

* support for parsing error messages from xml responses (#465)

* support for parsing error messages from xml responses

* fixing the linting

* removed some duplicate code

* fix bug introduced in refactoring

* added XML test and fixed bug it uncovered

* fix godoc comment for methods that are safe for concurrent use (#475)

* New Authorizers for Azure Storage (#416)

* Authorizers for Blob, File, Queue and Table Storage

* Adding a SharedKey authorizer

* refactor based on existing storage implementation

* add missing storage emulator account name

* replace hard-coded strings with constants

* changed to by-ref

* Adding a new Authorizer for SAS Token Authentication (#478)

* Adding a new Authorizer for SAS Token Authentication

This commit introduces a new Authorizer for authenticating with
Blob Storage using a SAS Token

```
$ go test -v ./autorest/ -run="TestSas"
=== RUN   TestSasNewSasAuthorizerEmptyToken
--- PASS: TestSasNewSasAuthorizerEmptyToken (0.00s)
=== RUN   TestSasNewSasAuthorizerEmptyTokenWithWhitespace
--- PASS: TestSasNewSasAuthorizerEmptyTokenWithWhitespace (0.00s)
=== RUN   TestSasNewSasAuthorizerValidToken
--- PASS: TestSasNewSasAuthorizerValidToken (0.00s)
=== RUN   TestSasAuthorizerRequest
--- PASS: TestSasAuthorizerRequest (0.00s)
    authorization_sas_test.go:76: [DEBUG] Testing Case "empty querystring without a prefix"..
    authorization_sas_test.go:76: [DEBUG] Testing Case "empty querystring with a prefix"..
    authorization_sas_test.go:76: [DEBUG] Testing Case "existing querystring without a prefix"..
    authorization_sas_test.go:76: [DEBUG] Testing Case "existing querystring with a prefix"..
PASS
ok  	github.com/Azure/go-autorest/autorest	0.011s
```

* minor clean-up

* token: support for a custom refresh func (#476)

* token: support for a custom refresh func

* pass closures by value

* minor clean-up

* Fix Dropped Errors (#480)

* autorest: fix dropped errror

* autorest/adal: fix dropped test error

* Duration order consistency when multiplying number by time unit (#499)

* Drain response bodies (#432)

The retry helpers and a few other methods weren't reading and closing
response bodies leading to connection leaks.

* Enable exponential back-off when retrying on 429 (#503)

* Enable exponential back-off when retrying on 429

* enforce a 2-minute cap on delays if there isn't one

* updated comment

* fix type-o

* update version and CHANGELOG

Co-authored-by: Nick <muller_nicky@hotmail.com>
Co-authored-by: Tom Harvey <tombuildsstuff@users.noreply.github.com>
Co-authored-by: Sam Kreter <samkreter@gmail.com>
Co-authored-by: Delyan Raychev <delyan.raychev@microsoft.com>
Co-authored-by: Patrick Decat <pdecat@gmail.com>
Co-authored-by: Tony Abboud <tdabboud@hotmail.com>
Co-authored-by: Lars Lehtonen <lars.lehtonen@gmail.com>
Co-authored-by: Maxim Fominykh <vominyh@yandex.ru>
alrs pushed a commit to alrs/go-autorest that referenced this pull request Mar 6, 2020
* Deserialize additionalInfo in ARM error

* Allow a new authorizer to be created from a configuration file by specifying a resource instead of a base url.

This enables resource like KeyVault and Container Registry to use an authorizer configured from a configuration file.

* [WIP] Using the Context from the timeout if provided (Azure#315)

* Using the timeout from the context if available

- Makes PollingDuration optional

* Renaming the registration start time

* Making PollingDuration not a pointer

* fixing a broken reference

* Add NewAuthorizerFromCli method which uses Azure CLI to obtain a token for the currently logged in user, for  local development scenarios. (Azure#316)

* Adding User assigned identity support for the MSIConfig authorizor (Azure#332)

* Adding ByteSlicePtr (Azure#399)

* Adding a new `WithXML` method (Azure#402)

* Add HTTP status code response helpers (Azure#403)

Added IsHTTPStatus() and HasHTTPStatus() methods to autorest.Response

* adding a new preparer for `MERGE` used in the Storage API's (Azure#406)

* New Preparer/Responder for `Unmarshalling Bytes` (Azure#407)

* New Preparer: WithBytes

* New Responder: `ByUnmarshallingBytes`

* Reusing the bytes, rather than copying them

* Fixing the broken test / switching to read the bytes directly

* Support HTTP-Date in Retry-After header (Azure#410)

RFC specifies Retry-After header can be integer value expressing seconds
or an HTTP-Date indicating when to try again.
Removed superfluous check for HTTP status code.

* Add support for multi-tenant authentication (Azure#412)

* Add support for multi-tenant authentication

Support for multi-tenant via x-ms-authorization-auxiliary header has
been added for client credentials with secret scenario; this basically
bundles multiple OAuthConfig and ServicePrincipalToken types into
corresponding MultiTenant* types along with a new authorizer that adds
the primary and auxiliary token headers to the reqest.
The authenticaion helpers have been updated to support this scenario; if
environment var AZURE_AUXILIARY_TENANT_IDS is set with a semicolon
delimited list of tenants the multi-tenant codepath will kick in to
create the appropriate authorizer.

* feedback

* rename Options to OAuthOptions (Azure#415)

* Support custom SendDecorator chains via context (Azure#417)

* Support custom SendDecorator chains via context

Added `autorest.WithSendDecorators` and `autorest.GetSendDecorators` for
adding and retrieving a custom chain of SendDecorators to the provided
context.
Added `autorest.DoRetryForStatusCodesWithCap` and
`autorest.DelayForBackoffWithCap` to enforce an upper bound on the
duration between retries.
Fixed up some code comments.

* small refactor based on PR feedback

* remove some changes for dev branch

* merge master into dev (Azure#427)

* v12.3.0 (Azure#418)

* Deserialize additionalInfo in ARM error

* Allow a new authorizer to be created from a configuration file by specifying a resource instead of a base url.

This enables resource like KeyVault and Container Registry to use an authorizer configured from a configuration file.

* [WIP] Using the Context from the timeout if provided (Azure#315)

* Using the timeout from the context if available

- Makes PollingDuration optional

* Renaming the registration start time

* Making PollingDuration not a pointer

* fixing a broken reference

* Add NewAuthorizerFromCli method which uses Azure CLI to obtain a token for the currently logged in user, for  local development scenarios. (Azure#316)

* Adding User assigned identity support for the MSIConfig authorizor (Azure#332)

* Adding ByteSlicePtr (Azure#399)

* Adding a new `WithXML` method (Azure#402)

* Add HTTP status code response helpers (Azure#403)

Added IsHTTPStatus() and HasHTTPStatus() methods to autorest.Response

* adding a new preparer for `MERGE` used in the Storage API's (Azure#406)

* New Preparer/Responder for `Unmarshalling Bytes` (Azure#407)

* New Preparer: WithBytes

* New Responder: `ByUnmarshallingBytes`

* Reusing the bytes, rather than copying them

* Fixing the broken test / switching to read the bytes directly

* Support HTTP-Date in Retry-After header (Azure#410)

RFC specifies Retry-After header can be integer value expressing seconds
or an HTTP-Date indicating when to try again.
Removed superfluous check for HTTP status code.

* Add support for multi-tenant authentication (Azure#412)

* Add support for multi-tenant authentication

Support for multi-tenant via x-ms-authorization-auxiliary header has
been added for client credentials with secret scenario; this basically
bundles multiple OAuthConfig and ServicePrincipalToken types into
corresponding MultiTenant* types along with a new authorizer that adds
the primary and auxiliary token headers to the reqest.
The authenticaion helpers have been updated to support this scenario; if
environment var AZURE_AUXILIARY_TENANT_IDS is set with a semicolon
delimited list of tenants the multi-tenant codepath will kick in to
create the appropriate authorizer.

* feedback

* rename Options to OAuthOptions (Azure#415)

* Support custom SendDecorator chains via context (Azure#417)

* Support custom SendDecorator chains via context

Added `autorest.WithSendDecorators` and `autorest.GetSendDecorators` for
adding and retrieving a custom chain of SendDecorators to the provided
context.
Added `autorest.DoRetryForStatusCodesWithCap` and
`autorest.DelayForBackoffWithCap` to enforce an upper bound on the
duration between retries.
Fixed up some code comments.

* small refactor based on PR feedback

* remove some changes for dev branch

* v12.3.0

* add yaml file for azure devops CI (Azure#419)

* add status badge for azure devops CI (Azure#420)

* enable build and test on linux (Azure#421)

* enable build and test on linux

* fail on first error and use portable std*

* update test to run on devops

* Refactor azure devops pipeline (Azure#422)

Break monolithic script into separate scripts with useful names.
Moved formatting checks to the end with succeededOrFailed conditions.

* remove travis artifacts (Azure#423)

* remove unnecessary trigger section from devops (Azure#424)

* Use accessTokens.json from AZURE_CONFIG_DIR if AZURE_ACCESS_TOKEN_FILE is not set before falling back on ~/.azure/ (Azure#471)

* support for parsing error messages from xml responses (Azure#465)

* support for parsing error messages from xml responses

* fixing the linting

* removed some duplicate code

* fix bug introduced in refactoring

* added XML test and fixed bug it uncovered

* fix godoc comment for methods that are safe for concurrent use (Azure#475)

* New Authorizers for Azure Storage (Azure#416)

* Authorizers for Blob, File, Queue and Table Storage

* Adding a SharedKey authorizer

* refactor based on existing storage implementation

* add missing storage emulator account name

* replace hard-coded strings with constants

* changed to by-ref

* Adding a new Authorizer for SAS Token Authentication (Azure#478)

* Adding a new Authorizer for SAS Token Authentication

This commit introduces a new Authorizer for authenticating with
Blob Storage using a SAS Token

```
$ go test -v ./autorest/ -run="TestSas"
=== RUN   TestSasNewSasAuthorizerEmptyToken
--- PASS: TestSasNewSasAuthorizerEmptyToken (0.00s)
=== RUN   TestSasNewSasAuthorizerEmptyTokenWithWhitespace
--- PASS: TestSasNewSasAuthorizerEmptyTokenWithWhitespace (0.00s)
=== RUN   TestSasNewSasAuthorizerValidToken
--- PASS: TestSasNewSasAuthorizerValidToken (0.00s)
=== RUN   TestSasAuthorizerRequest
--- PASS: TestSasAuthorizerRequest (0.00s)
    authorization_sas_test.go:76: [DEBUG] Testing Case "empty querystring without a prefix"..
    authorization_sas_test.go:76: [DEBUG] Testing Case "empty querystring with a prefix"..
    authorization_sas_test.go:76: [DEBUG] Testing Case "existing querystring without a prefix"..
    authorization_sas_test.go:76: [DEBUG] Testing Case "existing querystring with a prefix"..
PASS
ok  	github.com/Azure/go-autorest/autorest	0.011s
```

* minor clean-up

* token: support for a custom refresh func (Azure#476)

* token: support for a custom refresh func

* pass closures by value

* minor clean-up

* v13.3.0
alrs added a commit to alrs/go-autorest that referenced this pull request Mar 6, 2020
* Deserialize additionalInfo in ARM error

* Allow a new authorizer to be created from a configuration file by specifying a resource instead of a base url.

This enables resource like KeyVault and Container Registry to use an authorizer configured from a configuration file.

* [WIP] Using the Context from the timeout if provided (Azure#315)

* Using the timeout from the context if available

- Makes PollingDuration optional

* Renaming the registration start time

* Making PollingDuration not a pointer

* fixing a broken reference

* Add NewAuthorizerFromCli method which uses Azure CLI to obtain a token for the currently logged in user, for  local development scenarios. (Azure#316)

* Adding User assigned identity support for the MSIConfig authorizor (Azure#332)

* Adding ByteSlicePtr (Azure#399)

* Adding a new `WithXML` method (Azure#402)

* Add HTTP status code response helpers (Azure#403)

Added IsHTTPStatus() and HasHTTPStatus() methods to autorest.Response

* adding a new preparer for `MERGE` used in the Storage API's (Azure#406)

* New Preparer/Responder for `Unmarshalling Bytes` (Azure#407)

* New Preparer: WithBytes

* New Responder: `ByUnmarshallingBytes`

* Reusing the bytes, rather than copying them

* Fixing the broken test / switching to read the bytes directly

* Support HTTP-Date in Retry-After header (Azure#410)

RFC specifies Retry-After header can be integer value expressing seconds
or an HTTP-Date indicating when to try again.
Removed superfluous check for HTTP status code.

* Add support for multi-tenant authentication (Azure#412)

* Add support for multi-tenant authentication

Support for multi-tenant via x-ms-authorization-auxiliary header has
been added for client credentials with secret scenario; this basically
bundles multiple OAuthConfig and ServicePrincipalToken types into
corresponding MultiTenant* types along with a new authorizer that adds
the primary and auxiliary token headers to the reqest.
The authenticaion helpers have been updated to support this scenario; if
environment var AZURE_AUXILIARY_TENANT_IDS is set with a semicolon
delimited list of tenants the multi-tenant codepath will kick in to
create the appropriate authorizer.

* feedback

* rename Options to OAuthOptions (Azure#415)

* Support custom SendDecorator chains via context (Azure#417)

* Support custom SendDecorator chains via context

Added `autorest.WithSendDecorators` and `autorest.GetSendDecorators` for
adding and retrieving a custom chain of SendDecorators to the provided
context.
Added `autorest.DoRetryForStatusCodesWithCap` and
`autorest.DelayForBackoffWithCap` to enforce an upper bound on the
duration between retries.
Fixed up some code comments.

* small refactor based on PR feedback

* remove some changes for dev branch

* merge master into dev (Azure#427)

* v12.3.0 (Azure#418)

* Deserialize additionalInfo in ARM error

* Allow a new authorizer to be created from a configuration file by specifying a resource instead of a base url.

This enables resource like KeyVault and Container Registry to use an authorizer configured from a configuration file.

* [WIP] Using the Context from the timeout if provided (Azure#315)

* Using the timeout from the context if available

- Makes PollingDuration optional

* Renaming the registration start time

* Making PollingDuration not a pointer

* fixing a broken reference

* Add NewAuthorizerFromCli method which uses Azure CLI to obtain a token for the currently logged in user, for  local development scenarios. (Azure#316)

* Adding User assigned identity support for the MSIConfig authorizor (Azure#332)

* Adding ByteSlicePtr (Azure#399)

* Adding a new `WithXML` method (Azure#402)

* Add HTTP status code response helpers (Azure#403)

Added IsHTTPStatus() and HasHTTPStatus() methods to autorest.Response

* adding a new preparer for `MERGE` used in the Storage API's (Azure#406)

* New Preparer/Responder for `Unmarshalling Bytes` (Azure#407)

* New Preparer: WithBytes

* New Responder: `ByUnmarshallingBytes`

* Reusing the bytes, rather than copying them

* Fixing the broken test / switching to read the bytes directly

* Support HTTP-Date in Retry-After header (Azure#410)

RFC specifies Retry-After header can be integer value expressing seconds
or an HTTP-Date indicating when to try again.
Removed superfluous check for HTTP status code.

* Add support for multi-tenant authentication (Azure#412)

* Add support for multi-tenant authentication

Support for multi-tenant via x-ms-authorization-auxiliary header has
been added for client credentials with secret scenario; this basically
bundles multiple OAuthConfig and ServicePrincipalToken types into
corresponding MultiTenant* types along with a new authorizer that adds
the primary and auxiliary token headers to the reqest.
The authenticaion helpers have been updated to support this scenario; if
environment var AZURE_AUXILIARY_TENANT_IDS is set with a semicolon
delimited list of tenants the multi-tenant codepath will kick in to
create the appropriate authorizer.

* feedback

* rename Options to OAuthOptions (Azure#415)

* Support custom SendDecorator chains via context (Azure#417)

* Support custom SendDecorator chains via context

Added `autorest.WithSendDecorators` and `autorest.GetSendDecorators` for
adding and retrieving a custom chain of SendDecorators to the provided
context.
Added `autorest.DoRetryForStatusCodesWithCap` and
`autorest.DelayForBackoffWithCap` to enforce an upper bound on the
duration between retries.
Fixed up some code comments.

* small refactor based on PR feedback

* remove some changes for dev branch

* v12.3.0

* add yaml file for azure devops CI (Azure#419)

* add status badge for azure devops CI (Azure#420)

* enable build and test on linux (Azure#421)

* enable build and test on linux

* fail on first error and use portable std*

* update test to run on devops

* Refactor azure devops pipeline (Azure#422)

Break monolithic script into separate scripts with useful names.
Moved formatting checks to the end with succeededOrFailed conditions.

* remove travis artifacts (Azure#423)

* remove unnecessary trigger section from devops (Azure#424)

* Use accessTokens.json from AZURE_CONFIG_DIR if AZURE_ACCESS_TOKEN_FILE is not set before falling back on ~/.azure/ (Azure#471)

* support for parsing error messages from xml responses (Azure#465)

* support for parsing error messages from xml responses

* fixing the linting

* removed some duplicate code

* fix bug introduced in refactoring

* added XML test and fixed bug it uncovered

* fix godoc comment for methods that are safe for concurrent use (Azure#475)

* New Authorizers for Azure Storage (Azure#416)

* Authorizers for Blob, File, Queue and Table Storage

* Adding a SharedKey authorizer

* refactor based on existing storage implementation

* add missing storage emulator account name

* replace hard-coded strings with constants

* changed to by-ref

* Adding a new Authorizer for SAS Token Authentication (Azure#478)

* Adding a new Authorizer for SAS Token Authentication

This commit introduces a new Authorizer for authenticating with
Blob Storage using a SAS Token

```
$ go test -v ./autorest/ -run="TestSas"
=== RUN   TestSasNewSasAuthorizerEmptyToken
--- PASS: TestSasNewSasAuthorizerEmptyToken (0.00s)
=== RUN   TestSasNewSasAuthorizerEmptyTokenWithWhitespace
--- PASS: TestSasNewSasAuthorizerEmptyTokenWithWhitespace (0.00s)
=== RUN   TestSasNewSasAuthorizerValidToken
--- PASS: TestSasNewSasAuthorizerValidToken (0.00s)
=== RUN   TestSasAuthorizerRequest
--- PASS: TestSasAuthorizerRequest (0.00s)
    authorization_sas_test.go:76: [DEBUG] Testing Case "empty querystring without a prefix"..
    authorization_sas_test.go:76: [DEBUG] Testing Case "empty querystring with a prefix"..
    authorization_sas_test.go:76: [DEBUG] Testing Case "existing querystring without a prefix"..
    authorization_sas_test.go:76: [DEBUG] Testing Case "existing querystring with a prefix"..
PASS
ok  	github.com/Azure/go-autorest/autorest	0.011s
```

* minor clean-up

* token: support for a custom refresh func (Azure#476)

* token: support for a custom refresh func

* pass closures by value

* minor clean-up

* Fix Dropped Errors (Azure#480)

* autorest: fix dropped errror

* autorest/adal: fix dropped test error

* Duration order consistency when multiplying number by time unit (Azure#499)

* Drain response bodies (Azure#432)

The retry helpers and a few other methods weren't reading and closing
response bodies leading to connection leaks.

* Enable exponential back-off when retrying on 429 (Azure#503)

* Enable exponential back-off when retrying on 429

* enforce a 2-minute cap on delays if there isn't one

* updated comment

* fix type-o

* update version and CHANGELOG

Co-authored-by: Nick <muller_nicky@hotmail.com>
Co-authored-by: Tom Harvey <tombuildsstuff@users.noreply.github.com>
Co-authored-by: Sam Kreter <samkreter@gmail.com>
Co-authored-by: Delyan Raychev <delyan.raychev@microsoft.com>
Co-authored-by: Patrick Decat <pdecat@gmail.com>
Co-authored-by: Tony Abboud <tdabboud@hotmail.com>
Co-authored-by: Lars Lehtonen <lars.lehtonen@gmail.com>
Co-authored-by: Maxim Fominykh <vominyh@yandex.ru>
jhendrixMSFT added a commit that referenced this pull request Jun 23, 2020
* Deserialize additionalInfo in ARM error

* Allow a new authorizer to be created from a configuration file by specifying a resource instead of a base url.

This enables resource like KeyVault and Container Registry to use an authorizer configured from a configuration file.

* [WIP] Using the Context from the timeout if provided (#315)

* Using the timeout from the context if available

- Makes PollingDuration optional

* Renaming the registration start time

* Making PollingDuration not a pointer

* fixing a broken reference

* Add NewAuthorizerFromCli method which uses Azure CLI to obtain a token for the currently logged in user, for  local development scenarios. (#316)

* Adding User assigned identity support for the MSIConfig authorizor (#332)

* Adding ByteSlicePtr (#399)

* Adding a new `WithXML` method (#402)

* Add HTTP status code response helpers (#403)

Added IsHTTPStatus() and HasHTTPStatus() methods to autorest.Response

* adding a new preparer for `MERGE` used in the Storage API's (#406)

* New Preparer/Responder for `Unmarshalling Bytes` (#407)

* New Preparer: WithBytes

* New Responder: `ByUnmarshallingBytes`

* Reusing the bytes, rather than copying them

* Fixing the broken test / switching to read the bytes directly

* Support HTTP-Date in Retry-After header (#410)

RFC specifies Retry-After header can be integer value expressing seconds
or an HTTP-Date indicating when to try again.
Removed superfluous check for HTTP status code.

* Add support for multi-tenant authentication (#412)

* Add support for multi-tenant authentication

Support for multi-tenant via x-ms-authorization-auxiliary header has
been added for client credentials with secret scenario; this basically
bundles multiple OAuthConfig and ServicePrincipalToken types into
corresponding MultiTenant* types along with a new authorizer that adds
the primary and auxiliary token headers to the reqest.
The authenticaion helpers have been updated to support this scenario; if
environment var AZURE_AUXILIARY_TENANT_IDS is set with a semicolon
delimited list of tenants the multi-tenant codepath will kick in to
create the appropriate authorizer.

* feedback

* rename Options to OAuthOptions (#415)

* Support custom SendDecorator chains via context (#417)

* Support custom SendDecorator chains via context

Added `autorest.WithSendDecorators` and `autorest.GetSendDecorators` for
adding and retrieving a custom chain of SendDecorators to the provided
context.
Added `autorest.DoRetryForStatusCodesWithCap` and
`autorest.DelayForBackoffWithCap` to enforce an upper bound on the
duration between retries.
Fixed up some code comments.

* small refactor based on PR feedback

* remove some changes for dev branch

* merge master into dev (#427)

* v12.3.0 (#418)

* Deserialize additionalInfo in ARM error

* Allow a new authorizer to be created from a configuration file by specifying a resource instead of a base url.

This enables resource like KeyVault and Container Registry to use an authorizer configured from a configuration file.

* [WIP] Using the Context from the timeout if provided (#315)

* Using the timeout from the context if available

- Makes PollingDuration optional

* Renaming the registration start time

* Making PollingDuration not a pointer

* fixing a broken reference

* Add NewAuthorizerFromCli method which uses Azure CLI to obtain a token for the currently logged in user, for  local development scenarios. (#316)

* Adding User assigned identity support for the MSIConfig authorizor (#332)

* Adding ByteSlicePtr (#399)

* Adding a new `WithXML` method (#402)

* Add HTTP status code response helpers (#403)

Added IsHTTPStatus() and HasHTTPStatus() methods to autorest.Response

* adding a new preparer for `MERGE` used in the Storage API's (#406)

* New Preparer/Responder for `Unmarshalling Bytes` (#407)

* New Preparer: WithBytes

* New Responder: `ByUnmarshallingBytes`

* Reusing the bytes, rather than copying them

* Fixing the broken test / switching to read the bytes directly

* Support HTTP-Date in Retry-After header (#410)

RFC specifies Retry-After header can be integer value expressing seconds
or an HTTP-Date indicating when to try again.
Removed superfluous check for HTTP status code.

* Add support for multi-tenant authentication (#412)

* Add support for multi-tenant authentication

Support for multi-tenant via x-ms-authorization-auxiliary header has
been added for client credentials with secret scenario; this basically
bundles multiple OAuthConfig and ServicePrincipalToken types into
corresponding MultiTenant* types along with a new authorizer that adds
the primary and auxiliary token headers to the reqest.
The authenticaion helpers have been updated to support this scenario; if
environment var AZURE_AUXILIARY_TENANT_IDS is set with a semicolon
delimited list of tenants the multi-tenant codepath will kick in to
create the appropriate authorizer.

* feedback

* rename Options to OAuthOptions (#415)

* Support custom SendDecorator chains via context (#417)

* Support custom SendDecorator chains via context

Added `autorest.WithSendDecorators` and `autorest.GetSendDecorators` for
adding and retrieving a custom chain of SendDecorators to the provided
context.
Added `autorest.DoRetryForStatusCodesWithCap` and
`autorest.DelayForBackoffWithCap` to enforce an upper bound on the
duration between retries.
Fixed up some code comments.

* small refactor based on PR feedback

* remove some changes for dev branch

* v12.3.0

* add yaml file for azure devops CI (#419)

* add status badge for azure devops CI (#420)

* enable build and test on linux (#421)

* enable build and test on linux

* fail on first error and use portable std*

* update test to run on devops

* Refactor azure devops pipeline (#422)

Break monolithic script into separate scripts with useful names.
Moved formatting checks to the end with succeededOrFailed conditions.

* remove travis artifacts (#423)

* remove unnecessary trigger section from devops (#424)

* Use accessTokens.json from AZURE_CONFIG_DIR if AZURE_ACCESS_TOKEN_FILE is not set before falling back on ~/.azure/ (#471)

* support for parsing error messages from xml responses (#465)

* support for parsing error messages from xml responses

* fixing the linting

* removed some duplicate code

* fix bug introduced in refactoring

* added XML test and fixed bug it uncovered

* fix godoc comment for methods that are safe for concurrent use (#475)

* New Authorizers for Azure Storage (#416)

* Authorizers for Blob, File, Queue and Table Storage

* Adding a SharedKey authorizer

* refactor based on existing storage implementation

* add missing storage emulator account name

* replace hard-coded strings with constants

* changed to by-ref

* Adding a new Authorizer for SAS Token Authentication (#478)

* Adding a new Authorizer for SAS Token Authentication

This commit introduces a new Authorizer for authenticating with
Blob Storage using a SAS Token

```
$ go test -v ./autorest/ -run="TestSas"
=== RUN   TestSasNewSasAuthorizerEmptyToken
--- PASS: TestSasNewSasAuthorizerEmptyToken (0.00s)
=== RUN   TestSasNewSasAuthorizerEmptyTokenWithWhitespace
--- PASS: TestSasNewSasAuthorizerEmptyTokenWithWhitespace (0.00s)
=== RUN   TestSasNewSasAuthorizerValidToken
--- PASS: TestSasNewSasAuthorizerValidToken (0.00s)
=== RUN   TestSasAuthorizerRequest
--- PASS: TestSasAuthorizerRequest (0.00s)
    authorization_sas_test.go:76: [DEBUG] Testing Case "empty querystring without a prefix"..
    authorization_sas_test.go:76: [DEBUG] Testing Case "empty querystring with a prefix"..
    authorization_sas_test.go:76: [DEBUG] Testing Case "existing querystring without a prefix"..
    authorization_sas_test.go:76: [DEBUG] Testing Case "existing querystring with a prefix"..
PASS
ok  	github.com/Azure/go-autorest/autorest	0.011s
```

* minor clean-up

* token: support for a custom refresh func (#476)

* token: support for a custom refresh func

* pass closures by value

* minor clean-up

* Fix Dropped Errors (#480)

* autorest: fix dropped errror

* autorest/adal: fix dropped test error

* Duration order consistency when multiplying number by time unit (#499)

* Drain response bodies (#432)

The retry helpers and a few other methods weren't reading and closing
response bodies leading to connection leaks.

* Enable exponential back-off when retrying on 429 (#503)

* Enable exponential back-off when retrying on 429

* enforce a 2-minute cap on delays if there isn't one

* updated comment

* fix type-o

* Expose OAuth token provider for use outside autorest (#520)

* feat: extract token creation to public method for MSI auth

* Add getter for token provider on BearerAuthorizer

* Fix Go module ambiguous import errors (#528)

* Fix Go module ambiguous import errors

This is an extension of the mitigations introduced in #455.
Unfortunately, the original mitigations didn't address the primary cause
of ambiguous import errors: the github.com/Azure/go-autorest module.

The issue stems from the fact that old versions of the root module
(github.com/Azure/go-autorest) provide the same packages as the newer
submodules.

To correct this situation, the _root module_ needs to be upgraded to a
version that no longer provides those packages (a version where the
submodules are present). Fortunately, the submodules can be leveraged to
provide the necessary version bump.

See: #414 (comment)

----

Caveat: in order for this to work, an importable version of the root
package needs to be referenceable.

PR #527 makes the root package importable.

The go.mod files assume that this importable version will be
referenceable as v14.2.0. If the version where the importable package is
available ends up being different, these files will need to be updated.

See also: #395, #413, #414, #455, #481, #524

* Update go.sum files

Co-authored-by: Joel Hendrix <jhendrix@microsoft.com>

* Update resourceManagerVMDNSSuffix for AzureUSGovernmentCloud (#531)

* This endpoint changed in AzureChinaCloud (#530)

See from Azurre China portal - this is now cloudapp.chinacloudapi.cn

Co-authored-by: Jin Soon Lim <jilim@microsoft.com>
Co-authored-by: Nick Muller <muller_nicky@hotmail.com>
Co-authored-by: Tom Harvey <tombuildsstuff@users.noreply.github.com>
Co-authored-by: Sam Kreter <samkreter@gmail.com>
Co-authored-by: Delyan Raychev <49918230+draychev@users.noreply.github.com>
Co-authored-by: Patrick Decat <pdecat@gmail.com>
Co-authored-by: Tony Abboud <tdabboud@hotmail.com>
Co-authored-by: Lars Lehtonen <lars.lehtonen@gmail.com>
Co-authored-by: Maxim Fominykh <vominyh@yandex.ru>
Co-authored-by: alespour <42931850+alespour@users.noreply.github.com>
Co-authored-by: Mark Severson <miquella@gmail.com>
Co-authored-by: Panic Stevenson <panic.stevenson@gmail.com>
Co-authored-by: Mauro Giusti <MaurGi@users.noreply.github.com>
jhendrixMSFT added a commit that referenced this pull request Aug 5, 2020
* Deserialize additionalInfo in ARM error

* Allow a new authorizer to be created from a configuration file by specifying a resource instead of a base url.

This enables resource like KeyVault and Container Registry to use an authorizer configured from a configuration file.

* [WIP] Using the Context from the timeout if provided (#315)

* Using the timeout from the context if available

- Makes PollingDuration optional

* Renaming the registration start time

* Making PollingDuration not a pointer

* fixing a broken reference

* Add NewAuthorizerFromCli method which uses Azure CLI to obtain a token for the currently logged in user, for  local development scenarios. (#316)

* Adding User assigned identity support for the MSIConfig authorizor (#332)

* Adding ByteSlicePtr (#399)

* Adding a new `WithXML` method (#402)

* Add HTTP status code response helpers (#403)

Added IsHTTPStatus() and HasHTTPStatus() methods to autorest.Response

* adding a new preparer for `MERGE` used in the Storage API's (#406)

* New Preparer/Responder for `Unmarshalling Bytes` (#407)

* New Preparer: WithBytes

* New Responder: `ByUnmarshallingBytes`

* Reusing the bytes, rather than copying them

* Fixing the broken test / switching to read the bytes directly

* Support HTTP-Date in Retry-After header (#410)

RFC specifies Retry-After header can be integer value expressing seconds
or an HTTP-Date indicating when to try again.
Removed superfluous check for HTTP status code.

* Add support for multi-tenant authentication (#412)

* Add support for multi-tenant authentication

Support for multi-tenant via x-ms-authorization-auxiliary header has
been added for client credentials with secret scenario; this basically
bundles multiple OAuthConfig and ServicePrincipalToken types into
corresponding MultiTenant* types along with a new authorizer that adds
the primary and auxiliary token headers to the reqest.
The authenticaion helpers have been updated to support this scenario; if
environment var AZURE_AUXILIARY_TENANT_IDS is set with a semicolon
delimited list of tenants the multi-tenant codepath will kick in to
create the appropriate authorizer.

* feedback

* rename Options to OAuthOptions (#415)

* Support custom SendDecorator chains via context (#417)

* Support custom SendDecorator chains via context

Added `autorest.WithSendDecorators` and `autorest.GetSendDecorators` for
adding and retrieving a custom chain of SendDecorators to the provided
context.
Added `autorest.DoRetryForStatusCodesWithCap` and
`autorest.DelayForBackoffWithCap` to enforce an upper bound on the
duration between retries.
Fixed up some code comments.

* small refactor based on PR feedback

* remove some changes for dev branch

* merge master into dev (#427)

* v12.3.0 (#418)

* Deserialize additionalInfo in ARM error

* Allow a new authorizer to be created from a configuration file by specifying a resource instead of a base url.

This enables resource like KeyVault and Container Registry to use an authorizer configured from a configuration file.

* [WIP] Using the Context from the timeout if provided (#315)

* Using the timeout from the context if available

- Makes PollingDuration optional

* Renaming the registration start time

* Making PollingDuration not a pointer

* fixing a broken reference

* Add NewAuthorizerFromCli method which uses Azure CLI to obtain a token for the currently logged in user, for  local development scenarios. (#316)

* Adding User assigned identity support for the MSIConfig authorizor (#332)

* Adding ByteSlicePtr (#399)

* Adding a new `WithXML` method (#402)

* Add HTTP status code response helpers (#403)

Added IsHTTPStatus() and HasHTTPStatus() methods to autorest.Response

* adding a new preparer for `MERGE` used in the Storage API's (#406)

* New Preparer/Responder for `Unmarshalling Bytes` (#407)

* New Preparer: WithBytes

* New Responder: `ByUnmarshallingBytes`

* Reusing the bytes, rather than copying them

* Fixing the broken test / switching to read the bytes directly

* Support HTTP-Date in Retry-After header (#410)

RFC specifies Retry-After header can be integer value expressing seconds
or an HTTP-Date indicating when to try again.
Removed superfluous check for HTTP status code.

* Add support for multi-tenant authentication (#412)

* Add support for multi-tenant authentication

Support for multi-tenant via x-ms-authorization-auxiliary header has
been added for client credentials with secret scenario; this basically
bundles multiple OAuthConfig and ServicePrincipalToken types into
corresponding MultiTenant* types along with a new authorizer that adds
the primary and auxiliary token headers to the reqest.
The authenticaion helpers have been updated to support this scenario; if
environment var AZURE_AUXILIARY_TENANT_IDS is set with a semicolon
delimited list of tenants the multi-tenant codepath will kick in to
create the appropriate authorizer.

* feedback

* rename Options to OAuthOptions (#415)

* Support custom SendDecorator chains via context (#417)

* Support custom SendDecorator chains via context

Added `autorest.WithSendDecorators` and `autorest.GetSendDecorators` for
adding and retrieving a custom chain of SendDecorators to the provided
context.
Added `autorest.DoRetryForStatusCodesWithCap` and
`autorest.DelayForBackoffWithCap` to enforce an upper bound on the
duration between retries.
Fixed up some code comments.

* small refactor based on PR feedback

* remove some changes for dev branch

* v12.3.0

* add yaml file for azure devops CI (#419)

* add status badge for azure devops CI (#420)

* enable build and test on linux (#421)

* enable build and test on linux

* fail on first error and use portable std*

* update test to run on devops

* Refactor azure devops pipeline (#422)

Break monolithic script into separate scripts with useful names.
Moved formatting checks to the end with succeededOrFailed conditions.

* remove travis artifacts (#423)

* remove unnecessary trigger section from devops (#424)

* Use accessTokens.json from AZURE_CONFIG_DIR if AZURE_ACCESS_TOKEN_FILE is not set before falling back on ~/.azure/ (#471)

* support for parsing error messages from xml responses (#465)

* support for parsing error messages from xml responses

* fixing the linting

* removed some duplicate code

* fix bug introduced in refactoring

* added XML test and fixed bug it uncovered

* fix godoc comment for methods that are safe for concurrent use (#475)

* New Authorizers for Azure Storage (#416)

* Authorizers for Blob, File, Queue and Table Storage

* Adding a SharedKey authorizer

* refactor based on existing storage implementation

* add missing storage emulator account name

* replace hard-coded strings with constants

* changed to by-ref

* Adding a new Authorizer for SAS Token Authentication (#478)

* Adding a new Authorizer for SAS Token Authentication

This commit introduces a new Authorizer for authenticating with
Blob Storage using a SAS Token

```
$ go test -v ./autorest/ -run="TestSas"
=== RUN   TestSasNewSasAuthorizerEmptyToken
--- PASS: TestSasNewSasAuthorizerEmptyToken (0.00s)
=== RUN   TestSasNewSasAuthorizerEmptyTokenWithWhitespace
--- PASS: TestSasNewSasAuthorizerEmptyTokenWithWhitespace (0.00s)
=== RUN   TestSasNewSasAuthorizerValidToken
--- PASS: TestSasNewSasAuthorizerValidToken (0.00s)
=== RUN   TestSasAuthorizerRequest
--- PASS: TestSasAuthorizerRequest (0.00s)
    authorization_sas_test.go:76: [DEBUG] Testing Case "empty querystring without a prefix"..
    authorization_sas_test.go:76: [DEBUG] Testing Case "empty querystring with a prefix"..
    authorization_sas_test.go:76: [DEBUG] Testing Case "existing querystring without a prefix"..
    authorization_sas_test.go:76: [DEBUG] Testing Case "existing querystring with a prefix"..
PASS
ok  	github.com/Azure/go-autorest/autorest	0.011s
```

* minor clean-up

* token: support for a custom refresh func (#476)

* token: support for a custom refresh func

* pass closures by value

* minor clean-up

* Fix Dropped Errors (#480)

* autorest: fix dropped errror

* autorest/adal: fix dropped test error

* Duration order consistency when multiplying number by time unit (#499)

* Drain response bodies (#432)

The retry helpers and a few other methods weren't reading and closing
response bodies leading to connection leaks.

* Enable exponential back-off when retrying on 429 (#503)

* Enable exponential back-off when retrying on 429

* enforce a 2-minute cap on delays if there isn't one

* updated comment

* fix type-o

* autorest: remove testing.T.Fatal() from test goroutine

* Expose OAuth token provider for use outside autorest (#520)

* feat: extract token creation to public method for MSI auth

* Add getter for token provider on BearerAuthorizer

* Fix Go module ambiguous import errors (#528)

* Fix Go module ambiguous import errors

This is an extension of the mitigations introduced in #455.
Unfortunately, the original mitigations didn't address the primary cause
of ambiguous import errors: the github.com/Azure/go-autorest module.

The issue stems from the fact that old versions of the root module
(github.com/Azure/go-autorest) provide the same packages as the newer
submodules.

To correct this situation, the _root module_ needs to be upgraded to a
version that no longer provides those packages (a version where the
submodules are present). Fortunately, the submodules can be leveraged to
provide the necessary version bump.

See: #414 (comment)

----

Caveat: in order for this to work, an importable version of the root
package needs to be referenceable.

PR #527 makes the root package importable.

The go.mod files assume that this importable version will be
referenceable as v14.2.0. If the version where the importable package is
available ends up being different, these files will need to be updated.

See also: #395, #413, #414, #455, #481, #524

* Update go.sum files

Co-authored-by: Joel Hendrix <jhendrix@microsoft.com>

* Update resourceManagerVMDNSSuffix for AzureUSGovernmentCloud (#531)

* This endpoint changed in AzureChinaCloud (#530)

See from Azurre China portal - this is now cloudapp.chinacloudapi.cn

Co-authored-by: Jin Soon Lim <jilim@microsoft.com>
Co-authored-by: Joel Hendrix <jhendrix@microsoft.com>
Co-authored-by: Nick Muller <muller_nicky@hotmail.com>
Co-authored-by: Tom Harvey <tombuildsstuff@users.noreply.github.com>
Co-authored-by: Sam Kreter <samkreter@gmail.com>
Co-authored-by: Delyan Raychev <49918230+draychev@users.noreply.github.com>
Co-authored-by: Patrick Decat <pdecat@gmail.com>
Co-authored-by: Tony Abboud <tdabboud@hotmail.com>
Co-authored-by: Maxim Fominykh <vominyh@yandex.ru>
Co-authored-by: alespour <42931850+alespour@users.noreply.github.com>
Co-authored-by: Mark Severson <miquella@gmail.com>
Co-authored-by: Panic Stevenson <panic.stevenson@gmail.com>
Co-authored-by: Mauro Giusti <MaurGi@users.noreply.github.com>
jhendrixMSFT added a commit that referenced this pull request Aug 7, 2020
* Deserialize additionalInfo in ARM error

* Allow a new authorizer to be created from a configuration file by specifying a resource instead of a base url.

This enables resource like KeyVault and Container Registry to use an authorizer configured from a configuration file.

* [WIP] Using the Context from the timeout if provided (#315)

* Using the timeout from the context if available

- Makes PollingDuration optional

* Renaming the registration start time

* Making PollingDuration not a pointer

* fixing a broken reference

* Add NewAuthorizerFromCli method which uses Azure CLI to obtain a token for the currently logged in user, for  local development scenarios. (#316)

* Adding User assigned identity support for the MSIConfig authorizor (#332)

* Adding ByteSlicePtr (#399)

* Adding a new `WithXML` method (#402)

* Add HTTP status code response helpers (#403)

Added IsHTTPStatus() and HasHTTPStatus() methods to autorest.Response

* adding a new preparer for `MERGE` used in the Storage API's (#406)

* New Preparer/Responder for `Unmarshalling Bytes` (#407)

* New Preparer: WithBytes

* New Responder: `ByUnmarshallingBytes`

* Reusing the bytes, rather than copying them

* Fixing the broken test / switching to read the bytes directly

* Support HTTP-Date in Retry-After header (#410)

RFC specifies Retry-After header can be integer value expressing seconds
or an HTTP-Date indicating when to try again.
Removed superfluous check for HTTP status code.

* Add support for multi-tenant authentication (#412)

* Add support for multi-tenant authentication

Support for multi-tenant via x-ms-authorization-auxiliary header has
been added for client credentials with secret scenario; this basically
bundles multiple OAuthConfig and ServicePrincipalToken types into
corresponding MultiTenant* types along with a new authorizer that adds
the primary and auxiliary token headers to the reqest.
The authenticaion helpers have been updated to support this scenario; if
environment var AZURE_AUXILIARY_TENANT_IDS is set with a semicolon
delimited list of tenants the multi-tenant codepath will kick in to
create the appropriate authorizer.

* feedback

* rename Options to OAuthOptions (#415)

* Support custom SendDecorator chains via context (#417)

* Support custom SendDecorator chains via context

Added `autorest.WithSendDecorators` and `autorest.GetSendDecorators` for
adding and retrieving a custom chain of SendDecorators to the provided
context.
Added `autorest.DoRetryForStatusCodesWithCap` and
`autorest.DelayForBackoffWithCap` to enforce an upper bound on the
duration between retries.
Fixed up some code comments.

* small refactor based on PR feedback

* remove some changes for dev branch

* merge master into dev (#427)

* v12.3.0 (#418)

* Deserialize additionalInfo in ARM error

* Allow a new authorizer to be created from a configuration file by specifying a resource instead of a base url.

This enables resource like KeyVault and Container Registry to use an authorizer configured from a configuration file.

* [WIP] Using the Context from the timeout if provided (#315)

* Using the timeout from the context if available

- Makes PollingDuration optional

* Renaming the registration start time

* Making PollingDuration not a pointer

* fixing a broken reference

* Add NewAuthorizerFromCli method which uses Azure CLI to obtain a token for the currently logged in user, for  local development scenarios. (#316)

* Adding User assigned identity support for the MSIConfig authorizor (#332)

* Adding ByteSlicePtr (#399)

* Adding a new `WithXML` method (#402)

* Add HTTP status code response helpers (#403)

Added IsHTTPStatus() and HasHTTPStatus() methods to autorest.Response

* adding a new preparer for `MERGE` used in the Storage API's (#406)

* New Preparer/Responder for `Unmarshalling Bytes` (#407)

* New Preparer: WithBytes

* New Responder: `ByUnmarshallingBytes`

* Reusing the bytes, rather than copying them

* Fixing the broken test / switching to read the bytes directly

* Support HTTP-Date in Retry-After header (#410)

RFC specifies Retry-After header can be integer value expressing seconds
or an HTTP-Date indicating when to try again.
Removed superfluous check for HTTP status code.

* Add support for multi-tenant authentication (#412)

* Add support for multi-tenant authentication

Support for multi-tenant via x-ms-authorization-auxiliary header has
been added for client credentials with secret scenario; this basically
bundles multiple OAuthConfig and ServicePrincipalToken types into
corresponding MultiTenant* types along with a new authorizer that adds
the primary and auxiliary token headers to the reqest.
The authenticaion helpers have been updated to support this scenario; if
environment var AZURE_AUXILIARY_TENANT_IDS is set with a semicolon
delimited list of tenants the multi-tenant codepath will kick in to
create the appropriate authorizer.

* feedback

* rename Options to OAuthOptions (#415)

* Support custom SendDecorator chains via context (#417)

* Support custom SendDecorator chains via context

Added `autorest.WithSendDecorators` and `autorest.GetSendDecorators` for
adding and retrieving a custom chain of SendDecorators to the provided
context.
Added `autorest.DoRetryForStatusCodesWithCap` and
`autorest.DelayForBackoffWithCap` to enforce an upper bound on the
duration between retries.
Fixed up some code comments.

* small refactor based on PR feedback

* remove some changes for dev branch

* v12.3.0

* add yaml file for azure devops CI (#419)

* add status badge for azure devops CI (#420)

* enable build and test on linux (#421)

* enable build and test on linux

* fail on first error and use portable std*

* update test to run on devops

* Refactor azure devops pipeline (#422)

Break monolithic script into separate scripts with useful names.
Moved formatting checks to the end with succeededOrFailed conditions.

* remove travis artifacts (#423)

* remove unnecessary trigger section from devops (#424)

* Use accessTokens.json from AZURE_CONFIG_DIR if AZURE_ACCESS_TOKEN_FILE is not set before falling back on ~/.azure/ (#471)

* support for parsing error messages from xml responses (#465)

* support for parsing error messages from xml responses

* fixing the linting

* removed some duplicate code

* fix bug introduced in refactoring

* added XML test and fixed bug it uncovered

* fix godoc comment for methods that are safe for concurrent use (#475)

* New Authorizers for Azure Storage (#416)

* Authorizers for Blob, File, Queue and Table Storage

* Adding a SharedKey authorizer

* refactor based on existing storage implementation

* add missing storage emulator account name

* replace hard-coded strings with constants

* changed to by-ref

* Adding a new Authorizer for SAS Token Authentication (#478)

* Adding a new Authorizer for SAS Token Authentication

This commit introduces a new Authorizer for authenticating with
Blob Storage using a SAS Token

```
$ go test -v ./autorest/ -run="TestSas"
=== RUN   TestSasNewSasAuthorizerEmptyToken
--- PASS: TestSasNewSasAuthorizerEmptyToken (0.00s)
=== RUN   TestSasNewSasAuthorizerEmptyTokenWithWhitespace
--- PASS: TestSasNewSasAuthorizerEmptyTokenWithWhitespace (0.00s)
=== RUN   TestSasNewSasAuthorizerValidToken
--- PASS: TestSasNewSasAuthorizerValidToken (0.00s)
=== RUN   TestSasAuthorizerRequest
--- PASS: TestSasAuthorizerRequest (0.00s)
    authorization_sas_test.go:76: [DEBUG] Testing Case "empty querystring without a prefix"..
    authorization_sas_test.go:76: [DEBUG] Testing Case "empty querystring with a prefix"..
    authorization_sas_test.go:76: [DEBUG] Testing Case "existing querystring without a prefix"..
    authorization_sas_test.go:76: [DEBUG] Testing Case "existing querystring with a prefix"..
PASS
ok  	github.com/Azure/go-autorest/autorest	0.011s
```

* minor clean-up

* token: support for a custom refresh func (#476)

* token: support for a custom refresh func

* pass closures by value

* minor clean-up

* Fix Dropped Errors (#480)

* autorest: fix dropped errror

* autorest/adal: fix dropped test error

* Duration order consistency when multiplying number by time unit (#499)

* Drain response bodies (#432)

The retry helpers and a few other methods weren't reading and closing
response bodies leading to connection leaks.

* Enable exponential back-off when retrying on 429 (#503)

* Enable exponential back-off when retrying on 429

* enforce a 2-minute cap on delays if there isn't one

* updated comment

* fix type-o

* Expose OAuth token provider for use outside autorest (#520)

* feat: extract token creation to public method for MSI auth

* Add getter for token provider on BearerAuthorizer

* Fix Go module ambiguous import errors (#528)

* Fix Go module ambiguous import errors

This is an extension of the mitigations introduced in #455.
Unfortunately, the original mitigations didn't address the primary cause
of ambiguous import errors: the github.com/Azure/go-autorest module.

The issue stems from the fact that old versions of the root module
(github.com/Azure/go-autorest) provide the same packages as the newer
submodules.

To correct this situation, the _root module_ needs to be upgraded to a
version that no longer provides those packages (a version where the
submodules are present). Fortunately, the submodules can be leveraged to
provide the necessary version bump.

See: #414 (comment)

----

Caveat: in order for this to work, an importable version of the root
package needs to be referenceable.

PR #527 makes the root package importable.

The go.mod files assume that this importable version will be
referenceable as v14.2.0. If the version where the importable package is
available ends up being different, these files will need to be updated.

See also: #395, #413, #414, #455, #481, #524

* Update go.sum files

Co-authored-by: Joel Hendrix <jhendrix@microsoft.com>

* Update resourceManagerVMDNSSuffix for AzureUSGovernmentCloud (#531)

* This endpoint changed in AzureChinaCloud (#530)

See from Azurre China portal - this is now cloudapp.chinacloudapi.cn

* allow MSI login with "mi_res_id" (#544)

* allow login with resourceID

* test

* tweaks

* fix

* tested with cmd

* fix unittest

* add new test, remove debug trace

* fix unittest

* fix with url encode

Co-authored-by: Jin Soon Lim <jilim@microsoft.com>
Co-authored-by: Nick Muller <muller_nicky@hotmail.com>
Co-authored-by: Tom Harvey <tombuildsstuff@users.noreply.github.com>
Co-authored-by: Sam Kreter <samkreter@gmail.com>
Co-authored-by: Delyan Raychev <49918230+draychev@users.noreply.github.com>
Co-authored-by: Patrick Decat <pdecat@gmail.com>
Co-authored-by: Tony Abboud <tdabboud@hotmail.com>
Co-authored-by: Lars Lehtonen <lars.lehtonen@gmail.com>
Co-authored-by: Maxim Fominykh <vominyh@yandex.ru>
Co-authored-by: alespour <42931850+alespour@users.noreply.github.com>
Co-authored-by: Mark Severson <miquella@gmail.com>
Co-authored-by: Panic Stevenson <panic.stevenson@gmail.com>
Co-authored-by: Mauro Giusti <MaurGi@users.noreply.github.com>
Co-authored-by: Haitao Chen <haitch@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants