[cloud-azure] Allow multiple repositories with different settings

[dadoonet]

As for now, azure cloud plugin only supports one single Azure configuration (account and key) as those settings are provided in elasticsearch.yml:

cloud:
    azure:
        storage:
            account: your_azure_storage_account
            key: your_azure_storage_key

In some cases, there is a need to have multiple azure connections, as explained for example at elastic/elasticsearch-cloud-azure#93

For this, we need to support providing settings when the repository is created:

PUT _snapshot/my_azure_repository1
{
    "type": "azure",
    "settings": {
        "account": "AZURE_ACCOUNT1",
        "key": "AZURE_KEY1"
    }
}

PUT _snapshot/my_azure_repository2
{
    "type": "azure",
    "settings": {
        "account": "AZURE_ACCOUNT2",
        "key": "AZURE_KEY2"
    }
}

If no account is defined in the repository creation call, then we fallback to the one in elasticsearch.yml and if no account is available, then we fail the creation.
Same for key parameter.

This will also fix #12446.

[dadoonet]

@clintongormley so based on your #12760 (comment), I guess we just don't want to support what I wrote in the above description:

Same for key parameter.

It would mean that we only support this:

PUT _snapshot/my_azure_repository2
{
    "type": "azure",
    "settings": {
        "account": "AZURE_ACCOUNT2"
    }
}

Correct?

[clintongormley]

i think that's right.

[craigwi]

The pull request I submitted back in June solves this: elastic/elasticsearch-cloud-azure#93. I learned on a call today that my release notes, which describes this in detail, was not surfaced well.

I added the release notes to the pull request: elastic/elasticsearch-cloud-azure#93.

Craig.

[dadoonet]

@craigwi see also discussion which happened at #12760

[craigwi]

The idea of chained repositories in #12760 is not a replacement for the feature discussed here and implemented by my pull request.

Azure storage accounts, when set up with replication of "READ-ACCESS GEO REDUNDANT" automatically replicate to a specific secondary region. In the case of a storage account "in" US West, the secondary region is (always) US East. To read from the secondary endpoint for the storage account one specifies a parameter LocationMode.SECONDARY_ONLY the APIs (in BlobRequestionOptions). Access to the storage account through to the secondary endpoint is always READ ONLY.

I have cases when I need to use different combinations of one storage account or the other and different values of LocationMode. Given other patterns I have seen in ES, it seemed simplest to enable both multiple storage accounts to be specified in the yml file AND any combination of those storage accounts with the location mode at the time the repository is registered.

I hope this helps to clarify the feature I built.

Craig.

[craigwi]

I created a pull request #13228 containing the changes I had made to the elasticsearch-cloud-azure plugin.

Craig.

[dadoonet]

This has been fixed by #13779