Add ability to support IAM roles for Service Accounts(IRSA) within Loki

[p1-repo-sync-bot]

Bug

Description

If you define an IAM Role for S3 access and have Loki create a service account with the proper annotations you are unable to access the S3 buckets due to the requirement by the helm charts to have an ENDPOINT configured. Loki will not use the service account if it sees an endpoint configured and will require AccessKey which is strongly discouraged by AWS in favor of IRSA.
Currently Big Bang will force Minio if the endpoint is defined even if a region is defined. It should allow either a region and/or endpoint.

Note a workaround currently used by me is to define the endpoints under objectStorage and override this via the loki.existingSecretForConfig that has the config.yaml without the endpoint.

To reproduce this issue:
Create a IAM Role with an inline policy to allow S3 access to your buckets.
Configure Loki within BB as follows:

loki:
  enabled: true
  strategy: "scalable"
  objectStorage:
    endpoint: s3-us-gov-west-1.amazonaws.com
    region: us-gov-west-1
    bucketNames:
      chunks: mybucket
      ruler: mybucket
  values:
    serviceAccount:
      create: true
      annotations:
        eks.amazonaws.com/role-arn: "arn:aws-us-gov:iam::<redacted>:role/loki-s3-role"
    write:
      replicas: 1
      persistence:
        size: 50Gi
    read:
      replicas: 1
      persistence:
        size: 50Gi

The loki services will not be able to connect to the buckets. Note if you create a configuration as above and leave out the endpoint, the helm chart will not deploy. If you instead add

  values:
    loki:
      existingSecretForConfig: loki-config-sercet

And create the above secret that excludes the endpoint from the config it will succeed.

BigBang Version

1.52