(datasource/custom): Always uses token abc instead of Bearer abc to the Authorization header

[Shegox]

How are you running Renovate?

Self-hosted Renovate

If you're self-hosting Renovate, tell us what version of Renovate you run.

36.107.2/93ab46e37e26e1e0d48e9d9f2a19041372ca7093

If you're self-hosting Renovate, select which platform you are using.

GitHub Enterprise Server

Was this something which used to work for you, and then stopped?

It used to work, and then stopped

Describe the problem

I've been using custom data sources with a JSON-API for a couple of weeks. However, I've noticed an issue following commit fadf1f2 (related to discussion #24377). Since this update, all requests to my custom data source now use the token prefix in the authorization-header when being executed.

The root issue appears to be that GITEA_API_USING_HOST_TYPES now includes the custom type. This leads to Renovate using the token prefix, assuming that it's interfacing with a GITEA compatible host.

Here is the relevant code snippet:

renovate/lib/util/http/auth.ts

Lines 31 to 36 in 69f7240

	if (options.token) {
	if (
	options.hostType &&
	GITEA_API_USING_HOST_TYPES.includes(options.hostType)
	) {
	options.headers.authorization = `token ${options.token}`;

Removing custom from GITEA isn't a viable solution as it would then proceed into the GITHUB if-branch, which also results in unexpected formatting of the authorization-header.

My Renovate configuration is as follows:

const config = {
    hostRules: [
        {
            matchHost: "https://api.gardener.company.com",
            token: "<gardenerToken>",
        }
    ],
    customDatasources: {
        "gardener-k8s-supported": {
            defaultRegistryUrlTemplate: 'https://api.gardener.company.com/apis/core.gardener.cloud/v1beta1/cloudprofiles/aws',
            transformTemplates: [
                `{"releases": $$.spec.kubernetes.versions[classification = 'supported'].{"version": $.version, "sourceUrl":"https://github.com/kubernetes/kubernetes", "changelogUrl":$join(["https://github.com/kubernetes/kubernetes/releases/tag/",$.version])},"sourceUrl": "https://github.com/kubernetes/kubernetes","homepage": "https://github.com/kubernetes/kubernetes"}`
            ],
        },
        "gardener-k8s-available": {
            defaultRegistryUrlTemplate: 'https://api.gardener.company.com/apis/core.gardener.cloud/v1beta1/cloudprofiles/aws',
            transformTemplates: [
                `{"releases": $$.spec.kubernetes.versions.{"version": $.version, "sourceUrl":"https://github.com/kubernetes/kubernetes", "changelogUrl":$join(["https://github.com/kubernetes/kubernetes/releases/tag/",$.version])},"sourceUrl": "https://github.com/kubernetes/kubernetes","homepage": "https://github.com/kubernetes/kubernetes"}`
            ],
        },
    }
}

I propose adding the following tests to check for the expected behavior in lib/util/http/auth.spec.ts:

    it('defaults to Bearer for custom datasource', () => {
      const opts: GotOptions = {
        headers: {},
        token: 'customToken',
        hostType: 'custom',
      };
      applyAuthorization(opts);

      expect(opts).toMatchInlineSnapshot(`
        {
          "headers": {
            "authorization": "Bearer customToken",
          },
          "hostType": "custom",
          "token": "customToken",
        }
      `);
    });

    it('uses authType for custom datasource', () => {
      const opts: GotOptions = {
        headers: {},
        token: 'customToken',
        hostType: 'custom',
        context: {
          authType: 'Basic',
        },
      };
      applyAuthorization(opts);

      expect(opts).toMatchInlineSnapshot(`
        {
          "context": {
            "authType": "Basic",
          },
          "headers": {
            "authorization": "Basic customToken",
          },
          "hostType": "custom",
          "token": "customToken",
        }
      `);
    });

Currently, the authorization header always contains token as shown in the image below, which works great for GitHub, GITEA and GitLab, but breaks for any other generic API.

Relevant debug logs

Logs

No logs provided; the logs didn't provide any meaningful insights into this.

Have you created a minimal reproduction repository?

I have explained in the description why a minimal reproduction is impossible

[rarkins]

That's a nasty bug. The _USING_HOST_TYPES lists are a nasty workaround to begin with. Do you mean that you have a fix ready to propose?

[Shegox]

I don't have a patch ready, I haven't found a (good) solution yet, which would support both:

• using custom "authorization header creation" for GitHub/Gitea/GitLab
• using no custom header creation for other endpoint

My idea/vision would maybe something to allow an user to specify a hostType in customDatasource and this is then used to determine how the header is formatted. E.g. if you use following it would use the github authorization formatting.

        "gardener-k8s-supported": {
            hostType: "github",
            defaultRegistryUrlTemplate: 'https://api.gardener.company.com/apis/core.gardener.cloud/v1beta1/cloudprofiles/aws',

            transformTemplates: [
                `{"releases": $$.spec.kubernetes.versions[classification = 'supported'].{"version": $.version, "sourceUrl":"https://github.com/kubernetes/kubernetes", "changelogUrl":$join(["https://github.com/kubernetes/kubernetes/releases/tag/",$.version])},"sourceUrl": "https://github.com/kubernetes/kubernetes","homepage": "https://github.com/kubernetes/kubernetes"}`
            ],
        },

But open for other ideas as well, as this is as well kinda hacky.

[secustor]

WDYT about a similar route as we have gone with custom datasources some predefined host types and the ability to define custom ones.

{
  "hostTypes": {
     "myType": {
        "headers": {
          "authorization": "Bearer {{ token }}"
        }
     }
  },
 "customDatasources": {
    "usingBuildInType": {
       "hostType": "github",
        "defaultRegistryUrlTemplate": "https://api.gardener.company.com/apis/core.gardener.cloud/v1beta1/cloudprofiles/aws",
         "transformTemplates": [
            "..."
          ]
    },
    "usingCustomType": {
       "hostType": "custom.myType",
        "defaultRegistryUrlTemplate": "https://api.gardener.company.com/apis/core.gardener.cloud/v1beta1/cloudprofiles/aws",
         "transformTemplates": [
            "..."
          ]
    }
  }
}

[rjacks24]

Hi @secustor, Is this fix for the datasources still something that is being considered? I'm not sure how to implement this or I'd help, but I sure am interested in getting it solved.

[secustor]

It is one of the possible solutions with #25677, but it has not been decided on which to implement.

Regarding getting this solved, I think configuring on hostRules is easier to implement. Tough this will not as easy as it looks on the surface as we have have also to consider security concerns like users of the APP overwriting some headers when Renovate reaches out to APIs.

Implementation wise there is the possibility to contribute it yourself or sponsor the feature.

[MvRoo]

This bug also impact calls into the Azure Graph API. We were trying to retrieve a version list from https://management.azure.com/subscriptions/[sub-id]/resourceGroups/[rg]/providers/Microsoft.Compute/galleries/[gallery-name]images/{{{depName}}}/versions?api-version=2022-08-03. This expects the authorization header with Bearer. Reverting back to a version before fadf1f2 fixed it for us.

Of course we'd like to run on the latest version of renovate, so very keen to see a fix for this.