custom data source for ivy with custom pattern

[lenntt]

How are you running Renovate?

Self-hosted

If you're self-hosting Renovate, tell us what version of Renovate you run.

36.7.5

If you're self-hosting Renovate, select which platform you are using.

Bitbucket Server

Was this something which used to work for you, and then stopped?

I never saw this working

Wanted end result.

I want renovate to version bump from my customized data source.
I find the documentation clear and granular, but I think I miss some of the concepts and how to tie things together.

i.e. I don't really understand how to apply a custom datasource, but not change the regex manager (because it seems to find the components and versions quite well in different type of files (.toml, .gradle, ...)

More on my situation:
Unfortunately, I have my own artifactory with a somewhat customized ivy artifact format and scheme and a monolithic build.
As I'm migrating parts to gradle, I'd still like to autobump when ivy legacy components change.

Renovate detects a package and version, but looking up the datasource fails (its interpreted up as maven - and it tries to look for a maven-metadata.xml)

To illustrate, here's how I configured my build.gradle

repositories {
    ivy {
        url "http://MYARTIFACTORY/artifactory/majorversion-ivy-repo"
        allowInsecureProtocol true
        patternLayout {
            artifact '[organisation]/[module]/[revision]/jars/[artifact]-[revision](-[classifier])(.[ext])'
            artifact '[organisation]/[module]/[revision]/schemas/[artifact]-[revision](-[classifier])(.[ext])'
            ivy '[organisation]/[module]/[revision]/ivy.xml'
        }
    }

What you tried so far.

I've tried defining my own custom data source, but it doesn't seem to be used.

I run with RENOVATE_TRACING_CONSOLE_EXPORTER=true npm run renovate > ren.log and try to see if a custom url is lookedup .

I tried this:

    customDatasources: {
        ivy: {
            defaultRegistryUrlTemplate: 'https://MYARTIFACTORY/artifactory/majorversion-ivy-repo/blabla/{{ groups }}/{{depName}}',
            transformTemplates: [
                '{"releases":[{"version": $$.(data.children.uri),data[id = \'stable\'].latest])}]}',
            ]
        },

Notice I added 'blabla', but In the logging, I don't see any urls with 'blabla' being hit.

more specifics:

renovate tries to get versions from: http://MYARTIFACTORY/artifactory/majorversion-ivy-repo/com/my/org/my-package/maven-metadata.xml

I found artifactory's storage api return json, so I think I could (ab)use (/transform) at this url: https://majorversion-ivy-repo/artifactory/api/storage/com/my/org/my-package

Relevant debug logs

Logs

The feedback in the PR is the most evident

....
:warning: Dependency Lookup Warnings :warning:
Please correct - or verify that you can safely ignore - these lookup failures before you merge this PR.
Failed to look up maven package com.my.org:my-package
Files affected: build.gradle

[secustor]

The gradle manager does define the datasource to use.
Therefore you have to use the regex manager to manually extract these changes.

Further defaultRegistryUrlTemplate will only be used if the manager is not providing a registryURL from the extraction.
Also group is currently not a valid template variable.

If this doesn't clear it up please provide a repository with your configuration and an example grade file.

[lenntt]

Thanks @secustor ,
I wasn't sure if it's the gradle manager or the maven manager in this case.

Therefore you have to use the regex manager to manually extract these changes.

Is there a possibility to extend the gradle manager, rather than a new entry in the regex manager (?) (which requires fields like fileMatch and matchStrings - can I 'copy' these from somewhere for gradle?)?

Further defaultRegistryUrlTemplate will only be used if the manager is not providing a registryURL from the extraction.

I'm still confused by this, I'm setting a registryURL in the package rules, does that mean, the defaultRegistryUrlTemplate will not be used?

[secustor]

What you can try is this:

{
  "packageRules": [
    {
      "matchManagers": ["gradle"],
      "datasource": "custom.ivy",
      "registryUrl": null
    }
  ]
}

Tough I'm not sure if this will work.

I'm still confused by this, I'm setting a registryURL in the package rules, does that mean, the defaultRegistryUrlTemplate will not be used?

Yes the defaultRegistryUrl is the fall back if no registryUrl is provided.

[Churro]

Tough I'm not sure if this will work.

I doubt it because the gradle manager is currently not matching ivy { url ... } patterns. Adding support for this would be trivial but repositories would be mapped to the maven datasource. An override is possible (as suggested by your package rule) but the remaining question is what will happen with all cases, where users don't specify a custom ivy datasource.

The docs say that ivy is generally compatible with maven metadata but I don't know if maven sources in ivy repos are the exception or the norm. Any clues?