Update github hashes in npm package.json files

[piotr-s-brainhub]

Sometimes I have dependencies in my package.json defined as a link to a git (GitHub, BitBucket or GitLab) repo, possibly with a branch name or a hash.

So it would be nice if renovate allowed me two options:

• replace git dependencies into NPM dependencies
• update git dependencies to the newest hash

[rarkins]

• replace git dependencies into NPM dependencies

We don't plan to support this automatically, because there's too many ways such PRs could be wrong/unwelcome. We assume most people will use a GitHub dependency instead of npm for a good reason, and we may not know it.

• update git dependencies to the newest hash

This is possible. Can you collect the possible syntaxes that would need to be supported?

[piotr-s-brainhub]

"@brainhubeu/gatsby-docs-kit": "RobertHebel/gatsby-docs-kit.git#upgrade-gatsby-to-v2",
"@brainhubeu/gatsby-docs-kit": "https://github.com/RobertHebel/gatsby-docs-kit.git#upgrade-gatsby-to-v2",
"react-native-camera": "git+https://git@github.com/react-native-community/react-native-camera",

at the begin:

• git+https://git@github.com/
• https://github.com/
• nothing so it's GitHub as default

at the end:

• hash
• branch name
• nothing so it fetches from the newest hash of the default branch

[rarkins]

How could Renovate know what to upgrade RobertHebel/gatsby-docs-kit.git#upgrade-gatsby-to-v2 to though? That seems like a very specific branch and not something to be updated. I had expected more like RobertHebel/gatsby-docs-kit.git#abcdef0 which then could be handled as so:

• Check the latest commit on the default branch
• If it's not abcdef0 then update it to the latest commit

Adding a commit hash to a URL that doesn't have one is a good idea, but definitely something we'd want people to opt into. We refer to updating digests as "digest updating" and adding digests as "digest pinning" btw.

[piotr-s-brainhub]

it wasn't a good example, the hash should be updated only if there's already a hash

[rarkins]

Phase 1: If a hash exists then update it
Phase 2: If no hash or commitish reference exists but pinDigests is enabled, then add a commit hash

[m4theushw]

I have this same need. In my case, I don't need support for commit hash but only the branch name at the end of the URL, which might not be the default one. I started to look on how to support this kind of dependency and we could reuse the git-refs datasource. The locked version set by the lock file should be bypassed or Renovate won't update. May I open a PR with an initial implementation of the support for Git URLs with a branch at the end?

[rarkins]

@m4theushw can you create a reproduction first and show manually what the PR should look like? Then I can advise if a PR would be a good idea.

[github-actions]

Hi there,

Help us by making a minimal reproduction repository.

Before we can start work on your issue we first need to know exactly what's causing the current behavior. A minimal reproduction helps us with this.

To get started, please read our guide on creating a minimal reproduction to understand what is needed.

We may close the issue if you (or someone else) have not provided a minimal reproduction within two weeks. If you need more time, or are stuck, please ask for help or more time in a comment.

Good luck,

The Renovate team

[m4theushw]

@rarkins Here's the reproduction: https://github.com/m4theushw/renovate-git-urls

The real application using this dependency is https://github.com/mui-org/material-ui-x/blob/master/package.json#L68

To add support for branch-based git urls, the first step would be to replace the skipReason below with the information for the dependency. The datasource would be set to git-refs. For the versioning, I'm not sure how far git could be reused. The versioning needed here is similar to docker.

renovate/lib/manager/npm/extract/index.ts

Line 326 in 4e69bfa

dep.skipReason = SkipReason.UnversionedReference;

The next step is to make sure that the versioning.matches will return true when "branchname" is given to it. By default, the git versioning always returns false.

renovate/lib/workers/repository/process/lookup/index.ts

Line 135 in 4e69bfa

(v) => unconstrainedValue || versioning.matches(v.version, currentValue)

I stopped here but additional steps may be needed.

[rarkins]

Can you manually create the PR you expect? I'm not sure what you're expecting because it's set to a non-versioned branch name

[oliviertassinari]

Can you manually create the PR you expect?

@rarkins This is the PR we would expect https://github.com/mui-org/material-ui-x/pull/3526/files.

How could Renovate know what to upgrade RobertHebel/gatsby-docs-kit.git#upgrade-gatsby-to-v2 to though?

I think that the answer on this one from @piotr-s-brainhub would be the latest git hash commit this branch refers to.

[viceice]

That's a lockfile only update. Use lockfile maintenance for this.

[m4theushw]

@viceice The dependency is not versioned for lockfile maintenance to work. There's a branch name at the end of the URL: https://github.com/RobertHebel/gatsby-docs-kit.git#upgrade-gatsby-to-v2. We need that Renovate upgrade the resolved value to be the last commit hash in the specified branch.

[viceice]

Lockfile maintenance should work, as renovate simply deletes the lockfile and let yarn re-create, so yarn should resolve to latest commit.

Renovate never directly changes the lockfile.

[m4theushw]

I added a versioned dependency (lodash in this case) to my test repo and enabled lockfile maintenance. Renovate created two PRs: one to update lodash and another "Lock file maintenance" updating the resolved hash of the git url. The problem is that the lockfile maintenance PR is also updating lodash.

Is there a way to update via lockfile maintenance only those dependencies whose a dedicated PR can't be created? Sometimes the update of a single dependency needs intervention to fix the CI and handling all updates at once (in the lockfile maintenance PR) is not feasible.

[oliviertassinari]

@m4theushw I approached it differently in mui/mui-x#3500

[m4theushw]

Any update here?

[rarkins]

A new reproduction is needed here