feat(gomod): add support for package replacements #29575

jamietanna · 2024-06-10T12:14:11Z

Changes

Modifies the gomod manager to add support for managing package replacements, if present.

Context

As part of #24883, towards being able to solve #29567, we want to add support for managing package replacement with Renovate.

To make it possible for folks using Go modules to be able to take
advantage of Renovate's package replacements, we should add support to
the gomod manager.

This adds support for managing the following updateTypes:

minor/patch
major
digest

For major and digest updates, we need to support the case that we're
performing a replacement, so add an additional check for the
updateType=replacement and whether there's a newMajor or
newDigest.

Additionally, as we're replacing the dependency's name, we want to have
a separate regex capture group for depName, which we can also use for
determining whether there are quotes around the dependency.

As part of #24883.

Documentation (please check one with an [x])

I have updated the documentation, or
No documentation update is required

How I've tested my work (please select one)

I have verified these changes via:

Code inspection only, or
Newly added/modified unit tests, or
No unit tests but ran on a real repository, or
Both unit tests + ran on a real repository

Example PR: jamietanna/renovate-iss-oapi#2

Note: should the Package column be updated in the PR description to be the new package?

jamietanna · 2024-06-10T16:44:28Z

lib/modules/manager/gomod/update.ts

@@ -36,11 +35,14 @@ export function updateDependency({
    const lineToChange = lines[upgrade.managerData.lineNumber];
    logger.trace({ upgrade, lineToChange }, 'go.mod current line');
    if (
-      !lineToChange.includes(depNameNoVersion) &&
+      !lineToChange.includes(fromPackageNameNoVersion) &&


Should we also check toPackageNameNoVersion here?

It depends if this code gets executed the "second" time (next run) after a replacement. If so it looks like you need to check both

To make it possible for folks using Go modules to be able to take advantage of Renovate's package replacements, we should add support to the `gomod` manager. This adds support for managing the following `updateType`s: - minor/patch - major - digest For `major` and `digest` updates, we need to support the case that we're performing a replacement, so add an additional check for the `updateType=replacement` and whether there's a `newMajor` or `newDigest`. Because we're now managing multiple `depName`s, we should rename this to `fromPackageName` and `toPackageName`, following how we've done this with the `hermit` manager. Additionally, as we're replacing the dependency's name, we want to have a separate regex capture group for `depName`, which we can also use for determining whether there are quotes around the dependency. As part of renovatebot#24883.

rarkins

I haven't reviewed the second half yet

lib/modules/manager/gomod/update.ts

rarkins · 2024-06-12T17:32:56Z

lib/modules/manager/gomod/update.ts

@@ -36,11 +35,14 @@ export function updateDependency({
    const lineToChange = lines[upgrade.managerData.lineNumber];
    logger.trace({ upgrade, lineToChange }, 'go.mod current line');
    if (
-      !lineToChange.includes(depNameNoVersion) &&
+      !lineToChange.includes(fromPackageNameNoVersion) &&


It depends if this code gets executed the "second" time (next run) after a replacement. If so it looks like you need to check both

lib/modules/manager/gomod/update.ts

rarkins · 2024-06-12T17:35:50Z

lib/modules/manager/gomod/update.ts

-          /^(?<depPart>\s+[^\s]+[\s]+[=][>]+\s+)(?<divider>[^\s]+\s+)[^\s]+/,
+          /^(?<depPart>\s+[^\s]+[\s]+[=][>]+\s+)(?<depName>[^\s]+)(?<divider>\s+)[^\s]+/,
        );
      } else {
        updateLineExp = regEx(
-          /^(?<depPart>replace\s+[^\s]+[\s]+[=][>]+\s+)(?<divider>[^\s]+\s+)[^\s]+/,
+          /^(?<depPart>replace\s+[^\s]+[\s]+[=][>]+\s+)(?<depName>[^\s]+)(?<divider>\s+)[^\s]+/,
        );
      }
    } else if (depType === 'require' || depType === 'indirect') {
      if (upgrade.managerData.multiLine) {
-        updateLineExp = regEx(/^(?<depPart>\s+[^\s]+)(?<divider>\s+)[^\s]+/);
+        updateLineExp = regEx(
+          /^(?<depPart>\s+)(?<depName>[^\s]+)(?<divider>\s+)[^\s]+/,
+        );
      } else {
        updateLineExp = regEx(
-          /^(?<depPart>require\s+[^\s]+)(?<divider>\s+)[^\s]+/,
+          /^(?<depPart>require\s+)(?<depName>[^\s]+)(?<divider>\s+)[^\s]+/,


I have concerns about these regex changes breaking something. Unfortunately the existing regex isn't documented

This regex isn't user-facing is it? I believe it's only used in this file, so - from what I can tell - the usages in this file it seems to be OK, any thoughts on how we can work out if there's any impact? What would you be thinking in terms of documentation here?

Co-authored-by: Rhys Arkins <rhys@arkins.net>

jamietanna · 2024-06-12T20:09:51Z

lib/modules/manager/gomod/update.ts

@@ -22,7 +22,7 @@ export function updateDependency({
    // newName will be available for replacement
    const toPackageName = newName ?? fromPackageName;
    // istanbul ignore if: should never happen
-    if (!fromPackageName || !toPackageName || !upgrade.managerData) {
+    if (!fromPackageName || !upgrade.managerData) {


we may need to undo this check to ensure that eslint knows that we've checked this

As part of a prefactor for renovatebot#29575, we should make this change independently, to make the changes in renovatebot#29575 easier to review. This is required as we will be adding replacements for `gomod` and so we'll have both a `fromPackageName` and a `toPackageName`.

jamietanna · 2024-07-04T09:55:39Z

Moved the renaming into #30030 - LMK if there's any more you think we could move out into a separate PR?

jamietanna · 2024-07-08T07:45:47Z

Sorry, re-requested review a bit early - will fix up

E: think we need both vars to do find/replace

jamietanna · 2024-07-20T12:02:53Z

While testing this - here - I've noticed two things:

Package update table

The package updates table presents the old package name with a replacement noted, like so:

Package	Type	Update	Change
github.com/deepmap/oapi-codegen	require	replacement	`v1.12.4` -> `v2.3.0`

Should we instead show the new name?

Package	Type	Update	Change
github.com/deepmap/oapi-codegen -> github.com/oapi-codegen/oapi-codegen	require	replacement	`v1.12.4` -> `v2.3.0`

This is likely a wider change, as it probably impacts other cases where we're doing replacements, so may be worth a separate PR.

`gomodUpdateImportPaths`

Also, I've noticed that the gomodUpdateImportPaths doesn't seem to be run - it looks like this is because of this line in which we only perform this check on a major update, but in this case, we're performing a major version bump.

Thoughts on whether these are things we should try and close off here before this PR goes in?

lib/modules/manager/gomod/update.ts

jamietanna · 2024-08-15T07:24:07Z

Hmm I need to retry jamietanna/renovate-iss-oapi#2 and check it also works when we update the module import path

jamietanna · 2024-08-16T07:02:55Z

OK, so retrying with an update to my test repo, I see jamietanna/renovate-iss-oapi#5 raised which does not take the postUpdateOptions into account 🤔

jamietanna · 2024-08-16T07:10:42Z

Adding support for that looks like it could be a little more involved - separate PR?

jamietanna · 2024-08-16T07:32:29Z

(This isn't yet working - I can't seem to trigger the postUpdateOptions for https://github.com/jamietanna/renovate-iss-oapi

jamietanna · 2024-08-16T17:27:40Z

Any thoughts on why we may not be able to see postUpdateOptions working? 🤔

jamietanna · 2024-09-09T20:04:22Z

lib/modules/manager/gomod/artifacts.ts

@@ -31,7 +31,7 @@ import { getExtraDepsNotice } from './artifacts-extra';
 const { major, valid } = semver;

 function getUpdateImportPathCmds(
-  updatedDeps: PackageDependency[],
+  updatedDeps: Upgrade[],


this isn't quite correct, and why things aren't working

jamietanna · 2024-09-09T20:24:45Z

lib/modules/manager/gomod/artifacts.ts

@@ -31,7 +31,7 @@ import { getExtraDepsNotice } from './artifacts-extra';
 const { major, valid } = semver;

 function getUpdateImportPathCmds(
-  updatedDeps: PackageDependency[],
+  updatedDeps: Upgrade[],
  { constraints }: UpdateArtifactsConfig,
 ): string[] {
  // Check if we fail to parse any major versions and log that they're skipped


For some reason, when we're seeing a replacement, we get the wrong data here 🤔

jamietanna force-pushed the feat/gomod-replace branch from d47e4e9 to be641b7 Compare June 10, 2024 15:56

jamietanna commented Jun 10, 2024

View reviewed changes

jamietanna force-pushed the feat/gomod-replace branch from d52c9ab to cb98d8e Compare June 10, 2024 16:51

jamietanna force-pushed the feat/gomod-replace branch from cb98d8e to bcc7977 Compare June 10, 2024 16:52

jamietanna marked this pull request as ready for review June 10, 2024 16:55

fixup! feat(gomod): add support for package replacements

542a379

rarkins requested changes Jun 12, 2024

View reviewed changes

fixup

bf22bb7

Co-authored-by: Rhys Arkins <rhys@arkins.net>

jamietanna commented Jun 12, 2024

View reviewed changes

jamietanna mentioned this pull request Jul 4, 2024

refactor(gomod): rename depName #30030

Merged

6 tasks

fixup! fixup

3919296

jamietanna requested a review from rarkins July 8, 2024 07:37

jamietanna added 2 commits July 8, 2024 08:47

fixup! fixup! fixup

992478b

sq

5e3ed02

jamietanna marked this pull request as draft July 20, 2024 11:00

jamietanna added 4 commits July 20, 2024 12:03

Merge remote-tracking branch 'origin/HEAD' into feat/gomod-replace

f5e619e

sq

a3b873d

sq

e9dff6a

Merge branch 'main' into feat/gomod-replace

5d10278

jamietanna marked this pull request as ready for review July 20, 2024 11:31

rarkins requested changes Jul 20, 2024

View reviewed changes

lib/modules/manager/gomod/update.ts Outdated Show resolved Hide resolved

lib/modules/manager/gomod/update.ts Outdated Show resolved Hide resolved

jamietanna and others added 3 commits July 20, 2024 18:13

sq

b0f7170

Merge branch 'main' into feat/gomod-replace

6177678

Merge branch 'main' into feat/gomod-replace

91a0342

jamietanna requested a review from rarkins August 13, 2024 16:20

WIP-replace-update

89f4276

viceice approved these changes Aug 16, 2024

View reviewed changes

jamietanna commented Sep 9, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(gomod): add support for package replacements #29575

feat(gomod): add support for package replacements #29575

jamietanna commented Jun 10, 2024 •

edited

Loading

jamietanna Jun 10, 2024

rarkins Jun 12, 2024

rarkins left a comment

rarkins Jun 12, 2024

rarkins Jun 12, 2024

jamietanna Jun 12, 2024

jamietanna Jun 12, 2024

jamietanna commented Jul 4, 2024

jamietanna commented Jul 8, 2024 •

edited

Loading

jamietanna commented Jul 20, 2024 •

edited

Loading

jamietanna commented Aug 15, 2024

jamietanna commented Aug 16, 2024

jamietanna commented Aug 16, 2024

jamietanna commented Aug 16, 2024

jamietanna commented Aug 16, 2024

jamietanna Sep 9, 2024

jamietanna Sep 9, 2024

feat(gomod): add support for package replacements #29575

Are you sure you want to change the base?

feat(gomod): add support for package replacements #29575

Conversation

jamietanna commented Jun 10, 2024 • edited Loading

Changes

Context

Documentation (please check one with an [x])

How I've tested my work (please select one)

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rarkins left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jamietanna commented Jul 4, 2024

jamietanna commented Jul 8, 2024 • edited Loading

jamietanna commented Jul 20, 2024 • edited Loading

Package update table

gomodUpdateImportPaths

jamietanna commented Aug 15, 2024

jamietanna commented Aug 16, 2024

jamietanna commented Aug 16, 2024

jamietanna commented Aug 16, 2024

jamietanna commented Aug 16, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jamietanna commented Jun 10, 2024 •

edited

Loading

jamietanna commented Jul 8, 2024 •

edited

Loading

jamietanna commented Jul 20, 2024 •

edited

Loading

`gomodUpdateImportPaths`