Support Go Modules

[garyburd]

This repo was moved from here from garyburd/redigo. At the time of the move, I made some minor breaking changes to the API and tagged a v2.0.0 release.

The v2.0.0 release turned out to be a problem when Go Modules were introduced. It's now clear that I should have deleted previous v1.x.x releases from the repo and tagged a new v1.0.0 release.

I was my intent that this repo have one major semantic version until the next breaking API change is made. Let's call the major semantic version as of today as version N.

It's possible that users who employ a dependency management tool are using the tags copied over from garyburd/redigo. These users are on major version N-1. Edit: These versions do not compile because they have a package import comment for garyburd/redigo. It's safe to assume that there are no users of these versions.

Users of Go 1.5+ who do not employ a dependency management tool should continue to get the latest of major version N. This works today and should continue to be supported until Go Modules is widely adopted.

Users who opt into Go Modules should not need to change their code.

Because I expect Go Modules to be widely adopted before major version N+1 of this package is created, it's acceptable to require a dependency management tool to get version N+1.

Adding a go.mod file to a v2.x.x release changes the path of the package to github.com/gomodule/redigo/v2. This requires a change to all importers of the packages and does not work for users on Go 1.8 and earlier.

See previous discussion in #365 and #363.

[huaguoping]

Hi, i can not build. get the error:

/workpath/go/pkg/mod/github.com/garyburd/redigo@v2.0.0+incompatible/redis/pool.go:28:2: use of internal package github.com/gomodule/redigo/internal not allowed

And i got 2 redigo in my go.mod...

require (
...
github.com/garyburd/redigo v2.0.0+incompatible
github.com/gomodule/redigo v2.0.0+incompatible // indirect
...
)

go version: v1.11

[garyburd]

@huaguoping Use github.com/garyburd/redigo v1.6.0 instead of v2.0.0. Better yet, use this package.

[huaguoping]

@huaguoping Use github.com/garyburd/redigo v1.6.0 instead of v2.0.0.

Thx. 👍

[jrefior]

Per "semantic import versioning", introduced with go modules in go 1.11, incrementing the major version of a module above 1 requires a new import path. See:

In semantic versioning, changing the major version number indicates a lack of backwards compatibility with earlier versions. To preserve import compatibility, the go command requires that modules with major version v2 or later use a module path with that major version as the final element. For example, version v2.0.0 of example.com/m must instead use module path example.com/m/v2, and packages in that module would use that path as their import path prefix, as in example.com/m/v2/sub/pkg. Including the major version number in the module path and import paths in this way is called "semantic import versioning".

Module compatibility and semantic versioning

The preferred way to do this is usually to create a v2 branch, append /v2 to the module name (first line of the go.mod file), update import paths to include the /v2, and tag a new v2.x.x release. That should be all that's required. Imports that leave off the v2 will get the highest v1.x.x availlable, while imports that include the v2 will get the highest v2.x.x available.

See From Repository to Modules.

[elithrar]

The challenge is that the repository is already v2, because it was tagged as such well before Modules existed (and they are still not “final”). You could publish a v2.0.1 and update the import path, but at risk of breaking existing users & module users alike.

…

On Fri, Nov 30, 2018 at 3:07 PM John Refior ***@***.***> wrote: Per "semantic import versioning", introduced with go modules in go 1.11, incrementing the major version of a module above 1 requires a new import path. See: In semantic versioning, changing the major version number indicates a lack of backwards compatibility with earlier versions. To preserve import compatibility, the go command requires that modules with major version v2 or later use a module path with that major version as the final element. For example, version v2.0.0 of example.com/m must instead use module path example.com/m/v2, and packages in that module would use that path as their import path prefix, as in example.com/m/v2/sub/pkg. Including the major version number in the module path and import paths in this way is called "semantic import versioning". Module compatibility and semantic versioning <https://golang.org/cmd/go/#hdr-Module_compatibility_and_semantic_versioning> The preferred way to do this is usually to create a v2 branch, append /v2 to the module name (first line of the go.mod file), update import paths to include the /v2, and tag a new v2.x.x release. That should be all that's required. Imports that leave off the v2 will get the highest v1.x.x availlable, while imports that include the v2 will get the highest v2.x.x available. See From Repository to Modules <https://research.swtch.com/vgo-module>. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#366 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABIcDYwhnhJPPlm9NVo3g1__LnXc1_iks5u0Y_kgaJpZM4W31s-> .

[jrefior]

Requirement: Existing users do not need to change code or tools to continue using the current version of the package. This includes users on Go 1.8 and earlier, users who fetch the code with go get, etc.

Glide allows Go 1.5+ users to lock packages at specific versions. But if I understand this requirement correctly, you are also trying to support users of Go 1.8 and earlier who do not employ a dependency management tool, and will always just get "latest" with go get -- is that correct?

Does the current "latest" work for them, or are you trying to move them back to some previous version via the next release?

While I personally prefer the v2 branch mechanism of moving to v2 with go modules, in some situations it only works if you are following semantic versioning and assuming that users of older versions of Go can/will/should obtain an older release tag that works for their version. Here it sounds like you want to be backward compatible even when that assumption is not true (what Go creators wanted in the beginning).

Then again, you have somewhat isolated internal import paths that would be affected by a /v2 import path change. Garyburd proposed a solution in #363 (link). At this point it looks promising to me, but I don't think I know enough about earlier versions of this repository and support for users of earlier versions of Go to be sure whether that gets you everything you want or not. Has the community determined whether it does?

Regarding support for Go modules in that proposal, I think as long as your go.mod defines the name as github.com/gomodule/redigo/v2 in your next v2.x.x release (tagged from some "v2" branch -- the branch names don't affect the behavior of the go tool), and import paths within the repo are updated to use it, I think you're good for "v2". Ideally I think there would also be a v1.x.x release tagged from another branch with a go.mod that defines the name without the /v2, and has appropriate import paths, to support "v1".

Alternatively, I think the directory approach to moving to v2 with go modules probably does meet your goals (if I understand the situation correctly). As Russ Cox described it:

As an alternative, vgo also supports a “major subdirectory” convention, in which major versions above v1 are developed in subdirectories. In this case, v2.0.0 is created not by forking the whole tree into a separate branch but by copying it into a subdirectory. Again the go.mod must be updated to say "my/thing/v2". Afterward, v1.x.x tags pointing at commits address the files in the root directory, excluding v2/, while v2.x.x tags pointing at commits address the files in the v2/ subdirectory only. The go.mod file lets vgo distinguishes the two cases. It would also be meaningful to have a v1.x.x and a v2.x.x tag pointing at the same commit: they would address different subtrees of the commit.

We expect that developers may feel strongly about choosing one convention or the other. Instead of taking sides, vgo supports both. Note that for major versions above v2, the major subdirectory approach may provide a more graceful transition for users of go get. On the other hand, users of dep or vendoring tools should be able to consume repositories using either convention. Certainly we will make sure dep can.

See From Repository to Modules

In that case, you could choose (by leaving the code outside the v2 directory) which version of code will be presented as "v1": what users will get on a simple "go get" without any dependency management tool; and also what users of modern versions of Go employing a dependency management tool will get if they don't use the v2 import path.

Then inside the v2 directory you could place the code that supports modern versions of Go for users choosing to move to "v2" by specifying the v2 import path.

[garyburd]

The current latest works Go 1.5 and up.

…

On Sun, Dec 2, 2018 at 6:37 AM John Refior ***@***.***> wrote: Existing users do not need to change code or tools to continue using the current version of the package. This includes users on Go 1.8 and earlier, users who fetch the code with go get, etc. Glide allows Go 1.5+ users to lock packages at specific versions. But if I understand this requirement correctly, you are also trying to support users of Go 1.8 and earlier who do not employ a dependency management tool, and will always just get "latest" with go get -- is that correct? Does the current "latest" work for them, or are you trying to move them back to some previous version via the next release? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#366 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AACCg3iWVKvJ0eyPrbsRhxCSiHvqBixFks5u0-WSgaJpZM4W31s-> .

[garyburd]

@jrefior: I updated the issue to clarify where we are today and what I see as the requirements. Let me know if that does not answer your questions.

All: I deleted comments regarding an unrelated issue.

Does it work to orphan v2.0.0 and continue continue releases on v1.x.x where a go.mod file can be added seamlessly?

[jrefior]

You've mentioned orphaning v2.0.0. If you leave the v2.0.0 tag in place and continue development on v1.x.x, someone or some dependency management tool will pull v2.0.0 when trying to get latest. So you'll have to delete the v2.0.0 tag, and create a new 1.x.0 tag in it's place.

Here I tried to do that. I forked the repository, deleted the v2.0.0 tag (git push --delete origin v2.0.0), added a go.mod file, and tagged a new v1.7.0 release: https://github.com/jrefior/redigo . You can try pulling it down or using various dependency management tools with different version specifications for testing if you like.

If you're willing to delete the v2.0.0 tag, then this may accomplish more of your goals than other solutions. Users may have to make changes in their dependency management configuration, but not to their code. Otherwise at some point users of new versions of Go will have to change their import path to include the /v2.

What do you think?

[elithrar]

What happens to users who pulled v2.0.0 already, and have it in their dependency manifest?

…

On Sat, Jan 19, 2019 at 11:23 AM John Refior ***@***.***> wrote: You've mentioned orphaning v2.0.0. If you leave the v2.0.0 tag in place and continue development on v1.x.x, someone or some dependency management tool will pull v2.0.0 when trying to get latest. So you'll have to delete the v2.0.0 tag, and create a new 1.x.0 tag in it's place. Here I tried to do that. I forked the repository, deleted the v2.0.0 tag (git push --delete origin v2.0.0), added a go.mod file, and tagged a new v1.7.0 release: https://github.com/jrefior/redigo . You can try pulling it down or using various dependency management tools with different version specifications for testing if you like. If you're willing to delete the v2.0.0 tag, then I think this accomplishes more of your goals than any other solution. Otherwise at some point users of new versions of Go will have to change their import path to include the /v2. But if you're willing to require the /v2 import path to get the latest code -- and not specifying v2 will just give you the highest v1.x.x version -- I think the branch model with build constraints will also work. What do you think? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#366 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABIcFYcGZ_mM6wmm_3R06P2zCx7HFZZks5vE3CvgaJpZM4W31s-> .

[jrefior]

Right, that's the question, because that's where there's possible downside. And the reason for doing some testing with different dependency management tools.

So for example, if I'm on go 1.11.4, and I "restore" v2.0.0 by deleting the go.mod file and committing and tagging, then import jrefior/redigo from some other project via go get, I'll see this:

$ go get github.com/jrefior/redigo/redis
go: finding github.com/jrefior/redigo/redis latest
go: finding github.com/jrefior/redigo v2.0.0+incompatible
go: downloading github.com/jrefior/redigo v2.0.0+incompatible
$ cat go.mod
module opensource/test01

require github.com/jrefior/redigo v2.0.0+incompatible // indirect
$ cat go.sum
github.com/jrefior/redigo v2.0.0+incompatible h1:DnvDEZJV+Z/ulWZhD5r/L7/bziq203lzvd2AUi5C9S8=
github.com/jrefior/redigo v2.0.0+incompatible/go.mod h1:msepy20fPwm2qIMjp3xi0iLG66J+DoYk66NT0Jt7Oso=

Now when I delete tag v2.0.0, and commit the go.mod file again and tag v1.8.0, now I try to build my test project again:

$ go build
$

The build just works. But that's because it's loading jrefior/redigo from my module cache ($GOPATH/pkg/mod). If I set a new module cache by changing my gopath, trying to build errors:

$ go build
go: finding github.com/jrefior/redigo v2.0.0+incompatible
go: github.com/jrefior/redigo@v2.0.0+incompatible: unknown revision v2.0.0
go: error loading module requirements

And the binary is not built. So then the user has to take some action to resolve the problem. In this situation a go get or go get -u won't resolve it, even if you specify the new version you want. Your only solution is to manually edit the go.mod file (either change the version or delete the requirement and let the go tool discover it again on build).

In my view, that's not a great user experience. This test could be repeated with glide and dep and other dependency management tools.

Unfortunately I think trying to continue with development/maintenance on v1.x.x only without deleting v2.0.0 is probably worse, as many users trying to get latest will inadvertently wind up with v2.0.0, failing silently to get the latest code.

To avoid both of those you can set the module name to github.com/gomodule/redigo/v2 and require go-module-aware users to add the v2 to their import path to get v2.x.x code, if you're willing to do that.

[garyburd]

How about this:

• Add go.mod file with content:

   module github.com/gomodule/redigo/v2
  

• Change tests to import github.com/gomodule/redigo/v2

• Tag a new v2.x.x. release

• Create a v1 branch and change go.mod on the branch to:

   module github.com/gomodule/redigo
  

• Tag v1.x.x release on v1 branch.

• Write script to:

  • Tag v2.x.x release
  • Merge changes from master to v1 branch
  • Tag v1.x.x release on the v1 branch

Tests will fail in the v1 branch. Everything else should work.

[urandom2]

Since it appears you intend for most consumers to use the v1.7.0 tag, not the old v2.0.0 tag, what if you release v1.7.1 with module github.com/gomodule/redigo and on a separate, say v2 branch, release v2.0.1 with module github.com/gomodule/redigo/v2.

I would imagine that the v2.0.0 codebase is not being supported, so this proposal serves to have module customers ignore the v2 line unless they ask for it specifically. The only downside is that it locks you into v2 api compatibility, but I think the v2.0.0 tag already does that.

If this is a viable solution, I think #413 can serve for the v1.7.1 change, would you like me to spin up another change for v2.0.1?

[garyburd]

@arnottcr Because it was a mistake to create the v2 tag, I'd like to ignore it. The problem is that people will assume that v2 is better than v1. The status of v2 can be explained in the README, but that will only help those who actually read the file.

Perhaps the best option is to maintain v1 and v2 in parallel with v1 as the main line of development, A script can be written to merge changes from v1 to v2 and create the v2 tags as needed. This is the opposite of what I described here.

[urandom2]

That makes sense, I think in that case you may want to consider the major subdirectory option and use type aliases in conjunction with a tool like goforward to make things easier on yourself. The only breakage you could see, is for people using tools like dep, glide, etc. who are currently consuming the v2.x.x line from the import github.com/gomodule/redigo.

#413 mocks up this proposal, but it could be changed to use v3 instead of v2, or if the the last commit is broken off into another branch you could adopt the major branch model.

[danqing]

Agreed that maybe we can just get a v3. Now every time I do things like go get -u my 1.7 will be bumped to 2.0 and I have to manually edit the files to lower the version, which is not ideal..

[dcormier]

Is there any reason to just not delete the v2.0.0 tag?

[stevenh]

It's not 100% ideal but a v3 may be the lesser of all evils, thoughts?

[urandom2]

Tags MUST not be deleted per the semver spec, it also cannot be deleted from the module proxy, so the delete will be a noop.

[dcormier]

It's not 100% ideal but a v3 may be the lesser of all evils, thoughts?

Sounds good to me. Better than the current situation.

[rkuska]

(Just fyi) To use the latest version of this module you can fallback to pseudoversioning:

go get github.com/gomodule/redigo@v0.0.0-20191011215927-4f569a470858

and then just replace your original gomodule/redigo in go.mod with github.com/gomodule/redigo v0.0.0-20191011215927-4f569a470858

[jrefior]

Noticed this post from rsc in another thread on adding Go Modules support to an open source package, and thought some people considering options in this issue might find it helpful:
Russ Cox reply to Badger v2 post

[Jim-Lambert-Bose]

You could just create a long lived v2 branch with a go.mod with the proper v2 module name, and then merge all released changes into that branch from master every time you cut a new v1 release... then use the v2 branch to make releases for v2...keeping them in sync with the v1 releases. This would allow go get installs that don't specify the v2 module name to work as expected and the v2 versions will also stay in sync.