Replace deprecated gobin with bingo task runner
#78
Comments
svengreb
added a commit
that referenced
this issue
Apr 26, 2021
GH-89 [1] supersedes GH-78 [2] which documents how the official deprecation [3] of `gobin` [4] in favor of the new Go 1.16 `go install pkg@version` [5] syntax feature should have been handled for this project. The idea was to replace the `gobin` task runner [6] with a one that leverages "bingo" [7], a project similar to `gobin`, that comes with many great features and also allows to manage development tools on a per-module basis. The problem is that `bingo` uses some non-default and nontransparent mechanisms under the hood and automatically generates files in the repository without the option to disable this behavior. It does not make use of the `go install` command but relies on custom dependency resolution mechanisms, making it prone to future changes in the Go toolchain and therefore not a good choice for the maintainability of projects. >>> `go install` is still not perfect Support for the new `go install` features, which allow to install commands without affecting the `main` module, have already been added in GH-71 [8] as an alternative to `gobin`, but one significant problem was still not addressed: install module/package executables globally without overriding already installed executables of different versions. Since `go install` will always place compiled binaries in the path defined by `go env GOBIN`, any already existing executable with the same name will be replaced. It is not possible to install a module command with two different versions since `go install` still messes up the local user environment. >>> The Workaround: Hybrid `go install` task runner This commit therefore implements the solution through a custom `Runner` [9] that uses `go install` under the hood, but places the compiled executable in a custom cache directory instead of `go env GOBIN`. The runner checks if the executable already exists, installs it if not so, and executes it afterwards. The concept of storing dependencies locally on a per-project basis is well-known from the `node_modules` directory [10] of the "Node" [11] package manager "npm" [12]. Storing executables in a cache directory within the repository (not tracked by Git) allows to use `go install` mechanisms while not affect the global user environment and executables stored in `go env GOBIN`. The runner achieves this by changing the `GOBIN` environment variable to the custom cache directory during the execution of `go install`. This way it bypasses the need for "dirty hacks" while using a custom output path. The only known disadvantage is the increased usage of storage disk space, but since most Go executables are small in size anyway, this is perfectly acceptable compared to the clearly outweighing advantages. Note that the runner dynamically runs executables based on the given task so `Validate() error` is a NOOP. >>> Upcoming Changes The solution described above works totally fine, but is still not a clean solution that uses the Go toolchain without any special logic so as soon as the following changes are made to the Go toolchain (Go 1.17 or later), the custom runner will be removed again: - golang/go/issues#42088 [13] — tracks the process of adding support for the Go module syntax to the `go run` command. This will allow to let the Go toolchain handle the way how compiled executable are stored, located and executed. - golang/go#44469 [14] — tracks the process of making `go install` aware of the `-o` flag like the `go build` command which is the only reason why the custom runner has been implemented. >>> Further Adjustments Because the new custom task runner dynamically runs executables based on the given task, the `Bootstrap` method [15] of the `Wand` [16] reference implementation `Elder` [17] additionally allows to pass Go module import paths, optionally including a version suffix (`pkg@version`), to install executables from Go module-based `main` packages into the local cache directory. This way the local development environment can be set up, for e.g. by running it as startup task [18] in "JetBrains" IDEs. The method also ensures that the local cache directory exists and creates a `.gitignore` file that includes ignore pattern for the cache directory. [1]: #89 [2]: #78 [3]: myitcv/gobin#103 [4]: https://github.com/myitcv/gobin [5]: https://pkg.go.dev/cmd/go#hdr-Compile_and_install_packages_and_dependencies [6]: https://pkg.go.dev/github.com/svengreb/wand@v0.5.0/pkg/task/gobin#Runner [7]: https://github.com/bwplotka/bingo [8]: #71 [9]: https://pkg.go.dev/github.com/svengreb/wand/pkg/task#Runner [10]: https://docs.npmjs.com/cli/v7/configuring-npm/folders#node-modules [11]: https://nodejs.org [12]: https://www.npmjs.com [13]: golang/go#42088 [14]: golang/go#44469 (comment) [15]: https://pkg.go.dev/github.com/svengreb/wand@v0.5.0/pkg/elder#Elder.Bootstrap [16]: https://pkg.go.dev/github.com/svengreb/wand#Wand [17]: https://pkg.go.dev/github.com/svengreb/wand/pkg/elder#Elder [18]: https://www.jetbrains.com/help/idea/settings-tools-startup-tasks.html GH-89
svengreb
added a commit
that referenced
this issue
Apr 26, 2021
GH-89 [1] supersedes GH-78 [2] which documents how the official deprecation [3] of `gobin` [4] in favor of the new Go 1.16 `go install pkg@version` [5] syntax feature should have been handled for this project. The idea was to replace the `gobin` task runner [6] with a one that leverages "bingo" [7], a project similar to `gobin`, that comes with many great features and also allows to manage development tools on a per-module basis. The problem is that `bingo` uses some non-default and nontransparent mechanisms under the hood and automatically generates files in the repository without the option to disable this behavior. It does not make use of the `go install` command but relies on custom dependency resolution mechanisms, making it prone to future changes in the Go toolchain and therefore not a good choice for the maintainability of projects. >>> `go install` is still not perfect Support for the new `go install` features, which allow to install commands without affecting the `main` module, have already been added in GH-71 [8] as an alternative to `gobin`, but one significant problem was still not addressed: install module/package executables globally without overriding already installed executables of different versions. Since `go install` will always place compiled binaries in the path defined by `go env GOBIN`, any already existing executable with the same name will be replaced. It is not possible to install a module command with two different versions since `go install` still messes up the local user environment. >>> The Workaround: Hybrid `go install` task runner This commit therefore implements the solution through a custom `Runner` [9] that uses `go install` under the hood, but places the compiled executable in a custom cache directory instead of `go env GOBIN`. The runner checks if the executable already exists, installs it if not so, and executes it afterwards. The concept of storing dependencies locally on a per-project basis is well-known from the `node_modules` directory [10] of the "Node" [11] package manager "npm" [12]. Storing executables in a cache directory within the repository (not tracked by Git) allows to use `go install` mechanisms while not affect the global user environment and executables stored in `go env GOBIN`. The runner achieves this by changing the `GOBIN` environment variable to the custom cache directory during the execution of `go install`. This way it bypasses the need for "dirty hacks" while using a custom output path. The only known disadvantage is the increased usage of storage disk space, but since most Go executables are small in size anyway, this is perfectly acceptable compared to the clearly outweighing advantages. Note that the runner dynamically runs executables based on the given task so `Validate() error` is a NOOP. >>> Upcoming Changes The solution described above works totally fine, but is still not a clean solution that uses the Go toolchain without any special logic so as soon as the following changes are made to the Go toolchain (Go 1.17 or later), the custom runner will be removed again: - golang/go/issues#42088 [13] — tracks the process of adding support for the Go module syntax to the `go run` command. This will allow to let the Go toolchain handle the way how compiled executable are stored, located and executed. - golang/go#44469 [14] — tracks the process of making `go install` aware of the `-o` flag like the `go build` command which is the only reason why the custom runner has been implemented. >>> Further Adjustments Because the new custom task runner dynamically runs executables based on the given task, the `Bootstrap` method [15] of the `Wand` [16] reference implementation `Elder` [17] additionally allows to pass Go module import paths, optionally including a version suffix (`pkg@version`), to install executables from Go module-based `main` packages into the local cache directory. This way the local development environment can be set up, for e.g. by running it as startup task [18] in "JetBrains" IDEs. The method also ensures that the local cache directory exists and creates a `.gitignore` file that includes ignore pattern for the cache directory. [1]: #89 [2]: #78 [3]: myitcv/gobin#103 [4]: https://github.com/myitcv/gobin [5]: https://pkg.go.dev/cmd/go#hdr-Compile_and_install_packages_and_dependencies [6]: https://pkg.go.dev/github.com/svengreb/wand@v0.5.0/pkg/task/gobin#Runner [7]: https://github.com/bwplotka/bingo [8]: #71 [9]: https://pkg.go.dev/github.com/svengreb/wand/pkg/task#Runner [10]: https://docs.npmjs.com/cli/v7/configuring-npm/folders#node-modules [11]: https://nodejs.org [12]: https://www.npmjs.com [13]: golang/go#42088 [14]: golang/go#44469 (comment) [15]: https://pkg.go.dev/github.com/svengreb/wand@v0.5.0/pkg/elder#Elder.Bootstrap [16]: https://pkg.go.dev/github.com/svengreb/wand#Wand [17]: https://pkg.go.dev/github.com/svengreb/wand/pkg/elder#Elder [18]: https://www.jetbrains.com/help/idea/settings-tools-startup-tasks.html GH-89
svengreb
added a commit
that referenced
this issue
Apr 26, 2021
…#90) GH-89 [1] supersedes GH-78 [2] which documents how the official deprecation [3] of `gobin` [4] in favor of the new Go 1.16 `go install pkg@version` [5] syntax feature should have been handled for this project. The idea was to replace the `gobin` task runner [6] with a one that leverages "bingo" [7], a project similar to `gobin`, that comes with many great features and also allows to manage development tools on a per-module basis. The problem is that `bingo` uses some non-default and nontransparent mechanisms under the hood and automatically generates files in the repository without the option to disable this behavior. It does not make use of the `go install` command but relies on custom dependency resolution mechanisms, making it prone to future changes in the Go toolchain and therefore not a good choice for the maintainability of projects. >>> `go install` is still not perfect Support for the new `go install` features, which allow to install commands without affecting the `main` module, have already been added in GH-71 [8] as an alternative to `gobin`, but one significant problem was still not addressed: install module/package executables globally without overriding already installed executables of different versions. Since `go install` will always place compiled binaries in the path defined by `go env GOBIN`, any already existing executable with the same name will be replaced. It is not possible to install a module command with two different versions since `go install` still messes up the local user environment. >>> The Workaround: Hybrid `go install` task runner This commit therefore implements the solution through a custom `Runner` [9] that uses `go install` under the hood, but places the compiled executable in a custom cache directory instead of `go env GOBIN`. The runner checks if the executable already exists, installs it if not so, and executes it afterwards. The concept of storing dependencies locally on a per-project basis is well-known from the `node_modules` directory [10] of the "Node" [11] package manager "npm" [12]. Storing executables in a cache directory within the repository (not tracked by Git) allows to use `go install` mechanisms while not affect the global user environment and executables stored in `go env GOBIN`. The runner achieves this by changing the `GOBIN` environment variable to the custom cache directory during the execution of `go install`. This way it bypasses the need for "dirty hacks" while using a custom output path. The only known disadvantage is the increased usage of storage disk space, but since most Go executables are small in size anyway, this is perfectly acceptable compared to the clearly outweighing advantages. Note that the runner dynamically runs executables based on the given task so `Validate() error` is a NOOP. >>> Upcoming Changes The solution described above works totally fine, but is still not a clean solution that uses the Go toolchain without any special logic so as soon as the following changes are made to the Go toolchain (Go 1.17 or later), the custom runner will be removed again: - golang/go/issues#42088 [13] — tracks the process of adding support for the Go module syntax to the `go run` command. This will allow to let the Go toolchain handle the way how compiled executable are stored, located and executed. - golang/go#44469 [14] — tracks the process of making `go install` aware of the `-o` flag like the `go build` command which is the only reason why the custom runner has been implemented. >>> Further Adjustments Because the new custom task runner dynamically runs executables based on the given task, the `Bootstrap` method [15] of the `Wand` [16] reference implementation `Elder` [17] additionally allows to pass Go module import paths, optionally including a version suffix (`pkg@version`), to install executables from Go module-based `main` packages into the local cache directory. This way the local development environment can be set up, for e.g. by running it as startup task [18] in "JetBrains" IDEs. The method also ensures that the local cache directory exists and creates a `.gitignore` file that includes ignore pattern for the cache directory. [1]: #89 [2]: #78 [3]: myitcv/gobin#103 [4]: https://github.com/myitcv/gobin [5]: https://pkg.go.dev/cmd/go#hdr-Compile_and_install_packages_and_dependencies [6]: https://pkg.go.dev/github.com/svengreb/wand@v0.5.0/pkg/task/gobin#Runner [7]: https://github.com/bwplotka/bingo [8]: #71 [9]: https://pkg.go.dev/github.com/svengreb/wand/pkg/task#Runner [10]: https://docs.npmjs.com/cli/v7/configuring-npm/folders#node-modules [11]: https://nodejs.org [12]: https://www.npmjs.com [13]: golang/go#42088 [14]: golang/go#44469 (comment) [15]: https://pkg.go.dev/github.com/svengreb/wand@v0.5.0/pkg/elder#Elder.Bootstrap [16]: https://pkg.go.dev/github.com/svengreb/wand#Wand [17]: https://pkg.go.dev/github.com/svengreb/wand/pkg/elder#Elder [18]: https://www.jetbrains.com/help/idea/settings-tools-startup-tasks.html Closes GH-89
svengreb
added a commit
that referenced
this issue
Apr 26, 2021
…#90) GH-89 [1] supersedes GH-78 [2] which documents how the official deprecation [3] of `gobin` [4] in favor of the new Go 1.16 `go install pkg@version` [5] syntax feature should have been handled for this project. The idea was to replace the `gobin` task runner [6] with a one that leverages "bingo" [7], a project similar to `gobin`, that comes with many great features and also allows to manage development tools on a per-module basis. The problem is that `bingo` uses some non-default and nontransparent mechanisms under the hood and automatically generates files in the repository without the option to disable this behavior. It does not make use of the `go install` command but relies on custom dependency resolution mechanisms, making it prone to future changes in the Go toolchain and therefore not a good choice for the maintainability of projects. >>> `go install` is still not perfect Support for the new `go install` features, which allow to install commands without affecting the `main` module, have already been added in GH-71 [8] as an alternative to `gobin`, but one significant problem was still not addressed: install module/package executables globally without overriding already installed executables of different versions. Since `go install` will always place compiled binaries in the path defined by `go env GOBIN`, any already existing executable with the same name will be replaced. It is not possible to install a module command with two different versions since `go install` still messes up the local user environment. >>> The Workaround: Hybrid `go install` task runner This commit therefore implements the solution through a custom `Runner` [9] that uses `go install` under the hood, but places the compiled executable in a custom cache directory instead of `go env GOBIN`. The runner checks if the executable already exists, installs it if not so, and executes it afterwards. The concept of storing dependencies locally on a per-project basis is well-known from the `node_modules` directory [10] of the "Node" [11] package manager "npm" [12]. Storing executables in a cache directory within the repository (not tracked by Git) allows to use `go install` mechanisms while not affect the global user environment and executables stored in `go env GOBIN`. The runner achieves this by changing the `GOBIN` environment variable to the custom cache directory during the execution of `go install`. This way it bypasses the need for "dirty hacks" while using a custom output path. The only known disadvantage is the increased usage of storage disk space, but since most Go executables are small in size anyway, this is perfectly acceptable compared to the clearly outweighing advantages. Note that the runner dynamically runs executables based on the given task so `Validate() error` is a NOOP. >>> Upcoming Changes The solution described above works totally fine, but is still not a clean solution that uses the Go toolchain without any special logic so as soon as the following changes are made to the Go toolchain (Go 1.17 or later), the custom runner will be removed again: - golang/go/issues#42088 [13] — tracks the process of adding support for the Go module syntax to the `go run` command. This will allow to let the Go toolchain handle the way how compiled executable are stored, located and executed. - golang/go#44469 [14] — tracks the process of making `go install` aware of the `-o` flag like the `go build` command which is the only reason why the custom runner has been implemented. >>> Further Adjustments Because the new custom task runner dynamically runs executables based on the given task, the `Bootstrap` method [15] of the `Wand` [16] reference implementation `Elder` [17] additionally allows to pass Go module import paths, optionally including a version suffix (`pkg@version`), to install executables from Go module-based `main` packages into the local cache directory. This way the local development environment can be set up, for e.g. by running it as startup task [18] in "JetBrains" IDEs. The method also ensures that the local cache directory exists and creates a `.gitignore` file that includes ignore pattern for the cache directory. [1]: #89 [2]: #78 [3]: myitcv/gobin#103 [4]: https://github.com/myitcv/gobin [5]: https://pkg.go.dev/cmd/go#hdr-Compile_and_install_packages_and_dependencies [6]: https://pkg.go.dev/github.com/svengreb/wand@v0.5.0/pkg/task/gobin#Runner [7]: https://github.com/bwplotka/bingo [8]: #71 [9]: https://pkg.go.dev/github.com/svengreb/wand/pkg/task#Runner [10]: https://docs.npmjs.com/cli/v7/configuring-npm/folders#node-modules [11]: https://nodejs.org [12]: https://www.npmjs.com [13]: golang/go#42088 [14]: golang/go#44469 (comment) [15]: https://pkg.go.dev/github.com/svengreb/wand@v0.5.0/pkg/elder#Elder.Bootstrap [16]: https://pkg.go.dev/github.com/svengreb/wand#Wand [17]: https://pkg.go.dev/github.com/svengreb/wand/pkg/elder#Elder [18]: https://www.jetbrains.com/help/idea/settings-tools-startup-tasks.html Closes GH-89
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Superseded by #87
As of Go 1.16gobinhas officially been deprecated sincego install $pkg@$versionallows to install commands without affecting themainmodule. See myitcv/gobin#103 for more details about thegobindeprecation and #22 for more information about thegobintask runner.Support for the newgo installfeatures have already been added in #71 as an alternative togobin, but this method still lacks an essential feature: install module/package executables globally without overriding already installed executables of different versions.Since
go installwill always place compiled binaries in$GOBIN(or whatever is set forgo env GOBIN), any already existing binary with the same name will be replaced. So installing a module command with two different versions is still not possible withgo installand still messes up the local environment.This and other well-known problems are tackled by bingo, a project similar togobin. It comes with many great features and allows to manage development tools on a per-module basis, almost like the mechanism of Node‘snode_modulesdirectory.Replacing the task runner that uses the deprecated
gobinproject with a one that usesbingounder the hood will bring back the features missing fromgo installwhile also adding many more goodies to simplify and improve development setups.The text was updated successfully, but these errors were encountered: