“gobin“ Go module caster #22
Labels
Milestone
Comments
svengreb
added a commit
that referenced
this issue
Nov 22, 2020
>>> Go Executable Installation When installing a Go executable from within a Go module [1] directory using the `go install` command [2], it is installed into the Go executable search path that is defined through the `GOBIN` environment variable [3] and can also be shown and modified using the `go env` command [4]. Even though the executable gets installed globally, the `go.mod` file [5] will be updated to include the installed packages since this is the default behavior of the `go get` command [6] running in "module mode" [7]. Next to this problem, the installed executable will also overwrite any executable of the same module/package that was installed already, but maybe from a different version. Therefore only one version of a executable can be installed at a time which makes it impossible to work on different projects that use the same tool but with different versions. >>>> History & Future The local installation of executables built from Go modules/packages has always been a somewhat controversial point which unfortunately, partly for historical reasons, does not offer an optimal and user-friendly solution up to now. The `go` command [8] is a fantastic toolchain that provides many great features one would expect to be provided out-of-the-box from a modern and well designed programming language without the requirement to use a third-party solution: from compiling code, running unit/integration/benchmark tests, quality and error analysis, debugging utilities and many more. Unfortunately the way the `go install` command [2] of Go versions less or equal to 1.15 handles the installation of an Go module/package executable is still not optimal. The general problem of tool dependencies is a long-time known issue/weak point of the current Go toolchain and is a highly rated change request from the Go community with discussions like golang/go#30515 [9], golang/go#25922 [10] and golang/go#27653 [11] to improve this essential feature, but they've been around for quite a long time without a solution that works without introducing breaking changes and most users and the Go team agree on. Luckily, this topic was finally picked up for the next upcoming Go release version 1.16 [12] and golang/go#40276 [13] introduces a way to install executables in module mode outside a module. The release note preview also already includes details about this change [14] and how installation of executables from Go modules will be handled in the future. >>>> The Workaround Beside the great news and anticipation about an official solution for the problem the usage of a workaround is almost inevitable until Go 1.16 is finally released. The official Go wiki [15] provides a section on "How can I track tool dependencies for a module?" [16] that describes a workaround that tracks tool dependencies. It allows to use the Go module logic by using a file like `tools.go` with a dedicated `tools` build tag that prevents the included module dependencies to be picked up included for normal executable builds. This approach works fine for non-main packages, but CLI tools that are only implemented in the `main` package can not be imported in such a file. In order to tackle this problem, a user from the community created "gobin" [17], an experimental, module-aware command to install/run main packages. It allows to install or run main-package commands without "polluting" the `go.mod` file by default. It downloads modules in version-aware mode into a binary cache path within the systems cache directory [18]. It prevents problems due to already globally installed executables by placing each version in its own directory. The decision to use a cache directory instead of sub-directories within the `GOBIN` path keeps the system clean. "gobin" is still in an early development state, but has already received a lot of positive feedback and is used in many projects. There are also members of the core Go team that have contributed to the project and the chance is high that the changes for Go 1.16 were influenced or partially ported from it. It is currently the best workaround to... 1. prevent the Go toolchain to pick up the `GOMOD` environment variable [4] (see [`go env GOMOD`][4]) that is initialized automatically with the path to the `go.mod` file in the current working directory. 2. install module/package executables globally without "polluting" the `go.mod` file. 3. install module/package executables globally without overriding already installed executables of different versions. See gobin's FAQ page [19] in the repository wiki for more details about the project. >>>> The Go Module Caster To allow to manage the tool dependency problem, wand now uses "gobin" through a new caster [20] `go.mod` file and allows to... 1. install `gobin` itself into `GOBIN` (`go env GOBIN` [4]). 2. cast any spell incantation [21] of kind `KindGoModule` [22] by installing the executable globally into the dedicated `gobin` cache. [1]: https://golang.org/ref/mod [10]: golang/go#25922 [11]: golang/go#27653 [12]: https://github.com/golang/go/milestone/145 [13]: golang/go#40276 [14]: https://tip.golang.org/doc/go1.16#modules [15]: https://github.com/golang/go/wiki [16]: https://github.com/golang/go/wiki/Modules#how-can-i-track-tool-dependencies-for-a-module [17]: https://github.com/myitcv/gobin [18]: https://golang.org/pkg/os/#UserCacheDir [19]: https://github.com/myitcv/gobin/wiki/FAQ [2]: https://golang.org/cmd/go#hdr-Compile_and_install_packages_and_dependencies [20]: https://pkg.go.dev/github.com/svengreb/wand/pkg/cast/gobin#Caster [21]: https://pkg.go.dev/github.com/svengreb/wand/pkg/spell#Incantation [22]: https://pkg.go.dev/github.com/svengreb/wand/pkg/spell#KindGoModule [3]: https://golang.org/cmd/go/#hdr-Environment_variables [4]: https://golang.org/cmd/go/#hdr-Print_Go_environment_information [5]: https://golang.org/ref/mod#go-mod-file [6]: https://golang.org/cmd/go/#hdr-Add_dependencies_to_current_module_and_install_them [7]: https://golang.org/ref/mod#mod-commands [8]: https://golang.org/cmd/go [9]: golang/go#30515 GH-22
Merged
svengreb
added a commit
that referenced
this issue
Nov 22, 2020
>>> Go Executable Installation When installing a Go executable from within a Go module [1] directory using the `go install` command [2], it is installed into the Go executable search path that is defined through the `GOBIN` environment variable [3] and can also be shown and modified using the `go env` command [4]. Even though the executable gets installed globally, the `go.mod` file [5] will be updated to include the installed packages since this is the default behavior of the `go get` command [6] running in "module mode" [7]. Next to this problem, the installed executable will also overwrite any executable of the same module/package that was installed already, but maybe from a different version. Therefore only one version of a executable can be installed at a time which makes it impossible to work on different projects that use the same tool but with different versions. >>>> History & Future The local installation of executables built from Go modules/packages has always been a somewhat controversial point which unfortunately, partly for historical reasons, does not offer an optimal and user-friendly solution up to now. The `go` command [8] is a fantastic toolchain that provides many great features one would expect to be provided out-of-the-box from a modern and well designed programming language without the requirement to use a third-party solution: from compiling code, running unit/integration/benchmark tests, quality and error analysis, debugging utilities and many more. Unfortunately the way the `go install` command [2] of Go versions less or equal to 1.15 handles the installation of an Go module/package executable is still not optimal. The general problem of tool dependencies is a long-time known issue/weak point of the current Go toolchain and is a highly rated change request from the Go community with discussions like golang/go#30515 [9], golang/go#25922 [10] and golang/go#27653 [11] to improve this essential feature, but they've been around for quite a long time without a solution that works without introducing breaking changes and most users and the Go team agree on. Luckily, this topic was finally picked up for the next upcoming Go release version 1.16 [12] and golang/go#40276 [13] introduces a way to install executables in module mode outside a module. The release note preview also already includes details about this change [14] and how installation of executables from Go modules will be handled in the future. >>>> The Workaround Beside the great news and anticipation about an official solution for the problem the usage of a workaround is almost inevitable until Go 1.16 is finally released. The official Go wiki [15] provides a section on "How can I track tool dependencies for a module?" [16] that describes a workaround that tracks tool dependencies. It allows to use the Go module logic by using a file like `tools.go` with a dedicated `tools` build tag that prevents the included module dependencies to be picked up included for normal executable builds. This approach works fine for non-main packages, but CLI tools that are only implemented in the `main` package can not be imported in such a file. In order to tackle this problem, a user from the community created "gobin" [17], an experimental, module-aware command to install/run main packages. It allows to install or run main-package commands without "polluting" the `go.mod` file by default. It downloads modules in version-aware mode into a binary cache path within the systems cache directory [18]. It prevents problems due to already globally installed executables by placing each version in its own directory. The decision to use a cache directory instead of sub-directories within the `GOBIN` path keeps the system clean. "gobin" is still in an early development state, but has already received a lot of positive feedback and is used in many projects. There are also members of the core Go team that have contributed to the project and the chance is high that the changes for Go 1.16 were influenced or partially ported from it. It is currently the best workaround to... 1. prevent the Go toolchain to pick up the `GOMOD` environment variable [4] (see [`go env GOMOD`][4]) that is initialized automatically with the path to the `go.mod` file in the current working directory. 2. install module/package executables globally without "polluting" the `go.mod` file. 3. install module/package executables globally without overriding already installed executables of different versions. See gobin's FAQ page [19] in the repository wiki for more details about the project. >>>> The Go Module Caster To allow to manage the tool dependency problem, wand now uses "gobin" through a new caster [20] `go.mod` file and allows to... 1. install `gobin` itself into `GOBIN` (`go env GOBIN` [4]). 2. cast any spell incantation [21] of kind `KindGoModule` [22] by installing the executable globally into the dedicated `gobin` cache. [1]: https://golang.org/ref/mod [10]: golang/go#25922 [11]: golang/go#27653 [12]: https://github.com/golang/go/milestone/145 [13]: golang/go#40276 [14]: https://tip.golang.org/doc/go1.16#modules [15]: https://github.com/golang/go/wiki [16]: https://github.com/golang/go/wiki/Modules#how-can-i-track-tool-dependencies-for-a-module [17]: https://github.com/myitcv/gobin [18]: https://golang.org/pkg/os/#UserCacheDir [19]: https://github.com/myitcv/gobin/wiki/FAQ [2]: https://golang.org/cmd/go#hdr-Compile_and_install_packages_and_dependencies [20]: https://pkg.go.dev/github.com/svengreb/wand/pkg/cast/gobin#Caster [21]: https://pkg.go.dev/github.com/svengreb/wand/pkg/spell#Incantation [22]: https://pkg.go.dev/github.com/svengreb/wand/pkg/spell#KindGoModule [3]: https://golang.org/cmd/go/#hdr-Environment_variables [4]: https://golang.org/cmd/go/#hdr-Print_Go_environment_information [5]: https://golang.org/ref/mod#go-mod-file [6]: https://golang.org/cmd/go/#hdr-Add_dependencies_to_current_module_and_install_them [7]: https://golang.org/ref/mod#mod-commands [8]: https://golang.org/cmd/go [9]: golang/go#30515 Closes GH-22
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Go Executable Installation
When installing a Go executable from within a Go module directory using the
go installcommand, it is installed into the Go executable search path that is defined through theGOBINenvironment variable and can also be shown and modified using thego envcommand. Even though the executable gets installed globally, thego.modfile will be updated to include the installed packages since this is the default behavior of thego getcommand when running in module mode.Next to this problem, the installed executable will also overwrite any executable of the same module/package that was installed already, but maybe from a different version. Therefore only one version of a executable can be installed at a time which makes it impossible to work on different projects that use the same tool but with different versions.
History & Future
The local installation of executables built from Go modules/packages has always been a somewhat controversial point which unfortunately, partly for historical reasons, does not offer an optimal and user-friendly solution up to now. The
gocommand is a fantastic toolchain that provides many great features one would expect to be provided out-of-the-box from a modern and well designed programming language without the requirement to use a third-party solution: from compiling code, running unit/integration/benchmark tests, quality and error analysis, debugging utilities and many more.Unfortunately the way the
go installcommand of Go versions less or equal to 1.15 handles the installation of an Go module/package executable is still not optimal.The general problem of tool dependencies is a long-time known issue/weak point of the current Go toolchain and is a highly rated change request from the Go community with discussions like golang/go#30515, golang/go#25922 and golang/go#27653 to improve this essential feature, but they‘ve been around for quite a long time without a solution that works without introducing breaking changes and most users and the Go team agree on.
Luckily, this topic was finally picked up for the next upcoming Go release version 1.16 and gh-golang/go#40276 introduces a way to install executables in module mode outside a module. The release note preview also already includes details about this change and how installation of executables from Go modules will be handled in the future.
The Workaround
Beside the great news and anticipation about an official solution for the problem the usage of a workaround is almost inevitable until Go 1.16 is finally released.
The official Go wiki provides a section on “How can I track tool dependencies for a module?” that describes a workaround that tracks tool dependencies. It allows to use the Go module logic by using a file like
tools.gowith a dedicatedtoolsbuild tag that prevents the included module dependencies to be picked up included for normal executable builds. This approach works fine for non-main packages, but CLI tools that are only implemented in themainpackage can not be imported in such a file.In order to tackle this problem, a user from the community created gobin, an experimental, module-aware command to install/run main packages.
It allows to install or run main-package commands without “polluting“ the
go.modfile by default. It downloads modules in version-aware mode into a binary cache path within the systems cache directory.It prevents problems due to already globally installed executables by placing each version in its own directory. The decision to use a cache directory instead of sub-directories within the
GOBINpath keeps the system clean.gobin is still in an early development state, but has already received a lot of positive feedback and is used in many projects. There are also members of the core Go team that have contributed to the project and the chance is high that the changes for Go 1.16 were influenced or partially ported from it.
It is currently the best workaround to…
GOMODenvironment variable (seego env GOMOD) that is initialized automatically with the path to thego.modfile in the current working directory.go.modfile.See gobin‘s FAQ page in the repository wiki for more details about the project.
The Go Module Caster
To allow to manage the tool dependency problem, wand, will use
gobinthrough a new caster that prevents the “pollution“ of the projectgo.modfile and allows to…gobinitself intoGOBIN(go env GOBIN).KindGoModuleby installing the executable globally into the dedicatedgobincache.The text was updated successfully, but these errors were encountered: