cmd/go: preserve basic GOPATH mode indefinitely

[rsc]

In Feb 2021 (https://go.dev/blog/go116-module-changes), we wrote:

It's still possible to build packages in GOPATH mode by setting the GO111MODULE environment variable to off. ... We plan to drop support for GOPATH mode in Go 1.17. In other words, Go 1.17 will ignore GO111MODULE. If you have projects that do not build in module-aware mode, now is the time to migrate. If there is a problem preventing you from migrating, please consider filing an issue or an experience report.

The time has come to decide what to do about GOPATH mode. There are at least two important problems with maintaining GOPATH mode as it exists today:

• The old, GOPATH-mode go get is being left behind as far as security improvements like the module proxy, checksum databases, and so on. All the focus is on modules.

• The old, GOPATH-mode source layout does not provide a way to identify the language version used by the source code. There is a bit of divergence here in that a go.mod with no go line assumes Go 1.16, while in GOPATH mode we assume the latest version of Go. This means if someone has downloadable packages they develop in GOPATH mode, they can use language features introduced after Go 1.16, like generics, but when clients download that code in module mode as a "legacy" module, the code is interpreted as Go 1.16 and does not compile. The divergence will become greater over time as other language features or changes are made, such as spec: less error-prone loop variable scoping #60078.

On the other hand, GOPATH mode remains the only way to work on dependency-free legacy packages that existing module code may still depend on (so-called "+incompatible" modules), and it remains the only way to build historical source code that predates modules, especially code that depends on the multiple layers of vendoring that Go 1.5 introduced (and modules removed).

Given these tensions, I propose that we do the following:

1. Commit to preserving the ability to build GOPATH-layout source trees when GO111MODULE=off, indefinitely.
2. Disable go get completely in GOPATH mode, since there is increasingly little code that it can download successfully and provides a worse and worse experience.
3. Assume a language setting of Go 1.21 in GOPATH mode, instead of continuing to assume "the latest Go version". This will ensure that if we change for loops in Go 1.22 (spec: less error-prone loop variable scoping #60078) or make other changes in the future, the old, legacy code we are aiming to keep compiling does keep compiling as written.

We would make these changes for Go 1.22. The GOPATH mode enabler would continue to be GO111MODULE=off rather than introducing a cleaner name like GOMODULE=off, because there's little point to upsetting whatever scripts people with legacy trees have already written.

[dominikh]

Do you expect that tools like gopls will continue to support GOPATH mode? While go list takes care of a lot of the details, it still requires some special handling in go/packages AFAIU — and possibly in the tools' themselves.

[rsc]

I would expect gopls and go/packages to continue to support GOPATH as well as they do today. There should be very little special code they need, and it's already written.

[rsc]

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

[willfaught]

What examples are there of GOPATH packages we want to keep working with the latest toolchains?

The steps to convert GOPATH packages to modules are:

• go mod init ...
• go get ...
• go mod tidy

Unless I missed something, that's not much effort. This is basically equivalent to forking a package that you know will be deleted soon. In a sense, GOPATH packages would be "deleted." Anyone who truly cares can fork their own copy, and turn it into a module for others to use too.

How much simpler would the toolchain be without GOPATH support?

[ChrisHines]

An anecdotal data point: My project at work still depends on GOPATH mode to build. Dependencies are managed with a 2015 era tool that automates populating the GOPATH with the desired versions of dependencies, but provides little help deciding what the versions should be. We would have upgraded to Go modules long ago had it been easy, but due to the size of the project, the accumulated complexity of its dependency tree, and other reasons it hasn't happened yet. Not for a lack of desire, by the way. The project policy for the toolchain version is to stay one major version behind, so it's currently built with Go 1.19 and would upgrade to 1.20 shortly after 1.21 is released.

Fortunately we have been able to remove some of the barriers to adopting modules over the last several months and I believe we will be upgraded sometime in the next several months. But there may be other projects similar to ours that are not publicly visible, that still need GOPATH, are still actively worked on, and regularly upgrade the toolchain, but are farther away from using modules. Continuing to support GOPATH mode helps projects like that.

[bcmills]

How much simpler would the toolchain be without GOPATH support?

The main quirks remaining to support GOPATH mode have to do with how we resolve dependencies in vendor directories, although even that has some overlap with how we support the vendored dependencies of the standard library.

Beyond that, as Russ mentions, the go get implementation for GOPATH mode is complex and fairly subtle.

Speaking as one of the primary maintainers of cmd/go, I feel that the cost/benefit tradeoff supports Russ's proposal. The go get support is subtle and not worth the cost to maintain, but maintaining go build support for the time being is fairly inexpensive and may be valuable for the long tail of migrations (especially for enterprise users).

[willfaught]

@ChrisHines Thanks for your example. I'm curious, how many man hours, working full-time 8 hour days without distraction, do you think it would take to convert that project to an equivalent module? I realize that it's probably non-trivial to do, but I suspect that a lot of situations like that (not necessarily yours, I'm speaking generally) are more a result of management deciding to spend their money on shiny new features instead of maintaining their code, especially since, until now, others have been willing to pay the price to keep them afloat in their current condition. I'm asking so we can have a better sense of the trade-off costs here.

[rsc]

Being able to run old (self-contained) code that predates modules is also worthwhile. That requires having GOPATH build support. Keeping build working is easy and it has real benefits.