GOPROXY environment not being passed

[lrstanley]

I'm not use to running on Windows, so excuse me if it's something specific I have to do on Windows.

Using the latest version of vscode-go, 0.6.92, I am seeing that the builtin functionality to install the analysis tools, the GOPROXY environment variable which is globally defined on windows, (and passed to vscode), isn't being passed when this library executes go.exe. When I start a terminal inside of vscode, it gets picked up as expected..

PS C:\Users\liam\code\my-package> go env
[...]
set GOPROXY=http://TRUNCATED.internal.com:3000
[...]

Trying to download modules through the Athens proxy, it is still throwing certificate errors when hitting Github (corp firewall restrictions). When doing it through the terminal, it downloads just fine.

Is there a way to get the go plugin to run 'go env' to see what environment variables it's getting passed?

Would also be worth while to add an additional setting for this as well (to override GOPROXY, much like GOROOT and GOPATH) -- I expect with GOPROXY making it much easier for corporations with restricted firewalls to use Go, this will be a frequently requested feature.

[ramya-rao-a]

Is there a way to get the go plugin to run 'go env' to see what environment variables it's getting passed?

Running go env in the terminal should give the same result as when it is run by the plugin.
To see the environment variables the plugin gets, run Help -> Toggle Developer Tools -> Console -> process.env

Speaking of proxies, can you see if anything in https://code.visualstudio.com/docs/setup/network helps?

[lrstanley]

What I see in process.env:

So, the plugin is getting the environment variable, but it's not passed through to the child process, go.exe.. as running the same command returns the expected output when GOPROXY is missing.

(this fails with ssl errors due to corp firewall):

Installing 1 tool at C:/Users/liam/go\bin
  goreturns

Installing github.com/sqs/goreturns FAILED

1 tools failed to install.

goreturns:
Error: Command failed: C:\Go\bin\go.exe get -u -v github.com/sqs/goreturns
github.com/sqs/goreturns (download)
# cd .; git clone https://github.com/sqs/goreturns C:\Users\liam\go\src\github.com\sqs\goreturns
Cloning into 'C:\Users\liam\go\src\github.com\sqs\goreturns'...
fatal: unable to access 'https://github.com/sqs/goreturns/': SSL certificate problem: unable to get local issuer certificate
package github.com/sqs/goreturns: exit status 128
github.com/sqs/goreturns (download)
# cd .; git clone https://github.com/sqs/goreturns C:\Users\liam\go\src\github.com\sqs\goreturns
Cloning into 'C:\Users\liam\go\src\github.com\sqs\goreturns'...
fatal: unable to access 'https://github.com/sqs/goreturns/': SSL certificate problem: unable to get local issuer certificate
package github.com/sqs/goreturns: exit status 128

When I go to the terminal in VSCode and run the exact same command, it properly sees GOPROXY, and uses it for the go modules:

PS C:\Users\liam\code\pipeline-gen> C:\Go\bin\go.exe get -u -v github.com/sqs/goreturns
go: finding github.com/sqs/goreturns latest
PS C:\Users\liam\code\pipeline-gen> goreturns
usage: goreturns [flags] [path ...]
  -b    remove bare returns
[...more output...]

In regards to VSCodes networking related proxy settings, this is unrelated. GOPROXY is something new with the new Go modules support -- it allows you to pull dependencies from a central location. Trying to use the go proxy as a normal http proxy will completely fail.

Here is the documentation for GOPROXY, pulled from one of my linux instances:

$ go help goproxy
The go command by default downloads modules from version control systems
directly, just as 'go get' always has. The GOPROXY environment variable allows
further control over the download source. If GOPROXY is unset, is the empty string,
or is the string "direct", downloads use the default direct connection to version
control systems. Setting GOPROXY to "off" disallows downloading modules from
any source. Otherwise, GOPROXY is expected to be the URL of a module proxy,
in which case the go command will fetch all modules from that proxy.
No matter the source of the modules, downloaded modules must match existing
entries in go.sum (see 'go help modules' for discussion of verification).

A Go module proxy is any web server that can respond to GET requests for
URLs of a specified form. The requests have no query parameters, so even
a site serving from a fixed file system (including a file:/// URL)
can be a module proxy.

The GET requests sent to a Go module proxy are:

GET $GOPROXY/<module>/@v/list returns a list of all known versions of the
given module, one per line.

GET $GOPROXY/<module>/@v/<version>.info returns JSON-formatted metadata
about that version of the given module.

GET $GOPROXY/<module>/@v/<version>.mod returns the go.mod file
for that version of the given module.

GET $GOPROXY/<module>/@v/<version>.zip returns the zip archive
for that version of the given module.

To avoid problems when serving from case-sensitive file systems,
the <module> and <version> elements are case-encoded, replacing every
uppercase letter with an exclamation mark followed by the corresponding
lower-case letter: github.com/Azure encodes as github.com/!azure.

The JSON-formatted metadata about a given module corresponds to
this Go data structure, which may be expanded in the future:

    type Info struct {
        Version string    // version string
        Time    time.Time // commit time
    }

The zip archive for a specific version of a given module is a
standard zip file that contains the file tree corresponding
to the module's source code and related files. The archive uses
slash-separated paths, and every file path in the archive must
begin with <module>@<version>/, where the module and version are
substituted directly, not case-encoded. The root of the module
file tree corresponds to the <module>@<version>/ prefix in the
archive.

Even when downloading directly from version control systems,
the go command synthesizes explicit info, mod, and zip files
and stores them in its local cache, $GOPATH/pkg/mod/cache/download,
the same as if it had downloaded them directly from a proxy.
The cache layout is the same as the proxy URL space, so
serving $GOPATH/pkg/mod/cache/download at (or copying it to)
https://example.com/proxy would let other users access those
cached module versions with GOPROXY=https://example.com/proxy.

[ramya-rao-a]

So, the plugin is getting the environment variable, but it's not passed through to the child process, go.exe..

The extension host process gets the same environment variables as what you saw in the console. The same are passed to the child process that runs go.exe.

Can you run the extension from source to see the exact environment variables being passed? Its is pretty straighforward.

• Build and Debug extension
• Add a breakpoint at https://github.com/Microsoft/vscode-go/blob/0.6.93/src/goInstallTools.ts#L378
• At that point look at the values for process.env['GOPROXY'] and envForTools['GOPROXY']

[lrstanley]

I'm sorry for the delay, I was in SF for hashiconf. I didn't run from source, as I don't have nodejs, and corp laptop makes it hard. I just modified the transpiled source to include the following two lines inside roughly the same place you mentioned (Promise.resolve([])).then):

        goStatus_1.outputChannel.appendLine('process.env: ', process.env['GOPROXY']);
        goStatus_1.outputChannel.appendLine('envForTools: ', envForTools['GOPROXY']);

and the output is:

Installing 7 tools at C:\Users\liam\go\bin
  gopkgs
  go-outline
  go-symbols
  guru
  gorename
  dlv
  godoc

Installing github.com/uudashr/gopkgs/cmd/gopkgs FAILED
Installing github.com/ramya-rao-a/go-outline FAILED
Installing github.com/acroca/go-symbols FAILED
Installing golang.org/x/tools/cmd/guru FAILED
Installing golang.org/x/tools/cmd/gorename FAILED
Installing github.com/derekparker/delve/cmd/dlv FAILED

process.env: 
envForTools: 
7 tools failed to install.

gopkgs:
Error: Command failed: C:\Go\bin\go.exe get -u -v github.com/uudashr/gopkgs/cmd/gopkgs
github.com/uudashr/gopkgs (download)
# cd .; git clone https://github.com/uudashr/gopkgs C:\Users\liam\go\src\github.com\uudashr\gopkgs
Cloning into 'C:\Users\liam\go\src\github.com\uudashr\gopkgs'...
fatal: unable to access 'https://github.com/uudashr/gopkgs/': SSL certificate problem: unable to get local issuer certificate
package github.com/uudashr/gopkgs/cmd/gopkgs: exit status 128
github.com/uudashr/gopkgs (download)
# cd .; git clone https://github.com/uudashr/gopkgs C:\Users\liam\go\src\github.com\uudashr\gopkgs
Cloning into 'C:\Users\liam\go\src\github.com\uudashr\gopkgs'...
fatal: unable to access 'https://github.com/uudashr/gopkgs/': SSL certificate problem: unable to get local issuer certificate
package github.com/uudashr/gopkgs/cmd/gopkgs: exit status 128
[...lots more failures...]

As you can see, it's printing nothing after the prefix..:

process.env: 
envForTools: 

And yet, when I open debug tools in vscode, and print the environment variable, it's there:

What I'm wondering is if it's being stripped out before it's being passed to the extension. Maybe vscode strips anything that has proxy in the name, as it assumes it's related to the network proxy settings? In any case, it doesn't look like it's getting passed to the go executable, even though it's being passed to vscode, and vscode itself sees it.

Can you replicate this? All it takes is to run the athens proxy docker container temporarily, and update your GOPROXY environment to point to it, so the athens proxy caches all of the downloaded go v1.11.x modules.

[lrstanley]

Any update to this?

[ramya-rao-a]

Hey @lrstanley,

Apologies, I didn't have time to try out athens myself. But by what you are sharing it looks like VS Code is getting the environment variable (because you see it in the integrated terminal), but it isn't passing it on to the extension host process that runs all the VS Code extensions (this is why you don't see it when you tried to print it out when you changed the source code).

I am including @arschles in this thread who uses athens as well as VS Code.

@arschles Have you seen anything like this before?

@lrstanley One last thing I would like you to try is to run VS Code directly from the command line. Just type code (or code-insiders if you are using the Insiders version of VS Code) from the command line. This ensures that VS Code inherits all the environment variables from the command line.

[lrstanley]

Yes, I have also invoked it from the command line, from the same environment where go env shows the GOPROXY environment variables (and go get <etc> requests through Athens work).

For @arschles -- note that this is solely for the portion where the vscode-go plugin attempts to go get dependencies. This is not related to using the embedded terminal to go get dependencies.

[ramya-rao-a]

Yes, I have also invoked it from the command line, from the same environment where go env shows the GOPROXY environment variables (and go get requests through Athens work).

And this is the same environment where echo %GOPROXY% is the same as go env GOPROXY?

[lrstanley]

Using Git Bash is how I usually invoke it on windows:

liam@hostname MINGW64 ~
$ echo $GOPROXY
http://TRUNCATED.internal.com:3000

liam@hostname MINGW64 ~
$ go env GOPROXY
http://TRUNCATED.internal.com:3000

Using command prompt though:

C:\Users\liam>echo %GOPROXY%
http://TRUNCATED.internal.com:3000

C:\Users\liam>go env GOPROXY
http://TRUNCATED.internal.com:3000

[ramya-rao-a]

Ok, so the "GOPROXY" environment variable is available in your Git Bash and command prompt, but when you run VS Code from either, it doesnt pass the same to the extension host.

I have logged an upstream bug microsoft/vscode#65113 in VS Code. Lets see if someone there can help

[lrstanley]

As a followup to what I mentioned in microsoft/vscode#65113; I think there may still be some issues in vscode-go, not vscode itself.

@ramya-rao-a, you told me to use:

process.env['GOPROXY'] and envForTools['GOPROXY']

I made the modification in the generated plugin at $HOME/.vscode/extensions/ms-vscode.go-0.8.0/out/src/goInstallTools.js. This time around, I am doing the modifications to the same file.

I just tested again, trying to print all of process.env using goStatus_1.outputChannel, and it was empty as well. Testing console.log(process.env) prints everything to the console properly. seems passing non-strings to goStatus_1.outputChannel doesn't work.. I see GOPROXY defined in console.log(process.env).

Using goStatus_1.outputChannel.appendLine('process.env', process.env['GOPROXY']) still doesn't work, not sure why. Now that I think about it, it probably only takes one argument, and we're trying to pass it multiple.

In any case, I think I know what's going on. It looks like you're explicitly setting:

envForTools['GO111MODULE'] = 'off';

Which.. GOPROXY doesn't work when module support is disabled. This is more of an issue with GOPROXY than with this plugin, since enabling module support would mean the tools would be added to the go.mod file.. which I don't think is what we'd want. However, since I am in a corporate environment, and we can't easily go get things, I don't think it's the worst to have these things dumped into the dependency list of the given project (and... downloaded over and over again for multiple projects).

[lrstanley]

I suspect people using GOPROXY are in corporate environments.. and as such, would be in the same situation. Maybe if GOPROXY is set (and not set to off), we don't set GO111MODULE=off (or maybe set to on?)

Or, maybe add an option to the extension to Download extension dependencies with go module support or similar?

[ramya-rao-a]

Good to know that the env variables are getting passed around properly.

So to sum this up, since I disable module support when running go get for the Go tools, GOPROXY doesn't work which then lands you with proxy issues in corporate environment.

I am in a corporate environment, and we can't easily go get things,

There have been other users who had these proxy issues in corporate environment before. Using the http.proxy setting has helped out in those cases. Can you give that a try?

I don't think it's the worst to have these things dumped into the dependency list of the given project (and... downloaded over and over again for multiple projects).

We really don't want to get into a place where the tools need to be downloaded into more than 1 location

[lrstanley]

There have been other users who had these proxy issues in corporate environment before. Using the http.proxy setting has helped out in those cases. Can you give that a try?

Unfortunately, we cannot setup a proxy in this fashion. The only way is via a controlled environment through something like Athens for our usecase.

We really don't want to get into a place where the tools need to be downloaded into more than 1 location

Which makes sense,, however, with the changes to go modules, GOPATH is being deprecated (the words "deprecated" show up in golang/go#24301, I'm not sure to what extent -- I expect support to still exist, at least for Go 1.x.x). A good amount of work has gone into things like cache being moved out of $GOPATH, as well as $GOPATH/pkg being moved into the new build cache locations (iirc, this is where they are now).

Moving forward, might be better to focus on the use of $GOBIN so things don't have to be downloaded in multiple locations.

I'm not saying what I recommended is the best option, but I do think it will become more of an issue, and warrants more discussion on alternatives.