cmd/go: allow package authors to retract older package versions as insecure, incompatible, or broken

[michael-schaller]

In order to achieve reproducible builds vgo keeps using specific package versions until an explicit upgrade is done. IMHO this is an excellent default but I'm worried about insecure package versions as currently vgo can't detect if the build contains an insecure package version.

Can vgo be changed so that a package author is able to specify that every version below X is deemed insecure and if an insecure package version is used during a build that the build will fail (with a flag to override)?

[kardianos]

@michael-schaller I'm not sure what new functionality you are asking for.

Right now vgo will not choose a version "below" a version you specify. So if there is an insecure package version, put the minimum version selector in your package or another package and it will not choose it. Maybe For modules that build main packages, you can also specify version ranges to exclude. Maybe I'm missing something?

[ALTree]

So if there is an insecure package version, put the minimum version selector in your package or another package and it will not choose it.

This only works if you know that a certain version is insecure. I think what he's asking for is a mechanism for package authors to broadcast to the world that a certain version is insecure; so that every time a user pulls it, they'll be warned that that version is deprecated and they'll know to update their mod file.

[kardianos]

@ALTree That makes sense. Thanks for clarifying for me. I think that is a fine question to ask and may go hand-in-hand with the question of how to distribute the content / function of go.modverify.

[michael-schaller]

@ALTree correct. :-)

One naive idea would be that 'vgo build' could check the 'go.mod' (or another machine readable file) of the latest package versions for security information. This would also be great for Continuous Integration as then a package author could notify of security issues via CI build failures that are (hopefully) monitored.

[michael-schaller]

@rsc mentioned Deprecated Versions (as part of the Defining Go Modules article) which is similar to this issue. He proposed to append +deprecated to a version tag which would also be a viable solution for this issue if +insecure would be honored by vgo.

IMHO that would be a pretty bare bones solution though as I presume that people would soon want to extend that further. For an instance I could see that someone would also want +buggy for a version with a serious bug (for an example a serious memory leak) or +broken for a version that is broken under certain circumstances (for an example the Windows build is broken). Furthermore this solution lacks a way to add more context as for deprecated versions one might be interested in the deprecation announcement or timeline and for insecure versions one might be interested in CVE, severity, ... and so on.

That said I think signaling via tags if a version is deprecated, insecure, ... is not adequate. Maybe even the proposal from my previous comment isn't. Maybe the discussion should rather go into the direction of a machine readable changelog which could be managed via vgo release.

[uluyol]

I'm not sure about the utility of this feature. If I release version 1.2.3, surely it must be fixing some bugs over 1.2.2 and I would probably mark 1.2.2. In other words, on almost every (point) release, I would mark all older versions as insecure. You might say this is only for bugs with a CVE, but I think the point still stands.

I don't see this providing much over just reporting that newer versions have been published.

[michael-schaller]

@uluyol there are actually multiple points for this feature (which I sadly failed to point out so far):

1. It enables machines to automatically react:
   It's a chore to check for new versions and humans are sadly error prone, especially in burdensome cases of chore like this one. Having a machine readable form to notify of insecure, deprecated, buggy, ... versions enables machines to react so that humans can spend their time on better things. In this case vgo would be able to detect an unhealthy build and could let the build fail.

2. Cut down reaction time:
   Once machines are able to react on such issues you can cut down reaction time drastically as for an instance Continuous Integration would fail and ideally you would automatically get a bug report for the build failure with the details why the build failed. This is especially valuable for projects that aren't any longer in the active development phase but rather receive infrequent updates on demand.

3. Indicator for healthiness of new dependencies:
   If you add a new dependency to your project you get immediately a sense of the healthiness of that new dependency. If the new dependency is unhealthy vgo should fail to build and with that you know that you either need to change your project, need to file a bug report against the new dependency or you need to fork and fix the new dependency (if you still want to use it). For an instance this would catch the error that a human tries to add the new dependency with the major version v1 albeit that version is deprecated and a newer major version should be used.

4. Indicator for healthiness of indirect dependencies:
   In the end you should have a cascading effect in case a package version is marked insecure as every other package that directly and most importantly indirectly depends on it should fail to build.

[uluyol]

I see. I agree that tags lack content, and that it would be useful to also have a reason (or changeling) listed for why a version is insecure. Then on build you could get messages like

build failed: insecure packages
oauth2: CVE-X-Y vulnerable to DoS
zstd: crash on invalid input

I can see why this would be useful. I do think that we'd want to let people override this behavior though.

[michael-schaller]

A definite +1 on the letting people override this behavior part. :-)

[jimmyfrasche]

I agree that additional in-band signalling would be useful way to let the maintainer know that action may need to be taken—but I disagree that every situation requires automatic, mechanical action.

Security fixes are important but automatically applying them can be as fraught as always ignoring them.

If the module in question is for a non-opensourced essential service using foo v1.0.5 and there's a foo v1.1.4+security that needs to be immediately investigated by a person. However, if its fixing a vulnerability introduced in foo v1.1.0 it may not necessarily be worth the effort and risk to drop everything and upgrade right now.

[4ad]

I would prefer if vgo would continue to work the way it does today, with another tool, perhaps vgo audit that could check online if the versions used have problems.

The idea is that this tool is run on-demand by the programmer, rather than automatically. If this tool is easy to use, vgo audit could become as natural as running go fmt.

[rsc]

The purpose of vgo is to add versions to the vocabulary of the toolchain, so that users and tools can talk to each other sensibly about versions. As I mentioned in the https://research.swtch.com/vgo-module article, I think it would make sense to have a v1.2.3+deprecated tag, using an annotated tag so that there's a commit message. The commit message can say anything it wants about why the release is deprecated, and we can show that to users. We could easily add a notation in the text for identifying security problems. What happens next is up to tools. Probably vgo list -m -u (tell me about pending module updates) would do well to show information about currently-used modules that have deprecation notices.