Skip to content

Performance issue with gosec on v1.54.2 #4039

Closed
@tobias-kuendig

Description

@tobias-kuendig

Welcome

Description of the problem

Using 1.54.2 of golang-ci-lint, the gosec linter suddenly is a lot slower than before. I am running this on a relatively large codebase:

v1.54.1 (runs in a few seconds)

$ curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin v1.54.1
$ golangci-lint version
golangci-lint has version 1.54.1 built with go1.21.0 from a9378d9b on 2023-08-11T12:49:16Z

$ golangci-lint run -v --no-config --disable-all --enable gosec ./...                                                             
INFO [lintersdb] Active 1 linters: [gosec]        
INFO [loader] Go packages loading at mode 575 (compiled_files|exports_file|files|deps|imports|name|types_sizes) took 1.53940274s 
INFO [runner/filename_unadjuster] Pre-built 0 adjustments in 90.851188ms 
INFO [linters_context/goanalysis] analyzers took 976.350117ms with top 10 stages: gosec: 974.013444ms, typecheck: 2.336673ms 
INFO [runner] Issues before processing: 55, after processing: 0 
INFO [runner] Processors filtering stat (out/in): exclude: 46/46, cgo: 55/55, skip_files: 55/55, exclude-rules: 9/46, nolint: 0/9, filename_unadjuster: 55/55, path_prettifier: 55/55, skip_dirs: 55/55, autogenerated_exclude: 46/55, identifier_marker: 46/46 
INFO [runner] processing took 4.736721ms with stages: nolint: 2.332846ms, autogenerated_exclude: 1.196407ms, exclude-rules: 373.173µs, identifier_marker: 328.37µs, skip_dirs: 250.949µs, path_prettifier: 239.91µs, cgo: 8.717µs, filename_unadjuster: 3.366µs, max_same_issues: 545ns, fixer: 359ns, source_code: 311ns, diff: 285ns, uniq_by_line: 283ns, skip_files: 260ns, path_shortener: 189ns, severity-rules: 186ns, exclude: 181ns, sort_results: 150ns, max_from_linter: 100ns, max_per_file_from_linter: 70ns, path_prefixer: 64ns 
INFO [runner] linters took 905.767983ms with stages: goanalysis_metalinter: 900.987883ms 
INFO File cache stats: 0 entries of total size 0B 
INFO Memory: 27 samples, avg is 136.6MB, max is 398.2MB 
INFO Execution took 2.539893412s 

v1.54.2 (takes minutes, hitting timeout)

I've had runs that passed, this one hit the timeout:

$ curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin v1.54.2
$ golangci-lint version
golangci-lint has version 1.54.2 built with go1.21.0 from 411e0bbb on 2023-08-21T12:04:32Z

$ golangci-lint run -v --no-config --disable-all --enable gosec ./...                                      
INFO [lintersdb] Active 1 linters: [gosec]        
INFO [loader] Go packages loading at mode 575 (imports|types_sizes|deps|exports_file|name|compiled_files|files) took 1.278350021s 
INFO [runner/filename_unadjuster] Pre-built 0 adjustments in 98.160985ms 

INFO Memory: 601 samples, avg is 441.7MB, max is 453.7MB 
INFO Execution took 1m0.000260426s                
INFO [linters_context/goanalysis] analyzers took 6m59.362370889s with top 10 stages: gosec: 6m59.360517105s, typecheck: 1.853784ms 
INFO [runner] Issues before processing: 56, after processing: 0 
INFO [runner] Processors filtering stat (out/in): exclude-rules: 10/47, identifier_marker: 47/47, nolint: 0/10, skip_files: 56/56, autogenerated_exclude: 47/56, exclude: 47/47, filename_unadjuster: 56/56, path_prettifier: 56/56, cgo: 56/56, skip_dirs: 56/56 
INFO [runner] processing took 5.211167ms with stages: nolint: 2.466589ms, autogenerated_exclude: 1.393037ms, exclude-rules: 541.334µs, identifier_marker: 390.287µs, path_prettifier: 250.543µs, skip_dirs: 157.991µs, cgo: 5.333µs, filename_unadjuster: 2.913µs, max_same_issues: 909ns, skip_files: 365ns, fixer: 350ns, diff: 293ns, sort_results: 263ns, severity-rules: 188ns, exclude: 187ns, uniq_by_line: 185ns, max_from_linter: 113ns, source_code: 95ns, max_per_file_from_linter: 67ns, path_shortener: 63ns, path_prefixer: 62ns 
INFO [runner] linters took 6m58.151346821s with stages: goanalysis_metalinter: 6m58.146074082s 
INFO File cache stats: 0 entries of total size 0B 
ERRO Timeout exceeded: try increasing it by passing --timeout option 

Running my default linter set with gosec disabled finishes in seconds:

$ golangci-lint version
golangci-lint has version 1.54.2 built with go1.21.0 from 411e0bbb on 2023-08-21T12:04:32Z

$ golangci-lint run -v ./...      

INFO [config_reader] Used config file .golangci.yml 
INFO [lintersdb] Active 46 linters: [asasalint asciicheck bidichk bodyclose decorder dogsled durationcheck errcheck errorlint execinquery exportloopref gochecknoinits gofmt goheader goimports gomoddirectives gomodguard goprintffuncname gosimple govet grouper importas loggercheck makezero misspell nakedret nestif nilerr noctx nolintlint nosprintfhostport predeclared promlinter reassign revive rowserrcheck staticcheck stylecheck tenv tparallel unconvert unparam unused usestdlibvars wastedassign whitespace] 
INFO [loader] Go packages loading at mode 575 (compiled_files|deps|types_sizes|exports_file|files|imports|name) took 1.334268938s 
INFO [runner/filename_unadjuster] Pre-built 0 adjustments in 92.982021ms 
INFO [linters_context] importas settings found, but no aliases listed. List aliases under alias: key. 
INFO [linters_context/goanalysis] analyzers took 0s with no stages 
INFO [runner/skip_dirs] Skipped 626 issues from dir applications/<test>/graphql/gqlserver by pattern applications/<test>/graphql/gqlserver 
INFO [runner] Issues before processing: 1595, after processing: 1 
INFO [runner] Processors filtering stat (out/in): exclude-rules: 462/889, fixer: 1/1, max_from_linter: 1/1, source_code: 1/1, path_prefixer: 1/1, sort_results: 1/1, nolint: 2/462, cgo: 1595/1595, filenameunadjuster: 1595/1595, path_prettifier: 1595/1595, skip_files: 1587/1595, skip_dirs: 961/1587, max_per_file_from_linter: 1/1, max_same_issues: 1/1, path_shortener: 1/1, autogenerated_exclude: 889/961, identifier_marker: 889/889, exclude: 889/889, uniq_by_line: 1/2, diff: 1/1, severity-rules: 1/1 
INFO [runner] processing took 40.59585ms with stages: nolint: 17.332562ms, exclude-rules: 11.281626ms, identifier_marker: 5.99838ms, path_prettifier: 2.925171ms, autogenerated_exclude: 2.204377ms, skip_dirs: 617.777µs, cgo: 93.45µs, skip_files: 73.536µs, filename_unadjuster: 46.562µs, source_code: 16.282µs, max_same_issues: 1.457µs, uniq_by_line: 1.375µs, path_shortener: 728ns, max_from_linter: 618ns, max_per_file_from_linter: 536ns, fixer: 337ns, exclude: 304ns, sort_results: 266ns, severity-rules: 222ns, diff: 218ns, path_prefixer: 66ns 
INFO [runner] linters took 153.458431ms with stages: goanalysis_metalinter: 112.812073ms 
INFO File cache stats: 1 entries of total size 10.8KiB 
INFO Memory: 17 samples, avg is 34.7MB, max is 101.6MB 
INFO Execution took 1.584228822s 

Running gosec manually runs quickly as well:

$ time gosec --exclude-dir=applications/<test>/graphql/gqlserver ./...  
35.50s user 14.62s system 1714% cpu 2.923 total

Version of golangci-lint

$ golangci-lint --version
golangci-lint has version 1.54.2 built with go1.21.0 from 411e0bbb on 2023-08-21T12:04:32Z

Configuration

golangci-lint run --no-config --disable-all --enable gosec ./...

Go environment

$ go version && go env
go version go1.20.4 linux/amd64
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/<me>/.cache/go-build"
GOENV="/home/<me>/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS="-mod=mod"
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/<me>/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/<me>/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.20.4"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/<me>/code/<project>/go.mod"
GOWORK=""
CGO_CFLAGS="-O2 -g"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-O2 -g"
CGO_FFLAGS="-O2 -g"
CGO_LDFLAGS="-O2 -g"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build123119041=/tmp/go-build -gno-record-gcc-switches"

Verbose output of running

$ golangci-lint cache clean
$ golangci-lint run -v --no-config --disable-all --enable gosec ./...
# paste output here

A minimal reproducible example or link to a public repository

// N/A

Validation

  • Yes, I've included all information above (version, config, etc.).

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingdependenciesRelates to an upstream dependencyfeedback requiredRequires additional feedback

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions