What's linters_context/goanalysis and why it is takes so much time on CI

[nickaein]

I have been experiencing slow lint on CI and tried to debug it using --verbose (logs are included below). There seems to be a linters_context/goanalysis job/stage that take almost zero time on local machine, but it takes around 2mins when running on CI.

As can been seen in the following logs, on my local machine I'm getting analyzers took 0s with no stages log for this job. Is it being ignored for some reason, e.g. is is there a cache in play? I do a go build in CI before linting (as suggested here), but that doesn't seem to help this stage.

golangci-lint version: 1.52.2

Here is the log when running local (takes ~700ms):

golangci-lint has version 1.52.2 built with go1.20.2 from da04413a on 2023-03-25T18:11:28Z
golangci-lint run --verbose --config=.golangci-lint.yml ./... --exclude comment
INFO [config_reader] Used config file .golangci-lint.yml 
INFO [lintersdb] Active 12 linters: [errcheck gofmt gosec gosimple govet ineffassign lll revive staticcheck typecheck unconvert unused] 
INFO [loader] Go packages loading at mode 575 (imports|compiled_files|deps|name|types_sizes|exports_file|files) took 398.976024ms 
INFO [runner/filename_unadjuster] Pre-built 0 adjustments in 14.286312ms 
INFO [linters_context/goanalysis] analyzers took 0s with no stages 
INFO [runner/skip_dirs] Skipped 45 issues from dir interfaces/mocks by pattern mocks 
INFO [runner] Issues before processing: 237, after processing: 0 
INFO [runner] Processors filtering stat (out/in): cgo: 237/237, filename_unadjuster: 237/237, autogenerated_exclude: 192/192, exclude: 0/192, path_prettifier: 237/237, skip_files: 237/237, identifier_marker: 192/192, skip_dirs: 192/237 
INFO [runner] processing took 5.208429ms with stages: identifier_marker: 3.512927ms, path_prettifier: 612.205µs, autogenerated_exclude: 575.171µs, exclude: 236.99µs, skip_dirs: 221.681µs, cgo: 27.382µs, filename_unadjuster: 17.677µs, max_same_issues: 818ns, nolint: 770ns, skip_files: 647ns, fixer: 295ns, max_from_linter: 295ns, uniq_by_line: 268ns, source_code: 251ns, exclude-rules: 203ns, diff: 180ns, severity-rules: 177ns, sort_results: 167ns, path_shortener: 162ns, max_per_file_from_linter: 90ns, path_prefixer: 73ns 
INFO [runner] linters took 319.495523ms with stages: goanalysis_metalinter: 314.222874ms 
INFO File cache stats: 0 entries of total size 0B 
INFO Memory: 9 samples, avg is 45.3MB, max is 64.4MB 
INFO Execution took 735.195196ms 

Here is the log in CI (takes ~2min):

golangci-lint has version 1.52.2 built with go1.20.2 from da04413a on 2023-03-25T18:11:28Z
golangci-lint run --verbose --config=.golangci-lint.yml ./... --exclude comment
level=info msg="[config_reader] Used config file .golangci-lint.yml"
level=info msg="[lintersdb] Active 12 linters: [errcheck gofmt gosec gosimple govet ineffassign lll revive staticcheck typecheck unconvert unused]"
level=info msg="[loader] Go packages loading at mode 575 (name|exports_file|files|imports|types_sizes|compiled_files|deps) took 2.026523491s"
level=info msg="[runner/filename_unadjuster] Pre-built 0 adjustments in 16.818378ms"
level=info msg="[linters_context/goanalysis] analyzers took 2m14.210196198s with top 10 stages: buildir: 1m53.096509277s, nilness: 3.455158096s, inspect: 3.254425348s, the_only_name: 3.077852533s, fact_purity: 1.929488278s, printf: 1.827449362s, ctrlflow: 1.637858208s, unconvert: 1.512965624s, fact_deprecated: 1.444998676s, typedness: 1.118064453s"
level=info msg="[runner/skip_dirs] Skipped 45 issues from dir interfaces/mocks by pattern mocks"
level=info msg="[runner] Issues before processing: 237, after processing: 0"
level=info msg="[runner] Processors filtering stat (out/in): skip_dirs: 192/237, exclude: 0/192, path_prettifier: 237/237, skip_files: 237/237, cgo: 237/237, identifier_marker: 192/192, filename_unadjuster: 237/237, autogenerated_exclude: 192/192"
level=info msg="[runner] processing took 6.636919ms with stages: identifier_marker: 4.774559ms, autogenerated_exclude: 723.805µs, path_prettifier: 477.354µs, exclude: 376.534µs, skip_dirs: 219.315µs, cgo: 35.28µs, filename_unadjuster: 19.521µs, max_same_issues: 2.062µs, nolint: 1.882µs, diff: 1.239µs, uniq_by_line: 1.181µs, skip_files: 815ns, source_code: 786ns, fixer: 551ns, sort_results: 365ns, severity-rules: 358ns, exclude-rules: 358ns, path_shortener: 350ns, max_from_linter: 340ns, path_prefixer: 141ns, max_per_file_from_linter: 123ns"
level=info msg="[runner] linters took 2m27.442162135s with stages: goanalysis_metalinter: 2m27.435381399s"
level=info msg="File cache stats: 0 entries of total size 0B"
level=info msg="Memory: 1496 samples, avg is 862.7MB, max is 1384.0MB"
level=info msg="Execution took 2m29.490664378s"

[ldez]

Hello,

INFO [runner/filename_unadjuster] Pre-built 0 adjustments in 14.286312ms

This log means that you are using the golangci-lint cache.

To clean the cache:

$ golangci-lint cache clean

To improve your CI, you have to "backup/restore" the cache.

[nickaein]

With a cleared cache I still get this in logs

INFO [runner/filename_unadjuster] Pre-built 0 adjustments in 9.426934ms 

but the lint time on the local machine is now indeed in the order of CI time (50sec).

To improve your CI, you have to "backup/restore" the cache.

I will take a look into that. Thanks for the tip!