x/tools/gopls: crash importing test variant of tools file

[painhardcore]

gopls version: v0.15.1/go1.22.0
gopls flags:
update flags: proxy
extension version: 0.41.1
environment: Visual Studio Code darwin
initialization error: undefined
issue timestamp: Fri, 01 Mar 2024 00:31:32 GMT
restart history:
Fri, 01 Mar 2024 00:30:44 GMT: activation (enabled: true)

Crashing gopls

panic: nil metadata for "github.com/mailru/easyjson/easyjson [command-line-arguments.test]"

goroutine 7085 [running]:
golang.org/x/tools/gopls/internal/cache.(*Snapshot).getPackageHandles.func1(0x1400d4f3950, {0x14009c2b040, 0x41})
	/Users/painhardcore/go/pkg/mod/golang.org/x/tools/gopls@v0.15.1/internal/cache/check.go:872 +0x53c
golang.org/x/tools/gopls/internal/cache.(*Snapshot).getPackageHandles.func1(0x0, {0x14009c54500, 0x7a})
	/Users/painhardcore/go/pkg/mod/golang.org/x/tools/gopls@v0.15.1/internal/cache/check.go:889 +0x468
golang.org/x/tools/gopls/internal/cache.(*Snapshot).getPackageHandles(0x140021d59e0, {0x10104c648, 0x14008365590}, {0x1400b2c9b08, 0x1, 0x0?})
	/Users/painhardcore/go/pkg/mod/golang.org/x/tools/gopls@v0.15.1/internal/cache/check.go:899 +0x17c
golang.org/x/tools/gopls/internal/cache.(*Snapshot).forEachPackage(0x140021d59e0, {0x10104c648, 0x14008365320}, {0x1400b2c9b08, 0x1, 0x1}, 0x1400a266c80, 0x1400cb41ca0)
	/Users/painhardcore/go/pkg/mod/golang.org/x/tools/gopls@v0.15.1/internal/cache/check.go:343 +0x134
golang.org/x/tools/gopls/internal/cache.(*Snapshot).PackageDiagnostics(0x140021d59e0, {0x10104c680?, 0x1400582e6e0?}, {0x1400b2c9b08, 0x1, 0x1})
	/Users/painhardcore/go/pkg/mod/golang.org/x/tools/gopls@v0.15.1/internal/cache/snapshot.go:686 +0x19c
golang.org/x/tools/gopls/internal/cache.(*Snapshot).orphanedFileDiagnostics(0x140021d59e0, {0x10104c680, 0x1400582e6e0}, {0x1400a266c00, 0x3, 0x10104aaf8?})
	/Users/painhardcore/go/pkg/mod/golang.org/x/tools/gopls@v0.15.1/internal/cache/snapshot.go:1452 +0x1c8
golang.org/x/tools/gopls/internal/cache.(*Session).OrphanedFileDiagnostics(0x1400048e150, {0x10104c680, 0x1400582e6e0})
	/Users/painhardcore/go/pkg/mod/golang.org/x/tools/gopls@v0.15.1/internal/cache/session.go:1165 +0x474
golang.org/x/tools/gopls/internal/server.(*server).diagnoseChangedViews(0x14000190000, {0x10104c680, 0x1400582e6e0}, 0x3, 0x140086b0b40, 0x0)
	/Users/painhardcore/go/pkg/mod/golang.org/x/tools/gopls@v0.15.1/internal/server/diagnostics.go:157 +0x4f0
golang.org/x/tools/gopls/internal/server.(*server).didModifyFiles.func2()
	/Users/painhardcore/go/pkg/mod/golang.org/x/tools/gopls@v0.15.1/internal/server/text_synchronization.go:269 +0x3c
created by golang.org/x/tools/gopls/internal/server.(*server).didModifyFiles in goroutine 108
	/Users/painhardcore/go/pkg/mod/golang.org/x/tools/gopls@v0.15.1/internal/server/text_synchronization.go:268 +0x468
[Error - 01:31:32] 

gopls stats -anon

{ "DirStats": { "Files": 8067, "TestdataFiles": 52, "GoFiles": 6683, "ModFiles": 4, "Dirs": 1348 }, "GOARCH": "arm64", "GOOS": "darwin", "GOPACKAGESDRIVER": "", "GOPLSCACHE": "", "GoVersion": "go1.22.0", "GoplsVersion": "v0.15.1", "InitialWorkspaceLoadDuration": "1.65849025s", "MemStats": { "HeapAlloc": 108681552, "HeapInUse": 144924672, "TotalAlloc": 294147416 }, "WorkspaceStats": { "Files": { "Total": 6971, "Largest": 7361676, "Errs": 0 }, "Views": [ { "GoCommandVersion": "go1.22.0", "AllPackages": { "Packages": 1426, "LargestPackage": 214, "CompiledGoFiles": 8363, "Modules": 177 }, "WorkspacePackages": { "Packages": 593, "LargestPackage": 214, "CompiledGoFiles": 4074, "Modules": 1 }, "Diagnostics": 0 } ] } }

[findleyr]

Thanks for the report. This looks like a dupe of golang/vscode-go#3207, and it's very interesting that the package ID in question is exactly the same. Are you by any chance a colleague of @amirrmonfared? I ask because if this isn't coming from the exact same code base, it's possible that there's something about the package layout of easyjson that exacerbates this bug.

[findleyr]

@painhardcore I should also ask: does this crash occur in an open source repository, from which we could try to reproduce?

[painhardcore]

@findleyr Yep, his my colleague and has similar issue. Not sure about open source repo that we can match(we have vendored packages and easyjson)...

1.15.0 and 1.15.1 are affected and have same panic trace, but 1.14.2 works fine

But so far I found a small trick to work

1. Close VScode
2. Downgrade to stable that works go install golang.org/x/tools/gopls@v0.14.2
3. Open VScode

VScode will update the gopls anyway, but the current gopls instance wont be effected. So if I need other brach in a new window(I use git worktree) - I just downgrade it before opening new VScode.

[findleyr]

But so far I found a small trick to work

To be clear, downgrading is not a long-term solution. We need to fix this bug.

I believe this problem is related to the use of a tools.go pattern, as is described here:
https://marcofranssen.nl/manage-go-tools-via-go-modules

"github.com/mailru/easyjson/easyjson" is a command, not an importable package (i.e. it is package main), so the only reason to have it in an import statement would be to intentionally introduce a tool dependency. However, I have tried many different ways of importing it, and can't produce a crash.

Can you search for this import in your codebase, and share details about the file containing the import? If possible, I would love to see the full header of the file with the import (the package name and imports, as well as any //go:build comments). I'd also like to know whether the file is a _test.go file. Does the crash occur as soon as you open your repository, or only when the importing file is open?

[painhardcore]

Yep, we have a file, that we keep to have everything in vendor.

//go:build tools
// +build tools

// This file tracks toolings that aren't directly used by repo
// but they are required to build the project.
// Refer to https://github.com/golang/go/wiki/Modules#how-can-i-track-tool-dependencies-for-a-module
package back

import (
	_ "github.com/mailru/easyjson/easyjson"
	_ "github.com/quasilyte/go-ruleguard/dsl" // For .github/gorules/rules.go.
)

I don't understand how its connected with _test.go files, but apparently something triggers background scan or smth:

I can go and edit .go files and everything is OK. Until I go to _test.go file and do "go to definition" it's crashing.

But if I remove that file with tools - it works OK, and crashes only when I bring it back.

[painhardcore]

BTW the new link tools is https://go.dev/wiki/Modules#how-can-i-track-tool-dependencies-for-a-module

[painhardcore]

@findleyr and I found the issue, its because the file with these tools is called tools_test.go 🤣
And apparently if we name it tools.go - everything work without panics. We need to doublecheck with @amirrmonfared just to be sure.
I don't know it's a bug but it worked before.

[findleyr]

Thanks. For some reason I still can't reproduce, but this is enough information for me to put together a fix. Transferring to the gopls issue tracker.

[gopherbot]

Change https://go.dev/cl/569035 mentions this issue: gopls/internal/cache: prune broken edges to command-line-arguments pkgs

[painhardcore]

@findleyr reproducable repo https://github.com/painhardcore/goplscrash . Just go to tools/tools_test.go in VScode and gopls will crash.

[findleyr]

Indeed it does! Apparently the file must be in a nested directory. Thank you for the reproducer!

[painhardcore]

Yep, in our case file tools_test.go was in root folder, so every test file in a big repo was like a trigger 😄

[gopherbot]

Change https://go.dev/cl/569437 mentions this issue: [gopls-release-branch.0.15] gopls/internal/cache: prune broken edges to command-line-arguments pkgs

[findleyr]

@painhardcore and @amirrmonfared, huge thanks for your conscientious help resolving this bug. The CL above should fix this crash. However, we have a couple other issues to sort out before we cut gopls@v0.15.2 with this fix.

[painhardcore]

@painhardcore and @amirrmonfared, huge thanks for your conscientious help resolving this bug. The CL above should fix this crash. However, we have a couple other issues to sort out before we cut gopls@v0.15.2 with this fix.

Thanks.
I fixed the issue in our repo, so we can work on 0.15.1

[amirrmonfared]

Thanks a lot @findleyr for your support and thanks to @painhardcore for solving the issue in our repo ❤️

[findleyr]

This fix is included in our prerelease:

go install golang.org/x/tools/gopls@v0.15.2-pre.1