Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

debug: hard to debug a binary built with -trimpath #1985

Open
hyangah opened this issue Dec 31, 2021 · 4 comments
Open

debug: hard to debug a binary built with -trimpath #1985

hyangah opened this issue Dec 31, 2021 · 4 comments
Labels
debug/remote issues related to remote debugging support Debug Issues related to the debugging functionality of the extension. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.

Comments

@hyangah
Copy link
Contributor

hyangah commented Dec 31, 2021

go version: go1.18beta1 (but the problem applies to older versions of go too)
dlv version: 2f13672765fe

When a binary is built with -trimpath, part of file paths are removed from the debug info.

It is recommended to configure substitutePath to handle this.
But, it is challenging to come up with good substitutePath rules since -trimpath produces file paths in the form of:

  • standard libs: /
    e.g.: runtime/map.go
  • 3rd party dependencies in module cache: module@version/package/path/file.go
    e.g.: golang.org/x/mod@v0.5.1/semver/semver.go (this can be interpreted as a path relative to MODCACHE)
  • files in main module (workspace): module/package/path/file.go
    e.g: work/main.go (when the main module's name is work)

More complication:

  • vendored 3rd party dependencies or replaced 3rd party dependencies
    -> original module@version/package/path/file.go is used instead of file path in vendor or replace directory
<autogenerated>
golang.org/x/mod@v0.5.1/semver/semver.go
...
runtime/map.go
sort/sort.go
...
work/main.go

Repro:

~/ww % cat go.mod
module work

go 1.17

require golang.org/x/mod v0.5.1

~/ww % cat main.go
package main

import (
        "golang.org/x/mod/semver"
)

func main() {
        if semver.IsValid("1.0.0") {
                println("valid")
        }
}

Launch configuration

        {
            "name": "Launch Package",
            "type": "go",
            "request": "launch",
            "mode": "debug",
            "program": "${fileDirname}",
            "buildFlags": "-trimpath",
            "stopOnEntry": true,
            "showLog": true,
            "logOutput": "dap"
        }

Delve DAP log

2021-12-31T17:18:43-05:00 debug layer=dap [<- from client]{"seq":15,"type":"request","command":"disconnect","arguments":{"restart":true,"terminateDebuggee":true}}
2021-12-31T17:18:43-05:00 debug layer=dap halting
2021-12-31T17:18:43-05:00 debug layer=dap process not running
2021-12-31T17:18:43-05:00 debug layer=dap [-> to client]{"seq":0,"type":"event","event":"output","body":{"category":"console","output":"Detaching and terminating target process\n","source":{}}}
Detaching and terminating target process
2021-12-31T17:18:43-05:00 debug layer=dap [-> to client]{"seq":0,"type":"response","request_seq":15,"success":true,"command":"disconnect"}
2021-12-31T17:18:43-05:00 debug layer=dap [-> to client]{"seq":0,"type":"event","event":"terminated","body":{}}
2021-12-31T17:18:43-05:00 debug layer=dap DAP server stopping...
2021-12-31T17:18:43-05:00 debug layer=dap DAP server stopped
dlv dap (81795) exited with code: 0
Starting: /Users/hakim/go/bin/dlv-dap dap --check-go-version=false --listen=127.0.0.1:50368 --log=true --log-output=dap --log-dest=3 from /Users/hakim/ww
DAP server listening at: 127.0.0.1:50368
2021-12-31T17:18:43-05:00 debug layer=dap DAP server pid = 81923
2021-12-31T17:18:43-05:00 debug layer=dap DAP connection 1 started
2021-12-31T17:18:43-05:00 debug layer=dap [<- from client]{"seq":1,"type":"request","command":"initialize","arguments":{"clientID":"vscode","clientName":"Visual Studio Code","adapterID":"go","locale":"en-us","linesStartAt1":true,"columnsStartAt1":true,"pathFormat":"path","supportsVariableType":true,"supportsVariablePaging":true,"supportsRunInTerminalRequest":true,"supportsMemoryReferences":true,"supportsProgressReporting":true,"supportsInvalidatedEvent":true}}
2021-12-31T17:18:43-05:00 debug layer=dap [-> to client]{"seq":0,"type":"response","request_seq":1,"success":true,"command":"initialize","body":{"supportsConfigurationDoneRequest":true,"supportsFunctionBreakpoints":true,"supportsConditionalBreakpoints":true,"supportsEvaluateForHovers":true,"supportsSetVariable":true,"supportsExceptionInfoRequest":true,"supportTerminateDebuggee":true,"supportsDelayedStackTraceLoading":true,"supportsLogPoints":true,"supportsDisassembleRequest":true,"supportsClipboardContext":true,"supportsSteppingGranularity":true,"supportsInstructionBreakpoints":true}}
2021-12-31T17:18:43-05:00 debug layer=dap [<- from client]{"seq":2,"type":"request","command":"launch","arguments":{"name":"Launch Package","type":"go","request":"launch","mode":"debug","program":".","buildFlags":"-trimpath","stopOnEntry":true,"showLog":true,"logOutput":"dap","__configurationTarget":5,"packagePathToGoModPathMap":{"/Users/hakim/ww":"/Users/hakim/ww"},"debugAdapter":"dlv-dap","dlvToolPath":"/Users/hakim/go/bin/dlv-dap","env":{},"__buildDir":"/Users/hakim/ww","__sessionId":"23c200ff-23a4-4336-9b50-c03565f1a05c"}}
2021-12-31T17:18:43-05:00 debug layer=dap parsed launch config: {
	"mode": "debug",
	"program": ".",
	"buildFlags": "-trimpath",
	"stopOnEntry": true,
	"backend": "default",
	"stackTraceDepth": 50
}
2021-12-31T17:18:43-05:00 debug layer=dap building from "/Users/hakim/ww": [go build -o /var/folders/bw/6r6k9d113sv1_vvzk_1kfxbm001py5/T/__debug_bin2675042589 -gcflags all=-N -l -trimpath .]
2021-12-31T17:18:43-05:00 debug layer=dap launching binary '/var/folders/bw/6r6k9d113sv1_vvzk_1kfxbm001py5/T/__debug_bin2675042589' with config: {
	"mode": "debug",
	"program": "/Users/hakim/ww",
	"cwd": "/Users/hakim/ww",
	"buildFlags": "-trimpath",
	"output": "/var/folders/bw/6r6k9d113sv1_vvzk_1kfxbm001py5/T/__debug_bin2675042589",
	"dlvCwd": "/Users/hakim/ww",
	"stopOnEntry": true,
	"backend": "default",
	"stackTraceDepth": 50
}
WARNING: undefined behavior - version of Delve is too old for Go version 1.18.-1 (maximum supported version 1.17)
2021-12-31T17:18:44-05:00 debug layer=dap [-> to client]{"seq":0,"type":"event","event":"initialized"}
2021-12-31T17:18:44-05:00 debug layer=dap [-> to client]{"seq":0,"type":"response","request_seq":2,"success":true,"command":"launch"}
2021-12-31T17:18:44-05:00 debug layer=dap [<- from client]{"seq":3,"type":"request","command":"setBreakpoints","arguments":{"source":{"name":"main.go","path":"/Users/hakim/ww/main.go"},"breakpoints":[{"line":8}],"lines":[8]}}
2021-12-31T17:18:44-05:00 debug layer=dap [-> to client]{"seq":0,"type":"response","request_seq":3,"success":true,"command":"setBreakpoints","body":{"breakpoints":[{"verified":false,"message":"could not find file /Users/hakim/ww/main.go","source":{}}]}}
2021-12-31T17:18:44-05:00 debug layer=dap [<- from client]{"seq":4,"type":"request","command":"setFunctionBreakpoints","arguments":{"breakpoints":[]}}
2021-12-31T17:18:44-05:00 debug layer=dap [-> to client]{"seq":0,"type":"response","request_seq":4,"success":true,"command":"setFunctionBreakpoints","body":{"breakpoints":[]}}
2021-12-31T17:18:44-05:00 debug layer=dap [<- from client]{"seq":5,"type":"request","command":"setInstructionBreakpoints","arguments":{"breakpoints":[]}}
2021-12-31T17:18:44-05:00 debug layer=dap [-> to client]{"seq":0,"type":"response","request_seq":5,"success":true,"command":"setInstructionBreakpoints","body":{"breakpoints":[]}}
2021-12-31T17:18:44-05:00 debug layer=dap [<- from client]{"seq":6,"type":"request","command":"configurationDone","arguments":{}}
2021-12-31T17:18:44-05:00 debug layer=dap [-> to client]{"seq":0,"type":"event","event":"stopped","body":{"reason":"entry","threadId":1,"allThreadsStopped":true}}
2021-12-31T17:18:44-05:00 debug layer=dap [-> to client]{"seq":0,"type":"event","event":"output","body":{"category":"console","output":"Type 'dlv help' for list of commands.\n","source":{}}}
2021-12-31T17:18:44-05:00 debug layer=dap [-> to client]{"seq":0,"type":"response","request_seq":6,"success":true,"command":"configurationDone"}
Type 'dlv help' for list of commands.
2021-12-31T17:18:44-05:00 debug layer=dap [<- from client]{"seq":7,"type":"request","command":"threads"}
2021-12-31T17:18:44-05:00 debug layer=dap [-> to client]{"seq":0,"type":"response","request_seq":7,"success":true,"command":"threads","body":{"threads":[{"id":1,"name":"Dummy"}]}}
2021-12-31T17:18:44-05:00 debug layer=dap [<- from client]{"seq":8,"type":"request","command":"threads"}
2021-12-31T17:18:44-05:00 debug layer=dap [-> to client]{"seq":0,"type":"response","request_seq":8,"success":true,"command":"threads","body":{"threads":[{"id":1,"name":"Dummy"}]}}
2021-12-31T17:18:44-05:00 debug layer=dap [<- from client]{"seq":9,"type":"request","command":"stackTrace","arguments":{"threadId":1,"levels":1,"format":{}}}
2021-12-31T17:18:44-05:00 debug layer=dap Unable to produce stack trace: unknown goroutine 1
2021-12-31T17:18:44-05:00 debug layer=dap [-> to client]{"seq":0,"type":"response","request_seq":9,"success":false,"command":"stackTrace","message":"Unable to produce stack trace","body":{"error":{"id":2004,"format":"Unable to produce stack trace: unknown goroutine 1","showUser":false}}}
2021-12-31T17:18:44-05:00 debug layer=dap [<- from client]{"seq":10,"type":"request","command":"stackTrace","arguments":{"threadId":1,"levels":20,"format":{}}}
2021-12-31T17:18:44-05:00 debug layer=dap Unable to produce stack trace: unknown goroutine 1
2021-12-31T17:18:44-05:00 debug layer=dap [-> to client]{"seq":0,"type":"response","request_seq":10,"success":false,"command":"stackTrace","message":"Unable to produce stack trace","body":{"error":{"id":2004,"format":"Unable to produce stack trace: unknown goroutine 1","showUser":false}}}
2021-12-31T17:18:44-05:00 debug layer=dap [<- from client]{"seq":11,"type":"request","command":"stackTrace","arguments":{"threadId":1,"levels":19,"format":{}}}
2021-12-31T17:18:44-05:00 debug layer=dap Unable to produce stack trace: unknown goroutine 1
2021-12-31T17:18:44-05:00 debug layer=dap [-> to client]{"seq":0,"type":"response","request_seq":11,"success":false,"command":"stackTrace","message":"Unable to produce stack trace","body":{"error":{"id":2004,"format":"Unable to produce stack trace: unknown goroutine 1","showUser":false}}}
dlv sources main.go
2021-12-31T17:18:51-05:00 debug layer=dap [<- from client]{"seq":12,"type":"request","command":"evaluate","arguments":{"expression":"dlv sources main.go","context":"repl","format":{}}}
2021-12-31T17:18:51-05:00 debug layer=dap [-> to client]{"seq":0,"type":"response","request_seq":12,"success":true,"command":"evaluate","body":{"result":"work/main.go","presentationHint":{},"variablesReference":0}}
work/main.go
dlv sources semver.go
2021-12-31T17:18:56-05:00 debug layer=dap [<- from client]{"seq":13,"type":"request","command":"evaluate","arguments":{"expression":"dlv sources semver.go","context":"repl","format":{}}}
2021-12-31T17:18:56-05:00 debug layer=dap [-> to client]{"seq":0,"type":"response","request_seq":13,"success":true,"command":"evaluate","body":{"result":"golang.org/x/mod@v0.5.1/semver/semver.go","presentationHint":{},"variablesReference":0}}
golang.org/x/mod@v0.5.1/semver/semver.go
@hyangah hyangah added the Debug Issues related to the debugging functionality of the extension. label Dec 31, 2021
@gopherbot gopherbot added this to the Untriaged milestone Dec 31, 2021
@hyangah hyangah added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jan 4, 2022
@hyangah hyangah modified the milestones: Untriaged, Unplanned Jan 4, 2022
@hyangah
Copy link
Contributor Author

hyangah commented Jan 4, 2022

@suzmue @polinasok This may affect blaze users and google internal binary debugging.

@polinasok
Copy link
Contributor

Related discussion in go-delve/delve#2754

@polinasok polinasok added the debug/remote issues related to remote debugging support label Mar 30, 2022
@firelizzard18
Copy link
Contributor

I think this is a better place for it, so I'm copying my comment from #2609.

Capitalization is another potential pitfall. The project I am working on has a dependency with a capitalized name in the path. With -trimpath the binary has paths such as github.com/AccumulateNetwork/jsonrpc2/v15@v15.0.0-20220517212445-953ad957e040/handler.go which needs to be resolved to /home/ME/go/pkg/mod/github.com/!accumulate!network/jsonrpc2/v15@v15.0.0-20220517212445-953ad957e040/handler.go. I have { "to": "", "from": "${env:HOME}/go/pkg/mod/" } as my last substitution rule, but that doesn't cover the capitalization case. Ultimately the conflict is that the go module system uses the !a substitution rule for filesystem paths, but does not apply that rule when synthesizing paths for -trimpath.

@dlipovetsky
Copy link
Contributor

In fa820d4, we document a strategy for constructing substitution paths, namely:

"substitutePath": [
  // Main module.
  {
   "from": "${workspaceFolder}",
   "to": "moduleName",
  },
  // Module cache paths.
  {
   "from": "${env:HOME}/go/pkg/mod/github.com",
   "to": "github.com",
  },
  {
   "from": "${env:HOME}/go/pkg/mod/golang.org",
   "to": "golang.org",
  },
  ...
  // Standard library paths.
  // This rule should come last since the empty "to" will match every path.
  { "from": "/path/to/local/goroot/pkg" , "to": ""}
 ],

I use a different strategy. My main module depends on many modules in the cache, and this set of modules changes over time. The standard library packages, on the other hand, are relatively stable. I wanted to avoid enumerating the modules in the cache, so instead I enumerated the standard library packages.

Because the standard library packages have no prefix (e.g. just fmt/errors.go, rather than go/fmt/errors.go), I had to create a substitution for every package. I also used the executable's go version to create the substitutions. (I use different go versions at the same time, so I needed to use the correct one for this executable.)

"substitutePath": [
    // Main module, built with -trimpath
    {
        "from": "${workspaceFolder}",
        "to": "moduleName",
    },
    // Packages in GOROOT (standard library)
    {
        "from": "/home/myuser/.local/go/1.20.1/src/archive",
        "to": "archive"
    },
    // ... Other entries omitted to save space ...
    {
        "from": "/home/myuser/.local/go/1.20.1/src/vendor",
        "to": "vendor"
    },
    // Modules in GOMODCACHE
    {
        "from": "/home/myuser/.local/go/pkg/mod",
        "to": "",
    },
]

While I find my strategy easier to use manually, I think would be easier to automatically generate substitutions for modules in the module cache. We would still need to generate one substitution for the standard library packages, and for that we would need (a) the go version of the executable (available as metadata in the executable), and (b) the path to the GOROOT of that go version (must be provided by the user).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
debug/remote issues related to remote debugging support Debug Issues related to the debugging functionality of the extension. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

5 participants