Rewrite debugger for Godot 4 support + improved maintainability

[DaelonSuzuka]

This is a continuation of #400 by @RedMser. I'm opening this as a draft PR so I have a place to keep some notes.

Current state:

The main thing holding this back, besides some issues with the debugger, is compat with 3.x.

Keeping compatibility with multiple Godot versions will probably double the size of this PR, due to having to create duplicates of every system (debugger, connection, etc.) which must be interchangeable. Not to speak of automatic version detection, which I'm not sure how it should be done just yet.

It may be possible to integrate the changes in #387 into this PR.

[Emi1305]

What's the status of this PR? I'm creating a new project on Godot 4 and this would be really helpful to me

[DaelonSuzuka]

What's the status of this PR?

Not done.

Due to some other PRs from the queue getting merged this week, it's possible that I'll be able to make progress on this soon ™️, but I'm not making any promises.

[DaelonSuzuka]

I have both debugger versions living together now. 3 works in both launch and attach mode, but 4 ONLY works in launch mode. I can't get it to connect to a running editor at all. There seem to be some changes in godot's debugger internals, but its difficult for me to tell the extent of them.

There are a number of other less severe problems with the scene tree or inspector or cleaning up after a session stops, but those should all be fixable with a little effort.

[moonraymurray]

Hello I ended up building this PR and testing it with my project in Godot 4, so far it is working correctly, had to make a few minor changes to my settings and launch to adjust to your modifications. I get debug messages now and also I can pause the code. Also seems as though the variables viewer and breakpoints work as intended. I would review the code but I am not well versed in JS it so I am just leaving a comment.
EDIT: I couldn't get the scene preview working but I have never really used that so I don't know what working looks like either. So far this PR ticks all the boxes, if I encounter anything else I will make another comment.

[DaelonSuzuka]

Thanks, @moonraymurray, that's great to hear. Can I assume you were using the debugger in launch mode, ie "launch_game_instance": true? I still haven't gotten attach mode to work ("launch_game_instance": false), and it would be very interesting if it that worked for you.

[moonraymurray]

using "launch_game_instance": true,
Yes I was able to use the debugger using that setting. Here is my launch code.

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Godot",
            "type": "godot4",
            "request": "launch",
            "project": "${workspaceFolder}",
            "port": 6007,
            "address": "127.0.0.1",
            "launch_game_instance": true,
            "launch_scene": false,
        }
    ]
}

And here is my settings.json, I am using a one or 2 other extensions aswell.

{
    "editor.inlineSuggest.enabled": true,
    "explorer.confirmDelete": false,
    "update.showReleaseNotes": false,
    "launch": {
        
        "configurations": [],
        "compounds": []
    },
    "theme-by-language.themes": {
        "*": "Default Dark+",
        "gdscript": "GDScript"
    },
    "godotTools.editorPath.godot4": "C:/Users/Neo/Downloads/Godot_v4.0.1-stable_win64.exe//Godot_v4.0.1-stable_win64.exe",
    "godotTools.lsp.serverPort": 6005,
}

I had to delete references to the other editor paths(default or 3) to get it to run. Sorry about the formatting

[DaelonSuzuka]

Made some progress on the Godot 3 debugger. I removed the launch_game_instance: true configuration option because it's the same thing as the request: launch option, which is an actual VSCode feature. After implementing the various API interfaces and splitting the code, it looks like most of the behavior for both session types is correct now. Some issues remain around closing the game/killing the process but the actual connection works now.

My next task is to doing the same thing to the Godot 4 side, and then hopefully these improvements will help me figure out why Godot 4 attach sessions don't work.

I intend to call this PR done once all 4 debugger modes (G3 Launch, G3 Attach, G4 Launch, G4 Attach) are working more-or-less correctly.

[Emi1305]

I'm not able to use the debugger (Compiled from you branch) on windows with the following launch.json file:

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "GDScript Godot",
            "type": "godot4",
            "request": "launch",
            "project": "${workspaceFolder}",
            "port": 6007,
            "debugServer": 6006,
            "address": "127.0.0.1",
            "launch_game_instance": true,
            "launch_scene": true
        }
    ]
}

Using wireshark to debug the communication I observe this error message:

{"body":{"error":{"format":"The editor and client are working on different paths; the client is on \"{clientPath}\", but the editor is on \"{editorPath}\"","id":1,"sendTelemetry":false,"showUser":false,"variables":{"clientPath":"C:\\Users\\Emiliano\\Documents\\Sound of Defeat","editorPath":"C:/Users/Emiliano/Documents/Sound of Defeat"}}},"command":"launch","message":"wrong_path","request_seq":2,"seq":3,"success":false,"type":"response"}

The error comes from vscode sending the project directory using \\ to split the directories while godot users /. Hardcoding the path to use / on launch.json is a workaround to get it working

Edit: I just noticed that, while the workaround allows me to launch the game and the external debugger to get attached. Breakpoints are not synced due to similar issue. This is the output I see on wireshark:

{"command":"setBreakpoints","arguments":{"source":{"name":"Player3D.gd","path":"c:\\Users\\Emiliano\\Documents\\Sound of Defeat\\src\\player\\Player3D.gd"},"lines":[],"breakpoints":[],"sourceModified":false},"type":"request","seq":7}

{"body":{"error":{"format":"The editor and client are working on different paths; the client is on \"{clientPath}\", but the editor is on \"{editorPath}\"","id":1,"sendTelemetry":false,"showUser":false,"variables":{"clientPath":"c:\\Users\\Emiliano\\Documents\\Sound of Defeat\\src\\player\\Player3D.gd","editorPath":"C:/Users/Emiliano/Documents/Sound of Defeat"}}},"command":"setBreakpoints","message":"wrong_path","request_seq":7,"seq":497,"success":false,"type":"response"}

[DaelonSuzuka]

@Emi1305 That's excellent information, thank you. I'm planning to get some work done on the godot 4 side of things this week, and with any luck I can get this path stuff sorted out.

It would be great if you could check back and test it again next weekend, maybe.

[clorl]

Would love to see this become a reality in the near future

[seffa1]

This is amazing thank you for the work you've done so far on this !

[DaelonSuzuka]

Quick status update: I've mostly completely a massive internal refactor to simplify the debugger code. Lots of features still don't work or don't work perfectly, but now it should actually be possible for me to trace out what's happening.

• Godot 3
  • launch: mostly working, some issues with closing/stopping game and resetting cleanly
  • attach: working, some issues with resetting cleanly
  • output: working
  • breakpoints: mostly working
  • scene tree: mostly working
  • inspector: mostly working
• Godot 4
  • launch: mostly working,
  • attach: not working
  • output: working
  • breakpoints: game stops, vscode ui does not react
  • scene tree: mostly working
  • inspector: not working

Features I haven't even touched yet:

• stack dumps
• variable inspection
• watch points
• evaluating expressions

I want to get sessions (launch/attach, starting, connecting, stopping, resetting) and breakpoints(pause/resume, stepping, stop) working solidly in both versions, and then I can get some volunteer testers and assess the remaining feature list.

[2fd5]

I want to get sessions (launch/attach, starting, connecting, stopping, resetting) and breakpoints(pause/resume, stepping, stop) working solidly in both versions, and then I can get some volunteer testers and assess the remaining feature list.

Count me in for testing

[ryanabx]

I want to get sessions (launch/attach, starting, connecting, stopping, resetting) and breakpoints(pause/resume, stepping, stop) working solidly in both versions, and then I can get some volunteer testers and assess the remaining feature list.

I'd also like to test whenever this is available :)
I'm excited to see if vscode can become more of a first-class editor to Godot with these debugger changes! I just recently rebased some changes that hopefully should make the language server more feature packed as well (godotengine/godot#80973)

Let me know when testing begins!

[Calinou]

If you want to test this PR's current state, install the VSIX contained in this ZIP archive in VS Code (see instructions):

• https://nightly.link/godotengine/godot-vscode-plugin/actions/artifacts/846981313.zip

[snkaupe]

I'm unable to launch a debugging session using the provided VSIX. Every attempt to do so results in the following error message:

Failed to launch Godot instance: Error: Command failed: "godot" --path "<project-path>" --remote-debug "tcp://tcp://127.0.0.1:6007"
Der Befehl ""godot"" ist entweder falsch geschrieben oder
konnte nicht gefunden werden.

The part in German says that the executable "godot" was not found. Which is not surprising, because it is not part of my PATH variable. I did set the value for the new Godot 4 Editor setting, however, in the GUI settings editor, it is always reverted to undefined. It is, however, found and set correctly in the settings.json:

"godotTools.lsp.serverPort": 6005,
"godotTools.editorPath.godot4": "E:\\Entwicklung\\Godot\\Godot_v4.1.1-stable_win64.exe"

Additionally, when I try to exit debugging mode by clicking the "Stop" button, the following error message appears and I have to click it a second time to make the debug bar disappear:

ReferenceError: TerminatedEvent is not defined
	at Oc.terminateRequest (c:\Users\Sebastian\.vscode\extensions\geequlim.godot-tools-1.3.1\out\extension.js:274:81967)
	at Oc.dispatchRequest (c:\Users\Sebastian\.vscode\extensions\geequlim.godot-tools-1.3.1\out\extension.js:51:5822)
	at Oc.dispatchRequest (c:\Users\Sebastian\.vscode\extensions\geequlim.godot-tools-1.3.1\out\extension.js:57:2045)
	at Oc.handleMessage (c:\Users\Sebastian\.vscode\extensions\geequlim.godot-tools-1.3.1\out\extension.js:45:7461)
	at C.sendMessage (e:\Entwicklung\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:119:705)
	at c.$sendDAMessage (e:\Entwicklung\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:118:9830)
	at t.N (e:\Entwicklung\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:114:7983)
	at t.M (e:\Entwicklung\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:114:7749)
	at t.H (e:\Entwicklung\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:114:6830)
	at t.G (e:\Entwicklung\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:114:5906)
	at r.value (e:\Entwicklung\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:114:4736)
	at f.w (e:\Entwicklung\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:63:1902)
	at f.fire (e:\Entwicklung\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:63:2119)
	at y.fire (e:\Entwicklung\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:79:13931)
	at r.value (e:\Entwicklung\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:130:30355)
	at f.w (e:\Entwicklung\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:63:1902)
	at f.fire (e:\Entwicklung\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:63:2119)
	at y.fire (e:\Entwicklung\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:79:13931)
	at MessagePortMain.<anonymous> (e:\Entwicklung\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:130:28635)
	at MessagePortMain.emit (node:events:513:28)
	at MessagePortMain._internalPort.emit (node:electron/js2c/utility_init:2:367)

[ryanabx]

I get this error when trying to debug and run from vscode:

and when trying to end the failed debug session:

[DaelonSuzuka]

New things that work:

• Godot4 launch mode
  • breakpoints + continuing
  • stack dump
  • variable inspection

New things that are broken:

• Godot 4 scene tree
• all editorPath settings
• maybe more idk

Video evidence:

[Haffi921]

Video evidence:
...

@DaelonSuzuka
I downloaded and packaged this PR but I still get the same non-functionality that I get with the normal extension. Could you share the launch.json configuration for godot4 you have in this video demo?

[DaelonSuzuka]

@Haffi921 this should be it:

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Launch",
            "type": "godot4",
            "request": "launch",
            "editor_path": "godot4",
            "project": "${workspaceFolder}"
        },
        {
            "name": "Attach",
            "type": "godot4",
            "request": "attach"
        }
    ]
}

[DaelonSuzuka]

@Geometror

Unfortunately I still can't walk through breakpoints/pause the project. As soon as I set any breakpoint the project freezes on the splash screen.

Apparently the debugger message format changed between 4.1 and 4.2. It was an easy fix now that I found it.

Breakpoints also weren't working for me in the benchmark repo because I added -- --run-benchmarks to additional_options in the launch configuration. The additional_options was being added BEFORE the breakpoint string, and the -- forces everything that comes after to be ignored by the engine and passed to the game. I fixed it so that additional_options is now always last, and breakpoints are working in both versions (for me).

[DaelonSuzuka]

Completed a bunch of user-facing polish. Error handling, prompts/dialogs for remediation, logging goes to an output panel (only for the debugger so far).

I've removed the commands Run workspace as Godot project and Run workspace as Godot project with visible collision shapes and navigation meshes because they're just a worse version of launching a debug session.

Open workspace with Godot editor is staying, and it's been rewritten to A) not be completely illegible, and B) validate the workspace version against the Godot version it's about to launch + handle other errors.

Log output screenshot:

[Geometror]

Works like a charm now, fantastic work!
(only tested with Godot 4.1/4.2 beta 5)

The only thing still missing are the icons in the active screen tree view, but that's not really a blocker:

[Geometror]

Btw is this PR's title still aptly? This looks more like a complete rewrite than just "debugger fixes" :)

[DaelonSuzuka]

Works like a charm now

Thanks for taking the time to test this branch, your feedback has been extremely valuable!

Btw is this PR's title still aptly? This looks more like a complete rewrite than just "debugger fixes" :)

We usually fix the PR titles right before merging.

[rsubtil]

The only thing still missing are the icons in the active screen tree view, but that's not really a blocker:

Regarding this, would it make sense to implement some functionality in Godot for debuggers to request editor icons, like a request using the existing debugging protocol to fetch a particular icon? I think it's a more reliable solution than to bundle the editor's icons like it is currently. If you think this is useful lemme know and I can start a proposal to see what the community's opinion is.

[DaelonSuzuka]

That's a good idea, but it makes me concerned about the volume of data we'd end up shuffling around. There's already an issue with slowdowns after a breakpoint hits and we have to recursively fetch inspections for all the variables.

Also thanks for reminding me that I need to update the icons!

[atirut-w]

Possibly offtopic, but this hyped me up so much. Can't wait for the next release!