Input lag when working with big scenes

[qarmin]

Godot version:
3.2 49b6423

OS/device including version:
Ubuntu 18.04 and also Windows 10

Issue description:
If I working with bigger scene and engine is under heavy load and then I click on the UP key for e.g. 10 seconds to change selection in scene tree, then engine stops operation a few seconds after releasing the key.

Steps to reproduce:
https://streamable.com/kmcuo
I stop pressing RIGHT button at 0:10 and Godot stops selecting nodes at 0:16

Minimal reproduction project:
TPS Demo

[qarmin]

Note that this happens mostly with keyboard.
Mouse input, even if visible, is small.

[kkga]

I think i'm experiencing the same issue here #31177

Have you found any workarounds?

[KoBeWi]

Can anyone still reproduce this bug in Godot 3.2.3 rc4 or any later release?

[Zireael07]

Related to #41495?

[akien-mga]

As per #41495 all reporters seem to be under Linux.