Mac native window disconnected from titlebar

[bandaloo]

When going into fullscreen (with the green button) and then exiting fullscreen, the window gets strangely disconnected from the titlebar as seen in this screenshot:

Furthermore, the window is stretched and there are black bars above and below. I'm able to reproduce it simply by doing the steps above. I'm running macOS Montery 12.0.1. It happens on my 1440p external display and the builtin display.