Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Terminal window "not responding" #16332

Closed
cool-RR opened this issue Nov 17, 2023 · 12 comments · Fixed by #16588
Closed

Terminal window "not responding" #16332

cool-RR opened this issue Nov 17, 2023 · 12 comments · Fixed by #16588
Labels
Area-Windowing Window frame, quake mode, tearout In-PR This issue has a related PR Issue-Bug It either shouldn't be doing this or needs an investigation. Needs-Repro We can't figure out how to make this happen. Please help find a simplified repro. Needs-Tag-Fix Doesn't match tag requirements Product-Terminal The new Windows Terminal.

Comments

@cool-RR
Copy link

cool-RR commented Nov 17, 2023

Windows Terminal version

1.18.2822.0

Windows build number

10.0.19045.0

Other Software

No response

Steps to reproduce

I'm using a terminal window, and then not using it for a while and doing other stuff.

Expected Behavior

No response

Actual Behavior

The terminal window becomes nonresponsive. The content is gone. The window title shows "not responding" and I can only kill the process at that point.

@cool-RR cool-RR added Issue-Bug It either shouldn't be doing this or needs an investigation. Needs-Triage It's a new issue that the core contributor team needs to triage at the next triage meeting labels Nov 17, 2023
@zadjii-msft
Copy link
Member

Does this happen consistently/? Or was this a on-off kinda thing? I'd guess that this is because we just pushed an update to the app on the Store, and the Store tried to kill the Terminal to install the update.

If that's the case, the Terminal version should probably now be https://github.com/microsoft/terminal/releases/tag/v1.18.3181.0

@microsoft-github-policy-service microsoft-github-policy-service bot added the Needs-Author-Feedback The original author of the issue/PR needs to come back and respond to something label Nov 17, 2023
@cool-RR
Copy link
Author

cool-RR commented Nov 17, 2023

This happened at least 10 times in the last few weeks, but I can't make it happen at will. My version right now is 1.18.2822.0.

Is there any way to find logs or something on my machine that would give us information about what's happening?

@microsoft-github-policy-service microsoft-github-policy-service bot added Needs-Attention The core contributors need to come back around and look at this ASAP. and removed Needs-Author-Feedback The original author of the issue/PR needs to come back and respond to something labels Nov 17, 2023
@lhecker
Copy link
Member

lhecker commented Nov 17, 2023

That sounds a bit like a deadlock, so log files probably won't be of help. You could try to capture a dump of a hung process before killing it: https://github.com/microsoft/terminal/wiki/Troubleshooting-Tips#capturing-and-sending-dumps

That way we can see what threads are currently stuck and what the UI thread is most likely waiting on.

@cool-RR
Copy link
Author

cool-RR commented Nov 17, 2023 via email

@cool-RR
Copy link
Author

cool-RR commented Nov 19, 2023

Today this happened again. I opened Process Explorer and saw these processes:

image

I wasn't sure which process you wanted a dump of, and whether it should be a mini-dump or a full dump, so I just made both mini- and full dumps of each of the 4 processes. They're available for download here: https://www.dropbox.com/scl/fo/ngzixgtvl9b4gms1ya0bl/h?rlkey=00vm814nojext99o8agdwwzu3&dl=0

Does that help you figure out why it's hanging?

@lhecker
Copy link
Member

lhecker commented Nov 21, 2023

The only notable thing I can see is that the active window thread is stuck in a call to DefWindowProc with a WM_INPUTLANGCHANGEREQUEST message. Unfortunately, the dump ends at NtUserMessageCall which is just a syscall stub. Basically, the kernel-side stack trace is missing in the dump and I don't know how to fix that.

But assuming that it calls into the venerable xxxRealDefWindowProc, then the default WM_INPUTLANGCHANGEREQUEST handler tries to pass the request to the currently focused window using SendMessage which is synchronous. In other words, if the currently focused window is unresponsive then the currently active window will also deadlock. I'm not familiar with the ntuser implementation so I can't really tell if "currently focused window" is only in the current process or system-wide.

In case of the former the issue might be caused by the 2nd window thread in your dump because that one is frozen (aka: A previously closed window. Does the issue occur if you never had more than 1 window of the Terminal open at a time?). In case of the latter it might be caused by another application you have installed that is unresponsive. Also in case of the latter, it might be caused by xxxActivateKeyboardLayout which is called if the currently active window is already focused. I suspect that this call interacts with the "Text Input Management Service" via COM/RPC and that might also hang if it has a bug.

@cool-RR
Copy link
Author

cool-RR commented Nov 21, 2023

i-know-some-of-these-words-mhmm

Thanks for the analysis. In that link were both mini-dumps and full dumps. Was the kernel-side stack trace missing also from the full dump?

@lhecker
Copy link
Member

lhecker commented Nov 21, 2023

Yep! I'm unfortunately not experienced enough with dumps to know why that is and a quick Google Bing didn't bring any answers to that either...

@zadjii-msft
Copy link
Member

I mean, maybe a dumb question is: are you changing the input language / method / something like that?

If you send /feedback via the Feedback Hub, we might be able to see if there's something else crashing on your machine that's correlated here. That's a real long shot though.

@microsoft-github-policy-service microsoft-github-policy-service bot added the Needs-Author-Feedback The original author of the issue/PR needs to come back and respond to something label Nov 29, 2023
@cool-RR
Copy link
Author

cool-RR commented Dec 1, 2023

@zadjii-msft

I mean, maybe a dumb question is: are you changing the input language / method / something like that?

I do change language often, between English and Hebrew. I usually don't do it in the shell because I don't use Hebrew in the shell. It's possible that this problem is caused by an input language change, but I'm not noticing that it's happening immediately after a language change. Right now I opened terminal and changed language a bunch of times, both in the terminal app and outside it. I also typed some Hebrew in the terminal for good measure. The terminal did not crash.

Therefore I can't really do the recording action suggested above. The problem is not reproducible, though it keeps happening again and again.

@microsoft-github-policy-service microsoft-github-policy-service bot removed the Needs-Author-Feedback The original author of the issue/PR needs to come back and respond to something label Dec 1, 2023
@carlos-zamora carlos-zamora added Needs-Repro We can't figure out how to make this happen. Please help find a simplified repro. Product-Terminal The new Windows Terminal. Area-Windowing Window frame, quake mode, tearout and removed Needs-Triage It's a new issue that the core contributor team needs to triage at the next triage meeting Needs-Attention The core contributors need to come back around and look at this ASAP. labels Dec 6, 2023
@carlos-zamora carlos-zamora added this to the Terminal v1.20 milestone Dec 6, 2023
@zadjii-msft
Copy link
Member

We had a bit of a brain blast: Does this only occur after you've opened two simultaneous terminal windows, and closed one of them (but not both)?

We suspect that this hang is due to the window refrigeration code in Terminal 1.18 that gets used on Windows 10. We're guessing the message gets sent to the closed, refrigerated window, which just doesn't pump it.

@cool-RR
Copy link
Author

cool-RR commented Dec 9, 2023

Does this only occur after you've opened two simultaneous terminal windows, and closed one of them (but not both)?

I tried that now, it didn't make the problem happen. But I often have multiple terminal windows open, and I routinely open new ones and close existing ones.

lhecker added a commit that referenced this issue Jan 23, 2024
At the time of writing, closing the last tab of a window inexplicably
doesn't lead to the destruction of the remaining TermControl instance.
On top of that, on Win10 we don't destroy window threads due to bugs in
DesktopWindowXamlSource. In other words, we leak TermControl instances.

Additionally, the XAML timer class is "self-referential".
Releasing all references to an instance will not stop the timer.
Only calling Stop() explicitly will achieve that.

The result is that the message loop of a frozen window thread has so
far received 1-2 messages per second due to the blink timer not being
stopped. This may have filled the message queue and lead to bugs as
described in #16332 where keyboard input stopped working.
@microsoft-github-policy-service microsoft-github-policy-service bot added the In-PR This issue has a related PR label Jan 23, 2024
DHowett pushed a commit that referenced this issue Jan 25, 2024
At the time of writing, closing the last tab of a window inexplicably
doesn't lead to the destruction of the remaining TermControl instance.
On top of that, on Win10 we don't destroy window threads due to bugs in
DesktopWindowXamlSource. In other words, we leak TermControl instances.

Additionally, the XAML timer class is "self-referential".
Releasing all references to an instance will not stop the timer.
Only calling Stop() explicitly will achieve that.

The result is that the message loop of a frozen window thread has so
far received 1-2 messages per second due to the blink timer not being
stopped. This may have filled the message queue and lead to bugs as
described in #16332 where keyboard input stopped working.

(cherry picked from commit 521a300)
Service-Card-Id: 91620224
Service-Version: 1.18
DHowett pushed a commit that referenced this issue Jan 25, 2024
At the time of writing, closing the last tab of a window inexplicably
doesn't lead to the destruction of the remaining TermControl instance.
On top of that, on Win10 we don't destroy window threads due to bugs in
DesktopWindowXamlSource. In other words, we leak TermControl instances.

Additionally, the XAML timer class is "self-referential".
Releasing all references to an instance will not stop the timer.
Only calling Stop() explicitly will achieve that.

The result is that the message loop of a frozen window thread has so
far received 1-2 messages per second due to the blink timer not being
stopped. This may have filled the message queue and lead to bugs as
described in #16332 where keyboard input stopped working.

(cherry picked from commit 521a300)
Service-Card-Id: 91642474
Service-Version: 1.19
DHowett pushed a commit that referenced this issue Jan 29, 2024
This changeset ensures that the message queue of frozen windows is
always being serviced. This should ensure that it won't fill up and
lead to deadlocks, freezes, or similar. I've tried _a lot_ of different
approaches before settling on this one. Introducing a custom `WM_APP`
message has the benefit of being the least intrusive to the existing
code base.

The approach that I would have favored the most would be to never
destroy the `AppHost` instance in the first place, as I imagined that
this would be more robust in general and resolve other (rare) bugs.
However, I found that this requires rewriting some substantial parts
of the code base around `AppHost` and it could be something that may
be of interest in the future.

Closes #16332
Depends on #16587 and #16575
@microsoft-github-policy-service microsoft-github-policy-service bot added the Needs-Tag-Fix Doesn't match tag requirements label Jan 29, 2024
DHowett pushed a commit that referenced this issue Jan 29, 2024
This changeset ensures that the message queue of frozen windows is
always being serviced. This should ensure that it won't fill up and
lead to deadlocks, freezes, or similar. I've tried _a lot_ of different
approaches before settling on this one. Introducing a custom `WM_APP`
message has the benefit of being the least intrusive to the existing
code base.

The approach that I would have favored the most would be to never
destroy the `AppHost` instance in the first place, as I imagined that
this would be more robust in general and resolve other (rare) bugs.
However, I found that this requires rewriting some substantial parts
of the code base around `AppHost` and it could be something that may
be of interest in the future.

Closes #16332
Depends on #16587 and #16575

(cherry picked from commit 5d2fa47)
Service-Card-Id: 91642478
Service-Version: 1.18
DHowett pushed a commit that referenced this issue Jan 29, 2024
This changeset ensures that the message queue of frozen windows is
always being serviced. This should ensure that it won't fill up and
lead to deadlocks, freezes, or similar. I've tried _a lot_ of different
approaches before settling on this one. Introducing a custom `WM_APP`
message has the benefit of being the least intrusive to the existing
code base.

The approach that I would have favored the most would be to never
destroy the `AppHost` instance in the first place, as I imagined that
this would be more robust in general and resolve other (rare) bugs.
However, I found that this requires rewriting some substantial parts
of the code base around `AppHost` and it could be something that may
be of interest in the future.

Closes #16332
Depends on #16587 and #16575

(cherry picked from commit 5d2fa47)
Service-Card-Id: 91642479
Service-Version: 1.19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area-Windowing Window frame, quake mode, tearout In-PR This issue has a related PR Issue-Bug It either shouldn't be doing this or needs an investigation. Needs-Repro We can't figure out how to make this happen. Please help find a simplified repro. Needs-Tag-Fix Doesn't match tag requirements Product-Terminal The new Windows Terminal.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants