Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix crash in terminal when tab closed while data is being output #8982

Merged
2 commits merged into from
Feb 1, 2021

Conversation

j4james
Copy link
Collaborator

@j4james j4james commented Feb 1, 2021

If there is data being output when a tab is closed, that can sometimes
result in the application crashing, because the renderer may still be in
use at the time is it destroyed. This PR attempts to prevent that from
happening by adding a lock in the TermControl::Close method.

What we're trying to prevent is the connection thread still being
active, and potentially accessing the renderer, after it has been
destroyed. So by acquiring the terminal lock in TermControl::Close,
after we've stopped accepting new output, we can be sure that the
connection thread is no longer active (it holds the lock while it is
processing output). So once we've acquired and released the lock, it
should be safe to tear down and destroy the renderer.

Validation Steps Performed

While this crash is difficult to reproduce in general usage, it occurred
quite frequently when testing my DECPS implementation (there is some
tricky thread synchronisation, which seems more likely to trigger the
issue). With this patch applied, though, those crashes have stopped
occurring.

I've also stepped through the shutdown code in the debugger, manually
freezing threads to get them aligned in the right way to trigger the
crash (as explained in issue #8734). Again with the patch applied, I can
no longer get the crash to occur.

Closes #8734

@ghost ghost added Issue-Bug It either shouldn't be doing this or needs an investigation. Priority-1 A description (P1) Product-Terminal The new Windows Terminal. Severity-Crash Crashes are real bad news. labels Feb 1, 2021
@carlos-zamora carlos-zamora added zPreview-Service-Queued-1.13 A floating label that tracks the current Preview version for servicing purposes. zStable-Service-Queued-1.12 A floating label that tracks the current Stable version for servicing purposes. labels Feb 1, 2021
Co-authored-by: Mike Griese <migrie@microsoft.com>
Copy link
Member

@DHowett DHowett left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clever fix. I hate it. I mean, I very much appreciate you fixing it, but I hate that our locking story is so poorly-told that you need to take and then surrender a lock as a sort of acq-rel barrier. 😄

@DHowett DHowett added the AutoMerge Marked for automatic merge by the bot when requirements are met label Feb 1, 2021
@ghost
Copy link

ghost commented Feb 1, 2021

Hello @DHowett!

Because this pull request has the AutoMerge label, I will be glad to assist with helping to merge this pull request once all check-in policies pass.

p.s. you can customize the way I help with merging this pull request, such as holding this pull request until a specific person approves. Simply @mention me (@msftbot) and give me an instruction to get started! Learn more here.

@ghost ghost merged commit 40ebe5a into microsoft:main Feb 1, 2021
@DHowett
Copy link
Member

DHowett commented Feb 1, 2021

(Thank you, as always!)

@j4james j4james deleted the fix-close-crash branch February 5, 2021 00:21
@DHowett DHowett removed zStable-Service-Queued-1.12 A floating label that tracks the current Stable version for servicing purposes. zPreview-Service-Queued-1.13 A floating label that tracks the current Preview version for servicing purposes. labels Feb 5, 2021
DHowett pushed a commit that referenced this pull request Feb 5, 2021
If there is data being output when a tab is closed, that can sometimes
result in the application crashing, because the renderer may still be in
use at the time is it destroyed. This PR attempts to prevent that from
happening by adding a lock in the `TermControl::Close` method.

What we're trying to prevent is the connection thread still being
active, and potentially accessing the renderer, after it has been
destroyed. So by acquiring the terminal lock in `TermControl::Close`,
after we've stopped accepting new output, we can be sure that the
connection thread is no longer active (it holds the lock while it is
processing output). So once we've acquired and released the lock, it
should be safe to tear down and destroy the renderer.

## Validation Steps Performed

While this crash is difficult to reproduce in general usage, it occurred
quite frequently when testing my `DECPS` implementation (there is some
tricky thread synchronisation, which seems more likely to trigger the
issue). With this patch applied, though, those crashes have stopped
occurring.

I've also stepped through the shutdown code in the debugger, manually
freezing threads to get them aligned in the right way to trigger the
crash (as explained in issue #8734). Again with the patch applied, I can
no longer get the crash to occur.

Closes #8734

(cherry picked from commit 40ebe5a)
DHowett pushed a commit that referenced this pull request Feb 5, 2021
If there is data being output when a tab is closed, that can sometimes
result in the application crashing, because the renderer may still be in
use at the time is it destroyed. This PR attempts to prevent that from
happening by adding a lock in the `TermControl::Close` method.

What we're trying to prevent is the connection thread still being
active, and potentially accessing the renderer, after it has been
destroyed. So by acquiring the terminal lock in `TermControl::Close`,
after we've stopped accepting new output, we can be sure that the
connection thread is no longer active (it holds the lock while it is
processing output). So once we've acquired and released the lock, it
should be safe to tear down and destroy the renderer.

## Validation Steps Performed

While this crash is difficult to reproduce in general usage, it occurred
quite frequently when testing my `DECPS` implementation (there is some
tricky thread synchronisation, which seems more likely to trigger the
issue). With this patch applied, though, those crashes have stopped
occurring.

I've also stepped through the shutdown code in the debugger, manually
freezing threads to get them aligned in the right way to trigger the
crash (as explained in issue #8734). Again with the patch applied, I can
no longer get the crash to occur.

Closes #8734

(cherry picked from commit 40ebe5a)
@ghost
Copy link

ghost commented Feb 11, 2021

🎉Windows Terminal v1.5.10411.0 has been released which incorporates this pull request.:tada:

Handy links:

@ghost
Copy link

ghost commented Feb 11, 2021

🎉Windows Terminal Preview v1.6.10412.0 has been released which incorporates this pull request.:tada:

Handy links:

This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
AutoMerge Marked for automatic merge by the bot when requirements are met Issue-Bug It either shouldn't be doing this or needs an investigation. Priority-1 A description (P1) Product-Terminal The new Windows Terminal. Severity-Crash Crashes are real bad news.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Windows Terminal can crash if a tab is closed while data is being output
4 participants