Skip to content

Commit

Permalink
Make conhost act in VtIo mode earlier in startup (#15298)
Browse files Browse the repository at this point in the history
We need to act like a ConPTY just a little earlier in startup. My relevant notes start here: #15245 (comment). 

Basically, we'd create the first screen buffer with 9001 rows, because it would be created _before_ VtIo would be in a state to say "yes, we're a conpty". Then, if a CLI app emits an entire screenful of text _before_ the terminal has a chance to resize the conpty, then the conpty will explode during `_DoLineFeed`. That method is absolutely not expecting the buffer to get resized (and the old text buffer deallocated). 

Instead, this will treat the console as in ConPty mode as soon as `VtIo::Initialize` is called (this is during `ConsoleCreateIoThread`, which is right at the end of `ConsoleEstablishHandoff`, which is before the API server starts to process the client connect message).  THEORETICALLY, `VtIo` could `Initialize` then fail to create objects in `CreateIoHandlers` (which is what we used to treat as the moment that we were in conpty mode). However, if we do fail out of `CreateIoHandlers`, then the console itself will fail to start up, and just die. So I don't think that's needed.

This fixes #15245. I think this is PROBABLY also the solution to #14512, but I'm not gonna explicitly mark closed. We'll loop back on it.
  • Loading branch information
zadjii-msft authored May 10, 2023
1 parent 4dd9493 commit 6ad8cd0
Show file tree
Hide file tree
Showing 2 changed files with 3 additions and 6 deletions.
8 changes: 3 additions & 5 deletions src/host/VtIo.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,6 @@ using namespace Microsoft::Console::Interactivity;

VtIo::VtIo() :
_initialized(false),
_objectsCreated(false),
_lookingForCursorPosition(false),
_IoMode(VtIoMode::INVALID)
{
Expand Down Expand Up @@ -224,13 +223,12 @@ VtIo::VtIo() :
}
CATCH_RETURN();

_objectsCreated = true;
return S_OK;
}

bool VtIo::IsUsingVt() const
{
return _objectsCreated;
return _initialized;
}

// Routine Description:
Expand All @@ -246,7 +244,7 @@ bool VtIo::IsUsingVt() const
[[nodiscard]] HRESULT VtIo::StartIfNeeded()
{
// If we haven't been set up, do nothing (because there's nothing to start)
if (!_objectsCreated)
if (!_initialized)
{
return S_FALSE;
}
Expand Down Expand Up @@ -514,7 +512,7 @@ void VtIo::EndResize()
// - <none>
void VtIo::EnableConptyModeForTests(std::unique_ptr<Microsoft::Console::Render::VtEngine> vtRenderEngine)
{
_objectsCreated = true;
_initialized = true;
_pVtRenderEngine = std::move(vtRenderEngine);
}
#endif
Expand Down
1 change: 0 additions & 1 deletion src/host/VtIo.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,6 @@ namespace Microsoft::Console::VirtualTerminal
VtIoMode _IoMode;

bool _initialized;
bool _objectsCreated;

bool _lookingForCursorPosition;

Expand Down

0 comments on commit 6ad8cd0

Please sign in to comment.