Extra newline when copying line with wide glyph on the edge

[Kagami]

Windows Terminal version (or Windows build number)

10.0.19043.1348, 1.12.2931.0

Other Software

Ubuntu 21.04 in WSL

Steps to reproduce

1. Open tab with some WSL distro, shouldn't matter which.

2. Create python3 file with the following content:

import os
import sys
cols = os.get_terminal_size().columns
sys.stdout.buffer.write(b'a'*(cols-1))
sys.stdout.buffer.write(b'\xea\xb0\x9c')

3. Run it:

python3 1.py

4. Copy output with the mouse and paste to some editor.

Expected Behavior

There should be only single line of text.

Actual Behavior

There're two lines of text, see screenshots.

The problem doesn't appear if that 3-byte symbol is not on the edge of the line. Also 2-byte symbols work fine too.

[zadjii-msft]

Hmm. Doesn't repro in conhost. So this is probably a conpty wrapping bug. Probably related to #5800 in some way I don't fully know yet. Thanks for the minimal repro!

[Kagami]

GNOME Terminal in native Linux also wraps the line -- as it should, I think.

The problem is not the line wrapping, it's the extra newline when I copy the output to the clipboard.

This problem doesn't happen with two-byte symbols (single column), e.g.:

sys.stdout.buffer.write(b'a'*(cols-1))
sys.stdout.buffer.write('тест'.encode('utf-8'))

[Kagami]

Works fine in this case too:

sys.stdout.buffer.write(b'a'*(cols-2))
sys.stdout.buffer.write(b'\xea\xb0\x9c')
sys.stdout.buffer.write(b'\xea\xb0\x9c')

It seems be related to the fact that there's only 1 column left on the current line but the character takes 2 columns.

[eryksun]

The problem is not the line wrapping, it's the extra newline when I copy the output to the clipboard.

Sorry, I misunderstood that the problem was the newline itself in the output. I see now that it shouldn't be there and normally isn't, except when a wide glyph is wrapped at the edge.

To me, the following example more clearly emphasizes that the character is known to be a wide glyph that prints in two columns (hence dividing the column count by 2), and that it's expected to wrap in the terminal by one character. The bug is just that the terminal adds a line-feed (LF) character where it wraps the line for display.

>>> os.get_terminal_size().columns
133
>>> print('\uac1c' * (os.get_terminal_size().columns // 2 + 1))
개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개
개

>>> os.get_terminal_size().columns
132
>>> print('\uac1c' * (os.get_terminal_size().columns // 2 + 1))
개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개

The output copied from the terminal with an odd number of columns (133) should not have the LF before the last "개" character. This example works fine with an even number of columns, for which a wide glyph doesn't straddle the edge.

Now that I know what I'm looking for, I see that conhost in Windows 10 and 11 is also wrong, but in a different way. For example:

>>> os.get_terminal_size().columns
133
>>> print('\uac1c' * (os.get_terminal_size().columns // 2 + 1))
개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개 개

>>> os.get_terminal_size().columns
132
>>> print('\uac1c' * (os.get_terminal_size().columns // 2 + 1))
개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개개

That's an ASCII space (0x20) inserted where the line was wrapped for display. It's not present when the line wrap occurs on a single-column character, or for this example when there is an even number of columns.

[Kagami]

Thanks for clarification, I modified issue's title accordingly.

[eryksun]

I updated my last comment to use the same screen sizes in Windows Terminal and conhost, and also to show that the bug only occurs for me when the number of columns in the terminal is odd, which causes a wide glyph to straddle the edge.