Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OSC52 does not handle UTF-8 correctly ("copying content using tmux copy mode doesn't paste properly") #7819

Closed
Axedyson opened this issue Oct 3, 2020 · 4 comments · Fixed by #7870
Closed

OSC52 does not handle UTF-8 correctly ("copying content using tmux copy mode doesn't paste properly") #7819

Axedyson opened this issue Oct 3, 2020 · 4 comments · Fixed by #7870
Labels
Area-VT Virtual Terminal sequence support Impact-Correctness It be wrong. Issue-Bug It either shouldn't be doing this or needs an investigation. Priority-2 A description (P2) Product-Terminal The new Windows Terminal. Resolution-Fix-Committed Fix is checked in, but it might be 3-4 weeks until a release.
Milestone
Terminal v1.5

Comments

@Axedyson
Copy link

Axedyson commented Oct 3, 2020 •
edited
Loading

Environment

Windows build number: Microsoft Windows [Version 10.0.19041.508]
Windows Terminal version: 1.3.2651.0

Other software:

Tmux version: tmux 3.0a
Running Ubuntu 20.04.1 LTS inside of WSL2

Steps to reproduce

All the steps to reproduce this issue are stated in this issue that I created.
Please read that issue as it contains some more information about my environment and some relevant comments.

It seems like this issue doesn't relate to a problem with tmux but rather a problem with the Windows Terminal or the windows clipboard.

Expected behavior

When I copy content using tmux copy mode I would like to have the ability to paste it outside of tmux containing the exact same characters which can be seen here in a browser search bar:
image

Actual behavior

The content is not pasted properly as this character shows up as cryptic characters:
image
@ghost ghost added Needs-Triage It's a new issue that the core contributor team needs to triage at the next triage meeting Needs-Tag-Fix Doesn't match tag requirements labels Oct 3, 2020
@DHowett
Copy link
Member

DHowett commented Oct 5, 2020

/cc @uzxmx
It looks like s_Decode in base64 is transforming the raw bytes that come out of the base64 encoded clipboard data directly into UCS-2 codepoints instead of running them through UTF-8->UTF-16 conversion.

@DHowett DHowett added Area-VT Virtual Terminal sequence support Impact-Correctness It be wrong. Issue-Bug It either shouldn't be doing this or needs an investigation. Priority-2 A description (P2) Product-Terminal The new Windows Terminal. and removed Needs-Triage It's a new issue that the core contributor team needs to triage at the next triage meeting labels Oct 5, 2020
@ghost ghost removed the Needs-Tag-Fix Doesn't match tag requirements label Oct 5, 2020
@DHowett DHowett added this to the Terminal v1.5 milestone Oct 5, 2020
@DHowett DHowett mentioned this issue Oct 9, 2020
Closed
@DHowett DHowett changed the title Copying content using tmux copy mode doesn't paste properly outside of tmux using ctrl+v OSC52 does not handle UTF-8 correctly ("copying content using tmux copy mode doesn't paste properly") Oct 9, 2020
@amaya382 amaya382 mentioned this issue Oct 9, 2020
Merged
@ghost ghost added the In-PR This issue has a related PR label Oct 9, 2020
@PetterS
Copy link

PetterS commented Oct 13, 2020

In the meantime, you can execute tmux show-buffer | clip.exe which will put the correct string on the Windows clipboard.

@ghost ghost closed this as completed in #7870 Oct 16, 2020
@ghost ghost removed the In-PR This issue has a related PR label Oct 16, 2020
ghost pushed a commit that referenced this issue Oct 16, 2020
@amaya382
Fix garbling when copying multibyte text via OSC 52 (#7870)
743283e 
This commit adds a missing conversion utf8 to utf16 in decoding base64
for handling multibyte text in copying via OSC 52.

## Validation Steps Performed
* automatically
    * Tests w/ multibyte characters
* manually
    * case1
        * Executed `printf "\x1b]52;;%s\x1b\\" "$(printf '👍👍🏻👍🏼👍🏽👍🏾👍🏿' | base64)"`
        * Verified `👍👍🏻👍🏼👍🏽👍🏾👍🏿` in my clipboard
    * case2
        * Copied `👍👍🏻👍🏼👍🏽👍🏾👍🏿` by tmux 2.6 default copy function (OSC 52)
        * Verified `👍👍🏻👍🏼👍🏽👍🏾👍🏿` in my clipboard

Closes #7819
@ghost ghost added the Resolution-Fix-Committed Fix is checked in, but it might be 3-4 weeks until a release. label Oct 16, 2020
DHowett pushed a commit that referenced this issue Oct 19, 2020
@amaya382 @DHowett
Fix garbling when copying multibyte text via OSC 52 (#7870)
72bedcc 
This commit adds a missing conversion utf8 to utf16 in decoding base64
for handling multibyte text in copying via OSC 52.

## Validation Steps Performed
* automatically
    * Tests w/ multibyte characters
* manually
    * case1
        * Executed `printf "\x1b]52;;%s\x1b\\" "$(printf '👍👍🏻👍🏼👍🏽👍🏾👍🏿' | base64)"`
        * Verified `👍👍🏻👍🏼👍🏽👍🏾👍🏿` in my clipboard
    * case2
        * Copied `👍👍🏻👍🏼👍🏽👍🏾👍🏿` by tmux 2.6 default copy function (OSC 52)
        * Verified `👍👍🏻👍🏼👍🏽👍🏾👍🏿` in my clipboard

Closes #7819

(cherry picked from commit 743283e)
@ghost
Copy link

ghost commented Nov 11, 2020

🎉This issue was addressed in #7870, which has now been successfully released as Windows Terminal v1.4.3141.0.:tada:

Handy links:

@ghost
Copy link

ghost commented Nov 11, 2020

🎉This issue was addressed in #7870, which has now been successfully released as Windows Terminal Preview v1.5.3142.0.:tada:

Handy links:

This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Area-VT Virtual Terminal sequence support Impact-Correctness It be wrong. Issue-Bug It either shouldn't be doing this or needs an investigation. Priority-2 A description (P2) Product-Terminal The new Windows Terminal. Resolution-Fix-Committed Fix is checked in, but it might be 3-4 weeks until a release.
Projects
None yet
Milestone
Terminal v1.5
Development

Successfully merging a pull request may close this issue.

Fix garbling when copying multibyte text via OSC 52 amaya382/terminal
3 participants
@DHowett @PetterS @Axedyson