Feature Request: Character Set translation

[doggy8088]

Description of the new feature/enhancement

In PuTTY, there is a convenient feature called Character set translation. See below:

Sometimes we have to deal with some files or programs that can only deal with non-unicode characters. In Windows Terminal, it can only deal with UTF-8 character set. This limitation will lead us unable to deal with that kind of situations.

Proposed technical implementation details (optional)

[DHowett]

I'm curious about your specific use case. Windows Terminal uses the Windows console host to handle all of its translation, and that supports all of the codepages and data encodings that the Windows console has always supported. For example, this does work:

#include <windows.h>
int main() {
	SetConsoleOutputCP(932);
	WriteFile(GetStdHandle(STD_OUTPUT_HANDLE), "\xA6", 1, nullptr, nullptr);
	return 0;
}

0xA6 is the Shift-JIS encoding (notably: not UTF-8!) for that glyph.

[DHowett]

(Which application are you using that is not functioning properly without the ability to switch the encoding?)

[doggy8088]

@DHowett I can reproduce the problem I met.

1. Login into any Linux

   $ lsb_release -a
   No LSB modules are available.
   Distributor ID: Ubuntu
   Description:    Ubuntu 20.04.2 LTS
   Release:        20.04
   Codename:       focal

2. Download a file encoded in Big5 charset.

   curl https://blog.miniasp.com/big5-example.txt -o big5-example.txt

3. Simply cat it out. It can't display correctly.

   

4. If I change it to UTF-8, then all okay!

   

5. I also tried to change LANG and LC_ALL, still not working.

   

[doggy8088]

I also tried luit to translate big5 to utf8, but it looks weird. For the same command, I can run correctly from x-terminal in elementary OS. It probably Windows Terminal's issue.

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.2 LTS
Release:        20.04
Codename:       focal
$ locale -a
C
C.UTF-8
POSIX
en_US.utf8
zh_HK.utf8
zh_TW
zh_TW.big5
zh_TW.utf8
$ cat big5-example.txt | iconv -f big5 -t utf8
測試
$ luit -encoding big5 cat big5-example.txt
4z8U
$ LC_ALL=zh_TW.big5 luit cat big5-example.txt
4z8U
$ LC_ALL=zh_TW.big5 luit -encoding big5 cat big5-example.txt
4z8U

[zadjii-msft]

Hey I'm trying to reconcile this with #1802, #15678, et. al. I feel like we may have missed an important note:

Login into any Linux

how are you logging in? WSL? ssh.exe? ssh in WSL? Sommething else?

[microsoft-github-policy-service]

This issue has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 3 days of this comment.