Terminal doesn't support RTL languages

[mostafa-drz]

• VSCode Version: Code 1.13.0 (376c52b, 2017-06-08T16:34:53.678Z)
• OS Version: Darwin x64 15.6.0
• Extensions:

Extension	Author	Version
vscode-html-css	ecmel	0.1.6

---

Steps to Reproduce:

1.In this new version there are problems regarding showing Unicode in terminal. (languages like Arabic, Persian ). It happened in this new recent update
2.

[ramya-rao-a]

@mostafa69d Can you share an example or screenshot?

[mostafa-drz]

@ramya-rao-a
As you see, all the fields containing a Unicode Characters are messed up. It was working fine before the update.
P.S: A word like "ایران" ---new version-----> "ن ا ر ی ا"

[Tyriar]

@mostafa69d do you have an example of another terminal that handles this well? This change was intentional to force the characters onto a grid. Sure it doesn't make some RTL languages look good but when the characters aren't aligned to the grid, bad stuff happens.

I checked Hyper (on hterm), terminator and gnome-terminal and all behave as in vscode 1.13.

[mostafa-drz]

@Tyriar Here you go.
This screenshot is from my Mac OS terminal:

And this one is from the VS Code terminal ( the same query with the same result)

I'm pretty sure something happened to the VS Code terminal, because I'm working on the same project for at least one month with VS code and it was working fine. After I updated the VS Code this morning it messed up.

[CherryDT]

@Tyriar I believe "not looking good" would be an understatement - it is pretty problematic that the text is reversed, probably the spacing is less of an issue, but I guess even that can still make it hard to read for languages where letters are supposed to be connected and appear differently in that case (Arabic). I can't judge it though because the only RTL language I can read is Hebrew.

As for examples: ConEmu seems to handle it in a better way in that the characters are still aligned in a grid, but RTL rules are adhered so that the text is shown the right way round. This may of course be confusing or cause issues combined with mouse selection or other things where characters are assumed to exist in the buffer at the position they are shown at, but working with LTR+RTL is weird in any environment I think.

Bottom line, I think destroying the readability by reversing the text makes the terminal unusable for those languages. It's true that many other terminals (e.g. xterm) do the same, but here the "better" solution already existed and people were probably happy about it... so this feels like a step back.

[mostafa-drz]

@CherryDT
Exactly as you mentioned all the IDEs have some problem with RTL languages especially when you hard code some characters in your code(I hope VS Code find a solution for this problem too). Anyway at least VS Code was working perfectly in terminal in the previous version( In my case Persian which its rules for RTL are same as Arabic)

[Tyriar]

@CherryDT the previous solution (just dumping the unstyled text) is no longer feasible due to optimizations and features coming into the terminal. The new selection model I've been working on for example assumes everything is on a grid containing only half-width and full width chars.

Reversing the text but still aligning it to the grid as in Terminal.app and ConEmu definitely seems doable though and seems to be the right way to do it. I created xtermjs/xterm.js#701 to track the upstream issue.

[avigdorfin]

ON the question "do you have an example of another terminal that handles this well?"
I'd like to suggest the Windows Application PuTTY by Simon Tatham from Release 0.66 up (maybe earlier also) that has a flag in the option to make the terminal behave exactly as requested.
It has a version for Linux that run correctly as well.
I can send an example to demonstrate, if requested.

[avigdorfin]

Well, the best example is the way this forum editor is working. Any way I copied the text from the terminal, it got corrected by the forum editor, used to send this message.
$ cat Downloads/example.txt
Jun 21 21:49 Downloads/לוח סילוקין.docx
On gnome-terminal the Hebrew letters appear " ןיקוליס חול"
For the demonstration I typed the letters in reverse.
The is a "bidi" library C code somewhere that handle the complex mixed Latin/RTL quiet good.
Best regards.

[saeedhei]

Why Persian Font is Okey in const, But is not okey in Functions & Output?

const StartKeyboard = [
    ['وضعیت من 🌎']
]
bot.onText(/\وضعیت من 🌎/, async  msg => {

});

VS Code: 1.17.2

[boazdori]

the problem for me occures not in the editor but when running the code if i run it to the jypter notebook it works fine. when i run it to the repel or use the run comand i get jiberish

[Tyriar]

Let's track this upstream xtermjs/xterm.js#701