fix(font-encoding): fix invalid characters on windows, fix #639

[CharlesChiuGit]

1. set default shell from cmd to powershell.¹
2. set fileendoing to use utf-8 only

All screenshots are taken in windows terminal

TODO:

1. icon overlapping (if anyone have this issue, just try another nerdfont)
2. icon won't show in toggleterm.nvim, it only works in the neovim built-in terminal.
   

Results:

Footnotes

1. pwsh would be better, but it's not installed on windows by default. Only cmd and powershell are. ↩

[CharlesChiuGit]

for the icon overlapping issue could be fixed if use wezterm as a workaround. it seems like a long term issue in windows terminal.

[CharlesChiuGit]

Wezterm results on windows:
Built-in terminal:

toggleterm:

So it seems the remain issues are all about windows terminal...

[CharlesChiuGit]

Actually, It's a font issue. After trying other fonts from nerdfonts on windows terminal. There's no more icon overlapping!

But the icon still doesn't show in toggleterm in windows terminal...

I think it's better to use wezterm for the sake of our sanity.

[Jint-lzxy]

@CharlesChiuGit Sorry for the late reply. I just did some testings on my Windows 10 VM. The known issues are:

• cmd.exe requires some tweaks to fully support UTF-16LE (by setting codepage to a more appropriate value [CCSID 850 by default], which might need further discussion).
• The flags for launching pwsh should be further adjusted to support command mode IO. Not tested yet.
• Looks like those incorrect behaviors are all related to the terminal invoking nvim. This is not related to the configuration of nvim itself (well, except for the options: 'shell*')
• The "icon overlapping issue" should be related to using incorrect font variants. AFAIK some terminal emulators will automatically adjust this behavior, but most will not.

For the first three items, I'll propose a draft solution ASAP.

For the last item, manually installing all fonts ending in Mono Windows Compatible can address this problem on my VM (and on any terminal), probably b/c the mono variant of Nerd Fonts should be more suitable for supporting terminal icons (as its name "mono" suggests). e.g.:

Then use JetBrainsMono Nerd Font Mono as the font name. Not sure if this method can solve the problem on your end.

[CharlesChiuGit]

• cmd.exe requires some tweaks to fully support UTF-16LE (by setting codepage to a more appropriate value [CCSID 850 by default], which might need further discussion).

I think supporting cmd is unnecessary, since every Windows ships with windows powershell by default. We can just set it as default shell in the nvim config.

• The flags for launching pwsh should be further adjusted to support command mode IO. Not tested yet.

Can u explain more on "command mode IO"? I not sure what do u mean. I can run most Windows command with :!.

• Looks like those incorrect behaviors are all related to the terminal invoking nvim. This is not related to the configuration of nvim itself (well, except for the options: 'shell*')

Ys, I follow the step in :h shell-powershell to set those things when the the detected OS is Windows.

• The "icon overlapping issue" should be related to using incorrect font variants. AFAIK some terminal emulators will automatically adjust this behavior, but most will not.

Ys, I also discovered that installing nerd-font via third-party package manager like scoop might cause issue as I mentioned in #639 (comment). (at least it looks different then downloading the ttf files directly from https://www.nerdfonts.com/)

For the last item, manually installing all fonts ending in Mono Windows Compatible can address this problem on my VM (and on any terminal), probably b/c the mono variant of Nerd Fonts should be more suitable for supporting terminal icons (as its name "mono" suggests). e.g.:

Ys, use the mono or the NFM variants did the same work on solving the icon overlapping issues. The result of mono variant is exactly the results in my above comment.

[Jint-lzxy]

I think supporting cmd is unnecessary, since every Windows ships with windows powershell by default. We can just set it as default shell in the nvim config.

IMO we could vim.notify those users who haven't installed PowerShell on their machine - to avoid some extremely rare situations (such as running nvim on Windows Vista or accidentally uninstalled windows powershell 🤣)

Can u explain more on "command mode IO"? I not sure what do u mean. I can run most Windows command with :!.

I mean we should further specify how PowerShell works - e.g., whether to start in non interactive-mode/read user-configuration/set useful environment variables, with the goal of simulating bash's -c flag.

Ys, I also discovered that installing nerd-font via third-party package manager like scoop might cause issue as I mentioned in #639 (comment). (at least it looks different then downloading the ttf files directly from https://www.nerdfonts.com/)

I just tested this and it seems feasible to install the mono variant directly:

scoop install JetBrainsMono-NF-Mono

btw, are the icons displaying in Windows Terminal (toggleterm) now? Looks like the byte string wasn't properly pasted to toggleterm. Try using pure ASCII:

Write-Host "`u{ebb1}"

[CharlesChiuGit]

actually "pwsh" is not the same as "windows powershell" lol

pwsh is the cross-platform version.

[CharlesChiuGit]

scoop install JetBrainsMono-NF-Mono

interesting! didn't knw that lol

[CharlesChiuGit]

btw, are the icons displaying in Windows Terminal (toggleterm) now? Looks like the byte string wasn't properly pasted to toggleterm. Try using pure ASCII:

Write-Host "`u{ebb1}"

will try once i got home.

[CharlesChiuGit]

I just tested this and it seems feasible to install the mono variant directly:

scoop install JetBrainsMono-NF-Mono

This works! Cool! It will install these two font family.

btw, are the icons displaying in Windows Terminal (toggleterm) now? Looks like the byte string wasn't properly pasted to toggleterm. Try using pure ASCII:

Write-Host "`u{ebb1}"

• In windows terminal(pwsh):
  

• In wezterm(pwsh):
  

• In windows terminal(nvim built-in terminal, pwsh):
  

• In windows terminal(nvim toggleterm, pwsh):
  

• In wezterm(nvim built-in terminal, pwsh):
  

• In wezterm(nvim toggleterm, pwsh):
  

IMO we could vim.notify those users who haven't installed PowerShell on their machine - to avoid some extremely rare situations (such as running nvim on Windows Vista or accidentally uninstalled windows powershell 🤣)

Agreed.

I mean we should further specify how PowerShell works - e.g., whether to start in non interactive-mode/read user-configuration/set useful environment variables, with the goal of simulating bash's -c flag.

Got it, but I'm not sure how to test it lol. We can discuses more later.

[Jint-lzxy]

This works! Cool! It will install these two font family.

Great! NL here probably stands for no ligature, AFAIU. IMO we can add this to the wiki.

Got it, but I'm not sure how to test it lol. We can discuses more later.

We're gonna need massive reference to about_PowerShell_exe and about_Pwsh 😄

[CharlesChiuGit]

• In windows terminal(nvim built-in terminal, pwsh):

Updated. It seems fine using pure ASCII, no matter where u use it. But
Write-Host ("E492：不是编辑器的命令", "", "", "", "XXXXXXXXX") -Separator " || " is still not valid in windows terminal(nvim toggleterm, pwsh).

[Jint-lzxy]

But <...> is still not valid in windows terminal(nvim toggleterm, pwsh).

Hmm maybe this is because those multi-byte icons were not properly pasted to toggleterm. Is pasting  (fa-lock) directly to the terminal supported? (i.e., is the icon displayed before execution?)

[Jint-lzxy]

@CharlesChiuGit The following are all possible flags that we could further discuss:

• -NoLogo - Hides the copyright banner at startup.
• -ExecutionPolicy - Sets the default execution policy for the current session.
  • I prefer RemoteSigned -Scope Process -Force for maximum security & portability.
• -Command - Executes the specified commands (and any parameters) as though they were typed at the PowerShell command prompt, and then exits, unless the NoExit parameter is specified.
  • I prefer New-Object and full module name for the script block:

- -Command [Console]::InputEncoding=[Console]::OutputEncoding=[System.Text.UTF8Encoding]::new();$PSDefaultParameterValues['Out-File:Encoding']='utf-8';
+ -Command $OutputEncoding = [console]::InputEncoding = [console]::OutputEncoding = New-Object System.Text.UTF8Encoding

• -NoProfile - Does not load the PowerShell profile.
• -NonInteractive - This switch is used to create sessions that shouldn't require user input. This is useful for scripts that run in scheduled tasks or CI/CD pipelines. Any attempts to use interactive features, like Read-Host or confirmation prompts, result in statement terminating errors rather than hanging.
  • I'm prone to add this flag since :! (bang) and system() are not interactive in neovim on *nix like systems.

---

LEGEND:

• - should be added, but details need further discussion;
• - whether to add this option requires further discussion.

[Jint-lzxy]

8282c80 cc @CharlesChiuGit If you have time to review & discuss this!

[CharlesChiuGit]

Damn! I totally forgot to test this. Will test soon!

[CharlesChiuGit]

But <...> is still not valid in windows terminal(nvim toggleterm, pwsh).

Hmm maybe this is because those multi-byte icons were not properly pasted to toggleterm. Is pasting  (fa-lock) directly to the terminal supported? (i.e., is the icon displayed before execution?)

This works!

[CharlesChiuGit]

• -NoLogo somehow doesn't seem to work, the copyright banner still appears.
• -ExecutionPolicy RemoteSigned -Scope Process -Force seems problematic, nvim process got shut down immediately after I open toggleterm, -ExecutionPolicy RemoteSigned -Scope Process cause the toggleterm to shut down immediately, but user will still remain in nvim.
• -Command LGTM, but I also doesn't really know the syntax. The original one is from the nvim help :h shell_powershell.
• NoProfile: I don't feel like adding it, b/c ppl might define aliases in their profile. Loading profiles is more like using shell in *nix-like OS. Also this doesn't seem effective, nothing changed after I add this flag.
• -NonInteractive LGTM

[CharlesChiuGit]

I also notice sth. I'm not sure why, but current settings seem to increase noticeable startup time comparing to my original draft.
Maybe some api is a bit slower, will check some doc to analyze.

[Jint-lzxy]

• -NoLogo somehow doesn't seem to work, the copyright banner still appears.

It's actually working, notice that:

Copyright (c) Microsoft Corporation. All rights reserved.

https://aka.ms/pscore6-docs
Type 'help' to get help.

have disappeared.

• -ExecutionPolicy RemoteSigned -Scope Process -Force seems problematic, nvim process got shut down immediately after I open toggleterm, -ExecutionPolicy RemoteSigned -Scope Process cause the toggleterm to shut down immediately, but user will still remain in nvim.

Oh yes, I forgot toggleterm will also use these options. "nvim process got shut down immediately" is b/c, from the documentation:

Executes the specified commands (and any parameters) as though they were typed at the PowerShell command prompt, and then exits, unless the -NoExit parameter is specified.

We probably need to add -NoExit, but I'm not sure if the emulator will work properly in :! (shell bang).

• NoProfile: I don't feel like adding it, b/c ppl might define aliases in their profile. Loading profiles is more like using shell in *nix-like OS.

Sounds very reasonable 😄 Will remove this flag.

Also this doesn't seem effective, nothing changed after I add this flag.

Maybe u are missing something from about_Profiles?

but current settings seem to increase noticeable startup time comparing to my original draft.

Hmm that's strange... The current settings work fine with PowerShell core on macOS, will test this issue on Windows later ;)

[Jint-lzxy]

e5485ee

[CharlesChiuGit]

From pwsh -h, and about Scopes, I think we can't set -Scope as a flag to pwsh. -Scope seems to only used for limiting the scopes when using Get-Variable, not for limiting current pwsh process.

---

-NoExit will somehow output an extra line of weird raw utf8 code straing in :!.

w/ -NoExit:

w/o -NoExit:

---

I think I found why toggleterm will still output profile loading time and copyright, that's b/c we need to append the flags after vim.o.shell like so:

		shell = string.format(
			"%s %s;",
			vim.o.shell,
			"-NoLogo -NoLogo -NoProfileLoadTime -NonInteractive -ExecutionPolicy RemoteSigned"
		), -- change the default shell

However, flags after -Command seems not been taken and it caused errors. Still need to research on that more.

---

The extra delay startup time seems to related to waiting for lsp to start, not directly related to the PR.

[CharlesChiuGit]

After hours of debugging and reading the docs, I just don't think manual-inputed icons can be display correctly on toggleterm on windows terminal.

But everything just works fine in wezterm. So I think it's a native flaw on windows terminal.

I think we need to cut the losses, we had already invested too much time in this trial font issue.

[Jint-lzxy]

-NoExit will somehow output an extra line of weird raw utf8 code straing in :!.

This is exactly what I was concerned about before:

We probably need to add -NoExit, but I'm not sure if the emulator will work properly in :! (shell bang).

That "extra line" containing ANSI escape sequences is actually the user's PowerShell custom prompt. If -NoExit is not supplied PowerShell would exit right after the execution of :! with no trailing user prompt. Not sure if toggleterm will work properly if we ignore -NoExit.

Also, we should consider users who use powershell.exe, AFAIK powershell.exe provides fewer options than pwsh.exe. A proposed workaround:

local basecmd = "-NoLogo -MTA -ExecutionPolicy RemoteSigned"
local ctrlcmd = "-Command [console]::InputEncoding = [console]::OutputEncoding = [System.Text.Encoding]::UTF8"

The following configuration will handle the issue of toggleterm shutting down immediately right after the execution of -Command (tested on macOS, not sure if we also need to make this change on Windows):

vim.api.nvim_set_option_value("shellredir", '-RedirectStandardOutput %s -NoNewWindow -Wait', {})
vim.api.nvim_set_option_value("shellpipe", '2>&1 | Out-File -Encoding UTF8 %s; exit $LastExitCode', {})

I think we need to cut the lost, we had already invested too much time in this trial font issue.

💯💯 agreed. If the above proposal works properly, we should just merge this PR. As for the actual behavior of toggleterm (such as whether to ignore copyright information or not), users can customize this by setting shell = <...> in toggleterm.setup(). And since the :term feature of nvim was originally proposed to simulate system terminals, IMO displaying all these information happens to be the way native terminals work, out-of-the-box, so there is no need to make any changes to the appearance of the terminal buffer(s) 😃 The same applies to localized strings.

[CharlesChiuGit]

Not sure if toggleterm will work properly if we ignore -NoExit.

Ys, toggleterm would work without -NoExit.

Also, we should consider users who use powershell.exe,

That's reasonable. Agreed.

As for the actual behavior of toggleterm (such as whether to ignore copyright information or not), users can customize this by setting shell = <...> in toggleterm.setup().

Agreed.

---

local basecmd = "-NoLogo -MTA -ExecutionPolicy RemoteSigned"
local ctrlcmd = "-Command [console]::InputEncoding = [console]::OutputEncoding = [System.Text.Encoding]::UTF8"
    

vim.api.nvim_set_option_value("shellredir", '-RedirectStandardOutput %s -NoNewWindow -Wait', {})
vim.api.nvim_set_option_value("shellpipe", '2>&1 | Out-File -Encoding UTF8 %s; exit $LastExitCode', {})
    

Both of those new settings works fine on my win11.

[Jint-lzxy]

A minor patch for consistent code style: f2f3c2c.

[CharlesChiuGit]

I just observe some interesting things.

Even in wezterm, sth the icons won't show up too!

wezterm, toggleterm:

This often happens when I just opened wezterm, as if wezterm itself haven't been "warm up".

If I open a second wezterm tab, everything just work again.

Such a mystery...

[Jint-lzxy]

@CharlesChiuGit I just ran into a similar problem, and after some investigation, this also seems to be an encoding issue b/c:

• Pasting a glyph directly worked (i.e., let the terminal convert the encoding if necessary);
• Providing an Unicode code point explicitly (via `u{x}) worked (i.e., let the terminal handle the encoding details);
• These characters were not replaced by the "replacement character" � (*nothing was translated), indicating that they are interpreted as something else, probably functional keys (like ins) that won't be fed to PowerShell.

Perhaps this is indeed related to the way WezTerm (or other similar emulators), PowerShell, and PSReadLine handle the input stack. The reason why everything works fine except for the glyphs provided by NF is probably that they all have fixed representation. E.g.:

Windows          終端機                    �  <- Glyph representing PowerShell
|Latin| |CJK Unified Ideograph|   |Private Use Area|

After WezTerm "warms up", all components should be configured as UTF-8 compatible, so everything will work normally from that time on.

[CharlesChiuGit]

Cool! Let's really interesting! Thanks for your observation!