-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Terminal does not handle custom keyboard layouts with dead-keys mapping to themselves #3516
Comments
@lhecker, this might be vaguely related to the alt gr stuff... |
This is a really awesome issue description @springcomp. Thanks! 🙂 But yeah, this is a bug I've already noticed to some extent - it's urgency simply hasn't been "triaged" yet. @miniksa I've been really packed with tasks in the last few weeks, but since this issue falls into the scope of "my" work on that |
@springcomp I'm starting to have more time for side projects like this again. 🙂 The issue seems to be that you cannot reliably call Observations while reproducing the behaviorThe only other thing that I had to do, was to fully restart my PC after installing your keyboard layout, otherwise Windows refused to view it as a valid one. Alright let's assume I press AltGrnSpace (in that order)... Due to recent changes in the rendering pipeline this looks a bit different now: To ensure the cause is the AltGr handling code I then slightly modified the mentioned The output for the above sequence is:
...while it actually should return Your observation @springcomp is sligthly incorrect in that regard though as mentioned before: |
Thanks for taking the time to look into this. Your analysis is consistent with my observations and my belief is that you cannot reliably call I stand by my previous analysis, and what you observe is consistent with the keyboard layout. My keyboard layout has, among other mappings, the following behaviour:
This is by design and a recommendation of the recent standard I'm trying to be compliant with. That is as much as possible, use dead keys where possible, but sometimes, you can use combining characters to produce characters with more than one diacritical. That said, let's analyse what happens when you press a key: The user presses AltGr+n.
Therefore, Thus, Terminal displays this character even though the user does not expect it. When the user then presses o, Terminal receives The situation is exactly the same as with the traditional AZERTY layout. Again, let's analyse what happens when you press a key: The user presses AltGr+2.
Several observations:
I'll try and hack a proof of concept userland function in the coming days. Although, I think the correct and future proof solution would rather be to
|
I wanted to say thanks for taking the time. |
Thanks for the explanation about I'm not entirely clear on one part yet though: Apart from this: What we could try and do is to return early here if |
P.S.: I can tackle this soon-ish. As I said I'm pretty positive that I'll find free time to work on this more often from now on. 🙂 |
Yes! That’s what I’m saying. I’m pretty positive this is what is happening.
I’m pretty sure the output buffer would be populated with the string |
@springcomp @miniksa To test my idea I made a hacky WIPPPP modification which ignores all key events that contain characters (i.e. You can find it here. (And this is the diff.) I'm fairly concerned that I might break some delicate handling though, since I need to perfectly replicate the |
I love this! |
I've been slacking off a bit over Christmas regarding this issue. I'm sorry for that. 'Been learning some vue.js - the framework with 'tegridy. 😄 |
My basic idea was that `WM_CHAR` is just the better `WM_KEYDOWN`. The latter fails to properly support common dead key sequences like in #3516. As such I added some logic to `Terminal::SendKeyEvent` to make it return false if the pressed key represents a printable character. This causes us to receive a character event with a (hopefully) correctly composed code unit, which then gets sent to `Terminal::SendCharEvent`. `Terminal::SendCharEvent` in turn had to be modified to support potentially pressed modifier keys, since `Terminal::SendKeyEvent` isn't doing that for us anymore. Lastly `TerminalInput` had to be modified heavily to support character events with modifier key states. In order to do so I merged its `HandleKey` and `HandleChar` methods into a single one, that now handles both cases. Since key events will now contain character data and character events key codes the decision logic in `TerminalInput::HandleKey` had to be rewritten. ## PR Checklist * [x] CLA signed * [x] Tests added/passed * [x] I've discussed this with core contributors already. ## Validation Steps Performed * See #3516. * I don't have any keyboard that generates surrogate characters. Due to this I modified `TermControl::_SendPastedTextToConnection` to send the data to `_terminal->SendCharEvent()` instead. I then pasted the test string ""𐐌𐐜𐐬" and ensured that the new `TerminalInput::_SendChar` method still correctly assembles surrogate pairs. Closes #3516 Closes #3554 (obsoleted by this PR) Potentially impacts #391, which sounds like a duplicate of #3516
🎉This issue was addressed in #4192, which has now been successfully released as Handy links: |
The Microsoft Terminal does not seem to correctly handle dead-keys for a custom keyboard layout made with Microsoft Keyboard Layout Creator.
In fact, it does not correctly handle dead-keys which have a mapping for themselves in the keyboard layout.
Environment
Additionnal software:
Steps to reproduce
This issue happens with many dead-keys. For the purpose of this discussion, let us focus on one particular dead-key.
On a French AZERTY keyboard, the AltGr+2 combination is a dead-key used to input the
~
(TILDE, U+007E) diacritical mark. By itself, this sequence does not produce any output to the Terminal.Obviously, when this sequence if followed by a supported character, for instance o, the corresponding
õ
character is typed into the Terminal. A spacing version of the~
character can be typed into the Terminal by following the dead-key sequence with a Space.However, when using a custom AZERTY-NF keyboard layout, the corresponding sequence does not work.
In order to reproduce this issue:
~
(COMBINING TILDE, U+0303) has been typed into the Terminal.õ
has been typed into the Terminal.Alternatively, you can reproduce this issue by following this other set of steps:
Install MSKLC 1.4.
Install the French AZERTY keyboard layout.
With MSKLC, using the
File|Open Existing Keyboard…
menu command, open the French AZERTY keyboard layout (named Français in the list)Add an additionnal mapping for the AltGr+2 dead key like so:
é
key (VK_2) and push theAll…
button in the corresponding dialog.ctrl-alt-<key>
dead key whose value is0x007e
and push the…
button.U+007e
(TILDE) to the compositeU+0303
(COMBINING TILDE) as illustrated by the following screenshot.Project|Build DLL and Setup Package
menu command, compile the resulting layout. It should be generated in your%USERPROFILE%\Documents\layout01
folder.Install the custom layout by running
Setup.exe
.Close your Windows session and login again.
Select this layout as the currently active layout with Win+Space
In the Microsoft Terminal, type the following sequence: AltGr+2.
Like previously described, observe that a character
~
(COMBINING TILDE, U+0303) has been typed into the Terminal.Proceed to type the following key: o.
Observe that the character
õ
has been typed into the Terminal.Expected behavior
Similarly to what happens with the regular French keyboard, the AltGr+n combination on an AZERTY-NF keyboard layout is a dead-key and should not produce any output to the Terminal.
Followed by a supported character, for instance o, this dead-key sequence corresponds to the
~
diacritical mark and should type the characterõ
by itself on the terminal.Actual behavior
When typing AltGr+n, o, the string
~õ
is typed into the Terminal, whereas the stringõ
was expected instead.Analysis summary
The heart of the problem lies in the following conditions:
WM_CHAR
message that ultimately calls theToUnicodeEx
function.I believe calling
ToUnicodeEx
as part of theWM_CHAR
handler is incorrect, because the effect of this function depends on the previous internal keyboard state.Extended analysis
Obviously, the expected behaviour is correctly observed in the builtin CMD, PowerShell and WSL terminals on Windows.
By compiling the latest version of the source code for Microsoft Terminal, I can see that the expected behaviour is also correctly observed when running the
OpenConsole.exe
program. The faulty behaviour only happens in theWindowsTerminal
application launched from theCascadiaPackage
.I was able to pinpoint a troublesome behaviour at this location in the source code.
There, the
ToUnicodeEx
^ function does not return the correct result.The reason for that is that this function is called inside a
WM_CHAR
handler where the keyboard internal state already contains the current character. However, theToUnicodeEx
function changes its behaviour depending on the current keyboard state.For instance, consider the following table, that summarizes the parameters supplied to the
ToUnicodeEx
function:When called by itself in a a simple program, the
ToUnicodeEx
returns the correct result for both layouts, which is:-1
indicating a dead key."~"
(TILDE, U+007E).However, when called from within the
WM_CHAR
handler, the keyboard state contains the current (dead key character), theToUnicodeEx
function tries to combine the character corresponding to the supplied virtual key code, scan code and key states with the current internal keyboard state for the process.2
indicating that a two-character string is returned."~~"
(TILDE, U+007E; TILDE, U+007E)In fact, the
ToUnicodeEx
function combines the supplied character with the previous keyboard state. In the case of virtually all keyboard layouts on the planet, there is no such valid combination. Therefore, theToUnicodeEx
function emits a string where the corresponding dead key character is emitted twice. That is what happens on Windows in all applications when typing the sequence AltGr+2 twice on a French AZERTY keyboard.Microsoft Terminal takes advantage of this fact, and when the
ToUnicodeEx
function returns anything other than1
(a single character) or-1
(a dead key character), it does not produce any output.As for the AZERTY-NF layout of the customized layout described in the section for how to reproduce this issue, the
ToUnicodeEx
function behaves like so:1
indicating that a single character string is returned."~"
(COMBINING TILDE, U+0303).This is consistent because the internal keyboard state already contains the dead key character and when combined with itself again, the keyboard layout is instructed to emit this particular character.
Because this results in a single character string, Microsoft Terminal writes this character to the output.
Conclusion
I understand that this is a freak issue but I believe virtually all other Windows terminal-like apps handle the custom keyboard layout correctly, which seem to indicate that this is a bug in Microsoft Terminal only.
The text was updated successfully, but these errors were encountered: