Windows 11 - files with umlauts

[komfort64]

hello,

I have installed ntvdmx64 within "windows 11 pro" to use an application "komforttext". I can start the application without problems.
If I use umlauts "äüö" within the filename of my files there are problems. After saving the file, the name in the filesystem is
without the umlauts, but another sign. The same problem occurs with the edit-command.

There are no problems using the application with Windows 10.

Who can help me?

grettings, Jürgen

[leecher1337]

This doesn't seem to ne NTVDM-related, but is a general bug in the Windows 11 NLS-implementation.
Just try to open a normal cmd.exe Console window (using the Legacy Console, which is default enyway due to NTVDMx64 installation) and try to enter Umlauts in there. They are just displayed wrong:

So this doesn't have to do with NTVDM, but is a severe Console/NLS bug that needs to be reported to Microsoft and fixed by them.
It may be related to this console host bug (missing NLS table initialization) which also still hasn't been fixed by M$:
929c853

[leecher1337]

Analysis clearly shows that the RtlUnicodeStringToOemString API is broken on Windows 11!

Take the following code snippet:

int main(int argc, char** argv)
{
    ANSI_STRING aStr;
    OEM_STRING oStr;
    UNICODE_STRING uStr;

    // 1) Take ANSI String and convert it to OEM CP String (CP850 on German systems):
    RtlInitAnsiString(&aStr, "äöüÄÖÜß");
    RtlAnsiStringToUnicodeString(&uStr, &aStr, TRUE);
    printf("Original UNICODE string: %wZ\n", &uStr);

    RtlUnicodeStringToOemString(&oStr, &uStr, TRUE);
    RtlFreeUnicodeString(&uStr);
    printf("OEM String: %.*s\n", oStr.Length, oStr.Buffer);

    // 2) Now create an OEM_STRING for translation and translate it to Unicode
    RtlOemStringToUnicodeString(&uStr, &oStr, TRUE);
    RtlFreeOemString(&oStr);

    // 3) Output converted UNICODE string which should resemble input data
    printf("RtlOemStringToUnicodeString: %wZ\n", &uStr);

    RtlFreeUnicodeString(&uStr);

    return 0;
}

On non-broken Windows versions, when you execute this on a console, first and last string look "garbled,", OEM string looks OK (or if you pipe it to an output file and open it with an editor, first and last string look OK, OEM string looks garbled).
This is correct, as it does the conversion and you are looking at the result in a specific codepage.
When executing the same piece of code on Windows 11, all 3 strings are the same (supposedly because of mentioned Windows 11 bug that NLS tables are not initialized upon loading).

[leecher1337]

Definitely a Windows bug, see here:

[leecher1337]

Windows bug is here:

int __fastcall RtlpInitCodePageTables(unsigned __int16 a1, unsigned __int16 a2)
{
  bool v3; // bl
  int v5; // eax

  v3 = 0;
  if ( a1 == 65001 || a2 == 65001 )
    goto LABEL_8;
  if ( (int)NtGetNlsSectionPtr(11, a1, 0, &dword_4B3A6878, 0) < 0 )
  {
LABEL_7:
    v5 = 0;
    dword_4B3A687C = 0;
    dword_4B3A6878 = 0;
    goto LABEL_9;
  }
  if ( a2 != a1 )
  {
    if ( (int)NtGetNlsSectionPtr(11, a2, 0, &dword_4B3A687C, 0) < 0 )
      goto LABEL_7;
LABEL_8:
    v5 = dword_4B3A6878;
    goto LABEL_9;
  }
  v5 = dword_4B3A6878;
  dword_4B3A687C = dword_4B3A6878;
LABEL_9:
  RtlInitCodePageTable((unsigned __int16 *)v5, GlobalRtlNlsState);
  RtlInitCodePageTable((unsigned __int16 *)dword_4B3A687C, oem_cp_table);// Here it correctly initialized the OEMCP table
...

But look at :

unsigned int __stdcall RtlOemToUnicodeN(
        PWCH UnicodeString,
        ULONG MaxBytesInUnicodeString,
        PULONG BytesInUnicodeString,
        PCH CustomCPString,
        ULONG BytesInCustomCPString)
{
  struct _CPTABLEINFO *CodePageData; // eax

  CodePageData = (struct _CPTABLEINFO *)RtlpGetCodePageData();
  return RtlCustomCPToUnicodeN(
           CodePageData,
           (int)UnicodeString,
           MaxBytesInUnicodeString,
           (int *)BytesInUnicodeString,
           (unsigned __int8 *)CustomCPString,
           BytesInCustomCPString);
}

When we check RtlpGetCodePageData:

__int16 *__stdcall RtlpGetCodePageData()
{
  signed __int32 v0; // ecx
  signed __int32 v2; // [esp+0h] [ebp-4h] BYREF

  v2 = v0;
  if ( RtlpIsUtf8Process() )
    return &Utf8TableInfo;
  _InterlockedOr(&v2, 0);
  return GlobalRtlNlsState;
}

NO Reference to loaded OEM codepage!! Because using this function for OEM CP is simply WRONG!
Do they test anything at least once before releasing it to the customer? I guess no... :-(

[leecher1337]

Status report from Microsoft:
From what I can tell this is MSFT-42362468 and a fix has merged into build version 25285.

So I guess you have to wait for this Win 11 build .

[komfort64]

Thank you for your analysis. I'll wait for the build version 25285.

[komfort64]

Do you know when the build version 25285 will be released?

[leecher1337]

You can try to ask Microsoft over at this thread, maybe they know: microsoft/terminal#15038

[komfort64]

hello,
within the newest update of windows 11 the problems with the umlauts are solved.
I've tested our application. Do you know, how i can improve the performance?

[leecher1337]

Use HAXM build

[komfort64]

hello,
what's wrong when this message appears?

[leecher1337]

IT usually means that you may have installed the original HAXM driver from Intel instead of the custom IntelHaxmNTVDM driver and this driver is running.
Try to do a:

net stop IntelHAXM

and see if it stops the original Intel HAXM driver. Then the IntelHAXMNTVDM driver may start during next launch of NTVDM if installed properly.
If unsure, you can also uninstall the original Intel HAXM driver from the system to ensure that it is not running/runnable.
Some people have installed this driver because they are using Android development tools and may need it for this pupose.