Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Weird fonts on Windows Server 2022 #11032

Closed
faxx1080 opened this issue Aug 25, 2021 · 36 comments · Fixed by #11764
Closed

Weird fonts on Windows Server 2022 #11032

faxx1080 opened this issue Aug 25, 2021 · 36 comments · Fixed by #11764
Assignees
Labels
Area-Fonts Related to the font Culprit-Centennial Help Wanted We encourage anyone to jump in on these. Issue-Bug It either shouldn't be doing this or needs an investigation. Priority-3 A description (P3) Product-Terminal The new Windows Terminal. Resolution-Fix-Committed Fix is checked in, but it might be 3-4 weeks until a release.

Comments

@faxx1080
Copy link

Windows Terminal version (or Windows build number)

1.10.1933.0

Other Software

Fresh install of Windows Server 2022 Evaluation, straight from https://www.microsoft.com/en-us/evalcenter/. Did not yet check for updates.

Steps to reproduce

Install latest preview appxbundle using Add-AppxPackage.
Running Windows Server on VMWare Workstation 16.1.2 build-17966106.

Expected Behavior

Capture

This is how cascadia code normally looks.

Actual Behavior

This is what I got. Doesn't quite seem right.

Capture-bad

@ghost ghost added Needs-Triage It's a new issue that the core contributor team needs to triage at the next triage meeting Needs-Tag-Fix Doesn't match tag requirements labels Aug 25, 2021
@j4james
Copy link
Collaborator

j4james commented Aug 25, 2021

This looks it might be the same thing as #11017.

@zadjii-msft
Copy link
Member

This is wild that we got the same report 3 times in a week. @faxx1080 what font are you using in the Terminal? What DPI scaling?

@zadjii-msft zadjii-msft added the Needs-Author-Feedback The original author of the issue/PR needs to come back and respond to something label Aug 26, 2021
@faxx1080
Copy link
Author

faxx1080 commented Aug 26, 2021 via email

@ghost ghost added Needs-Attention The core contributors need to come back around and look at this ASAP. and removed Needs-Author-Feedback The original author of the issue/PR needs to come back and respond to something labels Aug 26, 2021
@faxx1080
Copy link
Author

**I'm not 100% sure it was Cascadia Code; it was whatever font is the default after installing the Appx package.

Note that I did this on the out of box Administrator account which has UAC turned off.

@zadjii-msft
Copy link
Member

Okay this is wild. If you go into something like Microsoft Word, or anything that has a font picker, and go through that list, does "Cascadia Code" appear in that list? And does it appear in the dropdown list in the Terminal? We just got an internal report of the same thing that suggested that the font was uninstalled?

@rmbolger
Copy link

rmbolger commented Sep 2, 2021

I'm having the same issue on a recently installed, non-domain joined, Windows Server 2022 Standard (version 21H2, build 20348.202) VM. It sure seems like whatever the default font is just doesn't get installed.

The Font face field in the Terminal appearance settings is just empty for the profile. It's also not listed in the picker. I haven't tried to make any settings changes yet.

terminal-settings-no-fontface

WordPad doesn't show any Cascadia related fonts in the list.

wordpad-no-cascadia

I installed the Microsoft.WindowsTerminal_1.10.2383.0_8wekyb3d8bbwe.msixbundle via Add-AppxPackage. The only other things I currently have installed on this box are Pwsh 7.1.4, the latest VS Code (user install), and VMware Tools 11.1.

One more note. I had previously accidentally installed the Microsoft.WindowsTerminalPreview_1.11.2421.0_8wekyb3d8bbwe.msixbundle package which I uninstalled after realizing it was the preview instead of the release.

@zadjii-msft
Copy link
Member

The Font face field in the Terminal appearance settings is just empty for the profile. It's also not listed in the picker.

WordPad doesn't show any Cascadia related fonts in the list.

WHAT THE

That's WILDLY unexpected, but definitely explains why you're seeing what you're seeing. It doesn't make sense, but that at least explains it. The Cascadia family of fonts should be shipping in our package:

<Extensions>
<uap7:Extension Category="windows.sharedFonts">
<uap7:SharedFonts>
<uap4:Font File="CascadiaCode.ttf" />
<uap4:Font File="CascadiaCodeItalic.ttf" />
<uap4:Font File="CascadiaMono.ttf" />
<uap4:Font File="CascadiaMonoItalic.ttf" />
</uap7:SharedFonts>
</uap7:Extension>
</Extensions>

But if they're not getting installed when you install the Terminal, then a bunch of stuff will break. We pretty consistently rely on those fonts existing.

@rmbolger
Copy link

rmbolger commented Sep 2, 2021

I notice in that same file, there's a Dependencies section that seems to only target Windows.Desktop. Might that be part of the problem? (I know zero about MSIX, just grasping at straws)

<Dependencies>
<TargetDeviceFamily Name="Windows.Desktop" MinVersion="10.0.18362.0" MaxVersionTested="10.0.19041.0" />
</Dependencies>

@j4james
Copy link
Collaborator

j4james commented Sep 2, 2021

Something interesting I noticed about the SharedFonts extension: the documentation here suggests it's in the uap7 namespace, the linked article here claims it's in the uap2 namespace, and the example package in that article uses the uap4 namespace. It also lists uap4 in the IgnorableNamespaces attribute, which seems like a good idea, because clearly nobody knows what namespace it really belongs to! 😉

I also noticed that example includes the extension at the Application level rather than the Package level, which might make a difference.

I too know zero about MSIX, so maybe none of this matters.

@zadjii-msft
Copy link
Member

That seems unlikely to me - #11041 looks like the same thing, and that wasn't on a Server SKU, and the internal report we had of this wasn't either. But I like where your head is at!

We've had some notorious difficulty with the MSIX platform and our font in the past. In the last release, this actually just caused the Terminal to crash on startup if the font wasn't found. Now I suppose we survive, but find ourselves in a state where the font can't be found in XAML. We'll need to think about this.

@zadjii-msft
Copy link
Member

Something interesting I noticed about the SharedFonts extension: the documentation here suggests it's in the uap7 namespace, the linked article here claims it's in the uap2 namespace, and the example package in that article uses the uap4 namespace. It also lists uap4 in the IgnorableNamespaces attribute, which seems like a good idea, because clearly nobody knows what namespace it really belongs to! 😉

What the heck. That's a great observation

@DHowett
Copy link
Member

DHowett commented Sep 2, 2021

So, shared fonts I believe were actually introduced with uap4, as an application extension. You couldn't have a font package that didn't have an application, so things like "Arial Nova" shipped with . . . a web app. An empty web app. As their application. 🤦🏻‍♂️

uap7 came around and promoted it to a package extension, so it didn't need to be tied to an application. It kept uap4:Font entries inside it for manifest validation purposes, but moved somewhere else and needed a new namespace to hold it.

@DHowett
Copy link
Member

DHowett commented Sep 2, 2021

I notice in that same file, there's a Dependencies section that seems to only target Windows.Desktop. Might that be part of the problem? (I know zero about MSIX, just grasping at straws)

<Dependencies>
<TargetDeviceFamily Name="Windows.Desktop" MinVersion="10.0.18362.0" MaxVersionTested="10.0.19041.0" />
</Dependencies>

I believed that Windows.Desktop covered "all things where Windows presents as a normal desktop OS" (including Server), but we can validate that. That dependency, however, blocks installation if it is not present...

@DHowett
Copy link
Member

DHowett commented Sep 2, 2021

Would one of you mind collecting a recursive export of the registry key HKEY_CURRENT_USER\Software\Microsoft\Windows NT\CurrentVersion\Fonts?

@rmbolger
Copy link

rmbolger commented Sep 2, 2021

That key exists, but is empty on my Server 2022 VM. Here's the export and a view from the regedit GUI.

Windows Registry Editor Version 5.00

[HKEY_CURRENT_USER\Software\Microsoft\Windows NT\CurrentVersion\Fonts]

hkcu-fonts

The HKLM equiv path has plenty of font entries in it. Though none appear to be Cascadia related.

@DHowett
Copy link
Member

DHowett commented Sep 2, 2021

A-HA!

It looks like the "Shared Font" deployment helper (a deployment helper is something that runs when an MSIX is installed) isn't built into Windows Server. That's great for us to know. Thanks!

@DHowett
Copy link
Member

DHowett commented Sep 2, 2021

The thing I don't totally understand is why our side-by-side font loader didn't load it. @miniksa might know more, or be able to troubleshoot it from here. We explicitly look for fonts in our own package and try to load them...

This might suggest a root cause for the weird letter selection, if it's not working totally properly, or if we're doing font fallback with a collection that doesn't contain Cascadia.

@ianjoneill
Copy link
Contributor

I was intrigued by this, so I span up a Windows Server 2022 VM and deployed the code from main to it using the VS remote debugging tools.

The DxFontInfo class is successfully finding the font files next to the application. The first time it deployed it didn't copy the Cascadia fonts into the remote deployment directory, which lead to the font falling back to Consolas and the fallback font dialog being displayed.

I then copied the font into the remote deployment directory and got the weird font rendering.

Adding in some simple OutputDebugString() statements to DxFontInfo::_FindFontFace(), I found that _NearbyCollection() is successfully finding the fonts, and _FindFontFace() is successfully loading the font family and creating the font face.

Unfortunately I haven't had time to do any further debugging, but it looks like the issue is higher up in the application than the font loading.

@zadjii-msft zadjii-msft added Needs-Discussion Something that requires a team discussion before we can proceed and removed Needs-Discussion Something that requires a team discussion before we can proceed labels Sep 7, 2021
@ianjoneill
Copy link
Contributor

On a related note, the settings UI won't show the Cascadia font either, because it only loads the fonts from the system font collection.

@miniksa
Copy link
Member

miniksa commented Sep 9, 2021

Nnnnnnnnnnnnnnnnnnnnn. OK. Thanks.

@miniksa
Copy link
Member

miniksa commented Sep 9, 2021

OK I think I've got it. We need to change the way we handle the fallback to the font in the package.

Instead of running this process where we check the system collection and then fallback and check the nearby collection... I believe we need to make it so there's just one merged collection of both and then use that ANYWHERE in the application where a collection is needed.

So the _nearbyCollection is probably no longer a property of DxFontInfo as it has little to do with the chosen font and is more just a representation of the world.

We just need to... on startup of the rendering pipeline... use ::GetSystemFontCollection on one of the IDWriteFactory levels to get the system collection. (Probably IDWriteFactory3 so it gives us a IDWriteFontCollection1 straight up.) Then we need to use IDWriteFontCollection1::GetFontSet to get the underlying IDWriteFontSet from it. We then follow most of the contents of DxFontInfo::_NearbyCollection to create an IDWriteFontSetBuilder and use IDWriteFontSetBuilder::AddFontSet to put the system font set into the set builder first. Then we continue to use IDWriteFontSetBuilder2::AddFontFile to append the files next to our binary into the collection. Then we return that as a IDWriteFontCollection/1 and use that anywhere in the application where we're currently passing nullptr and having it load the system collection like for creating the IDWriteTextFormat and probably somehow exposing it to TerminalSettingsModel so it can pick it up and use it in ProfileViewModel::UpdateFontList().

EDIT: Oh and if we're not on Windows 10... then we have our _terminalCollection pointer as nullptr and dutifully pass that into all locations that ask for a collection and they'll resolve the system one since the fallback only works on 10+ anyway.

@miniksa miniksa self-assigned this Sep 9, 2021
@DHowett
Copy link
Member

DHowett commented Sep 9, 2021

If you do this, is it possible to ensure that the system fonts win over the nearby ones? Eventually, when we decouple Cascadia, we will be able to freeze down a "backup" version and we don't want it to defeat the system-installed version if it is newer.

@miniksa
Copy link
Member

miniksa commented Sep 9, 2021

If you do this, is it possible to ensure that the system fonts win over the nearby ones? Eventually, when we decouple Cascadia, we will be able to freeze down a "backup" version and we don't want it to defeat the system-installed version if it is newer.

I am 78% sure that putting the System Font Set in first and then appending the package one next will make that be the order of priority.

@miniksa
Copy link
Member

miniksa commented Sep 9, 2021

@fdwr... opinion on my shenanigans?

@fdwr
Copy link

fdwr commented Sep 9, 2021

I am 78% sure that putting the System Font Set in first and then appending the package one next will make that be the order of priority.

@miniksa : Yes, a tie between two of the same font in the font set (or in the old hierarchical WWS collection) is broken by addition order, favoring earlier indices. (there was a brief bug in Win 8.1 selfhost where it was sometimes not fully deterministic if you overflowed the font collection cache size, but I fixed that, and it should AFAIK still behave with first-index-wins-the-tie)

This is what I got. Doesn't quite seem right.

Interesting screenshot. It seems the m and w are being uniformly scaled to fit into the grid cell (maybe for the sake of fat emoji). At least for m and w (and other alphabetic letters), it might look better to scale them with a non-uniform aspect ratio (that is, only squeeze them horizontally to make them skinnier - granted, this would make the stems thinner too). Though, I'm assuming Terminal (under normal circumstances) doesn't select proportional fonts in the first place.

@miniksa
Copy link
Member

miniksa commented Sep 9, 2021

I am 78% sure that putting the System Font Set in first and then appending the package one next will make that be the order of priority.

@miniksa : Yes, a tie between two of the same font in the font set (or in the old hierarchical WWS collection) is broken by addition order, favoring earlier indices. (there was a brief bug in Win 8.1 selfhost where it was sometimes not fully deterministic if you overflowed the font collection cache size, but I fixed that, and it should AFAIK still behave with first-index-wins-the-tie)

Excellent. Thank you.

This is what I got. Doesn't quite seem right.

Interesting screenshot. It seems the m and w are being uniformly scaled to fit into the grid cell (maybe for the sake of fat emoji). At least for m and w (and other alphabetic letters), it might look better to scale them with a non-uniform aspect ratio (that is, only squeeze them horizontally to make them skinnier - granted, this would make the stems thinner too). Though, I'm assuming Terminal (under normal circumstances) doesn't select proportional fonts in the first place.

Yes and yes. You're correct that my scaling algorithm is probably suboptimal here using a uniform aspect ratio and that the Terminal strongly avoids selecting proportional fonts under normal circumstances.

@faxx1080
Copy link
Author

faxx1080 commented Sep 9, 2021

I've been keeping up with this intermittently and really want to say a thank you!! to all I've learned about font edge cases here, as well as what's really changed since classic Windows (XP and up).

Not being familiar with most (sadly) of the C++ code of terminal; I "solved" this on my VM by going into the appx directory, right clicking install on all the Cascadia Code fonts. But that's even worse than the above quick fix, since I needed admin for that, and that was a system-level change. But it did solve the issue!

I also had a Windows 11 test nearby and installed Windows Terminal Preview there, and found that Cascadia Code works fine in the terminal and settings; as well as Notepad / Old Conhost, but doesn't appear in C:\Windows\Fonts; nor does it appear in the old Fonts control panel.

@fdwr
Copy link

fdwr commented Sep 10, 2021

but doesn't appear in C:\Windows\Fonts; nor does it appear in the old Fonts control panel.

@faxx1080: IIRC, the old fontext.dll enumerated the registry itself directly rather than going through GDI or DirectWrite. So it would not know about "per user" fonts, unless additional work was done to it 🤔. I think the shell verb "Install" (forget which OS version this happened) now installs fonts into %LOCALAPPDATA%\Microsoft\Windows\Fonts\ (e.g. c:\Users\fdwr\AppData\Local\Microsoft\Windows\Fonts\NotoColorEmoji.ttf) using Computer\HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Fonts, whereas "Install for all users" installs to the old c:\Windows\Fonts\ using Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Fonts.

@ghost ghost added the In-PR This issue has a related PR label Nov 15, 2021
@ghost ghost closed this as completed in #11764 Nov 16, 2021
@ghost ghost added Resolution-Fix-Committed Fix is checked in, but it might be 3-4 weeks until a release. and removed In-PR This issue has a related PR labels Nov 16, 2021
ghost pushed a commit that referenced this issue Nov 16, 2021
This commit is a minimal fix in order to pass the
`IDWriteFontCollection` we create out of .ttf files residing next to our
binaries to the `IDWriteFontFallback::MapCharacters` call. The
`IDWriteTextFormat` is used in order to carry the font collection over
into `CustomTextLayout`.

## Validation
* Put `JetBrainsMono-Regular.ttf` into the binary output directory
* Modify `HKCU:\Console\*\FaceName`  to `JetBrains Mono`
* Launch OpenConsole.exe
* OpenConsole uses JetBrains Mono ✔️

Closes #11032
Closes #11648
@elevul
Copy link

elevul commented Nov 23, 2021

Problem was solved for me on 2 Server 2022 installs after installing the Cascadia Code fonts: https://github.com/microsoft/cascadia-code

DHowett pushed a commit that referenced this issue Dec 13, 2021
This commit is a minimal fix in order to pass the
`IDWriteFontCollection` we create out of .ttf files residing next to our
binaries to the `IDWriteFontFallback::MapCharacters` call. The
`IDWriteTextFormat` is used in order to carry the font collection over
into `CustomTextLayout`.

## Validation
* Put `JetBrainsMono-Regular.ttf` into the binary output directory
* Modify `HKCU:\Console\*\FaceName`  to `JetBrains Mono`
* Launch OpenConsole.exe
* OpenConsole uses JetBrains Mono ✔️

Closes #11032
Closes #11648

(cherry picked from commit 131f5d2)
DHowett pushed a commit that referenced this issue Dec 13, 2021
This commit is a minimal fix in order to pass the
`IDWriteFontCollection` we create out of .ttf files residing next to our
binaries to the `IDWriteFontFallback::MapCharacters` call. The
`IDWriteTextFormat` is used in order to carry the font collection over
into `CustomTextLayout`.

## Validation
* Put `JetBrainsMono-Regular.ttf` into the binary output directory
* Modify `HKCU:\Console\*\FaceName`  to `JetBrains Mono`
* Launch OpenConsole.exe
* OpenConsole uses JetBrains Mono ✔️

Closes #11032
Closes #11648

(cherry picked from commit 131f5d2)
@ghost
Copy link

ghost commented Dec 14, 2021

🎉This issue was addressed in #11764, which has now been successfully released as Windows Terminal Preview v1.12.3472.0.:tada:

Handy links:

@ghost
Copy link

ghost commented Dec 14, 2021

🎉This issue was addressed in #11764, which has now been successfully released as Windows Terminal v1.11.3471.0.:tada:

Handy links:

ghost pushed a commit that referenced this issue Apr 14, 2022
The original research for a solution all the way back in #11032 contained an
unfortunate flaw. The nearby font loading code was written under the assumption
that Cascadia is missing in the system font collection, leading to our issues.
Adding nearby fonts last into the collection would thus ensure that we use
the system fonts whenever possible, but only have nearby fonts as a fallback.

This didn't work and we figured that we'd have to always prefer loading nearby
fonts over system fonts. #12554 tried to achieve this, but failed to change
the order in which the font set is built. In order to prefer nearby fonts
over system ones, we have to add the system font collection last.

## PR Checklist

* [x] Closes #11648
* [x] I work here
* [x] Tests added/passed
* [x] Embarrassment for my incompetence

## Validation Steps Performed

* Put Jetbrains Mono into the AppX directory of the Debug build
* Jetbrains Mono shows up in the font selector and is useable

Additionally a more complex mini-test was built:
Using FontForge I've cloned arial.ttf and removed all characters except for
the letter "0". Afterwards I've build a custom font collection the same way
we do it in Terminal, extracted a `FontFace` named "Arial" and called
`IDWriteFont::HasCharacter` for the letter "1".
Loading the system font collection first results in `TRUE` and loading it last
results in `FALSE` (since my custom arial.ttf doesn't have the letter "1").
This confirms that we need to load the system font collection last.
DHowett pushed a commit that referenced this issue Apr 21, 2022
The original research for a solution all the way back in #11032 contained an
unfortunate flaw. The nearby font loading code was written under the assumption
that Cascadia is missing in the system font collection, leading to our issues.
Adding nearby fonts last into the collection would thus ensure that we use
the system fonts whenever possible, but only have nearby fonts as a fallback.

This didn't work and we figured that we'd have to always prefer loading nearby
fonts over system fonts. #12554 tried to achieve this, but failed to change
the order in which the font set is built. In order to prefer nearby fonts
over system ones, we have to add the system font collection last.

## PR Checklist

* [x] Closes #11648
* [x] I work here
* [x] Tests added/passed
* [x] Embarrassment for my incompetence

## Validation Steps Performed

* Put Jetbrains Mono into the AppX directory of the Debug build
* Jetbrains Mono shows up in the font selector and is useable

Additionally a more complex mini-test was built:
Using FontForge I've cloned arial.ttf and removed all characters except for
the letter "0". Afterwards I've build a custom font collection the same way
we do it in Terminal, extracted a `FontFace` named "Arial" and called
`IDWriteFont::HasCharacter` for the letter "1".
Loading the system font collection first results in `TRUE` and loading it last
results in `FALSE` (since my custom arial.ttf doesn't have the letter "1").
This confirms that we need to load the system font collection last.

(cherry picked from commit eeb8970)
Service-Card-Id: 80641214
Service-Version: 1.13
DHowett pushed a commit that referenced this issue Apr 21, 2022
The original research for a solution all the way back in #11032 contained an
unfortunate flaw. The nearby font loading code was written under the assumption
that Cascadia is missing in the system font collection, leading to our issues.
Adding nearby fonts last into the collection would thus ensure that we use
the system fonts whenever possible, but only have nearby fonts as a fallback.

This didn't work and we figured that we'd have to always prefer loading nearby
fonts over system fonts. #12554 tried to achieve this, but failed to change
the order in which the font set is built. In order to prefer nearby fonts
over system ones, we have to add the system font collection last.

## PR Checklist

* [x] Closes #11648
* [x] I work here
* [x] Tests added/passed
* [x] Embarrassment for my incompetence

## Validation Steps Performed

* Put Jetbrains Mono into the AppX directory of the Debug build
* Jetbrains Mono shows up in the font selector and is useable

Additionally a more complex mini-test was built:
Using FontForge I've cloned arial.ttf and removed all characters except for
the letter "0". Afterwards I've build a custom font collection the same way
we do it in Terminal, extracted a `FontFace` named "Arial" and called
`IDWriteFont::HasCharacter` for the letter "1".
Loading the system font collection first results in `TRUE` and loading it last
results in `FALSE` (since my custom arial.ttf doesn't have the letter "1").
This confirms that we need to load the system font collection last.

(cherry picked from commit eeb8970)
Service-Card-Id: 80641213
Service-Version: 1.12
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area-Fonts Related to the font Culprit-Centennial Help Wanted We encourage anyone to jump in on these. Issue-Bug It either shouldn't be doing this or needs an investigation. Priority-3 A description (P3) Product-Terminal The new Windows Terminal. Resolution-Fix-Committed Fix is checked in, but it might be 3-4 weeks until a release.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

10 participants