issues with the fontconfig rules

[fabiangreffrath]

Hi guys,

something is wrong with your fontconfig rules files, i.e. either with the rules themselves or with the priority that you suggest for installing them. Let's have a look at Nimbus Mono PS specifically.

This happens on my system before installing any of your fontconfig rules:

> fc-match monospace
DejaVuSansMono.ttf: "DejaVu Sans Mono" "Book"

This is the expected result, we will lates see why. This, however, is what happens once I install your urw-nimbus-mono-ps.conf fontconfg rules file as /etc/fonts/conf.d/30-urw-nimbus-mono-ps.conf:

> fc-match monospace
NimbusMonoPS-Regular.t1: "Nimbus Mono PS" "Regular"

As you can see, due to this rule, Nimbus Mono PS does a system-wide hijack of the "monospace" fontconfig generic, which has an immediate effect on e.g. my terminal and text editor. This is neither expected nor -- I hope -- intended.

It all happens because of this specific rule in the urw-nimbus-mono-ps.conf file:

  <!-- Generic name aliasing -->
  <alias>
    <family>monospace</family>
    <prefer>
      <family>Nimbus Mono PS</family>
    </prefer>
  </alias>

Please note that (a variant of) this rule is already in fontconfig's upstream rule file in /etc/fonts/conf.d/60-latin.conf:

        <alias>
                <family>monospace</family>
                <prefer>
                        <family>Bitstream Vera Sans Mono</family>
                        <family>DejaVu Sans Mono</family>
                        <family>Inconsolata</family>
                        <family>Andale Mono</family>
                        <family>Courier New</family>
                        <family>Cumberland AMT</family>
                        <family>Luxi Mono</family>
                        <family>Nimbus Mono L</family>
                        <family>Nimbus Mono</family>
                        <family>Nimbus Mono PS</family>
                        <family>Courier</family>
                </prefer>
        </alias>

As you can see, there is a specific priority in the monospace fonts, probably based on the (admittedly subjective) suitability as a terminal font -- and DejaVu Sans Mono comes clearly before Nimbus Mono PS. However, by installing the urw-nimbus-mono-ps.conf file with a higher priority than 60-latin.conf, this ordering is messed up and Nimbus Mono PS is installed with an unsuitably high priority.

I see two ways out of this unfortunate situation: Either, you recommend to install the fontconfig rules files with a priority lower than that of 60-latin.conf, e.g. 61, or you remove the redundant rules that are already part of fontconfig upstream's rules set out of your fontconfig rules files.

[fabiangreffrath]

To be more precice, I think only the rules removed here should be shipped by this project:
https://cgit.freedesktop.org/fontconfig/commit/conf.d?id=cc67d7df172431cb345ed42c27eb852e2ee65ae2

This means, especially, that prefer rules like the following will have to get removed:

  <alias>
    <family>serif</family>
    <prefer>
      <family>Standard Symbols PS</family>
    </prefer>
  </alias>

Gosh, you even apply this to Standard Symbols PS, thus setting it to be the preferred font for the serif generic. What did you think when you added this?

[deekej]

Hello @fabiangreffrath, so to clarify everything written here...

This happens on my system before installing any of your fontconfig rules:

fc-match monospace
DejaVuSansMono.ttf: "DejaVu Sans Mono" "Book"

This is the expected result, we will lates see why. This, however, is what happens once I install your urw-nimbus-mono-ps.conf fontconfg rules file as /etc/fonts/conf.d/30-urw-nimbus-mono-ps.conf:

fc-match monospace
NimbusMonoPS-Regular.t1: "Nimbus Mono PS" "Regular"

As you can see, due to this rule, Nimbus Mono PS does a system-wide hijack of the "monospace" fontconfig generic, which has an immediate effect on e.g. my terminal and text editor. This is neither expected nor -- I hope -- intended.

Yes, you're right, and the hijack actually happens. We are already dealing with this in Fedora for example.

I see two ways out of this unfortunate situation: Either, you recommend to install the fontconfig rules files with a priority lower than that of 60-latin.conf, e.g. 61, or you remove the redundant rules that are already part of fontconfig upstream's rules set out of your fontconfig rules files.

After agreement with fontconfig upstream we have used value 60 in Fedora to resolve this issue.

To be more precice, I think only the rules removed here should be shipped by this project:
https://cgit.freedesktop.org/fontconfig/commit/conf.d?id=cc67d7df172431cb345ed42c27eb852e2ee65ae2

They actually are, and they are shipped in separate fontconfig files - one fontconfig file for each font, and three special fontconfig files (urw-fallback-*) to deal with some special case scenarios like e.g. mapping of old (URW)++ fonts to latest versions, etc.

This means, especially, that prefer rules like the following will have to get removed:

I don't think so. These rules were created by the fontconfig upstream's guidelines. The only problem with these fontconfig files is choosing correct priority/ordering value for yourself.

Please note that these changes were done in coordination with fontconfig upstream, because they prefer that the each font family/project keeps their own fontconfig files for the fonts.

So I have kind of volunteered to "maintain" the fontconfig files for these (URW)++ fonts, because Artifex does not actually need them for the Ghostscript in any way. But I'm not a fontconfig guru, so that's why I'm opened to any feedback and discussion. :)

Gosh, you even apply this to Standard Symbols PS, thus setting it to be the preferred font for the serif generic. What did you think when you added this?

If you would look at the previous versions of (URW)++ fonts and older versions of fontconfig, you could actually see that those fontconfig configuration were "living" as part of default fontconfig configuration (shipped with fontconfig by default, at least in Fedora). These fontconfig rules had the priority 30-*, and that's the reason why I suggested to use the same value in the fontconfig/README.md file .

Which takes me back on how to correctly select the priority/ordering value for these specific fontconfig files... First, we have to realize that each Linux distribution using fontconfig can have their own rules on priority/ordering of the fonts used/installed on their operating system. As far as I know (correct me if I'm wrong), fontconfig does not provide any "official" guidelines on how to choose priority/ordering for each font family. This is left entirely on each fontconfig maintainer for each Linux distro.

Therefore if Fedora would choose 60, that still might not be good enough value for other distributions, because those might prefer different fonts/look/feeling of their distribution, and that's (IMHO) how they create their own ("internal") guidelines on how to set correct fontconfig priority/ordering inside their distribution.

And that's the reason why it is said in fontconfig/README.md file that the value 30 is only a suggestion. I really didn't know what would be the good value for this (and I don't think there is any good universal value that we can use/suggest). I used that value, because it was used inside the fontconfig's default configuration files before the split was made.

Just an example of Fedora's fontconfig priority/ordering guidelines:

Fontconfig configuration file ordering

Fontconfig parses files in /etc/fonts/conf.d/ by filename order. The following
conventions are used to attibute the appropriate prefix to new files:

┌────────┬────────────────────────────────────────────────────────────────────┐
│ Prefix │ Description                                                        │
├────────┼────────────────────────────────────────────────────────────────────┤
│ 50     │ User override                                                      │
│ 51     │ Local system overrides                                             │
│ 55-59  │ High priority LGC (Latin Greek Cyrillic) fonts                     │
│        │ (distribution general-purpose default fonts)                       │
│ 60     │ Main LGC font list                                                 │
│ 61-64  │ Low priority LGC fonts                                             │
│ 65-69  │ Fonts with less common encodings, ending with fonts that provide   │
│        │ coverage of exotic unicode blocks at the expense of drawing        │
│        │ quality                                                            │
└────────┴────────────────────────────────────────────────────────────────────┘

Fontconfig uses an inverted priority scheme where files with lower-number
prefixes take precedence over files with higher-number prefixes.

It is almost always safe to use high numbers in the prefix range your font
belongs to, if you do not want to perturb distribution defaults.

See also /etc/fonts/conf.d/README

To sum up - I agree that the suggested value 30 is unfortunate, and we should suggest something with lower priority/ordering value. (Does 60 sounds good to you?) And I think it should be stated more clearly that this value is not universally correct, and each user of these fontconfig files should first consult with their distribution's official guidelines on which value they should use.

Was my explanation okay for you, or is something unclear? Or would you like to make any other changes? :)

[fabiangreffrath]

So I have kind of volunteered to "maintain" the fontconfig files for these (URW)++ fonts, because Artifex does not actually need them for the Ghostscript in any way. But I'm not a fontconfig guru, so that's why I'm opened to any feedback and discussion. :)

Please don't mistake my criticism as offensive! I just had the impression that the supplemental files lacked some review so far and tried to contribute this.

I don't think so. These rules were created by the fontconfig upstream's guidelines.

Following guidelines doesn't necessarily mean that the result is correct. In the example in my previous post you declared Standard Symbols PS as the preferred font for the "serif" family, which I find pretty hard to follow.

These fontconfig rules had the priority 30-*, and that's the reason why I suggested to use the same value in the fontconfig/README.md file .

This would make a lot more sense if you just replaced the rules that have been removed upstream by fontconfig:

https://cgit.freedesktop.org/fontconfig/commit/conf.d?id=cc67d7df172431cb345ed42c27eb852e2ee65ae2

But you added a lot more rules and some of them apparently don't belong into or contradict existing rules in that priority range.

As far as I know (correct me if I'm wrong), fontconfig does not provide any "official" guidelines on how to choose priority/ordering for each font family.

Fontconfig upstream comes with a perfectly sorted list of default rules, and I am sure that you are not supposed to change them:

https://cgit.freedesktop.org/fontconfig/tree/conf.d

To sum up - I agree that the suggested value 30 is unfortunate, and we should suggest something with lower priority/ordering value

I'd change the suggestion to "place the rules with a lower priority than upstream's 60-latin.confrule". This should pretty much cover all possible cases of distro-specific rule-reordering. ;)

Or would you like to make any other changes?

I'd prefer if you could remove the redundant rules (as I pointed out, some of the rules you added are already in upstream's 60-latin.conf) and the ones that don't make immediate sense (such as declaring Standard Symbols PS as the preferred serif font).

[deekej]

Please don't mistake my criticism as offensive! I just had the impression that the supplemental files lacked some review so far and tried to contribute this.

Don't worry, I'm not. :) This is a necessary part of all FOSS projects, and sometimes it's like discussing politics... :D It might be boring for many people, but it's often necessary. ;) (And sometimes you need to "beat people down" with good arguments to change their mind. :D)

Following guidelines doesn't necessarily mean that the result is correct.

That's true, and it can certainly be this case.

In the example in my previous post you declared Standard Symbols PS as the preferred font for the "serif" family, which I find pretty hard to follow.

I had a problems dealing with this myself as well. After that I have decided to follow what Wikipedia says about it (https://en.wikipedia.org/wiki/Symbol_(typeface)):

"It contains a complete unaccented Greek alphabet (upper and lower case) and a selection of commonly used mathematical symbols. Insofar as it fits into any standard classification, it is a serif font designed in the style of Times Roman."

So, this fonts falls into LGC (Latin Greek Cyrilic) group of fonts, because it can be used to display Greek correctly. And the font intself contains serifs as well. Ergo, by the definition of serif fonts, this Standard Symbols PS font is a serif font, even though its main purpose is for displaying mathematical symbols.

To follow up on this - here's the decision making algorithm/process used for deciding to which generic group the font falls into:

Fontconfig generic font families

The CSS classification system currently used by fontconfig is the following:

   1. a font that can not be used to write long “professional” texts is a 
      “fantasy” font.
   2. otherwise:
         1. if it's monospaced, it's a “monospace” font,
         2. if it's variable-width and:
               1. it simulates hand-written text, it's a “cursive” font,
               2. it uses serifs¹, it's a “serif” font,
               3. it does not use serifs, it's a “sans-serif” font, 
         3. otherwise, are you really sure it's not a “fantasy” font? 

Some font authors also make their fonts self-categorize, thus checking the 
font metadata in fontforge may provide another hint². However, sometimes this 
metadata is missing or plain false, so do not rely on it 100%. 

¹ http://en.wikipedia.org/wiki/Serif
² <CTRL> + <SHIFT> + <F> then check the OS/2 tab.

© 2009 Nicolas Mailhot <nim at fedoraproject dot org>

As you can see, by that process/algorithm, it falls into serif group. However, I don't want to stand up behind this argument and not being opened to other opinions. So here's my suggestions:

1. I will discuss the use of serif for Standard Symbols PS directly with fontconfig upstream, to see what's their opinion on this. (One of the fontconfig upstream developers is my colleague, but on the other side of the World.)
2. We will use whatever solution/workaround the upstream will suggest or provide to us, no matter what they will say about the font belonging into serif group or not.
3. In case the font will remain in serif group, I would expect that it's priority should be lowered even more (behind 60-latif.conf).

Does it sound reasonable enough for you? :)

This would make a lot more sense if you just replaced the rules that have been removed upstream by fontconfig:

Using what upstream removed (based on my patch) would result only (IMHO) in fontconfig working partially with these fonts. Again, I followed up the guidelines, thats why the fontconfig for the fonts now contains additional mappings. All of these mappings should help in different scenarios, which could potentionally happen. Not having these mappings complete could results that these fonts wouldn't be used as they should, and some other similar fonts would be used instead.

I had several discussions with upstream about these mapping rules. However, even if you try comprehense it all, sometimes you just miss something, because the number of font mappings currently messing/working with each other is really complex.

But you added a lot more rules and some of them apparently don't belong into or contradict existing rules in that priority range.

So far, you have only mentioned the Standard Symbols PS. If there are any other problems, please tell me exactly which ones.

Fontconfig upstream comes with a perfectly sorted list of default rules, and I am sure that you are not supposed to change them:

As I said, I proposed that patch to upstream which removed some of the old (URW)++ aliases, and we kept only small subset for compatibility reason there. And actually fontconfig says it totally up to each distribution if they will use the fontconfig default values or not, or if they will decide to somehow "override" and to which extent.

Also, the point here is not to change them - for that I would send a patch to upstream. If we get back to the root cause of this - fontconfig having wrong priority - that also takes us back to what upstream said - each user/distribution are free to override/change the prirority rules/ordering based on their own preferences. This is FOSS after all. :)

So in this case, when you are using the fonts on your own, based on the fontconfig/README.md, you also know that the value 30 mentioned there was just a suggestion, not an unbreakable rule. ;) You are free to change the value yourself. Same applies to any fontconfig file you have. Because if you are using it "on your own", you are also using it "on your own risk". It totally up to you to decide which value is correct priority/ordering for (URW)++ fonts.

If you are using it as a package from your Linux distribution, then you need to contact a maintainer of this fonts package, and ask him to set the fontconfig priority correctly for that specific Linux distribution you are using... ;)

IMHO, we're kind of beating the dead horse here, so to get back to what we can really change here:

1. Check if serif should be used for Standard Symbols PS - I totally agree on this.
2. Improve wording in fontconfig/README.md to be more specific - I agree on this.
3. Suggest a value higher than 60, because of upstream's default fontconfig file 60-latin.conf - again, I totaly agree on this.

NOTE: And than you for pointing that out - because I actually wasn't aware that (URW)++ mapping/aliasing is done in other default fontconfig files. That actually explains nicely your problem, and also points out that my patch suggested to upstream was incomplete. I will propose an additional patch to upstream for them to correctly reflect the (URW)++ fonts.

I don't see any other issues that we can fix here now. If you know about some, please let us know. :)

Ah, and small note regarding this:

I'd prefer if you could remove the redundant rules (as I pointed out, some of the rules you added are already in upstream's 60-latin.conf).

There are also some redundant rules in 30-metric-aliases.conf as well, and after discussion with upstream we have decided to keep them there. The reason for this is that fontconfig shouldn't have a problem with same rules appearing multiple times (according to upstream), and there could be a situations where:
A) Some distribution/user would decide to not ship/use the fontconfig's default config files, thus the (URW)++ fontconfig files needs to be able to "work on their own" properly.
B) Some distribution/user might have documents that require (URW)++ fonts, but they might not have them installed on their system. When this happens, the duplicate aliasing/mapping in fontconfig's default files will help to choose a different font with similar look/weight.

Uff, I hope my explanations and other notes makes sense... :D

[fabiangreffrath]

Thank you very much for the detailed elaboration, @deekej !

Does it sound reasonable enough for you?

Yes, this (i.e. the three points you made up) sounds reasonable to me!

If there are any other problems, please tell me exactly which ones.

I am not convinced that each font should become the preferred font in its generic category and then the rule gets shifted so far back in priority that other rules take precedence and the initial rule becomes worth as much as an <accept> rule.

If you are using it as a package from your Linux distribution, then you need to contact a maintainer of this fonts package, and ask him to set the fontconfig priority correctly for that specific Linux distribution you are using... ;)

I am the Debian maintainer for this package, that's why I am so picky about these rules. ;)

https://tracker.debian.org/pkg/fonts-urw-base35

Check if serif should be used for Standard Symbols PS - I totally agree on this.
Improve wording in fontconfig/README.md to be more specific - I agree on this.
Suggest a value higher than 60, because of upstream's default fontconfig file 60-latin.conf - again, I totaly agree on this.

Absolutely, I agree that these are the three things that need to happen to close this issue.
For the record, I'll move the rules to priority 61 in the next upload for the Debian package to make it clear that they are supposed to sort after 60-latin.conf.

I will propose an additional patch to upstream for them to correctly reflect the (URW)++ fonts.

Thank you! I am convinced that if you work closely together with fontconfig upstream and the Fedora fontconfig maintainer and let them have a look over the rules provided here, that everything gets back into reasonable shape soon. ;)

Cheers,

• Fabian

[deekej]

I am not convinced that each font should become the preferred font in its generic category and then the rule gets shifted so far back in priority that other rules take precedence and the initial rule becomes worth as much as an rule.

Could you please elaborate on this? You probably know more about this topic than I do. :) Because if I recall correctly, there were 2 accept rules before (for Standard Symbols PS and D05000L aka Dingbats fonts). When I was discussing the proposed patch to upstream, they didn't recall why the <accept> was used there in the first place, and so we have come to a conclusion that "standard" rules should be enough...

It would be nice if you good provide some more explanation, so we could fix this if needed. ;)

For the record, I'll move the rules to priority 61 in the next upload for the Debian package to make it clear that they are supposed to sort after 60-latin.conf.

I will be lowering the fontconfig files' priority in Fedora as well to 61, or maybe even lower (down to 64 max.)... I am also a maintainer of these fonts - in Fedora. :D

[fabiangreffrath]

Could you please elaborate on this? You probably know more about this topic than I do.

Not sure, let's see. Please excuse my naive language, I am also not a guru at all. ;)

It is my understanding that <prefer> and <accept> rules do about the same, i.e. they add a font to the prioritized list of alternatives that is suitable as an alias for a specific font family. The difference is that <accept> rules add the font to the bottom of the priority list, whereas <prefer> rules put the font on top of that list.

This is where the hijacking came from: The urw-base35 rules were installed with higher priority than upstream's 60-latin.conf, so each rule pushed its corresponding urw font on top of the priority list for the "serif", "sanf-serif" and "monospace" generics. The properly sorted list of alternatives in 60-latin.conf was treated as second priority then.

On the other hand, if we sort the urw-base35 rules after upstream's 60-latin.conf rule, the top of the priority list is already taken by the fonts listed there, so all that is left to do is sort after them towards the bottom of the priority list. This, in turn, has the same effect as registering them with an <accept> rule in the first place.

I hope this makes sense. ;)

I am also a maintainer of these fonts

So we are the maintainers! hugh

[deekej]

Hey @fabiangreffrath, that makes a lot of sense to me now, thanks! :) I'm currently hold up with my main job responsibilities, so I hope to get on with this during the next week. ;)

[fabiangreffrath]

No need to rush. As I pointed out, if applied with a priority of 61 or lower, the impact of the <prefer> rules is pretty harmless.

[deekej]

Hello @fabiangreffrath, sorry for the late response. I created an update to README.md for the fontconfig files. Could you please check it is okay? :)

I still need to contact fontconfig upstream about the serif usage for Standard Symbol PS, but this commit is independent on that.

[fabiangreffrath]

Looks good to me, thanks!