Map the checking routines performed by third-party tools and how/if they overlap with native FontBakery checks

[m4rc1e]

I feel the new Font Validator covers these checks better, https://github.com/googlefonts/fontbakery/blob/master/Lib/fontbakery/specifications/googlefonts.py#L1411-L1564

I would like the results of the MS Validator reports to get logged to the cmdline.

Also, the headless Font Forge is a truly ugly dependency

---

@felipesanches edited this on 2017-Sept-22

The goal here is to come up with a comparative table describing the checks provided by each of the third-party font-linters such as Font-Forge, MSFVal, etc and how do they relate to / overlap with FontBakery explicit suite of font checks.

This table can help us decide whether to keep the dependencies or if we'd be better porting the (few?) additional checks provided by them into native FontBakery python code and ditch the dependencies.

[graphicore]

I'm not sure. But sometimes I think they report useful stuff.

I would like the results of the MS Validator reports to get logged to the cmdline.

They are logged, or? We could improve that a bit though.

[m4rc1e]

Ah yes, sorry they are logged. I'll look into them in more detail soon.

[HinTak]

I am all for people using Font Validator more (and commission/fund me working on it...), but a couple of things that font forge does hasn't got equivalent tests in Font Validator (yet). Also font forge has a rather large memory foot print, so indeed, you might/should use the Font Validator's tests if they are available.

BTW, a 4000-line python file should probably be re-factored :-). The equivalent part to the python code you refered to above is part of the glyf table test and the maxp table tests.

I think you want to have a look at the T.maxp_GlyphStats section (i.e. line 136 to 297) of part of the maxp table related test:
https://github.com/HinTak/Font-Validator/blob/master/OTFontFileVal/val_maxp.cs
what it does and does not do.

Sorry there is no overall view of the glyf test - TODO - the best I could suggest is to look at the chm file https://github.com/HinTak/Font-Validator/blob/master/bin/FontValidatorHelp.chm and have a look at all the E11XX/W11XX/I11XX/P11XX/A11XX status messages. It certainly detects "contour intersects", "Contours have correct directions", but perhaps not "Glyphs have points at extremas" .

Argh, I was wrong about the last part. Font Validator can emit a "W1112: Not all extremes are marked with the on-curve control points" message. Perhaps the best summary on what glyf table test does and does not do , I can give you, is this issue I filed myself:
microsoft/Font-Validator#15 .

[HinTak]

BTW, maxp related status codes are E19XX/W19XX/I19XX's. E for error, w for warning, I for information, A for application error [i.e. FontVal itself choked on something], P for pass, then the two digits are organized by table. 11 is glyf and 19 is maxp. They were alphabetical in the beginning, but newer tables are added to the end...

As you see from microsoft/Font-Validator#15 , the documentation for glyf table test is not as organized/centralized as the other tables . The bulk of the content of the chm is generated from https://github.com/HinTak/Font-Validator/blob/master/GenerateFValData/OurData.xml, except the glyf table tests.

[graphicore]

BTW, a 4000-line python file should probably be re-factored :-).

It's an ongoing effort.

[felipesanches]

I think that a list of all tests from all tools (native FontBakery, MS FVal, FontForge, etc...) and the equivalences among them would be neat to have if anyone is willing to put the effort into documenting what currently exists...

[davelab6]

@m4rc1e you asked,

I would like the results of [FontVal] to get logged to the cmdline

I believe this is now done. Please confirm and close.

[davelab6]

I feel the new Font Validator covers these checks better

100% of them?

The headless Font Forge is a truly ugly dependency, but, if there is a single thing it checks that nothing else checks, we must keep it.

[felipesanches]

OK. There are a few different issues described here:

• Whether or not to ditch any of the third-party tools (fontforge, MSFVal, etc)
• How these tools overlap with FontBakery tests and among themselves
• The burden of bundling together some of these dependencies

I believe that we can rename this issue so that the goal is to map the equivalency of the test-suites provided by each of the third-party tools and how they relate to the FontBakery native tests. And this can be a long-term issue that may or may not lead to ditching some dependencies in the future.

[davelab6]

This seems related to #1602 and #1515

[felipesanches]

trying to summarize, these are separate issues because:

• This one is about what our checks do versus what other suites of checks do (and if they're not 100% redundant with our current checks).
• Map all FontBakery dependencies and try to minimize the dependency tree #1602 is about reducing the python module dependency footprint.
• Modular approach to subcommand functionality (and associated dependencies) #1515 seems to be either already addressed by the gftools spin-off, or maybe it is about something fancier. Let's see what @chrissimpkins says there.

[felipesanches]

This mapping of third-party features is not doable within this week, so I'll move it to a future milestone.