⚠️ Social cards broken – update to 9.5.17

[aronmolnar]

Note

Edit by @squidfunk: Please update to the latest version before raising another issue or opening a discussion. The problem has been fixed and released as part of 9.5.17, and is reported to fix the BadZipFile error.

---

Context

The social plugin tries to fetch fonts from Google and to unzip them:
https://github.com/squidfunk/mkdocs-material-insiders/blob/4ff6a572151f684fe2e21527582e1096a88805d0/src/plugins/social/plugin.py#L857

Bug description

Google no longer provides direct access to the zip file, so building sites fails.

  File "/usr/local/lib/python3.12/site-packages/material/plugins/social/plugin.py", line 522, in _render_typography
    path = self._resolve_font(family, style, variant)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/material/plugins/social/plugin.py", line 809, in _resolve_font
    self._fetch_font_from_google_fonts(family)
  File "/usr/local/lib/python3.12/site-packages/material/plugins/social/plugin.py", line 876, in _fetch_font_from_google_fonts
    archive = ZipFile(f)
              ^^^^^^^^^^
  File "/usr/local/lib/python3.12/zipfile/__init__.py", line 1341, in __init__
    self._RealGetContents()
  File "/usr/local/lib/python3.12/zipfile/__init__.py", line 1408, in _RealGetContents
    raise BadZipFile("File is not a zip file")
zipfile.BadZipFile: File is not a zip file

Workaround was disabling the social plugin.

Related links

• Reporting a bug

Reproduction

Using the code referenced above shows that the link provides HTML data.

Steps to reproduce

import requests
family= "Noto Sans"
url = f"https://fonts.google.com/download?family={family}"
res = requests.get(url, stream = True)
for chunk in res.iter_content(chunk_size = 8192):
    print(chunk)

<script data-id="_gd" nonce="lai0FBQz_50-Cv_k8L_woA">window.WIZ_global_data
...snip...

Browser

No response

Before submitting

• I have read and followed the bug reporting guidelines.
• I have attached links to the documentation, and possibly related issues and discussions.
• I assure that I have removed all customizations before submitting this bug report.
• I have attached a .zip file with a minimal reproduction using the built-in info plugin.

[alexvoss]

This has also been observed by others: #6982 (comment)

[kcgthb]

Confirming the issue. It's also being reported upstream: google/fonts#7481

[alexvoss]

Assuming that Google don't reverse their change, what would be a sensible solution? I could imagine downloading the needed fonts manually and pointing the social plugin at them. That way I could also use fonts that are not on GF. If they did reverse it we'd still win that.

[rathboma]

I'm also having this issue building my docs - https://github.com/beekeeper-studio/beekeeper-studio/actions/runs/8469768189

This only started when I upgraded to 9.5.15, but seems like that is unrelated?

[alexvoss]

Yes, this is a change on Google's side that broke the automated font downloading.

[alexvoss]

@kamilkrzyskow should one of us do a PR that introduces a "I have the fonts locally" option? Should not be too difficult to bypass the downloading and unzipping from Google but it does not make sense for us to do this in parallel. Squidfunk can then do what he likes with it - incorporate it, provide an alternative, fix it, whatever. Seems important to fix this asap as it will affect a lot of people?

[peterschmidt85]

Any workaround folks? I can't deploy a new release.

[alexvoss]

For right now the only thing I can suggest is to turn off the social plugin. There should be a fix tomorrow, I think. IMHO downloading the fonts from Google Fonts was a convenience function that now no longer works. I will start looking at a PR to fix this issue with a local font option. That leaves open the possibility of automating it should that become feasible again but gives us a plan B.

[discdiver]

Affecting us also.

[alexvoss]

Ok, here is a workaround of sorts in the hope that this helps some people in the short term. Edit: this is for the Insider Edition, instructions for the public version see post #6983 (comment) by @jbott below.

Download the font(s) you need from Google Fonts manually and install them in the .cache directory following this recipe, adjusting it as necessary:

export FONT_FAMILY=Inter
mkdir -p .cache/plugin/social/fonts/$FONT_FAMILY
cd .cache/plugin/social/fonts/$FONT_FAMILY
unzip ~/Downloads/$FONT_FAMILY.zip .
mv static/*.ttf . 
mv static/*.otf .

Now, rename the fonts to have only the variant in the name, i.e., get rid of the family name in it. That should leave them in the cache in the way the plugin expects them and it will skip the trip to Google Fonts.

Would be good if someone could try this and give feedback in case I made a mistake. I constructed the cache layout from what I got after running the code in my PR, which I am about to send off to @squidfunk. Might take a few hours before he sees it, please bear with us.

Edit: simplified by unpacking from Downloads directory

Also, this is what the result looks like for the Inter font family:

$ ls -l .cache/plugin/social/fonts/Inter          
total 5544
-rw-rw-r--@ 1 avoss  staff  316848 Sep 15  2023 Black.ttf
-rw-rw-r--@ 1 avoss  staff  316584 Sep 15  2023 Bold.ttf
-rw-rw-r--@ 1 avoss  staff  317184 Sep 15  2023 ExtraBold.ttf
-rw-rw-r--@ 1 avoss  staff  311232 Sep 15  2023 ExtraLight.ttf
-rw-rw-r--@ 1 avoss  staff  310832 Sep 15  2023 Light.ttf
-rw-rw-r--@ 1 avoss  staff  315132 Sep 15  2023 Medium.ttf
-rw-rw-r--@ 1 avoss  staff  310252 Sep 15  2023 Regular.ttf
-rw-rw-r--@ 1 avoss  staff  316220 Sep 15  2023 SemiBold.ttf
-rw-rw-r--@ 1 avoss  staff  310984 Sep 15  2023 Thin.ttf

[alexvoss]

Just a note that the PR I am working on will not require manual manipulations of the fonts beyond the downloading and putting the .zip in a directory somewhere that can be configured with a fonts_dir option.

Edit: PR has just gone in. Hope it is of some use and we'll see a fix released soon. Might take a little because of timezones.

[kcgthb]

The issue reported to the Google Fonts repo has been closed as "completed" with the following comment:

fonts.google.com/download (and all other endpoints under fonts.google.com/*) are internal and not meant for external use as they may change at any time without warning. If you need access to font metadata and the font binaries you can instead use the public developer api: https://developers.google.com/fonts/docs/developer_api

[alexvoss]

Does anyone know if there are any rate limits on the use of the API? That is one possible problem that I could see. It would be super important that caching is used to prevent fonts being downloaded again and again. The example Github Actions workflow provided in the documentation contains settings to prevent this.

Edit: ok, so the API key needed to access the API is a secret? That ain't gonna work. Hope I am holding the wrong end of the stick there.

[squidfunk]

Wow, what just happened while I was sleeping? I'm currently on vacation, and albeit I try to stay on top of things, answer questions, fix bugs, and be responsive nonetheless, I will likely not be able to resolve this in the next few days, as we need to think carefully here before we decide on what to do, as it is a very, very fundamental change impacting DX big time. I'm confident that we can find other ways to cope with this while I'm gone, maybe even work towards a solution.

Context

Before we started using the downloads endpoint, we fetched the font files from Google Fonts' CSS files, but that led to some problems where characters were not available, as those files are heavily optimized. I can't really find the commit now, but a user suggested that we could switch to the downloads endpoint for a better experience, and it served us well for years.

However, I wasn't aware that it was internal. Google switching this endpoint off is inconvenient for us, to put it mildly, but I think we can't really complain if we have used an unofficial API until now. Thus, we need to look for other solutions.

Directions

We cannot instruct the social plugin to use the developer API, as it requires an API key. This means that every user of Material for MkDocs that wants to use the social plugin would need to register for a Google API key prior to using it. It would also mean this key would need to be passed along as a secret everywhere the project is built, which for Open Source projects would mean that contributors would not be able to build your documentation, since the API key must be kept secret and considered sensitive, or it would defeat its purpose. Thus, this is a dead end from my perspective.

Before we decide how we proceed, we should evaluate multiple possibilites. A first PR by @alexvoss is up that might help to remedy the situation, so you can help us collect feedback on this PR. We're open to other suggestions, including entirely different font catalogs (i.e. switching away from Google Fonts), etc., but we can't just bandaid this, since everybody using the social plugin is impacted.

Mitigation

The only save mitigation that does not rely on caching is to disable the social plugin. If you're using Insiders, you could theoretically just download the font via Google Fonts web interface and unpack it in .cache/plugins/social/fonts. This does not work for the community edition, since the Insiders edition is a more elaborate rewrite. We can match folder locations if that helps, though.

I'll try to find some time here and there during my vacation and be as helpful as possible, but please understand that right now, I might be a little slow to respond.

[capcom6]

Why not use fonts directly from https://github.com/google/fonts ?

The main difference is that this repository contains multi-variant fonts in single files, while fonts.google.com provided archives of single-variant files.

[janaka]

I'm facing this also with 9.5.17 in @docqai/docq docs for whatever reason. Local build works in CI it doesn't.

Just a +1 report for now but maybe the above action log helps.

[kamilkrzyskow]

@janaka I never worked with poetry too deeply, so I'm not sure if there are any missing logs etc., but the failed CI says it installed version 9.5.13:
https://github.com/docqai/docq/actions/runs/8590257742/job/23537555949#step:6:296

And in the other working CI where you disabled the social plugin it says it installed 9.5.17:
https://github.com/docqai/docq/actions/runs/8590353932/job/23537780963#step:6:296

Quite bizarre, perhaps the issue lies in the 2 different version definitions?
https://github.com/docqai/docq/blob/8c88f4c7019654167d6d1075414ee79fdf393cfe/pyproject.toml#L51
https://github.com/docqai/docq/blob/8c88f4c7019654167d6d1075414ee79fdf393cfe/pyproject.toml#L77

pip with requirements.txt is so much simpler 😸
perhaps you could clear the caches too 🤔

[vedranmiletic]

Just noticed today that my naive requirements.txt without version pins:

mkdocs[i18n]
mkdocs-material[recommended, imaging]

results in Material for MkDocs version 9.5.2 getting installed. This could be related to MkDocs 1.6.0 now available. Anyhow, I pinned Material to >=9.5.17 and it works fine, but reporting here just in case other use unpinned requirements and stumble upon this issue.

Edit: FWIW, specifying only mkdocs-material[recommended, imaging] also works.

[squidfunk]

We don't yet support MkDocs 1.6, because MkDocs has had a history of breaking changes in minor releases, which is why we've limited to <1.6 for now. We'll try to upgrade to 1.6 quickly, but I currently need to finish other things for Material for MkDocs, so I'm happy to collaborate if somebody wants to go ahead and do the legwork ☺️

Edit: This is now tracked in #7076.

[komradski]

Is it possible this fix hasn't been merged into the Insiders version?

I figured I'd ask before writing up a New Issue. Thanks!

[squidfunk]

The fix was merged into Insiders – everything is up to date. We're successfully building our own documentation with Insiders. Maybe you can try to remove the .cache directory, albeit that shouldn't be necessary.