Error rendering file using windows executable

[manuvarkey]

OS: Windows 11
WeasyPrint: v62.3

When rendering an html file with or without css using weasyprint.exe, following repeated error messages are displayed and the rendered pdf has no font characters displayed.

.\weasyprint.exe .\report.html out.pdf -s .\report.css -v

INFO: Step 1 - Fetching and parsing HTML - .\report.html
INFO: Step 2 - Fetching and parsing CSS - CSS string
INFO: Step 2 - Fetching and parsing CSS - file:///C:/Users/User/Desktop/Testing/report.css
INFO: Step 3 - Applying CSS
INFO: Step 4 - Creating formatting structure
INFO: Step 5 - Creating layout - Page 1

(process:2544): Pango-CRITICAL **: 11:37:03.157: pango_context_set_font_map: assertion '!font_map || PANGO_IS_FONT_MAP (font_map)' failed

(process:2544): Pango-CRITICAL **: 11:37:03.170: pango_itemize_with_font: assertion 'context->font_map != NULL' failed

(process:2544): Pango-CRITICAL **: 11:37:03.170: pango_context_load_font: assertion 'context->font_map != NULL' failed

(process:2544): Pango-CRITICAL **: 11:37:03.170: pango_context_set_font_map: assertion '!font_map || PANGO_IS_FONT_MAP (font_map)' failed

(process:2544): Pango-CRITICAL **: 11:37:03.170: pango_itemize_with_font: assertion 'context->font_map != NULL' failed

(process:2544): Pango-CRITICAL **: 11:37:03.170: pango_context_load_font: assertion 'context->font_map != NULL' failed

(process:2544): Pango-CRITICAL **: 11:37:03.170: pango_context_set_font_map: assertion '!font_map || PANGO_IS_FONT_MAP (font_map)' failed

...

INFO: Step 6 - Creating PDF
INFO: Step 7 - Adding PDF metadata

report.zip

The html file displays fine in web browser and renders alright while using weasyprint python API under Linux (Ubuntu 24.04 with latest weasyprint)

[liZe]

Hi!

I’ve never seen this error on my Windows VM or on CI. Is that a fresh Windows install on a server, or on a laptop?

Looks like there’s something wrong with fonts. Do you have fonts installed on your Windows system, and can your user reach them?

[manuvarkey]

Hi, I am using WinDev2407Eval on VirtualBox. Other applications are able to use fonts without any issue. For ex: I am able to use fonts in WordPad. Is there any test I can do ?

[liZe]

Hi, I am using WinDev2407Eval on VirtualBox. Other applications are able to use fonts without any issue. For ex: I am able to use fonts in WordPad. Is there any test I can do ?

I’ll try the VM on my computer.

[manuvarkey]

I think I have figured out the cause of the error. I have in my path variable set the path to a build of GTK with Pango build without fontconfig & pangoft2 support. Once this path is removed, the errors are not shown and the pdf is generated fine.

It seems weasyprint.exe is using the external pango libraries when available in path.

[manuvarkey]

I also tested this on a different system with an older build of GTK which had Pango build with support for both libraries. The error is not displayed and pdf is generated fine in this case. But when I delete the dlls for fontconfig and pangoft2, the same error is reproduced.

[liZe]

It seems weasyprint.exe is using the external pango libraries when available in path.

Since Python 3.8, Python doesn’t search libraries in the PATH environment variable. The executable is built with Python 3.12, so PATH shouldn’t change anything. But WeasyPrint looks in the default install folders of GTK and MSYS2, so maybe it could interfere.

Are you sure that removing GTK from PATH is what makes WeasyPrint work on your system?

[manuvarkey]

Are you sure that removing GTK from PATH is what makes WeasyPrint work on your system?

I have the build of Gtk including Pango and fontconfig setup at C:\gtk-build\gtk\x64\release\bin. This location is set in the PATH variable. The error occurs when I delete fonconfig-1.dll.

Once this location is deleted from PATH, the error is resolved and pdf is rendered fine.

[manuvarkey]

Please see screencast.

screencast.zip

[liZe]

Thanks for the screencast.

Your GTK install is not even in the default install path, the problem doesn’t come from this:

WeasyPrint/weasyprint/text/ffi.py

Lines 474 to 481 in f17bd4b

	if hasattr(os, 'add_dll_directory'): # pragma: no cover
	dll_directories = os.getenv(
	'WEASYPRINT_DLL_DIRECTORIES',
	'C:\\Program Files\\GTK3-Runtime Win64\\bin;'
	'C:\\msys64\\mingw64\\bin').split(';')
	for dll_directory in dll_directories:
	with suppress((OSError, FileNotFoundError)):
	os.add_dll_directory(dll_directory)

I just don’t understand what’s going on.

Many users complained that with Python 3.8+, setting the PATH variable wasn’t enough, and the Python documentation explicitly explains that it doesn’t look in PATH anymore.

To be honest, we’ve already spent so many days trying to provide a simple way to get WeasyPrint working on Windows that it’s becoming ridiculous. Nobody knows how DLL loading works on Windows, and we have random different behaviors on systems that are supposed to be the same.

If you (or anyone else) wants to spend some time understanding what exactly happens here, and then provide a PR to fix this, I’ll be happy to merge it. But otherwise, I’ll just close this issue and let users tweak their install paths and environment variables until it works. At least we now know that having GTK installed in a folder set in the PATH can break the rendering, and that removing the path from PATH is a working workaround.

[manuvarkey]

Hi, I did some troubleshooting and I think I have figured out the source of the issue.

Error seems to be related to the order of libraries listed here.

WeasyPrint/weasyprint/text/ffi.py

Lines 483 to 502 in f17bd4b

	gobject = _dlopen(
	ffi, 'gobject-2.0-0', 'gobject-2.0', 'libgobject-2.0-0',
	'libgobject-2.0.so.0', 'libgobject-2.0.dylib', 'libgobject-2.0-0.dll')
	pango = _dlopen(
	ffi, 'pango-1.0-0', 'pango-1.0', 'libpango-1.0-0', 'libpango-1.0.so.0',
	'libpango-1.0.dylib', 'libpango-1.0-0.dll')
	harfbuzz = _dlopen(
	ffi, 'harfbuzz', 'harfbuzz-0.0', 'libharfbuzz-0',
	'libharfbuzz.so.0', 'libharfbuzz.so.0', 'libharfbuzz.0.dylib',
	'libharfbuzz-0.dll')
	harfbuzz_subset = _dlopen(
	ffi, 'harfbuzz-subset', 'harfbuzz-subset-0.0', 'libharfbuzz-subset-0',
	'libharfbuzz-subset.so.0', 'libharfbuzz-subset.so.0', 'libharfbuzz-subset.0.dylib',
	'libharfbuzz-subset-0.dll', allow_fail=True)
	fontconfig = _dlopen(
	ffi, 'fontconfig-1', 'fontconfig', 'libfontconfig', 'libfontconfig.so.1',
	'libfontconfig.1.dylib', 'libfontconfig-1.dll')
	pangoft2 = _dlopen(
	ffi, 'pangoft2-1.0-0', 'pangoft2-1.0', 'libpangoft2-1.0-0',
	'libpangoft2-1.0.so.0', 'libpangoft2-1.0.dylib', 'libpangoft2-1.0-0.dll')

The _dlopen function, searches over the list of names until a match is made.

WeasyPrint/weasyprint/text/ffi.py

Lines 456 to 458 in f17bd4b

	for name in names:
	with suppress(OSError):
	return ffi.dlopen(name)

Since the libraries, copied by PyInstaller are named libgobject-2.0-0.dll, libpango-1.0-0.dll, libharfbuzz-0.dll, libharfbuzz-subset-0.dll, libfontconfig-1.dll and libpangoft2-1.0-0.dll, which are listed last in the list of names, the search for the dll goes past the PyInstaller private folder, folders added through os.add_dll_directory to the directories that are listed in the PATH environment variable. This leads to selection of libraries from GTK installation in the PATH over the one in the private folder copied by PyInstaller from MSYS.

The error could be resolved by changing the order of names for modules to give preference to the libraries packaged by PyInstaller as follows.

gobject = _dlopen(
    ffi, 'libgobject-2.0-0', 'gobject-2.0-0', 'gobject-2.0',
    'libgobject-2.0.so.0', 'libgobject-2.0.dylib')
pango = _dlopen(
    ffi, 'libpango-1.0-0', 'pango-1.0-0', 'pango-1.0', 'libpango-1.0.so.0',
    'libpango-1.0.dylib')
harfbuzz = _dlopen(
    ffi, 'libharfbuzz-0', 'harfbuzz', 'harfbuzz-0.0',
    'libharfbuzz.so.0', 'libharfbuzz.so.0', 'libharfbuzz.0.dylib')
harfbuzz_subset = _dlopen(
    ffi, 'libharfbuzz-subset-0', 'harfbuzz-subset', 'harfbuzz-subset-0.0',
    'libharfbuzz-subset.so.0', 'libharfbuzz-subset.so.0', 'libharfbuzz-subset.0.dylib', allow_fail=True)
fontconfig = _dlopen(
    ffi, 'libfontconfig-1', 'fontconfig-1', 'fontconfig', 'libfontconfig', 'libfontconfig.so.1',
    'libfontconfig.1.dylib')
pangoft2 = _dlopen(
    ffi, 'libpangoft2-1.0-0', 'pangoft2-1.0-0', 'pangoft2-1.0', 'libpangoft2-1.0-0',
    'libpangoft2-1.0.so.0', 'libpangoft2-1.0.dylib')

Reference:
For search order of DLL.
https://learn.microsoft.com/en-us/windows/win32/dlls/dynamic-link-library-search-order#standard-search-order-for-unpackaged-apps
LoadLibraryW used by ffi.dlopen()
https://learn.microsoft.com/en-us/windows/win32/api/libloaderapi/nf-libloaderapi-loadlibraryw

[manuvarkey]

I did a build using PyInstaller following the procedure given in https://github.com/Kozea/WeasyPrint/actions/runs/10339644406/workflow on my system using the modified code. Using the new build, the error is no longer reported.

[liZe]

Thanks A LOT for this information.

I understand why changing the order works, and that we should reference the name as used by MSYS before other names (used by GTK for example).

the search for the dll goes past the PyInstaller private folder, folders added through os.add_dll_directory to the directories that are listed in the PATH environment variable.

That’s the last thing I don’t understand. Windows searches DLL files in PATH, but I’m sure that Python doesn’t. So, why is PATH relevant? Could it be possible that importing Pango with Python requires Windows to find Fontconfig, because it’s a dependency of Pango? And in this case, if we import Fontconfig with Python before importing Pango, does Windows use the Fontconfig already imported by Python instead of trying to find it by itself?

[manuvarkey]

That’s the last thing I don’t understand. Windows searches DLL files in PATH, but I’m sure that Python doesn’t. So, why is PATH relevant? Could it be possible that importing Pango with Python requires Windows to find Fontconfig, because it’s a dependency of Pango? And in this case, if we import Fontconfig with Python before importing Pango, does Windows use the Fontconfig already imported by Python instead of trying to find it by itself?

I think it is to do with the flags passed to ffi.dlopen as mentioned here https://cffi.readthedocs.io/en/latest/cdef.html#ffi-dlopen-loading-libraries-in-abi-mode

New in version 1.17: on Windows, ffi.dlopen(filename, flags=0) now passes the flags to LoadLibraryEx(). Moreover, if you use the default value of 0 but filename contains a slash or backslash character, it will instead use LOAD_LIBRARY_SEARCH_DEFAULT_DIRS | LOAD_LIBRARY_SEARCH_DLL_LOAD_DIR. This ensures that dependent DLLs from the same path are also found. It is what ctypes does too.

Values for the flag are listed here https://learn.microsoft.com/en-us/windows/win32/api/libloaderapi/nf-libloaderapi-loadlibraryexa

For the LOAD_LIBRARY_SEARCH_DEFAULT_DIRS mode, the flag value is 0x00001000

This value is a combination of
LOAD_LIBRARY_SEARCH_APPLICATION_DIR,
LOAD_LIBRARY_SEARCH_SYSTEM32, and
LOAD_LIBRARY_SEARCH_USER_DIRS.
Directories in the standard search path are not searched. This value cannot be combined with LOAD_WITH_ALTERED_SEARCH_PATH.

WeasyPrint/weasyprint/text/ffi.py

Line 458 in f17bd4b

return ffi.dlopen(name)

Modifying the above line as follows to include the flag should limit the dll search to the given library locations.

return ffi.dlopen(name, flags=0x00001000)

[manuvarkey]

Without modifying the library order with the above fix alone, I did a build using PyInstaller and can confirm that the error is not reported even when the GTK path was set in PATH. I was also able to cause the error if I set the GTK build path to WEASYPRINT_DLL_DIRECTORIES. This seems to confirm that the DLL search is not extending to the PATH variable.

PS: The above code modification will have to be limited to Windows.

[liZe]

I think it is to do with the flags passed to ffi.dlopen
New in version 1.17

WeasyPrint 62.3’s executable has been built with cffi 1.16, so that’s probably unrelated to your original problem. Moreover, from what I can understand, the behavior only changed when filenames contain a slash or a backslash, but that’s not our case.

Modifying the above line as follows to include the flag should limit the dll search to the given library locations.

Well, it may work, thanks a lot for the hint, I’ll try. I suppose that it requires cffi 1.17, but that the flags are just discarded for older versions, so we don’t care.

And if anybody reports a problem with that, I’ll burn all the Windows DVDs of the world until Bill Gates comes and finds a solution himself.

[manuvarkey]

WeasyPrint 62.3’s executable has been built with cffi 1.16, so that’s probably unrelated to your original problem. Moreover, from what I can understand, the behavior only changed when filenames contain a slash or a backslash, but that’s not our case.

The dlopen function calls LoadLibraryExA on windows with flag = 0 on cffi v1.17.

https://github.com/python-cffi/cffi/blob/release-1.17/src/c/misc_win32.h#L194-L205

From LoadLibraryExA documentation,

You select these optional behaviors by setting the dwFlags parameter; if dwFlags is zero, LoadLibraryEx behaves identically to LoadLibrary.

For cffi v1.16, the dlopen function calls LoadLibraryA on windows.

https://github.com/python-cffi/cffi/blob/release-1.16/src/c/misc_win32.h#L194-L197

When LoadLibraryA is used, standard search order is used which includes the PATH variable. Please see below.

https://learn.microsoft.com/en-us/windows/win32/dlls/dynamic-link-library-search-order#standard-search-order-for-unpackaged-apps

[liZe]

I’ve open #2238, could you please check that everything’s OK for you?

[manuvarkey]

Changes seem OK to me.

I think it may also be better to change the order of libraries to give preference to the pyinstaller bundled libraries. Especially since following locations are hardcoded, and in future there is no guarantee that they ship compatible builds.

'C:\Program Files\GTK3-Runtime Win64\bin;'
'C:\msys64\mingw64\bin'

[liZe]

Done!

[liZe]

Let’s merge it! Thanks a lot for your time and your advice.