OSError: Too many open files

[sander76]

I am creating pdf's from a (static) site which is built every night.
But after some time weasyprints ends with the above OSError.
I have been trying to simplify the issue and have come up with the following python script:
(Running Python 3.5.2, Ubuntu 16.04.1 LTS (GNU/Linux 4.4.0-51-generic x86_64))

from weasyprint import HTML
import logging

logging.basicConfig(level=logging.DEBUG)

URL = "http://tools.hde.nl/menc/site/guides/Pliss%C3%A9%20%26%20Duette%C2%AE%20Bottom-Up%20programming/"
OUT = "/home/admin-s/test.pdf"

for lp in range(0, 300):
    try:
        HTML(URL).write_pdf(OUT)
    except OSError as e:
        logging.exception("**************** ERROR AT ATTEMPT: {} *******************".format(lp))
        break

In my case the error occurs after approx 60 loops (varies per situation).
Stacktrace:

ERROR:root:**************** ERROR AT ATTEMPT: 63 *******************
Traceback (most recent call last):
  File "tester.py", line 11, in <module>
  File "/home/admin-s/envs/weasy/lib/python3.5/site-packages/weasyprint/__init__.py", line 178, in write_pdf
  File "/home/admin-s/envs/weasy/lib/python3.5/site-packages/weasyprint/__init__.py", line 143, in render
  File "/home/admin-s/envs/weasy/lib/python3.5/site-packages/weasyprint/document.py", line 325, in _render
  File "/home/admin-s/envs/weasy/lib/python3.5/site-packages/weasyprint/css/__init__.py", line 712, in get_all_computed_styles
  File "/home/admin-s/envs/weasy/lib/python3.5/site-packages/weasyprint/css/__init__.py", line 162, in find_stylesheets
  File "/home/admin-s/envs/weasy/lib/python3.5/site-packages/weasyprint/__init__.py", line 261, in __init__
  File "/home/admin-s/envs/weasy/lib/python3.5/site-packages/weasyprint/css/__init__.py", line 692, in preprocess_stylesheet
  File "/home/admin-s/envs/weasy/lib/python3.5/site-packages/weasyprint/fonts.py", line 229, in add_font_face
OSError: [Errno 24] Too many open files: '/tmp/tmp1a03t715'

[sander76]

Doing a little digging around I found out the following:

ulimit -n shows the amount of allowed open files. It was set at 1024 which was pretty low compared to a docker instance I tried (where the issue was not occurring).

I edited /etc/security/limits.conf by adding

*               soft    nofile          65535
*               hard    nofile          65535

Now ulimit -n shows the number above and the issue like described in the first post is gone.
Don't know though whether this is a workaround for an actual weasyprint issue. But for now it works for me !

[liZe]

There's a real problem about this, temporary files used for fonts management are deleted but still open by the python process. That's why the 5th item in #376 is needed for example.

[sander76]

@liZe Alright. But the 5th item is windows related if I understand correctly. My issue is all linux...

[liZe]

Yes, the file is open on both Windows and Linux, but Windows doesn't accept deleting open files when Linux does. The file is marked as open+deleted on Linux, when it should be closed and deleted afterwards.

[sander76]

I know it is asking for the obvious, but shouldn't you drop windows support? Having tried *many* times, on many machines and after a lot of hair pulling Weasy print on windows doesn't work for me. I really wonder if anyone else has succeeded...

…

On Wed, 7 Dec 2016, 09:43 Guillaume Ayoub, ***@***.***> wrote: Yes, the file is open on both Windows and Linux, but Windows doesn't accept deleting open files when Linux does. The file is marked as open+deleted on Linux, when it should be closed and deleted afterwards. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#396 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABnEiE9Dr6kEUppLgLYIO-3SG5UCRIeZks5rFnHFgaJpZM4LCbOy> .

[liZe]

Obvious fact: this bug is not related at all to the Windows support 😄.

I really wonder if
anyone else has succeeded...

At least I did, and it works well.

There are lots of bugs related to the Windows support in the bug tracker, so I assume that it's working for some people too. There's almost no Windows-specific code in WeasyPrint, the only hard part is the installation. The documentation about that has been improved during the last months, you should give another try!

[liZe]

It's now fixed, thanks!

[sander76]

Greatn

[sander76]

Great!