Cloudflare: skip running the worker if >1Mb

[humitos]

Don't execute our worker if the file size is bigger than 1Mb. This is to avoid a "Parser error: The memory limit has been exceeded." for now, but we should be able to find a better solution that also works for these big files.

Closes https://github.com/readthedocs/readthedocs-ops/issues/1513

[ericholscher]

Seems worth a try -- wish we could just run the worker without pulling the whole file into memory, but I guess if we're doing string compares on it, we need it in memory. I wonder if we can just do a head or similar on it?

[humitos]

We could probably do a HEAD request instead of a GET, but that means will be doing two request per file always just to protect us for this edge case. However, we will still need to do the real GET request to be able to return it back to the client. I think we won't be earning anything by doing a HEAD first, actually.

Honestly, it doesn't make sense to me that parsing a 4Mb HTML consumes more than 128Mb, but 🤷🏼 -- in particular, if we are working only over the head tag. Maybe we can tune a little more the CSS selector to be based on head tag like head > script[...] instead of just script[...] but I'm not sure if that will help.

[humitos]

I just deploy this as a test but it didn't work because it seems the NGINX from production doesn't return Content-Length. The version we use in our local development environment, does 😞

[humitos]

It seems that Content-Length is not compatible with gzip since it uses chunked in that case. So, I think we will need a different approach here.

Yeap, immediately I enabled gzip on locally, it stops sending the Content-Length...

[humitos]

Good news... I was able to replicate the error locally by using the repository with the big file:

ContentType: text/html; charset=utf-8
X-RTD-Force-Addons: true
X-RTD-Hosting-Integrations: false
addPreloads
addProjectVersionSlug. projectSlug=jgdtrans-py versionSlug=dev-doc resolverFilename=/docs/sample.html

wrangler_1     | ✘ [ERROR] workerd/server/server.c++:2838: error: Uncaught exception: workerd/api/html-rewriter.c++:95: failed: remote.jsg.TypeError: Parser error: The memory limit has been exceeded.
wrangler_1     | 
wrangler_1     |   stack: /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-64/bin/workerd@20a4618 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-64/bin/workerd@20a09cb /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-64/bin/workerd@1f44225 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-64/bin/workerd@209f370 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-64/bin/workerd@1f3d350 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-64/bin/workerd@1f3e0d5 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-64/bin/workerd@1deebf0 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-64/bin/workerd@1def160 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-64/bin/workerd@1df0040 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-64/bin/workerd@1df0670 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-64/bin/workerd@1e334b0 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-64/bin/workerd@18a3f70 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-64/bin/workerd@28b295b /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-64/bin/workerd@28b1200 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-64/bin/workerd@189d530 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-64/bin/workerd@1f3d610 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-64/bin/workerd@1f3da44 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-64/bin/workerd@1df0670 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-64/bin/workerd@18bf78b /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-64/bin/workerd@28c4d8c /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-64/bin/workerd@28c5b93 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-64/bin/workerd@28c6194 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-64/bin/workerd@288b0b0
wrangler_1     | 
wrangler_1     | 
nginx_1        | 2024/07/10 13:45:42 [warn] 35#35: *318 an upstream response is buffered to a temporary file /var/cache/nginx/proxy_temp/9/00/0000000009 while reading upstream, client: 10.10.0.5, server: proxito, request: "GET /en/dev-doc/_static/vendor/fontawesome/6.5.2/js/all.min.js?digest=dfe6caa3a7d634c4db9b HTTP/1.1", upstream: "http://10.10.0.3:9000/media/html/jgdtrans-py/d
wrangler_1     | ✘ [ERROR] Uncaught (in response) TypeError: Parser error: The memory limit has been exceeded.
wrangler_1     | 
wrangler_1     | 
wrangler_1     | ✘ [ERROR] Uncaught (async) TypeError: Parser error: The memory limit has been exceeded.
wrangler_1     |

This will help us to debug this problem more in deep, at least.

[humitos]

Meh, even calling just HTMLReWrite().transform(originalResponse) produces an exception for memory limit. I don't think there is too much we can do here since we are not applying any transformation to the original response. I'm thinking this is probably a bug on CF at this point 🤷🏼

wrangler_1     | ✘ [ERROR] workerd/server/server.c++:2838: error: Uncaught exception: workerd/api/html-rewriter.c++:95: failed: remote.jsg.TypeError: Parser error: The memory limit has been exceeded.

[humitos]

With the latest version of wrangler, 3.63.2 I was able to reproduce the error in the same way than in production: blank page. With the version we are currently using, it returns the page and sometimes it also adds the addons. So, the old version works better than the new one 🙃

[humitos]

I filled a ticket on Cloudflare to ask them if this is the expected behavior.

[humitos]

Closing for now as we don't have a good way to solve this issue yet. We will continue the conversation in the issue.