-
Notifications
You must be signed in to change notification settings - Fork 38
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cloudflare: skip running the worker if >1Mb #218
Conversation
Don't execute our worker if the file size is bigger than 1Mb. This is to avoid a "Parser error: The memory limit has been exceeded." for now, but we should be able to find a better solution that also works for these big files. readthedocs/readthedocs-ops#1513
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems worth a try -- wish we could just run the worker without pulling the whole file into memory, but I guess if we're doing string compares on it, we need it in memory. I wonder if we can just do a head
or similar on it?
We could probably do a HEAD request instead of a GET, but that means will be doing two request per file always just to protect us for this edge case. However, we will still need to do the real GET request to be able to return it back to the client. I think we won't be earning anything by doing a HEAD first, actually. Honestly, it doesn't make sense to me that parsing a 4Mb HTML consumes more than 128Mb, but 🤷🏼 -- in particular, if we are working only over the |
I just deploy this as a test but it didn't work because it seems the NGINX from production doesn't return |
It seems that Yeap, immediately I enabled |
Good news... I was able to replicate the error locally by using the repository with the big file:
This will help us to debug this problem more in deep, at least. |
Meh, even calling just
|
With the latest version of wrangler, |
I filled a ticket on Cloudflare to ask them if this is the expected behavior. |
Closing for now as we don't have a good way to solve this issue yet. We will continue the conversation in the issue. |
Don't execute our worker if the file size is bigger than 1Mb. This is to avoid a "Parser error: The memory limit has been exceeded." for now, but we should be able to find a better solution that also works for these big files.
Closes https://github.com/readthedocs/readthedocs-ops/issues/1513