Allow easier method of extending GUNICORN's worker timeout for Docker images.

[michaelsolla]

This is something that was an issue for my team since we use docker images to run ReDash and we deal with pretty large datasets at times. After some research it turned out the timeout was the issue of why some of our larger queries where failing. I set it to 30 seconds which is the normal default but for our needs I set it to 5 minutes via the environment variable. Hopefully this will get posted to help those that need an easy way to increase the timeout for those large result sets (think 200k rows or more).

Let me know if any questions, thank you!

[vercel]

This pull request is automatically deployed with Now.

To access deployments, click Details below or on the icon next to each push.

[jezdez]

Before we add env vars for every Gunicorn configuration value, I'd like to point at the ability to specify a GUNICORN_CMD_ARGS env var to define additional command line parameters (since version 19.7).

So in your case you could specify GUNICORN_CMD_ARGS="--timeout=150" in your custom Dockerfile, docker-compose.yml or whatever your environment allows. I'm wondering if we need to do anything else to allow passing the variable into the Redash container. @michaelsolla Could you try it out?

[michaelsolla]

@jezdez Sure thing, will try it out soon (within a week or two since will need time to get this rolling on our test servers) and update here what I find. The tricky part is even if this works we may need to update the environment list of variables since currently I don't see it listed there and would be a handy place for anyone else in the future:

https://redash.io/help/open-source/admin-guide/env-vars-settings

We use AWS ECS to spin these up using docker images from the official repo. While creating the slightly custom one with settings does get us through I would love to keep us on the official docker images and just config using environment variables I can pass in via the task definitions.

Thanks!

[jezdez]

IIUC it would suffice to add GUNICORN_CMD_ARGS as an empty env var to the Dockerfile and it would be able to be overwritten on ECS when using the official image. That would make the change to Redash small and solve this for all kinds of gunicorn config problems. Looking forward to your report back!

[michaelsolla]

Hello @jezdez ,

So looks like what you suggested worked out just the same, which is great because I would like to keep us as close to the tested official images than custom make them here, unless we one day have to and to which I would rather try to share those changes here to we don't diverge too much.

I will say though that while the variable I am using does not seem needed at this time it would be useful to have this reference documented in that guide I linked earlier. That way future people who run into this issue (which I can tell you is almost a sure thing if Redash is to be stood up for a company with people wanting result sets at times in the 100s of thousands of rows) this know they can access GUNICORN settings and string them into that field. Any way we can just update the admin guide for environment settings and call it a day for this issue / PR?

[arikfr]

Thanks for mentioning GUNICORN_CMD_ARGS, @jezdez.

I'm not sure any changes to the Docker image are needed to use GUNICORN_CMD_ARGS. If I'm not mistaken, if your environment (like env variables settings in task definition on AWS ECS) defines a variable it will be picked up by the processes.

But if a change like this is needed, you're welcome to update the pull request.

Also you're more than welcome to suggest edits to the knowledge base page, its source is here: https://github.com/getredash/website/blob/master/website/_kb/open-source/admin-guide/env-vars-settings.md.

Thanks!

[arikfr]

Closing this as there was no recent activity. @michaelsolla If a change is indeed needed to support GUNICORN_CMD_ARGS then please reopen.

Thanks.