Allow installing additional dependencies in Docker deployment

[jezdez]

Issue Summary

To be able to use the Python extension API (and soon the frontend extension API) it would be useful to be able to easily install extra dependencies when using the Redash docker image. I wonder if we could add an optional environment variables, e.g. REDASH_EXTRA_PYTHON_REQUIREMENTS and the analog way for Node dependencies, to be picked up when inheriting from the main Redash Docker image.

It's simple enough to do that right now by simply inheriting from the Redash image and running additional commands, e.g.:

FROM redash/redash:latest

COPY requirements_extra.txt /tmp/
RUN pip install -r /tmp/requirements_extra.txt

If we want to encourage users to use the extension APIs in their projects, a guideline how to integrate it in the Docker based deployment workflow would alleviate the risk of the application bundle build process from breaking.

The above example is simple enough, but when considering that the current Dockerfile installs the Node dependencies only to build the application bundle and then delete the node_modules/ directory again, it's a lot more brittle that I think is worthwhile going forward.

For the sake of consistency between the Python and Node dependencies it would make sense to have a common API to add additional dependencies using the parent Redash Docker image.

Maybe an ideal way would be to use the ON BUILD command to do the installation and build process if certain environment variables are defined? OTOH the *-onbuild images of the various language images have recently been deprecated, so maybe this is betting on the wrong horse.

As mentioned in the ticket, Sentry uses ON BUILD in their "onpremise" Docker image though, which seems to be an accepted use case. The base onbuild image contains a few ON BUILD commands to install additional dependencies if available:

FROM sentry:9.0

# ...

# Hook for installing additional plugins
ONBUILD RUN if [ -s requirements.txt ]; then pip install -r requirements.txt; fi

# Hook for installing a local app as an addon
ONBUILD RUN if [ -s setup.py ]; then pip install -e .; fi

I'm not sure what the best way would be to do the same for NPM though since we'll need to install both Redash's own and custom dependencies at the same time to successfully build the application bundle.

Technical details:

• Redash Version: 5.0beta
• Browser/OS: Firefox/macOS
• How did you install Redash: Docker

[jezdez]

/cc @emtwo This is right along your alley for the frontend extension API..

[arikfr]

If I understand it correctly, what ONBUILD allows is running additional commands with a new context (files) after the original image was built already. It only saves the end user the need to write some commands and adds standardization, but won't change how things are built.

TL;DR it's only useful for installing additional Python packages, but not for NPM/bundling the frontend assets.

I don't mind adding this extensions to make things easier for all involved.

As a side note, on the topic of improving our Dockerfile, I was thinking of switching to a multi stage Docker build to optimize the final size and speed up things (by not having to install Node utilities for example but using a Node Docker image).