Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

docs(drivers): refresh guide on adding a db driver in docker #26038

Merged
merged 4 commits into from
Nov 29, 2023
Merged

docs(drivers): refresh guide on adding a db driver in docker #26038

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
5df5668
refresh adding a db driver in docker
sfirke Nov 20, 2023
23e13b7
tiny wording tweaks
sfirke Nov 20, 2023
24808b4
fix linting
sfirke Nov 20, 2023
e68dd07
undo manual line breaks
sfirke Nov 22, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 21 additions & 51 deletions docs/docs/databases/docker-add-drivers.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,87 +7,57 @@ version: 1

## Adding New Database Drivers in Docker

Superset requires a Python database driver to be installed for each additional type of database you
want to connect to. When setting up Superset locally via `docker compose`, the drivers and packages
contained in
[requirements.txt](https://github.com/apache/superset/blob/master/requirements.txt) and
[requirements-dev.txt](https://github.com/apache/superset/blob/master/requirements-dev.txt)
will be installed automatically.
Superset requires a Python database driver to be installed for each additional type of database you want to connect to.

In this section, we'll walk through how to install the MySQL connector library. The connector
library installation process is the same for all additional libraries and we'll end this section
with the recommended connector library for each database.
In this example, we'll walk through how to install the MySQL connector library. The connector library installation process is the same for all additional libraries.

### 1. Determine the driver you need

To figure out how to install the [database driver](/docs/databases/installing-database-drivers) of your choice.
Consult the [list of database drivers](/docs/databases/installing-database-drivers) and find the PyPI package needed to connect to your database. In this example, we're connecting to a MySQL database, so we'll need the `mysqlclient` connector library.

In the example, we'll walk through the process of installing a MySQL driver in Superset.
### 2. Install the driver in the container

### 2. Install MySQL Driver
We need to get the `mysqlclient` library installed into the Superset docker container (it doesn't matter if it's installed on the host machine). We could enter the running container with `docker exec -it <container_name> bash` and run `pip install mysqlclient` there, but that wouldn't persist permanently.

As we are currently running inside of a Docker container via `docker compose`, we cannot simply run
`pip install mysqlclient` on our local shell and expect the drivers to be installed within the
Docker containers for superset.
To address this, the Superset `docker compose` deployment uses the convention of a `requirements-local.txt` file. All packages listed in this file will be installed into the container from PyPI at runtime. This file will be ignored by Git for the purposes of local development.

In order to address this, the Superset `docker compose` setup comes with a mechanism for you to
install packages locally, which will be ignored by Git for the purposes of local development. Please
follow these steps:

Create `requirements-local.txt`
Create the file `requirements-local.txt` in a subdirectory called `docker` that exists in the directory with your `docker-compose.yml` or `docker-compose-non-dev.yml` file.

```
# From the repo root...
# Run from the repo root:
touch ./docker/requirements-local.txt
```

Add the driver selected in step above:
Add the driver identified in step above. You can use a text editor or do it from the command line like:

```
echo "mysqlclient" >> ./docker/requirements-local.txt
```

Rebuild your local image with the new driver baked in:
**If you are running a stock (non-customized) Superset image**, you are done. Launch Superset with `docker compose -f docker-compose-non-dev.yml up` and the driver should be present.

```
docker compose build --force-rm
```
You can check its presence by entering the running container with `docker exec -it <container_name> bash` and running `pip freeze`. The PyPI package should be present in the printed list.

After the rebuild of the Docker images is complete (which may take a few minutes) you can relaunch using the following command:
**If you're running a customized docker image**, rebuild your local image with the new driver baked in:

```
docker compose up
docker compose build --force-rm
sfirke marked this conversation as resolved.
Show resolved Hide resolved
```

The other option is to start Superset via Docker Compose is using the recipe in `docker-compose-non-dev.yml`, which will use pre-built frontend assets and skip the building of front-end assets:

```
docker compose -f docker-compose-non-dev.yml pull
docker compose -f docker-compose-non-dev.yml up
```
After the rebuild of the Docker images is complete, relaunch Superset by running `docker compose up`.

### 3. Connect to MySQL

Now that you've got a MySQL driver installed locally, you should be able to test it out.

We can now create a Datasource in Superset that can be used to connect to a MySQL instance. Assuming
your MySQL instance is running locally and can be accessed via localhost, use the following
connection string in “SQL Alchemy URI”, by going to Sources > Databases > + icon (to add a new
datasource) in Superset.
Now that you've got a MySQL driver installed in your container, you should be able to connect to your database via the Superset web UI.

For Docker running in Linux:
As an admin user, go to Settings -> Data: Database Connections and click the +DATABASE button. From there, follow the steps on the [Using Database Connection UI page](https://superset.apache.org/docs/databases/db-connection-ui).

```
mysql://mysqluser:mysqluserpassword@localhost/example?charset=utf8
```
Consult the page for your specific database type in the Superset documentation to determine the connection string and any other parameters you need to input. For instance, on the [MySQL page](https://superset.apache.org/docs/databases/mysql), we see that the connection string to a local MySQL database differs depending on whether the setup is running on Linux or Mac.

For Docker running in OSX:
Click the “Test Connection” button, which should result in a popup message saying, "Connection looks good!".

```
mysql://mysqluser:mysqluserpassword@docker.for.mac.host.internal/example?charset=utf8
```
### 4. Troubleshooting

Then click “Test Connection”, which should give you an “OK” message. If not, please look at your
terminal for error messages, and reach out for help.
If the test fails, review your docker logs for error messages. Superset uses SQLAlchemy to connect to databases; to troubleshoot the connection string for your database, you might start Python in the Superset application container or host environment and try to connect directly to the desired database and fetch data. This eliminates Superset for the purposes of isolating the problem.

You can repeat this process for every database you want superset to be able to connect to.
Repeat this process for each different type of database you want Superset to be able to connect to.
Loading