Skip to content

Commit

Permalink
chore(docs): create architecture page (apache#28481)
Browse files Browse the repository at this point in the history
Co-authored-by: Evan Rusackas <evan@preset.io>
  • Loading branch information
sfirke and rusackas authored Jul 16, 2024
1 parent d51a2ad commit e90a9b3
Show file tree
Hide file tree
Showing 7 changed files with 79 additions and 19 deletions.
20 changes: 6 additions & 14 deletions docs/docs/configuration/configuring-superset.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,9 @@ variables and objects exposed act as a public interface of the bulk of what you
to configure, alter and interface with. In this python module, you'll find all these
parameters, sensible defaults, as well as rich documentation in the form of comments

To configure your application, you need to create you own configuration module, which
To configure your application, you need to create your own configuration module, which
will allow you to override few or many of these parameters. Instead of altering the core module,
You'll want to define your own module (typically a file named `superset_config.py`.
you'll want to define your own module (typically a file named `superset_config.py`).
Add this file to your `PYTHONPATH` or create an environment variable
`SUPERSET_CONFIG_PATH` specifying the full path of the `superset_config.py`.

Expand All @@ -28,15 +28,15 @@ For example, if deploying on Superset directly on a Linux-based system where you
export SUPERSET_CONFIG_PATH=/app/superset_config.py
```

If you are using your own custom Dockerfile with official Superset image as base image,
If you are using your own custom Dockerfile with the official Superset image as base image,
then you can add your overrides as shown below:

```bash
COPY --chown=superset superset_config.py /app/
ENV SUPERSET_CONFIG_PATH /app/superset_config.py
```

Docker compose deployments handle application configuration differently using specific conventions..
Docker compose deployments handle application configuration differently using specific conventions.
Refer to the [docker-compose tips & configuration](/docs/installation/docker-compose#docker-compose-tips--configuration)
for details.

Expand Down Expand Up @@ -87,23 +87,15 @@ can be altered in your local `superset_config.py`. Administrators will want to r
to understand what can be configured locally as well as the default values in place.

Since `superset_config.py` acts as a Flask configuration module, it can be used to alter the
settings Flask itself, as well as Flask extensions that Superset bundles like
settings of Flask itself, as well as Flask extensions that Superset bundles like
`flask-wtf`, `flask-caching`, `flask-migrate`,
and `flask-appbuilder`. Each one of these extensions offers intricate configurability.
Flask App Builder, the web framework used by Superset, also offers many
configuration settings. Please consult the
[Flask App Builder Documentation](https://flask-appbuilder.readthedocs.org/en/latest/config.html)
for more information on how to configure it.

You'll want to change:

- `SECRET_KEY`: to a long random string
- `SQLALCHEMY_DATABASE_URI`: that by default points to sqlite database located at
~/.superset/superset.db

```
WTF_CSRF_EXEMPT_LIST = [‘’]
```
At the very least, you'll want to change `SECRET_KEY` and `SQLALCHEMY_DATABASE_URI`. Continue reading for more about each of these.

## Specifying a SECRET_KEY

Expand Down
68 changes: 68 additions & 0 deletions docs/docs/installation/architecture.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
---
title: Architecture
hide_title: true
sidebar_position: 1
version: 1
---

import useBaseUrl from "@docusaurus/useBaseUrl";

# Architecture

This page is meant to give new administrators an understanding of Superset's components.

## Components

A Superset installation is made up of these components:
1. The Superset application itself
2. A metadata database
3. A caching layer (optional, but necessary for some features)
4. A worker & beat (optional, but necessary for some features)

### Optional components and associated features

The optional components above are necessary to enable these features:
- [Alerts and Reports](/docs/configuration/alerts-reports)
- [Caching](/docs/configuration/cache)
- [Async Queries](/docs/configuration/async-queries-celery/)
- [Dashboard Thumbnails](/docs/configuration/cache/#caching-thumbnails)

If you install with Kubernetes or Docker Compose, all of these components will be created.

However, installing from PyPI only creates the application itself. Users installing from PyPI will need to configure a caching layer, worker, and beat on their own if they wish to enable the above features. Configuration of those components for a PyPI install is not currently covered in this documentation.

Here are further details on each component.

### The Superset Application

This is the core application. Superset operates like this:
- A user visits a chart or dashboard
- That triggers a SQL query to the data warehouse holding the underlying dataset
- The resulting data is served up in a data visualization
- The Superset application is comprised of the Python (Flask) backend application (server), API layer, and the React frontend, built via Webpack, and static assets needed for the application to work

### Metadata Database

This is where chart and dashboard definitions, user information, logs, etc. are stored. Superset is tested to work with PostgreSQL and MySQL databases as the metadata database (not be confused with a data source like your data warehouse, which could be a much greater variety of options like Snowflake, Redshift, etc.).

Some installation methods like our Quickstart and PyPI come configured by default to use a SQLite on-disk database. And in a Docker Compose installation, the data would be stored in a PostgresQL container volume. Neither of these cases are recommended for production instances of Superset.

For production, a properly-configured, managed, standalone database is recommended. No matter what database you use, you should plan to back it up regularly.

### Caching Layer

The caching layer serves two main functions:
- Store the results of queries to your data warehouse so that when a chart is loaded twice, it pulls from the cache the second time, speeding up the application and reducing load on your data warehouse.
- Act as a message broker for the worker, enabling the Alerts & Reports, async queries, and thumbnail caching features.

Most people use Redis for their cache, but Superset supports other options too. See the [cache docs](/docs/configuration/cache/) for more.

### Worker and Beat

This is one or more workers who execute tasks like run async queries or take snapshots of reports and send emails, and a "beat" that acts as the scheduler and tells workers when to perform their tasks. Most installations use Celery for these components.

## Other components

Other components can be incorporated into Superset. The best place to learn about additional configurations is the [Configuration page](/docs/configuration/configuring-superset). For instance, you could set up a load balancer or reverse proxy to implement HTTPS in front of your Superset application, or specify a Mapbox URL to enable geospatial charts, etc.

Superset won't even start without certain configuration settings established, so it's essential to review that page.
2 changes: 1 addition & 1 deletion docs/docs/installation/docker-builds.mdx
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Docker Builds
hide_title: true
sidebar_position: 5
sidebar_position: 6
version: 1
---

Expand Down
2 changes: 1 addition & 1 deletion docs/docs/installation/docker-compose.mdx
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Docker Compose
hide_title: true
sidebar_position: 3
sidebar_position: 4
version: 1
---

Expand Down
2 changes: 1 addition & 1 deletion docs/docs/installation/kubernetes.mdx
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Kubernetes
hide_title: true
sidebar_position: 1
sidebar_position: 2
version: 1
---

Expand Down
2 changes: 1 addition & 1 deletion docs/docs/installation/pypi.mdx
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: PyPI
hide_title: true
sidebar_position: 2
sidebar_position: 3
version: 1
---

Expand Down
2 changes: 1 addition & 1 deletion docs/docs/installation/upgrading-superset.mdx
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Upgrading Superset
hide_title: true
sidebar_position: 4
sidebar_position: 5
version: 1
---

Expand Down

0 comments on commit e90a9b3

Please sign in to comment.