Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: use Scarf Gateway for Superset helm charts/Docker compose downloads #24432

Merged
merged 9 commits into from
Jul 19, 2023
Merged

feat: use Scarf Gateway for Superset helm charts/Docker compose downloads #24432

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
ee9b298
feat: use Scarf Gateway for Superset helm charts/Docker compose downl…
Jun 16, 2023
d0b5f6a
bycatch: ignoring zshrc
rusackas Jul 7, 2023
cfda998
docs updates
rusackas Jul 7, 2023
6c6ad70
helm-docs bump
rusackas Jul 7, 2023
d3fe778
bumping chart version
rusackas Jul 8, 2023
28ce165
Merge branch 'master' into scarf-docker-compose-helm-downloads
rusackas Jul 8, 2023
c029c25
fixing erroneous conflict resolution
rusackas Jul 8, 2023
bde06cc
pt 2 of conflict resolution fix
rusackas Jul 8, 2023
92ab712
fixing busted markdown link
rusackas Jul 10, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docker-compose-non-dev.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
x-superset-image: &superset-image apache/superset:${TAG:-latest-dev}
x-superset-image: &superset-image apachesuperset.docker.scarf.sh/apache/superset:${TAG:-latest-dev}
x-superset-depends-on: &superset-depends-on
- db
- redis
Expand Down
2 changes: 1 addition & 1 deletion docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
x-superset-image: &superset-image apache/superset:${TAG:-latest-dev}
x-superset-image: &superset-image apachesuperset.docker.scarf.sh/apache/superset:${TAG:-latest-dev}
x-superset-user: &superset-user root
x-superset-depends-on: &superset-depends-on
- db
Expand Down
2 changes: 2 additions & 0 deletions docs/.gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,5 @@
npm-debug.log*
yarn-debug.log*
yarn-error.log*

docs/.zshrc
6 changes: 6 additions & 0 deletions docs/docs/frequently-asked-questions.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -282,3 +282,9 @@ guarantees and are not recommended but may fit your use case temporarily:
In the Edit Dataset view, you can specify a time offset. This field lets you configure the
number of hours to be added or subtracted from the time column.
This can be used, for example, to convert UTC time to local time.

### Does Superset collect any telemetry data?

Superset uses [Scarf](https://about.scarf.sh/) by default to collect basic telemetry data upon installing and/or running Superset. This data helps the maintainers of Superset better understand which versions of Superset are being used, in order to prioritize patch/minor releases and security fixes.
We use the [Scarf Gateway](https://docs.scarf.sh/gateway/) to sit in front of container registries, and the [scarf-js](https://about.scarf.sh/package-sdks) package to track `npm` installations.
Scarf purges PII and provides aggregated statistics. Superset users can easily opt out of analytics in various ways documented [here](https://docs.scarf.sh/gateway/#do-not-track) and [here](https://docs.scarf.sh/package-analytics/#as-a-user-of-a-package-using-scarf-js-how-can-i-opt-out-of-analytics). Additional opt-out instructions for Docker users are available on the [Docker Installation](https://superset.apache.org/docs/installation/installing-superset-using-docker-compose) page.
9 changes: 8 additions & 1 deletion docs/docs/installation/installing-superset-using-docker-compose.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,14 @@ You can configure the Docker Compose environment varirables for dev and non-dev

One important variable is `SUPERSET_LOAD_EXAMPLES` which determines whether the `superset_init` container will load example data and visualizations into the database and Superset. These examples are quite helpful for most people, but probably unnecessary for experienced users. The loading process can sometimes take a few minutes and a good amount of CPU, so you may want to disable it on a resource-constrained device.

**Note:** Users often want to connect to other databases from Superset. Currently, the easiest way to do this is to modify the `docker-compose-non-dev.yml` file and add your database as a service that the other services depend on (via `x-superset-depends-on`). Others have attempted to set `network_mode: host` on the Superset services, but these generally break the installation, because the configuration requires use of the Docker Compose DNS resolver for the service names. If you have a good solution for this, let us know!

:::note
Users often want to connect to other databases from Superset. Currently, the easiest way to do this is to modify the `docker-compose-non-dev.yml` file and add your database as a service that the other services depend on (via `x-superset-depends-on`). Others have attempted to set `network_mode: host` on the Superset services, but these generally break the installation, because the configuration requires use of the Docker Compose DNS resolver for the service names. If you have a good solution for this, let us know!
:::

:::note
Superset uses [Scarf Gateway](https://about.scarf.sh/scarf-gateway) to collect telmetry data to better understand and support the need for patch versions of Sueprset. Scarf purges PII and provides aggregated statistics. Superset users can easily opt out of analytics in various ways documented [here](https://docs.scarf.sh/gateway/#do-not-track). However, if you wish to opt-out of this in your Docker-based installation, you can simply edit your `docker-compose.yml` or `docker-compose-non-dev.yml` file and remove `apachesuperset.docker.scarf.sh/` from the `x-superset-image` setting, so that it's simply pulling `apache/superset:${TAG:-latest-dev}`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@arjundevarajan and @rusackas would it be safer to make it opt-out by default? Granted one might not get the same scale of telemetry data, but it feels significantly less intrusive.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @john-bodley mostly chiming in to agree with @rusackas that opting for the non-default option will significantly decrease the amount of useful data that's being collected. It should be reemphasized that all of this data is de-identified no matter what, and that the ASF has approved Scarf as a verified external service provider in the past for other ASF projects (see Privacy Policy here), which have deployed Scarf live to their projects for several years now

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @arjundevarajan for the context.

:::

### 4. Log in to Superset

Expand Down
4 changes: 4 additions & 0 deletions docs/docs/installation/running-on-kubernetes.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,10 @@ init:
. {{ .Values.configMountPath }}/superset_init.sh
```

:::note
Superset uses [Scarf Gateway](https://about.scarf.sh/scarf-gateway) to collect telmetry data to better understand and support the need for patch versions of Sueprset. Scarf purges PII and provides aggregated statistics. Superset users can easily opt out of analytics in various ways documented [here](https://docs.scarf.sh/gateway/#do-not-track). However, if you wish to opt-out of this in your Helm-based installation, you can simply edit your `helm/superset/values.yaml` file and remove `apachesuperset.docker.scarf.sh/` from the `repository` field, so that it's simply pulling `apache/superset`
:::

#### Dependencies

Install additional packages and do any other bootstrap configuration in the bootstrap script.
Expand Down
2 changes: 1 addition & 1 deletion helm/superset/Chart.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ maintainers:
- name: craig-rueda
email: craig@craigrueda.com
url: https://github.com/craig-rueda
version: 0.10.4
version: 0.10.5
dependencies:
- name: postgresql
version: 12.1.6
Expand Down
4 changes: 2 additions & 2 deletions helm/superset/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ NOTE: This file is generated by helm-docs: https://github.com/norwoodj/helm-docs

# superset

![Version: 0.10.4](https://img.shields.io/badge/Version-0.10.4-informational?style=flat-square)
![Version: 0.10.5](https://img.shields.io/badge/Version-0.10.5-informational?style=flat-square)

Apache Superset is a modern, enterprise-ready business intelligence web application

Expand Down Expand Up @@ -70,7 +70,7 @@ helm install my-superset superset/superset
| fullnameOverride | string | `nil` | Provide a name to override the full names of resources |
| hostAliases | list | `[]` | Custom hostAliases for all superset pods # https://kubernetes.io/docs/tasks/network/customize-hosts-file-for-pods/ |
| image.pullPolicy | string | `"IfNotPresent"` | |
| image.repository | string | `"apache/superset"` | |
| image.repository | string | `"apachesuperset.docker.scarf.sh/apache/superset"` | |
| image.tag | string | `""` | |
| imagePullSecrets | list | `[]` | |
| ingress.annotations | object | `{}` | |
Expand Down
2 changes: 1 addition & 1 deletion helm/superset/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -176,7 +176,7 @@ configMountPath: "/app/pythonpath"
extraConfigMountPath: "/app/configs"

image:
repository: apache/superset
repository: apachesuperset.docker.scarf.sh/apache/superset
tag: ""
pullPolicy: IfNotPresent

Expand Down