Merge branch 'master' of https://github.com/preset-io/superset into f…

…ix-change-database

.github/workflows/embedded-sdk-release.yml

@@ -0,0 +1,23 @@
+name: Embedded SDK Release
+
+on:
+  push:
+    branches:
+      - 'master'
+
+jobs:
+  build:
+    runs-on: ubuntu-20.04
+    defaults:
+      run:
+        working-directory: superset-embedded-sdk
+    steps:
+      - uses: actions/checkout@v2
+      - uses: actions/setup-node@v2
+        with:
+          node-version: "16"
+          registry-url: 'https://registry.npmjs.org'
+      - run: npm ci
+      - run: npm run ci:release
+        env:
+          NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

.github/workflows/embedded-sdk-test.yml

@@ -0,0 +1,23 @@
+name: Embedded SDK PR Checks
+
+on:
+  pull_request:
+    paths:
+      - "superset-embedded-sdk/**"
+    types: [synchronize, opened, reopened, ready_for_review]
+
+jobs:
+  embedded-sdk-test:
+    if: github.event.pull_request.draft == false
+    runs-on: ubuntu-20.04
+    defaults:
+      run:
+        working-directory: superset-embedded-sdk
+    steps:
+      - uses: actions/checkout@v2
+      - uses: actions/setup-node@v2
+        with:
+          node-version: "16"
+          registry-url: 'https://registry.npmjs.org'
+      - run: npm ci
+      - run: npm run build

.github/workflows/superset-docs.yml

@@ -23,7 +23,7 @@ jobs:
           submodules: recursive
       - name: yarn install
         run: |
-          yarn install --immutable --immutable-cache --check-cache
+          yarn install --check-cache
       - name: yarn build
         run: |
           yarn build

.gitignore

@@ -89,6 +89,9 @@ scripts/*.zip
 venv
 @eaDir/
 
+# PyCharm
+.run
+
 # Test data
 celery_results.sqlite
 celerybeat-schedule

docs/docs/Contributing/contributing-page.mdx

@@ -1,8 +1,7 @@
 ---
-name: General Resources
-menu: Contributing
-route: /docs/contributing/contribution-guidelines
-index: 1
+title: Contributing to Superset
+hide_title: true
+sidebar_position: 1
 version: 1
 ---
 

docs/docs/Contributing/translations.mdx

@@ -7,14 +7,19 @@ version: 1
 
 ## Translating
 
-We use [Babel](http://babel.pocoo.org/en/latest/) to translate Superset.
-In Python files, we import the magic `_` function using:
+We use [Flask-Babel](https://flask-babel.tkte.ch/) to translate Superset.
+In Python files, we use the following
+[translation functions](https://flask-babel.tkte.ch/#using-translations) from
+`Flask-Babel`:
+- `gettext` and `lazy_gettext` (usually aliased to `_`): for translating singular
+  strings.
+- `ngettext`: for translating strings that might become plural.
 
 ```python
 from flask_babel import lazy_gettext as _
 ```
 
-then wrap our translatable strings with it, e.g. `_('Translate me')`.
+then wrap the translatable strings with it, e.g. `_('Translate me')`.
 During extraction, string literals passed to `_` will be added to the
 generated `.po` file for each language for later translation.
 
@@ -43,29 +48,80 @@ LANGUAGES = {
 }
 ```
 
+### Creating a new language dictionary
+
+First check if the language code for your target language already exists. Check if the
+[two letter ISO 639-1 code](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes)
+for your target language already exists in the `superset/translations` directory:
+
+```bash
+ls superset/translations | grep -E "^[a-z]{2}\/"
+```
+
+If your language already has a pre-existing translation, skip to the next section
+
+The following languages are already supported by Flask AppBuilder, and will make it
+easier to translate the application to your target language:
+[Flask AppBuilder i18n documentation](https://flask-appbuilder.readthedocs.io/en/latest/i18n.html)
+
+To create a dictionary for a new language, first make sure the necessary dependencies are installed:
+```bash
+pip install -r superset/translations/requirements.txt
+```
+
+Then run the following, where `LANGUAGE_CODE` is replaced with the language code for your target
+language:
+
+```bash
+pybabel init -i superset/translations/messages.pot -d superset/translations -l LANGUAGE_CODE
+```
+
+For instance, to add a translation for Finnish (language code `fi`), run the following:
+
+```bash
+pybabel init -i superset/translations/messages.pot -d superset/translations -l fi
+```
+
 ### Extracting new strings for translation
 
+This step needs to be done every time application strings change. This happens fairly
+frequently, so if you want to ensure that your translation has good coverage, this
+step needs to be run fairly frequently and the updated strings merged to the upstream
+codebase via PRs. To update the template file `superset/translations/messages.pot`
+with current application strings, run the following command:
+
 ```bash
 pybabel extract -F superset/translations/babel.cfg -o superset/translations/messages.pot -k _ -k __ -k t -k tn -k tct .
 ```
 
-This will update the template file `superset/translations/messages.pot` with current application strings. Do not forget to update
-this file with the appropriate license information.
+Do not forget to update this file with the appropriate license information.
 
 ### Updating language files
 
+Run the following command to update the language files with the new extracted strings.
+
 ```bash
  pybabel update -i superset/translations/messages.pot -d superset/translations --ignore-obsolete
 ```
 
-This will update language files with the new extracted strings.
-
 You can then translate the strings gathered in files located under
-`superset/translation`, where there's one per language. You can use [Poedit](https://poedit.net/features)
+`superset/translation`, where there's one folder per language. You can use [Poedit](https://poedit.net/features)
 to translate the `po` file more conveniently.
 There are some [tutorials in the wiki](https://wiki.lxde.org/en/Translate_*.po_files_with_Poedit).
 
-In the case of JS translation, we need to convert the PO file into a JSON file, and we need the global download of the npm package po2json.
+To perform the translation on MacOS, you can install `poedit` via Homebrew:
+
+```bash
+brew install poedit
+```
+
+After this, just start the `poedit` application and open the `messages.po` file. In the
+case of the Finnish translation, this would be `superset/translations/fi/LC_MESSAGES/messages.po`.
+
+### Applying translations
+
+To make the translations available on the frontend, we need to convert the PO file into
+a JSON file. To do this, we need to globally install the npm package `po2json`.
 
 ```bash
 npm install -g po2json
@@ -84,20 +140,9 @@ the executable path (e.g. `/usr/local/bin/po2json` instead of `po2json`).
 If you get a lot of `[null,***]` in `messages.json`, just delete all the `null,`.
 For example, `"year":["年"]` is correct while `"year":[null,"年"]`is incorrect.
 
-For the translations to take effect we need to compile translation catalogs into binary MO files.
+Finally, for the translations to take effect we need to compile translation catalogs into
+binary MO files.
 
 ```bash
 pybabel compile -d superset/translations
 ```
-
-### Creating a new language dictionary
-
-To create a dictionary for a new language, run the following, where `LANGUAGE_CODE` is replaced with
-the language code for your target language, e.g. `es` (see [Flask AppBuilder i18n documentation](https://flask-appbuilder.readthedocs.io/en/latest/i18n.html) for more details):
-
-```bash
-pip install -r superset/translations/requirements.txt
-pybabel init -i superset/translations/messages.pot -d superset/translations -l LANGUAGE_CODE
-```
-
-Then, [extract strings for the new language](#extracting-new-strings-for-translation).

docs/docusaurus.config.js

@@ -101,6 +101,10 @@ const config = {
             to: '/docs/installation/alerts-reports',
             from: '/docs/installation/email-reports',
           },
+          {
+            to: '/docs/intro',
+            from: '/docs/roadmap',
+          },
         ],
       },
     ],
@@ -113,7 +117,7 @@ const config = {
       ({
         docs: {
           sidebarPath: require.resolve('./sidebars.js'),
-          editUrl: 'https://github.com/apache/superset/tree/master/docs-v2',
+          editUrl: 'https://github.com/apache/superset/tree/master/docs',
         },
         blog: {
           showReadingTime: true,

superset-embedded-sdk/.gitignore

@@ -0,0 +1,3 @@
+bundle
+dist
+lib

superset-embedded-sdk/CONTRIBUTING.md

@@ -0,0 +1,77 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+  http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Contributing to the Superset Embedded SDK
+
+The superset-embedded-sdk directory is a self contained sub-project in the Superset codebase.
+
+This is because the SDK has different requirements from other parts of the Superset codebase:
+Namely, we need to export a lightweight frontend library that can be used in as many environments as possible.
+Having separate configs allows for better separation of concerns and allows the SDK code to remain simple.
+
+## Testing
+
+The functions used in the sdk so far are very closely tied to browser behavior,
+and therefore are not easily unit-testable. We have instead opted to test the sdk behavior using end-to-end tests.
+This way, the tests can assert that the sdk actually mounts the iframe and communicates with it correctly.
+
+At time of writing, these tests are not written yet, because we haven't yet put together the demo app that they will leverage.
+### Things to e2e test once we have a demo app:
+
+**happy path:**
+
+fetch valid guest token and pass it to the sdk, verify that the dashboard shows up
+
+**security:**
+
+it should fail if you pass a fake guest token
+it should fail if your guest token doesn't have permission to access this resource
+it should apply rls filters correctly
+it should not apply rls filters to a dataset that isn't included
+
+**edge cases:**
+
+what happens if superset is offline
+what happens if the superset domain is invalid or incorrect
+what happens if dashboard id doesn't exist
+
+## Publishing
+
+To publish a new version, first determine whether it will be a major/minor/patch version according to [semver rules](https://semver.org/).
+Run `npm version [major|minor|patch]`, and include the resulting version change in your PR.
+
+Building the package and publishing to npm will be handled by github actions automatically on merge to master,
+provided that the currently specified package version isn't already published.
+
+## Building
+
+Builds are handled by CI, so there is no need to run the build yourself unless you are curious about it.
+
+The library is built in two modes: one for consumption by package managers
+and subsequent build systems, and one for consumption directly by a web browser.
+
+Babel is used to build the sdk into a relatively modern js package in the `lib` directory.
+This is used by consumers who install the embedded sdk via npm, yarn, or other package manager.
+
+Webpack is used to bundle the `bundle` directory,
+for use directly in the browser with no build step e.g. when importing via unpkg.
+
+Typescript outputs type definition files to the `dist` directory.
+
+Which of these outputs is used by the library consumer is determined by our package.json's `main`, `module`, and `types` fields.