Merge branch 'pexdax/db-connection-ui' of https://github.com/apache/s…

…uperset into ch16648_headerSteps

docs/installation.rst

@@ -433,7 +433,7 @@ For other strategies, check the `superset/tasks/cache.py` file.
 Caching Thumbnails
 ------------------
 
-This is an optional feature that can be turned on by activating it's feature flag on config:
+This is an optional feature that can be turned on by activating its feature flag on config:
 
 .. code-block:: python
 
@@ -972,7 +972,7 @@ environment variable: ::
 Event Logging
 -------------
 
-Superset by default logs special action event on it's database. These log can be accessed on the UI navigating to
+Superset by default logs special action event on its database. These logs can be accessed on the UI navigating to
 "Security" -> "Action Log". You can freely customize these logs by implementing your own event log class.
 
 Example of a simple JSON to Stdout class::
@@ -1358,7 +1358,7 @@ The available validators and names can be found in `sql_validators/`.
 **Scheduling queries**
 
 You can optionally allow your users to schedule queries directly in SQL Lab.
-This is done by addding extra metadata to saved queries, which are then picked
+This is done by adding extra metadata to saved queries, which are then picked
 up by an external scheduled (like [Apache Airflow](https://airflow.apache.org/)).
 
 To allow scheduled queries, add the following to your `config.py`:

docs/src/pages/docs/installation/configuring.mdx

@@ -113,6 +113,8 @@ RequestHeader set X-Forwarded-Proto "https"
 Beyond FAB supported providers (Github, Twitter, LinkedIn, Google, Azure, etc), its easy to connect
 Superset with other OAuth2 Authorization Server implementations that support “code” authorization.
 
+Make sure the pip package [`Authlib`](https://authlib.org/) is installed on the webserver.
+
 First, configure authorization in Superset `superset_config.py`.
 
 ```python
@@ -134,7 +136,7 @@ OAUTH_PROVIDERS = [
             'access_token_headers':{    # Additional headers for calls to access_token_url
                 'Authorization': 'Basic Base64EncodedClientIdAndSecret'
             },
-            'base_url':'https://myAuthorizationServer/oauth2AuthorizationServer/',
+            'api_base_url':'https://myAuthorizationServer/oauth2AuthorizationServer/',
             'access_token_url':'https://myAuthorizationServer/oauth2AuthorizationServer/token',
             'authorize_url':'https://myAuthorizationServer/oauth2AuthorizationServer/authorize'
         }
@@ -175,6 +177,29 @@ from custom_sso_security_manager import CustomSsoSecurityManager
 CUSTOM_SECURITY_MANAGER = CustomSsoSecurityManager
 ```
 
+**Notes**
+
+- The redirect URL will be `https://<superset-webserver>/oauth-authorized/<provider-name>`
+  When configuring an OAuth2 authorization provider if needed. For instance, the redirect URL will
+  be `https://<superset-webserver>/oauth-authorized/egaSSO` for the above configuration.
+
+- If an OAuth2 authorization server supports OpenID Connect 1.0, you could configure its configuration
+  document URL only without providing `api_base_url`, `access_token_url`, `authorize_url` and other
+  required options like user info endpoint, jwks uri etc. For instance:
+  ```python
+  OAUTH_PROVIDERS = [
+    {   'name':'egaSSO',
+        'token_key':'access_token', # Name of the token in the response of access_token_url
+        'icon':'fa-address-card',   # Icon for the provider
+        'remote_app': {
+            'client_id':'myClientId',  # Client Id (Identify Superset application)
+            'client_secret':'MySecret', # Secret for this Client Id (Identify Superset application)
+            'server_metadata_url': 'https://myAuthorizationServer/.well-known/openid-configuration'
+        }
+    }
+  ]
+  ```
+
 ### Feature Flags
 
 To support a diverse set of users, Superset has some features that are not enabled by default. For

docs/src/pages/docs/installation/index.mdx

@@ -79,7 +79,7 @@ The following is for users who want to configure how Superset starts up in Docke
 
 You can configure the Docker Compose settings for dev and non-dev mode with `docker/.env` and `docker/.env-non-dev` respectively. These environment files set the environment for most containers in the Docker Compose setup, and some variables affect multiple containers and others only single ones.
 
-One important variable is `SUPERSET_LOAD_EXAMPLES` which determines whether the `superset_init` container will load example data and visualizations into the database and Superset. Thiese examples are quite helpful for most people, but probably unnecessary for experienced users. The loading process can sometimes take a few minutes and a good amount of CPU, so you may want to disable it on a resource-constrained device.
+One important variable is `SUPERSET_LOAD_EXAMPLES` which determines whether the `superset_init` container will load example data and visualizations into the database and Superset. These examples are quite helpful for most people, but probably unnecessary for experienced users. The loading process can sometimes take a few minutes and a good amount of CPU, so you may want to disable it on a resource-constrained device.
 
 **Note:** Users often want to connect to other databases from Superset. Currently, the easiest way to do this is to modify the `docker-compose-non-dev.yml` file and add your database as a service that the other services depend on (via `x-superset-depends-on`). Others have attempted to set `network_mode: host` on the Superset services, but these generally break the installation, because the configuration requires use of the Docker Compose DNS resolver for the service names. If you have a good solution for this, let us know!
 

scripts/benchmark_migration.py

@@ -25,10 +25,12 @@
 from typing import Dict, List, Set, Type
 
 import click
+from flask import current_app
 from flask_appbuilder import Model
 from flask_migrate import downgrade, upgrade
 from graphlib import TopologicalSorter  # pylint: disable=wrong-import-order
-from sqlalchemy import inspect
+from sqlalchemy import create_engine, inspect, Table
+from sqlalchemy.ext.automap import automap_base
 
 from superset import db
 from superset.utils.mock_data import add_sample_rows
@@ -83,11 +85,18 @@ def find_models(module: ModuleType) -> List[Type[Model]]:
         elif isinstance(obj, dict):
             queue.extend(obj.values())
 
-    # add implicit models
-    # pylint: disable=no-member, protected-access
-    for obj in Model._decl_class_registry.values():
-        if hasattr(obj, "__table__") and obj.__table__.fullname in tables:
-            models.append(obj)
+    # build models by automapping the existing tables, instead of using current
+    # code; this is needed for migrations that modify schemas (eg, add a column),
+    # where the current model is out-of-sync with the existing table after a
+    # downgrade
+    sqlalchemy_uri = current_app.config["SQLALCHEMY_DATABASE_URI"]
+    engine = create_engine(sqlalchemy_uri)
+    Base = automap_base()
+    Base.prepare(engine, reflect=True)
+    for table in tables:
+        model = getattr(Base.classes, table)
+        model.__tablename__ = table
+        models.append(model)
 
     # sort topologically so we can create entities in order and
     # maintain relationships (eg, create a database before creating
@@ -133,15 +142,6 @@ def main(
     ).scalar()
     print(f"Current version of the DB is {current_revision}")
 
-    print("\nIdentifying models used in the migration:")
-    models = find_models(module)
-    model_rows: Dict[Type[Model], int] = {}
-    for model in models:
-        rows = session.query(model).count()
-        print(f"- {model.__name__} ({rows} rows in table {model.__tablename__})")
-        model_rows[model] = rows
-    session.close()
-
     if current_revision != down_revision:
         if not force:
             click.confirm(
@@ -152,6 +152,15 @@ def main(
             )
         downgrade(revision=down_revision)
 
+    print("\nIdentifying models used in the migration:")
+    models = find_models(module)
+    model_rows: Dict[Type[Model], int] = {}
+    for model in models:
+        rows = session.query(model).count()
+        print(f"- {model.__name__} ({rows} rows in table {model.__tablename__})")
+        model_rows[model] = rows
+    session.close()
+
     print("Benchmarking migration")
     results: Dict[str, float] = {}
     start = time.time()

superset-frontend/spec/fixtures/mockDatasource.js

@@ -45,6 +45,8 @@ export default {
         verbose_name: 'sum__num',
         metric_name: 'sum__num',
         description: null,
+        extra:
+          '{"certification":{"details":"foo", "certified_by":"someone"},"warning_markdown":"bar"}',
       },
       {
         expression: 'AVG(birth_names.num)',

superset-frontend/spec/javascripts/datasource/DatasourceEditor_spec.jsx

@@ -21,6 +21,9 @@ import { shallow } from 'enzyme';
 import configureStore from 'redux-mock-store';
 import fetchMock from 'fetch-mock';
 import thunk from 'redux-thunk';
+import userEvent from '@testing-library/user-event';
+import { render, screen } from 'spec/helpers/testing-library';
+
 import { Radio } from 'src/components/Radio';
 
 import Icon from 'src/components/Icon';
@@ -56,6 +59,10 @@ describe('DatasourceEditor', () => {
     inst = wrapper.instance();
   });
 
+  afterEach(() => {
+    wrapper.unmount();
+  });
+
   it('is valid', () => {
     expect(React.isValidElement(el)).toBe(true);
   });
@@ -209,3 +216,21 @@ describe('DatasourceEditor', () => {
     isFeatureEnabledMock.mockRestore();
   });
 });
+
+describe('DatasourceEditor RTL', () => {
+  it('properly renders the metric information', async () => {
+    render(<DatasourceEditor {...props} />, { useRedux: true });
+    const metricButton = screen.getByTestId('collection-tab-Metrics');
+    userEvent.click(metricButton);
+    const expandToggle = await screen.findAllByLabelText(/toggle expand/i);
+    userEvent.click(expandToggle[0]);
+    const certificationDetails = await screen.findByPlaceholderText(
+      /certification details/i,
+    );
+    expect(certificationDetails.value).toEqual('foo');
+    const warningMarkdown = await await screen.findByPlaceholderText(
+      /certified by/i,
+    );
+    expect(warningMarkdown.value).toEqual('someone');
+  });
+});

superset-frontend/src/SqlLab/components/QueryTable/index.jsx

@@ -88,6 +88,14 @@ const statusAttributes = {
       status: 'running',
     },
   },
+  fetching: {
+    color: ({ theme }) => theme.colors.primary.base,
+    config: {
+      name: 'queued',
+      label: t('fetching'),
+      status: 'fetching',
+    },
+  },
   timed_out: {
     color: ({ theme }) => theme.colors.grayscale.light1,
     config: {
@@ -97,14 +105,20 @@ const statusAttributes = {
     },
   },
   scheduled: {
-    name: 'queued',
-    label: t('Scheduled'),
-    status: 'queued',
+    color: ({ theme }) => theme.colors.greyscale.base,
+    config: {
+      name: 'queued',
+      label: t('Scheduled'),
+      status: 'queued',
+    },
   },
   pending: {
-    name: 'queued',
-    label: t('Scheduled'),
-    status: 'queued',
+    color: ({ theme }) => theme.colors.greyscale.base,
+    config: {
+      name: 'queued',
+      label: t('Scheduled'),
+      status: 'queued',
+    },
   },
 };
 

superset-frontend/src/components/ErrorMessage/DatabaseErrorMessage.test.tsx

@@ -44,6 +44,7 @@ const mockedProps = {
     message: 'Error message',
   },
   source: 'dashboard' as ErrorSource,
+  subtitle: 'Error message',
 };
 
 test('should render', () => {

superset-frontend/src/components/ErrorMessage/DatabaseErrorMessage.tsx

@@ -35,6 +35,7 @@ interface DatabaseErrorExtra {
 function DatabaseErrorMessage({
   error,
   source = 'dashboard',
+  subtitle,
 }: ErrorMessageComponentProps<DatabaseErrorExtra>) {
   const { extra, level, message } = error;
 
@@ -81,7 +82,7 @@ ${extra.issue_codes.map(issueCode => issueCode.message).join('\n')}`;
   return (
     <ErrorAlert
       title={t('%s Error', extra.engine_name || t('DB engine'))}
-      subtitle={message}
+      subtitle={subtitle}
       level={level}
       source={source}
       copyText={copyText}

superset-frontend/src/components/ErrorMessage/ErrorMessageWithStackTrace.tsx

@@ -50,7 +50,13 @@ export default function ErrorMessageWithStackTrace({
       error.error_type,
     );
     if (ErrorMessageComponent) {
-      return <ErrorMessageComponent error={error} source={source} />;
+      return (
+        <ErrorMessageComponent
+          error={error}
+          source={source}
+          subtitle={subtitle}
+        />
+      );
     }
   }
 

superset-frontend/src/components/ErrorMessage/ParameterErrorMessage.test.tsx

@@ -44,6 +44,7 @@ const mockedProps = {
     message: 'Error message',
   },
   source: 'dashboard' as ErrorSource,
+  subtitle: 'Error message',
 };
 
 test('should render', () => {

superset-frontend/src/components/ErrorMessage/ParameterErrorMessage.tsx

@@ -54,6 +54,7 @@ const findMatches = (undefinedParameters: string[], templateKeys: string[]) => {
 function ParameterErrorMessage({
   error,
   source = 'sqllab',
+  subtitle,
 }: ErrorMessageComponentProps<ParameterErrorExtra>) {
   const { extra, level, message } = error;
 
@@ -112,7 +113,7 @@ ${extra.issue_codes.map(issueCode => issueCode.message).join('\n')}`;
   return (
     <ErrorAlert
       title={t('Parameter error')}
-      subtitle={message}
+      subtitle={subtitle}
       level={level}
       source={source}
       copyText={copyText}

superset-frontend/src/components/ErrorMessage/types.ts

@@ -88,6 +88,7 @@ export type ErrorMessageComponentProps<
 > = {
   error: SupersetError<ExtraType>;
   source?: ErrorSource;
+  subtitle?: React.ReactNode;
 };
 
 export type ErrorMessageComponent = React.ComponentType<ErrorMessageComponentProps>;

superset-frontend/src/components/Form/LabeledErrorBoundInput.tsx

@@ -62,6 +62,11 @@ const alertIconStyles = (theme: SupersetTheme, hasError: boolean) => css`
     }`}
 `;
 const StyledFormGroup = styled('div')`
+  input::-webkit-outer-spin-button,
+  input::-webkit-inner-spin-button {
+    -webkit-appearance: none;
+    margin: 0;
+  }
   margin-bottom: ${({ theme }) => theme.gridUnit * 5}px;
   .ant-form-item {
     margin-bottom: 0;

superset-frontend/src/components/Icon/index.tsx

@@ -145,6 +145,7 @@ import { ReactComponent as WarningIcon } from 'images/icons/warning.svg';
 import { ReactComponent as WarningSolidIcon } from 'images/icons/warning_solid.svg';
 import { ReactComponent as XLargeIcon } from 'images/icons/x-large.svg';
 import { ReactComponent as XSmallIcon } from 'images/icons/x-small.svg';
+import { ReactComponent as DefaultDatabaseIcon } from 'images/icons/default_db_image.svg';
 
 export type IconName =
   | 'alert'
@@ -184,6 +185,7 @@ export type IconName =
   | 'copy'
   | 'cursor-target'
   | 'database'
+  | 'default-database'
   | 'dataset-physical'
   | 'dataset-virtual'
   | 'dataset-virtual-greyscale'
@@ -299,6 +301,7 @@ export const iconsRegistry: Record<
   'circle-check-solid': CircleCheckSolidIcon,
   'color-palette': ColorPaletteIcon,
   'cursor-target': CursorTargeIcon,
+  'default-database': DefaultDatabaseIcon,
   'dataset-physical': DatasetPhysicalIcon,
   'dataset-virtual': DatasetVirtualIcon,
   'dataset-virtual-greyscale': DatasetVirtualGreyscaleIcon,