Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Visualize SqlLab.Query model data in Explore 📈 #20281

Merged
merged 102 commits into from
Jul 15, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
102 commits
Select commit Hold shift + click to select a range
6fc435d
add POC ExploreMixin
hughhhh Jun 6, 2022
632ce21
Working POC
hughhhh Jun 6, 2022
6f22618
Created/tested query dataset dropdown
lyndsiWilliams Jun 7, 2022
236c489
Add isValidDatasourceType to @superset-ui/core and hide query dropdown
lyndsiWilliams Jun 8, 2022
24261d1
fix merge conflicts
hughhhh Jun 8, 2022
bd3b6a9
Visual updates to explore datasource panel
lyndsiWilliams Jun 8, 2022
64f4181
Temporarily make Query icon visible
lyndsiWilliams Jun 8, 2022
7c07534
Remove Query icon visibility
lyndsiWilliams Jun 8, 2022
6fab57f
Removed isValidDatasourceType check
lyndsiWilliams Jun 8, 2022
3a46152
Added Query preview Modal from DatasourceControl if the data source t…
eric-briscoe Jun 8, 2022
613d78a
> fix integration point with frontend
hughhhh Jun 8, 2022
e076f13
Merge branch 'master' of https://github.com/preset-io/superset into c…
hughhhh Jun 13, 2022
31dfdd2
Adjusts conditional logic approach to be extensible for additional ty…
eric-briscoe Jun 13, 2022
bbb91e0
Merge branch 'chart-power-query' into lyndsi/dataset-panel-updates
eric-briscoe Jun 13, 2022
8513cc5
Merge branch 'lyndsi/dataset-panel-updates' into ericbriscoe/sc-41493…
eric-briscoe Jun 13, 2022
cd54603
refactor
hughhhh Jun 14, 2022
a173cfa
Merge branch 'master' of https://github.com/preset-io/superset into c…
hughhhh Jun 15, 2022
0295818
Merge pull request #218 from preset-io/ericbriscoe/sc-41493/query-pre…
eric-briscoe Jun 15, 2022
de20276
set field for sql
hughhhh Jun 15, 2022
bd7db0d
Fixes issue where Missing query parameters error was showing in datas…
eric-briscoe Jun 16, 2022
e344197
add query_language
hughhhh Jun 16, 2022
9c5bc48
fix ds main_dttm
hughhhh Jun 17, 2022
b6fbc33
Fixes issue where menu.tsx was blocking access to redux debugging for…
eric-briscoe Jun 17, 2022
6af2961
Fixes issue where database id was not available to save query as data…
eric-briscoe Jun 17, 2022
12546f5
Merge branch 'master' into chart-power-query
eric-briscoe Jun 17, 2022
ff59846
Merge branch 'master' into chart-power-query
hughhhh Jun 21, 2022
bdbf2d1
oops
hughhhh Jun 21, 2022
559e1a2
fix pre-commit to 50 errors now
hughhhh Jun 21, 2022
890be43
fix circuliar dep
hughhhh Jun 21, 2022
4b833be
Disables showing Metrics section in DatasourcePanel when Query is the…
eric-briscoe Jun 21, 2022
4254b7e
adds condition to use query.columns if query.results is not present e…
eric-briscoe Jun 22, 2022
4710b3d
down to 26 now
hughhhh Jun 22, 2022
017bde6
patch for pre-commit
hughhhh Jun 24, 2022
075c5d8
one more pre-commit
hughhhh Jun 24, 2022
d73504c
added explore_json error
AAfghahi Jun 24, 2022
74ae38c
added error messages
AAfghahi Jun 24, 2022
97ee4eb
Fixes frontend lint and TypeScript errors unit test fixes will be nex…
eric-briscoe Jun 28, 2022
276bc28
Aditional TypeScript error fix
eric-briscoe Jun 28, 2022
a8ff273
Fixes unit test failure
eric-briscoe Jun 28, 2022
044945b
added frontend piece
AAfghahi Jun 28, 2022
8d8a868
Merge branch master
eric-briscoe Jun 28, 2022
7ace7a4
Fixes bad import caused by merge from master and removes duplicate sh…
eric-briscoe Jun 28, 2022
e436582
Fixes for DartasourceControl Test Suite
eric-briscoe Jun 28, 2022
76c3505
Fix lint error
eric-briscoe Jun 28, 2022
3370439
Fixes unit test issues due to array instead of a component being pass…
eric-briscoe Jun 29, 2022
d1866e8
merged main branch
AAfghahi Jun 29, 2022
18b6fa1
Merge pull request #223 from preset-io/arash/error_messages
AAfghahi Jun 29, 2022
a7ded35
Fixes unit test failure for DatasourceControl and simplifies getDatas…
eric-briscoe Jun 30, 2022
c65c225
Merge branch 'master' into chart-power-query
eric-briscoe Jun 30, 2022
232d6dc
pylint
AAfghahi Jun 29, 2022
aa2756e
core_test fix
AAfghahi Jul 5, 2022
ebdfcf0
Merge pull request #225 from preset-io/arash/pylint_errors
hughhhh Jul 5, 2022
a52acd9
Merge branch 'master' into chart-power-query
eric-briscoe Jul 5, 2022
3653e40
Fixes line error post merge from master
eric-briscoe Jul 5, 2022
c0ab26b
merge master
hughhhh Jul 6, 2022
655fddf
fixed from master
hughhhh Jul 6, 2022
f752655
fixed from master
hughhhh Jul 6, 2022
c478add
Merge branch 'chart-power-query' of ssh://github.com/preset-io/supers…
eric-briscoe Jul 6, 2022
5e32c24
Fixes issue where Overwrite dataset does not work due to userid error
eric-briscoe Jul 6, 2022
6707808
Resolves TypeScript errors with changes made for SPA merging in and c…
eric-briscoe Jul 6, 2022
b28d79c
fix: top right panel view query functionality
hughhhh Jul 7, 2022
c619618
Merge branch 'master' of https://github.com/preset-io/superset into c…
hughhhh Jul 7, 2022
fa85488
remove unneeded code from core.py
hughhhh Jul 7, 2022
a09d781
working samples endpoint for query
hughhhh Jul 7, 2022
4850f9a
new error message for legacy chart
AAfghahi Jul 7, 2022
bb7acbb
Merge branch 'chart-power-query' into arash/newErrorMessage
AAfghahi Jul 7, 2022
d80576e
addressing comments
AAfghahi Jul 8, 2022
076e4fd
add owner check
hughhhh Jul 8, 2022
c5ca22d
update FE for it
hughhhh Jul 8, 2022
a35038d
handle columns are dict vs object
hughhhh Jul 8, 2022
661bb9e
fix exceptions
hughhhh Jul 8, 2022
b476b70
fix fe lint
hughhhh Jul 8, 2022
56fcdc9
fix test
hughhhh Jul 8, 2022
551ddc8
Merge pull request #233 from preset-io/chart-power-query-samples
hughhhh Jul 8, 2022
6e5443a
add tab_name to payload
hughhhh Jul 8, 2022
97ac731
Enables use of tab name from Query
eric-briscoe Jul 8, 2022
664773a
fix merge conflicts
hughhhh Jul 11, 2022
a073313
fix cypress test
hughhhh Jul 11, 2022
f717e18
save columns on execution
hughhhh Jul 11, 2022
16ea687
fix frontend build test
hughhhh Jul 11, 2022
3f6fa0a
remove parathesis around columns
hughhhh Jul 11, 2022
88a51b4
changing column types
AAfghahi Jul 11, 2022
a105ca6
fixing samples that has literal_columns
AAfghahi Jul 11, 2022
9e09af2
address comments
hughhhh Jul 11, 2022
8eb1709
all save to datasets now working
AAfghahi Jul 11, 2022
744f031
Merge remote-tracking branch 'origin/chart-power-query' into arash/ne…
AAfghahi Jul 12, 2022
d21f145
fix menu
hughhhh Jul 13, 2022
432aee7
fix build
hughhhh Jul 13, 2022
66fc9a6
remove viz error code and addressed more concerns
hughhhh Jul 14, 2022
ff963db
Update superset-frontend/src/explore/components/ExploreChartPanel.jsx
AAfghahi Jul 14, 2022
e4448fe
Merge branch 'chart-power-query' into arash/newErrorMessage
AAfghahi Jul 14, 2022
a203bf8
addressed hugh comments
AAfghahi Jul 14, 2022
15d530c
remove viz error code and addressed more concerns
hughhhh Jul 14, 2022
48f40a6
Merge branch 'master' of https://github.com/preset-io/superset into c…
hughhhh Jul 14, 2022
555d26b
address comments v2
hughhhh Jul 15, 2022
cc7a2d0
Merge remote-tracking branch 'origin/chart-power-query' into arash/ne…
AAfghahi Jul 15, 2022
eb99ad4
addressed last comment
AAfghahi Jul 15, 2022
4fa62e9
addressed last comment
AAfghahi Jul 15, 2022
afa34fd
failing test
AAfghahi Jul 15, 2022
c0d7c14
Merge pull request #231 from preset-io/arash/newErrorMessage
hughhhh Jul 15, 2022
0666568
Merge branch 'master' into chart-power-query
AAfghahi Jul 15, 2022
0e5767b
hide feature
hughhhh Jul 15, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,9 @@ describe('Test datatable', () => {
cy.get('.ant-empty-description').should('not.exist');
});
it('Datapane loads view samples', () => {
cy.intercept('/api/v1/dataset/*/samples?force=false').as('Samples');
cy.intercept(
'api/v1/explore/samples?force=false&datasource_type=table&datasource_id=*',
).as('Samples');
cy.contains('Samples')
.click()
.then(() => {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@ export default class DatasourceKey {
constructor(key: string) {
const [idStr, typeStr] = key.split('__');
this.id = parseInt(idStr, 10);
this.type =
typeStr === 'table' ? DatasourceType.Table : DatasourceType.Druid;
this.type = DatasourceType.Table; // default to SqlaTable model
this.type = typeStr === 'query' ? DatasourceType.Query : this.type;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make this into a switch statement

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can remove DatasourceType.Druid

}

public toString() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ export type PhysicalColumn = string;
* Column information defined in datasource.
*/
export interface Column {
id: number;
id?: number;
eschutho marked this conversation as resolved.
Show resolved Hide resolved
type?: string;
type_generic?: GenericDataType;
column_name: string;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ import { Metric } from './Metric';

export enum DatasourceType {
Table = 'table',
Druid = 'druid',
Query = 'query',
Dataset = 'dataset',
SlTable = 'sl_table',
Expand All @@ -47,11 +46,14 @@ export interface Datasource {
};
}

export const DEFAULT_METRICS = [
export const DEFAULT_METRICS: Metric[] = [
{
metric_name: 'COUNT(*)',
expression: 'COUNT(*)',
},
];

export const isValidDatasourceType = (datasource: DatasourceType) =>
Object.values(DatasourceType).includes(datasource);

export default {};
Original file line number Diff line number Diff line change
Expand Up @@ -334,6 +334,7 @@ export type QueryResults = {
expanded_columns: QueryColumn[];
selected_columns: QueryColumn[];
query: { limit: number };
query_id?: number;
};
};

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,17 +20,10 @@ import { DatasourceKey } from '@superset-ui/core';

describe('DatasourceKey', () => {
const tableKey = '5__table';
const druidKey = '5__druid';

it('should handle table data sources', () => {
const datasourceKey = new DatasourceKey(tableKey);
expect(datasourceKey.toString()).toBe(tableKey);
expect(datasourceKey.toObject()).toEqual({ id: 5, type: 'table' });
});

it('should handle druid data sources', () => {
const datasourceKey = new DatasourceKey(druidKey);
expect(datasourceKey.toString()).toBe(druidKey);
expect(datasourceKey.toObject()).toEqual({ id: 5, type: 'druid' });
});
});
Original file line number Diff line number Diff line change
Expand Up @@ -31,17 +31,6 @@ describe('buildQueryContext', () => {
expect(queryContext.result_format).toBe('json');
expect(queryContext.result_type).toBe('full');
});
it('should build datasource for druid sources and set force to true', () => {
const queryContext = buildQueryContext({
datasource: '5__druid',
granularity: 'ds',
viz_type: 'table',
force: true,
});
expect(queryContext.datasource.id).toBe(5);
expect(queryContext.datasource.type).toBe('druid');
expect(queryContext.force).toBe(true);
});
it('should build datasource for table sources with columns', () => {
const queryContext = buildQueryContext(
{
Expand Down
30 changes: 27 additions & 3 deletions superset-frontend/src/SqlLab/components/ResultSet/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,11 @@ import Button from 'src/components/Button';
import shortid from 'shortid';
import { styled, t, QueryResponse } from '@superset-ui/core';
import ErrorMessageWithStackTrace from 'src/components/ErrorMessage/ErrorMessageWithStackTrace';
import { SaveDatasetModal } from 'src/SqlLab/components/SaveDatasetModal';
import {
ISaveableDatasource,
ISimpleColumn,
SaveDatasetModal,
} from 'src/SqlLab/components/SaveDatasetModal';
import { UserWithPermissionsAndRoles } from 'src/types/bootstrapTypes';
import ProgressBar from 'src/components/ProgressBar';
import Loading from 'src/components/Loading';
Expand Down Expand Up @@ -220,6 +224,15 @@ export default class ResultSet extends React.PureComponent<
const { showSaveDatasetModal } = this.state;
const { query } = this.props;

const datasource: ISaveableDatasource = {
columns: query.columns as ISimpleColumn[],
name: query?.tab || 'Untitled',
dbId: 1,
sql: query.sql,
templateParams: query.templateParams,
schema: query.schema,
};

return (
<ResultSetControls>
<SaveDatasetModal
Expand All @@ -230,14 +243,25 @@ export default class ResultSet extends React.PureComponent<
modalDescription={t(
'Save this query as a virtual dataset to continue exploring',
)}
datasource={query}
datasource={datasource}
/>
<ResultSetButtons>
{this.props.visualize &&
this.props.database?.allows_virtual_table_explore && (
<ExploreResultsButton
database={this.props.database}
onClick={() => this.setState({ showSaveDatasetModal: true })}
onClick={() => {
// There is currently redux / state issue where sometimes a query will have serverId
// and other times it will not. We need this attribute consistently for this to work
// const qid = this.props?.query?.results?.query_id;
// if (qid) {
// // This will open explore using the query as datasource
// window.location.href = `/explore/?dataset_type=query&dataset_id=${qid}`;
// } else {
// this.setState({ showSaveDatasetModal: true });
// }
this.setState({ showSaveDatasetModal: true });
}}
/>
)}
{this.props.csv && (
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,16 +17,42 @@
* under the License.
*/
import React from 'react';
import { QueryResponse, testQuery } from '@superset-ui/core';
import { SaveDatasetModal } from 'src/SqlLab/components/SaveDatasetModal';
import {
ISaveableDatasource,
SaveDatasetModal,
} from 'src/SqlLab/components/SaveDatasetModal';
import { render, screen } from 'spec/helpers/testing-library';
import { DatasourceType } from '@superset-ui/core';

const testQuery: ISaveableDatasource = {
name: 'unimportant',
dbId: 1,
sql: 'SELECT *',
columns: [
{
name: 'Column 1',
type: DatasourceType.Query,
is_dttm: false,
},
{
name: 'Column 3',
type: DatasourceType.Query,
is_dttm: false,
},
{
name: 'Column 2',
type: DatasourceType.Query,
is_dttm: true,
},
],
};

const mockedProps = {
visible: true,
onHide: () => {},
buttonTextOnSave: 'Save',
buttonTextOnOverwrite: 'Overwrite',
datasource: testQuery as QueryResponse,
datasource: testQuery,
};

describe('SaveDatasetModal RTL', () => {
Expand All @@ -36,6 +62,7 @@ describe('SaveDatasetModal RTL', () => {
const saveRadioBtn = screen.getByRole('radio', {
name: /save as new unimportant/i,
});

const fieldLabel = screen.getByText(/save as new/i);
const inputField = screen.getByRole('textbox');
const inputFieldText = screen.getByDisplayValue(/unimportant/i);
Expand Down
Loading