Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Workspace] Refactor 'Associate data sources' in create page to support direct query connections #7961

Merged
merged 29 commits into from
Sep 5, 2024
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
57db44d
support DQC
Kapian1234 Sep 2, 2024
6111eb3
Fix UTs in workspace form select data source panel
wanglam Sep 2, 2024
affc9ba
Remove no need IntlProvider
wanglam Sep 2, 2024
cff1491
Add aria-labelledby for permission inputs
wanglam Sep 2, 2024
8490ca0
Modify UTs
Kapian1234 Sep 2, 2024
473e058
Merge branch 'main' of https://github.com/opensearch-project/OpenSear…
Kapian1234 Sep 2, 2024
8701a3f
Merge branch 'creator_association' of github.com:Kapian1234/OpenSearc…
Kapian1234 Sep 2, 2024
b7916d0
Changeset file for PR #7961 created/updated
opensearch-changeset-bot[bot] Sep 2, 2024
a362c4d
Modify UTs
Kapian1234 Sep 2, 2024
fb07145
Merge branch 'creator_association' of github.com:Kapian1234/OpenSearc…
Kapian1234 Sep 2, 2024
18b4294
Resolve some issues
Kapian1234 Sep 3, 2024
00875c0
Modify UTs
Kapian1234 Sep 3, 2024
d70ead6
Merge branch 'main' of https://github.com/opensearch-project/OpenSear…
Kapian1234 Sep 3, 2024
d3bd99f
Fix UT errror
wanglam Sep 3, 2024
cd360e8
Merge branch 'creator_association' of github.com:Kapian1234/OpenSearc…
Kapian1234 Sep 3, 2024
c816ecd
update button text
Kapian1234 Sep 3, 2024
65eed8b
rename onSelectItems()
Kapian1234 Sep 4, 2024
06e7e74
Merge branch 'main' into creator_association
Kapian1234 Sep 4, 2024
53d0fc3
Fix an error
Kapian1234 Sep 4, 2024
439c4d8
Merge branch 'creator_association' of github.com:Kapian1234/OpenSearc…
Kapian1234 Sep 4, 2024
2b4362a
Refactor data source connection table
wanglam Sep 4, 2024
64363b4
resolve some issues
Kapian1234 Sep 4, 2024
308c318
resolve some issues
Kapian1234 Sep 4, 2024
5c8d72f
resolve conflicts
Kapian1234 Sep 5, 2024
46e78c3
Fix the data source URL reference
Kapian1234 Sep 5, 2024
1400600
Move restProps to tableProps
wanglam Sep 5, 2024
ea60cb3
Fix table not unmont after connection type changed
wanglam Sep 5, 2024
2451b92
Refactor selectedDataSources to selectedDataSourceConnections
wanglam Sep 5, 2024
b4b1ac7
Load direct query connections after data source tab selected
wanglam Sep 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions changelogs/fragments/7961.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
feat:
- Support DQCs in create page ([#7961](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/7961))
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@ import {
WorkspaceCreator as WorkspaceCreatorComponent,
WorkspaceCreatorProps,
} from './workspace_creator';
import { DataSourceEngineType } from '../../../../data_source/common/data_sources';
import { DataSourceConnectionType } from '../../../common/types';
import * as utils from '../../utils';

const workspaceClientCreate = jest
.fn()
Expand All @@ -32,21 +35,50 @@ const dataSourcesList = [
{
id: 'id1',
title: 'ds1',
description: 'Description of data source 1',
auth: '',
dataSourceEngineType: '' as DataSourceEngineType,
workspaces: [],
// This is used for mocking saved object function
get: () => {
return 'ds1';
},
},
{
id: '2',
id: 'id2',
title: 'ds2',
description: 'Description of data source 1',
auth: '',
dataSourceEngineType: '' as DataSourceEngineType,
workspaces: [],
get: () => {
return 'ds2';
},
},
];

const dataSourceConnectionsList = [
{
id: 'id1',
name: 'ds1',
connectionType: DataSourceConnectionType.OpenSearchConnection,
type: 'OpenSearch',
relatedConnections: [],
},
{
id: 'id2',
name: 'ds2',
connectionType: DataSourceConnectionType.OpenSearchConnection,
type: 'OpenSearch',
},
];

const mockCoreStart = coreMock.createStart();
jest.spyOn(utils, 'fetchDataSourceConnections').mockImplementation(async (passedDataSources) => {
return dataSourceConnectionsList.filter(({ id }) =>
passedDataSources.some((dataSource) => dataSource.id === id)
);
});

const WorkspaceCreator = ({
isDashboardAdmin = false,
Expand Down Expand Up @@ -304,7 +336,15 @@ describe('WorkspaceCreator', () => {
});

it('create workspace with customized selected dataSources', async () => {
const { getByTestId, getByTitle, getByText } = render(
Object.defineProperty(HTMLElement.prototype, 'offsetHeight', {
configurable: true,
value: 600,
});
Object.defineProperty(HTMLElement.prototype, 'offsetWidth', {
configurable: true,
value: 600,
});
const { getByTestId, getAllByText, getByText } = render(
<WorkspaceCreator isDashboardAdmin={true} />
);

Expand All @@ -317,10 +357,17 @@ describe('WorkspaceCreator', () => {
target: { value: 'test workspace name' },
});
fireEvent.click(getByTestId('workspaceUseCase-observability'));
fireEvent.click(getByTestId('workspaceForm-select-dataSource-addNew'));
fireEvent.click(getByTestId('workspaceForm-select-dataSource-comboBox'));
fireEvent.click(getByText('Select'));
fireEvent.click(getByTitle(dataSourcesList[0].title));
fireEvent.click(getByTestId('workspace-creator-dataSources-assign-button'));
await waitFor(() => {
expect(
getByText(
'Add data sources that will be available in the workspace. If a selected data source has related Direct Query connection, they will also be available in the workspace.'
)
).toBeInTheDocument();
expect(getByText(dataSourcesList[0].title)).toBeInTheDocument();
});
fireEvent.click(getByText(dataSourcesList[0].title));
fireEvent.click(getAllByText('Associate data sources')[1]);

fireEvent.click(getByTestId('workspaceForm-bottomBar-createButton'));
expect(workspaceClientCreate).toHaveBeenCalledWith(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ export const WorkspaceCreatorForm = (props: WorkspaceCreatorFormProps) => {

return (
<EuiFlexGroup className="workspaceCreateFormContainer">
<EuiFlexItem style={{ maxWidth: 650 }}>
<EuiFlexItem style={{ maxWidth: 768 }}>
<EuiForm
id={formId}
onSubmit={handleFormSubmit}
Expand Down Expand Up @@ -176,8 +176,9 @@ export const WorkspaceCreatorForm = (props: WorkspaceCreatorFormProps) => {
errors={formErrors.selectedDataSources}
onChange={setSelectedDataSources}
savedObjects={savedObjects}
selectedDataSources={formData.selectedDataSources}
assignedDataSources={formData.selectedDataSources}
data-test-subj={`workspaceForm-dataSourcePanel`}
isDashboardAdmin={true}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
isDashboardAdmin={true}
isDashboardAdmin={true}

Is it by intension?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's always true here, otherwise the component wouldn't be mounted. Above, there's isDashboardAdmin && isDataSourceEnabled && (

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest we rename isDashboardAdmin to showDataSourceManagement or something like that, because it's a UI behavior, isDashboardAdmin has been widely used as a term that related to permission

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But in this SelectDataSourcePanel component, isDashboardAdmin will continue to be passed to components like DataSourceConnectionTable, where there are some behaviors like 'select' and 'delete' do indeed need permissions verification. Using showDataSourceManagement during the pass may cause confusion.
In SelectDataSourcePanel:

  const renderTableContent = () => {
    return (
      <EuiPanel paddingSize="none" hasBorder={false}>
        <DataSourceConnectionTable
          isDashboardAdmin={isDashboardAdmin}
          dataSourceConnections={assignedDataSourceConnections}
          handleUnassignDataSources={handleUnassignDataSources}
          onSelectItems={getSelectedItems}
          inCreatePage={true}
          connectionType={AssociationDataSourceModalMode.OpenSearchConnections}
        />
      </EuiPanel>
    );
  };

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, so that's an existing props. I think you can change the props.isDashboardAdmin of SelectDataSourcePanel because this props is newly aded, let's fix this first if fixing DataSourceConnectionTable is out of the scope of this PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, so in SelectDataSourcePanel it would become:

        <DataSourceConnectionTable
          isDashboardAdmin={showDataSourceManagement}
          ...
        />

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wanglam @Kapian1234 maybe we can have a follow up task to align this prop name among these components.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that we should have a follow-up task update these components. The isDashboardAdmin in DataSourceConnectionTable controls two behaviors. The first one is to control whether table row can be selectable. The second one is to control whether data source connection can be unassociated. Shall we need break it into two properties?

/>
<EuiSpacer size="s" />
<EuiSpacer size="s" />
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,12 @@ import { DataSourceConnectionType } from '../../../common/types';
import React from 'react';
import { DataSourceConnectionTable } from './data_source_connection_table';
import { AssociationDataSourceModalMode } from '../../../common/constants';
import { useOpenSearchDashboards } from '../../../../opensearch_dashboards_react/public';

jest.mock('../../../../opensearch_dashboards_react/public', () => ({
...jest.requireActual('../../../../opensearch_dashboards_react/public'),
useOpenSearchDashboards: jest.fn(),
}));
const handleUnassignDataSources = jest.fn();
const dataSourceConnectionsMock = [
{
Expand Down Expand Up @@ -58,6 +63,18 @@ const dataSourceConnectionsMock = [
];

describe('DataSourceConnectionTable', () => {
beforeEach(() => {
const mockHttp = {
basePath: {
serverBasePath: '',
},
};
(useOpenSearchDashboards as jest.Mock).mockImplementation(() => ({
services: {
http: mockHttp,
},
}));
});
afterEach(() => {
jest.clearAllMocks();
});
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@
import './data_source_connection_table.scss';
import React, { useCallback, useEffect, useMemo, useState } from 'react';
import {
EuiSpacer,
EuiInMemoryTable,
EuiBasicTableColumn,
EuiTableSelectionType,
Expand All @@ -27,26 +26,40 @@
import { DataSourceConnection, DataSourceConnectionType } from '../../../common/types';
import { AssociationDataSourceModalMode } from '../../../common/constants';
import { DirectQueryConnectionIcon } from '../workspace_form';
import { useOpenSearchDashboards } from '../../../../opensearch_dashboards_react/public';
import { CoreStart } from '../../../../../core/public';

interface DataSourceConnectionTableProps {
isDashboardAdmin: boolean;
connectionType: string;
dataSourceConnections: DataSourceConnection[];
handleUnassignDataSources: (dataSources: DataSourceConnection[]) => Promise<void>;
handleUnassignDataSources: (dataSources: DataSourceConnection[]) => void;
onSelectedItems?: (dataSources: DataSourceConnection[]) => void;
inCreatePage?: boolean;
}

export const DataSourceConnectionTable = ({
isDashboardAdmin,
connectionType,
dataSourceConnections,
handleUnassignDataSources,
onSelectedItems,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
onSelectedItems,
onSelectItems,

Nit:

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

inCreatePage = false,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you leave a comment to describe the UI results of setting inCreatePage to true/false?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

}: DataSourceConnectionTableProps) => {
const [selectedItems, setSelectedItems] = useState<DataSourceConnection[]>([]);
const [modalVisible, setModalVisible] = useState(false);
const [popoversState, setPopoversState] = useState<Record<string, boolean>>({});
const [itemIdToExpandedRowMap, setItemIdToExpandedRowMap] = useState<
Record<string, React.ReactNode>
>({});
const {
services: { http },
} = useOpenSearchDashboards<CoreStart>();
useEffect(() => {
if (onSelectedItems) {
onSelectedItems(selectedItems);
}
}, [selectedItems, onSelectedItems]);

useEffect(() => {
// Reset selected items when connectionType changes
Expand Down Expand Up @@ -152,19 +165,20 @@
]
: []),
{
width: '25%',
width: '20%',
field: 'name',
name: i18n.translate('workspace.detail.dataSources.table.title', {
defaultMessage: 'Title',
}),
truncateText: true,
render: (name: string, record) => {
const origin = window.location.origin;
const basePath = http.basePath.serverBasePath;
let url: string;
if (record.connectionType === DataSourceConnectionType.OpenSearchConnection) {
url = `${origin}/app/dataSources/${record.id}`;
url = `${origin}${basePath}/app/dataSources/${record.id}`;
Copy link
Member

@SuZhou-Joe SuZhou-Joe Sep 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use
basePath.prepend(`/app/dataSources/${record.id}`, { withoutClientBasePath: true })?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

basePath is a string and there's no prepend in basePath

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think Suzhou refers to http.basePath

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right, please use http.basePath to get the final path. We should avoid string concatenation when generating the path.

} else {
url = `${origin}/app/dataSources/manage/${name}?dataSourceMDSId=${record.parentId}`;
url = `${origin}${basePath}/app/dataSources/manage/${name}?dataSourceMDSId=${record.parentId}`;
}
return (
<EuiLink href={url} className="eui-textTruncate">
Expand All @@ -174,27 +188,26 @@
},
},
{
width: '10%',
width: '20%',
field: 'type',
name: i18n.translate('workspace.detail.dataSources.table.type', {
defaultMessage: 'Type',
}),
truncateText: true,
},
{
width: '35%',
field: 'description',
name: i18n.translate('workspace.detail.dataSources.table.description', {
defaultMessage: 'Description',
}),
truncateText: true,
},
{
width: '140px',
field: 'relatedConnections',
name: i18n.translate('workspace.detail.dataSources.table.relatedConnections', {
defaultMessage: 'Related connections',
}),
align: 'right',
truncateText: true,
render: (relatedConnections: DataSourceConnection[], record) =>
relatedConnections?.length > 0 ? (
Expand Down Expand Up @@ -267,12 +280,17 @@
icon: 'unlink',
type: 'icon',
onClick: (item: DataSourceConnection) => {
setSelectedItems([item]);
setModalVisible(true);
if (inCreatePage) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When in create page, click un-assign button will call handleUnassignDataSources directly, otherwise, it will display a confirmation dialog, am I right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes

handleUnassignDataSources([item]);

Check warning on line 284 in src/plugins/workspace/public/components/workspace_detail/data_source_connection_table.tsx

View check run for this annotation

Codecov / codecov/patch

src/plugins/workspace/public/components/workspace_detail/data_source_connection_table.tsx#L284

Added line #L284 was not covered by tests
} else {
setSelectedItems([item]);
setModalVisible(true);
}
},
'data-test-subj': 'workspace-detail-dataSources-table-actions-remove',
},
],
width: '10%',
} as EuiTableActionsColumnType<DataSourceConnection>,
]
: []),
Expand All @@ -285,22 +303,34 @@

return (
<>
<EuiInMemoryTable
items={openSearchConnections}
itemId="id"
columns={columns}
selection={selection}
search={search}
key={connectionType}
isSelectable={true}
itemIdToExpandedRowMap={itemIdToExpandedRowMap}
isExpandable={true}
pagination={{
initialPageSize: 10,
pageSizeOptions: [10, 20, 30],
}}
/>
<EuiSpacer />
{inCreatePage ? (
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like the only difference is we show pagination when not in create page?
Nit: what do this?

interface DataSourceConnectionTableProps extends Pick<EuiInMemoryTableProps<DataSourceConnection>, 'pagination'> {
...
}

Then you can pass pagination as props to DataSourceConnectionTable whenever needed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another difference is that we show search bar when not in create page, then we have to pass two props. The search props looks like this, we may not need to pass it.

   const renderToolsLeft = useCallback(() => {
    return selectedItems.length > 0 && !modalVisible
      ? [
          <EuiSmallButton
            color="danger"
            onClick={() => setModalVisible(true)}
            data-test-subj="workspace-detail-dataSources-table-bulkRemove"
          >
            {i18n.translate('workspace.detail.dataSources.table.remove.button', {
              defaultMessage: 'Remove {numberOfSelect} association(s)',
              values: { numberOfSelect: selectedItems.length },
            })}
          </EuiSmallButton>,
        ]
      : [];
  }, [selectedItems, modalVisible]);

  const search: EuiSearchBarProps = {
    toolsLeft: renderToolsLeft(),
    box: {
      incremental: true,
    },
    filters: [
      {
        type: 'field_value_selection',
        field: 'type',
        name: 'Type',
        multiSelect: 'or',
        options: Array.from(
          new Set(openSearchConnections.map(({ type }) => type).filter(Boolean))
        ).map((type) => ({
          value: type!,
          name: type!,
        })),
      },
    ],
  };

Copy link
Member

@SuZhou-Joe SuZhou-Joe Sep 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This component needs a refactor. If we are trying to make this component a reusable component, we should let the parent of the component to pass in different props to control the behavior of the component. For this component, I think we should split this component into 2 components:

  1. A table component that renders the built-in columns, and gives a onUnlinkDatasource callback and tableProps: EuiInMemoryTableProps, without the inCreatePage props.
  2. The ConfirmModal should be moved to workspace detail page and all the different props should be handled by workspace details page and workspace creation page, the table component just take whatever props their parents pass in and spread to the EuiInMemoryTable.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree with @SuZhou-Joe 's comments. Current implementation of DataSourceConnectionTable contains too much business logic should handled by workspace detail or create page self. I've updated the code. Feel free to help me review it. Thank you.

<EuiInMemoryTable
items={openSearchConnections}
itemId="id"
columns={columns}
selection={selection}
key={connectionType}
isSelectable={true}
itemIdToExpandedRowMap={itemIdToExpandedRowMap}
isExpandable={true}
/>
) : (
<EuiInMemoryTable
items={openSearchConnections}
itemId="id"
columns={columns}
selection={selection}
search={search}
key={connectionType}
isSelectable={true}
itemIdToExpandedRowMap={itemIdToExpandedRowMap}
isExpandable={true}
pagination={{
initialPageSize: 10,
pageSizeOptions: [10, 20, 30],
}}
/>
)}
{modalVisible && (
<EuiConfirmModal
data-test-subj="workspaceForm-cancelModal"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -123,3 +123,7 @@ export const DetailTabTitles: { [key in DetailTab]: string } = {
defaultMessage: 'Collaborators',
}),
};

export const PERMISSION_TYPE_LABEL_ID = 'workspace-form-permission-type-label';
export const PERMISSION_COLLABORATOR_LABEL_ID = 'workspace-form-permission-collaborator-label';
export const PERMISSION_ACCESS_LEVEL_LABEL_ID = 'workspace-form-permission-access-level-label';
Loading
Loading