[Multiple Datasource] Expose filterfn in datasource menu component to allow filter data sources before rendering in navigation bar #6113

BionIT · 2024-03-11T18:43:51Z

Description

In #6057, we exposed a few properties in data source selector component for plugins to consume, and in #6082, we created a new data source menu component to be mounted into navigation bar. Based on requirements, we should also expose this filter function to allow plugins to exclude data sources that don't make sense to render in options

Issues Resolved

Screenshot

exportfilterinselectable.mp4

Testing the changes

The following steps were performed in the recording:

create a data source with no authentication
go to add sample data page, and see that the data source picker shows all available data sources including the one configured with no authentication
go to query workbench plugin, and see that the data source menu doesn't show the data source with no auth
go to the query workbench implementation, and see the filterfn passed in will exclude the data sources with no auth

Check List

All tests pass
- yarn test:jest
- yarn test:jest_integration
New functionality includes testing.
New functionality has been documented.
Update CHANGELOG.md
Commits are signed per the DCO using --signoff

Signed-off-by: Lu Yu <nluyu@amazon.com>

codecov · 2024-03-11T18:57:37Z

Codecov Report

Attention: Patch coverage is 90.00000% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 67.15%. Comparing base (4b89ad0) to head (1ce11a3).

Files	Patch %	Lines
...onents/data_source_menu/data_source_selectable.tsx	90.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6113      +/-   ##
==========================================
+ Coverage   67.14%   67.15%   +0.01%     
==========================================
  Files        3326     3326              
  Lines       64394    64400       +6     
  Branches    10358    10361       +3     
==========================================
+ Hits        43237    43249      +12     
+ Misses      18627    18620       -7     
- Partials     2530     2531       +1

Flag	Coverage Δ
Linux_1	`31.73% <ø> (ø)`
Linux_2	`55.23% <ø> (ø)`
Linux_3	`?`
Linux_4	`35.08% <0.00%> (-0.01%)`	⬇️
Windows_1	`31.75% <ø> (ø)`
Windows_2	`55.20% <ø> (ø)`
Windows_3	`44.75% <90.00%> (+0.02%)`	⬆️
Windows_4	`35.08% <0.00%> (-0.01%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Flyingliuhub · 2024-03-12T16:50:56Z

...plugins/data_source_management/public/components/data_source_menu/data_source_selectable.tsx

@@ -69,18 +70,24 @@ export class DataSourceSelectable extends React.Component<

  async componentDidMount() {
    this._isMounted = true;
-    getDataSources(this.props.savedObjectsClient)
+    getDataSourcesWithFields(this.props.savedObjectsClient, ['id', 'title', 'auth.type'])


how we extend this, which not only can filter auth.type?

we can extend by adding more fields, the fields entered here are to specify which fields should be returned from the API for data source object, and before, it restricts to id, title, description, auth type is required for security plugin to remove no auth data sources since they would indicate non-FGAC domains

Flyingliuhub · 2024-03-12T16:53:20Z

...ns/data_source_management/public/components/data_source_menu/data_source_selectable.test.tsx

      perPage: 10000,
      type: 'data-source',
    });
    expect(toasts.addWarning).toBeCalledTimes(0);
  });
+
+  it('should filter options if configured', async () => {


Where we configure this filterfn, is configurable in the yml file or somewhere or must change the code to configure the filter? thanks

This filterFn is passed by plugins when they consume the data source menu component, it is used then in the component when it is mounted to remove unwanted data sources before rendering

Signed-off-by: Lu Yu <nluyu@amazon.com>

xinruiba · 2024-03-12T17:11:33Z

...plugins/data_source_management/public/components/data_source_menu/data_source_selectable.tsx

-            a.label.toLowerCase().localeCompare(b.label.toLowerCase())
-          );
+          if (filteredDataSources.length === 0) {
+            filteredDataSources = fetchedDataSources;


If no filteredDataSources exist after apply filter, we will show all dataSource instead. And those datasource won't meet the filter condition.

Will it introduce in some confusions? What if we just show nothing.

Thanks

This filter is for plugins to decide how they want to render the options, there is possibility the filter would remove all options then there will be no options displayed. If plugins decide not to add filter options, then it should show all data sources, why it is confusing?

This comment is not a blocker of this PR~

Suppose we have following dataSources:
[{datasource1, NoAuth}, {datasource2, NoAuth}, {datasource3, NoAuth}]

And filter out all "NoAuth" dataSources.

Looks like we will return [{datasource1, NoAuth}, {datasource2, NoAuth}, {datasource3, NoAuth}] here,
instead of an empty array. Is that expected?

Thanks~

… allow filter data sources before rendering in navigation bar (#6113) * expose filterfn in datasource menu component Signed-off-by: Lu Yu <nluyu@amazon.com> * add changelog Signed-off-by: Lu Yu <nluyu@amazon.com> --------- Signed-off-by: Lu Yu <nluyu@amazon.com> (cherry picked from commit 45096bb) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> # Conflicts: # CHANGELOG.md

… allow filter data sources before rendering in navigation bar (#6113) (#6261) * expose filterfn in datasource menu component Signed-off-by: Lu Yu <nluyu@amazon.com> * add changelog Signed-off-by: Lu Yu <nluyu@amazon.com> --------- Signed-off-by: Lu Yu <nluyu@amazon.com> (cherry picked from commit 45096bb) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> # Conflicts: # CHANGELOG.md Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

expose filterfn in datasource menu component

6088b41

Signed-off-by: Lu Yu <nluyu@amazon.com>

BionIT requested review from ananzh, kavilla, AMoo-Miki, ashwin-pc, joshuarrrr, abbyhu2000, zengyan-amazon, kristenTian, zhongnansu, manasvinibs, ZilongX, Flyingliuhub, BSFishy, curq, bandinib-amzn, SuZhou-Joe and ruanyl as code owners March 11, 2024 18:43

github-actions bot added the seasoned-contributor label Mar 11, 2024

BionIT changed the title ~~[Multiple Datasource] Expose filterfn in datasource menu component~~ [Multiple Datasource] Expose filterfn in datasource menu component to allow filter data sources before rendering in navigation bar Mar 11, 2024

add changelog

2846c0a

Signed-off-by: Lu Yu <nluyu@amazon.com>

BionIT mentioned this pull request Mar 11, 2024

[MD] Create data source selectable menu using context menu and allow it to be consumed by plugins via Nav bar #6035

Closed

2 tasks

yujin-emma approved these changes Mar 12, 2024

View reviewed changes

Merge branch 'main' into addfilterFn

a534d7f

Flyingliuhub reviewed Mar 12, 2024

View reviewed changes

Merge branch 'main' into addfilterFn

1ce11a3

Signed-off-by: Lu Yu <nluyu@amazon.com>

xinruiba reviewed Mar 12, 2024

View reviewed changes

seraphjiang added the multiple datasource multiple datasource project label Mar 12, 2024

Flyingliuhub assigned BionIT Mar 12, 2024

Flyingliuhub approved these changes Mar 12, 2024

View reviewed changes

ZilongX approved these changes Mar 12, 2024

View reviewed changes

xinruiba approved these changes Mar 12, 2024

View reviewed changes

BionIT merged commit 45096bb into opensearch-project:main Mar 12, 2024
70 of 71 checks passed

seraphjiang added the v2.14.0 label Mar 12, 2024

BionIT mentioned this pull request Mar 19, 2024

[Meta] [2.14] Security, bug fix and enhancemanet for dashboards anywhere #5872

Closed

77 tasks

BionIT added the backport 2.x label Mar 25, 2024

opensearch-trigger-bot bot mentioned this pull request Mar 25, 2024

[Backport 2.x] [Multiple Datasource] Expose filterfn in datasource menu component to allow filter data sources before rendering in navigation bar #6261

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Multiple Datasource] Expose filterfn in datasource menu component to allow filter data sources before rendering in navigation bar #6113

[Multiple Datasource] Expose filterfn in datasource menu component to allow filter data sources before rendering in navigation bar #6113

BionIT commented Mar 11, 2024

codecov bot commented Mar 11, 2024 •

edited

Loading

Flyingliuhub Mar 12, 2024

BionIT Mar 12, 2024

Flyingliuhub Mar 12, 2024

BionIT Mar 12, 2024

xinruiba Mar 12, 2024 •

edited

Loading

BionIT Mar 12, 2024

xinruiba Mar 12, 2024 •

edited

Loading

[Multiple Datasource] Expose filterfn in datasource menu component to allow filter data sources before rendering in navigation bar #6113

[Multiple Datasource] Expose filterfn in datasource menu component to allow filter data sources before rendering in navigation bar #6113

Conversation

BionIT commented Mar 11, 2024

Description

Issues Resolved

Screenshot

Testing the changes

Check List

codecov bot commented Mar 11, 2024 • edited Loading

Codecov Report

Flyingliuhub Mar 12, 2024

Choose a reason for hiding this comment

BionIT Mar 12, 2024

Choose a reason for hiding this comment

Flyingliuhub Mar 12, 2024

Choose a reason for hiding this comment

BionIT Mar 12, 2024

Choose a reason for hiding this comment

xinruiba Mar 12, 2024 • edited Loading

Choose a reason for hiding this comment

BionIT Mar 12, 2024

Choose a reason for hiding this comment

xinruiba Mar 12, 2024 • edited Loading

Choose a reason for hiding this comment

codecov bot commented Mar 11, 2024 •

edited

Loading

xinruiba Mar 12, 2024 •

edited

Loading

xinruiba Mar 12, 2024 •

edited

Loading