From 58f392710db6021a6d3bd70c1ce7365386699c38 Mon Sep 17 00:00:00 2001 From: Kristen Tian <105667444+kristenTian@users.noreply.github.com> Date: Mon, 19 Sep 2022 13:19:26 -0700 Subject: [PATCH] MultiDataSource feature merge (#2334) * Instantiate credential management plugin code structure (#1996) Signed-off-by: Kristen Tian * Data source inside stack management setup (#2017) (#2030) Signed-off-by: Kristen Tian * enable CI for feature branch (#2010) Signed-off-by: Zhongnan Su Signed-off-by: Kristen Tian * Add empty data source plugin (#2052) Adds empty data source plugin. Signed-off-by: Kristen Tian * [MD] Add initial credential management CRUD pages (#2040) * Add credential management CRUD pages 1. List all credentials 2. Create your saved credential 3. Edit your credential 4. Delete credentials Signed-off-by: Louis Chu Signed-off-by: Kristen Tian * Register Data source to savedObject & Update license header (#2037) Signed-off-by: Kristen Tian * Move credential saved object to data source plugin (#2062) Signed-off-by: Louis Chu Move credential saved object to data source plugin Resolve follow up comments on UI Signed-off-by: Kristen Tian * breadcrumbfix for datasource management (#2066) * breadcrumbfix for datasource management Signed-off-by: mpabba3003 * breadcrumbfix for datasource management - refactoring code Signed-off-by: mpabba3003 * using services to update breadcrumb on data sources management page Signed-off-by: mpabba3003 * Changing the license header on breadcrumbs.ts datasource management Signed-off-by: Kristen Tian * Fix breadcrumb on listing page and update saved object mapping (#2069) Signed-off-by: Louis Chu Signed-off-by: Kristen Tian * Add data source step into IndexPattern with Mock switch (#2064) (#2086) Signed-off-by: Kristen Tian * Add delete button for credential detailed page (#2067) Signed-off-by: Yibo Wang Signed-off-by: Kristen Tian * Link datasource to indexpattern (#2118) Signed-off-by: Kristen Tian Signed-off-by: Kristen Tian * Add encrypt/decrypt module on data source plugin (#2120) Signed-off-by: Louis Chu 1. Add encrypt/decrypt module with UT 2. Add client factory wrapper for encrypt credential 3. Add encryption config support 4. Bugfix on Jest interpret Buffer Signed-off-by: Kristen Tian * Integrate index pattern with new data client (#2146) Signed-off-by: Kristen Tian * Add noAuth to dataSource attributes (#2154) Signed-off-by: Kristen Tian Signed-off-by: Kristen Tian * [MD] Datasource Management - creation & listing - UI only (#2128) * data source management - creation & Listing UI only * data source management - creation & Listing UI only * Create/edit data source feature * toggling default value * refactoring code as per review comments * toggling server flag to false Signed-off-by: mpabba3003 Signed-off-by: Kristen Tian * Apply dataSource plugin as dependent for cm and dsm plugins (#2150) Signed-off-by: Louis Chu Apply dataSource plugin as dependent for cm and dsm plugins (#2150) Signed-off-by: Kristen Tian * Leverage datasource enablement in index pattern management Signed-off-by: Kristen Tian * [MD] Add client management module and register `getClient()` to route handler context (#2121) * Add client management module and register `getClient()` interface to route handler context Signed-off-by: Zhongnan Su Signed-off-by: Kristen Tian * Make step info in index pattern creation dynamic (#2164) Signed-off-by: Kristen Tian Signed-off-by: Kristen Tian * Refactor for credential listing page & add loading effect (#2142) Signed-off-by: Yibo Wang Signed-off-by: Yibo Wang Signed-off-by: Kristen Tian * Integration sequal - replace data client placeholders (#2167) Signed-off-by: Kristen Tian Signed-off-by: Kristen Tian * Integrate with crypto module to decrpt password (#2170) Signed-off-by: Zhongnan Su Signed-off-by: Kristen Tian * [MD] noAuth integration, credential & endpoint validation (#2165) * noAuth integration, credential & endpoint validation Signed-off-by: mpabba3003 * Refactoring validation message Signed-off-by: mpabba3003 * Adding back accidentally deleted file home/tutorials/haproxy_metrics/index.ts Signed-off-by: mpabba3003 Signed-off-by: mpabba3003 Signed-off-by: Kristen Tian * [MD] Refactor search strategy to conditionally user datasource client (#2171) Signed-off-by: Kristen Tian * adding relation between credential selection and no auth checkbox (#2175) Signed-off-by: mpabba3003 Signed-off-by: mpabba3003 Signed-off-by: Kristen Tian * Update getIndicesViaSearch with datasource (#2176) Signed-off-by: Kristen Tian Signed-off-by: Kristen Tian * Add null header to child client spawn (#2188) Signed-off-by: Su Signed-off-by: Kristen Tian * [MD][IP]Update button position & Add UT & Add skip option (#2195) * Add UT - index pattern related Signed-off-by: Kristen Tian * Update button position Signed-off-by: Kristen Tian * Add skip option to allow use default os data source Signed-off-by: Kristen Tian Signed-off-by: Kristen Tian * [MD]Refactor layout and validate input fields for listing and create pages (#2202) Signed-off-by: Louis Chu Signed-off-by: Louis Chu Signed-off-by: Kristen Tian * Enable datasource link in saveObjectManagement (#2209) Signed-off-by: Kristen Tian Signed-off-by: Kristen Tian * Update configure data source per UX input (#2235) Signed-off-by: Kristen Tian Signed-off-by: Kristen Tian * Enable data source audit log to file (#2215) Signed-off-by: Kristen Tian Signed-off-by: Kristen Tian * [MD] Refactor of credential editing page layout & refactor backend field validation method (#2222) * Refactor of credential editing page layout & refactor backend field validation method * Resolved the comments & fix the multiple call for one operation Signed-off-by: Yibo Wang Signed-off-by: Yibo Wang Signed-off-by: Kristen Tian * Tweak fetch data back to original (#2238) Signed-off-by: Kristen Tian Signed-off-by: Kristen Tian * [MD] Revamped UX for data source management (#2239) * revamped UX for data source management Signed-off-by: mpabba3003 * refactored datasource screens as per PR comments Signed-off-by: mpabba3003 Signed-off-by: mpabba3003 Signed-off-by: Kristen Tian * [MD] Credentials security redesign (#2253) 1. Data model changes for data source saved object 2. Server side changes for data source saved object a. Implement data_source_saved_objects_client_wrapper to integrate with CryptographyClient for password encryption / decryption. b. Change data_source_service to fetch credentials directly from data source (still decrypt via CryptographyClient) c. Fix unit tests accordingly Signed-off-by: Louis Chu Signed-off-by: Kristen Tian * Add step data source UI test (#2264) Signed-off-by: Kristen Tian Signed-off-by: Kristen Tian * [MD]Improve datasource server side error handling (#2236) Signed-off-by: Su Signed-off-by: Kristen Tian * [MD] Datasource management new API change integrations (#2282) * changing datasource management design to integrate with new API changes Signed-off-by: mpabba3003 * Update edit_data_source_form.tsx moving masked password to constants Signed-off-by: mpabba3003 Signed-off-by: Kristen Tian * Update stream test to bypass CI env generate domain attribute Signed-off-by: Kristen Tian * Delete credential management Signed-off-by: Kristen Tian * Address comments Signed-off-by: Kristen Tian * [MD] Datasource Management - Create data source - Unit tests (#2341) * Unit test cases for data source management - create Signed-off-by: mpabba3003 * adding tests to utils.ts & changing it to test Signed-off-by: mpabba3003 Signed-off-by: mpabba3003 Signed-off-by: Kristen Tian * Fix update data source & block update endpint (#2364) Signed-off-by: Kristen Tian Signed-off-by: Kristen Tian * MD datasource management-datasource table-UTs (#2350) Signed-off-by: Yibo Wang Signed-off-by: Yibo Wang Signed-off-by: Kristen Tian * reafctor based on PR 2334 comments to merge to main (#2375) Signed-off-by: mpabba3003 Signed-off-by: mpabba3003 Signed-off-by: Kristen Tian * removing invalid urls as CI fails on link checker for inavlid urls in git (#2376) Signed-off-by: mpabba3003 Signed-off-by: mpabba3003 Signed-off-by: Kristen Tian * Address comments Signed-off-by: Kristen Tian * Fix DS snapshot test Signed-off-by: Kristen Tian * Add https://test.com/ to lychee exclude Signed-off-by: Kristen Tian * Address comments Signed-off-by: Kristen Tian * Remove unnessacry check Signed-off-by: Kristen Tian * Remove not needed check Signed-off-by: Kristen Tian Signed-off-by: Kristen Tian Signed-off-by: Zhongnan Su Signed-off-by: Louis Chu Signed-off-by: Yibo Wang Signed-off-by: mpabba3003 Signed-off-by: Su Co-authored-by: Louis Chu Co-authored-by: Zhongnan Su Co-authored-by: Yan Zeng <46499415+zengyan-amazon@users.noreply.github.com> Co-authored-by: Manideep Pabba <109986843+mpabba3003@users.noreply.github.com> Co-authored-by: Yibo Wang <109543558+yibow98@users.noreply.github.com> --- .github/workflows/pr_check_workflow.yml | 22 +- .lycheeexclude | 1 + package.json | 1 + .../src/types/stream_type.test.ts | 19 +- src/core/server/http/router/router.ts | 12 + src/dev/jest/config.js | 3 + .../stubbed_saved_object_index_pattern.ts | 16 +- .../__snapshots__/index_pattern.test.ts.snap | 643 ++ .../__snapshots__/index_patterns.test.ts.snap | 27 + .../index_patterns/index_pattern.test.ts | 68 + .../index_patterns/index_pattern.ts | 24 +- .../index_patterns/index_patterns.test.ts | 26 + .../index_patterns/index_patterns.ts | 12 +- .../data/common/index_patterns/types.ts | 7 + .../common/search/opensearch_search/types.ts | 1 + .../search_source/search_source.test.ts | 20 + .../search/search_source/search_source.ts | 12 +- src/plugins/data/opensearch_dashboards.json | 7 +- .../index_patterns_api_client.ts | 7 +- .../fetcher/index_patterns_fetcher.ts | 6 +- .../field_capabilities/field_capabilities.ts | 4 +- .../fetcher/lib/opensearch_api.ts | 33 +- .../fetcher/lib/resolve_time_pattern.ts | 7 +- .../data/server/index_patterns/routes.ts | 16 +- .../search/opensearch_search/decide_client.ts | 19 + .../opensearch_search_strategy.test.ts | 64 +- .../opensearch_search_strategy.ts | 8 +- src/plugins/data_source/README.md | 13 + src/plugins/data_source/audit_config.ts | 42 + .../data_source/common/data_sources/index.ts | 6 + .../data_source/common/data_sources/types.ts | 26 + src/plugins/data_source/common/index.ts | 8 + src/plugins/data_source/config.ts | 42 + .../data_source/opensearch_dashboards.json | 9 + src/plugins/data_source/public/index.ts | 14 + src/plugins/data_source/public/plugin.ts | 20 + src/plugins/data_source/public/types.ts | 10 + .../server/audit/logging_auditor.ts | 23 + .../server/client/client_config.test.ts | 29 + .../server/client/client_config.ts | 29 + .../server/client/client_pool.test.ts | 39 + .../data_source/server/client/client_pool.ts | 76 + .../client/configure_client.test.mocks.ts | 18 + .../server/client/configure_client.test.ts | 130 + .../server/client/configure_client.ts | 130 + .../data_source/server/client/index.ts | 7 + .../cryptography/cryptography_client.test.ts | 117 + .../cryptography/cryptography_client.ts | 70 + .../data_source/server/cryptography/index.ts | 6 + .../server/data_source_service.test.ts | 38 + .../data_source/server/data_source_service.ts | 56 + src/plugins/data_source/server/index.ts | 18 + .../data_source/server/lib/error.test.ts | 40 + src/plugins/data_source/server/lib/error.ts | 21 + src/plugins/data_source/server/plugin.ts | 170 + .../server/saved_objects/data_source.ts | 37 + ...ata_source_saved_objects_client_wrapper.ts | 268 + .../data_source/server/saved_objects/index.ts | 7 + src/plugins/data_source/server/types.ts | 22 + src/plugins/data_source_management/README.md | 11 + .../data_source_management/common/index.ts | 7 + .../opensearch_dashboards.json | 9 + .../public/components/breadcrumbs.test.ts | 29 + .../public/components/breadcrumbs.ts | 40 + .../__snapshots__/create_button.test.tsx.snap | 16 + .../create_button/create_button.test.tsx | 31 + .../create_button/create_button.tsx | 30 + .../public/components/create_button/index.tsx | 6 + .../create_data_source_wizard.test.tsx.snap | 1058 ++++ .../create_data_source_form.test.tsx.snap | 5176 +++++++++++++++++ .../create_data_source_form.test.tsx | 162 + .../create_form/create_data_source_form.tsx | 323 + .../components/create_form/index.ts | 6 + .../header/__snapshots__/header.test.tsx.snap | 18 + .../components/header/header.test.tsx | 29 + .../components/header/header.tsx | 53 + .../components/header/index.ts | 6 + .../create_data_source_wizard.test.tsx | 81 + .../create_data_source_wizard.tsx | 100 + .../create_data_source_wizard/index.ts | 6 + .../data_source_table.test.tsx.snap | 2201 +++++++ .../data_source_table.test.tsx | 178 + .../data_source_table/data_source_table.tsx | 335 ++ .../components/data_source_table/index.ts | 6 + .../edit_form/edit_data_source_form.tsx | 539 ++ .../components/header/header.tsx | 99 + .../components/header/index.ts | 6 + .../components/update_password_modal/index.ts | 6 + .../update_password_modal.tsx | 146 + .../edit_data_source/edit_data_source.tsx | 162 + .../components/edit_data_source/index.ts | 6 + .../__snapshots__/loading_mask.test.tsx.snap | 9 + .../public/components/loading_mask/index.ts | 6 + .../loading_mask/loading_mask.test.tsx | 15 + .../components/loading_mask/loading_mask.tsx | 15 + .../public/components/text_content/index.ts | 6 + .../components/text_content/text_content.ts | 283 + .../public/components/utils.test.ts | 175 + .../public/components/utils.ts | 89 + .../datasource_form_validation.test.ts | 74 + .../validation/datasource_form_validation.ts | 122 + .../public/components/validation/index.ts | 6 + .../data_source_management/public/index.ts | 13 + .../public/management_app/index.ts | 6 + .../mount_management_section.tsx | 70 + .../data_source_management/public/mocks.ts | 180 + .../data_source_management/public/plugin.ts | 42 + .../data_source_management/public/types.ts | 83 + .../opensearch_dashboards.json | 3 +- .../create_index_pattern_wizard.test.tsx.snap | 36 +- .../step_data_source.test.tsx.snap | 19 + .../header/__snapshots__/header.test.tsx.snap | 238 + .../components/header/header.scss | 3 + .../components/header/header.test.tsx | 98 + .../components/header/header.tsx | 168 + .../components/header/index.ts | 6 + .../components/step_data_source/index.ts | 6 + .../step_data_source.test.tsx | 21 + .../step_data_source/step_data_source.tsx | 46 + .../header/__snapshots__/header.test.tsx.snap | 18 +- .../components/header/header.test.tsx | 2 + .../components/header/header.tsx | 9 +- .../step_index_pattern/step_index_pattern.tsx | 70 +- .../header/__snapshots__/header.test.tsx.snap | 9 +- .../components/header/header.test.tsx | 8 +- .../components/header/header.tsx | 10 +- .../step_time_field/step_time_field.tsx | 16 +- .../create_index_pattern_wizard.test.tsx | 5 +- .../create_index_pattern_wizard.tsx | 101 +- .../lib/creation_flow.test.ts | 44 + .../lib/creation_flow.ts | 79 + .../lib/get_indices.test.ts | 42 +- .../lib/get_indices.ts | 81 +- .../create_index_pattern_wizard/lib/index.ts | 2 + .../create_index_pattern_wizard/types.ts | 5 + .../index_pattern_table.tsx | 43 +- .../mount_management_section.tsx | 4 +- .../index_pattern_management/public/plugin.ts | 2 + .../index_pattern_management/public/types.ts | 4 + .../server/routes/resolve_index.ts | 13 + .../server/capabilities_provider.ts | 1 + yarn.lock | 379 +- 142 files changed, 16139 insertions(+), 153 deletions(-) create mode 100644 src/plugins/data/server/search/opensearch_search/decide_client.ts create mode 100755 src/plugins/data_source/README.md create mode 100644 src/plugins/data_source/audit_config.ts create mode 100644 src/plugins/data_source/common/data_sources/index.ts create mode 100644 src/plugins/data_source/common/data_sources/types.ts create mode 100644 src/plugins/data_source/common/index.ts create mode 100644 src/plugins/data_source/config.ts create mode 100644 src/plugins/data_source/opensearch_dashboards.json create mode 100644 src/plugins/data_source/public/index.ts create mode 100644 src/plugins/data_source/public/plugin.ts create mode 100644 src/plugins/data_source/public/types.ts create mode 100644 src/plugins/data_source/server/audit/logging_auditor.ts create mode 100644 src/plugins/data_source/server/client/client_config.test.ts create mode 100644 src/plugins/data_source/server/client/client_config.ts create mode 100644 src/plugins/data_source/server/client/client_pool.test.ts create mode 100644 src/plugins/data_source/server/client/client_pool.ts create mode 100644 src/plugins/data_source/server/client/configure_client.test.mocks.ts create mode 100644 src/plugins/data_source/server/client/configure_client.test.ts create mode 100644 src/plugins/data_source/server/client/configure_client.ts create mode 100644 src/plugins/data_source/server/client/index.ts create mode 100644 src/plugins/data_source/server/cryptography/cryptography_client.test.ts create mode 100644 src/plugins/data_source/server/cryptography/cryptography_client.ts create mode 100644 src/plugins/data_source/server/cryptography/index.ts create mode 100644 src/plugins/data_source/server/data_source_service.test.ts create mode 100644 src/plugins/data_source/server/data_source_service.ts create mode 100644 src/plugins/data_source/server/index.ts create mode 100644 src/plugins/data_source/server/lib/error.test.ts create mode 100644 src/plugins/data_source/server/lib/error.ts create mode 100644 src/plugins/data_source/server/plugin.ts create mode 100644 src/plugins/data_source/server/saved_objects/data_source.ts create mode 100644 src/plugins/data_source/server/saved_objects/data_source_saved_objects_client_wrapper.ts create mode 100644 src/plugins/data_source/server/saved_objects/index.ts create mode 100644 src/plugins/data_source/server/types.ts create mode 100755 src/plugins/data_source_management/README.md create mode 100644 src/plugins/data_source_management/common/index.ts create mode 100644 src/plugins/data_source_management/opensearch_dashboards.json create mode 100644 src/plugins/data_source_management/public/components/breadcrumbs.test.ts create mode 100644 src/plugins/data_source_management/public/components/breadcrumbs.ts create mode 100644 src/plugins/data_source_management/public/components/create_button/__snapshots__/create_button.test.tsx.snap create mode 100644 src/plugins/data_source_management/public/components/create_button/create_button.test.tsx create mode 100644 src/plugins/data_source_management/public/components/create_button/create_button.tsx create mode 100644 src/plugins/data_source_management/public/components/create_button/index.tsx create mode 100644 src/plugins/data_source_management/public/components/create_data_source_wizard/__snapshots__/create_data_source_wizard.test.tsx.snap create mode 100644 src/plugins/data_source_management/public/components/create_data_source_wizard/components/create_form/__snapshots__/create_data_source_form.test.tsx.snap create mode 100644 src/plugins/data_source_management/public/components/create_data_source_wizard/components/create_form/create_data_source_form.test.tsx create mode 100644 src/plugins/data_source_management/public/components/create_data_source_wizard/components/create_form/create_data_source_form.tsx create mode 100644 src/plugins/data_source_management/public/components/create_data_source_wizard/components/create_form/index.ts create mode 100644 src/plugins/data_source_management/public/components/create_data_source_wizard/components/header/__snapshots__/header.test.tsx.snap create mode 100644 src/plugins/data_source_management/public/components/create_data_source_wizard/components/header/header.test.tsx create mode 100644 src/plugins/data_source_management/public/components/create_data_source_wizard/components/header/header.tsx create mode 100644 src/plugins/data_source_management/public/components/create_data_source_wizard/components/header/index.ts create mode 100644 src/plugins/data_source_management/public/components/create_data_source_wizard/create_data_source_wizard.test.tsx create mode 100644 src/plugins/data_source_management/public/components/create_data_source_wizard/create_data_source_wizard.tsx create mode 100644 src/plugins/data_source_management/public/components/create_data_source_wizard/index.ts create mode 100644 src/plugins/data_source_management/public/components/data_source_table/__snapshots__/data_source_table.test.tsx.snap create mode 100644 src/plugins/data_source_management/public/components/data_source_table/data_source_table.test.tsx create mode 100644 src/plugins/data_source_management/public/components/data_source_table/data_source_table.tsx create mode 100644 src/plugins/data_source_management/public/components/data_source_table/index.ts create mode 100644 src/plugins/data_source_management/public/components/edit_data_source/components/edit_form/edit_data_source_form.tsx create mode 100644 src/plugins/data_source_management/public/components/edit_data_source/components/header/header.tsx create mode 100644 src/plugins/data_source_management/public/components/edit_data_source/components/header/index.ts create mode 100644 src/plugins/data_source_management/public/components/edit_data_source/components/update_password_modal/index.ts create mode 100644 src/plugins/data_source_management/public/components/edit_data_source/components/update_password_modal/update_password_modal.tsx create mode 100644 src/plugins/data_source_management/public/components/edit_data_source/edit_data_source.tsx create mode 100644 src/plugins/data_source_management/public/components/edit_data_source/index.ts create mode 100644 src/plugins/data_source_management/public/components/loading_mask/__snapshots__/loading_mask.test.tsx.snap create mode 100644 src/plugins/data_source_management/public/components/loading_mask/index.ts create mode 100644 src/plugins/data_source_management/public/components/loading_mask/loading_mask.test.tsx create mode 100644 src/plugins/data_source_management/public/components/loading_mask/loading_mask.tsx create mode 100644 src/plugins/data_source_management/public/components/text_content/index.ts create mode 100644 src/plugins/data_source_management/public/components/text_content/text_content.ts create mode 100644 src/plugins/data_source_management/public/components/utils.test.ts create mode 100644 src/plugins/data_source_management/public/components/utils.ts create mode 100644 src/plugins/data_source_management/public/components/validation/datasource_form_validation.test.ts create mode 100644 src/plugins/data_source_management/public/components/validation/datasource_form_validation.ts create mode 100644 src/plugins/data_source_management/public/components/validation/index.ts create mode 100644 src/plugins/data_source_management/public/index.ts create mode 100644 src/plugins/data_source_management/public/management_app/index.ts create mode 100644 src/plugins/data_source_management/public/management_app/mount_management_section.tsx create mode 100644 src/plugins/data_source_management/public/mocks.ts create mode 100644 src/plugins/data_source_management/public/plugin.ts create mode 100644 src/plugins/data_source_management/public/types.ts create mode 100644 src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_data_source/__snapshots__/step_data_source.test.tsx.snap create mode 100644 src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_data_source/components/header/__snapshots__/header.test.tsx.snap create mode 100644 src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_data_source/components/header/header.scss create mode 100644 src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_data_source/components/header/header.test.tsx create mode 100644 src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_data_source/components/header/header.tsx create mode 100644 src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_data_source/components/header/index.ts create mode 100644 src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_data_source/index.ts create mode 100644 src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_data_source/step_data_source.test.tsx create mode 100644 src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_data_source/step_data_source.tsx create mode 100644 src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/lib/creation_flow.test.ts create mode 100644 src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/lib/creation_flow.ts diff --git a/.github/workflows/pr_check_workflow.yml b/.github/workflows/pr_check_workflow.yml index 7a8609b4725..6f95ced784a 100644 --- a/.github/workflows/pr_check_workflow.yml +++ b/.github/workflows/pr_check_workflow.yml @@ -38,7 +38,7 @@ jobs: - name: Setup Node uses: actions/setup-node@v2 with: - node-version-file: ".nvmrc" + node-version-file: '.nvmrc' registry-url: 'https://registry.npmjs.org' - name: Setup Yarn @@ -71,7 +71,7 @@ jobs: image: docker://opensearchstaging/ci-runner:ci-runner-rockylinux8-opensearch-dashboards-integtest-v2 options: --user 1001 name: Run functional tests - strategy: + strategy: matrix: group: [ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13 ] steps: @@ -83,7 +83,7 @@ jobs: - name: Setup Node uses: actions/setup-node@v2 with: - node-version-file: ".nvmrc" + node-version-file: '.nvmrc' registry-url: 'https://registry.npmjs.org' - name: Setup Yarn @@ -119,7 +119,7 @@ jobs: defaults: run: working-directory: ./artifacts - strategy: + strategy: matrix: include: - name: Linux x64 @@ -139,7 +139,7 @@ jobs: - name: Setup Node uses: actions/setup-node@v2 with: - node-version-file: "./artifacts/.nvmrc" + node-version-file: './artifacts/.nvmrc' registry-url: 'https://registry.npmjs.org' - name: Setup Yarn @@ -167,9 +167,9 @@ jobs: name: ${{ matrix.suffix }}-${{ env.VERSION }} path: ./artifacts/target/${{ env.ARTIFACT_BUILD_NAME }} retention-days: 1 - + bwc-tests: - needs: [ build-min-artifact-tests ] + needs: [build-min-artifact-tests] runs-on: ubuntu-latest container: image: docker://opensearchstaging/ci-runner:ci-runner-rockylinux8-opensearch-dashboards-integtest-v2 @@ -178,7 +178,7 @@ jobs: defaults: run: working-directory: ./artifacts - strategy: + strategy: matrix: version: [ osd-1.3.2, osd-2.0.0, osd-2.1.0, osd-2.2.0 ] steps: @@ -193,7 +193,7 @@ jobs: - name: Setup Node uses: actions/setup-node@v2 with: - node-version-file: "./artifacts/.nvmrc" + node-version-file: './artifacts/.nvmrc' registry-url: 'https://registry.npmjs.org' - name: Setup Yarn @@ -215,7 +215,7 @@ jobs: if curl -I -L ${{ env.OPENSEARCH_URL }}; then echo "::set-output name=version-exists::true" fi - + - name: Skipping tests if: steps.verify-opensearch-exists.outputs.version-exists != 'true' run: echo Tests were skipped because an OpenSearch release build does not exist for this version yet! @@ -223,7 +223,7 @@ jobs: - name: Setting environment variable to run tests for ${{ matrix.version }} if: steps.verify-opensearch-exists.outputs.version-exists == 'true' run: echo "BWC_VERSIONS=${{ matrix.version }}" >> $GITHUB_ENV - + - name: Download OpenSearch Dashboards uses: actions/download-artifact@v3 id: download diff --git a/.lycheeexclude b/.lycheeexclude index a16068878e1..b82de263d3d 100644 --- a/.lycheeexclude +++ b/.lycheeexclude @@ -35,6 +35,7 @@ http://noone.nowhere.none/ http://bar http://foo http://test.com/ +https://test.com/ https://manifest.foobar https://files.foobar/ https://tiles.foobar/ diff --git a/package.json b/package.json index b5086af4fe5..370ccbff439 100644 --- a/package.json +++ b/package.json @@ -107,6 +107,7 @@ ] }, "dependencies": { + "@aws-crypto/client-node": "^3.1.1", "@elastic/datemath": "5.0.3", "@elastic/eui": "npm:@opensearch-project/oui@1.0.0", "@elastic/good": "^9.0.1-kibana3", diff --git a/packages/osd-config-schema/src/types/stream_type.test.ts b/packages/osd-config-schema/src/types/stream_type.test.ts index 8eb83306869..dd12897a86f 100644 --- a/packages/osd-config-schema/src/types/stream_type.test.ts +++ b/packages/osd-config-schema/src/types/stream_type.test.ts @@ -68,13 +68,18 @@ test('includes namespace in failure', () => { describe('#defaultValue', () => { test('returns default when undefined', () => { const value = new Stream(); - expect(schema.stream({ defaultValue: value }).validate(undefined)).toMatchInlineSnapshot(` - Stream { - "_events": Object {}, - "_eventsCount": 0, - "_maxListeners": undefined, - } - `); + expect(schema.stream({ defaultValue: value }).validate(undefined)).toHaveProperty( + '_events', + {} + ); + expect(schema.stream({ defaultValue: value }).validate(undefined)).toHaveProperty( + '_eventsCount', + 0 + ); + expect(schema.stream({ defaultValue: value }).validate(undefined)).toHaveProperty( + '_maxListeners', + undefined + ); }); test('returns value when specified', () => { diff --git a/src/core/server/http/router/router.ts b/src/core/server/http/router/router.ts index a426eef40e7..752706879d0 100644 --- a/src/core/server/http/router/router.ts +++ b/src/core/server/http/router/router.ts @@ -300,11 +300,23 @@ export class Router implements IRouter { if (LegacyOpenSearchErrorHelpers.isNotAuthorizedError(e)) { return e; } + + if (isDataSourceConfigError(e)) { + return hapiResponseAdapter.handle( + opensearchDashboardsResponseFactory.badRequest({ body: e.message }) + ); + } + // TODO: add legacy data source client config error handling + return hapiResponseAdapter.toInternalError(); } } } +const isDataSourceConfigError = (error: any) => { + return error.constructor.name === 'DataSourceConfigError' && error.statusCode === 400; +}; + const convertOpenSearchUnauthorized = ( e: OpenSearchNotAuthorizedError ): ErrorHttpResponseOptions => { diff --git a/src/dev/jest/config.js b/src/dev/jest/config.js index 295a1a27674..6384fb98692 100644 --- a/src/dev/jest/config.js +++ b/src/dev/jest/config.js @@ -110,4 +110,7 @@ export default { '/node_modules/enzyme-to-json/serializer', ], reporters: ['default', '/src/dev/jest/junit_reporter.js'], + globals: { + Uint8Array: Uint8Array, + }, }; diff --git a/src/fixtures/stubbed_saved_object_index_pattern.ts b/src/fixtures/stubbed_saved_object_index_pattern.ts index 20a0474e1bd..1100a987e28 100644 --- a/src/fixtures/stubbed_saved_object_index_pattern.ts +++ b/src/fixtures/stubbed_saved_object_index_pattern.ts @@ -33,8 +33,8 @@ import stubbedLogstashFields from './logstash_fields'; const mockLogstashFields = stubbedLogstashFields(); -export function stubbedSavedObjectIndexPattern(id: string | null = null) { - return { +export function stubbedSavedObjectIndexPattern(id: string | null = null, withDataSource?: false) { + const indexPattern: any = { id, type: 'index-pattern', attributes: { @@ -45,4 +45,16 @@ export function stubbedSavedObjectIndexPattern(id: string | null = null) { }, version: '2', }; + + if (withDataSource) { + indexPattern.reference = [ + { + id: 'id', + name: 'name', + type: 'data-source', + }, + ]; + } + + return indexPattern; } diff --git a/src/plugins/data/common/index_patterns/index_patterns/__snapshots__/index_pattern.test.ts.snap b/src/plugins/data/common/index_patterns/index_patterns/__snapshots__/index_pattern.test.ts.snap index ed84aceb60e..32dd9fa3fa2 100644 --- a/src/plugins/data/common/index_patterns/index_patterns/__snapshots__/index_pattern.test.ts.snap +++ b/src/plugins/data/common/index_patterns/index_patterns/__snapshots__/index_pattern.test.ts.snap @@ -2,6 +2,649 @@ exports[`IndexPattern toSpec should match snapshot 1`] = ` Object { + "dataSourceRef": undefined, + "fields": Object { + "@tags": Object { + "aggregatable": true, + "conflictDescriptions": undefined, + "count": 0, + "esTypes": Array [ + "keyword", + ], + "format": Object { + "id": "number", + "params": Object { + "pattern": "$0,0.[00]", + }, + }, + "lang": undefined, + "name": "@tags", + "readFromDocValues": true, + "script": undefined, + "scripted": false, + "searchable": true, + "subType": undefined, + "type": "string", + }, + "@timestamp": Object { + "aggregatable": true, + "conflictDescriptions": undefined, + "count": 30, + "esTypes": Array [ + "date", + ], + "format": Object { + "id": "number", + "params": Object { + "pattern": "$0,0.[00]", + }, + }, + "lang": undefined, + "name": "@timestamp", + "readFromDocValues": true, + "script": undefined, + "scripted": false, + "searchable": true, + "subType": undefined, + "type": "date", + }, + "_id": Object { + "aggregatable": true, + "conflictDescriptions": undefined, + "count": 0, + "esTypes": Array [ + "_id", + ], + "format": Object { + "id": "number", + "params": Object { + "pattern": "$0,0.[00]", + }, + }, + "lang": undefined, + "name": "_id", + "readFromDocValues": false, + "script": undefined, + "scripted": false, + "searchable": true, + "subType": undefined, + "type": "string", + }, + "_source": Object { + "aggregatable": true, + "conflictDescriptions": undefined, + "count": 0, + "esTypes": Array [ + "_source", + ], + "format": Object { + "id": "number", + "params": Object { + "pattern": "$0,0.[00]", + }, + }, + "lang": undefined, + "name": "_source", + "readFromDocValues": false, + "script": undefined, + "scripted": false, + "searchable": true, + "subType": undefined, + "type": "_source", + }, + "_type": Object { + "aggregatable": true, + "conflictDescriptions": undefined, + "count": 0, + "esTypes": Array [ + "_type", + ], + "format": Object { + "id": "number", + "params": Object { + "pattern": "$0,0.[00]", + }, + }, + "lang": undefined, + "name": "_type", + "readFromDocValues": false, + "script": undefined, + "scripted": false, + "searchable": true, + "subType": undefined, + "type": "string", + }, + "area": Object { + "aggregatable": true, + "conflictDescriptions": undefined, + "count": 0, + "esTypes": Array [ + "geo_shape", + ], + "format": Object { + "id": "number", + "params": Object { + "pattern": "$0,0.[00]", + }, + }, + "lang": undefined, + "name": "area", + "readFromDocValues": false, + "script": undefined, + "scripted": false, + "searchable": true, + "subType": undefined, + "type": "geo_shape", + }, + "bytes": Object { + "aggregatable": true, + "conflictDescriptions": undefined, + "count": 10, + "esTypes": Array [ + "long", + ], + "format": Object { + "id": "number", + "params": Object { + "pattern": "$0,0.[00]", + }, + }, + "lang": undefined, + "name": "bytes", + "readFromDocValues": true, + "script": undefined, + "scripted": false, + "searchable": true, + "subType": undefined, + "type": "number", + }, + "custom_user_field": Object { + "aggregatable": true, + "conflictDescriptions": undefined, + "count": 0, + "esTypes": Array [ + "conflict", + ], + "format": Object { + "id": "number", + "params": Object { + "pattern": "$0,0.[00]", + }, + }, + "lang": undefined, + "name": "custom_user_field", + "readFromDocValues": true, + "script": undefined, + "scripted": false, + "searchable": true, + "subType": undefined, + "type": "conflict", + }, + "extension": Object { + "aggregatable": true, + "conflictDescriptions": undefined, + "count": 0, + "esTypes": Array [ + "text", + ], + "format": Object { + "id": "number", + "params": Object { + "pattern": "$0,0.[00]", + }, + }, + "lang": undefined, + "name": "extension", + "readFromDocValues": false, + "script": undefined, + "scripted": false, + "searchable": true, + "subType": undefined, + "type": "string", + }, + "extension.keyword": Object { + "aggregatable": true, + "conflictDescriptions": undefined, + "count": 0, + "esTypes": Array [ + "keyword", + ], + "format": Object { + "id": "number", + "params": Object { + "pattern": "$0,0.[00]", + }, + }, + "lang": undefined, + "name": "extension.keyword", + "readFromDocValues": true, + "script": undefined, + "scripted": false, + "searchable": true, + "subType": Object { + "multi": Object { + "parent": "extension", + }, + }, + "type": "string", + }, + "geo.coordinates": Object { + "aggregatable": true, + "conflictDescriptions": undefined, + "count": 0, + "esTypes": Array [ + "geo_point", + ], + "format": Object { + "id": "number", + "params": Object { + "pattern": "$0,0.[00]", + }, + }, + "lang": undefined, + "name": "geo.coordinates", + "readFromDocValues": true, + "script": undefined, + "scripted": false, + "searchable": true, + "subType": undefined, + "type": "geo_point", + }, + "geo.src": Object { + "aggregatable": true, + "conflictDescriptions": undefined, + "count": 0, + "esTypes": Array [ + "keyword", + ], + "format": Object { + "id": "number", + "params": Object { + "pattern": "$0,0.[00]", + }, + }, + "lang": undefined, + "name": "geo.src", + "readFromDocValues": true, + "script": undefined, + "scripted": false, + "searchable": true, + "subType": undefined, + "type": "string", + }, + "hashed": Object { + "aggregatable": false, + "conflictDescriptions": undefined, + "count": 0, + "esTypes": Array [ + "murmur3", + ], + "format": Object { + "id": "number", + "params": Object { + "pattern": "$0,0.[00]", + }, + }, + "lang": undefined, + "name": "hashed", + "readFromDocValues": false, + "script": undefined, + "scripted": false, + "searchable": true, + "subType": undefined, + "type": "murmur3", + }, + "ip": Object { + "aggregatable": true, + "conflictDescriptions": undefined, + "count": 0, + "esTypes": Array [ + "ip", + ], + "format": Object { + "id": "number", + "params": Object { + "pattern": "$0,0.[00]", + }, + }, + "lang": undefined, + "name": "ip", + "readFromDocValues": true, + "script": undefined, + "scripted": false, + "searchable": true, + "subType": undefined, + "type": "ip", + }, + "machine.os": Object { + "aggregatable": true, + "conflictDescriptions": undefined, + "count": 0, + "esTypes": Array [ + "text", + ], + "format": Object { + "id": "number", + "params": Object { + "pattern": "$0,0.[00]", + }, + }, + "lang": undefined, + "name": "machine.os", + "readFromDocValues": false, + "script": undefined, + "scripted": false, + "searchable": true, + "subType": undefined, + "type": "string", + }, + "machine.os.raw": Object { + "aggregatable": true, + "conflictDescriptions": undefined, + "count": 0, + "esTypes": Array [ + "keyword", + ], + "format": Object { + "id": "number", + "params": Object { + "pattern": "$0,0.[00]", + }, + }, + "lang": undefined, + "name": "machine.os.raw", + "readFromDocValues": true, + "script": undefined, + "scripted": false, + "searchable": true, + "subType": Object { + "multi": Object { + "parent": "machine.os", + }, + }, + "type": "string", + }, + "non-filterable": Object { + "aggregatable": true, + "conflictDescriptions": undefined, + "count": 0, + "esTypes": Array [ + "text", + ], + "format": Object { + "id": "number", + "params": Object { + "pattern": "$0,0.[00]", + }, + }, + "lang": undefined, + "name": "non-filterable", + "readFromDocValues": false, + "script": undefined, + "scripted": false, + "searchable": false, + "subType": undefined, + "type": "string", + }, + "non-sortable": Object { + "aggregatable": false, + "conflictDescriptions": undefined, + "count": 0, + "esTypes": Array [ + "text", + ], + "format": Object { + "id": "number", + "params": Object { + "pattern": "$0,0.[00]", + }, + }, + "lang": undefined, + "name": "non-sortable", + "readFromDocValues": false, + "script": undefined, + "scripted": false, + "searchable": false, + "subType": undefined, + "type": "string", + }, + "phpmemory": Object { + "aggregatable": true, + "conflictDescriptions": undefined, + "count": 0, + "esTypes": Array [ + "integer", + ], + "format": Object { + "id": "number", + "params": Object { + "pattern": "$0,0.[00]", + }, + }, + "lang": undefined, + "name": "phpmemory", + "readFromDocValues": true, + "script": undefined, + "scripted": false, + "searchable": true, + "subType": undefined, + "type": "number", + }, + "point": Object { + "aggregatable": true, + "conflictDescriptions": undefined, + "count": 0, + "esTypes": Array [ + "geo_point", + ], + "format": Object { + "id": "number", + "params": Object { + "pattern": "$0,0.[00]", + }, + }, + "lang": undefined, + "name": "point", + "readFromDocValues": true, + "script": undefined, + "scripted": false, + "searchable": true, + "subType": undefined, + "type": "geo_point", + }, + "request_body": Object { + "aggregatable": true, + "conflictDescriptions": undefined, + "count": 0, + "esTypes": Array [ + "attachment", + ], + "format": Object { + "id": "number", + "params": Object { + "pattern": "$0,0.[00]", + }, + }, + "lang": undefined, + "name": "request_body", + "readFromDocValues": true, + "script": undefined, + "scripted": false, + "searchable": true, + "subType": undefined, + "type": "attachment", + }, + "script date": Object { + "aggregatable": true, + "conflictDescriptions": undefined, + "count": 0, + "esTypes": Array [ + "date", + ], + "format": Object { + "id": "number", + "params": Object { + "pattern": "$0,0.[00]", + }, + }, + "lang": "painless", + "name": "script date", + "readFromDocValues": false, + "script": "1234", + "scripted": true, + "searchable": true, + "subType": undefined, + "type": "date", + }, + "script murmur3": Object { + "aggregatable": true, + "conflictDescriptions": undefined, + "count": 0, + "esTypes": Array [ + "murmur3", + ], + "format": Object { + "id": "number", + "params": Object { + "pattern": "$0,0.[00]", + }, + }, + "lang": "expression", + "name": "script murmur3", + "readFromDocValues": false, + "script": "1234", + "scripted": true, + "searchable": true, + "subType": undefined, + "type": "murmur3", + }, + "script number": Object { + "aggregatable": true, + "conflictDescriptions": undefined, + "count": 0, + "esTypes": Array [ + "long", + ], + "format": Object { + "id": "number", + "params": Object { + "pattern": "$0,0.[00]", + }, + }, + "lang": "expression", + "name": "script number", + "readFromDocValues": false, + "script": "1234", + "scripted": true, + "searchable": true, + "subType": undefined, + "type": "number", + }, + "script string": Object { + "aggregatable": true, + "conflictDescriptions": undefined, + "count": 0, + "esTypes": Array [ + "text", + ], + "format": Object { + "id": "number", + "params": Object { + "pattern": "$0,0.[00]", + }, + }, + "lang": "expression", + "name": "script string", + "readFromDocValues": false, + "script": "'i am a string'", + "scripted": true, + "searchable": true, + "subType": undefined, + "type": "string", + }, + "ssl": Object { + "aggregatable": true, + "conflictDescriptions": undefined, + "count": 20, + "esTypes": Array [ + "boolean", + ], + "format": Object { + "id": "number", + "params": Object { + "pattern": "$0,0.[00]", + }, + }, + "lang": undefined, + "name": "ssl", + "readFromDocValues": true, + "script": undefined, + "scripted": false, + "searchable": true, + "subType": undefined, + "type": "boolean", + }, + "time": Object { + "aggregatable": true, + "conflictDescriptions": undefined, + "count": 30, + "esTypes": Array [ + "date", + ], + "format": Object { + "id": "number", + "params": Object { + "pattern": "$0,0.[00]", + }, + }, + "lang": undefined, + "name": "time", + "readFromDocValues": true, + "script": undefined, + "scripted": false, + "searchable": true, + "subType": undefined, + "type": "date", + }, + "utc_time": Object { + "aggregatable": true, + "conflictDescriptions": undefined, + "count": 0, + "esTypes": Array [ + "date", + ], + "format": Object { + "id": "number", + "params": Object { + "pattern": "$0,0.[00]", + }, + }, + "lang": undefined, + "name": "utc_time", + "readFromDocValues": true, + "script": undefined, + "scripted": false, + "searchable": true, + "subType": undefined, + "type": "date", + }, + }, + "id": "test-pattern", + "sourceFilters": undefined, + "timeFieldName": "timestamp", + "title": "title", + "type": "index-pattern", + "typeMeta": undefined, + "version": "2", +} +`; + +exports[`IndexPatternWithDataSource toSpec should match snapshot 1`] = ` +Object { + "dataSourceRef": Object { + "id": "id", + "type": "data-source", + }, "fields": Object { "@tags": Object { "aggregatable": true, diff --git a/src/plugins/data/common/index_patterns/index_patterns/__snapshots__/index_patterns.test.ts.snap b/src/plugins/data/common/index_patterns/index_patterns/__snapshots__/index_patterns.test.ts.snap index e015fd0b5e1..445ec63df13 100644 --- a/src/plugins/data/common/index_patterns/index_patterns/__snapshots__/index_patterns.test.ts.snap +++ b/src/plugins/data/common/index_patterns/index_patterns/__snapshots__/index_patterns.test.ts.snap @@ -2,6 +2,33 @@ exports[`IndexPatterns savedObjectToSpec 1`] = ` Object { + "dataSourceRef": undefined, + "fields": Object {}, + "id": "id", + "intervalName": undefined, + "sourceFilters": Array [ + Object { + "value": "item1", + }, + Object { + "value": "item2", + }, + ], + "timeFieldName": "@timestamp", + "title": "opensearch-dashboards-*", + "type": "", + "typeMeta": Object {}, + "version": "version", +} +`; + +exports[`IndexPatterns savedObjectToSpecWithDataSource 1`] = ` +Object { + "dataSourceRef": Object { + "id": "id", + "name": "dataSource", + "type": "data-source", + }, "fields": Object {}, "id": "id", "intervalName": undefined, diff --git a/src/plugins/data/common/index_patterns/index_patterns/index_pattern.test.ts b/src/plugins/data/common/index_patterns/index_patterns/index_pattern.test.ts index a5ccece4994..089927dbc1e 100644 --- a/src/plugins/data/common/index_patterns/index_patterns/index_pattern.test.ts +++ b/src/plugins/data/common/index_patterns/index_patterns/index_pattern.test.ts @@ -90,6 +90,24 @@ function create(id: string) { }); } +function createWithDataSource(id: string) { + const { + type, + version, + attributes: { timeFieldName, fields, title }, + reference, + } = stubbedSavedObjectIndexPattern(id, true); + + const dataSourceRef = { id: reference[0].id, type: reference[0].type }; + return new IndexPattern({ + spec: { id, type, version, timeFieldName, fields, title, dataSourceRef }, + savedObjectsClient: {} as any, + fieldFormats: fieldFormatsMock, + shortDotsEnable: false, + metaFields: [], + }); +} + describe('IndexPattern', () => { let indexPattern: IndexPattern; @@ -252,3 +270,53 @@ describe('IndexPattern', () => { }); }); }); + +describe('IndexPatternWithDataSource', () => { + let indexPattern: IndexPattern; + + // create an indexPattern instance for each test + beforeEach(() => { + indexPattern = createWithDataSource('test-pattern'); + }); + + describe('toSpec', () => { + test('should match snapshot', () => { + const formatter = { + toJSON: () => ({ id: 'number', params: { pattern: '$0,0.[00]' } }), + } as FieldFormat; + indexPattern.getFormatterForField = () => formatter; + expect(indexPattern.toSpec()).toMatchSnapshot(); + }); + + test('can restore from spec', () => { + const formatter = { + toJSON: () => ({ id: 'number', params: { pattern: '$0,0.[00]' } }), + } as FieldFormat; + indexPattern.getFormatterForField = () => formatter; + const spec = indexPattern.toSpec(); + const restoredPattern = new IndexPattern({ + spec, + savedObjectsClient: {} as any, + fieldFormats: fieldFormatsMock, + shortDotsEnable: false, + metaFields: [], + }); + expect(restoredPattern.id).toEqual(indexPattern.id); + expect(restoredPattern.title).toEqual(indexPattern.title); + expect(restoredPattern.timeFieldName).toEqual(indexPattern.timeFieldName); + expect(restoredPattern.fields.length).toEqual(indexPattern.fields.length); + expect(restoredPattern.fieldFormatMap.bytes instanceof MockFieldFormatter).toEqual(true); + expect(restoredPattern.dataSourceRef).toEqual(indexPattern.dataSourceRef); + }); + }); + + describe('getSaveObjectReference', () => { + test('should get index pattern saved object reference', function () { + expect(indexPattern.getSaveObjectReference()[0]?.id).toEqual(indexPattern.dataSourceRef?.id); + expect(indexPattern.getSaveObjectReference()[0]?.type).toEqual( + indexPattern.dataSourceRef?.type + ); + expect(indexPattern.getSaveObjectReference()[0]?.name).toEqual('dataSource'); + }); + }); +}); diff --git a/src/plugins/data/common/index_patterns/index_patterns/index_pattern.ts b/src/plugins/data/common/index_patterns/index_patterns/index_pattern.ts index ebe893479cf..7a966d2849c 100644 --- a/src/plugins/data/common/index_patterns/index_patterns/index_pattern.ts +++ b/src/plugins/data/common/index_patterns/index_patterns/index_pattern.ts @@ -43,7 +43,13 @@ import { IndexPatternField, IIndexPatternFieldList, fieldList } from '../fields' import { formatHitProvider } from './format_hit'; import { flattenHitWrapper } from './flatten_hit'; import { FieldFormatsStartCommon, FieldFormat } from '../../field_formats'; -import { IndexPatternSpec, TypeMeta, SourceFilter, IndexPatternFieldMap } from '../types'; +import { + IndexPatternSpec, + TypeMeta, + SourceFilter, + IndexPatternFieldMap, + SavedObjectReference, +} from '../types'; import { SerializedFieldFormat } from '../../../../expressions/common'; interface IndexPatternDeps { @@ -67,6 +73,7 @@ interface SavedObjectBody { type FormatFieldFn = (hit: Record, fieldName: string) => any; +const DATA_SOURCE_REFERNECE_NAME = 'dataSource'; export class IndexPattern implements IIndexPattern { public id?: string; public title: string = ''; @@ -86,6 +93,7 @@ export class IndexPattern implements IIndexPattern { // savedObject version public version: string | undefined; public sourceFilters?: SourceFilter[]; + public dataSourceRef?: SavedObjectReference; private originalSavedObjectBody: SavedObjectBody = {}; private shortDotsEnable: boolean = false; private fieldFormats: FieldFormatsStartCommon; @@ -128,6 +136,7 @@ export class IndexPattern implements IIndexPattern { this.fieldFormatMap = _.mapValues(fieldFormatMap, (mapping) => { return this.deserializeFieldFormatMap(mapping); }); + this.dataSourceRef = spec.dataSourceRef; } /** @@ -215,13 +224,13 @@ export class IndexPattern implements IIndexPattern { return { id: this.id, version: this.version, - title: this.title, timeFieldName: this.timeFieldName, sourceFilters: this.sourceFilters, fields: this.fields.toSpec({ getFormatterForField: this.getFormatterForField.bind(this) }), typeMeta: this.typeMeta, type: this.type, + dataSourceRef: this.dataSourceRef, }; } @@ -357,6 +366,17 @@ export class IndexPattern implements IIndexPattern { }; } + getSaveObjectReference() { + return this.dataSourceRef + ? [ + { + ...this.dataSourceRef, + name: DATA_SOURCE_REFERNECE_NAME, + }, + ] + : []; + } + /** * Provide a field, get its formatter * @param field diff --git a/src/plugins/data/common/index_patterns/index_patterns/index_patterns.test.ts b/src/plugins/data/common/index_patterns/index_patterns/index_patterns.test.ts index fe39a05b6b7..4361e12dec1 100644 --- a/src/plugins/data/common/index_patterns/index_patterns/index_patterns.test.ts +++ b/src/plugins/data/common/index_patterns/index_patterns/index_patterns.test.ts @@ -223,4 +223,30 @@ describe('IndexPatterns', () => { expect(indexPatterns.savedObjectToSpec(savedObject)).toMatchSnapshot(); }); + + test('savedObjectToSpecWithDataSource', () => { + const savedObject = { + id: 'id', + version: 'version', + attributes: { + title: 'opensearch-dashboards-*', + timeFieldName: '@timestamp', + fields: '[]', + sourceFilters: '[{"value":"item1"},{"value":"item2"}]', + fieldFormatMap: '{"field":{}}', + typeMeta: '{}', + type: '', + }, + type: 'index-pattern', + references: [ + { + id: 'id', + type: 'data-source', + name: 'dataSource', + }, + ], + }; + + expect(indexPatterns.savedObjectToSpec(savedObject)).toMatchSnapshot(); + }); }); diff --git a/src/plugins/data/common/index_patterns/index_patterns/index_patterns.ts b/src/plugins/data/common/index_patterns/index_patterns/index_patterns.ts index f3d605472b9..bd5bc48bba7 100644 --- a/src/plugins/data/common/index_patterns/index_patterns/index_patterns.ts +++ b/src/plugins/data/common/index_patterns/index_patterns/index_patterns.ts @@ -30,7 +30,6 @@ import { i18n } from '@osd/i18n'; import { SavedObjectsClientCommon } from '../..'; - import { createIndexPatternCache } from '.'; import { IndexPattern } from './index_pattern'; import { @@ -238,6 +237,7 @@ export class IndexPatternsService { metaFields, type: options.type, params: options.params || {}, + dataSourceId: options.dataSourceId, }); }; @@ -254,6 +254,7 @@ export class IndexPatternsService { ...options, type: indexPattern.type, params: indexPattern.typeMeta && indexPattern.typeMeta.params, + dataSourceId: indexPattern.dataSourceRef?.id, }); /** @@ -355,12 +356,14 @@ export class IndexPatternsService { typeMeta, type, }, + references, } = savedObject; const parsedSourceFilters = sourceFilters ? JSON.parse(sourceFilters) : undefined; const parsedTypeMeta = typeMeta ? JSON.parse(typeMeta) : undefined; const parsedFieldFormatMap = fieldFormatMap ? JSON.parse(fieldFormatMap) : {}; const parsedFields: FieldSpec[] = fields ? JSON.parse(fields) : []; + const dataSourceRef = Array.isArray(references) ? references[0] : undefined; this.addFormatsToFields(parsedFields, parsedFieldFormatMap); return { @@ -373,6 +376,7 @@ export class IndexPatternsService { fields: this.fieldArrayToMap(parsedFields), typeMeta: parsedTypeMeta, type, + dataSourceRef, }; }; @@ -401,7 +405,7 @@ export class IndexPatternsService { } const spec = this.savedObjectToSpec(savedObject); - const { title, type, typeMeta } = spec; + const { title, type, typeMeta, dataSourceRef } = spec; const parsedFieldFormats: FieldFormatMap = savedObject.attributes.fieldFormatMap ? JSON.parse(savedObject.attributes.fieldFormatMap) : {}; @@ -415,6 +419,7 @@ export class IndexPatternsService { metaFields: await this.config.get(UI_SETTINGS.META_FIELDS), type, params: typeMeta && typeMeta.params, + dataSourceId: dataSourceRef?.id, }) : spec.fields; } catch (err) { @@ -562,8 +567,11 @@ export class IndexPatternsService { } const body = indexPattern.getAsSavedObjectBody(); + const references = indexPattern.getSaveObjectReference(); + const response = await this.savedObjectsClient.create(savedObjectType, body, { id: indexPattern.id, + references, }); indexPattern.id = response.id; indexPatternCache.set(indexPattern.id, indexPattern); diff --git a/src/plugins/data/common/index_patterns/types.ts b/src/plugins/data/common/index_patterns/types.ts index 01ab9e41bab..108a93a3725 100644 --- a/src/plugins/data/common/index_patterns/types.ts +++ b/src/plugins/data/common/index_patterns/types.ts @@ -113,6 +113,7 @@ export interface GetFieldsOptions { params?: any; lookBack?: boolean; metaFields?: string[]; + dataSourceId?: string; } export interface IIndexPatternsApiClient { @@ -184,6 +185,11 @@ export interface FieldSpec { export type IndexPatternFieldMap = Record; +export interface SavedObjectReference { + name?: string; + id: string; + type: string; +} export interface IndexPatternSpec { id?: string; version?: string; @@ -194,6 +200,7 @@ export interface IndexPatternSpec { fields?: IndexPatternFieldMap; typeMeta?: TypeMeta; type?: string; + dataSourceRef?: SavedObjectReference; } export interface SourceFilter { diff --git a/src/plugins/data/common/search/opensearch_search/types.ts b/src/plugins/data/common/search/opensearch_search/types.ts index 5ff0fb689f8..a45d4d0660c 100644 --- a/src/plugins/data/common/search/opensearch_search/types.ts +++ b/src/plugins/data/common/search/opensearch_search/types.ts @@ -52,6 +52,7 @@ export type ISearchRequestParams> = { export interface IOpenSearchSearchRequest extends IOpenSearchDashboardsSearchRequest { indexType?: string; + dataSourceId?: string; } export type IOpenSearchSearchResponse = IOpenSearchDashboardsSearchResponse< diff --git a/src/plugins/data/common/search/search_source/search_source.test.ts b/src/plugins/data/common/search/search_source/search_source.test.ts index 40193e1a4a3..92cc0682a13 100644 --- a/src/plugins/data/common/search/search_source/search_source.test.ts +++ b/src/plugins/data/common/search/search_source/search_source.test.ts @@ -59,6 +59,16 @@ const indexPattern2 = ({ getSourceFiltering: () => mockSource2, } as unknown) as IndexPattern; +const dataSourceId = 'dataSourceId'; +const indexPattern3 = ({ + dataSourceRef: { + id: dataSourceId, + type: 'dataSource', + }, + getComputedFields, + getSourceFiltering: () => mockSource, +} as unknown) as IndexPattern; + describe('SearchSource', () => { let mockSearchMethod: any; let searchSourceDependencies: SearchSourceDependencies; @@ -209,6 +219,16 @@ describe('SearchSource', () => { await searchSource.fetch(options); expect(mockSearchMethod).toBeCalledTimes(1); }); + + test('index pattern with dataSourceId will generate request with dataSourceId', async () => { + const searchSource = new SearchSource({}, searchSourceDependencies); + searchSource.setField('index', indexPattern3); + const options = {}; + await searchSource.fetch(options); + const request = searchSource.history[0]; + expect(mockSearchMethod).toBeCalledTimes(1); + expect(request.dataSourceId).toEqual(dataSourceId); + }); }); describe('#serialize', () => { diff --git a/src/plugins/data/common/search/search_source/search_source.ts b/src/plugins/data/common/search/search_source/search_source.ts index 229357abffe..9a85ec85ce8 100644 --- a/src/plugins/data/common/search/search_source/search_source.ts +++ b/src/plugins/data/common/search/search_source/search_source.ts @@ -282,6 +282,9 @@ export class SearchSource { if (getConfig(UI_SETTINGS.COURIER_BATCH_SEARCHES)) { response = await this.legacyFetch(searchRequest, options); } else { + const indexPattern = this.getField('index'); + searchRequest.dataSourceId = indexPattern?.dataSourceRef?.id; + response = await this.fetchSearch(searchRequest, options); } @@ -335,9 +338,10 @@ export class SearchSource { getConfig, }); - return search({ params, indexType: searchRequest.indexType }, options).then(({ rawResponse }) => - onResponse(searchRequest, rawResponse) - ); + return search( + { params, indexType: searchRequest.indexType, dataSourceId: searchRequest.dataSourceId }, + options + ).then(({ rawResponse }) => onResponse(searchRequest, rawResponse)); } /** @@ -466,7 +470,7 @@ export class SearchSource { } } - private flatten() { + flatten() { const searchRequest = this.mergeProps(); searchRequest.body = searchRequest.body || {}; diff --git a/src/plugins/data/opensearch_dashboards.json b/src/plugins/data/opensearch_dashboards.json index b8d1134cba6..1193def7cb9 100644 --- a/src/plugins/data/opensearch_dashboards.json +++ b/src/plugins/data/opensearch_dashboards.json @@ -3,11 +3,8 @@ "version": "opensearchDashboards", "server": true, "ui": true, - "requiredPlugins": [ - "expressions", - "uiActions" - ], - "optionalPlugins": ["usageCollection"], + "requiredPlugins": ["expressions", "uiActions"], + "optionalPlugins": ["usageCollection", "dataSource"], "extraPublicDirs": ["common", "common/utils/abort_utils"], "requiredBundles": [ "usageCollection", diff --git a/src/plugins/data/public/index_patterns/index_patterns/index_patterns_api_client.ts b/src/plugins/data/public/index_patterns/index_patterns/index_patterns_api_client.ts index c1486943ef4..5f048e5cf75 100644 --- a/src/plugins/data/public/index_patterns/index_patterns/index_patterns_api_client.ts +++ b/src/plugins/data/public/index_patterns/index_patterns/index_patterns_api_client.ts @@ -60,7 +60,7 @@ export class IndexPatternsApiClient implements IIndexPatternsApiClient { } getFieldsForTimePattern(options: GetFieldsOptions = {}) { - const { pattern, lookBack, metaFields } = options; + const { pattern, lookBack, metaFields, dataSourceId } = options; const url = this._getUrl(['_fields_for_time_pattern']); @@ -68,11 +68,12 @@ export class IndexPatternsApiClient implements IIndexPatternsApiClient { pattern, look_back: lookBack, meta_fields: metaFields, + data_source: dataSourceId, }).then((resp: any) => resp.fields); } getFieldsForWildcard(options: GetFieldsOptions = {}) { - const { pattern, metaFields, type, params } = options; + const { pattern, metaFields, type, params, dataSourceId } = options; let url; let query; @@ -83,12 +84,14 @@ export class IndexPatternsApiClient implements IIndexPatternsApiClient { pattern, meta_fields: metaFields, params: JSON.stringify(params), + data_source: dataSourceId, }; } else { url = this._getUrl(['_fields_for_wildcard']); query = { pattern, meta_fields: metaFields, + data_source: dataSourceId, }; } diff --git a/src/plugins/data/server/index_patterns/fetcher/index_patterns_fetcher.ts b/src/plugins/data/server/index_patterns/fetcher/index_patterns_fetcher.ts index 9144e505133..a53e06cdea0 100644 --- a/src/plugins/data/server/index_patterns/fetcher/index_patterns_fetcher.ts +++ b/src/plugins/data/server/index_patterns/fetcher/index_patterns_fetcher.ts @@ -28,7 +28,7 @@ * under the License. */ -import { LegacyAPICaller } from 'opensearch-dashboards/server'; +import { LegacyAPICaller, OpenSearchClient } from 'opensearch-dashboards/server'; import { getFieldCapabilities, resolveTimePattern, createNoMatchingIndicesError } from './lib'; @@ -48,9 +48,9 @@ interface FieldSubType { } export class IndexPatternsFetcher { - private _callDataCluster: LegacyAPICaller; + private _callDataCluster: LegacyAPICaller | OpenSearchClient; - constructor(callDataCluster: LegacyAPICaller) { + constructor(callDataCluster: LegacyAPICaller | OpenSearchClient) { this._callDataCluster = callDataCluster; } diff --git a/src/plugins/data/server/index_patterns/fetcher/lib/field_capabilities/field_capabilities.ts b/src/plugins/data/server/index_patterns/fetcher/lib/field_capabilities/field_capabilities.ts index 0aee1222a36..57d59ce5a48 100644 --- a/src/plugins/data/server/index_patterns/fetcher/lib/field_capabilities/field_capabilities.ts +++ b/src/plugins/data/server/index_patterns/fetcher/lib/field_capabilities/field_capabilities.ts @@ -30,7 +30,7 @@ import { defaults, keyBy, sortBy } from 'lodash'; -import { LegacyAPICaller } from 'opensearch-dashboards/server'; +import { LegacyAPICaller, OpenSearchClient } from 'opensearch-dashboards/server'; import { callFieldCapsApi } from '../opensearch_api'; import { FieldCapsResponse, readFieldCapsResponse } from './field_caps_response'; import { mergeOverrides } from './overrides'; @@ -47,7 +47,7 @@ import { FieldDescriptor } from '../../index_patterns_fetcher'; * @return {Promise>} */ export async function getFieldCapabilities( - callCluster: LegacyAPICaller, + callCluster: LegacyAPICaller | OpenSearchClient, indices: string | string[] = [], metaFields: string[] = [], fieldCapsOptions?: { allowNoIndices: boolean } diff --git a/src/plugins/data/server/index_patterns/fetcher/lib/opensearch_api.ts b/src/plugins/data/server/index_patterns/fetcher/lib/opensearch_api.ts index 9e090a41462..5fcbb01ce07 100644 --- a/src/plugins/data/server/index_patterns/fetcher/lib/opensearch_api.ts +++ b/src/plugins/data/server/index_patterns/fetcher/lib/opensearch_api.ts @@ -28,7 +28,7 @@ * under the License. */ -import { LegacyAPICaller } from 'opensearch-dashboards/server'; +import { LegacyAPICaller, OpenSearchClient } from 'opensearch-dashboards/server'; import { convertOpenSearchError } from './errors'; import { FieldCapsResponse } from './field_capabilities'; @@ -57,10 +57,23 @@ export interface IndexAliasResponse { * @return {Promise} */ export async function callIndexAliasApi( - callCluster: LegacyAPICaller, + callCluster: LegacyAPICaller | OpenSearchClient, indices: string[] | string ): Promise { try { + // This approach of identify between OpenSearchClient vs LegacyAPICaller + // will be deprecated after support data client with legacy client + // https://github.com/opensearch-project/OpenSearch-Dashboards/issues/2133 + if ('transport' in callCluster) { + return ( + await callCluster.indices.getAlias({ + index: indices, + ignore_unavailable: true, + allow_no_indices: true, + }) + ).body as IndicesAliasResponse; + } + return (await callCluster('indices.getAlias', { index: indices, ignoreUnavailable: true, @@ -84,11 +97,25 @@ export async function callIndexAliasApi( * @return {Promise} */ export async function callFieldCapsApi( - callCluster: LegacyAPICaller, + callCluster: LegacyAPICaller | OpenSearchClient, indices: string[] | string, fieldCapsOptions: { allowNoIndices: boolean } = { allowNoIndices: false } ) { try { + // This approach of identify between OpenSearchClient vs LegacyAPICaller + // will be deprecated after support data client with legacy client + // https://github.com/opensearch-project/OpenSearch-Dashboards/issues/2133 + if ('transport' in callCluster) { + return ( + await callCluster.fieldCaps({ + index: indices, + fields: '*', + ignore_unavailable: true, + allow_no_indices: fieldCapsOptions.allowNoIndices, + }) + ).body as FieldCapsResponse; + } + return (await callCluster('fieldCaps', { index: indices, fields: '*', diff --git a/src/plugins/data/server/index_patterns/fetcher/lib/resolve_time_pattern.ts b/src/plugins/data/server/index_patterns/fetcher/lib/resolve_time_pattern.ts index cff81b65020..7b19ff78646 100644 --- a/src/plugins/data/server/index_patterns/fetcher/lib/resolve_time_pattern.ts +++ b/src/plugins/data/server/index_patterns/fetcher/lib/resolve_time_pattern.ts @@ -31,7 +31,7 @@ import { chain } from 'lodash'; import moment from 'moment'; -import { LegacyAPICaller } from 'opensearch-dashboards/server'; +import { LegacyAPICaller, OpenSearchClient } from 'opensearch-dashboards/server'; import { timePatternToWildcard } from './time_pattern_to_wildcard'; import { callIndexAliasApi, IndicesAliasResponse } from './opensearch_api'; @@ -47,7 +47,10 @@ import { callIndexAliasApi, IndicesAliasResponse } from './opensearch_api'; * and the indices that actually match the time * pattern (matches); */ -export async function resolveTimePattern(callCluster: LegacyAPICaller, timePattern: string) { +export async function resolveTimePattern( + callCluster: LegacyAPICaller | OpenSearchClient, + timePattern: string +) { const aliases = await callIndexAliasApi(callCluster, timePatternToWildcard(timePattern)); const allIndexDetails = chain(aliases) diff --git a/src/plugins/data/server/index_patterns/routes.ts b/src/plugins/data/server/index_patterns/routes.ts index 83632615cd8..f41c15ccb38 100644 --- a/src/plugins/data/server/index_patterns/routes.ts +++ b/src/plugins/data/server/index_patterns/routes.ts @@ -53,11 +53,12 @@ export function registerRoutes(http: HttpServiceSetup) { meta_fields: schema.oneOf([schema.string(), schema.arrayOf(schema.string())], { defaultValue: [], }), + data_source: schema.maybe(schema.string()), }), }, }, async (context, request, response) => { - const { callAsCurrentUser } = context.core.opensearch.legacy.client; + const callAsCurrentUser = await decideClient(context, request); const indexPatterns = new IndexPatternsFetcher(callAsCurrentUser); const { pattern, meta_fields: metaFields } = request.query; @@ -112,11 +113,13 @@ export function registerRoutes(http: HttpServiceSetup) { meta_fields: schema.oneOf([schema.string(), schema.arrayOf(schema.string())], { defaultValue: [], }), + data_source: schema.maybe(schema.string()), }), }, }, async (context: RequestHandlerContext, request: any, response: any) => { - const { callAsCurrentUser } = context.core.opensearch.legacy.client; + const callAsCurrentUser = await decideClient(context, request); + const indexPatterns = new IndexPatternsFetcher(callAsCurrentUser); const { pattern, interval, look_back: lookBack, meta_fields: metaFields } = request.query; @@ -147,3 +150,12 @@ export function registerRoutes(http: HttpServiceSetup) { } ); } + +const decideClient = async (context: RequestHandlerContext, request: any) => { + const dataSourceId = request.query.data_source; + if (dataSourceId) { + return await context.dataSource.opensearch.getClient(dataSourceId); + } + + return context.core.opensearch.legacy.client.callAsCurrentUser; +}; diff --git a/src/plugins/data/server/search/opensearch_search/decide_client.ts b/src/plugins/data/server/search/opensearch_search/decide_client.ts new file mode 100644 index 00000000000..5f38b82ffb4 --- /dev/null +++ b/src/plugins/data/server/search/opensearch_search/decide_client.ts @@ -0,0 +1,19 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { OpenSearchClient, RequestHandlerContext } from 'src/core/server'; +import { IOpenSearchSearchRequest } from '..'; + +export const decideClient = async ( + context: RequestHandlerContext, + request: IOpenSearchSearchRequest +): Promise => { + // if data source feature is disabled, return default opensearch client of current user + const client = + request.dataSourceId && context.dataSource + ? await context.dataSource.opensearch.getClient(request.dataSourceId) + : context.core.opensearch.client.asCurrentUser; + return client; +}; diff --git a/src/plugins/data/server/search/opensearch_search/opensearch_search_strategy.test.ts b/src/plugins/data/server/search/opensearch_search/opensearch_search_strategy.test.ts index 6207273c95c..fe95e3d7d4e 100644 --- a/src/plugins/data/server/search/opensearch_search/opensearch_search_strategy.test.ts +++ b/src/plugins/data/server/search/opensearch_search/opensearch_search_strategy.test.ts @@ -36,7 +36,7 @@ describe('OpenSearch search strategy', () => { const mockLogger: any = { debug: () => {}, }; - const mockApiCaller = jest.fn().mockResolvedValue({ + const body = { body: { _shards: { total: 10, @@ -45,7 +45,19 @@ describe('OpenSearch search strategy', () => { successful: 7, }, }, - }); + }; + const mockOpenSearchApiCaller = jest.fn().mockResolvedValue(body); + const mockDataSourceApiCaller = jest.fn().mockResolvedValue(body); + const dataSourceId = 'test-data-source-id'; + const mockDataSourceContext = { + dataSource: { + opensearch: { + getClient: () => { + return { search: mockDataSourceApiCaller }; + }, + }, + }, + }; const mockContext = { core: { uiSettings: { @@ -53,13 +65,18 @@ describe('OpenSearch search strategy', () => { get: () => {}, }, }, - opensearch: { client: { asCurrentUser: { search: mockApiCaller } } }, + opensearch: { client: { asCurrentUser: { search: mockOpenSearchApiCaller } } }, }, }; + const mockDataSourceEnabledContext = { + ...mockContext, + ...mockDataSourceContext, + }; const mockConfig$ = pluginInitializerContextConfigMock({}).legacy.globalConfig$; beforeEach(() => { - mockApiCaller.mockClear(); + mockOpenSearchApiCaller.mockClear(); + mockDataSourceApiCaller.mockClear(); }); it('returns a strategy with `search`', async () => { @@ -74,8 +91,8 @@ describe('OpenSearch search strategy', () => { await opensearchSearch.search((mockContext as unknown) as RequestHandlerContext, { params }); - expect(mockApiCaller).toBeCalled(); - expect(mockApiCaller.mock.calls[0][0]).toEqual({ + expect(mockOpenSearchApiCaller).toBeCalled(); + expect(mockOpenSearchApiCaller.mock.calls[0][0]).toEqual({ ...params, ignore_unavailable: true, track_total_hits: true, @@ -88,8 +105,8 @@ describe('OpenSearch search strategy', () => { await opensearchSearch.search((mockContext as unknown) as RequestHandlerContext, { params }); - expect(mockApiCaller).toBeCalled(); - expect(mockApiCaller.mock.calls[0][0]).toEqual({ + expect(mockOpenSearchApiCaller).toBeCalled(); + expect(mockOpenSearchApiCaller.mock.calls[0][0]).toEqual({ ...params, track_total_hits: true, }); @@ -111,4 +128,35 @@ describe('OpenSearch search strategy', () => { expect(response).toHaveProperty('loaded'); expect(response).toHaveProperty('rawResponse'); }); + + it('dataSource enabled, send request with dataSourceId get data source client', async () => { + const opensearchSearch = await opensearchSearchStrategyProvider(mockConfig$, mockLogger); + + await opensearchSearch.search( + (mockDataSourceEnabledContext as unknown) as RequestHandlerContext, + { + dataSourceId, + } + ); + expect(mockDataSourceApiCaller).toBeCalled(); + expect(mockOpenSearchApiCaller).not.toBeCalled(); + }); + + it('dataSource disabled, send request with dataSourceId get default client', async () => { + const opensearchSearch = await opensearchSearchStrategyProvider(mockConfig$, mockLogger); + + await opensearchSearch.search((mockContext as unknown) as RequestHandlerContext, { + dataSourceId, + }); + expect(mockOpenSearchApiCaller).toBeCalled(); + expect(mockDataSourceApiCaller).not.toBeCalled(); + }); + + it('dataSource enabled, send request without dataSourceId get default client', async () => { + const opensearchSearch = await opensearchSearchStrategyProvider(mockConfig$, mockLogger); + + await opensearchSearch.search((mockContext as unknown) as RequestHandlerContext, {}); + expect(mockOpenSearchApiCaller).toBeCalled(); + expect(mockDataSourceApiCaller).not.toBeCalled(); + }); }); diff --git a/src/plugins/data/server/search/opensearch_search/opensearch_search_strategy.ts b/src/plugins/data/server/search/opensearch_search/opensearch_search_strategy.ts index a29885a0a9f..4ccc8db0572 100644 --- a/src/plugins/data/server/search/opensearch_search/opensearch_search_strategy.ts +++ b/src/plugins/data/server/search/opensearch_search/opensearch_search_strategy.ts @@ -42,6 +42,7 @@ import { getShardTimeout, shimAbortSignal, } from '..'; +import { decideClient } from './decide_client'; export const opensearchSearchStrategyProvider = ( config$: Observable, @@ -70,10 +71,9 @@ export const opensearchSearchStrategyProvider = ( }); try { - const promise = shimAbortSignal( - context.core.opensearch.client.asCurrentUser.search(params), - options?.abortSignal - ); + const client = await decideClient(context, request); + const promise = shimAbortSignal(client.search(params), options?.abortSignal); + const { body: rawResponse } = (await promise) as ApiResponse>; if (usage) usage.trackSuccess(rawResponse.took); diff --git a/src/plugins/data_source/README.md b/src/plugins/data_source/README.md new file mode 100755 index 00000000000..53a96aa2146 --- /dev/null +++ b/src/plugins/data_source/README.md @@ -0,0 +1,13 @@ +# data_source + +An OpenSearch Dashboards plugin + +This plugin introduces OpenSearch data source into OpenSearch Dashboards, and provides related functions to connect to OpenSearch data sources. + +--- + +## Development + +See the [OpenSearch Dashboards contributing +guide](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/CONTRIBUTING.md) for instructions +setting up your development environment. diff --git a/src/plugins/data_source/audit_config.ts b/src/plugins/data_source/audit_config.ts new file mode 100644 index 00000000000..d8c99fbfa84 --- /dev/null +++ b/src/plugins/data_source/audit_config.ts @@ -0,0 +1,42 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { schema } from '@osd/config-schema'; +// eslint-disable-next-line @osd/eslint/no-restricted-paths +import { DateConversion } from '../../../src/core/server/logging/layouts/conversions'; + +const patternSchema = schema.string({ + validate: (string) => { + DateConversion.validate!(string); + }, +}); + +const patternLayout = schema.object({ + highlight: schema.maybe(schema.boolean()), + kind: schema.literal('pattern'), + pattern: schema.maybe(patternSchema), +}); + +const jsonLayout = schema.object({ + kind: schema.literal('json'), +}); + +export const fileAppenderSchema = schema.object( + { + kind: schema.literal('file'), + layout: schema.oneOf([patternLayout, jsonLayout]), + path: schema.string(), + }, + { + defaultValue: { + kind: 'file', + layout: { + kind: 'pattern', + highlight: true, + }, + path: '/tmp/opensearch-dashboards-data-source-audit.log', + }, + } +); diff --git a/src/plugins/data_source/common/data_sources/index.ts b/src/plugins/data_source/common/data_sources/index.ts new file mode 100644 index 00000000000..9f269633f30 --- /dev/null +++ b/src/plugins/data_source/common/data_sources/index.ts @@ -0,0 +1,6 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +export * from './types'; diff --git a/src/plugins/data_source/common/data_sources/types.ts b/src/plugins/data_source/common/data_sources/types.ts new file mode 100644 index 00000000000..afcf3d662fe --- /dev/null +++ b/src/plugins/data_source/common/data_sources/types.ts @@ -0,0 +1,26 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { SavedObjectAttributes } from 'src/core/types'; + +export interface DataSourceAttributes extends SavedObjectAttributes { + title: string; + description?: string; + endpoint: string; + auth: { + type: AuthType; + credentials: UsernamePasswordTypedContent | undefined; + }; +} + +export interface UsernamePasswordTypedContent extends SavedObjectAttributes { + username: string; + password: string; +} + +export enum AuthType { + NoAuth = 'no_auth', + UsernamePasswordType = 'username_password', +} diff --git a/src/plugins/data_source/common/index.ts b/src/plugins/data_source/common/index.ts new file mode 100644 index 00000000000..a98825eb97f --- /dev/null +++ b/src/plugins/data_source/common/index.ts @@ -0,0 +1,8 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +export const PLUGIN_ID = 'dataSource'; +export const PLUGIN_NAME = 'data_source'; +export const DATA_SOURCE_SAVED_OBJECT_TYPE = 'data-source'; diff --git a/src/plugins/data_source/config.ts b/src/plugins/data_source/config.ts new file mode 100644 index 00000000000..d7579026c6e --- /dev/null +++ b/src/plugins/data_source/config.ts @@ -0,0 +1,42 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { schema, TypeOf } from '@osd/config-schema'; +import { fileAppenderSchema } from './audit_config'; + +const KEY_NAME_MIN_LENGTH: number = 1; +const KEY_NAME_MAX_LENGTH: number = 100; +// Wrapping key size shoule be 32 bytes, as used in envelope encryption algorithms. +const WRAPPING_KEY_SIZE: number = 32; + +export const configSchema = schema.object({ + enabled: schema.boolean({ defaultValue: false }), + encryption: schema.object({ + wrappingKeyName: schema.string({ + minLength: KEY_NAME_MIN_LENGTH, + maxLength: KEY_NAME_MAX_LENGTH, + defaultValue: 'changeme', + }), + wrappingKeyNamespace: schema.string({ + minLength: KEY_NAME_MIN_LENGTH, + maxLength: KEY_NAME_MAX_LENGTH, + defaultValue: 'changeme', + }), + wrappingKey: schema.arrayOf(schema.number(), { + minSize: WRAPPING_KEY_SIZE, + maxSize: WRAPPING_KEY_SIZE, + defaultValue: new Array(32).fill(0), + }), + }), + clientPool: schema.object({ + size: schema.number({ defaultValue: 5 }), + }), + audit: schema.object({ + enabled: schema.boolean({ defaultValue: true }), + appender: fileAppenderSchema, + }), +}); + +export type DataSourcePluginConfigType = TypeOf; diff --git a/src/plugins/data_source/opensearch_dashboards.json b/src/plugins/data_source/opensearch_dashboards.json new file mode 100644 index 00000000000..71183a411c7 --- /dev/null +++ b/src/plugins/data_source/opensearch_dashboards.json @@ -0,0 +1,9 @@ +{ + "id": "dataSource", + "version": "opensearchDashboards", + "opensearchDashboardsVersion": "opensearchDashboards", + "server": true, + "ui": true, + "requiredPlugins": [], + "optionalPlugins": [] +} diff --git a/src/plugins/data_source/public/index.ts b/src/plugins/data_source/public/index.ts new file mode 100644 index 00000000000..411838d0b1b --- /dev/null +++ b/src/plugins/data_source/public/index.ts @@ -0,0 +1,14 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { DataSourcePublicPlugin } from './plugin'; + +// This exports static code and TypeScript types, +// as well as, OpenSearch Dashboards Platform `plugin()` initializer. +export function plugin() { + return new DataSourcePublicPlugin(); +} + +export { DataSourcePublicPluginSetup, DataSourcePublicPluginStart } from './types'; diff --git a/src/plugins/data_source/public/plugin.ts b/src/plugins/data_source/public/plugin.ts new file mode 100644 index 00000000000..a84091be1d9 --- /dev/null +++ b/src/plugins/data_source/public/plugin.ts @@ -0,0 +1,20 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { CoreSetup, CoreStart, Plugin } from '../../../core/public'; +import { DataSourcePublicPluginSetup, DataSourcePublicPluginStart } from './types'; + +export class DataSourcePublicPlugin + implements Plugin { + public setup(core: CoreSetup): DataSourcePublicPluginSetup { + return {}; + } + + public start(core: CoreStart): DataSourcePublicPluginStart { + return {}; + } + + public stop() {} +} diff --git a/src/plugins/data_source/public/types.ts b/src/plugins/data_source/public/types.ts new file mode 100644 index 00000000000..95bab57ed14 --- /dev/null +++ b/src/plugins/data_source/public/types.ts @@ -0,0 +1,10 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +// eslint-disable-next-line @typescript-eslint/no-empty-interface +export interface DataSourcePublicPluginSetup {} + +// eslint-disable-next-line @typescript-eslint/no-empty-interface +export interface DataSourcePublicPluginStart {} diff --git a/src/plugins/data_source/server/audit/logging_auditor.ts b/src/plugins/data_source/server/audit/logging_auditor.ts new file mode 100644 index 00000000000..bc1f5bc57dc --- /dev/null +++ b/src/plugins/data_source/server/audit/logging_auditor.ts @@ -0,0 +1,23 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { AuditableEvent, Auditor, Logger, OpenSearchDashboardsRequest } from 'src/core/server'; + +export class LoggingAuditor implements Auditor { + constructor( + private readonly request: OpenSearchDashboardsRequest, + private readonly logger: Logger + ) {} + + public withAuditScope(name: string) {} + + public add(event: AuditableEvent) { + const message = event.message; + const meta = { + type: event.type, + }; + this.logger.info(message, meta); + } +} diff --git a/src/plugins/data_source/server/client/client_config.test.ts b/src/plugins/data_source/server/client/client_config.test.ts new file mode 100644 index 00000000000..39a3607ccba --- /dev/null +++ b/src/plugins/data_source/server/client/client_config.test.ts @@ -0,0 +1,29 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ +import { DataSourcePluginConfigType } from '../../config'; +import { parseClientOptions } from './client_config'; + +const TEST_DATA_SOURCE_ENDPOINT = 'http://datasource.com'; + +const config = { + enabled: true, + clientPool: { + size: 5, + }, +} as DataSourcePluginConfigType; + +describe('parseClientOptions', () => { + test('include the ssl client configs as defaults', () => { + expect(parseClientOptions(config, TEST_DATA_SOURCE_ENDPOINT)).toEqual( + expect.objectContaining({ + node: TEST_DATA_SOURCE_ENDPOINT, + ssl: { + requestCert: true, + rejectUnauthorized: true, + }, + }) + ); + }); +}); diff --git a/src/plugins/data_source/server/client/client_config.ts b/src/plugins/data_source/server/client/client_config.ts new file mode 100644 index 00000000000..5973e5a0813 --- /dev/null +++ b/src/plugins/data_source/server/client/client_config.ts @@ -0,0 +1,29 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { ClientOptions } from '@opensearch-project/opensearch'; +import { DataSourcePluginConfigType } from '../../config'; + +/** + * Parse the client options from given data source config and endpoint + * + * @param config The config to generate the client options from. + * @param endpoint endpoint url of data source + */ +export function parseClientOptions( + // TODO: will use client configs, that comes from a merge result of user config and default opensearch client config, + config: DataSourcePluginConfigType, + endpoint: string +): ClientOptions { + const clientOptions: ClientOptions = { + node: endpoint, + ssl: { + requestCert: true, + rejectUnauthorized: true, + }, + }; + + return clientOptions; +} diff --git a/src/plugins/data_source/server/client/client_pool.test.ts b/src/plugins/data_source/server/client/client_pool.test.ts new file mode 100644 index 00000000000..92320e9610a --- /dev/null +++ b/src/plugins/data_source/server/client/client_pool.test.ts @@ -0,0 +1,39 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { loggingSystemMock } from '../../../../core/server/mocks'; +import { DataSourcePluginConfigType } from '../../config'; +import { OpenSearchClientPool } from './client_pool'; + +const logger = loggingSystemMock.create(); + +describe('Client Pool', () => { + let service: OpenSearchClientPool; + let config: DataSourcePluginConfigType; + + beforeEach(() => { + const mockLogger = logger.get('dataSource'); + service = new OpenSearchClientPool(mockLogger); + config = { + enabled: true, + clientPool: { + size: 5, + }, + } as DataSourcePluginConfigType; + }); + + afterEach(() => { + service.stop(); + jest.clearAllMocks(); + }); + + describe('setup()', () => { + test('exposes proper contract', async () => { + const setup = await service.setup(config); + expect(setup).toHaveProperty('getClientFromPool'); + expect(setup).toHaveProperty('addClientToPool'); + }); + }); +}); diff --git a/src/plugins/data_source/server/client/client_pool.ts b/src/plugins/data_source/server/client/client_pool.ts new file mode 100644 index 00000000000..be1957bc769 --- /dev/null +++ b/src/plugins/data_source/server/client/client_pool.ts @@ -0,0 +1,76 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { Client } from '@opensearch-project/opensearch'; +import LRUCache from 'lru-cache'; +import { Logger } from 'src/core/server'; +import { DataSourcePluginConfigType } from '../../config'; + +export interface OpenSearchClientPoolSetup { + getClientFromPool: (id: string) => Client | undefined; + addClientToPool: (endpoint: string, client: Client) => void; +} + +/** + * OpenSearch client pool. + * + * This client pool uses an LRU cache to manage OpenSearch Js client objects. + * It reuse TPC connections for each OpenSearch endpoint. + */ +export class OpenSearchClientPool { + // LRU cache + // key: data source endpoint url + // value: OpenSearch client object + private cache?: LRUCache; + private isClosed = false; + + constructor(private logger: Logger) {} + + public async setup(config: DataSourcePluginConfigType) { + const logger = this.logger; + const { size } = config.clientPool; + + this.cache = new LRUCache({ + max: size, + maxAge: 15 * 60 * 1000, // by default, TCP connection times out in 15 minutes + + async dispose(endpoint, client) { + try { + await client.close(); + } catch (error: any) { + // log and do nothing since we are anyways evicting the client object from cache + logger.warn( + `Error closing OpenSearch client when removing from client pool: ${error.message}` + ); + } + }, + }); + this.logger.info(`Created data source client pool of size ${size}`); + + const getClientFromPool = (endpoint: string) => { + return this.cache!.get(endpoint); + }; + + const addClientToPool = (endpoint: string, client: Client) => { + this.cache!.set(endpoint, client); + }; + + return { + getClientFromPool, + addClientToPool, + }; + } + + start() {} + + // close all clients in the pool + async stop() { + if (this.isClosed) { + return; + } + await Promise.all(this.cache!.values().map((client) => client.close())); + this.isClosed = true; + } +} diff --git a/src/plugins/data_source/server/client/configure_client.test.mocks.ts b/src/plugins/data_source/server/client/configure_client.test.mocks.ts new file mode 100644 index 00000000000..38a585ff202 --- /dev/null +++ b/src/plugins/data_source/server/client/configure_client.test.mocks.ts @@ -0,0 +1,18 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +export const ClientMock = jest.fn(); +jest.doMock('@opensearch-project/opensearch', () => { + const actual = jest.requireActual('@opensearch-project/opensearch'); + return { + ...actual, + Client: ClientMock, + }; +}); + +export const parseClientOptionsMock = jest.fn(); +jest.doMock('./client_config', () => ({ + parseClientOptions: parseClientOptionsMock, +})); diff --git a/src/plugins/data_source/server/client/configure_client.test.ts b/src/plugins/data_source/server/client/configure_client.test.ts new file mode 100644 index 00000000000..f8f8f2cb580 --- /dev/null +++ b/src/plugins/data_source/server/client/configure_client.test.ts @@ -0,0 +1,130 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { SavedObjectsClientContract } from '../../../../core/server'; +import { loggingSystemMock, savedObjectsClientMock } from '../../../../core/server/mocks'; +import { DATA_SOURCE_SAVED_OBJECT_TYPE } from '../../common'; +import { DataSourceAttributes, AuthType } from '../../common/data_sources/types'; +import { DataSourcePluginConfigType } from '../../config'; +import { ClientMock, parseClientOptionsMock } from './configure_client.test.mocks'; +import { OpenSearchClientPoolSetup } from './client_pool'; +import { configureClient } from './configure_client'; +import { ClientOptions } from '@opensearch-project/opensearch'; +// eslint-disable-next-line @osd/eslint/no-restricted-paths +import { opensearchClientMock } from '../../../../core/server/opensearch/client/mocks'; +import { CryptographyClient } from '../cryptography'; + +const DATA_SOURCE_ID = 'a54b76ec86771ee865a0f74a305dfff8'; +const cryptoClient = new CryptographyClient('test', 'test', new Array(32).fill(0)); + +// TODO: improve UT +describe('configureClient', () => { + let logger: ReturnType; + let config: DataSourcePluginConfigType; + let savedObjectsMock: jest.Mocked; + let clientPoolSetup: OpenSearchClientPoolSetup; + let clientOptions: ClientOptions; + let dataSourceAttr: DataSourceAttributes; + let dsClient: ReturnType; + + beforeEach(() => { + dsClient = opensearchClientMock.createInternalClient(); + logger = loggingSystemMock.createLogger(); + savedObjectsMock = savedObjectsClientMock.create(); + config = { + enabled: true, + clientPool: { + size: 5, + }, + } as DataSourcePluginConfigType; + clientOptions = { + nodes: 'http://localhost', + ssl: { + requestCert: true, + rejectUnauthorized: true, + }, + } as ClientOptions; + dataSourceAttr = { + title: 'title', + endpoint: 'http://localhost', + auth: { + type: AuthType.UsernamePasswordType, + credentials: { + username: 'username', + password: 'password', + }, + }, + } as DataSourceAttributes; + + clientPoolSetup = { + getClientFromPool: jest.fn(), + addClientToPool: jest.fn(), + }; + + savedObjectsMock.get.mockResolvedValueOnce({ + id: DATA_SOURCE_ID, + type: DATA_SOURCE_SAVED_OBJECT_TYPE, + attributes: dataSourceAttr, + references: [], + }); + + ClientMock.mockImplementation(() => { + return dsClient; + }); + }); + + afterEach(() => { + ClientMock.mockReset(); + }); + + test('configure client with auth.type == no_auth, will call new Client() to create client', async () => { + savedObjectsMock.get.mockReset().mockResolvedValueOnce({ + id: DATA_SOURCE_ID, + type: DATA_SOURCE_SAVED_OBJECT_TYPE, + attributes: { + ...dataSourceAttr, + auth: { + type: AuthType.NoAuth, + }, + }, + references: [], + }); + + parseClientOptionsMock.mockReturnValue(clientOptions); + + const client = await configureClient( + DATA_SOURCE_ID, + savedObjectsMock, + cryptoClient, + clientPoolSetup, + config, + logger + ); + + expect(parseClientOptionsMock).toHaveBeenCalled(); + expect(ClientMock).toHaveBeenCalledTimes(1); + expect(ClientMock).toHaveBeenCalledWith(clientOptions); + expect(savedObjectsMock.get).toHaveBeenCalledTimes(1); + expect(client).toBe(dsClient.child.mock.results[0].value); + }); + + test('configure client with auth.type == username_password, will first call decrypt()', async () => { + const spy = jest.spyOn(cryptoClient, 'decodeAndDecrypt').mockResolvedValue('password'); + + const client = await configureClient( + DATA_SOURCE_ID, + savedObjectsMock, + cryptoClient, + clientPoolSetup, + config, + logger + ); + + expect(ClientMock).toHaveBeenCalledTimes(1); + expect(savedObjectsMock.get).toHaveBeenCalledTimes(1); + expect(spy).toHaveBeenCalledTimes(1); + expect(client).toBe(dsClient.child.mock.results[0].value); + }); +}); diff --git a/src/plugins/data_source/server/client/configure_client.ts b/src/plugins/data_source/server/client/configure_client.ts new file mode 100644 index 00000000000..8cfa9769de7 --- /dev/null +++ b/src/plugins/data_source/server/client/configure_client.ts @@ -0,0 +1,130 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { Client } from '@opensearch-project/opensearch'; +import { Logger, SavedObject, SavedObjectsClientContract } from '../../../../../src/core/server'; +import { DATA_SOURCE_SAVED_OBJECT_TYPE } from '../../common'; +import { + AuthType, + DataSourceAttributes, + UsernamePasswordTypedContent, +} from '../../common/data_sources'; +import { DataSourcePluginConfigType } from '../../config'; +import { CryptographyClient } from '../cryptography'; +import { DataSourceConfigError } from '../lib/error'; +import { parseClientOptions } from './client_config'; +import { OpenSearchClientPoolSetup } from './client_pool'; + +export const configureClient = async ( + dataSourceId: string, + savedObjects: SavedObjectsClientContract, + cryptographyClient: CryptographyClient, + openSearchClientPoolSetup: OpenSearchClientPoolSetup, + config: DataSourcePluginConfigType, + logger: Logger +): Promise => { + try { + const dataSource = await getDataSource(dataSourceId, savedObjects); + const rootClient = getRootClient(dataSource.attributes, config, openSearchClientPoolSetup); + + return await getQueryClient(rootClient, dataSource, cryptographyClient); + } catch (error: any) { + logger.error(`Fail to get data source client for dataSourceId: [${dataSourceId}]`); + logger.error(error); + // Re-throw as DataSourceConfigError + throw new DataSourceConfigError('Fail to get data source client: ', error); + } +}; + +export const getDataSource = async ( + dataSourceId: string, + savedObjects: SavedObjectsClientContract +): Promise> => { + const dataSource = await savedObjects.get( + DATA_SOURCE_SAVED_OBJECT_TYPE, + dataSourceId + ); + return dataSource; +}; + +export const getCredential = async ( + dataSource: SavedObject, + cryptographyClient: CryptographyClient +): Promise => { + const { username, password } = dataSource.attributes.auth.credentials!; + const decodedPassword = await cryptographyClient.decodeAndDecrypt(password); + const credential = { + username, + password: decodedPassword, + }; + + return credential; +}; + +/** + * Create a child client object with given auth info. + * + * @param rootClient root client for the connection with given data source endpoint. + * @param dataSource data source saved object + * @param cryptographyClient cryptography client for password encryption / decrpytion + * @returns child client. + */ +const getQueryClient = async ( + rootClient: Client, + dataSource: SavedObject, + cryptographyClient: CryptographyClient +): Promise => { + if (AuthType.NoAuth === dataSource.attributes.auth.type) { + return rootClient.child(); + } else { + const credential = await getCredential(dataSource, cryptographyClient); + + return getBasicAuthClient(rootClient, credential); + } +}; + +/** + * Gets a root client object of the OpenSearch endpoint. + * Will attempt to get from cache, if cache miss, create a new one and load into cache. + * + * @param dataSourceAttr data source saved objects attributes. + * @param config data source config + * @returns OpenSearch client for the given data source endpoint. + */ +const getRootClient = ( + dataSourceAttr: DataSourceAttributes, + config: DataSourcePluginConfigType, + { getClientFromPool, addClientToPool }: OpenSearchClientPoolSetup +): Client => { + const endpoint = dataSourceAttr.endpoint; + const cachedClient = getClientFromPool(endpoint); + if (cachedClient) { + return cachedClient; + } else { + const clientOptions = parseClientOptions(config, endpoint); + + const client = new Client(clientOptions); + addClientToPool(endpoint, client); + + return client; + } +}; + +const getBasicAuthClient = ( + rootClient: Client, + credential: UsernamePasswordTypedContent +): Client => { + const { username, password } = credential; + return rootClient.child({ + auth: { + username, + password, + }, + // Child client doesn't allow auth option, adding null auth header to bypass, + // so logic in child() can rebuild the auth header based on the auth input. + // See https://github.com/opensearch-project/OpenSearch-Dashboards/issues/2182 for details + headers: { authorization: null }, + }); +}; diff --git a/src/plugins/data_source/server/client/index.ts b/src/plugins/data_source/server/client/index.ts new file mode 100644 index 00000000000..8adc96115b9 --- /dev/null +++ b/src/plugins/data_source/server/client/index.ts @@ -0,0 +1,7 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +export { OpenSearchClientPool } from './client_pool'; +export { configureClient } from './configure_client'; diff --git a/src/plugins/data_source/server/cryptography/cryptography_client.test.ts b/src/plugins/data_source/server/cryptography/cryptography_client.test.ts new file mode 100644 index 00000000000..1f8d2596a3c --- /dev/null +++ b/src/plugins/data_source/server/cryptography/cryptography_client.test.ts @@ -0,0 +1,117 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { CryptographyClient } from './cryptography_client'; +import { randomBytes } from 'crypto'; + +const dummyWrappingKeyName = 'dummy_wrapping_key_name'; +const dummyWrappingKeyNamespace = 'dummy_wrapping_key_namespace'; + +test('Invalid wrapping key size throws error', () => { + const dummyRandomBytes = [...randomBytes(31)]; + const expectedErrorMsg = `Wrapping key size shoule be 32 bytes, as used in envelope encryption. Current wrapping key size: '${dummyRandomBytes.length}' bytes`; + expect(() => { + new CryptographyClient(dummyWrappingKeyName, dummyWrappingKeyNamespace, dummyRandomBytes); + }).toThrowError(new Error(expectedErrorMsg)); +}); + +describe('Test encrpyt and decrypt module', () => { + const dummyPlainText = 'dummy'; + const dummyNumArray1 = [...randomBytes(32)]; + const dummyNumArray2 = [...randomBytes(32)]; + + describe('Positive test cases', () => { + test('Encrypt and Decrypt with same in memory keyring', async () => { + const cryptographyClient = new CryptographyClient( + dummyWrappingKeyName, + dummyWrappingKeyNamespace, + dummyNumArray1 + ); + const encrypted = await cryptographyClient.encryptAndEncode(dummyPlainText); + const outputText = await cryptographyClient.decodeAndDecrypt(encrypted); + expect(outputText).toBe(dummyPlainText); + }); + test('Encrypt and Decrypt with two different keyrings with exact same identifiers', async () => { + const cryptographyClient1 = new CryptographyClient( + dummyWrappingKeyName, + dummyWrappingKeyNamespace, + dummyNumArray1 + ); + const encrypted = await cryptographyClient1.encryptAndEncode(dummyPlainText); + + const cryptographyClient2 = new CryptographyClient( + dummyWrappingKeyName, + dummyWrappingKeyNamespace, + dummyNumArray1 + ); + const outputText = await cryptographyClient2.decodeAndDecrypt(encrypted); + expect(cryptographyClient1 === cryptographyClient2).toBeFalsy(); + expect(outputText).toBe(dummyPlainText); + }); + }); + + describe('Negative test cases', () => { + const defaultWrappingKeyName = 'changeme'; + const defaultWrappingKeyNamespace = 'changeme'; + const expectedErrorMsg = 'unencryptedDataKey has not been set'; + test('Encrypt and Decrypt with different key names', async () => { + const cryptographyClient1 = new CryptographyClient( + dummyWrappingKeyName, + dummyWrappingKeyNamespace, + dummyNumArray1 + ); + const encrypted = await cryptographyClient1.encryptAndEncode(dummyPlainText); + + const cryptographyClient2 = new CryptographyClient( + defaultWrappingKeyName, + dummyWrappingKeyNamespace, + dummyNumArray1 + ); + try { + await cryptographyClient2.decodeAndDecrypt(encrypted); + } catch (error) { + expect(error.message).toMatch(expectedErrorMsg); + } + }); + test('Encrypt and Decrypt with different key namespaces', async () => { + const cryptographyClient1 = new CryptographyClient( + dummyWrappingKeyName, + dummyWrappingKeyNamespace, + dummyNumArray1 + ); + const encrypted = await cryptographyClient1.encryptAndEncode(dummyPlainText); + + const cryptographyClient2 = new CryptographyClient( + dummyWrappingKeyName, + defaultWrappingKeyNamespace, + dummyNumArray1 + ); + try { + await cryptographyClient2.decodeAndDecrypt(encrypted); + } catch (error) { + expect(error.message).toMatch(expectedErrorMsg); + } + }); + test('Encrypt and Decrypt with different wrapping keys', async () => { + const cryptographyClient1 = new CryptographyClient( + dummyWrappingKeyName, + dummyWrappingKeyNamespace, + dummyNumArray1 + ); + const encrypted = await cryptographyClient1.encryptAndEncode(dummyPlainText); + + const cryptographyClient2 = new CryptographyClient( + dummyWrappingKeyName, + dummyWrappingKeyNamespace, + dummyNumArray2 + ); + try { + await cryptographyClient2.decodeAndDecrypt(encrypted); + } catch (error) { + expect(error.message).toMatch(expectedErrorMsg); + } + }); + }); +}); diff --git a/src/plugins/data_source/server/cryptography/cryptography_client.ts b/src/plugins/data_source/server/cryptography/cryptography_client.ts new file mode 100644 index 00000000000..f5968ae13ad --- /dev/null +++ b/src/plugins/data_source/server/cryptography/cryptography_client.ts @@ -0,0 +1,70 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { + buildClient, + CommitmentPolicy, + RawAesKeyringNode, + RawAesWrappingSuiteIdentifier, +} from '@aws-crypto/client-node'; + +export const ENCODING_STRATEGY: BufferEncoding = 'base64'; +export const WRAPPING_KEY_SIZE: number = 32; + +export class CryptographyClient { + private readonly commitmentPolicy = CommitmentPolicy.REQUIRE_ENCRYPT_REQUIRE_DECRYPT; + private readonly wrappingSuite = RawAesWrappingSuiteIdentifier.AES256_GCM_IV12_TAG16_NO_PADDING; + + private keyring: RawAesKeyringNode; + + private readonly encrypt: Function; + private readonly decrypt: Function; + + /** + * @param {string} wrappingKeyName name value to identify the AES key in a keyring + * @param {string} wrappingKeyNamespace namespace value to identify the AES key in a keyring, + * @param {number[]} wrappingKey 32 Bytes raw wrapping key used to perform envelope encryption + */ + constructor(wrappingKeyName: string, wrappingKeyNamespace: string, wrappingKey: number[]) { + if (wrappingKey.length !== WRAPPING_KEY_SIZE) { + const wrappingKeySizeMismatchMsg = `Wrapping key size shoule be 32 bytes, as used in envelope encryption. Current wrapping key size: '${wrappingKey.length}' bytes`; + throw new Error(wrappingKeySizeMismatchMsg); + } + + // Create raw AES keyring + this.keyring = new RawAesKeyringNode({ + keyName: wrappingKeyName, + keyNamespace: wrappingKeyNamespace, + unencryptedMasterKey: new Uint8Array(wrappingKey), + wrappingSuite: this.wrappingSuite, + }); + + // Destructuring encrypt and decrypt functions from client + const { encrypt, decrypt } = buildClient(this.commitmentPolicy); + + this.encrypt = encrypt; + this.decrypt = decrypt; + } + + /** + * Input text content and output encrypted string encoded with ENCODING_STRATEGY + * @param {string} plainText + * @returns {Promise} + */ + public async encryptAndEncode(plainText: string): Promise { + const result = await this.encrypt(this.keyring, plainText); + return result.result.toString(ENCODING_STRATEGY); + } + + /** + * Input encrypted content and output decrypted string + * @param {string} encrypted + * @returns {Promise} + */ + public async decodeAndDecrypt(encrypted: string): Promise { + const result = await this.decrypt(this.keyring, Buffer.from(encrypted, ENCODING_STRATEGY)); + return result.plaintext.toString(); + } +} diff --git a/src/plugins/data_source/server/cryptography/index.ts b/src/plugins/data_source/server/cryptography/index.ts new file mode 100644 index 00000000000..857fa691bdd --- /dev/null +++ b/src/plugins/data_source/server/cryptography/index.ts @@ -0,0 +1,6 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +export { CryptographyClient } from './cryptography_client'; diff --git a/src/plugins/data_source/server/data_source_service.test.ts b/src/plugins/data_source/server/data_source_service.test.ts new file mode 100644 index 00000000000..53dfb6f273e --- /dev/null +++ b/src/plugins/data_source/server/data_source_service.test.ts @@ -0,0 +1,38 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { loggingSystemMock } from '../../../core/server/mocks'; +import { DataSourcePluginConfigType } from '../config'; +import { DataSourceService } from './data_source_service'; + +const logger = loggingSystemMock.create(); + +describe('Data Source Service', () => { + let service: DataSourceService; + let config: DataSourcePluginConfigType; + + beforeEach(() => { + const mockLogger = logger.get('dataSource'); + service = new DataSourceService(mockLogger); + config = { + enabled: true, + clientPool: { + size: 5, + }, + } as DataSourcePluginConfigType; + }); + + afterEach(() => { + service.stop(); + jest.clearAllMocks(); + }); + + describe('setup()', () => { + test('exposes proper contract', async () => { + const setup = await service.setup(config); + expect(setup).toHaveProperty('getDataSourceClient'); + }); + }); +}); diff --git a/src/plugins/data_source/server/data_source_service.ts b/src/plugins/data_source/server/data_source_service.ts new file mode 100644 index 00000000000..73f61b87cb3 --- /dev/null +++ b/src/plugins/data_source/server/data_source_service.ts @@ -0,0 +1,56 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { + Auditor, + Logger, + OpenSearchClient, + SavedObjectsClientContract, +} from '../../../../src/core/server'; +import { DataSourcePluginConfigType } from '../config'; +import { configureClient, OpenSearchClientPool } from './client'; +import { CryptographyClient } from './cryptography'; +export interface DataSourceServiceSetup { + getDataSourceClient: ( + dataSourceId: string, + // this saved objects client is used to fetch data source on behalf of users, caller should pass scoped saved objects client + savedObjects: SavedObjectsClientContract, + cryptographyClient: CryptographyClient + ) => Promise; +} +export class DataSourceService { + private readonly openSearchClientPool: OpenSearchClientPool; + + constructor(private logger: Logger) { + this.openSearchClientPool = new OpenSearchClientPool(logger); + } + + async setup(config: DataSourcePluginConfigType) { + const openSearchClientPoolSetup = await this.openSearchClientPool.setup(config); + + const getDataSourceClient = ( + dataSourceId: string, + savedObjects: SavedObjectsClientContract, + cryptographyClient: CryptographyClient + ): Promise => { + return configureClient( + dataSourceId, + savedObjects, + cryptographyClient, + openSearchClientPoolSetup, + config, + this.logger + ); + }; + + return { getDataSourceClient }; + } + + start() {} + + stop() { + this.openSearchClientPool.stop(); + } +} diff --git a/src/plugins/data_source/server/index.ts b/src/plugins/data_source/server/index.ts new file mode 100644 index 00000000000..f05b833817d --- /dev/null +++ b/src/plugins/data_source/server/index.ts @@ -0,0 +1,18 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { PluginConfigDescriptor, PluginInitializerContext } from 'src/core/server'; +import { DataSourcePlugin } from './plugin'; +import { configSchema, DataSourcePluginConfigType } from '../config'; + +export const config: PluginConfigDescriptor = { + schema: configSchema, +}; + +export function plugin(initializerContext: PluginInitializerContext) { + return new DataSourcePlugin(initializerContext); +} + +export { DataSourcePluginSetup, DataSourcePluginStart } from './types'; diff --git a/src/plugins/data_source/server/lib/error.test.ts b/src/plugins/data_source/server/lib/error.test.ts new file mode 100644 index 00000000000..fca7efb0904 --- /dev/null +++ b/src/plugins/data_source/server/lib/error.test.ts @@ -0,0 +1,40 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { SavedObjectsErrorHelpers } from '../../../../../src/core/server'; +import { DataSourceConfigError } from './error'; + +describe('DataSourceConfigError', () => { + it('create from savedObject bad request error should be 400 error', () => { + const error = SavedObjectsErrorHelpers.createBadRequestError('test reason message'); + expect(new DataSourceConfigError('test prefix: ', error)).toMatchObject({ + statusCode: 400, + message: 'test prefix: test reason message: Bad Request', + }); + }); + + it('create from savedObject not found error should be 400 error', () => { + const error = SavedObjectsErrorHelpers.decorateNotAuthorizedError(new Error()); + expect(new DataSourceConfigError('test prefix: ', error)).toHaveProperty('statusCode', 400); + }); + + it('create from savedObject service unavailable error should be a 500 error', () => { + const error = SavedObjectsErrorHelpers.decorateOpenSearchUnavailableError( + new Error('test reason message') + ); + expect(new DataSourceConfigError('test prefix: ', error)).toMatchObject({ + statusCode: 500, + message: 'test prefix: test reason message', + }); + }); + + it('create from non savedObject error should always be a 400 error', () => { + const error = new Error('test reason message'); + expect(new DataSourceConfigError('test prefix: ', error)).toMatchObject({ + statusCode: 400, + message: 'test prefix: test reason message', + }); + }); +}); diff --git a/src/plugins/data_source/server/lib/error.ts b/src/plugins/data_source/server/lib/error.ts new file mode 100644 index 00000000000..e921a8d8043 --- /dev/null +++ b/src/plugins/data_source/server/lib/error.ts @@ -0,0 +1,21 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { SavedObjectsErrorHelpers } from '../../../../../src/core/server'; +import { OsdError } from '../../../../../src/plugins/opensearch_dashboards_utils/common'; + +export class DataSourceConfigError extends OsdError { + // must have statusCode to avoid route handler in search.ts to return 500 + statusCode: number; + constructor(messagePrefix: string, error: any) { + const messageContent = SavedObjectsErrorHelpers.isSavedObjectsClientError(error) + ? error.output.payload.message + : error.message; + super(messagePrefix + messageContent); + // Cast all 5xx error returned by saveObjectClient to 500, 400 for both savedObject client + // 4xx errors, and other errors + this.statusCode = SavedObjectsErrorHelpers.isOpenSearchUnavailableError(error) ? 500 : 400; + } +} diff --git a/src/plugins/data_source/server/plugin.ts b/src/plugins/data_source/server/plugin.ts new file mode 100644 index 00000000000..b0b6718ae11 --- /dev/null +++ b/src/plugins/data_source/server/plugin.ts @@ -0,0 +1,170 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { Observable } from 'rxjs'; +import { first, map } from 'rxjs/operators'; +import { + Auditor, + AuditorFactory, + CoreSetup, + CoreStart, + IContextProvider, + Logger, + LoggerContextConfigInput, + OpenSearchDashboardsRequest, + Plugin, + PluginInitializerContext, + RequestHandler, +} from '../../../../src/core/server'; +import { DataSourcePluginConfigType } from '../config'; +import { LoggingAuditor } from './audit/logging_auditor'; +import { CryptographyClient } from './cryptography'; +import { DataSourceService, DataSourceServiceSetup } from './data_source_service'; +import { DataSourceSavedObjectsClientWrapper, dataSource } from './saved_objects'; +import { DataSourcePluginSetup, DataSourcePluginStart } from './types'; +import { DATA_SOURCE_SAVED_OBJECT_TYPE } from '../common'; + +// eslint-disable-next-line @osd/eslint/no-restricted-paths +import { ensureRawRequest } from '../../../../src/core/server/http/router'; +export class DataSourcePlugin implements Plugin { + private readonly logger: Logger; + private readonly dataSourceService: DataSourceService; + private readonly config$: Observable; + + constructor(private initializerContext: PluginInitializerContext) { + this.logger = this.initializerContext.logger.get(); + this.dataSourceService = new DataSourceService(this.logger.get('data-source-service')); + this.config$ = this.initializerContext.config.create(); + } + + public async setup(core: CoreSetup) { + this.logger.debug('dataSource: Setup'); + + // Register data source saved object type + core.savedObjects.registerType(dataSource); + + const config: DataSourcePluginConfigType = await this.config$.pipe(first()).toPromise(); + + // Fetch configs used to create credential saved objects client wrapper + const { wrappingKeyName, wrappingKeyNamespace, wrappingKey } = config.encryption; + + // Create data source saved objects client wrapper + const cryptographyClient = new CryptographyClient( + wrappingKeyName, + wrappingKeyNamespace, + wrappingKey + ); + const dataSourceSavedObjectsClientWrapper = new DataSourceSavedObjectsClientWrapper( + cryptographyClient + ); + + // Add data source saved objects client wrapper factory + core.savedObjects.addClientWrapper( + 1, + DATA_SOURCE_SAVED_OBJECT_TYPE, + dataSourceSavedObjectsClientWrapper.wrapperFactory + ); + + const dataSourceService: DataSourceServiceSetup = await this.dataSourceService.setup(config); + + core.logging.configure( + this.config$.pipe( + map((dataSourceConfig) => ({ + appenders: { + auditTrailAppender: dataSourceConfig.audit.appender, + }, + loggers: [ + { + context: 'audit', + level: dataSourceConfig.audit.enabled ? 'info' : 'off', + appenders: ['auditTrailAppender'], + }, + ], + })) + ) + ); + + const auditorFactory: AuditorFactory = { + asScoped: (request: OpenSearchDashboardsRequest) => { + return new LoggingAuditor(request, this.logger.get('audit')); + }, + }; + core.auditTrail.register(auditorFactory); + const auditTrailPromise = core.getStartServices().then(([coreStart]) => coreStart.auditTrail); + + // Register data source plugin context to route handler context + core.http.registerRouteHandlerContext( + 'dataSource', + this.createDataSourceRouteHandlerContext( + dataSourceService, + cryptographyClient, + this.logger, + auditTrailPromise + ) + ); + + return {}; + } + + public start(core: CoreStart) { + this.logger.debug('dataSource: Started'); + + return {}; + } + + public stop() { + this.dataSourceService!.stop(); + } + + private createDataSourceRouteHandlerContext = ( + dataSourceService: DataSourceServiceSetup, + cryptographyClient: CryptographyClient, + logger: Logger, + auditTrailPromise: Promise + ): IContextProvider, 'dataSource'> => { + return (context, req) => { + return { + opensearch: { + getClient: (dataSourceId: string) => { + const auditor = auditTrailPromise.then((auditTrail) => auditTrail.asScoped(req)); + this.logAuditMessage(auditor, dataSourceId, req); + + return dataSourceService.getDataSourceClient( + dataSourceId, + context.core.savedObjects.client, + cryptographyClient + ); + }, + }, + }; + }; + }; + + private async logAuditMessage( + auditorPromise: Promise, + dataSourceId: string, + request: OpenSearchDashboardsRequest + ) { + const auditor = await auditorPromise; + const auditMessage = this.getAuditMessage(request, dataSourceId); + + auditor.add({ + message: auditMessage, + type: 'opensearch.dataSourceClient.fetchClient', + }); + } + + private getAuditMessage(request: OpenSearchDashboardsRequest, dataSourceId: string) { + const rawRequest = ensureRawRequest(request); + const remoteAddress = rawRequest?.info?.remoteAddress; + const xForwardFor = request.headers['x-forwarded-for']; + const forwarded = request.headers.forwarded; + const forwardedInfo = forwarded ? forwarded : xForwardFor; + + return forwardedInfo + ? `${remoteAddress} attempted accessing through ${forwardedInfo} on data source: ${dataSourceId}` + : `${remoteAddress} attempted accessing on data source: ${dataSourceId}`; + } +} diff --git a/src/plugins/data_source/server/saved_objects/data_source.ts b/src/plugins/data_source/server/saved_objects/data_source.ts new file mode 100644 index 00000000000..9404a4bcf37 --- /dev/null +++ b/src/plugins/data_source/server/saved_objects/data_source.ts @@ -0,0 +1,37 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { SavedObjectsType } from 'opensearch-dashboards/server'; + +export const dataSource: SavedObjectsType = { + name: 'data-source', + namespaceType: 'agnostic', + hidden: false, + management: { + icon: 'apps', // todo: pending ux #2034 + defaultSearchField: 'title', + importableAndExportable: true, + getTitle(obj) { + return obj.attributes.title; + }, + getEditUrl(obj) { + return `/management/opensearch-dashboards/dataSources/${encodeURIComponent(obj.id)}`; + }, + getInAppUrl(obj) { + return { + path: `/app/management/opensearch-dashboards/dataSources/${encodeURIComponent(obj.id)}`, + uiCapabilitiesPath: 'management.opensearchDashboards.dataSources', + }; + }, + }, + mappings: { + dynamic: false, + properties: { + title: { + type: 'text', + }, + }, + }, +}; diff --git a/src/plugins/data_source/server/saved_objects/data_source_saved_objects_client_wrapper.ts b/src/plugins/data_source/server/saved_objects/data_source_saved_objects_client_wrapper.ts new file mode 100644 index 00000000000..b84dee70552 --- /dev/null +++ b/src/plugins/data_source/server/saved_objects/data_source_saved_objects_client_wrapper.ts @@ -0,0 +1,268 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { + SavedObjectsBulkCreateObject, + SavedObjectsBulkResponse, + SavedObjectsBulkUpdateObject, + SavedObjectsBulkUpdateOptions, + SavedObjectsBulkUpdateResponse, + SavedObjectsClientWrapperFactory, + SavedObjectsCreateOptions, + SavedObjectsUpdateOptions, + SavedObjectsUpdateResponse, +} from 'opensearch-dashboards/server'; + +import { SavedObjectsErrorHelpers } from '../../../../core/server'; + +import { CryptographyClient } from '../cryptography'; + +import { DATA_SOURCE_SAVED_OBJECT_TYPE } from '../../common'; +import { AuthType } from '../../common/data_sources'; + +/** + * Describes the Credential Saved Objects Client Wrapper class, + * which contains the factory used to create Saved Objects Client Wrapper instances + */ +export class DataSourceSavedObjectsClientWrapper { + constructor(private cryptographyClient: CryptographyClient) {} + + /** + * Describes the factory used to create instances of Saved Objects Client Wrappers + * for data source spcific operations such as credntials encryption + */ + public wrapperFactory: SavedObjectsClientWrapperFactory = (wrapperOptions) => { + const createWithCredentialsEncryption = async ( + type: string, + attributes: T, + options?: SavedObjectsCreateOptions + ) => { + if (DATA_SOURCE_SAVED_OBJECT_TYPE !== type) { + return await wrapperOptions.client.create(type, attributes, options); + } + + const encryptedAttributes = await this.validateAndEncryptAttributes(attributes); + + return await wrapperOptions.client.create(type, encryptedAttributes, options); + }; + + const bulkCreateWithCredentialsEncryption = async ( + objects: Array>, + options?: SavedObjectsCreateOptions + ): Promise> => { + objects = await Promise.all( + objects.map(async (object) => { + const { type, attributes } = object; + + if (DATA_SOURCE_SAVED_OBJECT_TYPE !== type) { + return object; + } + + return { + ...object, + attributes: await this.validateAndEncryptAttributes(attributes), + }; + }) + ); + return await wrapperOptions.client.bulkCreate(objects, options); + }; + + const updateWithCredentialsEncryption = async ( + type: string, + id: string, + attributes: Partial, + options: SavedObjectsUpdateOptions = {} + ): Promise> => { + if (DATA_SOURCE_SAVED_OBJECT_TYPE !== type) { + return await wrapperOptions.client.update(type, id, attributes, options); + } + + const encryptedAttributes: Partial = await this.validateAndUpdatePartialAttributes( + attributes + ); + + return await wrapperOptions.client.update(type, id, encryptedAttributes, options); + }; + + const bulkUpdateWithCredentialsEncryption = async ( + objects: Array>, + options?: SavedObjectsBulkUpdateOptions + ): Promise> => { + objects = await Promise.all( + objects.map(async (object) => { + const { type, attributes } = object; + + if (DATA_SOURCE_SAVED_OBJECT_TYPE !== type) { + return object; + } + + const encryptedAttributes: Partial = await this.validateAndUpdatePartialAttributes( + attributes + ); + + return { + ...object, + attributes: encryptedAttributes, + }; + }) + ); + + return await wrapperOptions.client.bulkUpdate(objects, options); + }; + + return { + ...wrapperOptions.client, + create: createWithCredentialsEncryption, + bulkCreate: bulkCreateWithCredentialsEncryption, + checkConflicts: wrapperOptions.client.checkConflicts, + delete: wrapperOptions.client.delete, + find: wrapperOptions.client.find, + bulkGet: wrapperOptions.client.bulkGet, + get: wrapperOptions.client.get, + update: updateWithCredentialsEncryption, + bulkUpdate: bulkUpdateWithCredentialsEncryption, + errors: wrapperOptions.client.errors, + addToNamespaces: wrapperOptions.client.addToNamespaces, + deleteFromNamespaces: wrapperOptions.client.deleteFromNamespaces, + }; + }; + + private isValidUrl(endpoint: string) { + try { + const url = new URL(endpoint); + return Boolean(url) && (url.protocol === 'http:' || url.protocol === 'https:'); + } catch (e) { + return false; + } + } + + private async validateAndEncryptAttributes(attributes: T) { + this.validateAttributes(attributes); + + const { auth } = attributes; + + switch (auth.type) { + case AuthType.NoAuth: + return { + ...attributes, + // Drop the credentials attribute for no_auth + credentials: undefined, + }; + case AuthType.UsernamePasswordType: + return { + ...attributes, + auth: await this.encryptCredentials(auth), + }; + default: + throw SavedObjectsErrorHelpers.createBadRequestError(`Invalid auth type: '${type}'`); + } + } + + private async validateAndUpdatePartialAttributes(attributes: T) { + const { auth, endpoint } = attributes; + + if (endpoint) { + throw SavedObjectsErrorHelpers.createBadRequestError( + `Updating a dataSource endpoint is not supported` + ); + } + + if (auth === undefined) { + return attributes; + } + + const { type, credentials } = auth; + + switch (type) { + case AuthType.NoAuth: + return { + ...attributes, + // Drop the credentials attribute for no_auth + credentials: undefined, + }; + case AuthType.UsernamePasswordType: + if (credentials?.password) { + return { + ...attributes, + auth: await this.encryptCredentials(auth), + }; + } else { + return attributes; + } + default: + throw SavedObjectsErrorHelpers.createBadRequestError(`Invalid credentials type: '${type}'`); + } + } + + private validateAttributes(attributes: T) { + const { title, endpoint, auth } = attributes; + if (!title?.trim?.().length) { + throw SavedObjectsErrorHelpers.createBadRequestError( + '"title" attribute must be a non-empty string' + ); + } + + if (!this.isValidUrl(endpoint)) { + throw SavedObjectsErrorHelpers.createBadRequestError('"endpoint" attribute is not valid'); + } + + if (auth === undefined) { + throw SavedObjectsErrorHelpers.createBadRequestError('"auth" attribute is required'); + } + + this.validateAuth(auth); + } + + private validateAuth(auth: T) { + const { type, credentials } = auth; + + if (!type) { + throw SavedObjectsErrorHelpers.createBadRequestError('"auth.type" attribute is required'); + } + + switch (type) { + case AuthType.NoAuth: + break; + case AuthType.UsernamePasswordType: + if (credentials === undefined) { + throw SavedObjectsErrorHelpers.createBadRequestError( + '"auth.credentials" attribute is required' + ); + } + + const { username, password } = credentials; + + if (!username) { + throw SavedObjectsErrorHelpers.createBadRequestError( + '"auth.credentials.username" attribute is required' + ); + } + + if (!password) { + throw SavedObjectsErrorHelpers.createBadRequestError( + '"auth.credentials.password" attribute is required' + ); + } + + break; + default: + throw SavedObjectsErrorHelpers.createBadRequestError(`Invalid auth type: '${type}'`); + } + } + + private async encryptCredentials(auth: T) { + const { + credentials: { username, password }, + } = auth; + + return { + ...auth, + credentials: { + username, + password: await this.cryptographyClient.encryptAndEncode(password), + }, + }; + } +} diff --git a/src/plugins/data_source/server/saved_objects/index.ts b/src/plugins/data_source/server/saved_objects/index.ts new file mode 100644 index 00000000000..76a332b84ce --- /dev/null +++ b/src/plugins/data_source/server/saved_objects/index.ts @@ -0,0 +1,7 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +export { dataSource } from './data_source'; +export { DataSourceSavedObjectsClientWrapper } from './data_source_saved_objects_client_wrapper'; diff --git a/src/plugins/data_source/server/types.ts b/src/plugins/data_source/server/types.ts new file mode 100644 index 00000000000..bad309b4b87 --- /dev/null +++ b/src/plugins/data_source/server/types.ts @@ -0,0 +1,22 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { OpenSearchClient } from 'src/core/server'; + +export interface DataSourcePluginRequestContext { + opensearch: { + getClient: (dataSourceId: string) => Promise; + }; +} +declare module 'src/core/server' { + interface RequestHandlerContext { + dataSource: DataSourcePluginRequestContext; + } +} + +// eslint-disable-next-line @typescript-eslint/no-empty-interface +export interface DataSourcePluginSetup {} +// eslint-disable-next-line @typescript-eslint/no-empty-interface +export interface DataSourcePluginStart {} diff --git a/src/plugins/data_source_management/README.md b/src/plugins/data_source_management/README.md new file mode 100755 index 00000000000..6d8556a1f32 --- /dev/null +++ b/src/plugins/data_source_management/README.md @@ -0,0 +1,11 @@ +# dataSourceManagement + +An OpenSearch Dashboards plugin + +--- + +## Development + +See the [OpenSearch Dashboards contributing +guide](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/CONTRIBUTING.md) for instructions +setting up your development environment. diff --git a/src/plugins/data_source_management/common/index.ts b/src/plugins/data_source_management/common/index.ts new file mode 100644 index 00000000000..e42b0c3fc51 --- /dev/null +++ b/src/plugins/data_source_management/common/index.ts @@ -0,0 +1,7 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +export const PLUGIN_ID = 'dataSourceManagement'; +export const PLUGIN_NAME = 'Data Sources'; diff --git a/src/plugins/data_source_management/opensearch_dashboards.json b/src/plugins/data_source_management/opensearch_dashboards.json new file mode 100644 index 00000000000..39877108d9c --- /dev/null +++ b/src/plugins/data_source_management/opensearch_dashboards.json @@ -0,0 +1,9 @@ +{ + "id": "dataSourceManagement", + "version": "opensearchDashboards", + "server": false, + "ui": true, + "requiredPlugins": ["management", "dataSource"], + "optionalPlugins": [], + "requiredBundles": ["opensearchDashboardsReact"] +} diff --git a/src/plugins/data_source_management/public/components/breadcrumbs.test.ts b/src/plugins/data_source_management/public/components/breadcrumbs.test.ts new file mode 100644 index 00000000000..a99cabc4596 --- /dev/null +++ b/src/plugins/data_source_management/public/components/breadcrumbs.test.ts @@ -0,0 +1,29 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { getCreateBreadcrumbs, getEditBreadcrumbs, getListBreadcrumbs } from './breadcrumbs'; +import { mockDataSourceAttributesWithAuth } from '../mocks'; + +describe('DataSourceManagement: breadcrumbs.ts', () => { + test('get listing breadcrumb', () => { + const bc = getListBreadcrumbs(); + expect(bc[0].text).toBe('Data Sources'); + expect(bc[0].href).toBe('/'); + }); + + test('get create breadcrumb', () => { + const bc = getCreateBreadcrumbs(); + expect(bc.length).toBe(2); + expect(bc[1].text).toBe('Create data source'); + expect(bc[1].href).toBe('/create'); + }); + + test('get edit breadcrumb', () => { + const bc = getEditBreadcrumbs(mockDataSourceAttributesWithAuth); + expect(bc.length).toBe(2); + expect(bc[1].text).toBe(mockDataSourceAttributesWithAuth.title); + expect(bc[1].href).toBe(`/${mockDataSourceAttributesWithAuth.id}`); + }); +}); diff --git a/src/plugins/data_source_management/public/components/breadcrumbs.ts b/src/plugins/data_source_management/public/components/breadcrumbs.ts new file mode 100644 index 00000000000..8287ea1e738 --- /dev/null +++ b/src/plugins/data_source_management/public/components/breadcrumbs.ts @@ -0,0 +1,40 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { i18n } from '@osd/i18n'; +import { DataSourceAttributes } from '../types'; + +export function getListBreadcrumbs() { + return [ + { + text: i18n.translate('dataSourcesManagement.dataSources.listBreadcrumb', { + defaultMessage: 'Data Sources', + }), + href: `/`, + }, + ]; +} + +export function getCreateBreadcrumbs() { + return [ + ...getListBreadcrumbs(), + { + text: i18n.translate('dataSourcesManagement.dataSources.createBreadcrumb', { + defaultMessage: 'Create data source', + }), + href: `/create`, + }, + ]; +} + +export function getEditBreadcrumbs(dataSource: DataSourceAttributes) { + return [ + ...getListBreadcrumbs(), + { + text: dataSource.title, + href: `/${dataSource.id}`, + }, + ]; +} diff --git a/src/plugins/data_source_management/public/components/create_button/__snapshots__/create_button.test.tsx.snap b/src/plugins/data_source_management/public/components/create_button/__snapshots__/create_button.test.tsx.snap new file mode 100644 index 00000000000..9cda669ba75 --- /dev/null +++ b/src/plugins/data_source_management/public/components/create_button/__snapshots__/create_button.test.tsx.snap @@ -0,0 +1,16 @@ +// Jest Snapshot v1, https://goo.gl/fbAQLP + +exports[`CreateButton should render normally 1`] = ` + + + +`; diff --git a/src/plugins/data_source_management/public/components/create_button/create_button.test.tsx b/src/plugins/data_source_management/public/components/create_button/create_button.test.tsx new file mode 100644 index 00000000000..64906045b00 --- /dev/null +++ b/src/plugins/data_source_management/public/components/create_button/create_button.test.tsx @@ -0,0 +1,31 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import React from 'react'; +import { shallow, ShallowWrapper } from 'enzyme'; +import { CreateButton } from './create_button'; +import { scopedHistoryMock } from '../../../../../core/public/mocks'; +import { ScopedHistory } from 'opensearch-dashboards/public'; + +const createButtonIdentifier = `[data-test-subj="createDataSourceButton"]`; + +describe('CreateButton', () => { + const history = (scopedHistoryMock.create() as unknown) as ScopedHistory; + let component: ShallowWrapper, React.Component<{}, {}, any>>; + + beforeEach(() => { + component = shallow(); + }); + + it('should render normally', () => { + expect(component).toMatchSnapshot(); + }); + + it('should click event normally', () => { + component.find(createButtonIdentifier).first().simulate('click'); + + expect(history.push).toBeCalledWith('/create'); + }); +}); diff --git a/src/plugins/data_source_management/public/components/create_button/create_button.tsx b/src/plugins/data_source_management/public/components/create_button/create_button.tsx new file mode 100644 index 00000000000..f5fca8b2789 --- /dev/null +++ b/src/plugins/data_source_management/public/components/create_button/create_button.tsx @@ -0,0 +1,30 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import React from 'react'; +import { History } from 'history'; + +import { EuiButton } from '@elastic/eui'; +import { FormattedMessage } from '@osd/i18n/react'; + +interface Props { + history: History; +} + +export const CreateButton = ({ history }: Props) => { + return ( + history.push('/create')} + iconType="plusInCircle" + > + + + ); +}; diff --git a/src/plugins/data_source_management/public/components/create_button/index.tsx b/src/plugins/data_source_management/public/components/create_button/index.tsx new file mode 100644 index 00000000000..84ddd6830b7 --- /dev/null +++ b/src/plugins/data_source_management/public/components/create_button/index.tsx @@ -0,0 +1,6 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +export { CreateButton } from './create_button'; diff --git a/src/plugins/data_source_management/public/components/create_data_source_wizard/__snapshots__/create_data_source_wizard.test.tsx.snap b/src/plugins/data_source_management/public/components/create_data_source_wizard/__snapshots__/create_data_source_wizard.test.tsx.snap new file mode 100644 index 00000000000..cbdcf40a647 --- /dev/null +++ b/src/plugins/data_source_management/public/components/create_data_source_wizard/__snapshots__/create_data_source_wizard.test.tsx.snap @@ -0,0 +1,1058 @@ +// Jest Snapshot v1, https://goo.gl/fbAQLP + +exports[`Datasource Management: Create Datasource Wizard should render normally 1`] = ` + + + + + +
+
+ +
+ +
+
+ +

+ Create data source connection +

+ + +
+ + +
+

+ + + A data source is an OpenSearch cluster endpoint (for now) to query against. + + +
+ + + + + Read documentation + + + + + + + + + (opens in a new tab or window) + + + + + +

+
+ +
+
+ +
+ +
+ +
+ + +
+ +
+
+ + + Connection Details + + +
+
+ + +
+ + +
+
+ + + +
+
+ + +
+
+ + + + +
+
+ + +
+
+ + +
+
+ + + +
+
+ + +
+
+ + + + +
+
+ + +
+
+ + +
+
+ + + +
+
+ + +
+
+ + + + +
+
+ + +
+
+ + +
+ + +
+
+ + + Authentication + + +
+
+ + +
+ + +
+
+ + + +
+
+ + + } + isOpen={false} + panelPaddingSize="none" + > + + [Function] + + } + buttonRef={[Function]} + className="euiInputPopover euiSuperSelect" + closePopover={[Function]} + display="block" + hasArrow={true} + isOpen={false} + ownFocus={false} + panelPaddingSize="none" + panelRef={[Function]} + > + +
+
+ +
+ + + +
+
+ + + + Select an option: Username & Password, is selected + + + + + +
+ + + + + +
+ +
+
+ + +
+ +
+
+ + + + +
+
+ + +
+
+ + + +
+
+ + +
+
+ + + + +
+
+ + +
+
+ + +
+
+ + + +
+
+ + , + ] + } + compressed={false} + fullWidth={false} + icon="lock" + isLoading={false} + > +
+
+ + + + +
+ + + + + +
+ +
+ + + +
+ + +
+
+ + +
+ + + + + + +
+ +
+ + + + +
+ + + +`; diff --git a/src/plugins/data_source_management/public/components/create_data_source_wizard/components/create_form/__snapshots__/create_data_source_form.test.tsx.snap b/src/plugins/data_source_management/public/components/create_data_source_wizard/components/create_form/__snapshots__/create_data_source_form.test.tsx.snap new file mode 100644 index 00000000000..923fcff47b4 --- /dev/null +++ b/src/plugins/data_source_management/public/components/create_data_source_wizard/components/create_form/__snapshots__/create_data_source_form.test.tsx.snap @@ -0,0 +1,5176 @@ +// Jest Snapshot v1, https://goo.gl/fbAQLP + +exports[`Datasource Management: Create Datasource form should create data source with No Auth when all fields are valid 1`] = ` + + + + +
+
+ +
+ +
+
+ +

+ Create data source connection +

+ + +
+ + +
+

+ + + A data source is an OpenSearch cluster endpoint (for now) to query against. + + +
+ + + + + Read documentation + + + + + + + + + (opens in a new tab or window) + + + + + +

+
+ +
+
+ +
+ +
+ +
+ + +
+ +
+
+ + + Connection Details + + +
+
+ + +
+ + +
+
+ + + +
+
+ + +
+
+ + + + +
+
+ + +
+
+ + +
+
+ + + +
+
+ + +
+
+ + + + +
+
+ + +
+
+ + +
+
+ + + +
+
+ + +
+
+ + + + +
+
+ + +
+
+ + +
+ + +
+
+ + + Authentication + + +
+
+ + +
+ + +
+
+ + + +
+
+ + + } + isOpen={false} + panelPaddingSize="none" + > + + [Function] + + } + buttonRef={[Function]} + className="euiInputPopover euiSuperSelect" + closePopover={[Function]} + display="block" + hasArrow={true} + isOpen={false} + ownFocus={false} + panelPaddingSize="none" + panelRef={[Function]} + > + +
+
+ +
+ + + +
+
+ + + + Select an option: No authentication, is selected + + + + + +
+ + + + + +
+ +
+
+ + +
+ +
+
+ + + + +
+
+ + +
+ + + + + + +
+ +
+ + + + +`; + +exports[`Datasource Management: Create Datasource form should create data source with username & password when all fields are valid 1`] = ` + + + + +
+
+ +
+ +
+
+ +

+ Create data source connection +

+ + +
+ + +
+

+ + + A data source is an OpenSearch cluster endpoint (for now) to query against. + + +
+ + + + + Read documentation + + + + + + + + + (opens in a new tab or window) + + + + + +

+
+ +
+
+ +
+ +
+ +
+ + +
+ +
+
+ + + Connection Details + + +
+
+ + +
+ + +
+
+ + + +
+
+ + +
+
+ + + + +
+
+ + +
+
+ + +
+
+ + + +
+
+ + +
+
+ + + + +
+
+ + +
+
+ + +
+
+ + + +
+
+ + +
+
+ + + + +
+
+ + +
+
+ + +
+ + +
+
+ + + Authentication + + +
+
+ + +
+ + +
+
+ + + +
+
+ + + } + isOpen={false} + panelPaddingSize="none" + > + + [Function] + + } + buttonRef={[Function]} + className="euiInputPopover euiSuperSelect" + closePopover={[Function]} + display="block" + hasArrow={true} + isOpen={false} + ownFocus={false} + panelPaddingSize="none" + panelRef={[Function]} + > + +
+
+ +
+ + + +
+
+ + + + Select an option: Username & Password, is selected + + + + + +
+ + + + + +
+ +
+
+ + +
+ +
+
+ + + + +
+
+ + +
+
+ + + +
+
+ + +
+
+ + + + +
+
+ + +
+
+ + +
+
+ + + +
+
+ + , + ] + } + compressed={false} + fullWidth={false} + icon="lock" + isLoading={false} + > +
+
+ + + + +
+ + + + + +
+ +
+ + + +
+ + +
+
+ + +
+ + + + + + +
+ +
+ + + + +`; + +exports[`Datasource Management: Create Datasource form should render normally 1`] = ` + + + + +
+
+ +
+ +
+
+ +

+ Create data source connection +

+ + +
+ + +
+

+ + + A data source is an OpenSearch cluster endpoint (for now) to query against. + + +
+ + + + + Read documentation + + + + + + + + + (opens in a new tab or window) + + + + + +

+
+ +
+
+ +
+ +
+ +
+ + +
+ +
+
+ + + Connection Details + + +
+
+ + +
+ + +
+
+ + + +
+
+ + +
+
+ + + + +
+
+ + +
+
+ + +
+
+ + + +
+
+ + +
+
+ + + + +
+
+ + +
+
+ + +
+
+ + + +
+
+ + +
+
+ + + + +
+
+ + +
+
+ + +
+ + +
+
+ + + Authentication + + +
+
+ + +
+ + +
+
+ + + +
+
+ + + } + isOpen={false} + panelPaddingSize="none" + > + + [Function] + + } + buttonRef={[Function]} + className="euiInputPopover euiSuperSelect" + closePopover={[Function]} + display="block" + hasArrow={true} + isOpen={false} + ownFocus={false} + panelPaddingSize="none" + panelRef={[Function]} + > + +
+
+ +
+ + + +
+
+ + + + Select an option: Username & Password, is selected + + + + + +
+ + + + + +
+ +
+
+ + +
+ +
+
+ + + + +
+
+ + +
+
+ + + +
+
+ + +
+
+ + + + +
+
+ + +
+
+ + +
+
+ + + +
+
+ + , + ] + } + compressed={false} + fullWidth={false} + icon="lock" + isLoading={false} + > +
+
+ + + + +
+ + + + + +
+ +
+ + + +
+ + +
+
+ + +
+ + + + + + +
+ +
+ + + + +`; + +exports[`Datasource Management: Create Datasource form should throw validation error when title is not valid & remove error on update valid title 1`] = ` + + + + +
+
+ +
+ +
+
+ +

+ Create data source connection +

+ + +
+ + +
+

+ + + A data source is an OpenSearch cluster endpoint (for now) to query against. + + +
+ + + + + Read documentation + + + + + + + + + (opens in a new tab or window) + + + + + +

+
+ +
+
+ +
+ +
+ +
+ + +
+ + +
+
+ + Please address the highlighted errors. + +
+ +
+ +
+
    +
  • + Title must not be empty +
    • +
+
+ +
+ +
+ + + +
+
+ + + Connection Details + + +
+
+ + +
+ + +
+
+ + + +
+
+ + +
+
+ + + + +
+
+ + + +
+ Title must not be empty +
+ +
+
+ + +
+
+ + + +
+
+ + +
+
+ + + + +
+
+ + +
+
+ + +
+
+ + + +
+
+ + +
+
+ + + + +
+
+ + +
+
+ + +
+ + +
+
+ + + Authentication + + +
+
+ + +
+ + +
+
+ + + +
+
+ + + } + isOpen={false} + panelPaddingSize="none" + > + + [Function] + + } + buttonRef={[Function]} + className="euiInputPopover euiSuperSelect" + closePopover={[Function]} + display="block" + hasArrow={true} + isOpen={false} + ownFocus={false} + panelPaddingSize="none" + panelRef={[Function]} + > + +
+
+ +
+ + + +
+
+ + + + Select an option: Username & Password, is selected + + + + + +
+ + + + + +
+ +
+
+ + +
+ +
+
+ + + + +
+
+ + +
+
+ + + +
+
+ + +
+
+ + + + +
+
+ + +
+
+ + +
+
+ + + +
+
+ + , + ] + } + compressed={false} + fullWidth={false} + icon="lock" + isLoading={false} + > +
+
+ + + + +
+ + + + + +
+ +
+ + + +
+ + +
+
+ + +
+ + + + + + +
+ +
+ + + + +`; + +exports[`Datasource Management: Create Datasource form should validate when submit button is clicked without any user input on any field 1`] = ` + + + + +
+
+ +
+ +
+
+ +

+ Create data source connection +

+ + +
+ + +
+

+ + + A data source is an OpenSearch cluster endpoint (for now) to query against. + + +
+ + + + + Read documentation + + + + + + + + + (opens in a new tab or window) + + + + + +

+
+ +
+
+ +
+ +
+ +
+ + +
+ + +
+
+ + Please address the highlighted errors. + +
+ +
+ +
+
    +
  • + Title must not be empty +
    • +
  • + Endpoint is not valid +
    • +
  • + Username should not be empty +
    • +
  • + Password should not be empty +
    • +
+
+ +
+ +
+ + + +
+
+ + + Connection Details + + +
+
+ + +
+ + +
+
+ + + +
+
+ + +
+
+ + + + +
+
+ + + +
+ Title must not be empty +
+ +
+
+ + +
+
+ + + +
+
+ + +
+
+ + + + +
+
+ + +
+
+ + +
+
+ + + +
+
+ + +
+
+ + + + +
+
+ + + +
+ Endpoint is not valid +
+ +
+
+ + +
+ + +
+
+ + + Authentication + + +
+
+ + +
+ + +
+
+ + + +
+
+ + + } + isOpen={false} + panelPaddingSize="none" + > + + [Function] + + } + buttonRef={[Function]} + className="euiInputPopover euiSuperSelect" + closePopover={[Function]} + display="block" + hasArrow={true} + isOpen={false} + ownFocus={false} + panelPaddingSize="none" + panelRef={[Function]} + > + +
+
+ +
+ + + +
+
+ + + + Select an option: Username & Password, is selected + + + + + +
+ + + + + +
+ +
+
+ + +
+ +
+
+ + + + +
+
+ + +
+
+ + + +
+
+ + +
+
+ + + + +
+
+ + + +
+ Username should not be empty +
+ +
+
+ + +
+
+ + + +
+
+ + , + ] + } + compressed={false} + fullWidth={false} + icon="lock" + isLoading={false} + > +
+
+ + + + +
+ + + + + +
+ +
+ + + +
+ + + +
+ Password should not be empty +
+ +
+
+ + +
+ + + + + + +
+ +
+ + + + +`; diff --git a/src/plugins/data_source_management/public/components/create_data_source_wizard/components/create_form/create_data_source_form.test.tsx b/src/plugins/data_source_management/public/components/create_data_source_wizard/components/create_form/create_data_source_form.test.tsx new file mode 100644 index 00000000000..54b33a0a0bb --- /dev/null +++ b/src/plugins/data_source_management/public/components/create_data_source_wizard/components/create_form/create_data_source_form.test.tsx @@ -0,0 +1,162 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import React from 'react'; +import { mockManagementPlugin } from '../../../../mocks'; +import { mount, ReactWrapper } from 'enzyme'; +import { wrapWithIntl } from 'test_utils/enzyme_helpers'; +import { OpenSearchDashboardsContextProvider } from '../../../../../../opensearch_dashboards_react/public'; +import { CreateDataSourceForm } from './create_data_source_form'; +import { findTestSubject } from '@elastic/eui/lib/test'; +import { AuthType } from '../../../../types'; + +const titleIdentifier = '[data-test-subj="createDataSourceFormTitleField"]'; +const descriptionIdentifier = `[data-test-subj="createDataSourceFormDescriptionField"]`; +const endpointIdentifier = '[data-test-subj="createDataSourceFormEndpointField"]'; +const authTypeIdentifier = '[data-test-subj="createDataSourceFormAuthTypeSelect"]'; +const usernameIdentifier = '[data-test-subj="createDataSourceFormUsernameField"]'; +const passwordIdentifier = '[data-test-subj="createDataSourceFormPasswordField"]'; + +describe('Datasource Management: Create Datasource form', () => { + const mockedContext = mockManagementPlugin.createDataSourceManagementContext(); + let component: ReactWrapper, React.Component<{}, {}, any>>; + const mockSubmitHandler = jest.fn(); + + const getFields = (comp: ReactWrapper, React.Component<{}, {}, any>>) => { + return { + title: comp.find(titleIdentifier).first(), + description: comp.find(descriptionIdentifier).first(), + endpoint: comp.find(endpointIdentifier).first(), + authType: comp.find(authTypeIdentifier).first(), + username: comp.find(usernameIdentifier).first(), + password: comp.find(passwordIdentifier).first(), + }; + }; + + const changeTextFieldValue = (testSubjId: string, value: string) => { + component.find(testSubjId).last().simulate('change', { + target: { + value, + }, + }); + }; + + const setAuthTypeValue = ( + comp: ReactWrapper, React.Component<{}, {}, any>>, + value: AuthType + ) => { + // @ts-ignore + comp + .find(authTypeIdentifier) + .first() + // @ts-ignore + .prop('onChange')(value); + comp.update(); + }; + + beforeEach(() => { + component = mount(wrapWithIntl(), { + wrappingComponent: OpenSearchDashboardsContextProvider, + wrappingComponentProps: { + services: mockedContext, + }, + }); + }); + + /* Scenario 1: Should render the page normally*/ + test('should render normally', () => { + expect(component).toMatchSnapshot(); + }); + + /* Scenario 2: submit without any input from user - should display validation error messages*/ + /* Default option: Username & Password*/ + test('should validate when submit button is clicked without any user input on any field', () => { + findTestSubject(component, 'createDataSourceButton').simulate('click'); + const { title, description, endpoint, authType, username, password } = getFields(component); + expect(component).toMatchSnapshot(); + expect(title.prop('isInvalid')).toBe(true); + expect(description.prop('isInvalid')).toBe(undefined); + expect(endpoint.prop('isInvalid')).toBe(true); + expect(authType.prop('isInvalid')).toBe(false); + expect(username.prop('isInvalid')).toBe(true); + expect(password.prop('isInvalid')).toBe(true); + }); + + /* Change option: No Authentication */ + test('should validate when auth type changed & previously submit button clicked', () => { + /* Update Eui Super Select Value to No Auth*/ + setAuthTypeValue(component, AuthType.NoAuth); + component.update(); + + /* Click on submit without any user input */ + findTestSubject(component, 'createDataSourceButton').simulate('click'); + + const { title, description, endpoint, authType, username, password } = getFields(component); + + expect(authType.prop('valueOfSelected')).toBe(AuthType.NoAuth); + expect(title.prop('isInvalid')).toBe(true); + expect(description.prop('isInvalid')).toBe(undefined); + expect(endpoint.prop('isInvalid')).toBe(true); + expect(authType.prop('isInvalid')).toBe(false); + expect(username.exists()).toBeFalsy(); // username field does not exist when No Auth option is selected + expect(password.exists()).toBeFalsy(); // password field does not exist when No Auth option is selected + }); + + test('should throw validation error when title is not valid & remove error on update valid title', () => { + changeTextFieldValue(descriptionIdentifier, 'test'); + changeTextFieldValue(endpointIdentifier, 'https://test.com'); + changeTextFieldValue(usernameIdentifier, 'test123'); + changeTextFieldValue(passwordIdentifier, 'test123'); + + findTestSubject(component, 'createDataSourceButton').simulate('click'); + + const { title, description, endpoint, authType, username, password } = getFields(component); + + expect(component).toMatchSnapshot(); + expect(title.prop('isInvalid')).toBe(true); + expect(description.prop('isInvalid')).toBe(undefined); + expect(endpoint.prop('isInvalid')).toBe(false); + expect(authType.prop('isInvalid')).toBe(false); + expect(username.prop('isInvalid')).toBe(false); + expect(password.prop('isInvalid')).toBe(false); + + /* Update title & remove validation*/ + findTestSubject(component, 'createDataSourceButton').simulate('click'); + changeTextFieldValue(titleIdentifier, 'test'); + findTestSubject(component, 'createDataSourceButton').simulate('click'); + expect(mockSubmitHandler).toHaveBeenCalled(); // should call submit as all fields are valid + }); + + /* Create data source with no errors */ + /* Username & Password */ + test('should create data source with username & password when all fields are valid', () => { + /* set form fields */ + setAuthTypeValue(component, AuthType.UsernamePasswordType); + changeTextFieldValue(titleIdentifier, 'test'); + changeTextFieldValue(descriptionIdentifier, 'test'); + changeTextFieldValue(endpointIdentifier, 'https://test.com'); + changeTextFieldValue(usernameIdentifier, 'test123'); + changeTextFieldValue(passwordIdentifier, 'test123'); + + findTestSubject(component, 'createDataSourceButton').simulate('click'); + + expect(component).toMatchSnapshot(); + expect(mockSubmitHandler).toHaveBeenCalled(); // should call submit as all fields are valid + }); + + /* No Auth - Username & Password */ + test('should create data source with No Auth when all fields are valid', () => { + /* set form fields */ + setAuthTypeValue(component, AuthType.NoAuth); // No auth + changeTextFieldValue(titleIdentifier, 'test'); + changeTextFieldValue(descriptionIdentifier, 'test'); + changeTextFieldValue(endpointIdentifier, 'https://test.com'); + + findTestSubject(component, 'createDataSourceButton').simulate('click'); + + expect(component).toMatchSnapshot(); + expect(mockSubmitHandler).toHaveBeenCalled(); // should call submit as all fields are valid + }); +}); diff --git a/src/plugins/data_source_management/public/components/create_data_source_wizard/components/create_form/create_data_source_form.tsx b/src/plugins/data_source_management/public/components/create_data_source_wizard/components/create_form/create_data_source_form.tsx new file mode 100644 index 00000000000..f278431d091 --- /dev/null +++ b/src/plugins/data_source_management/public/components/create_data_source_wizard/components/create_form/create_data_source_form.tsx @@ -0,0 +1,323 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import React from 'react'; +import { + EuiButton, + EuiFieldPassword, + EuiFieldText, + EuiForm, + EuiFormRow, + EuiHorizontalRule, + EuiPageContent, + EuiSpacer, + EuiSuperSelect, + EuiText, +} from '@elastic/eui'; +import { FormattedMessage } from '@osd/i18n/react'; +import { credentialSourceOptions, DataSourceManagementContextValue } from '../../../../types'; +import { Header } from '../header'; +import { context as contextType } from '../../../../../../opensearch_dashboards_react/public'; +import { + CreateEditDataSourceValidation, + defaultValidation, + performDataSourceFormValidation, +} from '../../../validation'; +import { AuthType, DataSourceAttributes, UsernamePasswordTypedContent } from '../../../../types'; +import { + createDataSourceCredentialSource, + createDataSourceDescriptionPlaceholder, + createDataSourceEndpointPlaceholder, + createDataSourceEndpointURL, + createDataSourcePasswordPlaceholder, + createDataSourceUsernamePlaceholder, + descriptionText, + passwordText, + titleText, + usernameText, +} from '../../../text_content'; + +export interface CreateDataSourceProps { + handleSubmit: (formValues: DataSourceAttributes) => void; +} +export interface CreateDataSourceState { + /* Validation */ + formErrors: string[]; + formErrorsByField: CreateEditDataSourceValidation; + /* Inputs */ + title: string; + description: string; + endpoint: string; + auth: { + type: AuthType; + credentials: UsernamePasswordTypedContent; + }; +} + +export class CreateDataSourceForm extends React.Component< + CreateDataSourceProps, + CreateDataSourceState +> { + static contextType = contextType; + public readonly context!: DataSourceManagementContextValue; + + constructor(props: CreateDataSourceProps, context: DataSourceManagementContextValue) { + super(props, context); + + this.state = { + formErrors: [], + formErrorsByField: { ...defaultValidation }, + title: '', + description: '', + endpoint: '', + auth: { + type: AuthType.UsernamePasswordType, + credentials: { + username: '', + password: '', + }, + }, + }; + } + + /* Validations */ + + isFormValid = () => { + const { formErrors, formErrorsByField } = performDataSourceFormValidation(this.state); + + this.setState({ + formErrors, + formErrorsByField, + }); + + return formErrors.length === 0; + }; + + /* Events */ + + onChangeTitle = (e: { target: { value: any } }) => { + this.setState({ title: e.target.value }, this.checkValidation); + }; + + onChangeDescription = (e: { target: { value: any } }) => { + this.setState({ description: e.target.value }, this.checkValidation); + }; + + onChangeEndpoint = (e: { target: { value: any } }) => { + this.setState({ endpoint: e.target.value }, this.checkValidation); + }; + + onChangeAuthType = (value: AuthType) => { + this.setState({ auth: { ...this.state.auth, type: value } }, this.checkValidation); + }; + + onChangeUsername = (e: { target: { value: any } }) => { + this.setState( + { + auth: { + ...this.state.auth, + credentials: { ...this.state.auth.credentials, username: e.target.value }, + }, + }, + this.checkValidation + ); + }; + + onChangePassword = (e: { target: { value: any } }) => { + this.setState( + { + auth: { + ...this.state.auth, + credentials: { ...this.state.auth.credentials, password: e.target.value }, + }, + }, + this.checkValidation + ); + }; + + checkValidation = () => { + if (this.state.formErrors.length) { + this.isFormValid(); + } + }; + + onClickCreateNewDataSource = () => { + if (this.isFormValid()) { + const formValues: DataSourceAttributes = { + title: this.state.title, + description: this.state.description, + endpoint: this.state.endpoint, + auth: { ...this.state.auth }, + }; + + /* Remove credentials object for NoAuth */ + if (this.state.auth.type === AuthType.NoAuth) { + delete formValues.auth.credentials; + } + /* Submit */ + this.props.handleSubmit(formValues); + } + }; + + /* Render methods */ + + /* Render header*/ + renderHeader = () => { + const { docLinks } = this.context.services; + return
; + }; + + /* Render Section header*/ + renderSectionHeader = (i18nId: string, defaultMessage: string) => { + return ( + <> + +
+ +
+ + + ); + }; + + /* Render create new credentials*/ + renderCreateNewCredentialsForm = () => { + return ( + <> + + + + + + + + ); + }; + + renderContent = () => { + return ( + + {this.renderHeader()} + + + {/* Endpoint section */} + {this.renderSectionHeader( + 'dataSourceManagement.connectToDataSource.connectionDetails', + 'Connection Details' + )} + + + {/* Title */} + + + + + {/* Description */} + + + + + {/* Endpoint URL */} + + + + + {/* Authentication Section: */} + + + + {this.renderSectionHeader( + 'dataSourceManagement.connectToDataSource.authenticationHeader', + 'Authentication' + )} + + {/* Credential source */} + + + this.onChangeAuthType(value)} + data-test-subj="createDataSourceFormAuthTypeSelect" + /> + + + {/* Create New credentials */} + {this.state.auth.type === AuthType.UsernamePasswordType + ? this.renderCreateNewCredentialsForm() + : null} + + + {/* Create Data Source button*/} + + Create a data source connection + + + + ); + }; + + render() { + return <>{this.renderContent()}; + } +} diff --git a/src/plugins/data_source_management/public/components/create_data_source_wizard/components/create_form/index.ts b/src/plugins/data_source_management/public/components/create_data_source_wizard/components/create_form/index.ts new file mode 100644 index 00000000000..fba4fc4a517 --- /dev/null +++ b/src/plugins/data_source_management/public/components/create_data_source_wizard/components/create_form/index.ts @@ -0,0 +1,6 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +export { CreateDataSourceForm } from './create_data_source_form'; diff --git a/src/plugins/data_source_management/public/components/create_data_source_wizard/components/header/__snapshots__/header.test.tsx.snap b/src/plugins/data_source_management/public/components/create_data_source_wizard/components/header/__snapshots__/header.test.tsx.snap new file mode 100644 index 00000000000..01005baf35a --- /dev/null +++ b/src/plugins/data_source_management/public/components/create_data_source_wizard/components/header/__snapshots__/header.test.tsx.snap @@ -0,0 +1,18 @@ +// Jest Snapshot v1, https://goo.gl/fbAQLP + +exports[`Datasource Management: Header should render normally 1`] = ` +
+`; diff --git a/src/plugins/data_source_management/public/components/create_data_source_wizard/components/header/header.test.tsx b/src/plugins/data_source_management/public/components/create_data_source_wizard/components/header/header.test.tsx new file mode 100644 index 00000000000..ce396c413eb --- /dev/null +++ b/src/plugins/data_source_management/public/components/create_data_source_wizard/components/header/header.test.tsx @@ -0,0 +1,29 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import React from 'react'; +import { Header } from '../header'; +import { shallow } from 'enzyme'; +import { DocLinksStart } from 'opensearch-dashboards/public'; +import { wrapWithIntl } from 'test_utils/enzyme_helpers'; +import { OpenSearchDashboardsContextProvider } from '../../../../../../opensearch_dashboards_react/public'; +import { docLinks } from '../../../../mocks'; +import { mockManagementPlugin } from '../../../../mocks'; + +describe('Datasource Management: Header', () => { + const mockedContext = mockManagementPlugin.createDataSourceManagementContext(); + const mockedDocLinks = docLinks as DocLinksStart; + + test('should render normally', () => { + const component = shallow(wrapWithIntl(
), { + wrappingComponent: OpenSearchDashboardsContextProvider, + wrappingComponentProps: { + services: mockedContext, + }, + }); + + expect(component).toMatchSnapshot(); + }); +}); diff --git a/src/plugins/data_source_management/public/components/create_data_source_wizard/components/header/header.tsx b/src/plugins/data_source_management/public/components/create_data_source_wizard/components/header/header.tsx new file mode 100644 index 00000000000..3a66b742614 --- /dev/null +++ b/src/plugins/data_source_management/public/components/create_data_source_wizard/components/header/header.tsx @@ -0,0 +1,53 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import React from 'react'; + +import { EuiSpacer, EuiTitle, EuiText, EuiLink, EuiFlexItem, EuiFlexGroup } from '@elastic/eui'; + +import { FormattedMessage } from '@osd/i18n/react'; +import { DocLinksStart } from 'opensearch-dashboards/public'; +import { useOpenSearchDashboards } from '../../../../../../opensearch_dashboards_react/public'; +import { DataSourceManagementContext } from '../../../../types'; +import { createDataSourceHeader } from '../../../text_content/text_content'; + +export const Header = ({ docLinks }: { docLinks: DocLinksStart }) => { + const changeTitle = useOpenSearchDashboards().services.chrome + .docTitle.change; + + changeTitle(createDataSourceHeader); + + return ( + + +
+ +

{createDataSourceHeader}

+ + + +

+ +
+ + + +

+ +
+ + + ); +}; diff --git a/src/plugins/data_source_management/public/components/create_data_source_wizard/components/header/index.ts b/src/plugins/data_source_management/public/components/create_data_source_wizard/components/header/index.ts new file mode 100644 index 00000000000..3c25d4c42f0 --- /dev/null +++ b/src/plugins/data_source_management/public/components/create_data_source_wizard/components/header/index.ts @@ -0,0 +1,6 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +export { Header } from './header'; diff --git a/src/plugins/data_source_management/public/components/create_data_source_wizard/create_data_source_wizard.test.tsx b/src/plugins/data_source_management/public/components/create_data_source_wizard/create_data_source_wizard.test.tsx new file mode 100644 index 00000000000..a1406e7947e --- /dev/null +++ b/src/plugins/data_source_management/public/components/create_data_source_wizard/create_data_source_wizard.test.tsx @@ -0,0 +1,81 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import React from 'react'; +import { mockDataSourceAttributesWithAuth, mockManagementPlugin } from '../../mocks'; +import { mount, ReactWrapper } from 'enzyme'; +import { wrapWithIntl } from 'test_utils/enzyme_helpers'; +import { OpenSearchDashboardsContextProvider } from '../../../../opensearch_dashboards_react/public'; +import { CreateDataSourceWizard } from './create_data_source_wizard'; +import { scopedHistoryMock } from '../../../../../core/public/mocks'; +import { ScopedHistory } from 'opensearch-dashboards/public'; +import { RouteComponentProps } from 'react-router-dom'; +import { act } from 'react-dom/test-utils'; +import * as utils from '../utils'; + +const formIdentifier = 'CreateDataSourceForm'; +const toastsIdentifier = 'EuiGlobalToastList'; +describe('Datasource Management: Create Datasource Wizard', () => { + const mockedContext = mockManagementPlugin.createDataSourceManagementContext(); + let component: ReactWrapper, React.Component<{}, {}, any>>; + const history = (scopedHistoryMock.create() as unknown) as ScopedHistory; + beforeEach(() => { + component = mount( + wrapWithIntl( + + ), + { + wrappingComponent: OpenSearchDashboardsContextProvider, + wrappingComponentProps: { + services: mockedContext, + }, + } + ); + }); + test('should render normally', () => { + expect(component).toMatchSnapshot(); + }); + test('should create datasource successfully', async () => { + spyOn(utils, 'createSingleDataSource').and.returnValue({}); + + await act(async () => { + // @ts-ignore + await component.find(formIdentifier).first().prop('handleSubmit')( + mockDataSourceAttributesWithAuth + ); + }); + expect(utils.createSingleDataSource).toHaveBeenCalled(); + expect(history.push).toBeCalledWith(''); + }); + test('should fail to create datasource', async () => { + spyOn(utils, 'createSingleDataSource').and.throwError('error'); + await act(async () => { + // @ts-ignore + await component.find(formIdentifier).first().prop('handleSubmit')( + mockDataSourceAttributesWithAuth + ); + }); + component.update(); + expect(utils.createSingleDataSource).toHaveBeenCalled(); + // @ts-ignore + expect(component.find(toastsIdentifier).props().toasts.length).toBe(1); // failure toast + + // remove toast message after failure of creating datasource + + act(() => { + // @ts-ignore + component.find(toastsIdentifier).first().prop('dismissToast')({ + id: 'dataSourcesManagement.createDataSource.createDataSourceFailMsg', + }); + }); + component.update(); + // @ts-ignore + expect(component.find(toastsIdentifier).props().toasts.length).toBe(0); // failure toast + }); +}); diff --git a/src/plugins/data_source_management/public/components/create_data_source_wizard/create_data_source_wizard.tsx b/src/plugins/data_source_management/public/components/create_data_source_wizard/create_data_source_wizard.tsx new file mode 100644 index 00000000000..f69205efb85 --- /dev/null +++ b/src/plugins/data_source_management/public/components/create_data_source_wizard/create_data_source_wizard.tsx @@ -0,0 +1,100 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { EuiGlobalToastList, EuiGlobalToastListToast } from '@elastic/eui'; +import React, { useState } from 'react'; +import { RouteComponentProps, withRouter } from 'react-router-dom'; +import { useEffectOnce } from 'react-use'; +import { FormattedMessage } from '@osd/i18n/react'; +import { useOpenSearchDashboards } from '../../../../opensearch_dashboards_react/public'; +import { DataSourceManagementContext, ToastMessageItem } from '../../types'; +import { getCreateBreadcrumbs } from '../breadcrumbs'; +import { CreateDataSourceForm } from './components/create_form'; +import { createSingleDataSource } from '../utils'; +import { LoadingMask } from '../loading_mask'; +import { DataSourceAttributes } from '../../types'; + +type CreateDataSourceWizardProps = RouteComponentProps; + +export const CreateDataSourceWizard: React.FunctionComponent = ( + props: CreateDataSourceWizardProps +) => { + /* Initialization */ + const { savedObjects, setBreadcrumbs } = useOpenSearchDashboards< + DataSourceManagementContext + >().services; + + const toastLifeTimeMs: number = 6000; + + /* State Variables */ + const [toasts, setToasts] = useState([]); + const [isLoading, setIsLoading] = useState(false); + + /* Set breadcrumb */ + useEffectOnce(() => { + setBreadcrumbs(getCreateBreadcrumbs()); + }); + + /* Handle submit - create data source*/ + const handleSubmit = async (attributes: DataSourceAttributes) => { + setIsLoading(true); + try { + await createSingleDataSource(savedObjects.client, attributes); + props.history.push(''); + } catch (e) { + setIsLoading(false); + handleDisplayToastMessage({ + id: 'dataSourcesManagement.createDataSource.createDataSourceFailMsg', + defaultMessage: 'Creation of the Data Source failed with some errors. Please try it again', + color: 'warning', + iconType: 'alert', + }); + } + }; + + const handleDisplayToastMessage = ({ id, defaultMessage, color, iconType }: ToastMessageItem) => { + const failureMsg = ; + setToasts([ + ...toasts, + { + title: failureMsg, + id: failureMsg.props.id, + color, + iconType, + }, + ]); + }; + + /* Render the creation wizard */ + const renderContent = () => { + return ( + <> + + {isLoading ? : null} + + ); + }; + + /* Remove toast on dismiss*/ + const removeToast = (id: string) => { + setToasts(toasts.filter((toast) => toast.id !== id)); + }; + + return ( + <> + {renderContent()} + { + removeToast(id); + }} + toastLifeTimeMs={toastLifeTimeMs} + /> + + ); +}; + +export const CreateDataSourceWizardWithRouter = withRouter(CreateDataSourceWizard); diff --git a/src/plugins/data_source_management/public/components/create_data_source_wizard/index.ts b/src/plugins/data_source_management/public/components/create_data_source_wizard/index.ts new file mode 100644 index 00000000000..a96e556dcd6 --- /dev/null +++ b/src/plugins/data_source_management/public/components/create_data_source_wizard/index.ts @@ -0,0 +1,6 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +export { CreateDataSourceWizardWithRouter } from './create_data_source_wizard'; diff --git a/src/plugins/data_source_management/public/components/data_source_table/__snapshots__/data_source_table.test.tsx.snap b/src/plugins/data_source_management/public/components/data_source_table/__snapshots__/data_source_table.test.tsx.snap new file mode 100644 index 00000000000..30b16d9ce01 --- /dev/null +++ b/src/plugins/data_source_management/public/components/data_source_table/__snapshots__/data_source_table.test.tsx.snap @@ -0,0 +1,2201 @@ +// Jest Snapshot v1, https://goo.gl/fbAQLP + +exports[`DataSourceTable should get datasources failed should show toast and remove toast normally 1`] = ` + + + + +
+ +
+ +
+ +

+ Data Sources +

+ + +
+ + +
+

+ Create and manage the data sources that help you retrieve your data from multiple Elasticsearch clusters +

+
+ +
+ + +
+ + + + + + + +
+ +
+ + +
+ + + + Delete + + connection + + + , + } + } + selection={ + Object { + "onSelectionChange": [Function], + } + } + sorting={ + Object { + "sort": Object { + "direction": "asc", + "field": "title", + }, + } + } + tableLayout="fixed" + > +
+ + + Delete + + connection + + + + } + > + +
+ +
+ + + +
+
+ + + + +
+ + + + + +
+ +
+
+ + + +
+ + +
+ +
+ + + + + +
+ +
+ +
+ + + +
+ + +
+
+ +
+ +
+ +
+ + +
+ +
+ +
+ + +
+ + +
+ +
+ + + + } + closePopover={[Function]} + display="inlineBlock" + hasArrow={true} + isOpen={false} + ownFocus={true} + panelPaddingSize="none" + > +
+
+ + + +
+
+ +
+ +
+ +
+ +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+
+ + +
+ +
+
+ + +
+
+ + + +
+
+ + No items found + +
+
+ +
+
+ +
+ +
+ + + +
+ + + +`; + +exports[`DataSourceTable should get datasources successful should render normally 1`] = ` + + + + +
+ +
+ +
+ +

+ Data Sources +

+ + +
+ + +
+

+ Create and manage the data sources that help you retrieve your data from multiple Elasticsearch clusters +

+
+ +
+ + +
+ + + + + + + +
+ +
+ + +
+ + + + Delete + + connection + + + , + } + } + selection={ + Object { + "onSelectionChange": [Function], + } + } + sorting={ + Object { + "sort": Object { + "direction": "asc", + "field": "title", + }, + } + } + tableLayout="fixed" + > +
+ + + Delete + + connection + + + + } + > + +
+ +
+ + + +
+
+ + + + +
+ + + + + +
+ +
+
+ + + +
+ + +
+ +
+ + + + + +
+ +
+ +
+ + + +
+ + +
+
+ +
+ +
+ +
+ + +
+ +
+ +
+ + +
+ + +
+ +
+ + + + } + closePopover={[Function]} + display="inlineBlock" + hasArrow={true} + isOpen={false} + ownFocus={true} + panelPaddingSize="none" + > +
+
+ + + +
+
+ +
+ +
+ +
+ +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+
+ + +
+ +
+
+ + +
+
+ + + +
+
+ + No items found + +
+
+ +
+
+ +
+ +
+ + + +
+ + + +`; diff --git a/src/plugins/data_source_management/public/components/data_source_table/data_source_table.test.tsx b/src/plugins/data_source_management/public/components/data_source_table/data_source_table.test.tsx new file mode 100644 index 00000000000..282e8878b62 --- /dev/null +++ b/src/plugins/data_source_management/public/components/data_source_table/data_source_table.test.tsx @@ -0,0 +1,178 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import React from 'react'; +import { act } from 'react-dom/test-utils'; +import * as utils from '../utils'; +import { DataSourceTable } from './data_source_table'; +import { mount, ReactWrapper } from 'enzyme'; +import { RouteComponentProps } from 'react-router-dom'; +import { wrapWithIntl } from 'test_utils/enzyme_helpers'; +import { ScopedHistory } from 'opensearch-dashboards/public'; +import { scopedHistoryMock } from '../../../../../core/public/mocks'; +import { OpenSearchDashboardsContextProvider } from '../../../../opensearch_dashboards_react/public'; +import { getMappedDataSources, mockManagementPlugin } from '../../mocks'; + +const deleteButtonIdentifier = '[data-test-subj="deleteDataSourceConnections"]'; +const toastsIdentifier = 'EuiGlobalToastList'; +const tableIdentifier = 'EuiInMemoryTable'; +const confirmModalIndentifier = 'EuiConfirmModal'; +const tableColumnHeaderIdentifier = 'EuiTableHeaderCell'; +const tableColumnHeaderButtonIdentifier = 'EuiTableHeaderCell .euiTableHeaderButton'; + +describe('DataSourceTable', () => { + const mockedContext = mockManagementPlugin.createDataSourceManagementContext(); + let component: ReactWrapper, React.Component<{}, {}, any>>; + const history = (scopedHistoryMock.create() as unknown) as ScopedHistory; + + describe('should get datasources failed', () => { + beforeEach(async () => { + spyOn(utils, 'getDataSources').and.returnValue(Promise.reject({})); + await act(async () => { + component = await mount( + wrapWithIntl( + + ), + { + wrappingComponent: OpenSearchDashboardsContextProvider, + wrappingComponentProps: { + services: mockedContext, + }, + } + ); + }); + }); + + it('should show toast and remove toast normally', () => { + expect(component).toMatchSnapshot(); + expect(utils.getDataSources).toHaveBeenCalled(); + component.update(); + // @ts-ignore + expect(component.find(toastsIdentifier).props().toasts.length).toBe(1); + + act(() => { + // @ts-ignore + component.find(toastsIdentifier).first().prop('dismissToast')({ + id: 'dataSourcesManagement.dataSourceListing.fetchDataSourceFailMsg', + }); + }); + component.update(); + // @ts-ignore + expect(component.find(toastsIdentifier).props().toasts.length).toBe(0); // failure toast + }); + }); + + describe('should get datasources successful', () => { + beforeEach(async () => { + spyOn(utils, 'getDataSources').and.returnValue(Promise.resolve(getMappedDataSources)); + await act(async () => { + component = await mount( + wrapWithIntl( + + ), + { + wrappingComponent: OpenSearchDashboardsContextProvider, + wrappingComponentProps: { + services: mockedContext, + }, + } + ); + }); + }); + + it('should render normally', () => { + expect(component).toMatchSnapshot(); + expect(utils.getDataSources).toHaveBeenCalled(); + }); + + it('should sort datasources based on description', () => { + expect(component.find(tableIdentifier).exists()).toBe(true); + act(() => { + component.find(tableColumnHeaderButtonIdentifier).last().simulate('click'); + }); + component.update(); + // @ts-ignore + expect(component.find(tableColumnHeaderIdentifier).last().props().isSorted).toBe(true); + }); + + it('should enable delete button when select datasources', () => { + expect(component.find(deleteButtonIdentifier).first().props().disabled).toBe(true); + + act(() => { + // @ts-ignore + component.find(tableIdentifier).props().selection.onSelectionChange(getMappedDataSources); + }); + component.update(); + expect(component.find(deleteButtonIdentifier).first().props().disabled).toBe(false); + }); + + it('should detele confirm modal pop up and cancel button work normally', () => { + act(() => { + // @ts-ignore + component.find(tableIdentifier).props().selection.onSelectionChange(getMappedDataSources); + }); + component.update(); + component.find(deleteButtonIdentifier).first().simulate('click'); + // test if modal pop up when click the delete button + expect(component.find(confirmModalIndentifier).exists()).toBe(true); + + act(() => { + // @ts-ignore + component.find(confirmModalIndentifier).first().props().onCancel(); + }); + component.update(); + expect(component.find(confirmModalIndentifier).exists()).toBe(false); + }); + + it('should detele confirm modal confirm button work normally', async () => { + spyOn(utils, 'deleteMultipleDataSources').and.returnValue(Promise.resolve({})); + + act(() => { + // @ts-ignore + component.find(tableIdentifier).props().selection.onSelectionChange(getMappedDataSources); + }); + component.update(); + component.find(deleteButtonIdentifier).first().simulate('click'); + expect(component.find(confirmModalIndentifier).exists()).toBe(true); + + await act(async () => { + // @ts-ignore + await component.find(confirmModalIndentifier).first().props().onConfirm(); + }); + component.update(); + expect(component.find(confirmModalIndentifier).exists()).toBe(false); + }); + + it('should show toast when delete datasources failed', async () => { + spyOn(utils, 'deleteMultipleDataSources').and.returnValue(Promise.reject({})); + act(() => { + // @ts-ignore + component.find(tableIdentifier).props().selection.onSelectionChange(getMappedDataSources); + }); + + component.update(); + component.find(deleteButtonIdentifier).first().simulate('click'); + expect(component.find(confirmModalIndentifier).exists()).toBe(true); + + await act(async () => { + // @ts-ignore + await component.find(confirmModalIndentifier).props().onConfirm(); + }); + component.update(); + expect(utils.deleteMultipleDataSources).toHaveBeenCalled(); + // @ts-ignore + expect(component.find(toastsIdentifier).props().toasts.length).toBe(1); + expect(component.find(confirmModalIndentifier).exists()).toBe(false); + }); + }); +}); diff --git a/src/plugins/data_source_management/public/components/data_source_table/data_source_table.tsx b/src/plugins/data_source_management/public/components/data_source_table/data_source_table.tsx new file mode 100644 index 00000000000..97551c62ee3 --- /dev/null +++ b/src/plugins/data_source_management/public/components/data_source_table/data_source_table.tsx @@ -0,0 +1,335 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { + EuiButton, + EuiButtonEmpty, + EuiConfirmModal, + EuiFlexGroup, + EuiFlexItem, + EuiGlobalToastList, + EuiGlobalToastListToast, + EuiInMemoryTable, + EuiPageContent, + EuiSpacer, + EuiText, + EuiTitle, +} from '@elastic/eui'; +import React, { useState } from 'react'; +import { RouteComponentProps, withRouter } from 'react-router-dom'; +import { FormattedMessage } from '@osd/i18n/react'; +import { useEffectOnce } from 'react-use'; +import { getListBreadcrumbs } from '../breadcrumbs'; +import { + reactRouterNavigate, + useOpenSearchDashboards, +} from '../../../../opensearch_dashboards_react/public'; +import { DataSourceManagementContext, DataSourceTableItem, ToastMessageItem } from '../../types'; +import { CreateButton } from '../create_button'; +import { deleteMultipleDataSources, getDataSources } from '../utils'; +import { LoadingMask } from '../loading_mask'; +import { + cancelText, + deleteText, + dsListingAriaRegion, + dsListingDeleteDataSourceConfirmation, + dsListingDeleteDataSourceDescription, + dsListingDeleteDataSourceTitle, + dsListingDeleteDataSourceWarning, + dsListingDescription, + dsListingPageTitle, + dsListingTitle, +} from '../text_content/text_content'; + +/* Table config */ +const pagination = { + initialPageSize: 10, + pageSizeOptions: [5, 10, 25, 50], +}; + +const sorting = { + sort: { + field: 'title', + direction: 'asc' as const, + }, +}; + +const toastLifeTimeMs = 6000; + +export const DataSourceTable = ({ history }: RouteComponentProps) => { + const { chrome, setBreadcrumbs, savedObjects } = useOpenSearchDashboards< + DataSourceManagementContext + >().services; + + /* Component state variables */ + const [dataSources, setDataSources] = useState([]); + const [selectedDataSources, setSelectedDataSources] = useState([]); + const [toasts, setToasts] = React.useState([]); + const [isLoading, setIsLoading] = React.useState(false); + const [isDeleting, setIsDeleting] = React.useState(false); + const [confirmDeleteVisible, setConfirmDeleteVisible] = React.useState(false); + + /* useEffectOnce hook to avoid these methods called multiple times when state is updated. */ + useEffectOnce(() => { + /* Update breadcrumb*/ + setBreadcrumbs(getListBreadcrumbs()); + + /* Browser - Page Title */ + chrome.docTitle.change(dsListingPageTitle); + + /* fetch data sources*/ + fetchDataSources(); + }); + + const fetchDataSources = () => { + setIsLoading(true); + getDataSources(savedObjects.client) + .then((response: DataSourceTableItem[]) => { + setDataSources(response); + }) + .catch(() => { + setDataSources([]); + handleDisplayToastMessage({ + id: 'dataSourcesManagement.dataSourceListing.fetchDataSourceFailMsg', + defaultMessage: + 'Error occurred while fetching the records for Data sources. Please try it again', + color: 'warning', + iconType: 'alert', + }); + }) + .finally(() => { + setIsLoading(false); + }); + }; + + /* Table search config */ + const renderDeleteButton = () => { + return ( + { + setConfirmDeleteVisible(true); + }} + data-test-subj="deleteDataSourceConnections" + disabled={selectedDataSources.length === 0} + > + Delete {selectedDataSources.length || ''} connection + {selectedDataSources.length >= 2 ? 's' : ''} + + ); + }; + + const renderToolsRight = () => { + return ( + + {renderDeleteButton()} + + ); + }; + + const search = { + toolsRight: renderToolsRight(), + box: { + incremental: true, + schema: { + fields: { title: { type: 'string' } }, + }, + }, + }; + + /* Table columns */ + const columns = [ + { + field: 'title', + name: 'Datasource', + render: ( + name: string, + index: { + id: string; + tags?: Array<{ + key: string; + name: string; + }>; + } + ) => ( + <> + + {name} + + + ), + dataType: 'string' as const, + sortable: ({ title }: { title: string }) => title, + }, + { + field: 'description', + name: 'Description', + truncateText: true, + mobileOptions: { + show: false, + }, + dataType: 'string' as const, + sortable: ({ description }: { description: string }) => description, + }, + ]; + + /* render delete modal*/ + const tableRenderDeleteModal = () => { + return confirmDeleteVisible ? ( + { + setConfirmDeleteVisible(false); + }} + onConfirm={() => { + setConfirmDeleteVisible(false); + onClickDelete(); + }} + cancelButtonText={cancelText} + confirmButtonText={deleteText} + defaultFocusedButton="confirm" + > +

{dsListingDeleteDataSourceDescription}

+

{dsListingDeleteDataSourceConfirmation}

+

{dsListingDeleteDataSourceWarning}

+ + ) : null; + }; + + /* Delete selected data sources*/ + const onClickDelete = () => { + setIsDeleting(true); + + deleteMultipleDataSources(savedObjects.client, selectedDataSources) + .then(() => { + setSelectedDataSources([]); + // Fetch data sources + fetchDataSources(); + setConfirmDeleteVisible(false); + }) + .catch(() => { + handleDisplayToastMessage({ + id: 'dataSourcesManagement.dataSourceListing.deleteDataSourceFailMsg', + defaultMessage: + 'Error occurred while deleting few/all selected records for Data sources. Please try it again', + color: 'warning', + iconType: 'alert', + }); + }) + .finally(() => { + setIsDeleting(false); + }); + }; + + /* Table selection handlers */ + const onSelectionChange = (selected: DataSourceTableItem[]) => { + setSelectedDataSources(selected); + }; + + const selection = { + onSelectionChange, + }; + + /* Toast Handlers */ + const removeToast = (id: string) => { + setToasts(toasts.filter((toast) => toast.id !== id)); + }; + + const handleDisplayToastMessage = ({ id, defaultMessage, color, iconType }: ToastMessageItem) => { + const failureMsg = ; + setToasts([ + ...toasts, + { + title: failureMsg, + id: failureMsg.props.id, + color, + iconType, + }, + ]); + }; + + /* Render Ui elements*/ + /* Create Data Source button */ + const createButton = ; + + /* Render header*/ + const renderHeader = () => { + return ( + + + +

{dsListingTitle}

+ + + +

{dsListingDescription}

+ + + {createButton} + + ); + }; + + /* Render table */ + const renderTableContent = () => { + return ( + <> + + {/* Header */} + {renderHeader()} + + + + {/* Delete confirmation modal*/} + {tableRenderDeleteModal()} + + {/* Data sources table*/} + + + {isDeleting ? : null} + + ); + }; + + const renderContent = () => { + return ( + <> + {renderTableContent()} + {} + + ); + }; + + return ( + <> + {renderContent()} + { + removeToast(id); + }} + toastLifeTimeMs={toastLifeTimeMs} + /> + + ); +}; + +export const DataSourceTableWithRouter = withRouter(DataSourceTable); diff --git a/src/plugins/data_source_management/public/components/data_source_table/index.ts b/src/plugins/data_source_management/public/components/data_source_table/index.ts new file mode 100644 index 00000000000..6750808169b --- /dev/null +++ b/src/plugins/data_source_management/public/components/data_source_table/index.ts @@ -0,0 +1,6 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +export { DataSourceTableWithRouter } from './data_source_table'; diff --git a/src/plugins/data_source_management/public/components/edit_data_source/components/edit_form/edit_data_source_form.tsx b/src/plugins/data_source_management/public/components/edit_data_source/components/edit_form/edit_data_source_form.tsx new file mode 100644 index 00000000000..bf59853d542 --- /dev/null +++ b/src/plugins/data_source_management/public/components/edit_data_source/components/edit_form/edit_data_source_form.tsx @@ -0,0 +1,539 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import React from 'react'; +import { + EuiBottomBar, + EuiButton, + EuiButtonEmpty, + EuiDescribedFormGroup, + EuiFieldText, + EuiFlexGroup, + EuiFlexItem, + EuiForm, + EuiFormRow, + EuiHorizontalRule, + EuiPanel, + EuiSpacer, + EuiSuperSelect, + EuiText, + EuiToolTip, +} from '@elastic/eui'; +import { + AuthType, + credentialSourceOptions, + DataSourceAttributes, + DataSourceManagementContextValue, + UpdatePasswordFormType, + UsernamePasswordTypedContent, +} from '../../../../types'; +import { Header } from '../header'; +import { context as contextType } from '../../../../../../opensearch_dashboards_react/public'; +import { + CreateEditDataSourceValidation, + defaultValidation, + performDataSourceFormValidation, +} from '../../../validation'; +import { UpdatePasswordModal } from '../update_password_modal'; +import { + authenticationDetailsDescription, + authenticationDetailsText, + authenticationMethodTitle, + authenticationTitle, + cancelChangesText, + connectionDetailsText, + createDataSourceDescriptionPlaceholder, + createDataSourceEndpointURL, + createDataSourcePasswordPlaceholder, + createDataSourceUsernamePlaceholder, + descriptionText, + endpointDescription, + endpointTitle, + objectDetailsDescription, + objectDetailsText, + passwordText, + saveChangesText, + titleText, + updatePasswordText, + usernameText, + validationErrorTooltipText, +} from '../../../text_content'; + +export interface EditDataSourceProps { + existingDataSource: DataSourceAttributes; + handleSubmit: (formValues: DataSourceAttributes) => void; + onDeleteDataSource?: () => void; +} +export interface EditDataSourceState { + formErrors: string[]; + formErrorsByField: CreateEditDataSourceValidation; + title: string; + description: string; + endpoint: string; + auth: { + type: AuthType; + credentials: UsernamePasswordTypedContent; + }; + showUpdatePasswordModal: boolean; + showUpdateOptions: boolean; + oldPassword: string; + newPassword: string; + confirmNewPassword: string; +} + +export class EditDataSourceForm extends React.Component { + static contextType = contextType; + public readonly context!: DataSourceManagementContextValue; + maskedPassword: string = '********'; + + constructor(props: EditDataSourceProps, context: DataSourceManagementContextValue) { + super(props, context); + + this.state = { + formErrors: [], + formErrorsByField: { ...defaultValidation }, + title: '', + description: '', + endpoint: '', + auth: { + type: AuthType.NoAuth, + credentials: { + username: '', + password: '', + }, + }, + showUpdatePasswordModal: false, + showUpdateOptions: false, + oldPassword: '', + newPassword: '', + confirmNewPassword: '', + }; + } + + componentDidMount() { + this.setFormValuesForEditMode(); + } + + resetFormValues = () => { + this.setFormValuesForEditMode(); + this.setState({ showUpdateOptions: false }, this.checkValidation); + }; + + setFormValuesForEditMode() { + if (this.props.existingDataSource) { + const { title, description, endpoint, auth } = this.props.existingDataSource; + + this.setState({ + title, + description: description || '', + endpoint, + auth: { + type: auth.type, + credentials: { + username: auth.type === AuthType.NoAuth ? '' : auth.credentials?.username || '', + password: auth.type === AuthType.NoAuth ? '' : this.maskedPassword, + }, + }, + }); + } + } + + /* Validations */ + + isFormValid = () => { + const { formErrors, formErrorsByField } = performDataSourceFormValidation(this.state); + + this.setState({ + formErrors, + formErrorsByField, + }); + + return formErrors.length === 0; + }; + + /* Events */ + + onChangeTitle = (e: { target: { value: any } }) => { + this.setState({ title: e.target.value }, () => { + if (this.state.formErrorsByField.title.length) { + this.isFormValid(); + } + }); + }; + + onChangeAuthType = (value: AuthType) => { + this.setState({ auth: { ...this.state.auth, type: value } }, () => { + this.onChangeFormValues(); + this.checkValidation(); + }); + }; + + onChangeDescription = (e: { target: { value: any } }) => { + this.setState({ description: e.target.value }); + }; + + onChangeUsername = (e: { target: { value: any } }) => { + this.setState( + { + auth: { + ...this.state.auth, + credentials: { ...this.state.auth.credentials, username: e.target.value }, + }, + }, + this.checkValidation + ); + }; + + onChangePassword = (e: { target: { value: any } }) => { + this.setState( + { + auth: { + ...this.state.auth, + credentials: { ...this.state.auth.credentials, password: e.target.value }, + }, + }, + this.checkValidation + ); + }; + + checkValidation = () => { + if (this.state.formErrors.length) { + this.isFormValid(); + } + }; + + onClickUpdateDataSource = () => { + if (this.isFormValid()) { + // update data source endpoint is currently not supported/allowed + const formValues: DataSourceAttributes = { + title: this.state.title, + description: this.state.description, + auth: this.state.auth, + }; + /* Remove credentials object for NoAuth */ + if (this.state.auth.type === AuthType.NoAuth) { + delete formValues.auth.credentials; + } else if (this.props.existingDataSource.auth.type === AuthType.UsernamePasswordType) { + /* Remove password if previously & currently username & password method is selected*/ + delete formValues.auth.credentials?.password; + } + + /* Submit */ + this.props.handleSubmit(formValues); + } + }; + + onClickDeleteDataSource = () => { + if (this.props.onDeleteDataSource) { + this.props.onDeleteDataSource(); + } + }; + + onChangeFormValues = () => { + setTimeout(() => { + this.didFormValuesChange(); + }, 0); + }; + + /* Create new credentials*/ + onClickUpdatePassword = () => { + this.setState({ showUpdatePasswordModal: true }); + }; + + updatePassword = (passwords: UpdatePasswordFormType) => { + // TODO: update password when API is ready + this.closePasswordModal(); + }; + + /* Render methods */ + + /* Render Modal for new credential */ + closePasswordModal = () => { + this.setState({ showUpdatePasswordModal: false }); + }; + + renderUpdatePasswordModal = () => { + return ( + <> + {updatePasswordText} + + {this.state.showUpdatePasswordModal ? ( + + ) : null} + + ); + }; + /* Render header*/ + renderHeader = () => { + return ( +
+ ); + }; + + /* Render Connection Details Panel */ + renderConnectionDetailsSection = () => { + return ( + + {connectionDetailsText} + + + + {objectDetailsText}} + description={

{objectDetailsDescription}

} + > + {/* Title */} + + + + {/* Description */} + + + + + + ); + }; + + /* Render Connection Details Panel */ + renderEndpointSection = () => { + return ( + + {endpointTitle} + + + + {createDataSourceEndpointURL}} + description={

{endpointDescription}

} + > + {/* Endpoint */} + + + + + + ); + }; + + /* Render Connection Details Panel */ + renderAuthenticationSection = () => { + return ( + + {authenticationTitle} + + + + {authenticationDetailsText}} + description={

{authenticationDetailsDescription}

} + > + {this.renderCredentialsSection()} + + + ); + }; + + /* Render Credentials Existing & new */ + renderCredentialsSection = () => { + return ( + <> + {/* Auth type select */} + + this.onChangeAuthType(value)} + /> + + + + + {this.state.auth.type !== AuthType.NoAuth ? this.renderUsernamePasswordFields() : null} + + ); + }; + + renderUsernamePasswordFields = () => { + return ( + <> + {/* Username */} + + + + + {/* Password */} + {this.props.existingDataSource.auth.type === AuthType.NoAuth + ? this.renderEmptyPasswordField() + : this.renderDisabledPasswordField()} + + ); + }; + + renderDisabledPasswordField = () => { + return ( + + + + ************* + + {this.renderUpdatePasswordModal()} + + + ); + }; + + renderEmptyPasswordField = () => { + return ( + + + + ); + }; + + didFormValuesChange = () => { + const formValues: DataSourceAttributes = { + title: this.state.title, + description: this.state.description, + endpoint: this.props.existingDataSource.endpoint, + auth: this.state.auth, + }; + + const { title, description, endpoint, auth } = this.props.existingDataSource; + const isUsernameChanged: boolean = + auth.type === formValues.auth.type && + auth.type === AuthType.UsernamePasswordType && + formValues.auth.credentials?.username !== auth.credentials?.username; + + if ( + formValues.title !== title || + formValues.description !== description || + formValues.endpoint !== endpoint || + formValues.auth.type !== auth.type || + isUsernameChanged + ) { + this.setState({ showUpdateOptions: true }); + } else { + this.setState({ showUpdateOptions: false }); + } + }; + + renderBottomBar = () => { + let bottomBar = null; + + if (this.state.showUpdateOptions) { + bottomBar = ( + + + + + + this.resetFormValues()} + aria-describedby="aria-describedby.countOfUnsavedSettings" + data-test-subj="datasource-edit-cancelButton" + > + {cancelChangesText} + + + + + + {saveChangesText} + + + + + + ); + } + return bottomBar; + }; + + renderContent = () => { + return ( + <> + {this.renderHeader()} + this.onChangeFormValues()} + data-test-subj="data-source-edit" + isInvalid={!!this.state.formErrors.length} + error={this.state.formErrors} + > + {this.renderConnectionDetailsSection()} + + + + {this.renderEndpointSection()} + + + + {this.renderAuthenticationSection()} + + + {this.renderBottomBar()} + + + + + ); + }; + + render() { + return <>{this.renderContent()}; + } +} diff --git a/src/plugins/data_source_management/public/components/edit_data_source/components/header/header.tsx b/src/plugins/data_source_management/public/components/edit_data_source/components/header/header.tsx new file mode 100644 index 00000000000..f9d957f090f --- /dev/null +++ b/src/plugins/data_source_management/public/components/edit_data_source/components/header/header.tsx @@ -0,0 +1,99 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import React, { useState } from 'react'; + +import { + EuiSpacer, + EuiTitle, + EuiFlexItem, + EuiFlexGroup, + EuiToolTip, + EuiButtonIcon, + EuiConfirmModal, +} from '@elastic/eui'; + +import { useOpenSearchDashboards } from '../../../../../../opensearch_dashboards_react/public'; +import { DataSourceManagementContext } from '../../../../types'; +import { + cancelText, + deleteText, + deleteThisDataSource, + dsListingDeleteDataSourceConfirmation, + dsListingDeleteDataSourceDescription, + dsListingDeleteDataSourceTitle, + dsListingDeleteDataSourceWarning, +} from '../../../text_content'; + +export const Header = ({ + showDeleteIcon, + onClickDeleteIcon, + dataSourceName, +}: { + showDeleteIcon: boolean; + onClickDeleteIcon: () => void; + dataSourceName: string; +}) => { + /* State Variables */ + const [isDeleteModalVisible, setIsDeleteModalVisible] = useState(false); + + const changeTitle = useOpenSearchDashboards().services.chrome + .docTitle.change; + + changeTitle(dataSourceName); + + const renderDeleteButton = () => { + return ( + <> + + { + setIsDeleteModalVisible(true); + }} + iconType="trash" + iconSize="m" + size="m" + aria-label={deleteThisDataSource} + /> + + + {isDeleteModalVisible ? ( + { + setIsDeleteModalVisible(false); + }} + onConfirm={() => { + setIsDeleteModalVisible(false); + onClickDeleteIcon(); + }} + cancelButtonText={cancelText} + confirmButtonText={deleteText} + defaultFocusedButton="confirm" + > +

{dsListingDeleteDataSourceDescription}

+

{dsListingDeleteDataSourceConfirmation}

+

{dsListingDeleteDataSourceWarning}

+ + ) : null} + + ); + }; + + return ( + + +
+ +

{dataSourceName}

+ + +
+ + {showDeleteIcon ? renderDeleteButton() : null} + + ); +}; diff --git a/src/plugins/data_source_management/public/components/edit_data_source/components/header/index.ts b/src/plugins/data_source_management/public/components/edit_data_source/components/header/index.ts new file mode 100644 index 00000000000..3c25d4c42f0 --- /dev/null +++ b/src/plugins/data_source_management/public/components/edit_data_source/components/header/index.ts @@ -0,0 +1,6 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +export { Header } from './header'; diff --git a/src/plugins/data_source_management/public/components/edit_data_source/components/update_password_modal/index.ts b/src/plugins/data_source_management/public/components/edit_data_source/components/update_password_modal/index.ts new file mode 100644 index 00000000000..ce84aab8989 --- /dev/null +++ b/src/plugins/data_source_management/public/components/edit_data_source/components/update_password_modal/index.ts @@ -0,0 +1,6 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +export { UpdatePasswordModal } from './update_password_modal'; diff --git a/src/plugins/data_source_management/public/components/edit_data_source/components/update_password_modal/update_password_modal.tsx b/src/plugins/data_source_management/public/components/edit_data_source/components/update_password_modal/update_password_modal.tsx new file mode 100644 index 00000000000..df85ce1fd9a --- /dev/null +++ b/src/plugins/data_source_management/public/components/edit_data_source/components/update_password_modal/update_password_modal.tsx @@ -0,0 +1,146 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import React, { useCallback, useEffect, useState } from 'react'; +import { + EuiButton, + EuiButtonEmpty, + EuiFieldPassword, + EuiForm, + EuiFormRow, + EuiModal, + EuiModalBody, + EuiModalFooter, + EuiModalHeader, + EuiModalHeaderTitle, +} from '@elastic/eui'; +import { UpdatePasswordFormType } from '../../../../types'; +import { + defaultPasswordValidationByField, + UpdatePasswordValidation, + validateUpdatePassword, +} from '../../../validation'; +import { confirmNewPasswordText, newPasswordText, oldPasswordText } from '../../../text_content'; + +export interface UpdatePasswordModalProps { + handleUpdatePassword: (passwords: UpdatePasswordFormType) => void; + closeUpdatePasswordModal: () => void; +} + +export const UpdatePasswordModal = ({ + handleUpdatePassword, + closeUpdatePasswordModal, +}: UpdatePasswordModalProps) => { + /* State Variables */ + const [formErrors, setFormErrors] = useState([]); + const [formErrorsByField, setFormErrorsByField] = useState( + defaultPasswordValidationByField + ); + const [oldPassword, setOldPassword] = useState(''); + const [newPassword, setNewPassword] = useState(''); + const [confirmNewPassword, setConfirmNewPassword] = useState(''); + + const getFormValues = useCallback(() => { + return { + oldPassword, + newPassword, + confirmNewPassword, + }; + }, [oldPassword, newPassword, confirmNewPassword]); + + const onClickUpdatePassword = () => { + if (isFormValid()) { + handleUpdatePassword(getFormValues()); + } + }; + + /* Validations */ + const isFormValid = useCallback(() => { + const { formValidationErrors, formValidationErrorsByField } = validateUpdatePassword( + getFormValues() + ); + + setFormErrors([...formValidationErrors]); + setFormErrorsByField({ ...formValidationErrorsByField }); + + return formValidationErrors.length === 0; + }, [getFormValues]); + + useEffect(() => { + if (formErrors.length) { + isFormValid(); + } + }, [oldPassword, newPassword, confirmNewPassword, formErrors.length, isFormValid]); + + const renderUpdatePasswordModal = () => { + return ( + + + +

Update password

+ + + + + + + setOldPassword(e.target.value)} + /> + + + setNewPassword(e.target.value)} + /> + + + setConfirmNewPassword(e.target.value)} + /> + + + + + + Cancel + + Update + + + + ); + }; + + /* Return the modal */ + return
{renderUpdatePasswordModal()}
; +}; diff --git a/src/plugins/data_source_management/public/components/edit_data_source/edit_data_source.tsx b/src/plugins/data_source_management/public/components/edit_data_source/edit_data_source.tsx new file mode 100644 index 00000000000..0d27d650c27 --- /dev/null +++ b/src/plugins/data_source_management/public/components/edit_data_source/edit_data_source.tsx @@ -0,0 +1,162 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { RouteComponentProps, withRouter } from 'react-router-dom'; +import React, { useState } from 'react'; +import { useEffectOnce } from 'react-use'; +import { EuiGlobalToastList, EuiGlobalToastListToast } from '@elastic/eui'; +import { FormattedMessage } from '@osd/i18n/react'; +import { useOpenSearchDashboards } from '../../../../opensearch_dashboards_react/public'; +import { DataSourceManagementContext, ToastMessageItem } from '../../types'; +import { deleteDataSourceById, getDataSourceById, updateDataSourceById } from '../utils'; +import { getEditBreadcrumbs } from '../breadcrumbs'; +import { EditDataSourceForm } from './components/edit_form/edit_data_source_form'; +import { LoadingMask } from '../loading_mask'; +import { AuthType, DataSourceAttributes } from '../../types'; +import { dataSourceNotFound } from '../text_content'; + +const defaultDataSource: DataSourceAttributes = { + title: '', + description: '', + endpoint: '', + auth: { + type: AuthType.NoAuth, + credentials: undefined, + }, +}; + +const EditDataSource: React.FunctionComponent> = ( + props: RouteComponentProps<{ id: string }> +) => { + /* Initialization */ + const { savedObjects, setBreadcrumbs } = useOpenSearchDashboards< + DataSourceManagementContext + >().services; + const dataSourceID: string = props.match.params.id; + + /* State Variables */ + const [dataSource, setDataSource] = useState(defaultDataSource); + const [isLoading, setIsLoading] = useState(false); + const [toasts, setToasts] = useState([]); + + const toastLifeTimeMs: number = 6000; + + /* Fetch data source by id*/ + useEffectOnce(() => { + (async function () { + setIsLoading(true); + try { + const fetchDataSourceById = await getDataSourceById(dataSourceID, savedObjects.client); + if (fetchDataSourceById) { + setDataSource(fetchDataSourceById); + setBreadcrumbs(getEditBreadcrumbs(fetchDataSourceById)); + } + } catch (e) { + handleDisplayToastMessage({ + id: 'dataSourcesManagement.editDataSource.editDataSourceFailMsg', + defaultMessage: 'Unable to find the Data Source. Please try it again.', + color: 'warning', + iconType: 'alert', + }); + + props.history.push(''); + } finally { + setIsLoading(false); + } + })(); + }); + + /* Handle submit - create data source*/ + const handleSubmit = async (attributes: DataSourceAttributes) => { + setIsLoading(true); + try { + await updateDataSourceById(savedObjects.client, dataSourceID, attributes); + props.history.push(''); + } catch (e) { + setIsLoading(false); + handleDisplayToastMessage({ + id: 'dataSourcesManagement.editDataSource.editDataSourceFailMsg', + defaultMessage: 'Updating the Data Source failed with some errors. Please try it again.', + color: 'warning', + iconType: 'alert', + }); + } + }; + + const handleDisplayToastMessage = ({ id, defaultMessage, color, iconType }: ToastMessageItem) => { + if (id && defaultMessage && color && iconType) { + const failureMsg = ; + setToasts([ + ...toasts, + { + title: failureMsg, + id: failureMsg.props.id, + color, + iconType, + }, + ]); + } + }; + + /* Handle delete - data source*/ + const handleDelete = async () => { + setIsLoading(true); + try { + await deleteDataSourceById(props.match.params.id, savedObjects.client); + props.history.push(''); + } catch (e) { + setIsLoading(false); + handleDisplayToastMessage({ + id: 'dataSourcesManagement.editDataSource.deleteDataSourceFailMsg', + defaultMessage: 'Unable to delete the Data Source due to some errors. Please try it again.', + color: 'warning', + iconType: 'alert', + }); + } + }; + + /* Render the edit wizard */ + const renderContent = () => { + if (!isLoading && (!dataSource || !dataSource.id)) { + return

Data Source not found!

; + } + return ( + <> + {dataSource && dataSource.endpoint ? ( + + ) : null} + {isLoading || !dataSource?.endpoint ? : null} + + ); + }; + + /* Remove toast on dismiss*/ + const removeToast = (id: string) => { + setToasts(toasts.filter((toast) => toast.id !== id)); + }; + + if (!isLoading && !dataSource?.endpoint) { + return

{dataSourceNotFound}

; + } + + return ( + <> + {renderContent()} + { + removeToast(id); + }} + toastLifeTimeMs={toastLifeTimeMs} + /> + + ); +}; + +export const EditDataSourceWithRouter = withRouter(EditDataSource); diff --git a/src/plugins/data_source_management/public/components/edit_data_source/index.ts b/src/plugins/data_source_management/public/components/edit_data_source/index.ts new file mode 100644 index 00000000000..815c4f53b99 --- /dev/null +++ b/src/plugins/data_source_management/public/components/edit_data_source/index.ts @@ -0,0 +1,6 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +export { EditDataSourceWithRouter } from './edit_data_source'; diff --git a/src/plugins/data_source_management/public/components/loading_mask/__snapshots__/loading_mask.test.tsx.snap b/src/plugins/data_source_management/public/components/loading_mask/__snapshots__/loading_mask.test.tsx.snap new file mode 100644 index 00000000000..ed6543e2e1c --- /dev/null +++ b/src/plugins/data_source_management/public/components/loading_mask/__snapshots__/loading_mask.test.tsx.snap @@ -0,0 +1,9 @@ +// Jest Snapshot v1, https://goo.gl/fbAQLP + +exports[`Datasource Management: Header should render normally 1`] = ` + + + +`; diff --git a/src/plugins/data_source_management/public/components/loading_mask/index.ts b/src/plugins/data_source_management/public/components/loading_mask/index.ts new file mode 100644 index 00000000000..c1251b1cac9 --- /dev/null +++ b/src/plugins/data_source_management/public/components/loading_mask/index.ts @@ -0,0 +1,6 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +export { LoadingMask } from './loading_mask'; diff --git a/src/plugins/data_source_management/public/components/loading_mask/loading_mask.test.tsx b/src/plugins/data_source_management/public/components/loading_mask/loading_mask.test.tsx new file mode 100644 index 00000000000..e15b9ea0a55 --- /dev/null +++ b/src/plugins/data_source_management/public/components/loading_mask/loading_mask.test.tsx @@ -0,0 +1,15 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import React from 'react'; +import { shallow } from 'enzyme'; +import { LoadingMask } from './loading_mask'; + +describe('Datasource Management: Header', () => { + test('should render normally', () => { + const component = shallow(); + expect(component).toMatchSnapshot(); + }); +}); diff --git a/src/plugins/data_source_management/public/components/loading_mask/loading_mask.tsx b/src/plugins/data_source_management/public/components/loading_mask/loading_mask.tsx new file mode 100644 index 00000000000..4868b6d9911 --- /dev/null +++ b/src/plugins/data_source_management/public/components/loading_mask/loading_mask.tsx @@ -0,0 +1,15 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import React from 'react'; +import { EuiLoadingSpinner, EuiOverlayMask } from '@elastic/eui'; + +export const LoadingMask = () => { + return ( + + + + ); +}; diff --git a/src/plugins/data_source_management/public/components/text_content/index.ts b/src/plugins/data_source_management/public/components/text_content/index.ts new file mode 100644 index 00000000000..6caa2514600 --- /dev/null +++ b/src/plugins/data_source_management/public/components/text_content/index.ts @@ -0,0 +1,6 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +export * from './text_content'; diff --git a/src/plugins/data_source_management/public/components/text_content/text_content.ts b/src/plugins/data_source_management/public/components/text_content/text_content.ts new file mode 100644 index 00000000000..ee6bca828eb --- /dev/null +++ b/src/plugins/data_source_management/public/components/text_content/text_content.ts @@ -0,0 +1,283 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { i18n } from '@osd/i18n'; + +/* Generic */ +export const cancelText = i18n.translate('cancel', { + defaultMessage: 'Cancel', +}); + +export const deleteText = i18n.translate('delete', { + defaultMessage: 'Delete', +}); + +export const titleText = i18n.translate('title', { + defaultMessage: 'Title', +}); + +export const descriptionText = i18n.translate('description', { + defaultMessage: 'Description', +}); + +export const usernameText = i18n.translate('username', { + defaultMessage: 'Username', +}); + +export const passwordText = i18n.translate('password', { + defaultMessage: 'Password', +}); + +/* Datasource listing page */ +export const dsListingAriaRegion = i18n.translate( + 'dataSourcesManagement.createDataSourcesLiveRegionAriaLabel', + { + defaultMessage: 'Data Sources', + } +); +export const dsListingTitle = i18n.translate('dataSourcesManagement.dataSourcesTable.title', { + defaultMessage: 'Data Sources', +}); + +export const dsListingDescription = i18n.translate( + 'dataSourcesManagement.dataSourcesTable.description', + { + defaultMessage: + 'Create and manage the data sources that help you retrieve your data from multiple Elasticsearch clusters', + } +); + +export const dsListingPageTitle = i18n.translate( + 'dataSourcesManagement.dataSourcesTable.dataSourcesTitle', + { + defaultMessage: 'Data Sources', + } +); + +export const dsListingDeleteDataSourceTitle = i18n.translate( + 'dataSourcesManagement.dataSourcesTable.deleteTitle', + { + defaultMessage: 'Delete Data Source connection(s) permanently?', + } +); + +export const dsListingDeleteDataSourceDescription = i18n.translate( + 'dataSourcesManagement.dataSourcesTable.deleteDescription', + { + defaultMessage: + 'This will delete data source connections(s) and all Index Patterns using this credential will be invalid for access.', + } +); + +export const dsListingDeleteDataSourceConfirmation = i18n.translate( + 'dataSourcesManagement.dataSourcesTable.deleteConfirmation', + { + defaultMessage: 'To confirm deletion, click delete button.', + } +); + +export const dsListingDeleteDataSourceWarning = i18n.translate( + 'dataSourcesManagement.dataSourcesTable.deleteWarning', + { + defaultMessage: 'Note: this action is irrevocable!', + } +); + +/* CREATE DATA SOURCE */ +export const createDataSourceHeader = i18n.translate( + 'dataSourcesManagement.createDataSourceHeader', + { + defaultMessage: 'Create data source connection', + } +); +export const createDataSourceDescriptionPlaceholder = i18n.translate( + 'dataSourcesManagement.createDataSource.descriptionPlaceholder', + { + defaultMessage: 'Description of the data source', + } +); +export const createDataSourceEndpointURL = i18n.translate( + 'dataSourcesManagement.createDataSource.endpointURL', + { + defaultMessage: 'Endpoint URL', + } +); +export const createDataSourceEndpointPlaceholder = i18n.translate( + 'dataSourcesManagement.createDataSource.endpointPlaceholder', + { + defaultMessage: 'The connection URL', + } +); +export const createDataSourceUsernamePlaceholder = i18n.translate( + 'dataSourcesManagement.createDataSource.usernamePlaceholder', + { + defaultMessage: 'Username to connect to data source', + } +); +export const createDataSourcePasswordPlaceholder = i18n.translate( + 'dataSourcesManagement.createDataSource.passwordPlaceholder', + { + defaultMessage: 'Password to connect to data source', + } +); +export const createDataSourceCredentialSource = i18n.translate( + 'dataSourcesManagement.createDataSource.credentialSource', + { + defaultMessage: 'Credential Source', + } +); + +/* Edit data source */ +export const dataSourceNotFound = i18n.translate( + 'dataSourcesManagement.editDataSource.dataSourceNotFound', + { + defaultMessage: 'Data Source not found!', + } +); +export const deleteThisDataSource = i18n.translate( + 'dataSourcesManagement.editDataSource.deleteThisDataSource', + { + defaultMessage: 'Delete this Data Source', + } +); +export const oldPasswordText = i18n.translate('dataSourcesManagement.editDataSource.oldPassword', { + defaultMessage: 'Old password', +}); +export const newPasswordText = i18n.translate('dataSourcesManagement.editDataSource.newPassword', { + defaultMessage: 'New password', +}); +export const confirmNewPasswordText = i18n.translate( + 'dataSourcesManagement.editDataSource.confirmNewPassword', + { + defaultMessage: 'Confirm new password', + } +); +export const updatePasswordText = i18n.translate( + 'dataSourcesManagement.editDataSource.updatePasswordText', + { + defaultMessage: 'Update password', + } +); +export const connectionDetailsText = i18n.translate( + 'dataSourcesManagement.editDataSource.connectionDetailsText', + { + defaultMessage: 'Connection Details', + } +); +export const objectDetailsText = i18n.translate( + 'dataSourcesManagement.editDataSource.objectDetailsText', + { + defaultMessage: 'Object Details', + } +); +export const objectDetailsDescription = i18n.translate( + 'dataSourcesManagement.editDataSource.objectDetailsDescription', + { + defaultMessage: + 'This connection information is used for reference in tables and when adding to a data source connection', + } +); +export const authenticationMethodTitle = i18n.translate( + 'dataSourcesManagement.editDataSource.authenticationMethodTitle', + { + defaultMessage: 'Authentication Method', + } +); +export const authenticationTitle = i18n.translate( + 'dataSourcesManagement.editDataSource.authenticationTitle', + { + defaultMessage: 'Authentication', + } +); +export const authenticationDetailsText = i18n.translate( + 'dataSourcesManagement.editDataSource.authenticationDetailsText', + { + defaultMessage: 'Authentication Details', + } +); +export const authenticationDetailsDescription = i18n.translate( + 'dataSourcesManagement.editDataSource.authenticationDetailsDescription', + { + defaultMessage: 'Modify these to update the authentication type and associated details', + } +); +export const endpointTitle = i18n.translate('dataSourcesManagement.editDataSource.endpointTitle', { + defaultMessage: 'Endpoint', +}); +export const endpointDescription = i18n.translate( + 'dataSourcesManagement.editDataSource.endpointDescription', + { + defaultMessage: + 'This connection information is used for reference in tables and when adding to a data source connection', + } +); + +export const cancelChangesText = i18n.translate( + 'dataSourcesManagement.editDataSource.cancelButtonLabel', + { + defaultMessage: 'Cancel changes', + } +); +export const saveChangesText = i18n.translate( + 'dataSourcesManagement.editDataSource.saveButtonLabel', + { + defaultMessage: 'Save changes', + } +); + +export const validationErrorTooltipText = i18n.translate( + 'dataSourcesManagement.editDataSource.saveButtonTooltipWithInvalidChanges', + { + defaultMessage: 'Fix invalid settings before saving.', + } +); + +/* Password validation */ + +export const dataSourceValidationOldPasswordEmpty = i18n.translate( + 'dataSourcesManagement.validation.oldPasswordEmpty', + { + defaultMessage: 'Old password cannot be empty', + } +); +export const dataSourceValidationNewPasswordEmpty = i18n.translate( + 'dataSourcesManagement.validation.newPasswordEmpty', + { + defaultMessage: 'New password cannot be empty', + } +); +export const dataSourceValidationNoPasswordMatch = i18n.translate( + 'dataSourcesManagement.validation.noPasswordMatch', + { + defaultMessage: 'Passwords do not match', + } +); + +/* Create/Edit validation */ + +export const dataSourceValidationTitleEmpty = i18n.translate( + 'dataSourcesManagement.validation.titleEmpty', + { + defaultMessage: 'Title must not be empty', + } +); +export const dataSourceValidationEndpointNotValid = i18n.translate( + 'dataSourcesManagement.validation.endpointNotValid', + { + defaultMessage: 'Endpoint is not valid', + } +); +export const dataSourceValidationUsernameEmpty = i18n.translate( + 'dataSourcesManagement.validation.usernameEmpty', + { + defaultMessage: 'Username should not be empty', + } +); +export const dataSourceValidationPasswordEmpty = i18n.translate( + 'dataSourcesManagement.validation.passwordEmpty', + { + defaultMessage: 'Password should not be empty', + } +); diff --git a/src/plugins/data_source_management/public/components/utils.test.ts b/src/plugins/data_source_management/public/components/utils.test.ts new file mode 100644 index 00000000000..7aeb00e14f7 --- /dev/null +++ b/src/plugins/data_source_management/public/components/utils.test.ts @@ -0,0 +1,175 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { + createSingleDataSource, + deleteDataSourceById, + deleteMultipleDataSources, + getDataSourceById, + getDataSources, + isValidUrl, + updateDataSourceById, +} from './utils'; +import { coreMock } from '../../../../core/public/mocks'; +import { + getDataSourceByIdWithCredential, + getDataSourceByIdWithoutCredential, + getDataSourcesResponse, + getMappedDataSources, + mockDataSourceAttributesWithAuth, + mockErrorResponseForSavedObjectsCalls, + mockResponseForSavedObjectsCalls, +} from '../mocks'; +import { AuthType } from '../types'; + +const { savedObjects } = coreMock.createStart(); + +describe('DataSourceManagement: Utils.ts', () => { + describe('Get data source', () => { + test('Success: getting data sources', async () => { + mockResponseForSavedObjectsCalls(savedObjects.client, 'find', getDataSourcesResponse); + const fetchDataSources = await getDataSources(savedObjects.client); + expect(fetchDataSources.length).toBe(getDataSourcesResponse.savedObjects.length); + }); + test('Success but no data sources found: getting data sources', async () => { + mockResponseForSavedObjectsCalls(savedObjects.client, 'find', {}); + const fetchDataSources = await getDataSources(savedObjects.client); + expect(fetchDataSources.length).toBe(0); + }); + test('failure: getting data sources', async () => { + try { + mockErrorResponseForSavedObjectsCalls(savedObjects.client, 'find'); + await getDataSources(savedObjects.client); + } catch (e) { + expect(e).toBeTruthy(); + } + }); + }); + + describe('Get data source by ID', () => { + test('Success: getting data source by ID with credential', async () => { + mockResponseForSavedObjectsCalls(savedObjects.client, 'get', getDataSourceByIdWithCredential); + const dsById = await getDataSourceById('alpha-test', savedObjects.client); + expect(dsById.title).toBe('alpha-test'); + expect(dsById.auth.type).toBe(AuthType.UsernamePasswordType); + }); + test('Success: getting data source by ID without credential', async () => { + mockResponseForSavedObjectsCalls( + savedObjects.client, + 'get', + getDataSourceByIdWithoutCredential + ); + const dsById = await getDataSourceById('alpha-test', savedObjects.client); + expect(dsById.title).toBe('alpha-test'); + expect(dsById.auth.type).toBe(AuthType.NoAuth); + }); + test('Success but no data: getting data source by ID without credential', async () => { + mockResponseForSavedObjectsCalls(savedObjects.client, 'get', {}); + const dsById = await getDataSourceById('alpha-test', savedObjects.client); + expect(dsById?.description).toBe(''); + }); + test('failure: getting data source by ID', async () => { + try { + mockErrorResponseForSavedObjectsCalls(savedObjects.client, 'get'); + await getDataSourceById('alpha-test', savedObjects.client); + } catch (e) { + expect(e).toBeTruthy(); + } + }); + }); + + describe('Create data source', () => { + test('Success: creating data source', async () => { + mockResponseForSavedObjectsCalls(savedObjects.client, 'create', {}); + const createDs = await createSingleDataSource( + savedObjects.client, + mockDataSourceAttributesWithAuth + ); + expect(createDs).toBeTruthy(); + }); + test('failure: creating data source', async () => { + try { + mockErrorResponseForSavedObjectsCalls(savedObjects.client, 'create'); + await createSingleDataSource(savedObjects.client, mockDataSourceAttributesWithAuth); + } catch (e) { + expect(e).toBeTruthy(); + } + }); + }); + + describe('Update data source by id', () => { + test('Success: updating data source', async () => { + mockResponseForSavedObjectsCalls(savedObjects.client, 'update', {}); + const createDs = await updateDataSourceById( + savedObjects.client, + 'ds-1234', + mockDataSourceAttributesWithAuth + ); + expect(createDs).toBeTruthy(); + }); + test('failure: updating data sources', async () => { + try { + mockErrorResponseForSavedObjectsCalls(savedObjects.client, 'update'); + await updateDataSourceById( + savedObjects.client, + 'ds-1234', + mockDataSourceAttributesWithAuth + ); + } catch (e) { + expect(e).toBeTruthy(); + } + }); + }); + + describe('Delete data source by id', () => { + test('Success: deleting data source', async () => { + mockResponseForSavedObjectsCalls(savedObjects.client, 'delete', {}); + const createDs = await deleteDataSourceById('ds-1234', savedObjects.client); + expect(createDs).toBeTruthy(); + }); + test('failure: deleting data sources', async () => { + try { + mockErrorResponseForSavedObjectsCalls(savedObjects.client, 'delete'); + await deleteDataSourceById('ds-1234', savedObjects.client); + } catch (e) { + expect(e).toBeTruthy(); + } + }); + }); + + describe('Delete multiple data sources by id', () => { + test('Success: deleting multiple data source', async () => { + try { + mockResponseForSavedObjectsCalls(savedObjects.client, 'delete', {}); + await deleteMultipleDataSources(savedObjects.client, getMappedDataSources); + expect(true).toBe(true); // This will be executed if multiple delete call is successful. + } catch (e) { + // this block should not execute as the test case name suggests + expect(e).toBeFalsy(); + } + }); + test('failure: deleting multiple data sources', async () => { + try { + mockErrorResponseForSavedObjectsCalls(savedObjects.client, 'delete'); + await deleteMultipleDataSources(savedObjects.client, getMappedDataSources); + } catch (e) { + expect(e).toBeTruthy(); + } + }); + }); + + test('check if url is valid', () => { + /* False cases */ + expect(isValidUrl('')).toBeFalsy(); + expect(isValidUrl('test')).toBeFalsy(); + + /* True cases */ + expect(isValidUrl('https://test.com')).toBeTruthy(); + expect(isValidUrl('http://test.com')).toBeTruthy(); + + /* True cases: port number scenario*/ + expect(isValidUrl('http://192.168.1.1:1234/')).toBeTruthy(); + }); +}); diff --git a/src/plugins/data_source_management/public/components/utils.ts b/src/plugins/data_source_management/public/components/utils.ts new file mode 100644 index 00000000000..51f190be1ba --- /dev/null +++ b/src/plugins/data_source_management/public/components/utils.ts @@ -0,0 +1,89 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { SavedObjectsClientContract } from 'src/core/public'; +import { DataSourceTableItem, DataSourceAttributes } from '../types'; + +export async function getDataSources(savedObjectsClient: SavedObjectsClientContract) { + return savedObjectsClient + .find({ + type: 'data-source', + fields: ['id', 'description', 'title'], + perPage: 10000, + }) + .then( + (response) => + response?.savedObjects?.map?.((source) => { + const id = source.id; + const title = source.get('title'); + const description = source.get('description'); + + return { + id, + title, + description, + sort: `${title}`, + }; + }) || [] + ); +} + +export async function getDataSourceById( + id: string, + savedObjectsClient: SavedObjectsClientContract +) { + return savedObjectsClient.get('data-source', id).then((response) => { + const attributes: any = response?.attributes || {}; + return { + id: response.id, + title: attributes.title, + endpoint: attributes.endpoint, + description: attributes.description || '', + auth: attributes.auth, + }; + }); +} + +export async function createSingleDataSource( + savedObjectsClient: SavedObjectsClientContract, + attributes: DataSourceAttributes +) { + return savedObjectsClient.create('data-source', attributes); +} + +export async function updateDataSourceById( + savedObjectsClient: SavedObjectsClientContract, + id: string, + attributes: DataSourceAttributes +) { + return savedObjectsClient.update('data-source', id, attributes); +} + +export async function deleteDataSourceById( + id: string, + savedObjectsClient: SavedObjectsClientContract +) { + return savedObjectsClient.delete('data-source', id); +} + +export async function deleteMultipleDataSources( + savedObjectsClient: SavedObjectsClientContract, + selectedDataSources: DataSourceTableItem[] +) { + await Promise.all( + selectedDataSources.map(async (selectedDataSource) => { + await deleteDataSourceById(selectedDataSource.id, savedObjectsClient); + }) + ); +} + +export const isValidUrl = (endpoint: string) => { + try { + const url = new URL(endpoint); + return Boolean(url) && (url.protocol === 'http:' || url.protocol === 'https:'); + } catch (e) { + return false; + } +}; diff --git a/src/plugins/data_source_management/public/components/validation/datasource_form_validation.test.ts b/src/plugins/data_source_management/public/components/validation/datasource_form_validation.test.ts new file mode 100644 index 00000000000..0c64abc0487 --- /dev/null +++ b/src/plugins/data_source_management/public/components/validation/datasource_form_validation.test.ts @@ -0,0 +1,74 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { AuthType } from '../../types'; +import { CreateDataSourceState } from '../create_data_source_wizard/components/create_form/create_data_source_form'; +import { EditDataSourceState } from '../edit_data_source/components/edit_form/edit_data_source_form'; +import { + defaultValidation, + performDataSourceFormValidation, + validateUpdatePassword, +} from './datasource_form_validation'; +import { mockDataSourceAttributesWithAuth } from '../../mocks'; + +describe('DataSourceManagement: Form Validation', () => { + describe('validate create/edit datasource', () => { + let form: CreateDataSourceState | EditDataSourceState = { + formErrors: [], + formErrorsByField: { ...defaultValidation }, + title: '', + description: '', + endpoint: '', + auth: { + type: AuthType.UsernamePasswordType, + credentials: { + username: '', + password: '', + }, + }, + }; + test('should fail validation on all fields', () => { + const result = performDataSourceFormValidation(form); + expect(result.formErrors.length).toBe(4); + }); + test('should NOT fail validation on empty username/password when No Auth is selected', () => { + form.auth.type = AuthType.NoAuth; + const result = performDataSourceFormValidation(form); + expect(result.formErrors.length).toBe(2); + expect(result.formErrorsByField.createCredential.username.length).toBe(0); + expect(result.formErrorsByField.createCredential.password.length).toBe(0); + }); + test('should NOT fail validation on all fields', () => { + form = { ...form, ...mockDataSourceAttributesWithAuth }; + const result = performDataSourceFormValidation(form); + expect(result.formErrors.length).toBe(0); + }); + }); + + describe('validate passwords', () => { + const passwords = { + oldPassword: '', + newPassword: '', + confirmNewPassword: '', + }; + test('should fail validation for all fields', () => { + const result = validateUpdatePassword(passwords); + expect(result.formValidationErrors.length).toBe(2); + }); + test('should fail validation when passwords do not match', () => { + passwords.oldPassword = 'test'; + passwords.newPassword = 'test123'; + const result = validateUpdatePassword(passwords); + expect(result.formValidationErrors.length).toBe(1); + expect(result.formValidationErrorsByField.confirmNewPassword.length).toBe(1); + }); + test('should NOT fail validation ', () => { + passwords.confirmNewPassword = 'test123'; + const result = validateUpdatePassword(passwords); + expect(result.formValidationErrors.length).toBe(0); + expect(result.formValidationErrorsByField.confirmNewPassword.length).toBe(0); + }); + }); +}); diff --git a/src/plugins/data_source_management/public/components/validation/datasource_form_validation.ts b/src/plugins/data_source_management/public/components/validation/datasource_form_validation.ts new file mode 100644 index 00000000000..9135f429292 --- /dev/null +++ b/src/plugins/data_source_management/public/components/validation/datasource_form_validation.ts @@ -0,0 +1,122 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { isValidUrl } from '../utils'; +import { CreateDataSourceState } from '../create_data_source_wizard/components/create_form/create_data_source_form'; +import { AuthType } from '../../types'; +import { EditDataSourceState } from '../edit_data_source/components/edit_form/edit_data_source_form'; +import { UpdatePasswordFormType } from '../../types'; +import { + dataSourceValidationEndpointNotValid, + dataSourceValidationNewPasswordEmpty, + dataSourceValidationNoPasswordMatch, + dataSourceValidationOldPasswordEmpty, + dataSourceValidationPasswordEmpty, + dataSourceValidationTitleEmpty, + dataSourceValidationUsernameEmpty, +} from '../text_content'; + +export interface CreateEditDataSourceValidation { + title: string[]; + endpoint: string[]; + createCredential: { + username: string[]; + password: string[]; + }; +} + +export interface UpdatePasswordValidation { + oldPassword: string[]; + newPassword: string[]; + confirmNewPassword: string[]; +} + +export const defaultValidation: CreateEditDataSourceValidation = { + title: [], + endpoint: [], + createCredential: { + username: [], + password: [], + }, +}; +export const defaultPasswordValidationByField: UpdatePasswordValidation = { + oldPassword: [], + newPassword: [], + confirmNewPassword: [], +}; + +export const performDataSourceFormValidation = ( + formValues: CreateDataSourceState | EditDataSourceState +) => { + const validationByField: CreateEditDataSourceValidation = { + title: [], + endpoint: [], + createCredential: { + username: [], + password: [], + }, + }; + const formErrorMessages: string[] = []; + /* Title validation */ + if (!formValues?.title?.trim?.().length) { + validationByField.title.push(dataSourceValidationTitleEmpty); + formErrorMessages.push(dataSourceValidationTitleEmpty); + } + + /* Endpoint Validation */ + if (!isValidUrl(formValues?.endpoint)) { + validationByField.endpoint.push(dataSourceValidationEndpointNotValid); + formErrorMessages.push(dataSourceValidationEndpointNotValid); + } + + /* Credential Validation */ + + /* Username & Password */ + if (formValues?.auth?.type === AuthType.UsernamePasswordType) { + /* Username */ + if (!formValues.auth.credentials?.username) { + validationByField.createCredential.username.push(dataSourceValidationUsernameEmpty); + formErrorMessages.push(dataSourceValidationUsernameEmpty); + } + + /* password */ + if (!formValues.auth.credentials?.password) { + validationByField.createCredential.password.push(dataSourceValidationPasswordEmpty); + formErrorMessages.push(dataSourceValidationPasswordEmpty); + } + } + + return { + formErrors: formErrorMessages, + formErrorsByField: { ...validationByField }, + }; +}; + +export const validateUpdatePassword = (passwords: UpdatePasswordFormType) => { + const validationByField: UpdatePasswordValidation = { + oldPassword: [], + newPassword: [], + confirmNewPassword: [], + }; + + const formErrorMessages: string[] = []; + + if (!passwords.oldPassword) { + validationByField.oldPassword.push(dataSourceValidationOldPasswordEmpty); + formErrorMessages.push(dataSourceValidationOldPasswordEmpty); + } + if (!passwords.newPassword) { + validationByField.newPassword.push(dataSourceValidationNewPasswordEmpty); + formErrorMessages.push(dataSourceValidationNewPasswordEmpty); + } else if (passwords.confirmNewPassword !== passwords.newPassword) { + validationByField.confirmNewPassword.push(dataSourceValidationNoPasswordMatch); + formErrorMessages.push(dataSourceValidationNoPasswordMatch); + } + + return { + formValidationErrors: formErrorMessages, + formValidationErrorsByField: { ...validationByField }, + }; +}; diff --git a/src/plugins/data_source_management/public/components/validation/index.ts b/src/plugins/data_source_management/public/components/validation/index.ts new file mode 100644 index 00000000000..116d4a4a576 --- /dev/null +++ b/src/plugins/data_source_management/public/components/validation/index.ts @@ -0,0 +1,6 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +export * from './datasource_form_validation'; diff --git a/src/plugins/data_source_management/public/index.ts b/src/plugins/data_source_management/public/index.ts new file mode 100644 index 00000000000..acae34449ce --- /dev/null +++ b/src/plugins/data_source_management/public/index.ts @@ -0,0 +1,13 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { DataSourceManagementPlugin } from './plugin'; + +// This exports static code and TypeScript types, +// as well as, OpenSearch Dashboards Platform `plugin()` initializer. +export function plugin() { + return new DataSourceManagementPlugin(); +} +export { DataSourceManagementPluginStart } from './types'; diff --git a/src/plugins/data_source_management/public/management_app/index.ts b/src/plugins/data_source_management/public/management_app/index.ts new file mode 100644 index 00000000000..5ccbfb94764 --- /dev/null +++ b/src/plugins/data_source_management/public/management_app/index.ts @@ -0,0 +1,6 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +export { mountManagementSection } from './mount_management_section'; diff --git a/src/plugins/data_source_management/public/management_app/mount_management_section.tsx b/src/plugins/data_source_management/public/management_app/mount_management_section.tsx new file mode 100644 index 00000000000..9fe1f240638 --- /dev/null +++ b/src/plugins/data_source_management/public/management_app/mount_management_section.tsx @@ -0,0 +1,70 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { StartServicesAccessor } from 'src/core/public'; + +import { I18nProvider } from '@osd/i18n/react'; +import React from 'react'; +import ReactDOM from 'react-dom'; +import { Route, Router, Switch } from 'react-router-dom'; +import { DataPublicPluginStart } from 'src/plugins/data/public'; +import { ManagementAppMountParams } from '../../../management/public'; + +import { OpenSearchDashboardsContextProvider } from '../../../opensearch_dashboards_react/public'; +import { CreateDataSourceWizardWithRouter } from '../components/create_data_source_wizard'; +import { DataSourceTableWithRouter } from '../components/data_source_table'; +import { DataSourceManagementContext } from '../types'; +import { EditDataSourceWithRouter } from '../components/edit_data_source'; + +export interface DataSourceManagementStartDependencies { + data: DataPublicPluginStart; +} + +export async function mountManagementSection( + getStartServices: StartServicesAccessor, + params: ManagementAppMountParams +) { + const [ + { chrome, application, savedObjects, uiSettings, notifications, overlays, http, docLinks }, + ] = await getStartServices(); + + const deps: DataSourceManagementContext = { + chrome, + application, + savedObjects, + uiSettings, + notifications, + overlays, + http, + docLinks, + setBreadcrumbs: params.setBreadcrumbs, + }; + + ReactDOM.render( + + + + + + + + + + + + + + + + + , + params.element + ); + + return () => { + chrome.docTitle.reset(); + ReactDOM.unmountComponentAtNode(params.element); + }; +} diff --git a/src/plugins/data_source_management/public/mocks.ts b/src/plugins/data_source_management/public/mocks.ts new file mode 100644 index 00000000000..2e0333121e2 --- /dev/null +++ b/src/plugins/data_source_management/public/mocks.ts @@ -0,0 +1,180 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { throwError } from 'rxjs'; +import { SavedObjectsClientContract } from 'opensearch-dashboards/public'; +import { AuthType } from './types'; +import { coreMock } from '../../../core/public/mocks'; + +/* Mock Types */ + +export const docLinks = { + links: { + noDocumentation: { + indexPatterns: { + introduction: '', + }, + scriptedFields: {}, + }, + }, +}; + +const createDataSourceManagementContext = () => { + const { + chrome, + application, + savedObjects, + uiSettings, + notifications, + overlays, + } = coreMock.createStart(); + const { http } = coreMock.createSetup(); + + return { + chrome, + application, + savedObjects, + uiSettings, + notifications, + overlays, + http, + docLinks, + setBreadcrumbs: () => {}, + }; +}; + +export const mockManagementPlugin = { + createDataSourceManagementContext, + docLinks, +}; + +/* Mock data responses - JSON*/ +export const getDataSourcesResponse = { + savedObjects: [ + { + id: 'test', + type: 'data-source', + description: 'test datasource', + title: 'test', + get(field: string) { + const me: any = this || {}; + return me[field]; + }, + }, + { + id: 'test2', + type: 'data-source', + description: 'test datasource2', + title: 'test', + get(field: string) { + const me: any = this || {}; + return me[field]; + }, + }, + { + id: 'alpha-test', + type: 'data-source', + description: 'alpha test datasource', + title: 'alpha-test', + get(field: string) { + const me: any = this || {}; + return me[field]; + }, + }, + { + id: 'beta-test', + type: 'data-source', + description: 'beta test datasource', + title: 'beta-test', + get(field: string) { + const me: any = this || {}; + return me[field]; + }, + }, + ], +}; + +export const getMappedDataSources = [ + { + id: 'test', + description: 'test datasource', + title: 'test', + sort: 'test', + }, + { + id: 'test2', + description: 'test datasource2', + title: 'test', + sort: 'test', + }, + { + id: 'alpha-test', + description: 'alpha test datasource', + title: 'alpha-test', + sort: 'alpha-test', + }, + { + id: 'beta-test', + description: 'beta test datasource', + title: 'beta-test', + sort: 'beta-test', + }, +]; + +export const mockDataSourceAttributesWithAuth = { + id: 'test', + title: 'create-test-ds', + description: 'jest testing', + endpoint: 'https://test.com', + auth: { + type: AuthType.UsernamePasswordType, + credentials: { + username: 'test123', + password: 'test123', + }, + }, +}; +export const getDataSourceByIdWithCredential = { + attributes: { + id: 'alpha-test', + title: 'alpha-test', + endpoint: 'https://test.com', + description: 'alpha test data source', + auth: { + type: AuthType.UsernamePasswordType, + credentials: { + username: 'test123', + }, + }, + }, +}; + +export const getDataSourceByIdWithoutCredential = { + attributes: { + ...getDataSourceByIdWithCredential.attributes, + auth: { + type: AuthType.NoAuth, + credentials: undefined, + }, + }, + references: [], +}; + +export const mockResponseForSavedObjectsCalls = ( + savedObjectsClient: SavedObjectsClientContract, + savedObjectsMethodName: 'get' | 'find' | 'create' | 'delete' | 'update', + response: any +) => { + (savedObjectsClient[savedObjectsMethodName] as jest.Mock).mockResolvedValue(response); +}; + +export const mockErrorResponseForSavedObjectsCalls = ( + savedObjectsClient: SavedObjectsClientContract, + savedObjectsMethodName: 'get' | 'find' | 'create' | 'delete' | 'update' +) => { + (savedObjectsClient[savedObjectsMethodName] as jest.Mock).mockRejectedValue( + throwError(new Error('Error while fetching data sources')) + ); +}; diff --git a/src/plugins/data_source_management/public/plugin.ts b/src/plugins/data_source_management/public/plugin.ts new file mode 100644 index 00000000000..31ab8237a44 --- /dev/null +++ b/src/plugins/data_source_management/public/plugin.ts @@ -0,0 +1,42 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { CoreSetup, CoreStart, Plugin } from '../../../core/public'; + +import { PLUGIN_NAME } from '../common'; + +import { ManagementSetup } from '../../management/public'; + +export interface DataSourceManagementSetupDependencies { + management: ManagementSetup; +} + +const DSM_APP_ID = 'dataSources'; + +export class DataSourceManagementPlugin + implements Plugin { + public setup(core: CoreSetup, { management }: DataSourceManagementSetupDependencies) { + const opensearchDashboardsSection = management.sections.section.opensearchDashboards; + + if (!opensearchDashboardsSection) { + throw new Error('`opensearchDashboards` management section not found.'); + } + + opensearchDashboardsSection.registerApp({ + id: DSM_APP_ID, + title: PLUGIN_NAME, + order: 1, + mount: async (params) => { + const { mountManagementSection } = await import('./management_app'); + + return mountManagementSection(core.getStartServices, params); + }, + }); + } + + public start(core: CoreStart) {} + + public stop() {} +} diff --git a/src/plugins/data_source_management/public/types.ts b/src/plugins/data_source_management/public/types.ts new file mode 100644 index 00000000000..c0aa502b583 --- /dev/null +++ b/src/plugins/data_source_management/public/types.ts @@ -0,0 +1,83 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { + ChromeStart, + ApplicationStart, + IUiSettingsClient, + OverlayStart, + SavedObjectsStart, + NotificationsStart, + DocLinksStart, + HttpSetup, +} from 'src/core/public'; +import { ManagementAppMountParams } from 'src/plugins/management/public'; +import { SavedObjectAttributes } from 'src/core/types'; +import { OpenSearchDashboardsReactContextValue } from '../../opensearch_dashboards_react/public'; + +// eslint-disable-next-line @typescript-eslint/no-empty-interface +export interface DataSourceManagementPluginStart {} + +export interface DataSourceManagementContext { + chrome: ChromeStart; + application: ApplicationStart; + savedObjects: SavedObjectsStart; + uiSettings: IUiSettingsClient; + notifications: NotificationsStart; + overlays: OverlayStart; + http: HttpSetup; + docLinks: DocLinksStart; + setBreadcrumbs: ManagementAppMountParams['setBreadcrumbs']; +} + +export interface DataSourceTableItem { + id: string; + title: string; + description: string; + sort: string; +} + +export interface ToastMessageItem { + id: string; + defaultMessage: string; + color: 'primary' | 'success' | 'warning' | 'danger'; + iconType: string; +} + +export type DataSourceManagementContextValue = OpenSearchDashboardsReactContextValue< + DataSourceManagementContext +>; + +export interface UpdatePasswordFormType { + oldPassword: string; + newPassword: string; + confirmNewPassword: string; +} + +/* Datasource types */ +export enum AuthType { + NoAuth = 'no_auth', + UsernamePasswordType = 'username_password', +} + +export const credentialSourceOptions = [ + { value: AuthType.UsernamePasswordType, inputDisplay: 'Username & Password' }, + { value: AuthType.NoAuth, inputDisplay: 'No authentication' }, +]; + +export interface DataSourceAttributes extends SavedObjectAttributes { + title: string; + description?: string; + endpoint?: string; + auth: { + type: AuthType; + credentials: UsernamePasswordTypedContent | undefined; + }; +} + +export interface UsernamePasswordTypedContent extends SavedObjectAttributes { + username: string; + password?: string; +} diff --git a/src/plugins/index_pattern_management/opensearch_dashboards.json b/src/plugins/index_pattern_management/opensearch_dashboards.json index 20bb7b90759..1efd05cb1a4 100644 --- a/src/plugins/index_pattern_management/opensearch_dashboards.json +++ b/src/plugins/index_pattern_management/opensearch_dashboards.json @@ -3,6 +3,7 @@ "version": "opensearchDashboards", "server": true, "ui": true, + "optionalPlugins": ["dataSource"], "requiredPlugins": ["management", "data", "urlForwarding"], - "requiredBundles": ["opensearchDashboardsReact", "opensearchDashboardsUtils"] + "requiredBundles": ["opensearchDashboardsReact", "opensearchDashboardsUtils", "savedObjects"] } diff --git a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/__snapshots__/create_index_pattern_wizard.test.tsx.snap b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/__snapshots__/create_index_pattern_wizard.test.tsx.snap index 2371b22ff61..1546eda9b55 100644 --- a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/__snapshots__/create_index_pattern_wizard.test.tsx.snap +++ b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/__snapshots__/create_index_pattern_wizard.test.tsx.snap @@ -40,6 +40,7 @@ exports[`CreateIndexPatternWizard renders index pattern step when there are indi ] } goToNextStep={[Function]} + goToPreviousStep={[Function]} indexPatternCreationType={ IndexPatternCreationConfig { "httpClient": null, @@ -51,6 +52,12 @@ exports[`CreateIndexPatternWizard renders index pattern step when there are indi } } showSystemIndices={true} + stepInfo={ + Object { + "currentStepNumber": 1, + "totalStepNumber": 2, + } + } /> `; -exports[`CreateIndexPatternWizard renders time field step when step is set to 2 1`] = ` +exports[`CreateIndexPatternWizard renders time field step when step is set to TIME_FIELD_STEP 1`] = `
+ +
+ + +`; diff --git a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_data_source/components/header/__snapshots__/header.test.tsx.snap b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_data_source/components/header/__snapshots__/header.test.tsx.snap new file mode 100644 index 00000000000..e15b9428b45 --- /dev/null +++ b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_data_source/components/header/__snapshots__/header.test.tsx.snap @@ -0,0 +1,238 @@ +// Jest Snapshot v1, https://goo.gl/fbAQLP + +exports[`Header should render data source finder when choose to use data source 1`] = ` +
+ +

+ +

+ + + + + + + } + onChange={[Function]} + /> + + + + + + } + onChange={[Function]} + /> + + + + + + + + + + + + + + + +
+`; + +exports[`Header should render normally 1`] = ` +
+ +

+ +

+ + + + + + + } + onChange={[Function]} + /> + + + + + + } + onChange={[Function]} + /> + + + + + + + + + + + +
+`; diff --git a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_data_source/components/header/header.scss b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_data_source/components/header/header.scss new file mode 100644 index 00000000000..048ae2981c0 --- /dev/null +++ b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_data_source/components/header/header.scss @@ -0,0 +1,3 @@ +.dataSourceRadioHelperText { + text-indent: 24px; +} diff --git a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_data_source/components/header/header.test.tsx b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_data_source/components/header/header.test.tsx new file mode 100644 index 00000000000..0805f6a5f78 --- /dev/null +++ b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_data_source/components/header/header.test.tsx @@ -0,0 +1,98 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import React from 'react'; +import { Header } from '../header'; +import { shallow } from 'enzyme'; + +describe('Header', () => { + it('should render normally', () => { + const component = shallow( +
{}} + dataSourceRef={{ type: 'type', id: 'id' }!} + goToNextStep={() => {}} + isNextStepDisabled={true} + stepInfo={{ totalStepNumber: 0, currentStepNumber: 0 }} + /> + ); + + expect(component).toMatchSnapshot(); + }); + + it('should render data source finder when choose to use data source', () => { + const component = shallow( +
{}} + dataSourceRef={{ type: 'type', id: 'id' }!} + goToNextStep={() => {}} + isNextStepDisabled={true} + stepInfo={{ totalStepNumber: 0, currentStepNumber: 0 }} + /> + ); + + component + .find('[data-test-subj="createIndexPatternStepDataSourceUseDataSourceRadio"]') + .simulate('change', { + target: { + checked: true, + }, + }); + + expect(component).toMatchSnapshot(); + }); + + it('should disable next step before select data source', () => { + const component = shallow( +
{}} + dataSourceRef={{ type: 'type', id: 'id' }!} + goToNextStep={() => {}} + isNextStepDisabled={true} + stepInfo={{ totalStepNumber: 0, currentStepNumber: 0 }} + /> + ); + + component + .find('[data-test-subj="createIndexPatternStepDataSourceUseDataSourceRadio"]') + .simulate('change', { + target: { + checked: true, + }, + }); + + expect( + component + .find('[data-test-subj="createIndexPatternStepDataSourceNextStepButton"]') + .prop('isDisabled') + ).toEqual(true); + }); + + it('should enable next step when pick default option', () => { + const component = shallow( +
{}} + dataSourceRef={{ type: 'type', id: 'id' }!} + goToNextStep={() => {}} + isNextStepDisabled={true} + stepInfo={{ totalStepNumber: 0, currentStepNumber: 0 }} + /> + ); + + component + .find('[data-test-subj="createIndexPatternStepDataSourceUseDefaultRadio"]') + .simulate('change', { + target: { + checked: true, + }, + }); + + expect( + component + .find('[data-test-subj="createIndexPatternStepDataSourceNextStepButton"]') + .prop('isDisabled') + ).toEqual(false); + }); +}); diff --git a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_data_source/components/header/header.tsx b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_data_source/components/header/header.tsx new file mode 100644 index 00000000000..ea463a97a70 --- /dev/null +++ b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_data_source/components/header/header.tsx @@ -0,0 +1,168 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import React, { useState } from 'react'; +import './header.scss'; + +import { + EuiTitle, + EuiSpacer, + EuiText, + EuiFlexItem, + EuiButton, + EuiFlexGroup, + EuiRadio, +} from '@elastic/eui'; + +import { FormattedMessage } from '@osd/i18n/react'; +import { i18n } from '@osd/i18n'; +import { + DataSourceRef, + IndexPatternManagmentContext, +} from 'src/plugins/index_pattern_management/public/types'; +import { SavedObjectFinderUi } from '../../../../../../../../../plugins/saved_objects/public'; +import { useOpenSearchDashboards } from '../../../../../../../../../plugins/opensearch_dashboards_react/public'; +import { StepInfo } from '../../../../types'; + +interface HeaderProps { + onDataSourceSelected: (id: string, type: string) => void; + dataSourceRef: DataSourceRef; + goToNextStep: (dataSourceRef: DataSourceRef) => void; + isNextStepDisabled: boolean; + stepInfo: StepInfo; +} + +const DATA_SOURCE_PAGE_SIZE = 5; + +export const Header: React.FC = (props: HeaderProps) => { + const { dataSourceRef, onDataSourceSelected, goToNextStep, isNextStepDisabled, stepInfo } = props; + const { currentStepNumber, totalStepNumber } = stepInfo; + + const [defaultChecked, setDefaultChecked] = useState(true); + const [dataSourceChecked, setDataSourceChecked] = useState(false); + + const { savedObjects, uiSettings } = useOpenSearchDashboards< + IndexPatternManagmentContext + >().services; + + const onChangeDefaultChecked = (e) => { + setDefaultChecked(e.target.checked); + setDataSourceChecked(!e.target.checked); + }; + + const onChangeDataSourceChecked = (e) => { + setDataSourceChecked(e.target.checked); + setDefaultChecked(!e.target.checked); + }; + + return ( +
+ +

+ +

+ + + + + + + } + checked={defaultChecked} + onChange={(e) => onChangeDefaultChecked(e)} + compressed + /> + + + + + + } + checked={dataSourceChecked} + onChange={(e) => onChangeDataSourceChecked(e)} + compressed + /> + + + + {dataSourceChecked && ( + + + 'apps', // todo: #2034 + name: i18n.translate( + 'indexPatternManagement.createIndexPattern.searchSelection.savedObjectType.dataSource', + { + defaultMessage: 'Data Source', + } + ), + }, + ]} + fixedPageSize={DATA_SOURCE_PAGE_SIZE} + uiSettings={uiSettings} + savedObjects={savedObjects} + /> + + )} + + + + goToNextStep(dataSourceRef)} + isDisabled={isNextStepDisabled && !defaultChecked} + > + + + + +
+ ); +}; diff --git a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_data_source/components/header/index.ts b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_data_source/components/header/index.ts new file mode 100644 index 00000000000..3c25d4c42f0 --- /dev/null +++ b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_data_source/components/header/index.ts @@ -0,0 +1,6 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +export { Header } from './header'; diff --git a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_data_source/index.ts b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_data_source/index.ts new file mode 100644 index 00000000000..6cea2c4957b --- /dev/null +++ b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_data_source/index.ts @@ -0,0 +1,6 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +export { StepDataSource } from './step_data_source'; diff --git a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_data_source/step_data_source.test.tsx b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_data_source/step_data_source.test.tsx new file mode 100644 index 00000000000..42c5ffa4cee --- /dev/null +++ b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_data_source/step_data_source.test.tsx @@ -0,0 +1,21 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import React from 'react'; +import { shallow } from 'enzyme'; +import { StepDataSource } from './step_data_source'; + +describe('StepDataSource', () => { + it('should render normally', () => { + const component = shallow( + {}} + stepInfo={{ totalStepNumber: 0, currentStepNumber: 0 }} + /> + ); + + expect(component).toMatchSnapshot(); + }); +}); diff --git a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_data_source/step_data_source.tsx b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_data_source/step_data_source.tsx new file mode 100644 index 00000000000..5e1b1c92f1a --- /dev/null +++ b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_data_source/step_data_source.tsx @@ -0,0 +1,46 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { EuiPageContent } from '@elastic/eui'; +import React, { useState } from 'react'; +import { DataSourceRef } from 'src/plugins/index_pattern_management/public/types'; +import { StepInfo } from '../../types'; + +import { Header } from './components/header'; + +interface StepDataSourceProps { + goToNextStep: (dataSourceRef: DataSourceRef) => void; + stepInfo: StepInfo; +} + +export const StepDataSource = (props: StepDataSourceProps) => { + const { goToNextStep, stepInfo } = props; + + const [selectedDataSource, setSelectedDataSource] = useState(); + const [isNextStepDisabled, setIsNextStepDisabled] = useState(true); + + const onDataSourceSelected = (id: string, selectedType: string) => { + const selected = { id, type: selectedType }; + + setSelectedDataSource(selected); + setIsNextStepDisabled(false); + }; + + const renderContent = () => { + return ( + +
goToNextStep(selectedDataSource!)} + isNextStepDisabled={isNextStepDisabled} + stepInfo={stepInfo} + /> + + ); + }; + + return <>{renderContent()}; +}; diff --git a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_index_pattern/components/header/__snapshots__/header.test.tsx.snap b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_index_pattern/components/header/__snapshots__/header.test.tsx.snap index 851e5cc4c2a..35804208c24 100644 --- a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_index_pattern/components/header/__snapshots__/header.test.tsx.snap +++ b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_index_pattern/components/header/__snapshots__/header.test.tsx.snap @@ -7,9 +7,14 @@ exports[`Header should mark the input as invalid 1`] = ` >

@@ -119,9 +124,14 @@ exports[`Header should render normally 1`] = ` >

diff --git a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_index_pattern/components/header/header.test.tsx b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_index_pattern/components/header/header.test.tsx index e4b9b62eeca..8a8446cc561 100644 --- a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_index_pattern/components/header/header.test.tsx +++ b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_index_pattern/components/header/header.test.tsx @@ -45,6 +45,7 @@ describe('Header', () => { isNextStepDisabled={false} onChangeIncludingSystemIndices={() => {}} isIncludingSystemIndices={false} + stepInfo={{ totalStepNumber: 0, currentStepNumber: 0 }} /> ); @@ -63,6 +64,7 @@ describe('Header', () => { isNextStepDisabled={true} onChangeIncludingSystemIndices={() => {}} isIncludingSystemIndices={false} + stepInfo={{ totalStepNumber: 0, currentStepNumber: 0 }} /> ); diff --git a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_index_pattern/components/header/header.tsx b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_index_pattern/components/header/header.tsx index d2876765f66..b4e1803a8d2 100644 --- a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_index_pattern/components/header/header.tsx +++ b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_index_pattern/components/header/header.tsx @@ -45,6 +45,7 @@ import { import { i18n } from '@osd/i18n'; import { FormattedMessage } from '@osd/i18n/react'; +import { StepInfo } from '../../../../types'; interface HeaderProps { isInputInvalid: boolean; @@ -57,6 +58,7 @@ interface HeaderProps { showSystemIndices?: boolean; onChangeIncludingSystemIndices: (event: EuiSwitchEvent) => void; isIncludingSystemIndices: boolean; + stepInfo: StepInfo; } export const Header: React.FC = ({ @@ -70,6 +72,7 @@ export const Header: React.FC = ({ showSystemIndices = false, onChangeIncludingSystemIndices, isIncludingSystemIndices, + stepInfo, ...rest }) => (
@@ -77,7 +80,11 @@ export const Header: React.FC = ({

diff --git a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_index_pattern/step_index_pattern.tsx b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_index_pattern/step_index_pattern.tsx index 0b81d14d0e0..cf6d9f1398f 100644 --- a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_index_pattern/step_index_pattern.tsx +++ b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_index_pattern/step_index_pattern.tsx @@ -29,7 +29,14 @@ */ import React, { Component } from 'react'; -import { EuiSpacer, EuiCallOut, EuiSwitchEvent } from '@elastic/eui'; +import { + EuiSpacer, + EuiCallOut, + EuiSwitchEvent, + EuiFlexGroup, + EuiFlexItem, + EuiButtonEmpty, +} from '@elastic/eui'; import { i18n } from '@osd/i18n'; import { FormattedMessage } from '@osd/i18n/react'; import { indexPatterns, IndexPatternAttributes, UI_SETTINGS } from '../../../../../../data/public'; @@ -46,15 +53,18 @@ import { IndicesList } from './components/indices_list'; import { Header } from './components/header'; import { context as contextType } from '../../../../../../opensearch_dashboards_react/public'; import { IndexPatternCreationConfig } from '../../../../../../../plugins/index_pattern_management/public'; -import { MatchedItem } from '../../types'; -import { IndexPatternManagmentContextValue } from '../../../../types'; +import { MatchedItem, StepInfo } from '../../types'; +import { DataSourceRef, IndexPatternManagmentContextValue } from '../../../../types'; interface StepIndexPatternProps { allIndices: MatchedItem[]; indexPatternCreationType: IndexPatternCreationConfig; + goToPreviousStep: () => void; goToNextStep: (query: string, timestampField?: string) => void; initialQuery?: string; showSystemIndices: boolean; + dataSourceRef?: DataSourceRef; + stepInfo: StepInfo; } interface StepIndexPatternState { @@ -116,6 +126,8 @@ export class StepIndexPattern extends Component { - const { indexPatternCreationType } = this.props; + const { indexPatternCreationType, dataSourceRef } = this.props; + const dataSourceId = dataSourceRef?.id; const { existingIndexPatterns } = this.state; const { http } = this.context.services; const getIndexTags = (indexName: string) => indexPatternCreationType.getIndexTags(indexName); @@ -168,7 +182,14 @@ export class StepIndexPattern extends Component + + + + + + + ); + } + renderStatusMessage(matchedIndices: { allIndices: MatchedItem[]; exactMatchedIndices: MatchedItem[]; @@ -300,7 +352,7 @@ export class StepIndexPattern extends Component ); } @@ -383,6 +436,7 @@ export class StepIndexPattern extends Component {this.renderList(matchedIndices)} + {this.dataSrouceEnabled && this.renderGoToPrevious()} ); } diff --git a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_time_field/components/header/__snapshots__/header.test.tsx.snap b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_time_field/components/header/__snapshots__/header.test.tsx.snap index 9efda4fdac7..5dd6d1ad7e7 100644 --- a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_time_field/components/header/__snapshots__/header.test.tsx.snap +++ b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_time_field/components/header/__snapshots__/header.test.tsx.snap @@ -7,9 +7,14 @@ exports[`Header should render normally 1`] = ` >

diff --git a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_time_field/components/header/header.test.tsx b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_time_field/components/header/header.test.tsx index e7701dd1f89..8ddd8c0ee21 100644 --- a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_time_field/components/header/header.test.tsx +++ b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_time_field/components/header/header.test.tsx @@ -34,7 +34,13 @@ import { shallow } from 'enzyme'; describe('Header', () => { it('should render normally', () => { - const component = shallow(
); + const component = shallow( +
+ ); expect(component).toMatchSnapshot(); }); diff --git a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_time_field/components/header/header.tsx b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_time_field/components/header/header.tsx index 823afbf9a78..595d9e2e87b 100644 --- a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_time_field/components/header/header.tsx +++ b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_time_field/components/header/header.tsx @@ -33,19 +33,25 @@ import React from 'react'; import { EuiTitle, EuiSpacer, EuiText } from '@elastic/eui'; import { FormattedMessage } from '@osd/i18n/react'; +import { StepInfo } from '../../../../types'; interface HeaderProps { indexPattern: string; indexPatternName: string; + stepInfo: StepInfo; } -export const Header: React.FC = ({ indexPattern, indexPatternName }) => ( +export const Header: React.FC = ({ indexPattern, indexPatternName, stepInfo }) => (

diff --git a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_time_field/step_time_field.tsx b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_time_field/step_time_field.tsx index 312e37fc90e..2ef8806191a 100644 --- a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_time_field/step_time_field.tsx +++ b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/components/step_time_field/step_time_field.tsx @@ -46,8 +46,9 @@ import { TimeField } from './components/time_field'; import { AdvancedOptions } from './components/advanced_options'; import { ActionButtons } from './components/action_buttons'; import { context } from '../../../../../../opensearch_dashboards_react/public'; -import { IndexPatternManagmentContextValue } from '../../../../types'; +import { DataSourceRef, IndexPatternManagmentContextValue } from '../../../../types'; import { IndexPatternCreationConfig } from '../../../..'; +import { StepInfo } from '../../types'; interface StepTimeFieldProps { indexPattern: string; @@ -55,6 +56,8 @@ interface StepTimeFieldProps { createIndexPattern: (selectedTimeField: string | undefined, indexPatternId: string) => void; indexPatternCreationType: IndexPatternCreationConfig; selectedTimeField?: string; + dataSourceRef?: DataSourceRef; + stepInfo: StepInfo; } interface StepTimeFieldState { @@ -116,7 +119,7 @@ export class StepTimeField extends Component { - const { indexPattern: pattern } = this.props; + const { indexPattern: pattern, dataSourceRef } = this.props; const { getFetchForWildcardOptions } = this.props.indexPatternCreationType; this.setState({ isFetchingTimeFields: true }); @@ -124,6 +127,7 @@ export class StepTimeField extends Component 0 @@ -254,7 +258,11 @@ export class StepTimeField extends Component -
+
({ StepIndexPattern: 'StepIndexPattern' })); jest.mock('./components/step_time_field', () => ({ StepTimeField: 'StepTimeField' })); @@ -136,7 +137,7 @@ describe('CreateIndexPatternWizard', () => { expect(component).toMatchSnapshot(); }); - test('renders time field step when step is set to 2', async () => { + test('renders time field step when step is set to TIME_FIELD_STEP', async () => { const component = createComponentWithContext( CreateIndexPatternWizard, { ...routeComponentPropsMock }, @@ -146,7 +147,7 @@ describe('CreateIndexPatternWizard', () => { component.setState({ isInitiallyLoadingIndices: false, allIndices: [{ name: 'myIndexPattern' }], - step: 2, + step: TIME_FIELD_STEP, }); await component.update(); diff --git a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/create_index_pattern_wizard.tsx b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/create_index_pattern_wizard.tsx index 59b0e8bfc6a..f0f0563eee8 100644 --- a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/create_index_pattern_wizard.tsx +++ b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/create_index_pattern_wizard.tsx @@ -47,14 +47,27 @@ import { LoadingState } from './components/loading_state'; import { context as contextType } from '../../../../opensearch_dashboards_react/public'; import { getCreateBreadcrumbs } from '../breadcrumbs'; -import { ensureMinimumTime, getIndices } from './lib'; +import { + DATA_SOURCE_STEP, + ensureMinimumTime, + getCurrentStepNumber, + getIndices, + getInitialStepName, + getNextStep, + getPrevStep, + getTotalStepNumber, + INDEX_PATTERN_STEP, + StepType, + TIME_FIELD_STEP, +} from './lib'; import { IndexPatternCreationConfig } from '../..'; -import { IndexPatternManagmentContextValue } from '../../types'; +import { DataSourceRef, IndexPatternManagmentContextValue } from '../../types'; import { MatchedItem } from './types'; import { DuplicateIndexPatternError, IndexPattern } from '../../../../data/public'; +import { StepDataSource } from './components/step_data_source'; interface CreateIndexPatternWizardState { - step: number; + step: StepType; indexPattern: string; allIndices: MatchedItem[]; remoteClustersExist: boolean; @@ -63,6 +76,7 @@ interface CreateIndexPatternWizardState { indexPatternCreationType: IndexPatternCreationConfig; selectedTimeField?: string; docLinks: DocLinksStart; + dataSourceRef?: DataSourceRef; } export class CreateIndexPatternWizard extends Component< @@ -73,6 +87,9 @@ export class CreateIndexPatternWizard extends Component< public readonly context!: IndexPatternManagmentContextValue; + dataSourceEnabled: boolean; + totalSteps: number; + constructor(props: RouteComponentProps, context: IndexPatternManagmentContextValue) { super(props, context); @@ -80,12 +97,16 @@ export class CreateIndexPatternWizard extends Component< const type = new URLSearchParams(props.location.search).get('type') || undefined; + this.dataSourceEnabled = context.services.dataSourceEnabled; + this.totalSteps = getTotalStepNumber(this.dataSourceEnabled); + const isInitiallyLoadingIndices = !this.dataSourceEnabled; + this.state = { - step: 1, + step: getInitialStepName(this.dataSourceEnabled), indexPattern: '', allIndices: [], remoteClustersExist: false, - isInitiallyLoadingIndices: true, + isInitiallyLoadingIndices, toasts: [], indexPatternCreationType: context.services.indexPatternManagementStart.creation.getType(type), docLinks: context.services.docLinks, @@ -93,7 +114,9 @@ export class CreateIndexPatternWizard extends Component< } async UNSAFE_componentWillMount() { - this.fetchData(); + if (!this.dataSourceEnabled) { + this.fetchData(); + } } catchAndWarn = async ( @@ -120,6 +143,8 @@ export class CreateIndexPatternWizard extends Component< fetchData = async () => { const { http } = this.context.services; + const { dataSourceRef } = this.state; + const dataSourceId = dataSourceRef?.id; const getIndexTags = (indexName: string) => this.state.indexPatternCreationType.getIndexTags(indexName); const searchClient = this.context.services.data.search.search; @@ -141,7 +166,7 @@ export class CreateIndexPatternWizard extends Component< // query local and remote indices, updating state independently ensureMinimumTime( this.catchAndWarn( - getIndices({ http, getIndexTags, pattern: '*', searchClient }), + getIndices({ http, getIndexTags, pattern: '*', searchClient, dataSourceId }), [], indicesFailMsg @@ -153,7 +178,7 @@ export class CreateIndexPatternWizard extends Component< this.catchAndWarn( // if we get an error from remote cluster query, supply fallback value that allows user entry. // ['a'] is fallback value - getIndices({ http, getIndexTags, pattern: '*:*', searchClient }), + getIndices({ http, getIndexTags, pattern: '*:*', searchClient, dataSourceId }), ['a'], clustersFailMsg @@ -165,13 +190,14 @@ export class CreateIndexPatternWizard extends Component< createIndexPattern = async (timeFieldName: string | undefined, indexPatternId: string) => { let emptyPattern: IndexPattern; const { history } = this.props; - const { indexPattern } = this.state; + const { indexPattern, dataSourceRef } = this.state; try { emptyPattern = await this.context.services.data.indexPatterns.createAndSave({ id: indexPatternId, title: indexPattern, timeFieldName, + dataSourceRef, ...this.state.indexPatternCreationType.getIndexPatternMappings(), }); } catch (err) { @@ -209,12 +235,28 @@ export class CreateIndexPatternWizard extends Component< history.push(`/patterns/${emptyPattern.id}`); }; - goToTimeFieldStep = (indexPattern: string, selectedTimeField?: string) => { - this.setState({ step: 2, indexPattern, selectedTimeField }); + goToNextFromIndexPattern = (indexPattern: string, selectedTimeField?: string) => { + this.setState({ indexPattern, selectedTimeField }); + this.goToNextStep(); + }; + + goToNextFromDataSource = (dataSourceRef: DataSourceRef) => { + this.setState({ isInitiallyLoadingIndices: true, dataSourceRef }, async () => { + this.fetchData(); + this.goToNextStep(); + }); }; - goToIndexPatternStep = () => { - this.setState({ step: 1 }); + goToNextStep = () => { + this.setState((prevState) => ({ + step: getNextStep(prevState.step, this.dataSourceEnabled)!, + })); + }; + + goToPreviousStep = () => { + this.setState((prevState) => ({ + step: getPrevStep(prevState.step, this.dataSourceEnabled)!, + })); }; renderHeader() { @@ -230,7 +272,12 @@ export class CreateIndexPatternWizard extends Component< } renderContent() { - const { allIndices, isInitiallyLoadingIndices, step, indexPattern } = this.state; + const { allIndices, isInitiallyLoadingIndices, step, indexPattern, dataSourceRef } = this.state; + + const stepInfo = { + totalStepNumber: this.totalSteps, + currentStepNumber: getCurrentStepNumber(step, this.dataSourceEnabled), + }; if (isInitiallyLoadingIndices) { return ; @@ -238,7 +285,17 @@ export class CreateIndexPatternWizard extends Component< const header = this.renderHeader(); - if (step === 1) { + if (step === DATA_SOURCE_STEP) { + return ( + + {header} + + + + ); + } + + if (step === INDEX_PATTERN_STEP) { const { location } = this.props; const initialQuery = new URLSearchParams(location.search).get('id') || undefined; @@ -250,26 +307,32 @@ export class CreateIndexPatternWizard extends Component< allIndices={allIndices} initialQuery={indexPattern || initialQuery} indexPatternCreationType={this.state.indexPatternCreationType} - goToNextStep={this.goToTimeFieldStep} + goToPreviousStep={this.goToPreviousStep} + goToNextStep={this.goToNextFromIndexPattern} showSystemIndices={ - this.state.indexPatternCreationType.getShowSystemIndices() && this.state.step === 1 + this.state.indexPatternCreationType.getShowSystemIndices() && + this.state.step === INDEX_PATTERN_STEP } + dataSourceRef={dataSourceRef} + stepInfo={stepInfo} /> ); } - if (step === 2) { + if (step === TIME_FIELD_STEP) { return ( {header} ); diff --git a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/lib/creation_flow.test.ts b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/lib/creation_flow.test.ts new file mode 100644 index 00000000000..639357cb996 --- /dev/null +++ b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/lib/creation_flow.test.ts @@ -0,0 +1,44 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +import { + DATA_SOURCE_STEP, + getInitialStepName, + getNextStep, + getPrevStep, + getTotalStepNumber, + INDEX_PATTERN_STEP, + TIME_FIELD_STEP, +} from './creation_flow'; + +describe('getInitialStepName', () => { + it('should get correct first step base on different flow', () => { + expect(getInitialStepName(false)).toEqual(INDEX_PATTERN_STEP); + expect(getInitialStepName(true)).toEqual(DATA_SOURCE_STEP); + }); +}); + +describe('getTotalStepNumber', () => { + it('should get correct total number base on different flow', () => { + expect(getTotalStepNumber(false)).toEqual(2); + expect(getTotalStepNumber(true)).toEqual(3); + }); +}); + +describe('getNextStep', () => { + it('should get correct next step base on different flow', () => { + expect(getNextStep(INDEX_PATTERN_STEP, false)).toEqual(TIME_FIELD_STEP); + expect(getNextStep(INDEX_PATTERN_STEP, true)).toEqual(TIME_FIELD_STEP); + expect(getNextStep(DATA_SOURCE_STEP, true)).toEqual(INDEX_PATTERN_STEP); + }); +}); + +describe('getPrevStep', () => { + it('should get correct previous step base on different flow', () => { + expect(getPrevStep(TIME_FIELD_STEP, false)).toEqual(INDEX_PATTERN_STEP); + expect(getPrevStep(INDEX_PATTERN_STEP, true)).toEqual(DATA_SOURCE_STEP); + expect(getPrevStep(INDEX_PATTERN_STEP, false)).toEqual(null); + }); +}); diff --git a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/lib/creation_flow.ts b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/lib/creation_flow.ts new file mode 100644 index 00000000000..86a1deaf645 --- /dev/null +++ b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/lib/creation_flow.ts @@ -0,0 +1,79 @@ +/* + * Copyright OpenSearch Contributors + * SPDX-License-Identifier: Apache-2.0 + */ + +export const INDEX_PATTERN_STEP = 'INDEX_PATTERN_STEP'; +export const TIME_FIELD_STEP = 'TIME_FIELD_STEP'; +export const DATA_SOURCE_STEP = 'DATA_SOURCE_STEP'; + +const CREATION_FLOW_WITH_DATA_SOURCE_MAP = new Map(); +CREATION_FLOW_WITH_DATA_SOURCE_MAP.set(DATA_SOURCE_STEP, { + next: INDEX_PATTERN_STEP, + prev: null, + stepNumber: 1, +}); +CREATION_FLOW_WITH_DATA_SOURCE_MAP.set(INDEX_PATTERN_STEP, { + next: TIME_FIELD_STEP, + prev: DATA_SOURCE_STEP, + stepNumber: 2, +}); +CREATION_FLOW_WITH_DATA_SOURCE_MAP.set(TIME_FIELD_STEP, { + next: null, + prev: INDEX_PATTERN_STEP, + stepNumber: 3, +}); + +const DEFAULT_CREATION_FLOW_MAP = new Map(); +DEFAULT_CREATION_FLOW_MAP.set(INDEX_PATTERN_STEP, { + next: TIME_FIELD_STEP, + prev: null, + stepNumber: 1, +}); +DEFAULT_CREATION_FLOW_MAP.set(TIME_FIELD_STEP, { + next: null, + prev: INDEX_PATTERN_STEP, + stepNumber: 2, +}); + +export type StepType = 'INDEX_PATTERN_STEP' | 'TIME_FIELD_STEP' | 'DATA_SOURCE_STEP'; + +export const getInitialStepName = (dataSourceEnabled: boolean) => { + if (dataSourceEnabled) { + return DATA_SOURCE_STEP; + } + + return INDEX_PATTERN_STEP; +}; + +export const getNextStep = (currentStep: StepType, dataSourceEnabled: boolean): StepType | null => { + if (dataSourceEnabled) { + return CREATION_FLOW_WITH_DATA_SOURCE_MAP.get(currentStep).next; + } + + return DEFAULT_CREATION_FLOW_MAP.get(currentStep).next; +}; + +export const getPrevStep = (currentStep: StepType, dataSourceEnabled: boolean): StepType | null => { + if (dataSourceEnabled) { + return CREATION_FLOW_WITH_DATA_SOURCE_MAP.get(currentStep).prev; + } + + return DEFAULT_CREATION_FLOW_MAP.get(currentStep).prev; +}; + +export const getCurrentStepNumber = (currentStep: StepType, dataSourceEnabled: boolean): number => { + if (dataSourceEnabled) { + return CREATION_FLOW_WITH_DATA_SOURCE_MAP.get(currentStep).stepNumber; + } + + return DEFAULT_CREATION_FLOW_MAP.get(currentStep).stepNumber; +}; + +export const getTotalStepNumber = (dataSourceEnabled: boolean): number => { + if (dataSourceEnabled) { + return CREATION_FLOW_WITH_DATA_SOURCE_MAP.size; + } + + return DEFAULT_CREATION_FLOW_MAP.size; +}; diff --git a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/lib/get_indices.test.ts b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/lib/get_indices.test.ts index 0d8b215394a..56ae8515ddd 100644 --- a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/lib/get_indices.test.ts +++ b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/lib/get_indices.test.ts @@ -74,11 +74,32 @@ const searchClient = () => observer.next(successfulSearchResponse); observer.complete(); }) as any; +const dataSourceId = 'dataSourceId'; const http = httpServiceMock.createStartContract(); http.get.mockResolvedValue(successfulResolveResponse); describe('getIndices', () => { + it('should work in a basic case with data source', async () => { + const uncalledSearchClient = jest.fn(); + const result = await getIndices({ + http, + getIndexTags, + pattern: 'opensearch-dashboards', + searchClient: uncalledSearchClient, + dataSourceId, + }); + expect(http.get).toHaveBeenCalled(); + expect(http.get).toHaveBeenCalledWith( + expect.anything(), + expect.objectContaining({ query: { data_source: dataSourceId } }) + ); + expect(uncalledSearchClient).not.toHaveBeenCalled(); + expect(result.length).toBe(3); + expect(result[0].name).toBe('f-alias'); + expect(result[1].name).toBe('foo'); + }); + it('should work in a basic case', async () => { const uncalledSearchClient = jest.fn(); const result = await getIndices({ @@ -88,6 +109,10 @@ describe('getIndices', () => { searchClient: uncalledSearchClient, }); expect(http.get).toHaveBeenCalled(); + expect(http.get).toHaveBeenCalledWith( + expect.anything(), + expect.not.objectContaining({ query: { data_source: dataSourceId } }) + ); expect(uncalledSearchClient).not.toHaveBeenCalled(); expect(result.length).toBe(3); expect(result[0].name).toBe('f-alias'); @@ -101,6 +126,7 @@ describe('getIndices', () => { getIndexTags, pattern: '*:opensearch-dashboards', searchClient, + dataSourceId, }); expect(http.get).toHaveBeenCalled(); @@ -112,14 +138,21 @@ describe('getIndices', () => { }); it('should ignore ccs query-all', async () => { - expect((await getIndices({ http, getIndexTags, pattern: '*:', searchClient })).length).toBe(0); + expect( + (await getIndices({ http, getIndexTags, pattern: '*:', searchClient, dataSourceId })).length + ).toBe(0); }); it('should ignore a single comma', async () => { - expect((await getIndices({ http, getIndexTags, pattern: ',', searchClient })).length).toBe(0); - expect((await getIndices({ http, getIndexTags, pattern: ',*', searchClient })).length).toBe(0); expect( - (await getIndices({ http, getIndexTags, pattern: ',foobar', searchClient })).length + (await getIndices({ http, getIndexTags, pattern: ',', searchClient, dataSourceId })).length + ).toBe(0); + expect( + (await getIndices({ http, getIndexTags, pattern: ',*', searchClient, dataSourceId })).length + ).toBe(0); + expect( + (await getIndices({ http, getIndexTags, pattern: ',foobar', searchClient, dataSourceId })) + .length ).toBe(0); }); @@ -168,6 +201,7 @@ describe('getIndices', () => { getIndexTags, pattern: 'opensearch-dashboards', searchClient, + dataSourceId, }); expect(result.length).toBe(0); }); diff --git a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/lib/get_indices.ts b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/lib/get_indices.ts index 9d68b93aa7f..39bd044d801 100644 --- a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/lib/get_indices.ts +++ b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/lib/get_indices.ts @@ -34,7 +34,11 @@ import { i18n } from '@osd/i18n'; import { map, scan } from 'rxjs/operators'; import { IndexPatternCreationConfig } from '../../../../../index_pattern_management/public'; import { MatchedItem, ResolveIndexResponse, ResolveIndexResponseItemIndexAttrs } from '../types'; -import { DataPublicPluginStart, IOpenSearchSearchResponse } from '../../../../../data/public'; +import { + DataPublicPluginStart, + IOpenSearchSearchRequest, + IOpenSearchSearchResponse, +} from '../../../../../data/public'; import { MAX_SEARCH_SIZE } from '../constants'; const aliasLabel = i18n.translate('indexPatternManagement.aliasLabel', { defaultMessage: 'Alias' }); @@ -84,30 +88,15 @@ export const getIndicesViaSearch = async ({ pattern, searchClient, showAllIndices, + dataSourceId, }: { getIndexTags: IndexPatternCreationConfig['getIndexTags']; pattern: string; searchClient: DataPublicPluginStart['search']['search']; showAllIndices: boolean; + dataSourceId?: string; }): Promise => - searchClient({ - params: { - ignoreUnavailable: true, - expand_wildcards: showAllIndices ? 'all' : 'open', - index: pattern, - body: { - size: 0, // no hits - aggs: { - indices: { - terms: { - field: '_index', - size: MAX_SEARCH_SIZE, - }, - }, - }, - }, - }, - }) + searchClient(buildSearchRequest(showAllIndices, pattern, dataSourceId)) .pipe(map(searchResponseToArray(getIndexTags, showAllIndices))) .pipe(scan((accumulator = [], value) => accumulator.join(value))) .toPromise() @@ -118,15 +107,19 @@ export const getIndicesViaResolve = async ({ getIndexTags, pattern, showAllIndices, + dataSourceId, }: { http: HttpStart; getIndexTags: IndexPatternCreationConfig['getIndexTags']; pattern: string; showAllIndices: boolean; -}) => - http + dataSourceId?: string; +}) => { + const query = buildQuery(showAllIndices, dataSourceId); + + return http .get(`/internal/index-pattern-management/resolve_index/${pattern}`, { - query: showAllIndices ? { expand_wildcards: 'all' } : undefined, + query, }) .then((response) => { if (!response) { @@ -135,6 +128,7 @@ export const getIndicesViaResolve = async ({ return responseToItemArray(response, getIndexTags); } }); +}; /** * Takes two MatchedItem[]s and returns a merged set, with the second set prrioritized over the first based on name @@ -168,12 +162,14 @@ export async function getIndices({ pattern: rawPattern, showAllIndices = false, searchClient, + dataSourceId, }: { http: HttpStart; getIndexTags?: IndexPatternCreationConfig['getIndexTags']; pattern: string; showAllIndices?: boolean; searchClient: DataPublicPluginStart['search']['search']; + dataSourceId?: string; }): Promise { const pattern = rawPattern.trim(); const isCCS = pattern.indexOf(':') !== -1; @@ -202,6 +198,7 @@ export async function getIndices({ getIndexTags, pattern, showAllIndices, + dataSourceId, }).catch(() => []); requests.push(promiseResolve); @@ -212,6 +209,7 @@ export async function getIndices({ pattern, searchClient, showAllIndices, + dataSourceId, }).catch(() => []); requests.push(promiseSearch); } @@ -264,3 +262,42 @@ export const responseToItemArray = ( return sortBy(source, 'name'); }; + +const buildQuery = (showAllIndices: boolean, dataSourceId?: string) => { + const query = {} as any; + if (showAllIndices) { + query.expand_wildcards = 'all'; + } + if (dataSourceId) { + query.data_source = dataSourceId; + } + + return query; +}; + +const buildSearchRequest = (showAllIndices: boolean, pattern: string, dataSourceId?: string) => { + const request: IOpenSearchSearchRequest = { + params: { + ignoreUnavailable: true, + expand_wildcards: showAllIndices ? 'all' : 'open', + index: pattern, + body: { + size: 0, // no hits + aggs: { + indices: { + terms: { + field: '_index', + size: MAX_SEARCH_SIZE, + }, + }, + }, + }, + }, + }; + + if (dataSourceId) { + request.dataSourceId = dataSourceId; + } + + return request; +}; diff --git a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/lib/index.ts b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/lib/index.ts index b9e7771be27..09644bc4d8d 100644 --- a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/lib/index.ts +++ b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/lib/index.ts @@ -39,3 +39,5 @@ export { getMatchedIndices } from './get_matched_indices'; export { containsIllegalCharacters } from './contains_illegal_characters'; export { extractTimeFields } from './extract_time_fields'; + +export * from './creation_flow'; diff --git a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/types.ts b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/types.ts index b427a0e6fa6..9705bbc4b3e 100644 --- a/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/types.ts +++ b/src/plugins/index_pattern_management/public/components/create_index_pattern_wizard/types.ts @@ -79,3 +79,8 @@ export interface Tag { key: string; color: string; } + +export interface StepInfo { + totalStepNumber: number; + currentStepNumber: number; +} diff --git a/src/plugins/index_pattern_management/public/components/index_pattern_table/index_pattern_table.tsx b/src/plugins/index_pattern_management/public/components/index_pattern_table/index_pattern_table.tsx index efc7390dcc1..c19c9f5b0c2 100644 --- a/src/plugins/index_pattern_management/public/components/index_pattern_table/index_pattern_table.tsx +++ b/src/plugins/index_pattern_management/public/components/index_pattern_table/index_pattern_table.tsx @@ -104,12 +104,13 @@ export const IndexPatternTable = ({ canSave, history }: Props) => { http, getMlCardState, data, + dataSourceEnabled, } = useOpenSearchDashboards().services; const [indexPatterns, setIndexPatterns] = useState([]); const [creationOptions, setCreationOptions] = useState([]); const [sources, setSources] = useState([]); const [remoteClustersExist, setRemoteClustersExist] = useState(false); - const [isLoadingSources, setIsLoadingSources] = useState(true); + const [isLoadingSources, setIsLoadingSources] = useState(!dataSourceEnabled); const [isLoadingIndexPatterns, setIsLoadingIndexPatterns] = useState(true); useMount(() => { @@ -153,14 +154,16 @@ export const IndexPatternTable = ({ canSave, history }: Props) => { }; useEffect(() => { - getIndices({ http, pattern: '*', searchClient }).then((dataSources) => { - setSources(dataSources.filter(removeAliases)); - setIsLoadingSources(false); - }); - getIndices({ http, pattern: '*:*', searchClient }).then((dataSources) => - setRemoteClustersExist(!!dataSources.filter(removeAliases).length) - ); - }, [http, creationOptions, searchClient]); + if (!dataSourceEnabled) { + getIndices({ http, pattern: '*', searchClient }).then((dataSources) => { + setSources(dataSources.filter(removeAliases)); + setIsLoadingSources(false); + }); + getIndices({ http, pattern: '*:*', searchClient }).then((dataSources) => + setRemoteClustersExist(!!dataSources.filter(removeAliases).length) + ); + } + }, [http, creationOptions, searchClient, dataSourceEnabled]); chrome.docTitle.change(title); @@ -214,16 +217,18 @@ export const IndexPatternTable = ({ canSave, history }: Props) => { const hasDataIndices = sources.some(({ name }: MatchedItem) => !name.startsWith('.')); if (!indexPatterns.length) { - if (!hasDataIndices && !remoteClustersExist) { - return ( - - ); + if (!dataSourceEnabled) { + if (!hasDataIndices && !remoteClustersExist) { + return ( + + ); + } } else { return ( MlCardState; + dataSourceEnabled: boolean; } export type IndexPatternManagmentContextValue = OpenSearchDashboardsReactContextValue< @@ -67,3 +69,5 @@ export enum MlCardState { DISABLED, ENABLED, } + +export type DataSourceRef = Pick; diff --git a/src/plugins/index_pattern_management/server/routes/resolve_index.ts b/src/plugins/index_pattern_management/server/routes/resolve_index.ts index 09103945821..510eeb367da 100644 --- a/src/plugins/index_pattern_management/server/routes/resolve_index.ts +++ b/src/plugins/index_pattern_management/server/routes/resolve_index.ts @@ -49,6 +49,7 @@ export function registerResolveIndexRoute(router: IRouter): void { schema.literal('none'), ]) ), + data_source: schema.maybe(schema.string()), }), }, }, @@ -56,6 +57,18 @@ export function registerResolveIndexRoute(router: IRouter): void { const queryString = req.query.expand_wildcards ? { expand_wildcards: req.query.expand_wildcards } : null; + + const dataSourceId = req.query.data_source; + if (dataSourceId) { + const result = await ( + await context.dataSource.opensearch.getClient(dataSourceId) + ).indices.resolveIndex({ + name: encodeURIComponent(req.params.query), + expand_wildcards: req.query.expand_wildcards, + }); + return res.ok({ body: result.body }); + } + const result = await context.core.opensearch.legacy.client.callAsCurrentUser( 'transport.request', { diff --git a/src/plugins/management/server/capabilities_provider.ts b/src/plugins/management/server/capabilities_provider.ts index e3df4e02c68..2786378c982 100644 --- a/src/plugins/management/server/capabilities_provider.ts +++ b/src/plugins/management/server/capabilities_provider.ts @@ -38,6 +38,7 @@ export const capabilitiesProvider = () => ({ settings: true, indexPatterns: true, objects: true, + dataSources: true, }, }, }); diff --git a/yarn.lock b/yarn.lock index 88324326d73..796c88061ea 100644 --- a/yarn.lock +++ b/yarn.lock @@ -9,6 +9,146 @@ dependencies: "@jridgewell/trace-mapping" "^0.3.0" +"@aws-crypto/cache-material@^3.1.0": + version "3.1.0" + resolved "https://registry.yarnpkg.com/@aws-crypto/cache-material/-/cache-material-3.1.0.tgz#8369ed971feeaa710ee03cd5ccae11623e86b51a" + integrity sha512-fC59BV0YoxSSzI8bIAWLjaKrP52iOqey0NSvsZ5/kV/UwJp8VtDfD7UC5UneORQh1luRw8ZnC0kx0wHbqG/+iw== + dependencies: + "@aws-crypto/material-management" "^3.1.0" + "@aws-crypto/serialize" "^3.1.0" + "@types/lru-cache" "^5.1.0" + lru-cache "^6.0.0" + tslib "^2.2.0" + +"@aws-crypto/caching-materials-manager-node@^3.1.0": + version "3.1.0" + resolved "https://registry.yarnpkg.com/@aws-crypto/caching-materials-manager-node/-/caching-materials-manager-node-3.1.0.tgz#6e41af1ee63dfea4417c51928fc2ea58bfd010ca" + integrity sha512-1cb/XVb43RRq1PYdzBPyycfH+1ASZ6DfQ2GbSLFrMgPLvhnxoyyv+Bd5QlvViIREn3xIXa1gupeGJz6BaayFXQ== + dependencies: + "@aws-crypto/cache-material" "^3.1.0" + "@aws-crypto/material-management-node" "^3.1.0" + tslib "^2.2.0" + +"@aws-crypto/client-node@^3.1.1": + version "3.1.1" + resolved "https://registry.yarnpkg.com/@aws-crypto/client-node/-/client-node-3.1.1.tgz#97062007ec0a86e01713ba772a40af49e0322735" + integrity sha512-WdbYBxbB5onE4LVab5q7fByNiAAmL4Y+pGCaDOl6RLLH17xHBfyQHHeG+5+Xloygh/aWR42So8LhLPhGmral3g== + dependencies: + "@aws-crypto/caching-materials-manager-node" "^3.1.0" + "@aws-crypto/decrypt-node" "^3.1.0" + "@aws-crypto/encrypt-node" "^3.1.1" + "@aws-crypto/kms-keyring-node" "^3.1.0" + "@aws-crypto/material-management-node" "^3.1.0" + "@aws-crypto/raw-aes-keyring-node" "^3.1.0" + "@aws-crypto/raw-rsa-keyring-node" "^3.1.0" + tslib "^2.2.0" + +"@aws-crypto/decrypt-node@^3.1.0": + version "3.1.0" + resolved "https://registry.yarnpkg.com/@aws-crypto/decrypt-node/-/decrypt-node-3.1.0.tgz#f5d8298a50a2f73f4635770a8ed688df618595cd" + integrity sha512-F0TlNkdy11m1BCfhPQAXvdKHg82tAeOIc+tYXCL6ND/I3zh0F8oqVdr5oIuv4B6aIOgSSnAym0ctbAs8jw3pBA== + dependencies: + "@aws-crypto/material-management-node" "^3.1.0" + "@aws-crypto/serialize" "^3.1.0" + "@types/duplexify" "^3.6.0" + duplexify "^4.1.1" + readable-stream "^3.6.0" + tslib "^2.2.0" + +"@aws-crypto/encrypt-node@^3.1.1": + version "3.1.1" + resolved "https://registry.yarnpkg.com/@aws-crypto/encrypt-node/-/encrypt-node-3.1.1.tgz#7dcd533a008a0201d24bae6b0304a0cb87a2b7c8" + integrity sha512-LfB6LC1DhpsMCN2M24SgtR9m8CC2lTyO5w5mBWCsk+8h4BvhQXcgfruvHabGfDn4L+2Lr97nT9/THstNCuVVyg== + dependencies: + "@aws-crypto/material-management-node" "^3.1.0" + "@aws-crypto/serialize" "^3.1.0" + "@types/duplexify" "^3.6.0" + duplexify "^4.1.1" + readable-stream "^3.6.0" + tslib "^2.2.0" + +"@aws-crypto/hkdf-node@^3.0.0": + version "3.0.0" + resolved "https://registry.yarnpkg.com/@aws-crypto/hkdf-node/-/hkdf-node-3.0.0.tgz#9cf3a177ad1c1f98e245af6a19289bf98833f3b8" + integrity sha512-boeNV9G3Jk6W5Q9Zcj8yGqIOoQ2PwB+yvA/xFUazlu6qTtGRTviqkVx0Bf6r68KrkvmIY/9STN0mlF9q3jzPcQ== + dependencies: + tslib "^2.2.0" + +"@aws-crypto/kms-keyring-node@^3.1.0": + version "3.1.0" + resolved "https://registry.yarnpkg.com/@aws-crypto/kms-keyring-node/-/kms-keyring-node-3.1.0.tgz#64686da89192392d89036bbb03c64b99326ea77d" + integrity sha512-8SVn+Vz0HU3FXfHc6jdm4yvrAVssN7t5F9s3zzvKdb8w8u7vLpzZiLEw5TonmZD4KwBqG4YbciTX8wtXx3PUIw== + dependencies: + "@aws-crypto/kms-keyring" "^3.1.0" + "@aws-crypto/material-management-node" "^3.1.0" + aws-sdk "^2.650.0" + tslib "^2.2.0" + +"@aws-crypto/kms-keyring@^3.1.0": + version "3.1.0" + resolved "https://registry.yarnpkg.com/@aws-crypto/kms-keyring/-/kms-keyring-3.1.0.tgz#406399557a0d61f2ab79abe65f3c01f5160733fb" + integrity sha512-Z7hlnLs+8tz4XeRY6ZLccJ0TC7E1YJHLj5RuirZf/80tVmk9yJuicquLzev/5n2f0a01t7U4oyO2n8ahN81Isw== + dependencies: + "@aws-crypto/material-management" "^3.1.0" + tslib "^2.2.0" + +"@aws-crypto/material-management-node@^3.1.0": + version "3.1.0" + resolved "https://registry.yarnpkg.com/@aws-crypto/material-management-node/-/material-management-node-3.1.0.tgz#85dc37837a7170b3d7989fbb1ecc4583b57e598a" + integrity sha512-QX4JUqVydtskQbsKgd9JgTjzmrgYVaTOeguoV2KGz9TfqT//GDFUAL9jI6x20FdPH0A5D6BfgAAXzcUqjSWN5g== + dependencies: + "@aws-crypto/hkdf-node" "^3.0.0" + "@aws-crypto/material-management" "^3.1.0" + "@aws-crypto/serialize" "^3.1.0" + tslib "^2.2.0" + +"@aws-crypto/material-management@^3.1.0": + version "3.1.0" + resolved "https://registry.yarnpkg.com/@aws-crypto/material-management/-/material-management-3.1.0.tgz#888d08c4c707f7d443afb8a9cacd5b730958a6f1" + integrity sha512-bkxu2wr+Wk2KXpN/mDaGFbx2j5UoqqACAEecWzTpP8XafW9z8rzdVqtDp/3hUeytXrS0w+UwFtZQw1A946C5Ow== + dependencies: + asn1.js "^5.3.0" + bn.js "^5.1.1" + tslib "^2.2.0" + +"@aws-crypto/raw-aes-keyring-node@^3.1.0": + version "3.1.0" + resolved "https://registry.yarnpkg.com/@aws-crypto/raw-aes-keyring-node/-/raw-aes-keyring-node-3.1.0.tgz#54f8d92cc896709742bdc46f7af2f9bd8ac93b92" + integrity sha512-td1BH2OJks1PMkuHrMcwPJwVmyzsCrj1H9vqNnuw26dz9ZIdW8+e8DTv68WWUoCv43Qc6osmnGsTy7JjvGPzMQ== + dependencies: + "@aws-crypto/material-management-node" "^3.1.0" + "@aws-crypto/raw-keyring" "^3.1.0" + "@aws-crypto/serialize" "^3.1.0" + tslib "^2.2.0" + +"@aws-crypto/raw-keyring@^3.1.0": + version "3.1.0" + resolved "https://registry.yarnpkg.com/@aws-crypto/raw-keyring/-/raw-keyring-3.1.0.tgz#6adef656f2fca85a42b97887804518d5098dc078" + integrity sha512-GIZvpQ7eEXeV0FOZ5Tvwzz5t4dMmigjs739Dypt6q5Er02zQUB2OVmteLJH6p33j11zLaD1AGW6Yq22gy2+NDQ== + dependencies: + "@aws-crypto/material-management" "^3.1.0" + "@aws-crypto/serialize" "^3.1.0" + tslib "^2.2.0" + +"@aws-crypto/raw-rsa-keyring-node@^3.1.0": + version "3.1.0" + resolved "https://registry.yarnpkg.com/@aws-crypto/raw-rsa-keyring-node/-/raw-rsa-keyring-node-3.1.0.tgz#3a9f6d695d3b46bf819226a92bcb18fd77d2b71f" + integrity sha512-3EPnGm0l8Ui29p+jYctKaJAHv381MHvnjcPGw2ZGaVL+HAve+5WSL9JjwK8gOXoRot7ntt1KC1nBBmI+cj/G0g== + dependencies: + "@aws-crypto/material-management-node" "^3.1.0" + "@aws-crypto/raw-keyring" "^3.1.0" + tslib "^2.2.0" + +"@aws-crypto/serialize@^3.1.0": + version "3.1.0" + resolved "https://registry.yarnpkg.com/@aws-crypto/serialize/-/serialize-3.1.0.tgz#3c76095e344c9064f797daed7eb099aa5611629d" + integrity sha512-Dn1cZLudJrRAmwA95iVNo5ocKCX2sFvJe+cGVWPuj54qyF2gLfKQGWq20DWsu5Y7sSemp9NYbWsHD4/sesKnfw== + dependencies: + "@aws-crypto/material-management" "^3.1.0" + asn1.js "^5.3.0" + bn.js "^5.1.1" + tslib "^2.2.0" + "@babel/cli@^7.16.0": version "7.17.6" resolved "https://registry.yarnpkg.com/@babel/cli/-/cli-7.17.6.tgz#169e5935f1795f0b62ded5a2accafeedfe5c5363" @@ -2867,6 +3007,13 @@ resolved "https://registry.yarnpkg.com/@types/delete-empty/-/delete-empty-2.0.0.tgz#1647ae9e68f708a6ba778531af667ec55bc61964" integrity sha512-sq+kwx8zA9BSugT9N+Jr8/uWjbHMZ+N/meJEzRyT3gmLq/WMtx/iSIpvdpmBUi/cvXl6Kzpvve8G2ESkabFwmg== +"@types/duplexify@^3.6.0": + version "3.6.1" + resolved "https://registry.yarnpkg.com/@types/duplexify/-/duplexify-3.6.1.tgz#5685721cf7dc4a21b6f0e8a8efbec6b4d2fbafad" + integrity sha512-n0zoEj/fMdMOvqbHxmqnza/kXyoGgJmEpsXjpP+gEqE1Ye4yNqc7xWipKnUoMpWhMuzJQSfK2gMrwlElly7OGQ== + dependencies: + "@types/node" "*" + "@types/ejs@^3.1.0": version "3.1.0" resolved "https://registry.yarnpkg.com/@types/ejs/-/ejs-3.1.0.tgz#ab8109208106b5e764e5a6c92b2ba1c625b73020" @@ -4608,7 +4755,7 @@ asap@^2.0.0: resolved "https://registry.yarnpkg.com/asap/-/asap-2.0.6.tgz#e50347611d7e690943208bbdafebcbc2fb866d46" integrity sha1-5QNHYR1+aQlDIIu9r+vLwvuGbUY= -asn1.js@^5.2.0: +asn1.js@^5.2.0, asn1.js@^5.3.0: version "5.4.1" resolved "https://registry.yarnpkg.com/asn1.js/-/asn1.js-5.4.1.tgz#11a980b84ebb91781ce35b0fdc2ee294e3783f07" integrity sha512-+I//4cYPccV8LdmBLiX8CYvf9Sp3vQsrqu2QNXRcrbiWvcx/UdlFiqUJJzxRQxgsZmvhXhn4cSKeSmoFjVdupA== @@ -4751,6 +4898,22 @@ available-typed-arrays@^1.0.5: resolved "https://registry.yarnpkg.com/available-typed-arrays/-/available-typed-arrays-1.0.5.tgz#92f95616501069d07d10edb2fc37d3e1c65123b7" integrity sha512-DMD0KiN46eipeziST1LPP/STfDU0sufISXmjSgvVsoU2tqxctQeASejWcfNtxYKqETM1UxQ8sp2OrSBWpHY6sw== +aws-sdk@^2.650.0: + version "2.1214.0" + resolved "https://registry.yarnpkg.com/aws-sdk/-/aws-sdk-2.1214.0.tgz#6a57945b5bc9db79f8ee5ed99128a06110a88f83" + integrity sha512-50WxqYgEDB5UxwPJ0IDFWXe3ipAHhHmqfRnMNaQaZhb2aJpprbT7c0zic8AH9E1xJ9s+6QkhYrwQf/vXEHnLwg== + dependencies: + buffer "4.9.2" + events "1.1.1" + ieee754 "1.1.13" + jmespath "0.16.0" + querystring "0.2.0" + sax "1.2.1" + url "0.10.3" + util "^0.12.4" + uuid "8.0.0" + xml2js "0.4.19" + aws-sign2@~0.7.0: version "0.7.0" resolved "https://registry.yarnpkg.com/aws-sign2/-/aws-sign2-0.7.0.tgz#b46e890934a9591f2d2f6f86d7e6a9f1b3fe76a8" @@ -5241,7 +5404,7 @@ buffer-xor@^1.0.3: resolved "https://registry.yarnpkg.com/buffer-xor/-/buffer-xor-1.0.3.tgz#26e61ed1422fb70dd42e6e36729ed51d855fe8d9" integrity sha1-JuYe0UIvtw3ULm42cp7VHYVf6Nk= -buffer@^4.3.0: +buffer@4.9.2, buffer@^4.3.0: version "4.9.2" resolved "https://registry.yarnpkg.com/buffer/-/buffer-4.9.2.tgz#230ead344002988644841ab0244af8c44bbe3ef8" integrity sha512-xq+q3SRMOxGivLhBNaUdC64hDTQwejJ+H0T/NB1XMtTVEwNTrfFF3gAxiyW0Bu/xWEGhjVKgUcMhCrUy2+uCWg== @@ -6885,6 +7048,14 @@ define-properties@^1.1.2, define-properties@^1.1.3: dependencies: object-keys "^1.0.12" +define-properties@^1.1.4: + version "1.1.4" + resolved "https://registry.yarnpkg.com/define-properties/-/define-properties-1.1.4.tgz#0b14d7bd7fbeb2f3572c3a7eda80ea5d57fb05b1" + integrity sha512-uckOqKcfaVvtBdsVkdPv3XjveQJsNQqmhXgRi8uhvWWuPYZCNlzT8qAyblUgNoXdHdjMTzAqeGjAoli8f+bzPA== + dependencies: + has-property-descriptors "^1.0.0" + object-keys "^1.1.1" + define-property@^0.2.5: version "0.2.5" resolved "https://registry.yarnpkg.com/define-property/-/define-property-0.2.5.tgz#c35b1ef918ec3c990f9a5bc57be04aacec5c8116" @@ -7263,6 +7434,16 @@ duplexify@^3.2.0, duplexify@^3.4.2, duplexify@^3.6.0: readable-stream "^2.0.0" stream-shift "^1.0.0" +duplexify@^4.1.1: + version "4.1.2" + resolved "https://registry.yarnpkg.com/duplexify/-/duplexify-4.1.2.tgz#18b4f8d28289132fa0b9573c898d9f903f81c7b0" + integrity sha512-fz3OjcNCHmRP12MJoZMPglx8m4rrFP8rovnk4vT8Fs+aonZoCwGg10dSsQsfP/E62eZcPTMSMP6686fu9Qlqtw== + dependencies: + end-of-stream "^1.4.1" + inherits "^2.0.3" + readable-stream "^3.1.1" + stream-shift "^1.0.0" + eachr@^4.5.0: version "4.5.0" resolved "https://registry.yarnpkg.com/eachr/-/eachr-4.5.0.tgz#495eb3aab6a41811da1e04e510424df32075cf04" @@ -7603,6 +7784,35 @@ es-abstract@^1.18.5, es-abstract@^1.19.0, es-abstract@^1.19.1: string.prototype.trimstart "^1.0.4" unbox-primitive "^1.0.1" +es-abstract@^1.19.5, es-abstract@^1.20.0: + version "1.20.2" + resolved "https://registry.yarnpkg.com/es-abstract/-/es-abstract-1.20.2.tgz#8495a07bc56d342a3b8ea3ab01bd986700c2ccb3" + integrity sha512-XxXQuVNrySBNlEkTYJoDNFe5+s2yIOpzq80sUHEdPdQr0S5nTLz4ZPPPswNIpKseDDUS5yghX1gfLIHQZ1iNuQ== + dependencies: + call-bind "^1.0.2" + es-to-primitive "^1.2.1" + function-bind "^1.1.1" + function.prototype.name "^1.1.5" + get-intrinsic "^1.1.2" + get-symbol-description "^1.0.0" + has "^1.0.3" + has-property-descriptors "^1.0.0" + has-symbols "^1.0.3" + internal-slot "^1.0.3" + is-callable "^1.2.4" + is-negative-zero "^2.0.2" + is-regex "^1.1.4" + is-shared-array-buffer "^1.0.2" + is-string "^1.0.7" + is-weakref "^1.0.2" + object-inspect "^1.12.2" + object-keys "^1.1.1" + object.assign "^4.1.4" + regexp.prototype.flags "^1.4.3" + string.prototype.trimend "^1.0.5" + string.prototype.trimstart "^1.0.5" + unbox-primitive "^1.0.2" + es-array-method-boxes-properly@^1.0.0: version "1.0.0" resolved "https://registry.yarnpkg.com/es-array-method-boxes-properly/-/es-array-method-boxes-properly-1.0.0.tgz#873f3e84418de4ee19c5be752990b2e44718d09e" @@ -8099,6 +8309,11 @@ eventemitter2@~0.4.13: resolved "https://registry.yarnpkg.com/eventemitter2/-/eventemitter2-0.4.14.tgz#8f61b75cde012b2e9eb284d4545583b5643b61ab" integrity sha1-j2G3XN4BKy6esoTUVFWDtWQ7Yas= +events@1.1.1: + version "1.1.1" + resolved "https://registry.yarnpkg.com/events/-/events-1.1.1.tgz#9ebdb7635ad099c70dcc4c2a1f5004288e8bd924" + integrity sha512-kEcvvCBByWXGnZy6JUlgAp2gBIUjfCAV6P6TgT1/aaQKcmuAEC4OZTV1I4EWQLz2gxZw76atuVyvHhTxvi0Flw== + events@^3.0.0: version "3.3.0" resolved "https://registry.yarnpkg.com/events/-/events-3.3.0.tgz#31a95ad0a924e2d2c419a813aeb2c4e878ea7400" @@ -8844,7 +9059,7 @@ function-bind@^1.1.1: resolved "https://registry.yarnpkg.com/function-bind/-/function-bind-1.1.1.tgz#a56899d3ea3c9bab874bb9773b7c5ede92f4895d" integrity sha512-yIovAzMX49sF8Yl58fSCWJ5svSLuaibPxXQJFLmBObTuCr0Mf1KiPopGM9NiFjiYBCbfaa2Fh6breQ6ANVTI0A== -function.prototype.name@^1.1.2, function.prototype.name@^1.1.3: +function.prototype.name@^1.1.2, function.prototype.name@^1.1.3, function.prototype.name@^1.1.5: version "1.1.5" resolved "https://registry.yarnpkg.com/function.prototype.name/-/function.prototype.name-1.1.5.tgz#cce0505fe1ffb80503e6f9e46cc64e46a12a9621" integrity sha512-uN7m/BzVKQnCUF/iW8jYea67v++2u7m5UgENbHRtdDVclOUP+FMPlCNdmk0h/ysGyo2tavMJEDqJAkJdRa1vMA== @@ -8915,6 +9130,15 @@ get-intrinsic@^1.0.1, get-intrinsic@^1.0.2, get-intrinsic@^1.1.0, get-intrinsic@ has "^1.0.3" has-symbols "^1.0.1" +get-intrinsic@^1.1.2: + version "1.1.2" + resolved "https://registry.yarnpkg.com/get-intrinsic/-/get-intrinsic-1.1.2.tgz#336975123e05ad0b7ba41f152ee4aadbea6cf598" + integrity sha512-Jfm3OyCxHh9DJyc28qGk+JmfkpO41A4XkneDSujN9MDXrm4oDKdHvndhZ2dN94+ERNfkYJWDclW6k2L/ZGHjXA== + dependencies: + function-bind "^1.1.1" + has "^1.0.3" + has-symbols "^1.0.3" + get-nonce@^1.0.0: version "1.0.1" resolved "https://registry.yarnpkg.com/get-nonce/-/get-nonce-1.0.1.tgz#fdf3f0278073820d2ce9426c18f07481b1e0cdf3" @@ -9496,6 +9720,11 @@ has-bigints@^1.0.1: resolved "https://registry.yarnpkg.com/has-bigints/-/has-bigints-1.0.1.tgz#64fe6acb020673e3b78db035a5af69aa9d07b113" integrity sha512-LSBS2LjbNBTf6287JEbEzvJgftkF5qFkmCo9hDRpAzKhUOlJ+hx8dd4USs00SgsUNwc4617J9ki5YtEClM2ffA== +has-bigints@^1.0.2: + version "1.0.2" + resolved "https://registry.yarnpkg.com/has-bigints/-/has-bigints-1.0.2.tgz#0871bd3e3d51626f6ca0966668ba35d5602d6eaa" + integrity sha512-tSvCKtBr9lkF0Ex0aQiP9N+OpV4zi2r/Nee5VkRDbaqv35RLYMzbwQfFSZZH0kR+Rd6302UJZ2p/bJCEoR3VoQ== + has-dynamic-import@^2.0.1: version "2.0.1" resolved "https://registry.yarnpkg.com/has-dynamic-import/-/has-dynamic-import-2.0.1.tgz#9bca87846aa264f2ad224fcd014946f5e5182f52" @@ -9521,6 +9750,13 @@ has-glob@^1.0.0: dependencies: is-glob "^3.0.0" +has-property-descriptors@^1.0.0: + version "1.0.0" + resolved "https://registry.yarnpkg.com/has-property-descriptors/-/has-property-descriptors-1.0.0.tgz#610708600606d36961ed04c196193b6a607fa861" + integrity sha512-62DVLZGoiEBDHQyqG4w9xCuZ7eJEwNmJRWw2VY84Oedb7WFcA27fiEVe8oUQx9hAUJ4ekurquucTGwsyO1XGdQ== + dependencies: + get-intrinsic "^1.1.1" + has-symbols@^1.0.0, has-symbols@^1.0.1, has-symbols@^1.0.2, has-symbols@^1.0.3: version "1.0.3" resolved "https://registry.yarnpkg.com/has-symbols/-/has-symbols-1.0.3.tgz#bb7b2c4349251dce87b125f7bdf874aa7c8b39f8" @@ -9950,6 +10186,11 @@ icss-utils@^5.0.0, icss-utils@^5.1.0: resolved "https://registry.yarnpkg.com/icss-utils/-/icss-utils-5.1.0.tgz#c6be6858abd013d768e98366ae47e25d5887b1ae" integrity sha512-soFhflCVWLfRNOPU3iv5Z9VUdT44xFRbzjLsEzSr5AQmgqPMTHdU3PMT1Cf1ssx8fLNJDA1juftYl+PUcv3MqA== +ieee754@1.1.13: + version "1.1.13" + resolved "https://registry.yarnpkg.com/ieee754/-/ieee754-1.1.13.tgz#ec168558e95aa181fd87d37f55c32bbcb6708b84" + integrity sha512-4vf7I2LYV/HaWerSo3XmlMkp5eZ83i+/CDluXi/IGTs/O1sejBNhTtnxzmRZfvOUqj7lZjqHkeTvpgSFDlWZTg== + ieee754@^1.1.13, ieee754@^1.1.4, ieee754@^1.2.1: version "1.2.1" resolved "https://registry.yarnpkg.com/ieee754/-/ieee754-1.2.1.tgz#8eb7a10a63fff25d15a57b001586d177d1b0d352" @@ -10424,6 +10665,13 @@ is-generator-fn@^2.0.0: resolved "https://registry.yarnpkg.com/is-generator-fn/-/is-generator-fn-2.1.0.tgz#7d140adc389aaf3011a8f2a2a4cfa6faadffb118" integrity sha512-cTIB4yPYL/Grw0EaSzASzg6bBy9gqCofvWN8okThAYIxKJZC+udlRAmGbM0XLeniEJSs8uEgHPGuHSe1XsOLSQ== +is-generator-function@^1.0.7: + version "1.0.10" + resolved "https://registry.yarnpkg.com/is-generator-function/-/is-generator-function-1.0.10.tgz#f1558baf1ac17e0deea7c0415c438351ff2b3c72" + integrity sha512-jsEjy9l3yiXEQ+PsXdmBwEPcOxaXWLspKdplFUVI9vq1iZgIekeC0L167qeu86czQaxed3q/Uzuw0swL0irL8A== + dependencies: + has-tostringtag "^1.0.0" + is-glob@^3.0.0: version "3.1.0" resolved "https://registry.yarnpkg.com/is-glob/-/is-glob-3.1.0.tgz#7ba5ae24217804ac70707b96922567486cc3e84a" @@ -10610,6 +10858,13 @@ is-shared-array-buffer@^1.0.1: resolved "https://registry.yarnpkg.com/is-shared-array-buffer/-/is-shared-array-buffer-1.0.1.tgz#97b0c85fbdacb59c9c446fe653b82cf2b5b7cfe6" integrity sha512-IU0NmyknYZN0rChcKhRO1X8LYz5Isj/Fsqh8NJOSf+N/hCOTwy29F32Ik7a+QszE63IdvmwdTPDd6cZ5pg4cwA== +is-shared-array-buffer@^1.0.2: + version "1.0.2" + resolved "https://registry.yarnpkg.com/is-shared-array-buffer/-/is-shared-array-buffer-1.0.2.tgz#8f259c573b60b6a32d4058a1a07430c0a7344c79" + integrity sha512-sqN2UDu1/0y6uvXyStCOzyhAjCSlHceFoMKJW8W9EU9cvic/QdsZ0kEU93HEy3IUEFZIiH/3w+AH/UQbPHNdhA== + dependencies: + call-bind "^1.0.2" + is-stream@^1.0.0, is-stream@^1.1.0: version "1.1.0" resolved "https://registry.yarnpkg.com/is-stream/-/is-stream-1.1.0.tgz#12d4a3dd4e68e0b79ceb8dbc84173ae80d91ca44" @@ -10639,6 +10894,17 @@ is-symbol@^1.0.2, is-symbol@^1.0.3: dependencies: has-symbols "^1.0.2" +is-typed-array@^1.1.3: + version "1.1.9" + resolved "https://registry.yarnpkg.com/is-typed-array/-/is-typed-array-1.1.9.tgz#246d77d2871e7d9f5aeb1d54b9f52c71329ece67" + integrity sha512-kfrlnTTn8pZkfpJMUgYD7YZ3qzeJgWUn8XfVYBARc4wnmNOmLbmuuaAs3q5fvB0UJOn6yHAKaGTPM7d6ezoD/A== + dependencies: + available-typed-arrays "^1.0.5" + call-bind "^1.0.2" + es-abstract "^1.20.0" + for-each "^0.3.3" + has-tostringtag "^1.0.0" + is-typed-array@^1.1.7: version "1.1.8" resolved "https://registry.yarnpkg.com/is-typed-array/-/is-typed-array-1.1.8.tgz#cbaa6585dc7db43318bc5b89523ea384a6f65e79" @@ -11309,6 +11575,11 @@ jju@~1.4.0: resolved "https://registry.yarnpkg.com/jju/-/jju-1.4.0.tgz#a3abe2718af241a2b2904f84a625970f389ae32a" integrity sha1-o6vicYryQaKykE+EpiWXDzia4yo= +jmespath@0.16.0: + version "0.16.0" + resolved "https://registry.yarnpkg.com/jmespath/-/jmespath-0.16.0.tgz#b15b0a85dfd4d930d43e69ed605943c802785076" + integrity sha512-9FzQjJ7MATs1tSpnco1K6ayiYE3figslrXA72G2HQ/n76RzvYlofyi5QM+iX4YRs/pu3yzxlVQSST23+dMDknw== + joi@^13.5.2: version "13.7.0" resolved "https://registry.yarnpkg.com/joi/-/joi-13.7.0.tgz#cfd85ebfe67e8a1900432400b4d03bbd93fb879f" @@ -13337,6 +13608,11 @@ object-inspect@^1.12.0, object-inspect@^1.7.0, object-inspect@^1.9.0: resolved "https://registry.yarnpkg.com/object-inspect/-/object-inspect-1.12.0.tgz#6e2c120e868fd1fd18cb4f18c31741d0d6e776f0" integrity sha512-Ho2z80bVIvJloH+YzRmpZVQe87+qASmBUKZDWgx9cu+KDrX2ZDH/3tMy+gXbZETVGs2M8YdxObOh7XAtim9Y0g== +object-inspect@^1.12.2: + version "1.12.2" + resolved "https://registry.yarnpkg.com/object-inspect/-/object-inspect-1.12.2.tgz#c0641f26394532f28ab8d796ab954e43c009a8ea" + integrity sha512-z+cPxW0QGUp0mcqcsgQyLVRDoXFQbXOwBaqyF7VIgI4TWNQsDHrBpUQslRmIfAoYWdYzs6UlKJtB2XJpTaNSpQ== + object-is@^1.0.2, object-is@^1.1.2, object-is@^1.1.4, object-is@^1.1.5: version "1.1.5" resolved "https://registry.yarnpkg.com/object-is/-/object-is-1.1.5.tgz#b9deeaa5fc7f1846a0faecdceec138e5778f53ac" @@ -13377,6 +13653,16 @@ object.assign@^4.0.4, object.assign@^4.1.0, object.assign@^4.1.2: has-symbols "^1.0.1" object-keys "^1.1.1" +object.assign@^4.1.4: + version "4.1.4" + resolved "https://registry.yarnpkg.com/object.assign/-/object.assign-4.1.4.tgz#9673c7c7c351ab8c4d0b516f4343ebf4dfb7799f" + integrity sha512-1mxKf0e58bvyjSCtKYY4sRe9itRk3PJpquJOjeIkz885CczcI4IvJJDLPS72oowuSh+pBxUFROpX+TU++hxhZQ== + dependencies: + call-bind "^1.0.2" + define-properties "^1.1.4" + has-symbols "^1.0.3" + object-keys "^1.1.1" + object.defaults@^1.1.0: version "1.1.0" resolved "https://registry.yarnpkg.com/object.defaults/-/object.defaults-1.1.0.tgz#3a7f868334b407dea06da16d88d5cd29e435fecf" @@ -15079,6 +15365,15 @@ regexp.prototype.flags@^1.3.0, regexp.prototype.flags@^1.4.1: call-bind "^1.0.2" define-properties "^1.1.3" +regexp.prototype.flags@^1.4.3: + version "1.4.3" + resolved "https://registry.yarnpkg.com/regexp.prototype.flags/-/regexp.prototype.flags-1.4.3.tgz#87cab30f80f66660181a3bb7bf5981a872b367ac" + integrity sha512-fjggEOO3slI6Wvgjwflkc4NFRCTZAu5CnNfBd5qOMYhWdn67nJBBu34/TkD++eeFmd8C9r9jfXJ27+nSiRkSUA== + dependencies: + call-bind "^1.0.2" + define-properties "^1.1.3" + functions-have-names "^1.2.2" + regexpp@^2.0.1: version "2.0.1" resolved "https://registry.yarnpkg.com/regexpp/-/regexpp-2.0.1.tgz#8d19d31cf632482b589049f8281f93dbcba4d07f" @@ -15630,6 +15925,18 @@ sass-loader@^10.2.0: schema-utils "^3.0.0" semver "^7.3.2" +sass@~1.26.11: + version "1.26.12" + resolved "https://registry.yarnpkg.com/sass/-/sass-1.26.12.tgz#79eddaa1773fff32ccf19e00d1ce380fc2afc7d0" + integrity sha512-hmSwtBOWoS9zwe0yAS+QmaseVCUELiGV22gXHDR7+9stEsVuEuxfY1GhC8XmUpC+Ir3Hwq7NxSUNbnmkznnF7g== + dependencies: + chokidar ">=2.0.0 <4.0.0" + +sax@1.2.1: + version "1.2.1" + resolved "https://registry.yarnpkg.com/sax/-/sax-1.2.1.tgz#7b8e656190b228e81a66aea748480d828cd2d37a" + integrity sha512-8I2a3LovHTOpm7NV5yOyO8IHqgVsfK4+UuySrXU8YXkSRX7k6hCV9b3HrkKCr3nMpgj+0bmocaJJWpvp1oc7ZA== + sax@>=0.6.0, sax@^1.2.4: version "1.2.4" resolved "https://registry.yarnpkg.com/sax/-/sax-1.2.4.tgz#2816234e2378bddc4e5354fab5caa895df7100d9" @@ -16478,6 +16785,15 @@ string.prototype.trimend@^1.0.4: call-bind "^1.0.2" define-properties "^1.1.3" +string.prototype.trimend@^1.0.5: + version "1.0.5" + resolved "https://registry.yarnpkg.com/string.prototype.trimend/-/string.prototype.trimend-1.0.5.tgz#914a65baaab25fbdd4ee291ca7dde57e869cb8d0" + integrity sha512-I7RGvmjV4pJ7O3kdf+LXFpVfdNOxtCW/2C8f6jNiW4+PQchwxkCDzlk1/7p+Wl4bqFIZeF47qAHXLuHHWKAxog== + dependencies: + call-bind "^1.0.2" + define-properties "^1.1.4" + es-abstract "^1.19.5" + string.prototype.trimstart@^1.0.4: version "1.0.4" resolved "https://registry.yarnpkg.com/string.prototype.trimstart/-/string.prototype.trimstart-1.0.4.tgz#b36399af4ab2999b4c9c648bd7a3fb2bb26feeed" @@ -16486,6 +16802,15 @@ string.prototype.trimstart@^1.0.4: call-bind "^1.0.2" define-properties "^1.1.3" +string.prototype.trimstart@^1.0.5: + version "1.0.5" + resolved "https://registry.yarnpkg.com/string.prototype.trimstart/-/string.prototype.trimstart-1.0.5.tgz#5466d93ba58cfa2134839f81d7f42437e8c01fef" + integrity sha512-THx16TJCGlsN0o6dl2o6ncWUsdgnLRSA23rRE5pyGBw/mLr3Ej/R2LaqCtgP8VNMGZsvMWnf9ooZPyY2bHvUFg== + dependencies: + call-bind "^1.0.2" + define-properties "^1.1.4" + es-abstract "^1.19.5" + string_decoder@^1.0.0, string_decoder@^1.1.1: version "1.3.0" resolved "https://registry.yarnpkg.com/string_decoder/-/string_decoder-1.3.0.tgz#42f114594a46cf1a8e30b0a84f56c78c3edac21e" @@ -17507,6 +17832,16 @@ unbox-primitive@^1.0.1: has-symbols "^1.0.2" which-boxed-primitive "^1.0.2" +unbox-primitive@^1.0.2: + version "1.0.2" + resolved "https://registry.yarnpkg.com/unbox-primitive/-/unbox-primitive-1.0.2.tgz#29032021057d5e6cdbd08c5129c226dff8ed6f9e" + integrity sha512-61pPlCD9h51VoreyJ0BReideM3MDKMKnh6+V9L08331ipq6Q8OFXZYiqP6n/tbHx4s5I9uRhcye6BrbkizkBDw== + dependencies: + call-bind "^1.0.2" + has-bigints "^1.0.2" + has-symbols "^1.0.3" + which-boxed-primitive "^1.0.2" + unc-path-regex@^0.1.2: version "0.1.2" resolved "https://registry.yarnpkg.com/unc-path-regex/-/unc-path-regex-0.1.2.tgz#e73dd3d7b0d7c5ed86fbac6b0ae7d8c6a69d50fa" @@ -17819,6 +18154,14 @@ url-template@^2.0.8: resolved "https://registry.yarnpkg.com/url-template/-/url-template-2.0.8.tgz#fc565a3cccbff7730c775f5641f9555791439f21" integrity sha1-/FZaPMy/93MMd19WQflVV5FDnyE= +url@0.10.3: + version "0.10.3" + resolved "https://registry.yarnpkg.com/url/-/url-0.10.3.tgz#021e4d9c7705f21bbf37d03ceb58767402774c64" + integrity sha512-hzSUW2q06EqL1gKM/a+obYHLIO6ct2hwPuviqTTOcfFVc61UbfJ2Q32+uGL/HCPxKqrdGB5QUwIe7UqlDgwsOQ== + dependencies: + punycode "1.3.2" + querystring "0.2.0" + url@^0.11.0: version "0.11.0" resolved "https://registry.yarnpkg.com/url/-/url-0.11.0.tgz#3838e97cfc60521eb73c525a8e55bfdd9e2e28f1" @@ -17881,6 +18224,18 @@ util@^0.11.0: dependencies: inherits "2.0.3" +util@^0.12.4: + version "0.12.4" + resolved "https://registry.yarnpkg.com/util/-/util-0.12.4.tgz#66121a31420df8f01ca0c464be15dfa1d1850253" + integrity sha512-bxZ9qtSlGUWSOy9Qa9Xgk11kSslpuZwaxCg4sNIDj6FLucDab2JxnHwyNTCpHMtK1MjoQiWQ6DiUMZYbSrO+Sw== + dependencies: + inherits "^2.0.3" + is-arguments "^1.0.4" + is-generator-function "^1.0.7" + is-typed-array "^1.1.3" + safe-buffer "^5.1.2" + which-typed-array "^1.1.2" + utility-types@^3.10.0: version "3.10.0" resolved "https://registry.yarnpkg.com/utility-types/-/utility-types-3.10.0.tgz#ea4148f9a741015f05ed74fd615e1d20e6bed82b" @@ -17891,6 +18246,11 @@ uuid@3.3.2: resolved "https://registry.yarnpkg.com/uuid/-/uuid-3.3.2.tgz#1b4af4955eb3077c501c23872fc6513811587131" integrity sha512-yXJmeNaw3DnnKAOKJE51sL/ZaYfWJRl1pK9dr19YFCu0ObS231AB1/LbqTKRAQ5kw8A90rA6fr4riOUpTZvQZA== +uuid@8.0.0: + version "8.0.0" + resolved "https://registry.yarnpkg.com/uuid/-/uuid-8.0.0.tgz#bc6ccf91b5ff0ac07bbcdbf1c7c4e150db4dbb6c" + integrity sha512-jOXGuXZAWdsTH7eZLtyXMqUb9EcWMGZNbL9YcGBJl4MH4nrxHmZJhEHvyLFrkxo+28uLb/NYRcStH48fnD0Vzw== + uuid@^2.0.1: version "2.0.3" resolved "https://registry.yarnpkg.com/uuid/-/uuid-2.0.3.tgz#67e2e863797215530dff318e5bf9dcebfd47b21a" @@ -18898,6 +19258,14 @@ xml-parse-from-string@^1.0.0: resolved "https://registry.yarnpkg.com/xml-parse-from-string/-/xml-parse-from-string-1.0.1.tgz#a9029e929d3dbcded169f3c6e28238d95a5d5a28" integrity sha1-qQKekp09vN7RafPG4oI42VpdWig= +xml2js@0.4.19: + version "0.4.19" + resolved "https://registry.yarnpkg.com/xml2js/-/xml2js-0.4.19.tgz#686c20f213209e94abf0d1bcf1efaa291c7827a7" + integrity sha512-esZnJZJOiJR9wWKMyuvSE1y6Dq5LCuJanqhxslH2bxM6duahNZ+HMpCLhBQGZkbX6xRf8x1Y2eJlgt2q3qo49Q== + dependencies: + sax ">=0.6.0" + xmlbuilder "~9.0.1" + xml2js@^0.4.22, xml2js@^0.4.5: version "0.4.23" resolved "https://registry.yarnpkg.com/xml2js/-/xml2js-0.4.23.tgz#a0c69516752421eb2ac758ee4d4ccf58843eac66" @@ -18916,6 +19284,11 @@ xmlbuilder@~11.0.0: resolved "https://registry.yarnpkg.com/xmlbuilder/-/xmlbuilder-11.0.1.tgz#be9bae1c8a046e76b31127726347d0ad7002beb3" integrity sha512-fDlsI/kFEx7gLvbecc0/ohLG50fugQp8ryHzMTuW9vSa1GJ0XYWKnhsUx7oie3G98+r56aTQIUB4kht42R3JvA== +xmlbuilder@~9.0.1: + version "9.0.7" + resolved "https://registry.yarnpkg.com/xmlbuilder/-/xmlbuilder-9.0.7.tgz#132ee63d2ec5565c557e20f4c22df9aca686b10d" + integrity sha512-7YXTQc3P2l9+0rjaUbLwMKRhtmwg1M1eDf6nag7urC7pIPYLD9W/jmzQ4ptRSUbodw5S0jfoGTflLemQibSpeQ== + xmlchars@^2.2.0: version "2.2.0" resolved "https://registry.yarnpkg.com/xmlchars/-/xmlchars-2.2.0.tgz#060fe1bcb7f9c76fe2a17db86a9bc3ab894210cb"