Migrate plugin to be compatible with OpenSearch Dashboards #1

ohltyler · 2021-04-08T19:08:40Z

This PR migrates the plugin to be compatible with OpenSearch Dashboards, and be able to work with the OpenSearch Anomaly Detection plugin. Compatible with OpenSearch Dashboards 7.10.2.

Major changes include:

Change plugin name (as written in package.json and as shown in the url via browser) from opendistro-anomaly-detection-kibana -> anomaly-detection-dashboards
Change artifact name from opendistroAnomalyDetectionKibana -> anomalyDetectionDashboards
Updated README with new name changes + updated and tested new links
Mentions of Kibana changed to OpenSearch Dashboards
Mentions of ES/Elasticsearch changed to OpenSearch
Mentions of kbn changed to osd
Mentions of odfe to opensearch
Updated links (to other pages within the plugin, to Discover page, etc.) & made sure they worked
Updated KibanaRequest / IKibanaResponse to OpenSearchDashboardsRequest / IOpenSearchDashboardsResponse, respectively
Changed app category from Open Distro for Elasticsearch to OpenSearch Plugins (this is not a finalized change)

What works:

All unit tests passing
All integration tests passing
Additional sanity test of plugin functionality by creating/starting/stopping/deleting detectors

What doesn't work:

Integration test & CD GitHub workflows - OpenSearch artifacts are not published yet.
Integration with alerting - will need to be tested after the OpenSearch + Dashboards plugins have been migrated

…naUtils

…erver files

…rvice

dblock

Some remaining opendistro's that we might not want. Would search the codebase for more.

dblock · 2021-04-08T20:09:48Z

.github/workflows/e2e-tests-workflow.yml

  ODFE_VERSION: 1.13.1
-  AD_KIBANA_PLUGIN_NAME: opendistroAnomalyDetectionKibana
+  AD_OPENSEARCH_DASHBOARDS_PLUGIN_NAME: opendistroAnomalyDetectionOpenSearchDashboards


Should this be anomalyDetectionOpenSearchDashboards or even anomalyDetectionDashboards?

Thanks for taking a look. Regarding the remaining opendistros and plugin name: I believe there is nothing finalized here regarding plugin naming convention changes, so have just simply replaced Kibana with OpenSearchDashboards for now. I was also told there is no change of branding for opendistro for now, which is why I have left it for now. Let me reach out again to get some clarification on this.

@dblock have changed the plugin name (as seen in the url) to anomaly-detection-dashboards, and build zip to anomalyDetectionDashboards, and have removed remaining mentions of opendistro/ open-distro / Open Distro for Elasticsearch, with a few exceptions:

Sample detectors and corresponding indices: this will be a very involved change that will affect current users if we change these names, lots to consider regarding backwards compatibility, etc. Will consider this in a follow-up PR. Have created Update sample detectors names & indices #3 to track)

Release notes - will leave these as-is until a new standard format is decided upon in the future

Links to documentation (currently still using opendistro website for this)

References to backend indices, which will be changed in the future

Left all references to odfe for now until there is a new standard acronym

Have re-ran all unit & integration tests; sanity tested workflows via browser.

Nice work. I'll re-read the PR for typos.

The only question I have is around ODFE that you called out above, for things like volume names in docker containers. Isn't that just becoming opensearch? I don't think it makes sense to continue abbreviating that to, say, os, that would be super confusing.

@dblock as discussed, we will not have a separate acronym for OpenSearch, and can replace odfe with opensearch. Have made those changes in the latest commit. Re-ran tests + did another round of sanity testing, looks good. No remaining instances of odfe besides release notes.

@ohltyler Except for the sample detectors, we need to consider migrating all AD ODFE indices to open search ? And change ODFE AD URL /_opendistro/_anomaly_detection to /_opensearch/_anomaly_detection.

Do we have some issue to track these? It needs both backend and frontend changes.

And we need to call some alerting APIs like search alerts, remember to change this part once alerting API changes done.

Yes, the changes regarding indices / api changes from opendistro -> opensearch is being treated as a separate step. Current efforts are to make the ODFE plugins compatible with OpenSearch / OpenSearch Dashboards first.

.github/workflows/e2e-tests-workflow.yml

.cypress/utils/constants.ts

.eslintrc

…to anomalyDetectionDashboards; remove most instances of opendistro / open distro

CONTRIBUTING.md

.github/workflows/e2e-tests-workflow.yml

public/pages/HistoricalDetectorList/containers/HistoricalDetectorList.tsx

…rting plugin

README.md

ylwu-amzn · 2021-04-12T19:40:17Z

Should we change the ODFE doc link now? Like this one : https://github.com/opendistro-for-elasticsearch/anomaly-detection-kibana-plugin/blob/main/public/pages/AnomalyCharts/components/AlertsFlyout/AlertsFlyout.tsx#L89
Not sure if we have migrated ODFE docs or not.

ohltyler · 2021-04-12T19:49:42Z

Should we change the ODFE doc link now? Like this one : https://github.com/opendistro-for-elasticsearch/anomaly-detection-kibana-plugin/blob/main/public/pages/AnomalyCharts/components/AlertsFlyout/AlertsFlyout.tsx#L89
Not sure if we have migrated ODFE docs or not.

We have not. I was told we are ok to continue referencing the Open Distro docs for now.

ylwu-amzn · 2021-04-12T19:54:46Z

Do we plan to keep ODFE repo in future? If yes, will we commit any bug fix/patch to it or just keep it immutable? If not, should we remove or modify the ODFE issue link in code and release code?

ohltyler · 2021-04-12T20:13:28Z

Do we plan to keep ODFE repo in future? If yes, will we commit any bug fix/patch to it or just keep it immutable? If not, should we remove or modify the ODFE issue link in code and release code?

Yes, bug fixes may be backported to the Open Distro repo for current users. The issues and projects can be moved to this repo once it is public to move community development focus here.

yizheliu-amazon

Looks good to me. Thanks for the change.

ylwu-amzn

LGTM. Thanks for the change!

dblock · 2021-04-14T00:00:18Z

.github/workflows/CD.yml

        uses: actions/checkout@v2
        with:
-          path: kibana/plugins/anomaly-detection-kibana-plugin
+          path: OpenSearch-Dashboards/plugins/anomaly-detection-dashboards-plugin


Should this have been lower-case opensearch-dashboards? It's a path.

When cloning the repo via the checkout action, the dir is OpenSearch-Dashboards, which is different than the cloned Kibana repo which was just kibana. Will confirm if it needs to be changed once these workflows are runnable.

ohltyler added 10 commits April 7, 2021 11:48

Update workflows except for artifacts; update base files; rename kiba…

96a7342

…naUtils

Update more Kibana references; update all request/response types in s…

5d4d66c

…erver files

Fix plugin name in README links and workflows

ff2e069

Update contributing doc + eslintrc, update test snapshots so all UT pass

023d9c4

Fix flaky cypress test

0f89856

Remove remaining mentions of Kibana

af13989

Update README; remove some mentions of Elasticsearch

da4bbf1

Replace most instances of Elasticsearch

2e31984

Remove mentions of Elasticsearch Query DSL

cecfb08

Rename routes/elasticsearch to routes/opensearch and corresponding se…

fda3591

…rvice

ohltyler added the enhancement New feature or request label Apr 8, 2021

ohltyler requested review from saratvemulapalli, ylwu-amzn and weicongs-amazon April 8, 2021 19:08

dblock reviewed Apr 8, 2021

View reviewed changes

Change plugin name to anomaly-detection-dashboards; change build zip …

6199cc7

…to anomalyDetectionDashboards; remove most instances of opendistro / open distro

ohltyler force-pushed the rename branch from e4b06bf to 6199cc7 Compare April 9, 2021 18:32

dblock mentioned this pull request Apr 9, 2021

Migrate notebooks frontend and backend plugin to OpenSearch opensearch-project/dashboards-notebooks#3

Merged

Change odfe to opensearch

f346592

yizheliu-amazon reviewed Apr 9, 2021

View reviewed changes

CONTRIBUTING.md Outdated Show resolved Hide resolved

.github/workflows/e2e-tests-workflow.yml Show resolved Hide resolved

public/pages/HistoricalDetectorList/containers/HistoricalDetectorList.tsx Outdated Show resolved Hide resolved

Update CONTRIBUTING.md target branch; update comments referencing Ale…

1a6bff8

…rting plugin

ylwu-amzn reviewed Apr 12, 2021

View reviewed changes

README.md Show resolved Hide resolved

yizheliu-amazon approved these changes Apr 12, 2021

View reviewed changes

ylwu-amzn approved these changes Apr 12, 2021

View reviewed changes

ohltyler merged commit 9050115 into opensearch-project:main Apr 12, 2021

ohltyler deleted the rename branch April 12, 2021 22:09

dblock reviewed Apr 14, 2021

View reviewed changes

ohltyler mentioned this pull request Jul 21, 2021

[SINGLE FLOW] Rebase to OpenSearch #56

Merged

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrate plugin to be compatible with OpenSearch Dashboards #1

Migrate plugin to be compatible with OpenSearch Dashboards #1

ohltyler commented Apr 8, 2021 •

edited

Loading

dblock left a comment

dblock Apr 8, 2021

ohltyler Apr 8, 2021

ohltyler Apr 9, 2021 •

edited

Loading

dblock Apr 9, 2021

dblock Apr 9, 2021

ohltyler Apr 9, 2021

ylwu-amzn Apr 12, 2021 •

edited

Loading

ylwu-amzn Apr 12, 2021

ohltyler Apr 12, 2021

ylwu-amzn commented Apr 12, 2021

ohltyler commented Apr 12, 2021

ylwu-amzn commented Apr 12, 2021 •

edited

Loading

ohltyler commented Apr 12, 2021

yizheliu-amazon left a comment

ylwu-amzn left a comment

dblock Apr 14, 2021 •

edited

Loading

ohltyler Apr 14, 2021 •

edited

Loading

Migrate plugin to be compatible with OpenSearch Dashboards #1

Migrate plugin to be compatible with OpenSearch Dashboards #1

Conversation

ohltyler commented Apr 8, 2021 • edited Loading

dblock left a comment

Choose a reason for hiding this comment

dblock Apr 8, 2021

Choose a reason for hiding this comment

ohltyler Apr 8, 2021

Choose a reason for hiding this comment

ohltyler Apr 9, 2021 • edited Loading

Choose a reason for hiding this comment

dblock Apr 9, 2021

Choose a reason for hiding this comment

dblock Apr 9, 2021

Choose a reason for hiding this comment

ohltyler Apr 9, 2021

Choose a reason for hiding this comment

ylwu-amzn Apr 12, 2021 • edited Loading

Choose a reason for hiding this comment

ylwu-amzn Apr 12, 2021

Choose a reason for hiding this comment

ohltyler Apr 12, 2021

Choose a reason for hiding this comment

ylwu-amzn commented Apr 12, 2021

ohltyler commented Apr 12, 2021

ylwu-amzn commented Apr 12, 2021 • edited Loading

ohltyler commented Apr 12, 2021

yizheliu-amazon left a comment

Choose a reason for hiding this comment

ylwu-amzn left a comment

Choose a reason for hiding this comment

dblock Apr 14, 2021 • edited Loading

Choose a reason for hiding this comment

ohltyler Apr 14, 2021 • edited Loading

Choose a reason for hiding this comment

ohltyler commented Apr 8, 2021 •

edited

Loading

ohltyler Apr 9, 2021 •

edited

Loading

ylwu-amzn Apr 12, 2021 •

edited

Loading

ylwu-amzn commented Apr 12, 2021 •

edited

Loading

dblock Apr 14, 2021 •

edited

Loading

ohltyler Apr 14, 2021 •

edited

Loading