[Auto Suggest] OpenSearch SQL autosuggest with ANTLR #7336

mengweieric · 2024-07-20T22:06:41Z

Description

OpenSearch SQL auto suggest support for query editor.

Issues Resolved

Screenshot

Testing the changes

Changelog

Check List

All tests pass
- yarn test:jest
- yarn test:jest_integration
New functionality includes testing.
New functionality has been documented.
Update CHANGELOG.md
Commits are signed per the DCO using --signoff

Signed-off-by: Eric <menwe@amazon.com>

github-actions · 2024-07-20T22:07:13Z

ℹ️ Manual Changeset Creation Reminder

Please ensure manual commit for changeset file 7336.yml under folder changelogs/fragments to complete this PR.

If you want to use the available OpenSearch Changeset Bot App to avoid manual creation of changeset file you can install it in your forked repository following this link.

For more information about formatting of changeset files, please visit OpenSearch Auto Changeset and Release Notes Tool.

Signed-off-by: Eric <menwe@amazon.com>

codecov · 2024-07-20T22:36:37Z

Codecov Report

Attention: Patch coverage is 47.10983% with 183 lines in your changes missing coverage. Please review.

Project coverage is 63.95%. Comparing base (34f1b73) to head (3e5b430).
Report is 5 commits behind head on main.

Files	Patch %	Lines
...ntlr/opensearch_sql/opensearch_sql_autocomplete.ts	25.97%	56 Missing and 1 partial ⚠️
...ata/public/antlr/opensearch_sql/code_completion.ts	7.14%	52 Missing ⚠️
.../plugins/data/public/antlr/opensearch_sql/table.ts	52.94%	34 Missing and 6 partials ⚠️
src/plugins/data/public/antlr/shared/cursor.ts	41.02%	19 Missing and 4 partials ⚠️
src/plugins/data/public/antlr/shared/utils.ts	82.14%	1 Missing and 4 partials ⚠️
.../plugins/data/public/antlr/opensearch_sql/parse.ts	85.00%	3 Missing ⚠️
src/plugins/data/public/plugin.ts	0.00%	2 Missing ⚠️
...s/data/public/antlr/opensearch_sql/symbol_table.ts	95.65%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #7336      +/-   ##
==========================================
- Coverage   67.62%   63.95%   -3.67%     
==========================================
  Files        3583     3595      +12     
  Lines       72174    78211    +6037     
  Branches    11810    12327     +517     
==========================================
+ Hits        48805    50017    +1212     
- Misses      20385    25177    +4792     
- Partials     2984     3017      +33

Flag	Coverage Δ
Linux_1	`?`
Linux_2	`55.50% <ø> (ø)`
Linux_3	`41.31% <45.69%> (-3.20%)`	⬇️
Linux_4	`31.55% <12.13%> (-2.53%)`	⬇️
Windows_1	`31.08% <12.13%> (-2.10%)`	⬇️
Windows_2	`55.45% <ø> (ø)`
Windows_3	`41.33% <45.69%> (-3.20%)`	⬇️
Windows_4	`31.55% <12.13%> (-2.53%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Eric <menwe@amazon.com>

mengweieric · 2024-07-23T00:43:10Z

need to unblock @paulstn for DQL, will follow up on increasing test coverage.

RyanL1997 · 2024-07-23T01:28:47Z

src/plugins/data/public/antlr/shared/utils.ts

+) =>
+  connectionsService.getSelectedConnection$().pipe(
+    distinctUntilChanged(),
+    switchMap((connection) => {


Is this the place to get multi datasource id?

yes, and if it's undefined, the request will hit local cluster.

Got it. Just wanna make sure the underlying logic can be relied on as a source of truth for the flag of data_source.enabled, since my work changed the registration behavior of datasource management plugin. And I've seen that some of the plugin used the datasource management plugin status to determine if multi-datasource is enabled or not.

RyanL1997 · 2024-07-23T01:30:25Z

src/plugins/data/public/antlr/opensearch_sql/grammar/.antlr/OpenSearchSQLLexer.java

+import org.antlr.v4.runtime.misc.*;
+
+@SuppressWarnings({"all", "warnings", "unchecked", "unused", "cast", "CheckReturnValue", "this-escape"})
+public class OpenSearchSQLLexer extends Lexer {


For my knowledge, why we need this java file?

these are auto-generated files from ANTLR engine for query parsing internally.

RyanL1997 · 2024-07-23T01:32:26Z

package.json

@@ -84,7 +84,8 @@
    "cypress:run-without-security": "env TZ=America/Los_Angeles NO_COLOR=1 cypress run --headless --env SECURITY_ENABLED=false",
    "cypress:run-with-security": "env TZ=America/Los_Angeles NO_COLOR=1 cypress run --headless --env SECURITY_ENABLED=true,openSearchUrl=https://localhost:9200,WAIT_FOR_LOADER_BUFFER_MS=500",
    "osd:ciGroup10": "echo \"dashboard_sanity_test_spec.js\"",
-    "osd:ciGroup11": "echo \"apps/vis_builder/*.js\""
+    "osd:ciGroup11": "echo \"apps/vis_builder/*.js\"",
+    "generate:opensearchsqlantlr": "./node_modules/antlr4ng-cli/index.js -Dlanguage=TypeScript -o ./src/plugins/data/public/antlr/opensearch_sql/.generated -visitor -no-listener -Xexact-output-dir ./src/plugins/data/public/antlr/opensearch_sql/grammar/OpenSearchSQLLexer.g4 ./src/plugins/data/public/antlr/opensearch_sql/grammar/OpenSearchSQLParser.g4"


Are this manual or auto generated? If it is manual, we may need better mechanism in the future to embed this into CI

for now it's manually generated, previously we add this to post commit hook but since the build pipeline does not have java runtime, we switch to manual and only need to run this when there's grammar changes

RyanL1997 · 2024-07-23T01:35:01Z

src/plugins/data/public/antlr/opensearch_sql/grammar/OpenSearchSQLParser.g4

+THE SOFTWARE.
+*/
+
+parser grammar OpenSearchSQLParser;


where is the source of this grammar? Is this coming from the sql backend?

yes, it's from SQL plugin.

zhongnansu · 2024-07-23T02:12:33Z

need to unblock @paulstn for DQL, will follow up on increasing test coverage.

@mengweieric could you create a issue for tracking purpose?

mengweieric · 2024-07-23T02:37:41Z

need to unblock @paulstn for DQL, will follow up on increasing test coverage.

@mengweieric could you create a issue for tracking purpose?

sure. #7387

opensearch-trigger-bot · 2024-07-23T02:40:36Z

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch-Dashboards/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch-Dashboards/backport-2.x
# Create a new branch
git switch --create backport/backport-7336-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 9348bd4276a6daf4a719f3709c3d854e002350e0
# Push it to GitHub
git push --set-upstream origin backport/backport-7336-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch-Dashboards/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-7336-to-2.x.

opensearch-trigger-bot · 2024-07-23T02:40:37Z

The backport to 2.16 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch-Dashboards/backport-2.16 2.16
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch-Dashboards/backport-2.16
# Create a new branch
git switch --create backport/backport-7336-to-2.16
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 9348bd4276a6daf4a719f3709c3d854e002350e0
# Push it to GitHub
git push --set-upstream origin backport/backport-7336-to-2.16
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch-Dashboards/backport-2.16

Then, create a pull request where the base branch is 2.16 and the compare/head branch is backport/backport-7336-to-2.16.

abbyhu2000 · 2024-07-23T19:58:42Z

src/plugins/data/public/ui/filter_bar/_global_filter_group.scss

@@ -1,6 +1,7 @@
 // SASSTODO: Probably not the right file for this selector, but temporary until the files get re-organized
 .globalQueryBar {
  padding: 0 $euiSizeS $euiSizeS $euiSizeS;
+  height: 160px;


This line should not be added. This makes dashboard and visualize top nav bar being displayed incorrectly.

abbyhu2000 · 2024-07-23T20:06:10Z

src/plugins/data/public/ui/query_editor/language_switcher.tsx

Is this file needed? This is not used anywhere

abbyhu2000 · 2024-07-23T20:11:41Z

src/plugins/data/public/ui/query_editor/_language_switcher.scss

And this too, it is not used anywhere

…ect#7336) * cherry pick Signed-off-by: Eric <menwe@amazon.com> * SQL keyword suggestion Signed-off-by: Eric <menwe@amazon.com> * column and values suggestions Signed-off-by: Eric <menwe@amazon.com> * adjust values suggestions and ignored token Signed-off-by: Eric <menwe@amazon.com> * suggestion type casting Signed-off-by: Eric <menwe@amazon.com> * ignore functions Signed-off-by: Eric <menwe@amazon.com> * case insensitive Signed-off-by: Eric <menwe@amazon.com> * change to use grammar for insensitive case Signed-off-by: Eric <menwe@amazon.com> * fix multi-line issue Signed-off-by: Eric <menwe@amazon.com> * code cleanup Signed-off-by: Eric <menwe@amazon.com> * move ppl out of scope Signed-off-by: Eric <menwe@amazon.com> * rename folder and add ignoring rules Signed-off-by: Eric <menwe@amazon.com> * resolve type issue Signed-off-by: Eric <menwe@amazon.com> * fix ppl suggestion provider issue Signed-off-by: Eric <menwe@amazon.com> * remove function suggestion Signed-off-by: Eric <menwe@amazon.com> * some code clean up and adding tests Signed-off-by: Eric <menwe@amazon.com> * cursor tests Signed-off-by: Eric <menwe@amazon.com> * remove testing setup Signed-off-by: Eric <menwe@amazon.com> * add changelog Signed-off-by: Eric <menwe@amazon.com> * update yarn file Signed-off-by: Eric <menwe@amazon.com> * add missing testing library Signed-off-by: Eric <menwe@amazon.com> * MDS integration Signed-off-by: Eric <menwe@amazon.com> * minor interface change and disable word based suggestion Signed-off-by: Eric <menwe@amazon.com> * update ID_LITERAL and recompile grammar Signed-off-by: Eric <menwe@amazon.com> * column suggest for agg function Signed-off-by: Eric <menwe@amazon.com> * revert version Signed-off-by: Eric <menwe@amazon.com> * opensearch sql syntax highlighting Signed-off-by: Eric <menwe@amazon.com> * add utility tests Signed-off-by: Eric <menwe@amazon.com> * symbol table test Signed-off-by: Eric <menwe@amazon.com> * error listener test Signed-off-by: Eric <menwe@amazon.com> * parse test Signed-off-by: Eric <menwe@amazon.com> * add suggestion provider to single line query editor Signed-off-by: Eric <menwe@amazon.com> * remove one test Signed-off-by: Eric <menwe@amazon.com> * add connection service type Signed-off-by: Eric <menwe@amazon.com> --------- Signed-off-by: Eric <menwe@amazon.com> (cherry picked from commit 9348bd4)

* Bump OUI to 1.8.0 (#7363) Signed-off-by: Miki <miki@amazon.com> (cherry picked from commit c1bae75) * [Auto Suggest] OpenSearch SQL autosuggest with ANTLR (#7336) * cherry pick Signed-off-by: Eric <menwe@amazon.com> * SQL keyword suggestion Signed-off-by: Eric <menwe@amazon.com> * column and values suggestions Signed-off-by: Eric <menwe@amazon.com> * adjust values suggestions and ignored token Signed-off-by: Eric <menwe@amazon.com> * suggestion type casting Signed-off-by: Eric <menwe@amazon.com> * ignore functions Signed-off-by: Eric <menwe@amazon.com> * case insensitive Signed-off-by: Eric <menwe@amazon.com> * change to use grammar for insensitive case Signed-off-by: Eric <menwe@amazon.com> * fix multi-line issue Signed-off-by: Eric <menwe@amazon.com> * code cleanup Signed-off-by: Eric <menwe@amazon.com> * move ppl out of scope Signed-off-by: Eric <menwe@amazon.com> * rename folder and add ignoring rules Signed-off-by: Eric <menwe@amazon.com> * resolve type issue Signed-off-by: Eric <menwe@amazon.com> * fix ppl suggestion provider issue Signed-off-by: Eric <menwe@amazon.com> * remove function suggestion Signed-off-by: Eric <menwe@amazon.com> * some code clean up and adding tests Signed-off-by: Eric <menwe@amazon.com> * cursor tests Signed-off-by: Eric <menwe@amazon.com> * remove testing setup Signed-off-by: Eric <menwe@amazon.com> * add changelog Signed-off-by: Eric <menwe@amazon.com> * update yarn file Signed-off-by: Eric <menwe@amazon.com> * add missing testing library Signed-off-by: Eric <menwe@amazon.com> * MDS integration Signed-off-by: Eric <menwe@amazon.com> * minor interface change and disable word based suggestion Signed-off-by: Eric <menwe@amazon.com> * update ID_LITERAL and recompile grammar Signed-off-by: Eric <menwe@amazon.com> * column suggest for agg function Signed-off-by: Eric <menwe@amazon.com> * revert version Signed-off-by: Eric <menwe@amazon.com> * opensearch sql syntax highlighting Signed-off-by: Eric <menwe@amazon.com> * add utility tests Signed-off-by: Eric <menwe@amazon.com> * symbol table test Signed-off-by: Eric <menwe@amazon.com> * error listener test Signed-off-by: Eric <menwe@amazon.com> * parse test Signed-off-by: Eric <menwe@amazon.com> * add suggestion provider to single line query editor Signed-off-by: Eric <menwe@amazon.com> * remove one test Signed-off-by: Eric <menwe@amazon.com> * add connection service type Signed-off-by: Eric <menwe@amazon.com> --------- Signed-off-by: Eric <menwe@amazon.com> (cherry picked from commit 9348bd4) * Move TopNavLinks to new Nav Bar Discover (#7326) This PR aims to introduce a new Nav bar in Discover that would contain the Date Picker as well as the Navigation links that are currently part of headers. The Navigation Links have been replaced with the corresponding Icons to provide a modern UX. These setting are currently controlled by query:enhancements:enabled Advanced Setting Flag and would be visible when the above flag is turned on. Signed-off-by: Suchit Sahoo <suchsah@amazon.com> (cherry picked from commit e46e54a) * [navigation-next]fix: breadcrumb issue found in data source management and other similar cases. (#7401) * feat: add scoped breadcrumbs Signed-off-by: SuZhou-Joe <suzhou@amazon.com> * Changeset file for PR #7401 created/updated * feat: move data source management register logic up a little bit Signed-off-by: SuZhou-Joe <suzhou@amazon.com> * feat: add unit test Signed-off-by: SuZhou-Joe <suzhou@amazon.com> * feat: revert the home related change Signed-off-by: SuZhou-Joe <suzhou@amazon.com> * fix: overview error in all use case when workspace is enabled Signed-off-by: SuZhou-Joe <suzhou@amazon.com> * fix: update snapshot Signed-off-by: SuZhou-Joe <suzhou@amazon.com> * Changeset file for PR #7401 created/updated * fix: hide nav groups that should be displayed Signed-off-by: SuZhou-Joe <suzhou@amazon.com> * feat: hide expand icon in left navigation Signed-off-by: SuZhou-Joe <suzhou@amazon.com> * feat: update Signed-off-by: SuZhou-Joe <suzhou@amazon.com> --------- Signed-off-by: SuZhou-Joe <suzhou@amazon.com> Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com> Co-authored-by: ZilongX <99905560+ZilongX@users.noreply.github.com> (cherry picked from commit 3f78c94) * [Bug][Data Source] Move data source manageable feature flag to DSM plugin (#7440) * Move data source manageable feature flag to DSM plugin Signed-off-by: yubonluo <yubonluo@amazon.com> * Changeset file for PR #7440 created/updated --------- Signed-off-by: yubonluo <yubonluo@amazon.com> Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com> (cherry picked from commit dfcd2e1) * [Discover-next] data set picker (#7426) * update using query manager Signed-off-by: Kawika Avilla <kavilla414@gmail.com> * aggs not working Signed-off-by: Kawika Avilla <kavilla414@gmail.com> * almost there Signed-off-by: Kawika Avilla <kavilla414@gmail.com> * stablish Signed-off-by: Kawika Avilla <kavilla414@gmail.com> * thanks ashwin Signed-off-by: Kawika Avilla <kavilla414@gmail.com> * update ref name Signed-off-by: Kawika Avilla <kavilla414@gmail.com> * fix timefields Signed-off-by: Kawika Avilla <kavilla414@gmail.com> * clean up some console logs Signed-off-by: Kawika Avilla <kavilla414@gmail.com> * go safer route of setting language Signed-off-by: Kawika Avilla <kavilla414@gmail.com> * its working again Signed-off-by: Kawika Avilla <kavilla414@gmail.com> * fix names Signed-off-by: Kawika Avilla <kavilla414@gmail.com> * restore code editor and indices Signed-off-by: Kawika Avilla <kavilla414@gmail.com> * sql df Signed-off-by: Kawika Avilla <kavilla414@gmail.com> * fix external datasources again Signed-off-by: Kawika Avilla <kavilla414@gmail.com> --------- Signed-off-by: Kawika Avilla <kavilla414@gmail.com> (cherry picked from commit 6a079d3) * Fix add filter for numeric scripted field (#7022) * Fix add filter for scripted field Signed-off-by: abbyhu2000 <abigailhu2000@gmail.com> * add unit test Signed-off-by: abbyhu2000 <abigailhu2000@gmail.com> * change from nested ternary to if else Signed-off-by: abbyhu2000 <abigailhu2000@gmail.com> * add more unit tests Signed-off-by: abbyhu2000 <abigailhu2000@gmail.com> * change unit tests Signed-off-by: abbyhu2000 <abigailhu2000@gmail.com> * fix typo Signed-off-by: abbyhu2000 <abigailhu2000@gmail.com> --------- Signed-off-by: abbyhu2000 <abigailhu2000@gmail.com> (cherry picked from commit 7c1f8fc) * [Bug][Workspace] Add permission validation at workspace detail page (#7435) * [Bug][Workspace] Add permission validate at workspace detail page Signed-off-by: yubonluo <yubonluo@amazon.com> * Changeset file for PR #7435 created/updated --------- Signed-off-by: yubonluo <yubonluo@amazon.com> Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com> (cherry picked from commit 45d34b5) * Revert "[Discover-next] data set picker (#7426)" (#7479) This reverts commit 6a079d3. --------- Co-authored-by: Miki <miki@amazon.com> Co-authored-by: Eric Wei <menwe@amazon.com> Co-authored-by: SuZhou-Joe <suzhou@amazon.com> Co-authored-by: yuboluo <yubonluo@amazon.com> Co-authored-by: Kawika Avilla <kavilla414@gmail.com> Co-authored-by: Qingyang(Abby) Hu <abigailhu2000@gmail.com> Co-authored-by: Sean Li <lnse@amazon.com>

* Bump OUI to 1.8.0 (#7363) Signed-off-by: Miki <miki@amazon.com> (cherry picked from commit c1bae75) * [Auto Suggest] OpenSearch SQL autosuggest with ANTLR (#7336) * cherry pick Signed-off-by: Eric <menwe@amazon.com> * SQL keyword suggestion Signed-off-by: Eric <menwe@amazon.com> * column and values suggestions Signed-off-by: Eric <menwe@amazon.com> * adjust values suggestions and ignored token Signed-off-by: Eric <menwe@amazon.com> * suggestion type casting Signed-off-by: Eric <menwe@amazon.com> * ignore functions Signed-off-by: Eric <menwe@amazon.com> * case insensitive Signed-off-by: Eric <menwe@amazon.com> * change to use grammar for insensitive case Signed-off-by: Eric <menwe@amazon.com> * fix multi-line issue Signed-off-by: Eric <menwe@amazon.com> * code cleanup Signed-off-by: Eric <menwe@amazon.com> * move ppl out of scope Signed-off-by: Eric <menwe@amazon.com> * rename folder and add ignoring rules Signed-off-by: Eric <menwe@amazon.com> * resolve type issue Signed-off-by: Eric <menwe@amazon.com> * fix ppl suggestion provider issue Signed-off-by: Eric <menwe@amazon.com> * remove function suggestion Signed-off-by: Eric <menwe@amazon.com> * some code clean up and adding tests Signed-off-by: Eric <menwe@amazon.com> * cursor tests Signed-off-by: Eric <menwe@amazon.com> * remove testing setup Signed-off-by: Eric <menwe@amazon.com> * add changelog Signed-off-by: Eric <menwe@amazon.com> * update yarn file Signed-off-by: Eric <menwe@amazon.com> * add missing testing library Signed-off-by: Eric <menwe@amazon.com> * MDS integration Signed-off-by: Eric <menwe@amazon.com> * minor interface change and disable word based suggestion Signed-off-by: Eric <menwe@amazon.com> * update ID_LITERAL and recompile grammar Signed-off-by: Eric <menwe@amazon.com> * column suggest for agg function Signed-off-by: Eric <menwe@amazon.com> * revert version Signed-off-by: Eric <menwe@amazon.com> * opensearch sql syntax highlighting Signed-off-by: Eric <menwe@amazon.com> * add utility tests Signed-off-by: Eric <menwe@amazon.com> * symbol table test Signed-off-by: Eric <menwe@amazon.com> * error listener test Signed-off-by: Eric <menwe@amazon.com> * parse test Signed-off-by: Eric <menwe@amazon.com> * add suggestion provider to single line query editor Signed-off-by: Eric <menwe@amazon.com> * remove one test Signed-off-by: Eric <menwe@amazon.com> * add connection service type Signed-off-by: Eric <menwe@amazon.com> --------- Signed-off-by: Eric <menwe@amazon.com> (cherry picked from commit 9348bd4) * Move TopNavLinks to new Nav Bar Discover (#7326) This PR aims to introduce a new Nav bar in Discover that would contain the Date Picker as well as the Navigation links that are currently part of headers. The Navigation Links have been replaced with the corresponding Icons to provide a modern UX. These setting are currently controlled by query:enhancements:enabled Advanced Setting Flag and would be visible when the above flag is turned on. Signed-off-by: Suchit Sahoo <suchsah@amazon.com> (cherry picked from commit e46e54a) * [navigation-next]fix: breadcrumb issue found in data source management and other similar cases. (#7401) * feat: add scoped breadcrumbs Signed-off-by: SuZhou-Joe <suzhou@amazon.com> * Changeset file for PR #7401 created/updated * feat: move data source management register logic up a little bit Signed-off-by: SuZhou-Joe <suzhou@amazon.com> * feat: add unit test Signed-off-by: SuZhou-Joe <suzhou@amazon.com> * feat: revert the home related change Signed-off-by: SuZhou-Joe <suzhou@amazon.com> * fix: overview error in all use case when workspace is enabled Signed-off-by: SuZhou-Joe <suzhou@amazon.com> * fix: update snapshot Signed-off-by: SuZhou-Joe <suzhou@amazon.com> * Changeset file for PR #7401 created/updated * fix: hide nav groups that should be displayed Signed-off-by: SuZhou-Joe <suzhou@amazon.com> * feat: hide expand icon in left navigation Signed-off-by: SuZhou-Joe <suzhou@amazon.com> * feat: update Signed-off-by: SuZhou-Joe <suzhou@amazon.com> --------- Signed-off-by: SuZhou-Joe <suzhou@amazon.com> Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com> Co-authored-by: ZilongX <99905560+ZilongX@users.noreply.github.com> (cherry picked from commit 3f78c94) * [Bug][Data Source] Move data source manageable feature flag to DSM plugin (#7440) * Move data source manageable feature flag to DSM plugin Signed-off-by: yubonluo <yubonluo@amazon.com> * Changeset file for PR #7440 created/updated --------- Signed-off-by: yubonluo <yubonluo@amazon.com> Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com> (cherry picked from commit dfcd2e1) * [Discover-next] data set picker (#7426) * update using query manager Signed-off-by: Kawika Avilla <kavilla414@gmail.com> * aggs not working Signed-off-by: Kawika Avilla <kavilla414@gmail.com> * almost there Signed-off-by: Kawika Avilla <kavilla414@gmail.com> * stablish Signed-off-by: Kawika Avilla <kavilla414@gmail.com> * thanks ashwin Signed-off-by: Kawika Avilla <kavilla414@gmail.com> * update ref name Signed-off-by: Kawika Avilla <kavilla414@gmail.com> * fix timefields Signed-off-by: Kawika Avilla <kavilla414@gmail.com> * clean up some console logs Signed-off-by: Kawika Avilla <kavilla414@gmail.com> * go safer route of setting language Signed-off-by: Kawika Avilla <kavilla414@gmail.com> * its working again Signed-off-by: Kawika Avilla <kavilla414@gmail.com> * fix names Signed-off-by: Kawika Avilla <kavilla414@gmail.com> * restore code editor and indices Signed-off-by: Kawika Avilla <kavilla414@gmail.com> * sql df Signed-off-by: Kawika Avilla <kavilla414@gmail.com> * fix external datasources again Signed-off-by: Kawika Avilla <kavilla414@gmail.com> --------- Signed-off-by: Kawika Avilla <kavilla414@gmail.com> (cherry picked from commit 6a079d3) * Fix add filter for numeric scripted field (#7022) * Fix add filter for scripted field Signed-off-by: abbyhu2000 <abigailhu2000@gmail.com> * add unit test Signed-off-by: abbyhu2000 <abigailhu2000@gmail.com> * change from nested ternary to if else Signed-off-by: abbyhu2000 <abigailhu2000@gmail.com> * add more unit tests Signed-off-by: abbyhu2000 <abigailhu2000@gmail.com> * change unit tests Signed-off-by: abbyhu2000 <abigailhu2000@gmail.com> * fix typo Signed-off-by: abbyhu2000 <abigailhu2000@gmail.com> --------- Signed-off-by: abbyhu2000 <abigailhu2000@gmail.com> (cherry picked from commit 7c1f8fc) * [Bug][Workspace] Add permission validation at workspace detail page (#7435) * [Bug][Workspace] Add permission validate at workspace detail page Signed-off-by: yubonluo <yubonluo@amazon.com> * Changeset file for PR #7435 created/updated --------- Signed-off-by: yubonluo <yubonluo@amazon.com> Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com> (cherry picked from commit 45d34b5) * Revert "[Discover-next] data set picker (#7426)" (#7479) This reverts commit 6a079d3. --------- Co-authored-by: Miki <miki@amazon.com> Co-authored-by: Eric Wei <menwe@amazon.com> Co-authored-by: SuZhou-Joe <suzhou@amazon.com> Co-authored-by: yuboluo <yubonluo@amazon.com> Co-authored-by: Kawika Avilla <kavilla414@gmail.com> Co-authored-by: Qingyang(Abby) Hu <abigailhu2000@gmail.com> Co-authored-by: Sean Li <lnse@amazon.com> (cherry picked from commit 0515fb2) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

opensearch-trigger-bot · 2024-07-25T20:48:39Z

The backport to 2.16 failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch-Dashboards/backport-2.16 2.16
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch-Dashboards/backport-2.16
# Create a new branch
git switch --create backport/backport-7336-to-2.16
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 9348bd4276a6daf4a719f3709c3d854e002350e0
# Push it to GitHub
git push --set-upstream origin backport/backport-7336-to-2.16
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch-Dashboards/backport-2.16

Then, create a pull request where the base branch is 2.16 and the compare/head branch is backport/backport-7336-to-2.16.

* Bump OUI to 1.8.0 (#7363) (cherry picked from commit c1bae75) * [Auto Suggest] OpenSearch SQL autosuggest with ANTLR (#7336) * cherry pick * SQL keyword suggestion * column and values suggestions * adjust values suggestions and ignored token * suggestion type casting * ignore functions * case insensitive * change to use grammar for insensitive case * fix multi-line issue * code cleanup * move ppl out of scope * rename folder and add ignoring rules * resolve type issue * fix ppl suggestion provider issue * remove function suggestion * some code clean up and adding tests * cursor tests * remove testing setup * add changelog * update yarn file * add missing testing library * MDS integration * minor interface change and disable word based suggestion * update ID_LITERAL and recompile grammar * column suggest for agg function * revert version * opensearch sql syntax highlighting * add utility tests * symbol table test * error listener test * parse test * add suggestion provider to single line query editor * remove one test * add connection service type --------- (cherry picked from commit 9348bd4) * Move TopNavLinks to new Nav Bar Discover (#7326) This PR aims to introduce a new Nav bar in Discover that would contain the Date Picker as well as the Navigation links that are currently part of headers. The Navigation Links have been replaced with the corresponding Icons to provide a modern UX. These setting are currently controlled by query:enhancements:enabled Advanced Setting Flag and would be visible when the above flag is turned on. (cherry picked from commit e46e54a) * [navigation-next]fix: breadcrumb issue found in data source management and other similar cases. (#7401) * feat: add scoped breadcrumbs * Changeset file for PR #7401 created/updated * feat: move data source management register logic up a little bit * feat: add unit test * feat: revert the home related change * fix: overview error in all use case when workspace is enabled * fix: update snapshot * Changeset file for PR #7401 created/updated * fix: hide nav groups that should be displayed * feat: hide expand icon in left navigation * feat: update --------- (cherry picked from commit 3f78c94) * [Bug][Data Source] Move data source manageable feature flag to DSM plugin (#7440) * Move data source manageable feature flag to DSM plugin * Changeset file for PR #7440 created/updated --------- (cherry picked from commit dfcd2e1) * [Discover-next] data set picker (#7426) * update using query manager * aggs not working * almost there * stablish * thanks ashwin * update ref name * fix timefields * clean up some console logs * go safer route of setting language * its working again * fix names * restore code editor and indices * sql df * fix external datasources again --------- (cherry picked from commit 6a079d3) * Fix add filter for numeric scripted field (#7022) * Fix add filter for scripted field * add unit test * change from nested ternary to if else * add more unit tests * change unit tests * fix typo --------- (cherry picked from commit 7c1f8fc) * [Bug][Workspace] Add permission validation at workspace detail page (#7435) * [Bug][Workspace] Add permission validate at workspace detail page * Changeset file for PR #7435 created/updated --------- (cherry picked from commit 45d34b5) * Revert "[Discover-next] data set picker (#7426)" (#7479) This reverts commit 6a079d3. --------- (cherry picked from commit 0515fb2) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Miki <miki@amazon.com> Co-authored-by: Eric Wei <menwe@amazon.com> Co-authored-by: SuZhou-Joe <suzhou@amazon.com> Co-authored-by: yuboluo <yubonluo@amazon.com> Co-authored-by: Kawika Avilla <kavilla414@gmail.com> Co-authored-by: Qingyang(Abby) Hu <abigailhu2000@gmail.com> Co-authored-by: Sean Li <lnse@amazon.com>

AMoo-Miki · 2024-09-13T15:51:55Z

This was manually backported to 2.x with #7477 and to 2.16 with #7487.

mengweieric added 19 commits July 17, 2024 18:17

cherry pick

df0a054

Signed-off-by: Eric <menwe@amazon.com>

SQL keyword suggestion

e5175af

Signed-off-by: Eric <menwe@amazon.com>

column and values suggestions

d5daf39

Signed-off-by: Eric <menwe@amazon.com>

adjust values suggestions and ignored token

f0de90e

Signed-off-by: Eric <menwe@amazon.com>

suggestion type casting

be42ea2

Signed-off-by: Eric <menwe@amazon.com>

ignore functions

f215024

Signed-off-by: Eric <menwe@amazon.com>

case insensitive

2a452e9

Signed-off-by: Eric <menwe@amazon.com>

change to use grammar for insensitive case

7ae59f4

Signed-off-by: Eric <menwe@amazon.com>

fix multi-line issue

64c266d

Signed-off-by: Eric <menwe@amazon.com>

code cleanup

7c62bf7

Signed-off-by: Eric <menwe@amazon.com>

move ppl out of scope

c9da113

Signed-off-by: Eric <menwe@amazon.com>

rename folder and add ignoring rules

140114a

Signed-off-by: Eric <menwe@amazon.com>

resolve type issue

f1014b7

Signed-off-by: Eric <menwe@amazon.com>

fix ppl suggestion provider issue

152a645

Signed-off-by: Eric <menwe@amazon.com>

remove function suggestion

79e7782

Signed-off-by: Eric <menwe@amazon.com>

some code clean up and adding tests

4d84d3b

Signed-off-by: Eric <menwe@amazon.com>

sync main and resolve conflicts

609a2a9

Signed-off-by: Eric <menwe@amazon.com>

cursor tests

ce01e2c

Signed-off-by: Eric <menwe@amazon.com>

remove testing setup

d7e3bd4

Signed-off-by: Eric <menwe@amazon.com>

github-actions bot added the valued-contributor label Jul 20, 2024

github-actions bot added the failed changeset label Jul 20, 2024

mengweieric added 2 commits July 20, 2024 22:21

add changelog

3aa0dff

Signed-off-by: Eric <menwe@amazon.com>

update yarn file

d1e2540

Signed-off-by: Eric <menwe@amazon.com>

mengweieric added 5 commits July 21, 2024 00:05

add missing testing library

a5d12e5

Signed-off-by: Eric <menwe@amazon.com>

MDS integration

a606aa8

Signed-off-by: Eric <menwe@amazon.com>

Merge branch 'main' into feature/antlr-autocomplete

12715a2

minor interface change and disable word based suggestion

4a8300b

Signed-off-by: Eric <menwe@amazon.com>

update ID_LITERAL and recompile grammar

9cb77b8

Signed-off-by: Eric <menwe@amazon.com>

mengweieric added 5 commits July 22, 2024 22:21

parse test

9009922

Signed-off-by: Eric <menwe@amazon.com>

add suggestion provider to single line query editor

f28f558

Signed-off-by: Eric <menwe@amazon.com>

remove one test

1f9080b

Signed-off-by: Eric <menwe@amazon.com>

add connection service type

2fa1dfb

Signed-off-by: Eric <menwe@amazon.com>

Merge branch 'main' into feature/antlr-autocomplete

3e5b430

joshuali925 approved these changes Jul 23, 2024

View reviewed changes

RyanL1997 approved these changes Jul 23, 2024

View reviewed changes

BionIT approved these changes Jul 23, 2024

View reviewed changes

RyanL1997 approved these changes Jul 23, 2024

View reviewed changes

zhongnansu approved these changes Jul 23, 2024

View reviewed changes

mengweieric added the backport 2.16 label Jul 23, 2024

mengweieric merged commit 9348bd4 into opensearch-project:main Jul 23, 2024
65 of 69 checks passed

opensearch-trigger-bot bot added the failed backport label Jul 23, 2024

abbyhu2000 reviewed Jul 23, 2024

View reviewed changes

src/plugins/data/public/ui/query_editor/language_switcher.tsx

Copy link

Member

abbyhu2000 Jul 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this file needed? This is not used anywhere

abbyhu2000 reviewed Jul 23, 2024

View reviewed changes

src/plugins/data/public/ui/query_editor/_language_switcher.scss

Copy link

Member

abbyhu2000 Jul 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And this too, it is not used anywhere

mengweieric added backport 2.16 and removed backport 2.16 labels Jul 25, 2024

AMoo-Miki removed the failed backport label Sep 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Auto Suggest] OpenSearch SQL autosuggest with ANTLR #7336

[Auto Suggest] OpenSearch SQL autosuggest with ANTLR #7336

mengweieric commented Jul 20, 2024 •

edited

Loading

github-actions bot commented Jul 20, 2024

codecov bot commented Jul 20, 2024 •

edited

Loading

mengweieric commented Jul 23, 2024

RyanL1997 Jul 23, 2024

mengweieric Jul 23, 2024

RyanL1997 Jul 23, 2024

RyanL1997 Jul 23, 2024

mengweieric Jul 23, 2024

RyanL1997 Jul 23, 2024

mengweieric Jul 23, 2024

RyanL1997 Jul 23, 2024

mengweieric Jul 23, 2024

zhongnansu commented Jul 23, 2024

mengweieric commented Jul 23, 2024

opensearch-trigger-bot bot commented Jul 23, 2024

opensearch-trigger-bot bot commented Jul 23, 2024

abbyhu2000 Jul 23, 2024

abbyhu2000 Jul 23, 2024

abbyhu2000 Jul 23, 2024

opensearch-trigger-bot bot commented Jul 25, 2024

AMoo-Miki commented Sep 13, 2024

[Auto Suggest] OpenSearch SQL autosuggest with ANTLR #7336

[Auto Suggest] OpenSearch SQL autosuggest with ANTLR #7336

Conversation

mengweieric commented Jul 20, 2024 • edited Loading

Description

Issues Resolved

Screenshot

Testing the changes

Changelog

Check List

github-actions bot commented Jul 20, 2024

ℹ️ Manual Changeset Creation Reminder

codecov bot commented Jul 20, 2024 • edited Loading

Codecov Report

mengweieric commented Jul 23, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zhongnansu commented Jul 23, 2024

mengweieric commented Jul 23, 2024

opensearch-trigger-bot bot commented Jul 23, 2024

opensearch-trigger-bot bot commented Jul 23, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

opensearch-trigger-bot bot commented Jul 25, 2024

AMoo-Miki commented Sep 13, 2024

mengweieric commented Jul 20, 2024 •

edited

Loading

codecov bot commented Jul 20, 2024 •

edited

Loading