[Auto Suggest] DQL Updates #7593

paulstn · 2024-07-31T00:08:03Z

Description

Update to DQL Autocomplete: #7391

Every file removal under grammar/.antlr can be ignored.

Issues Resolved

Screenshot

Testing the changes

Changelog

fix: Update DQL Autocomplete in code and functionality

Check List

All tests pass
- yarn test:jest
- yarn test:jest_integration
New functionality includes testing.
New functionality has been documented.
Update CHANGELOG.md
Commits are signed per the DCO using --signoff

Signed-off-by: Paul Sebastian <paulstn@amazon.com>

github-actions · 2024-07-31T00:08:43Z

❌ Empty Changelog Section

The Changelog section in your PR description is empty. Please add a valid changelog entry or entries. If you did add a changelog entry, check to make sure that it was not accidentally included inside the comment block in the Changelog section.

codecov · 2024-07-31T00:22:30Z

Codecov Report

Attention: Patch coverage is 78.68852% with 13 lines in your changes missing coverage. Please review.

Project coverage is 64.19%. Comparing base (1976ecf) to head (0c2888a).
Report is 3 commits behind head on main.

Files	Patch %	Lines
...c/plugins/data/public/antlr/dql/code_completion.ts	81.35%	0 Missing and 11 partials ⚠️
...s/data/public/autocomplete/autocomplete_service.ts	0.00%	1 Missing ⚠️
...ashboards_react/public/code_editor/code_editor.tsx	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #7593      +/-   ##
==========================================
+ Coverage   64.17%   64.19%   +0.01%     
==========================================
  Files        3659     3658       -1     
  Lines       80796    80814      +18     
  Branches    12866    12874       +8     
==========================================
+ Hits        51848    51875      +27     
+ Misses      25769    25757      -12     
- Partials     3179     3182       +3

Flag	Coverage Δ
Linux_1	`30.28% <0.00%> (-0.02%)`	⬇️
Linux_2	`55.89% <ø> (ø)`
Linux_3	`40.70% <80.00%> (+0.03%)`	⬆️
Linux_4	`31.44% <0.00%> (-0.02%)`	⬇️
Windows_1	`30.30% <0.00%> (-0.02%)`	⬇️
Windows_2	`55.85% <ø> (ø)`
Windows_3	`40.71% <80.00%> (+0.04%)`	⬆️
Windows_4	`31.44% <0.00%> (-0.02%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

github-actions · 2024-07-31T17:53:24Z

❌ Empty Changelog Section

The Changelog section in your PR description is empty. Please add a valid changelog entry or entries. If you did add a changelog entry, check to make sure that it was not accidentally included inside the comment block in the Changelog section.

github-actions · 2024-07-31T21:03:24Z

❌ Empty Changelog Section

The Changelog section in your PR description is empty. Please add a valid changelog entry or entries. If you did add a changelog entry, check to make sure that it was not accidentally included inside the comment block in the Changelog section.

github-actions · 2024-07-31T21:03:54Z

❌ Empty Changelog Section

The Changelog section in your PR description is empty. Please add a valid changelog entry or entries. If you did add a changelog entry, check to make sure that it was not accidentally included inside the comment block in the Changelog section.

Signed-off-by: Paul Sebastian <paulstn@amazon.com>

…groupings Signed-off-by: Paul Sebastian <paulstn@amazon.com>

Signed-off-by: Paul Sebastian <paulstn@amazon.com>

…cases Signed-off-by: Paul Sebastian <paulstn@amazon.com>

paulstn · 2024-08-26T22:12:32Z

src/plugins/data/public/antlr/dql/code_completion.ts

    return {
      text: field,
      type: monaco.languages.CompletionItemKind.Field,
      insertText: `${field}: `,
+      start: -1,


These start/end parameters will be handled in an upcoming pr

joshuali925 · 2024-08-26T22:18:33Z

src/plugins/data/public/antlr/shared/constants.ts

+  { text: '_index', type: 3, insertText: '_index: ', end: -1, start: -1 },
+  { text: '_score', type: 3, insertText: '_score: ', end: -1, start: -1 },
+  { text: '_source', type: 3, insertText: '_source: ', end: -1, start: -1 },
+  { text: '_type', type: 3, insertText: '_type: ', end: -1, start: -1 },


should constants used for test be in a *.test.ts file?

My assumption was that since the file didn't contain any tests, it shouldn't have that extension. I couldn't find any precedence for constants being used exclusively for tests, do you know if there is another way of handling these?

ah @joshuali925 similar comment https://github.com/opensearch-project/OpenSearch-Dashboards/pull/7593/files#r1732297731

you can use mock or just include this in the test.

exporting public fieldNameSuggestions could get confusing for other plugins since now any plugin can access it and this will get compiled down with the release distribution.

that makes sense, i'll move everything here into those tests

kavilla · 2024-08-27T07:28:35Z

seeing some failures in the ci.

also qq, i know @mengweieric was working on a feature branch for his updates. does these updates have those and account for changes related to that work. since it has clean ups?

kavilla · 2024-08-27T07:32:25Z

src/plugins/data/public/antlr/shared/constants.ts

-  { text: '_score', type: 3, insertText: '_score: ' },
-  { text: '_source', type: 3, insertText: '_source: ' },
-  { text: '_type', type: 3, insertText: '_type: ' },
+export const fieldNameSuggestions: Array<{


we shouldn't store test values in constants file. constants will get bundled up and will increase the release distribution size. test files do not get compiled with the distribution.

out of curiosity, could you point me to where its specified which files don't get compiled in the distribution? would it just be .test.ts files, or are there other files that wouldn't be included?

there are some hard coded ignores in the repo, like

OpenSearch-Dashboards/packages/osd-pm/src/utils/targeted_build.ts

Lines 58 to 59 in 25c23db

'--ignore',

'**/*.test.ts,**/*.test.tsx',

kavilla · 2024-08-27T07:35:38Z

src/plugins/data/public/autocomplete/autocomplete_service.ts

@@ -63,7 +61,7 @@ export class AutocompleteService {
    const provider = this.querySuggestionProviders.get(language);

    if (provider) {
-      return provider({ core: this.core, ...args });
+      return provider(args);


is this a breaking change?

no, i'm reverting a change that i made previously with dql autocomplete. this was before i knew about a clear way to get to the http core util. but now since we're using services, its redundant and so i put it back the way it was.

Signed-off-by: Paul Sebastian <paulstn@amazon.com>

paulstn · 2024-08-27T16:18:28Z

also qq, i know @mengweieric was working on a feature branch for his updates. does these updates have those and account for changes related to that work. since it has clean ups?

The updates here and the interface changes Eric made in the feature branch don't conflict with each other, since I've included the necessary changes to DQL Autocomplete that handle those changes within that feature branch.

Signed-off-by: Paul Sebastian <paulstn@amazon.com>

paulstn · 2024-08-27T17:03:22Z

seeing some failures in the ci.

The cypress failures seem to be coming and going for different tests with each trivial, unrelated commit. I'm not sure what I should be doing here other than retry until all the stars align.

kavilla · 2024-08-27T22:47:41Z

src/plugins/data/public/antlr/dql/code_completion.ts

-    body: JSON.stringify({ query: currentValue, field: fieldName, boolFilter: [] }),
-  });
-};
-
 const findValueSuggestions = async (
  index: IndexPattern,


is this the indexpattern service or just object. because actually can just take the query object which now has the dataset object which has all the information you will need.

This is the actual object, I believe its set by the query_editor.tsx function which comes from the dataset object.

Update to DQL Autocomplete: #7391 Every file removal under `grammar/.antlr` can be ignored. - [x] use official value suggestion methods instead of direct api call - [x] removed language specified configuration - [ ] ~~added memoization for value suggestion to reduce number of calls made~~ - [x] removed core start from query suggestion function - [x] added tests for value suggestion - [x] added more test coverage for other general cases - [ ] ~~[[RFC] Monaco Code Editor provider registration #7594](#7594) made changes based on this RFC~~ - [x] remove grammar/.antlr auto generated files - [x] updated types in code completion and related files - [x] fixed many group value suggestion bugs and edge cases with more robust parser visitor - [x] fix group value NOT suggestion bugs - [ ] ~~added basic keyword syntax highlighting~~ --------- Signed-off-by: Paul Sebastian <paulstn@amazon.com> Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com> (cherry picked from commit 741c0d6) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

Update to DQL Autocomplete: #7391 Every file removal under `grammar/.antlr` can be ignored. - [x] use official value suggestion methods instead of direct api call - [x] removed language specified configuration - [ ] ~~added memoization for value suggestion to reduce number of calls made~~ - [x] removed core start from query suggestion function - [x] added tests for value suggestion - [x] added more test coverage for other general cases - [ ] ~~[[RFC] Monaco Code Editor provider registration #7594](#7594) made changes based on this RFC~~ - [x] remove grammar/.antlr auto generated files - [x] updated types in code completion and related files - [x] fixed many group value suggestion bugs and edge cases with more robust parser visitor - [x] fix group value NOT suggestion bugs - [ ] ~~added basic keyword syntax highlighting~~ --------- (cherry picked from commit 741c0d6) Signed-off-by: Paul Sebastian <paulstn@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com>

remove grammar/.antlr auto generated files

85bcf71

Signed-off-by: Paul Sebastian <paulstn@amazon.com>

github-actions bot added the valued-contributor label Jul 31, 2024

github-actions bot added the failed changeset label Jul 31, 2024

github-actions bot added Skip-Changelog PRs that are too trivial to warrant a changelog or release notes entry and removed failed changeset labels Jul 31, 2024

paulstn added 13 commits July 31, 2024 14:17

update types

a6b6aab

Signed-off-by: Paul Sebastian <paulstn@amazon.com>

use value suggestion provider through utils

3e4e40a

Signed-off-by: Paul Sebastian <paulstn@amazon.com>

remove core from query suggestion function

593146f

Signed-off-by: Paul Sebastian <paulstn@amazon.com>

basic syntax highlighting

ab548b7

Signed-off-by: Paul Sebastian <paulstn@amazon.com>

remove dql monarch syntax highlighting

24fdc23

Signed-off-by: Paul Sebastian <paulstn@amazon.com>

update code editor links

6c0064b

Signed-off-by: Paul Sebastian <paulstn@amazon.com>

remove language specified configuration change

a76483f

Signed-off-by: Paul Sebastian <paulstn@amazon.com>

update some types

0a47e71

Signed-off-by: Paul Sebastian <paulstn@amazon.com>

add value suggestion tests with mocked values

51787d8

Signed-off-by: Paul Sebastian <paulstn@amazon.com>

fix issue where NOT and VALUES do not appear consistent within value …

b9e218b

…groupings Signed-off-by: Paul Sebastian <paulstn@amazon.com>

specify exception for quotes to avoid suggesting NOT

8e143d3

Signed-off-by: Paul Sebastian <paulstn@amazon.com>

expand code completion previous field value visitor to work for edge …

7a3086c

…cases Signed-off-by: Paul Sebastian <paulstn@amazon.com>

Merge branch 'main' into dql-autosuggest-updates

84c8fd6

paulstn marked this pull request as ready for review August 26, 2024 21:45

paulstn requested review from ananzh, kavilla, AMoo-Miki, ashwin-pc, joshuarrrr and abbyhu2000 as code owners August 26, 2024 21:45

paulstn commented Aug 26, 2024

View reviewed changes

joshuali925 previously approved these changes Aug 26, 2024

View reviewed changes

kavilla reviewed Aug 27, 2024

View reviewed changes

Merge branch 'main' into dql-autosuggest-updates

8e16722

Signed-off-by: Paul Sebastian <paulstn@amazon.com>

paulstn dismissed joshuali925’s stale review via 8e16722 August 27, 2024 16:10

move constants into test file

22b29c8

Signed-off-by: Paul Sebastian <paulstn@amazon.com>

paulstn requested review from joshuali925 and kavilla August 27, 2024 16:31

remove error catching in auto complete

b25282e

Signed-off-by: Paul Sebastian <paulstn@amazon.com>

joshuali925 previously approved these changes Aug 27, 2024

View reviewed changes

Merge branch 'main' into dql-autosuggest-updates

2aec5a6

kavilla reviewed Aug 27, 2024

View reviewed changes

kavilla previously approved these changes Aug 27, 2024

View reviewed changes

kavilla added backport 2.x v2.17.0 discover-next labels Aug 27, 2024

Changeset file for PR opensearch-project#7593 created/updated

0c2888a

github-actions bot removed the Skip-Changelog PRs that are too trivial to warrant a changelog or release notes entry label Aug 27, 2024

opensearch-changeset-bot bot dismissed stale reviews from kavilla and joshuali925 via 0c2888a August 27, 2024 22:58

kavilla approved these changes Aug 27, 2024

View reviewed changes

joshuali925 approved these changes Aug 27, 2024

View reviewed changes

joshuali925 merged commit 741c0d6 into opensearch-project:main Aug 27, 2024
53 of 54 checks passed

opensearch-trigger-bot bot mentioned this pull request Aug 27, 2024

[Backport 2.x] [Auto Suggest] DQL Updates #7869

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Auto Suggest] DQL Updates #7593

[Auto Suggest] DQL Updates #7593

paulstn commented Jul 31, 2024 •

edited

Loading

github-actions bot commented Jul 31, 2024

codecov bot commented Jul 31, 2024 •

edited

Loading

github-actions bot commented Jul 31, 2024

github-actions bot commented Jul 31, 2024

github-actions bot commented Jul 31, 2024

paulstn Aug 26, 2024

joshuali925 Aug 26, 2024

paulstn Aug 26, 2024

kavilla Aug 27, 2024

paulstn Aug 27, 2024

kavilla commented Aug 27, 2024

kavilla Aug 27, 2024

paulstn Aug 27, 2024

joshuali925 Aug 27, 2024

kavilla Aug 27, 2024

paulstn Aug 27, 2024

paulstn commented Aug 27, 2024

paulstn commented Aug 27, 2024

kavilla Aug 27, 2024

paulstn Aug 27, 2024

[Auto Suggest] DQL Updates #7593

[Auto Suggest] DQL Updates #7593

Conversation

paulstn commented Jul 31, 2024 • edited Loading

Description

Issues Resolved

Screenshot

Testing the changes

Changelog

Check List

github-actions bot commented Jul 31, 2024

❌ Empty Changelog Section

codecov bot commented Jul 31, 2024 • edited Loading

Codecov Report

github-actions bot commented Jul 31, 2024

❌ Empty Changelog Section

github-actions bot commented Jul 31, 2024

❌ Empty Changelog Section

github-actions bot commented Jul 31, 2024

❌ Empty Changelog Section

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kavilla commented Aug 27, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

paulstn commented Aug 27, 2024

paulstn commented Aug 27, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

paulstn commented Jul 31, 2024 •

edited

Loading

codecov bot commented Jul 31, 2024 •

edited

Loading