Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add range agg in noaa and big5 workloads #310

Merged
merged 3 commits into from
Jun 18, 2024

Conversation

bowenlan-amzn
Copy link
Member

@bowenlan-amzn bowenlan-amzn commented Jun 9, 2024

Description

In OpenSearch 13865, previous date histogram aggregation optimization is now applied to range aggregation. So adding operations to track the performance from now on.

Issues Resolved

Related opensearch-project/OpenSearch#13865

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
@bowenlan-amzn bowenlan-amzn changed the title add range agg in noaa add range agg in noaa and big5 workloads Jun 9, 2024
@bowenlan-amzn bowenlan-amzn marked this pull request as ready for review June 9, 2024 18:28
"field": "TMAX",
"ranges": [
{"to": -10},
{"from": -10, "to": 0},
Copy link
Collaborator

@IanHoang IanHoang Jun 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Can we standardize on the JSON format we used in big5?
From

 {"from":   0, "to": 10},

To

{
    "from":   0, 
    "to": 10
},

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this json, the format is like this. I think if we want to format, better to do in a different PR.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. I see it in the subsequent operations.

"name": "range-agg-1",
"operation-type": "search",
"index": "{{index_name | default('big5')}}",
"body": {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason why we don't include "request-timeout": 7200, in range-agg-1?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. This 2 range aggregation operations are actually copied from existing range-auto-date-histo and range-auto-date-histo-with-metrics. I think "request-timeout": 7200 is just needed when the request is by default taking long time. So I will remove this from range-agg-2

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
Copy link
Collaborator

@IanHoang IanHoang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@IanHoang IanHoang added backport 2 Backport to the "2" branch backport 1 backport 3 Backport to the "3" branch labels Jun 18, 2024
@IanHoang IanHoang merged commit 12504df into opensearch-project:main Jun 18, 2024
5 checks passed
opensearch-trigger-bot bot pushed a commit that referenced this pull request Jun 18, 2024
* add range agg in noaa

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* add range agg

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* address comments

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

---------

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
(cherry picked from commit 12504df)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
opensearch-trigger-bot bot pushed a commit that referenced this pull request Jun 18, 2024
* add range agg in noaa

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* add range agg

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* address comments

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

---------

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
(cherry picked from commit 12504df)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
opensearch-trigger-bot bot pushed a commit that referenced this pull request Jun 18, 2024
* add range agg in noaa

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* add range agg

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* address comments

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

---------

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
(cherry picked from commit 12504df)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
IanHoang pushed a commit that referenced this pull request Jun 18, 2024
* add range agg in noaa



* add range agg



* address comments



---------


(cherry picked from commit 12504df)

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
IanHoang pushed a commit that referenced this pull request Jun 18, 2024
* add range agg in noaa



* add range agg



* address comments



---------


(cherry picked from commit 12504df)

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
IanHoang pushed a commit that referenced this pull request Jun 18, 2024
* add range agg in noaa



* add range agg



* address comments



---------


(cherry picked from commit 12504df)

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@bowenlan-amzn bowenlan-amzn deleted the add-range-agg branch June 18, 2024 17:21
harshavamsi pushed a commit to harshavamsi/opensearch-benchmark-workloads that referenced this pull request Jul 16, 2024
* add range agg in noaa

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* add range agg

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* address comments

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

---------

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 1 backport 2 Backport to the "2" branch backport 3 Backport to the "3" branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants