Skip to content

Commit

Permalink
[data] Change counts of metrics to rates of metrics (#47236)
Browse files Browse the repository at this point in the history 
## Why are these changes needed?
This changes the counts of various metrics for ray data to rates for
those same metrics. This allows for easier visualization.

There are a variety of different approaches we can take here to
calculate the rates:
* Window of rate: We can compute the rate over a time window of
specified size, the relevancy of this depends on the function chosen.
* Function:
* `rate`: Calculates the average per-second rate of increase of a time
series over a specified range. Less volatile, produces more smoothing
depending on the window.
* `irate`: Calculates the per-second rate of increase based on the two
most recent data points. More volatile and reactive, less smoothing
applied, not as sensitive to the window size because last two data
points are used.
 
### Examples:
**Yellow: `rate[30s]`** (Have selected this as this seems to be the best
balance between the various options)
Orange: `irate[30s]`
Blue: `rate[5m]`

<img width="2688" alt="image"
src="https://github.com/user-attachments/assets/a2424318-e06a-4b32-8d8c-83fc1dfbb540">
 
#### Sample code
```python
import ray
import time

context = ray.init(include_dashboard=True)
print(context.dashboard_url)

def f(x):
    time.sleep(0.1)
    return x

def g(x):
    time.sleep(0.1)
    return x

ray.data.range(10000).map(f).materialize()
```


## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

---------

Signed-off-by: Matthew Owen <mowen@anyscale.com>
Signed-off-by: Matthew Owen <omatthew98@berkeley.edu>
Co-authored-by: Scott Lee <scottjlee@users.noreply.github.com>
  • Loading branch information
@omatthew98 @scottjlee
omatthew98 and scottjlee authored Sep 11, 2024
1 parent f422376 commit 03ab9b3
Showing 1 changed file with 60 additions and 62 deletions.
122 changes: 60 additions & 62 deletions python/ray/dashboard/modules/metrics/dashboards/data_dashboard_panels.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -114,27 +114,27 @@
),
Panel(
id=7,
title="Bytes Outputted",
description="Total bytes outputted by dataset operators.",
unit="bytes",
title="Bytes Output / Second",
description="Bytes output per second by dataset operators.",
unit="Bps",
targets=[
Target(
expr="sum(ray_data_output_bytes{{{global_filters}}}) by (dataset, operator)",
legend="Bytes Outputted: {{dataset}}, {{operator}}",
expr="sum(rate(ray_data_output_bytes{{{global_filters}}}[1m])) by (dataset, operator)",
legend="Bytes Output / Second: {{dataset}}, {{operator}}",
)
],
fill=0,
stack=False,
),
Panel(
id=11,
title="Rows Outputted",
description="Total rows outputted by dataset operators.",
unit="rows",
title="Rows Output / Second",
description="Total rows output per second by dataset operators.",
unit="rows/sec",
targets=[
Target(
expr="sum(ray_data_output_rows{{{global_filters}}}) by (dataset, operator)",
legend="Rows Outputted: {{dataset}}, {{operator}}",
expr="sum(rate(ray_data_output_rows{{{global_filters}}}[1m])) by (dataset, operator)",
legend="Rows Output / Second: {{dataset}}, {{operator}}",
)
],
fill=0,
Expand All @@ -143,147 +143,145 @@
# Ray Data Metrics (Inputs)
Panel(
id=17,
title="Input Blocks Received by Operator",
description="Number of input blocks received by operator.",
unit="blocks",
title="Input Blocks Received by Operator / Second",
description="Number of input blocks received by operator per second.",
unit="blocks/sec",
targets=[
Target(
expr="sum(ray_data_num_inputs_received{{{global_filters}}}) by (dataset, operator)",
legend="Blocks Received: {{dataset}}, {{operator}}",
expr="sum(rate(ray_data_num_inputs_received{{{global_filters}}}[1m])) by (dataset, operator)",
legend="Blocks Received / Second: {{dataset}}, {{operator}}",
)
],
fill=0,
stack=False,
),
Panel(
id=18,
title="Input Blocks Received by Operator",
description="Byte size of input blocks received by operator.",
unit="bytes",
title="Input Bytes Received by Operator / Second",
description="Byte size of input blocks received by operator per second.",
unit="Bps",
targets=[
Target(
expr="sum(ray_data_bytes_inputs_received{{{global_filters}}}) by (dataset, operator)",
legend="Bytes Received: {{dataset}}, {{operator}}",
expr="sum(rate(ray_data_bytes_inputs_received{{{global_filters}}}[1m])) by (dataset, operator)",
legend="Bytes Received / Second: {{dataset}}, {{operator}}",
)
],
fill=0,
stack=False,
),
Panel(
id=19,
title="Input Blocks Processed by Tasks",
title="Input Blocks Processed by Tasks / Second",
description=(
"Number of input blocks that operator's tasks " "have finished processing."
"Number of input blocks that operator's tasks have finished processing per second."
),
unit="blocks",
unit="blocks/sec",
targets=[
Target(
expr="sum(ray_data_num_task_inputs_processed{{{global_filters}}}) by (dataset, operator)",
legend="Blocks Processed: {{dataset}}, {{operator}}",
expr="sum(rate(ray_data_num_task_inputs_processed{{{global_filters}}}[1m])) by (dataset, operator)",
legend="Blocks Processed / Second: {{dataset}}, {{operator}}",
)
],
fill=0,
stack=False,
),
Panel(
id=20,
title="Input Bytes Processed by Tasks",
title="Input Bytes Processed by Tasks / Second",
description=(
"Byte size of input blocks that operator's tasks "
"have finished processing."
"Byte size of input blocks that operator's tasks have finished processing per second."
),
unit="bytes",
unit="Bps",
targets=[
Target(
expr="sum(ray_data_bytes_task_inputs_processed{{{global_filters}}}) by (dataset, operator)",
legend="Bytes Processed: {{dataset}}, {{operator}}",
expr="sum(rate(ray_data_bytes_task_inputs_processed{{{global_filters}}}[1m])) by (dataset, operator)",
legend="Bytes Processed / Second: {{dataset}}, {{operator}}",
)
],
fill=0,
stack=False,
),
Panel(
id=21,
title="Input Bytes Submitted to Tasks",
description="Byte size of input blocks passed to submitted tasks.",
unit="bytes",
title="Input Bytes Submitted to Tasks / Second",
description="Byte size of input blocks passed to submitted tasks per second.",
unit="Bps",
targets=[
Target(
expr="sum(ray_data_bytes_inputs_of_submitted_tasks{{{global_filters}}}) by (dataset, operator)",
legend="Bytes Submitted: {{dataset}}, {{operator}}",
expr="sum(rate(ray_data_bytes_inputs_of_submitted_tasks{{{global_filters}}}[1m])) by (dataset, operator)",
legend="Bytes Submitted / Second: {{dataset}}, {{operator}}",
)
],
fill=0,
stack=False,
),
Panel(
id=22,
title="Blocks Generated by Tasks",
description="Number of output blocks generated by tasks.",
unit="blocks",
title="Blocks Generated by Tasks / Second",
description="Number of output blocks generated by tasks per second.",
unit="blocks/sec",
targets=[
Target(
expr="sum(ray_data_num_task_outputs_generated{{{global_filters}}}) by (dataset, operator)",
legend="Blocks Generated: {{dataset}}, {{operator}}",
expr="sum(rate(ray_data_num_task_outputs_generated{{{global_filters}}}[1m])) by (dataset, operator)",
legend="Blocks Generated / Second: {{dataset}}, {{operator}}",
)
],
fill=0,
stack=False,
),
Panel(
id=23,
title="Bytes Generated by Tasks",
description="Byte size of output blocks generated by tasks.",
unit="bytes",
title="Bytes Generated by Tasks / Second",
description="Byte size of output blocks generated by tasks per second.",
unit="Bps",
targets=[
Target(
expr="sum(ray_data_bytes_task_outputs_generated{{{global_filters}}}) by (dataset, operator)",
legend="Bytes Generated: {{dataset}}, {{operator}}",
expr="sum(rate(ray_data_bytes_task_outputs_generated{{{global_filters}}}[1m])) by (dataset, operator)",
legend="Bytes Generated / Second: {{dataset}}, {{operator}}",
)
],
fill=0,
stack=False,
),
Panel(
id=24,
title="Rows Generated by Tasks",
description="Number of rows in generated output blocks from finished tasks.",
unit="rows",
title="Rows Generated by Tasks / Second",
description="Number of rows in generated output blocks from finished tasks per second.",
unit="rows/sec",
targets=[
Target(
expr="sum(ray_data_rows_task_outputs_generated{{{global_filters}}}) by (dataset, operator)",
legend="Rows Generated: {{dataset}}, {{operator}}",
expr="sum(rate(ray_data_rows_task_outputs_generated{{{global_filters}}}[1m])) by (dataset, operator)",
legend="Rows Generated / Second: {{dataset}}, {{operator}}",
)
],
fill=0,
stack=False,
),
Panel(
id=25,
title="Output Blocks Taken by Downstream Operators",
description="Number of output blocks that are already taken by downstream operators.",
unit="blocks",
title="Output Blocks Taken by Downstream Operators / Second",
description="Number of output blocks taken by downstream operators per second.",
unit="blocks/sec",
targets=[
Target(
expr="sum(ray_data_num_outputs_taken{{{global_filters}}}) by (dataset, operator)",
legend="Blocks Taken: {{dataset}}, {{operator}}",
expr="sum(rate(ray_data_num_outputs_taken{{{global_filters}}}[1m])) by (dataset, operator)",
legend="Blocks Taken / Second: {{dataset}}, {{operator}}",
)
],
fill=0,
stack=False,
),
Panel(
id=26,
title="Output Bytes Taken by Downstream Operators",
title="Output Bytes Taken by Downstream Operators / Second",
description=(
"Byte size of output blocks that are already "
"taken by downstream operators."
"Byte size of output blocks taken by downstream operators per second."
),
unit="bytes",
unit="Bps",
targets=[
Target(
expr="sum(ray_data_bytes_outputs_taken{{{global_filters}}}) by (dataset, operator)",
legend="Bytes Taken: {{dataset}}, {{operator}}",
expr="sum(rate(ray_data_bytes_outputs_taken{{{global_filters}}}[1m])) by (dataset, operator)",
legend="Bytes Taken / Second: {{dataset}}, {{operator}}",
)
],
fill=0,
Expand Down

0 comments on commit 03ab9b3

Please sign in to comment.