Skip to content

Commit

Permalink
test_page_service_batching: fix non-numeric metrics (#9998)
Browse files Browse the repository at this point in the history
## Problem

```
2024-12-03T15:42:46.5978335Z + poetry run python /__w/neon/neon/scripts/ingest_perf_test_result.py --ingest /__w/neon/neon/test_runner/perf-report-local
2024-12-03T15:42:49.5325077Z Traceback (most recent call last):
2024-12-03T15:42:49.5325603Z   File "/__w/neon/neon/scripts/ingest_perf_test_result.py", line 165, in <module>
2024-12-03T15:42:49.5326029Z     main()
2024-12-03T15:42:49.5326316Z   File "/__w/neon/neon/scripts/ingest_perf_test_result.py", line 155, in main
2024-12-03T15:42:49.5326739Z     ingested = ingest_perf_test_result(cur, item, recorded_at_timestamp)
2024-12-03T15:42:49.5327488Z                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-12-03T15:42:49.5327914Z   File "/__w/neon/neon/scripts/ingest_perf_test_result.py", line 99, in ingest_perf_test_result
2024-12-03T15:42:49.5328321Z     psycopg2.extras.execute_values(
2024-12-03T15:42:49.5328940Z   File "/github/home/.cache/pypoetry/virtualenvs/non-package-mode-_pxWMzVK-py3.11/lib/python3.11/site-packages/psycopg2/extras.py", line 1299, in execute_values
2024-12-03T15:42:49.5335618Z     cur.execute(b''.join(parts))
2024-12-03T15:42:49.5335967Z psycopg2.errors.InvalidTextRepresentation: invalid input syntax for type numeric: "concurrent-futures"
2024-12-03T15:42:49.5336287Z LINE 57:             'concurrent-futures',
2024-12-03T15:42:49.5336462Z                      ^
```

## Summary of changes
- `test_page_service_batching`: save non-numeric params as `labels`
- Add a runtime check that `metric_value` is NUMERIC
  • Loading branch information
bayandin authored Dec 3, 2024
1 parent 944c1ad commit 023821a
Show file tree
Hide file tree
Showing 2 changed files with 14 additions and 7 deletions.
10 changes: 10 additions & 0 deletions test_runner/fixtures/benchmark_fixture.py
Original file line number Diff line number Diff line change
Expand Up @@ -266,6 +266,16 @@ def record(
name = f"{self.PROPERTY_PREFIX}_{metric_name}"
if labels is None:
labels = {}

# Sometimes mypy can't catch non-numeric values,
# so adding a check here
try:
float(metric_value)
except ValueError as e:
raise ValueError(
f"`metric_value` (`{metric_value}`) must be a NUMERIC-friendly data type"
) from e

self.property_recorder(
name,
{
Expand Down
11 changes: 4 additions & 7 deletions test_runner/performance/pageserver/test_page_service_batching.py
Original file line number Diff line number Diff line change
Expand Up @@ -116,21 +116,18 @@ def test_throughput(
# name is not a metric, we just use it to identify the test easily in the `test_...[...]`` notation
}
)
params.update(
{
f"pipelining_config.{k}": (v, {})
for k, v in dataclasses.asdict(pipelining_config).items()
}
)
# For storing configuration as a metric, insert a fake 0 with labels with actual data
params.update({"pipelining_config": (0, {"labels": dataclasses.asdict(pipelining_config)})})

log.info("params: %s", params)

for param, (value, kwargs) in params.items():
zenbenchmark.record(
param,
metric_value=value,
metric_value=float(value),
unit=kwargs.pop("unit", ""),
report=MetricReport.TEST_PARAM,
labels=kwargs.pop("labels", None),
**kwargs,
)

Expand Down

1 comment on commit 023821a

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

7155 tests run: 6837 passed, 0 failed, 318 skipped (full report)


Flaky tests (3)

Postgres 17

Postgres 15

Postgres 14

Code coverage* (full report)

  • functions: 30.8% (8265 of 26844 functions)
  • lines: 47.8% (65174 of 136375 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
023821a at 2024-12-04T00:34:34.763Z :recycle:

Please sign in to comment.