watch-list: define a job type for the perf-dashboard

[p0lyn0mial]

What type of PR is this?

a separate job type will allow to display metrics
collected during watchlist per tests like
https://perf-dash.k8s.io/#/?jobname=gce-100Nodes-master&metriccategoryname=E2E&metricname=LoadResources&PodName=kube-apiserver-e2e-big-master%2Fkube-apiserver&Resource=memory

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

[p0lyn0mial]

/assign @Argh4k @mborsz

[marseel]

This should be watch-list I think? Based on file in artifacts ResourceUsageSummary_watch-list_2023-05-24T08:35:20Z.json

[p0lyn0mial]

oh, okay, thx

[p0lyn0mial]

I would also add something that would allow me to show latencies

[p0lyn0mial]

perhaps

"LoadResponsiveness": []TestDescription{{
				Name:             "watch-list",
				OutputFilePrefix: "APIResponsiveness",
				Parser:           parsePerfData,
			}},

?

[p0lyn0mial]

or

"LoadResponsiveness_PrometheusSimple": []TestDescription{{
				Name:             "watch-list",
				OutputFilePrefix: "APIResponsivenessPrometheus_simple",
				Parser:           parsePerfData,
			}},

?

[Argh4k]

If you want to you can run perfdash locally to play with and find all the metrics that you need. To do that modify perfdash/Makefile by replacing githubConfigDir with https://api.github.com/repos/kubernetes/test-infra/contents/config/jobs/kubernetes/sig-scalability and replace builds with some reasonable number like 2 so you don't have to wait minutes for perdash to scrape all the data. After that just run make run from perfdash directory and you should have perfdash hosted at localhost:8080

[wojtek-t]

Why do we even need to introduce this? Why can't we use the existing performance?

[p0lyn0mial]

honesty i don't understand why it doesn't work.

[p0lyn0mial]

the tests use the performance type - https://github.com/kubernetes/test-infra/blob/master/config/jobs/kubernetes/sig-scalability/sig-scalability-experimental-periodic-jobs.yaml#L198 atm

[p0lyn0mial]

I started a slack thread if you want to chime in - https://kubernetes.slack.com/archives/C09QZTRH7/p1684911329191689

[marseel]

It doesn't work due to the fact that perfdash has predefined file names here: https://github.com/kubernetes/perf-tests/pull/2276/files#diff-17340b391bfccaca6d6cf3409b032676038adfa8dead9fb15f65557a99bd098eR82
and watch-list/config.yaml has test name watch-list and not load

[wojtek-t]

This is... weird.

Can we then instead parametrized the map and reuse it? (i.e. make load/watch-list a parameter of function returning this map)?

[p0lyn0mial]

If we want to keep the new type then I don't see value in making the map reusable. IMO it would make the code less readable.

I would appreciate if the whole mechanism could be more generic. Ideally if add a new job would not require defining new types.

[wojtek-t]

I would appreciate if the whole mechanism could be more generic. Ideally if add a new job would not require defining new types.

I would like that too, but I don't want to overinvest in perfdash...

[Argh4k]

/lgtm

[wojtek-t]

OK, let's merge it to unblock stuff. And we can improve in a follow up.

/lgtm
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Argh4k, p0lyn0mial, wojtek-t

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• perfdash/OWNERS [wojtek-t]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[p0lyn0mial]

thanks, i have also opened kubernetes/test-infra#29604