OCPVE-353: chore: operf refactoring

[jakobmoellerdev]

performance test refactoring to prepare runnning in CI.

Supports running in either long-observation modes for idle tests against a cluster, or initiates stress tests automatically by creating pods/pvcs in the cluster and observing that window.

Collected aggregates:
99,95,90 quantiles on CPU and Memory over timeframe of either stress period, or of duration given by flag long-term-observation-window if run-stress=false

Writes a Quantile report as .toml in --output-directory or current working directory if not set.

go run ./test/performance --help
This script retrieves cpu and memory usage metrics for all the workloads created by the logical volume manager storage (lvms).

It creates <instances> of "busybox" pods using PVCs provisioned via the Storage class provided, when in stress test mode, or only collects metrics in idle.

A report is written as "metrics-<start_unix_ts>-<end_unix_ts>.toml" in <output-directory> or current working directory if not set.

Quantile Units in report:
CPU: mcores
Memory: MiB

Usage:
  go run ./test/performance [flags]

Examples:
Stress: go run ./test/performance -t $(oc whoami -t) -s lvms-vg1 -i 64
Idle: go run ./test/performance -t $(oc whoami -t) --run-stress false --long-term-observation-window=5m

Flags:
  -h, --help                                    help for performance
  -i, --instances int                           Number of Workloads/Pvcs (each Workload uses a different PVC) to create in the test (default 4)
  -w, --long-term-observation-window duration   only used when not running stress test, defines observation windows as duration into the past, e.g. 5m means the last 5 minutes (default 5m0s)
  -n, --namespace string                        Namespace where operator is deployed and the PVCs and test pods will be deployed/undeployed (default "openshift-storage")
  -o, --output-directory string                 output directory for the metrics report, working directry by default
  -p, --pattern string                          Pattern used to build the PVCs/Workloads names (default "operf")
  -r, --run-stress                              defines if stress tests should be used, if false uses long-term observation and does not attempt to create stress resources (default true)
  -s, --storage-class string                    Name of the topolvm storage class that will be used in the PVCs (default "lvms-vg1")
  -t, --token string                            authentication token needed to connect with the Openshift cluster.

How to test:

1. Create a cluster and login via user/pass (ideally kubeadmin)
2. Run one of the makefile commands, or alternatively create the LVMCluster manually and then run the tool manually.
3. Observe findings created from the .toml files in the working directory.

[openshift-ci-robot]

@jakobmoellerdev: This pull request references OCPVE-353 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.15.0" version, but no target version was set.

In response to this:

performance test refactoring to prepare runnning in CI.

Supports running in either long-observation modes for idle tests against a cluster, or initiates stress tests automatically by creating pods/pvcs in the cluster and observing that window.

Collected aggregates:
99,95,90 quantiles on CPU and Memory over timeframe of either stress period, or of duration given by flag long-term-observation-window if run-stress=false

Writes a Quantile report as .toml in --output-directory or current working directory if not set.

go run ./test/performance --help
This script retrieves cpu and memory usage metrics for all the workloads created by the logical volume manager storage (lvms).

It creates <instances> of "busybox" pods using PVCs provisioned via the Storage class provided, when in stress test mode, or only collects metrics in idle.

A report is written as "metrics-<start_unix_ts>-<end_unix_ts>.toml" in <output-directory> or current working directory if not set.

Quantile Units in report:
CPU: mcores
Memory: MiB

Usage:
 go run ./test/performance [flags]

Examples:
Stress: go run ./test/performance -t $(oc whoami -t) -s lvms-vg1 -i 64
Idle: go run ./test/performance -t $(oc whoami -t) --run-stress false --long-term-observation-window=5m

Flags:
 -h, --help                                    help for performance
 -i, --instances int                           Number of Workloads/Pvcs (each Workload uses a different PVC) to create in the test (default 4)
 -w, --long-term-observation-window duration   only used when not running stress test, defines observation windows as duration into the past, e.g. 5m means the last 5 minutes (default 5m0s)
 -n, --namespace string                        Namespace where operator is deployed and the PVCs and test pods will be deployed/undeployed (default "openshift-storage")
 -o, --output-directory string                 output directory for the metrics report, working directry by default
 -p, --pattern string                          Pattern used to build the PVCs/Workloads names (default "operf")
 -r, --run-stress                              defines if stress tests should be used, if false uses long-term observation and does not attempt to create stress resources (default true)
 -s, --storage-class string                    Name of the topolvm storage class that will be used in the PVCs (default "lvms-vg1")
 -t, --token string                            authentication token needed to connect with the Openshift cluster.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[openshift-ci-robot]

@jakobmoellerdev: This pull request references OCPVE-353 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.15.0" version, but no target version was set.

In response to this:

performance test refactoring to prepare runnning in CI.

Supports running in either long-observation modes for idle tests against a cluster, or initiates stress tests automatically by creating pods/pvcs in the cluster and observing that window.

Collected aggregates:
99,95,90 quantiles on CPU and Memory over timeframe of either stress period, or of duration given by flag long-term-observation-window if run-stress=false

Writes a Quantile report as .toml in --output-directory or current working directory if not set.

go run ./test/performance --help
This script retrieves cpu and memory usage metrics for all the workloads created by the logical volume manager storage (lvms).

It creates <instances> of "busybox" pods using PVCs provisioned via the Storage class provided, when in stress test mode, or only collects metrics in idle.

A report is written as "metrics-<start_unix_ts>-<end_unix_ts>.toml" in <output-directory> or current working directory if not set.

Quantile Units in report:
CPU: mcores
Memory: MiB

Usage:
 go run ./test/performance [flags]

Examples:
Stress: go run ./test/performance -t $(oc whoami -t) -s lvms-vg1 -i 64
Idle: go run ./test/performance -t $(oc whoami -t) --run-stress false --long-term-observation-window=5m

Flags:
 -h, --help                                    help for performance
 -i, --instances int                           Number of Workloads/Pvcs (each Workload uses a different PVC) to create in the test (default 4)
 -w, --long-term-observation-window duration   only used when not running stress test, defines observation windows as duration into the past, e.g. 5m means the last 5 minutes (default 5m0s)
 -n, --namespace string                        Namespace where operator is deployed and the PVCs and test pods will be deployed/undeployed (default "openshift-storage")
 -o, --output-directory string                 output directory for the metrics report, working directry by default
 -p, --pattern string                          Pattern used to build the PVCs/Workloads names (default "operf")
 -r, --run-stress                              defines if stress tests should be used, if false uses long-term observation and does not attempt to create stress resources (default true)
 -s, --storage-class string                    Name of the topolvm storage class that will be used in the PVCs (default "lvms-vg1")
 -t, --token string                            authentication token needed to connect with the Openshift cluster.

How to test:

1. Create a cluster and login via user/pass (ideally kubeadmin)
2. Run one of the makefile commands, or alternatively create the LVMCluster manually and then run the tool manually.
3. Observe findings created from the .toml files in the working directory.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[codecov-commenter]

Codecov Report

Merging #451 (d6d45f0) into main (58e7c6f) will decrease coverage by 2.53%.
Report is 16 commits behind head on main.
The diff coverage is n/a.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #451      +/-   ##
==========================================
- Coverage   59.41%   56.88%   -2.53%     
==========================================
  Files          21       22       +1     
  Lines        1614     1735     +121     
==========================================
+ Hits          959      987      +28     
- Misses        539      626      +87     
- Partials      116      122       +6     

see 8 files with indirect coverage changes

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jakobmoellerdev

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [jakobmoellerdev]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[suleymanakbas91]

Looking great in general! It just needs a bit of documentation.

[openshift-ci]

@jakobmoellerdev: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[suleymanakbas91]

Great work!
/lgtm