RFC: distributed vegeta

[MalloZup]

RFC Design of distributed vegeta:

Desc

Distributed vegeta operator. ( Load-testing tool) see https://github.com/tsenart/vegeta

Goal

A user can schedule vegeta attacks(load-testing) to web-service, in a k8s cluster.

Highlevel

features 01:

CRD API:
Viakubectl the user will schedule vegeta attacks: see https://github.com/MalloZup/vegeta-operator/blob/master/config/samples/vegeta_v1beta1_vegeta.yaml

Example:

• When doing the command:
  attack: echo "GET http://localhost/" | vegeta attack -duration=5s
  this will schedule a K8s JOB.

• numAttack, with this the user can specify the number of same type load tests. This will then scheduled 2 times along the cluster.

The operator/controller will execute the run in the nodes on the cluster and managing also the report/files with the storage.

Unknowns and need research.

storage of results/report

1. We need to store reports/logs of attacks.
   E.g once the cmd is done we might save maybe in a distributed databases the report, or consider some lightweight solution like the k8s logs https://kubernetes.io/docs/concepts/cluster-administration/logging/

[szuecs]

Just some random thoughts to be used as input (feel free to disagree):

The controller part to run vegeta is easy, but first you should think about how to get the results back. Maybe vegeta should have a different output to make this simple or you can try to get results from Kubernetes logs. Do you even need to return the results, if people have a monitoring in place (think Prometheus + grafana dashboard)?

Outputs supported by vegeta:

encode command:
  -output string
    	Output file (default "stdout")
  -to string
    	Output encoding [csv, gob, json] (default "json")

Maybe the execution engine that could wrap the vegeta could pick up the csv and post it to different data sinks (S3, gist, ...). Why CSV? It's easy to use with R or spreadsheets to do analytics or reports.

CRD could have the args typed or just a list of string that you concat. I would not "over engineer" the solution with a file, or you could just use a configmap, and mount it to all vegeta. The question is then what does the controller do? Why not having a result output by vegeta that is easy to concat?

More important questions:
How this should be used?
Do you configure and run one loadtest by creating one CRD?
Can you run multiple loadtests at the same time?

[MalloZup]

@szuecs thx for feedback! 💘 💘

For the report question yop you are right. I still don't have thought about that.

For the other questions,

• how it should be used:

I was thinking that an user could create a CRD , and then schedule it one time via kubectl.

This would be a 1 JOB object load testing, that will go to the cluster. The cluster will figure out where to execute it.

An User could specify also kind of count so we could have the same test 3 times.
If the CRD/vegeta cmmand change, then we will have 2 diff attacks, which could run by default 1time, but also we could have count time.

This would be a load-test.

• For the CRD i wanted to have a simple and generic CRD, this i need to find out a good approach.

• Can you run multiple loadtests at the same time?

Yes, load-tests can be distributed in the cluster.
..

All this answers are also fragmental and i need to investigate and learn also trying out things, so this can change.

thx for feedback anyways 🎅 🎅

[MalloZup]

i think k8s logging is also nice. I need to research a bit
https://kubernetes.io/docs/concepts/cluster-administration/logging/