Skip to content

Latest commit

 

History

History
99 lines (85 loc) · 5.16 KB

README.md

File metadata and controls

99 lines (85 loc) · 5.16 KB
Raw

Envoy performance automation infrastructure

Pre-requisite

  1. python2.7 Installation: sudo apt-get install python2.7
  2. gcloud Installation: Follow instructions here: https://cloud.google.com/sdk/docs/quickstart-debian-ubuntu
  3. Install the following packages: sudo pip install pexpect sudo apt-get install python-matplotlib

Follow the instructions below and set the values of the variables (e.g. $ENVOY_BINARY) appropriately:

  1. Keep your envoy-binary in an accessible location, $ENVOY_BINARY.
  2. Keep all the scripts, Makefiles in a directory, $SCRIPTS. The scripts, Makefiles refer to the same files that are included in this benchmarking package.
  3. Keep all the Envoy configurations in a directory, $ENVOY_CONFIG.
  4. Select a directory in which you want to keep the result, $RESULT.

Google Cloud-related setup:

  1. Create a Service Account.
  2. Grant proper roles to the newly created Service Account, such as roles/owner.

Run the benchmarking script, as follows with python2:

python2 benchmark.py --local_envoy_binary_path $ENVOY_BINARY --scripts_path $SCRIPTS --envoy_config_path $ENVOY_CONFIG --result_dir $RESULT --username $USERNAME

The above command will create a VM in the us-east1-b zone (default) with the name, envoy-vm. The VM will have, by default, 20 CPUs, 76GB RAM and run Ubuntu 16.04 LTS under envoy-ci project. All the output and errors will be written in a file, named benchmark.log. You can change these default settings by providing the following arguments to the above Python script:

  --vm_name		name of the virtual machine that you want to create
	                (default: envoy-vm)
  --local_envoy_binary_path
	                local relative path of the envoy binary (default:
	                ./envoy-fastbuild)
  --scripts_path	local relative path to the directory of all helper
	                scripts and configs (default: ./)
  --envoy_config_path
	                local relative path to the directory of the envoy
	                configs (default: ./envoy-configs)
  --result_dir
	                local relative path to the directory of the
	                benchmarking result file (default: ./)
  --username   		username on the VM in the cloud-platform (default:
	                envoy)
  --zone            the zone where you want to create the VM. (default:
	                us-east1-b)
  --cpu             number of CPU cores. (default: 20)
  --ram             amount of ram in the VM in MB. (default: 76)
  --os_img_family
	                the OS in which you want the benchmark. (default:
	                ubuntu-1604-lts)
  --os_img_project
	                the project in which the OS can be found. (default:
	                ubuntu-os-cloud)
  --project     	the project name. (default: envoy-ci)
  --logfile     	the local log file for this script. New log messages
	                are appended to this file. (default: benchmark.log)
 --num_retries      the number of retries for a single command. (default:
                    15)
  --sleep_between_retry
	                number of seconds to sleep between each retry.
	                (default: 5)
  --db_instance_name
                    the name of the gcloud instance (default: envoy-db-
                    instance)
  --tier            the tier of GCloud SQL service (default:
	                db-n1-standard-2)
  --db_username     username on the DB (default: root)
  --table_name      the table which stores the benchmarking data (default:
	                envoy_stat)
  --envoy_hash      the hash of envoy version (default: None)
  --runid           the run id of this benchmark (default: 0)
  --database 	    name of the database (default: envoy_stat_db)
--num_iter   		the number of times h2load will be executed,
                    separately for direct and Envoy (default: 5)
  --create_delete   if you want to create/delete new benchmarking VM.
	                (default: True)
  --no-create_delete
  --setup           if you want to run setup on benchmarking VM. (default:
	                True)
  --no-setup
  --create_db_instance  
  					turn on if you want to create a Google Cloud SQL
	                instance (default: True)
  --no-create_db_instance
  --create_db       turn on if you want to create the DB (default: True)
  --no-create_db
  --delete_db       turn on if you want to delete the DB (default: True)
  --no-delete_db
  --ssl                 turn on if you want to enable ssl for the benchmarking
                    (default: True)
  --no-ssl

Generating Graphs

You have to install matplotlib in your machine to generate graphs. Graphs depicting Envoy's performance can be generated by running the below command. The graph will compare between Envoy and direct connection, with respect to the field provided as argument, such as the deafault one total_req_per_sec. Multiple Envoy Hash ids and Run IDs can be provided for the comparison.

 python show_graph.py --envoy_h xxxxxy --runids 10 11 12 13 14 15

The above command will generate two graphs: 1) A bar graph for Envoy proxy and direct connection with Envoy Hash xxxxxy. 2) Another bar for Envoy and direct connection with Run IDs 10, 11, 12, 13, 14, 15.