Rust based HTTP load testing tool using yaml files.
Bombardier can take your simple yaml based files containing scenarios & environment variables and bombard your application with a defined load.
Bombardier needs 2 files at minimum and 4 files at max to carry out any load tests
config.yaml
(required) - Contains the execution configuration like no of threads, rampup time etc. Check config section for more detailsscenarios.yml
(required) - File containing scenarios which would be used to generate loadenvironments.yml
(optional) - Environment variables which would be replaced in scenarios file during executiondata.csv
(optional) - CSV file to supply test data
You need to create a yml file which can tell Bombardier about the load configuration.
version: 1.0
threadCount: 1
iterations: 10
threadDelay: 1
executionTime: 0
rampUpTime: 1
continueOnError: true
handleCookies: false
ssl:
ignoreSSL: true
acceptInvalidHostnames: false
certificate: "./ca_cert.pem"
keyStore: "./keystore.p12"
keyStorePassword: "P@$$w0rd123"
database:
type: influxDB
url: http://some-influxdb-url/
name: dbName
user: someUser
password: P@$$w0rd123
For more details regarding configuration json, please check configurations doc.
A simple scenarios.yml looks something like below. Every scenario can have N requests.
Each request is defined with its name, method, url, headers, body and extractors.
Extractors is an array which are more like post processors to be applied on the received response, they help to extract certain values from response which then can be used in the following requests.
To know more about extractors check this doc
version: 1.0
scenarios:
- name: scenario1
requests:
- name: echoGet
method: GET
url: "{{baseUrl}}/get"
headers:
authorization: bearer jwt
extractors:
- from: Body
type: GjsonPath
extract:
authHeader: "headers.Authorization"
host: "headers.Host"
- from: Body
type: RegEx
extract:
contentType: 'Root=.*'
- from: Headers
extract:
server: server
- name: echoPostWithUrlEncoded
method: POST
url: '{{baseUrl}}/post'
body:
urlencoded:
key23: '{{Country}}'
key24: '{{City}}'
extractors:
- type: GjsonPath
extract:
country: "form.key23"
- type: GjsonPath
extract:
city: "form.key24"
- name: scenario2
requests:
- name: echoPostFormData
method: POST
url: '{{baseUrl}}/post'
body:
formdata:
- name: key1
value: '{{AccentCity}}'
extractors:
- type: GjsonPath
extract:
keyname: "form.key1"
- name: echoPostFormDataWithFile
method: POST
url: '{{baseUrl}}/post'
body:
formdata:
- name: key21
value: key22
- name: file
value: '/Users/coding-yogi/work/repos/bombardier/examples/configdev.json'
type: File
extractors:
- type: GjsonPath
extract:
keyname: "files.file"
- name: echoPostJSON
method: POST
url: '{{baseUrl}}/post'
headers:
content-type: application/json
body:
raw: |
{
"test": "test"
}
extractors:
- type: GjsonPath
extract:
keyname: "json.test"
Many a times there would be need to have some variables which needs to be used through the tests, One such example you can see in the above tests is the url
value. As the baseURL would remain same, you would want to pull it out of the tests, so that it can be updated (if required) later at a single place. All such variables can go into a file environment.yml
, Below is an example of the same
version: 1.0
variables:
baseUrl: https://httpbin.org
Bombardier will do the necessary replacement of the values from environment file into the scenarios file at runtime
Data file is a simple CSV format file with 1st row as header values, Data is injected into the test by reading from csv file. At end of CSV file, tests start reading the data again from 1st row till the execution ends.
Similar to environments file and data which is read from the data file will be replaced as per the parameter name. Parameter name should match the column name in the csv file for successful substitution of the value
Make sure you have cargo and rust installed. Checkout the code and run below command.
If code builds successfully you should see the binary/executable in /target/release
folder
cargo build --release
If you face issues building the binaries from the source please check troubleshooting document
If you do not wish to build bombardier from source, you can always download the binaries and use the tool directly. Go to releases to download the binary of your choice
Bombardier can run inside a container too. You can build the image with below command or just pull the codingyogi/bombardier:latest
image from docker hub
docker build . -t bombardier
Container can be started using below command
docker run --name bombardier -v $PWD:/home/tests bombardier:latest bombard -c ./tests/config.yml -s ./tests/scenarios.yml -e ./tests/env.yml -r ./tests/report.csv
Note the volume used. Present working directory on host is mapped to /home
directory on the container.
With this approach you need not copy your config file or collections file into the container.
./bombardier bombard -c <path of config yml> -s <path of scenarios yml> -e <path of env yml> -d <path of data csv>
Distributed tests run in a Control-Data plan architecture. Hub is a control plane from where you can control the execution.
Nodes are part of data plane which are responsible for generating the actual load.
./bombardier hub -p <port for rest server> -s <port for websocket server>
Hub is responsible for distributing load to all its nodes and it runs 2 servers
One is a REST server for user to interfact with bombardier, second is a websocket server for the hub to communicate with its nodes.
Check api documentation for API details
./bombardier node -h <hub address>
Node requires hub's address to connect to, so make sure hub is up and running before nodes are started
export RUST_LOG=debug
Debug logs would be written only to log file. It is not advisable to enable debug logging during actual execution of tests
./bombardier report -r <path to csv report file>
Request | Total Hits | Hits/s | Min | Avg | Max | 90% | 95% | 99% | Errors | Error Rate |
---|---|---|---|---|---|---|---|---|---|---|
PostWithFormData | 1000 | 5.7471266 | 235 | 282 | 1312 | 300 | 304 | 398 | 0 | 0 |
PostWithJsonAndReplaceableParam | 1000 | 5.7471266 | 235 | 280 | 1308 | 296 | 304 | 335 | 0 | 0 |
PostWithFormUrlEncoded | 1000 | 5.7471266 | 235 | 281 | 882 | 296 | 304 | 783 | 0 | 0 |
GetWithQueryParams | 1000 | 5.7471266 | 234 | 284 | 1307 | 296 | 303 | 808 | 0 | 0 |
PostWithNoBody | 1000 | 5.7471266 | 234 | 279 | 2168 | 296 | 303 | 327 | 0 | 0 |
Total Execution Time (in secs) | Total Hits | Hits/s | Total Errors | Error Rate |
---|---|---|---|---|
174 | 6000 | 34.482758 | 0 | 0 |
I would like this tool to be benchmarked with other tools to see if it needs more improvement. You can find the benchmarks here