Load ramping test results - 29 September 2023

[CS76]

To quote Brendan Gregg's excellent Systems Performance book, chapter 12.3.7 Ramping Load:

"This is a simple method for determining the maximum throughput a system can handle. It involves adding load in small increments and measuring the delivered throughput until a limit is reached. The results can be graphed, showing a scalability profile. This profile can be studied visually[...]

[...] When following this approach, measure latency as well as the throughput, especially the latency distribution. [...] If you push load too high, latency may become so high that it is no longer reasonable to consider the result as valid. Ask yourself if the delivered latency would be acceptable to a customer."

We use Vegeta to run load ramping tests and the latest test results are shown below.

Performance testing: Latency distribution and success rate at each
request rate

Ramping Load: To determine the maximum throughput the
Cheminformatics Microservice public instance (Intel(R) Xeon(R) Gold
6226R CPU @ 2.90GHz and 16GB of RAM) can handle, we used the Vegeta
([https://github.com/tsenart/vegeta]{.underline})
tool to add load in small increments and measure the delivered
throughput until a limit was reached. The results were then graphed to
show the scalability profile. The microservice's create 2D
coordinates from SMILES endpoint was chosen as the test request, and
the scalability profiles were generated for requests with input
structures submitted as SMILES representations starting from 6 heavy
atoms to 125 heavy atoms (randomly sampled from the COCONUT database).
The scalability profiles were then inspected visually to assess the
server performance (SI Figure A - K).

(A) Scalability profile - Echo request

(B) Scalability profile - SMILES to Mol2D Conversion (6 heavy atoms)

(C) Scalability profile - SMILES to Mol2D Conversion (11 heavy atoms)

(D) Scalability profile - SMILES to Mol2D Conversion (13 heavy atoms)

(E) Scalability profile - SMILES to Mol2D Conversion (20 heavy atoms)

(F) Scalability profile - SMILES to Mol2D Conversion (24 heavy atoms)

(G) Scalability profile - SMILES to Mol2D Conversion (32 heavy atoms)

(H) Scalability profile - SMILES to Mol2D Conversion (54 heavy atoms)

(I) Scalability profile - SMILES to Mol2D Conversion (74 heavy atoms)

(J) Scalability profile - SMILES to Mol2D Conversion (99 heavy atoms)

(K) Scalability profile - SMILES to Mol2D Conversion (125 heavy atoms)

A snapshot of performance comparing the success rate with an increase in the number of requests per second as well as with an increase in the number of heavy atoms in the input. The grey line represents the maximum number of requests the server could process (echo request).