Server performance test #893
Conversation
bruntib
force-pushed
the
performance_test
branch
from
a6bcc85 to
234e8f1
Compare
| VERBOSE = False | ||
|
|
||
|
|
||
| def timeit(func): |
There was a problem hiding this comment.
Don't we have similar functionality somewhere? Wouldn't it be better to put that to a common place and use it?
There was a problem hiding this comment.
The name collision might be a bit annoying. This decorator does something else.
There was a problem hiding this comment.
Maybe it would be better to use a slightly different name? (So go to definition works better witihin IDEs).
bruntib
force-pushed
the
performance_test
branch
from
234e8f1 to
e8cd4f7
Compare
There was a problem hiding this comment.
- Add a short readme.md how to use this script.
- make the job generation rate configurable
A later development proposal:
-Now the client, when sends a job zips the results again, and again. We should save the clients resources to be able to generate higher rate. Could you modify the script so that the client side zipping is not repeated?
|
|
||
| def play(self): | ||
| for name, func, args in self._actions: | ||
| self._user_random_sleep() |
There was a problem hiding this comment.
This random sleep time should follow exponential distribution and its mean should be configurable from command line.
"0" should mean that we do not wait anything (0s no random wait) between generating events.
bruntib
force-pushed
the
performance_test
branch
from
e8cd4f7 to
263b83c
Compare
| print('-' * (40 + len('stdout'))) | ||
|
|
||
|
|
||
| def parse_arguments(): |
There was a problem hiding this comment.
Add a new argument -t which specifies the duration of the simulation and measurement. If not specified the test should run indefinitely long until CTRL-C is pressed. When exiting, the script should print the statistics.
|
I have a single concern: this script is too much of a tool which allows arbitrary users to attack and possibly overflow an arbitrary CodeChecker server. Shouldn't we somehow harden a live production deployment against such an attack? My real concern is filling the disks beyond repair, assuming the server itself withstands the load. |
bruntib
force-pushed
the
performance_test
branch
from
263b83c to
337c253
Compare
|
I think by default we do not accept any request that is not coming from the localhost. If you make your server public, authentication can protect you from such attacks. What kind of hardening can you think of? Database size quota? |
bruntib
force-pushed
the
performance_test
branch
from
337c253 to
79c303c
Compare
| @@ -0,0 +1,346 @@ | |||
| import argparse | |||
|
|
||
|
|
||
| def parse_arguments(): | ||
| parser = argparse.ArgumentParser( |
There was a problem hiding this comment.
It would be useful to include default values of the argument in the help. Use argparse.ArgumentDefaultsHelpFormatter for this:
formatter_class=argparse.ArgumentDefaultsHelpFormatter,| type=int, | ||
| default=1, | ||
| help="Number of users") | ||
| parser.add_argument('-t', '--time', |
There was a problem hiding this comment.
Wouldn't it be better to use --timeout instead of --time for full argument name? You wrote in the help that this argument is a timeout.
| metavar='PRODUCT_URL', | ||
| dest='product_url', | ||
| default='localhost:8001/Default', | ||
| required=True, |
There was a problem hiding this comment.
This should be false because you specified a default value for this argument.
| "of seconds. The random numbers have exponential " | ||
| "distribution of the beta parameter can be " | ||
| "provided here.") | ||
| parser.add_argument('-v', '--verbose', |
There was a problem hiding this comment.
I started to run this script without a running server. The output of this script was the following:
User 1 is sleeping 0.391030774259 seconds
Storage of performance_test_1 is started (~/.codechecker/tinyxml/)
Storage of performance_test_1 is doneThis tells me that the storage was successful.
When I run this test script with the verbose flag the output says the storage was unsuccessful.
User 1 is sleeping 5.28858773684 seconds
Storage of performance_test_1 is started (/home/ecsomar/.codechecker/tinyxml-resolved/)
Output of storage
--------------------stdout--------------------
[09:39] - Storing analysis results for run 'performance_test_1'
Connection refused
[Errno 111] Connection refused
--------------------stderr--------------------
----------------------------------------------
Storage of performance_test_1 is doneWouldn't it be better to write a better message to the user that the storage was successful or unsuccessful?
| """ | ||
| self._stats[user_id].append((task_name, duration)) | ||
|
|
||
| def print_stats(self, file_name): |
There was a problem hiding this comment.
The output of this function looks like this:
User,Storage,Comparison,Reports,Delete,Storage,Comparison
1,0.110867,0.12792,0.126015,0.090653,0.093675,0.113359For my eyes it would be better to write this statistics in the following format:
---==== Statistics ====---
--------------------------
Action | Time (ms)
--------------------------
Storage | 0.110867
Comparison | 0.12792
Reports | 0.126015
Delete | 0.090653
Storage | 0.093675
Comparison | 0.113359
--------------------------
----==================----
Number of users: 2
----==================----We use this at the CodeChecker parse command when we write the statistics to the output.
bruntib
force-pushed
the
performance_test
branch
2 times, most recently
from
33c0bc9 to
92ca3e1
Compare
bruntib
force-pushed
the
performance_test
branch
from
92ca3e1 to
707ba67
Compare
Resolves #808