Make zero wait time the default #1626
Conversation
… to specify it. Makes a big difference in readability of small User classes and reduces the cognitive overhead for locust beginners. Fixes #1308
Codecov Report
@@ Coverage Diff @@
## master #1626 +/- ##
==========================================
+ Coverage 82.49% 82.58% +0.08%
==========================================
Files 28 28
Lines 2571 2572 +1
Branches 392 392
==========================================
+ Hits 2121 2124 +3
+ Misses 355 352 -3
- Partials 95 96 +1
Continue to review full report at Codecov.
|
|
|
||
| class QuickstartUser(HttpUser): | ||
| wait_time = between(1, 2) |
There was a problem hiding this comment.
The removal of a wait time for this example doesn't really make sense (a typical web user - which I think is the most common usecase - doesn't have zero wait time between requests). It can also make users new to Locust DoS-attack themselves (either their web server, or the Locust client itself if the HTTP-server can keep up) if they simply run the example (case in point #1629).
There was a problem hiding this comment.
I dont really know what is the issue in that ticket, but it seems he got 100% cpu even when specifying a wait time according to his latest post...
There was a problem hiding this comment.
True, the #1629 seems to be an unrelated issue, but my point still stands. Even if you don't agree with my statement about it making more sense with a wait time, don't you see an issue with having an example that'll hit 100% CPU and trigger the Locust CPU warning when started?
|
|
||
| The class defines a ``wait_time`` that will make the simulated users wait between 1 and 2 seconds after each task (see below) | ||
| is executed. For more info see :ref:`wait-time`. | ||
|
|
There was a problem hiding this comment.
Wait time for Locust users its a really important concept when using Locust as originally intended (defining user behaviour in code, and then choosing how many simultaneous users using a website to simulate), and I definitely think it should exist in the quickstart documentation.
There are definitely use-cases for zero or super low wait time as well, but I don't think that's a reason to remove this short paragraph from the introduction docs.
There was a problem hiding this comment.
I am aware of your arguments, and you are of mine so I wont repeat myself. The community voted in favour of changing it in #1308
There was a problem hiding this comment.
The community voted in favour of changing it in #1308
"The community" did not vote wether to include wait time in the quick start documentation.
There was a problem hiding this comment.
I also think having community votes to decide these things is a bad idea, and it's nothing that we've decided to have.
Out of curiosity - and since I didn't recognize the usernames of most of those who had "voted" - I checked the number of commits they had done to the Locust project. Among those who had voted 👍 , besides you, only one other user (though two were hidden behind "2 more" by the Github UI) had a single commit which was a typo fix. Among the 4 who had voted 👎 , besides me one had multiple (although old) commits and one had a single small fix commit.
Having "community votes" to decide design disagreements is a really bad idea in my opinion, and definitely not something that we've decided to have.
There was a problem hiding this comment.
And out of the people who have taken their time to actually comment on the issue with arguments for the issue in question, none of them were arguing for defaulting the wait time to zero.
There was a problem hiding this comment.
You haven't really been around for so long, so I did not expect you to care, and when I saw the large support for the issue I went ahead and implemented it.
I dont agree that the votes supporting your point of view are worth 3 times the ones supporting mine. You interpret the lack of comments as a lack of support, when it is just as likely that they just found that my argumentation was reasonable and they had nothing more to add. Would it have been better if I had phrased myself less convincingly so that other people would fill in the gaps, thus adding to the comment count?
I agree that community votes to decide design disagreements is not good, but as a last resort I think it is better than nothing when the two people with the biggest "claim" on the project cannot agree (and tbh, our energy is better spent almost anywhere else :)
If we are going to continue discussing this, can I suggest we appoint some kind of neutral arbiter? Someone we both think is a reasonable person (maybe some contributor that has not yet weighed in on this issue), and whose decision we can both accept.
There was a problem hiding this comment.
You haven't really been around for so long, so I did not expect you to care
Well I do, and I have actively participated in arguing the issue in #1308.
I dont agree that the votes supporting your point of view are worth 3 times the ones supporting mine.
Like I said, I think having random votes decide design decisions is a really bad idea. I only commented on the vote because you brought it up. If I was to draw any conclusion from #1308, it would be that the only two people caring much about the issue is you and I.
You interpret the lack of comments as a lack of support, when it is just as likely that they just found that my argumentation was reasonable and they had nothing more to add
I really don't know how to interpret a bunch of thumbs up from random users, on a comment that only stated the arguments from one side of the issue. I would have liked for some counter-arguments to the arguments brought up by me. I think that would have been much more constructive.
and tbh, our energy is better spent almost anywhere else :)
I agree.
If we are going to continue discussing this, can I suggest we appoint some kind of neutral arbiter? Someone we both think is a reasonable person (maybe some contributor that has not yet weighed in on this issue), and whose decision we can both accept.
I think it would be great to hear the opinion of more contributors who haven't weighed in. Ideally, one of us could be convinced of the other's perspective, though I realize that might be hoping on too much :).
In the meantime, would you be okay with at least including wait_time in the Quickstart documentation? I think it's a very important aspect of Locust, and I'm convinced it makes the documentation better. I get that you don't think so, but maybe you could live with a small piece of "unnecessary" info in the introduction docs?
There was a problem hiding this comment.
really don't know how to interpret a bunch of thumbs up from random users, on a comment that only stated the arguments from one side of the issue. I would have liked for some counter-arguments to the arguments brought up by me. I think that would have been much more constructive.
Maybe all users did not read and understand the implications of the issue, but I think it is more likely than not that that people did read the ticket, as well as the followup reasoning by you and me. At least that is how I interpret the the 6 upvotes for my followup argument & clarification (and Anuj's counter-argument that got 3 upvotes).
In the meantime, would you be okay with at least including wait_time in the Quickstart documentation? I think it's a very important aspect of Locust, and I'm convinced it makes the documentation better. I get that you don't think so, but maybe you could live with a small piece of "unnecessary" info in the introduction docs?
I think the simplicity of beginner documentation is one of the key reasons for not forcing people to be explicit about wait times. Particularly I feel that forcing new users to learn & think about wait times before thinking about tasks is bad for useability/pedagogy (remember that I wanted to move wait_time to the end of the class in the quickstart example? you did have a good point about keeping attributes before functions though...)
But if you feel this strongly about it then yes, for the sake of moving on, we can put it back into quickstart. Maybe one of us will turn around completely later on, who knows :)
Sorry if come across as a stubborn idiot :)
|
|
||
| class MyUser(User): | ||
| wait_time = constant(1) |
There was a problem hiding this comment.
Again, using zero wait time in examples - especially if they're printing stuff - gives a horrible user experience for new users who copy & paste examples while familiarizing with Locust.
There was a problem hiding this comment.
Agreed, I tried to leave it in on those examples but missed this one. Fixed now in 81d9707
|
Hi! I think this is a strange default. I've used locust for a couple of projects and it has mostly been with a wait time set to simulate real user load. I can see that there are use cases for this, yet it feels more like a special case rather than the norm. I think we should optimize the examples towards new users of locust.
Having an enforced Apart from the docs confusion I think we should expect that people will not always think about what service they target or how many users they try with. Having a decent wait time in examples will prevent some unintended DOS. I did not voice my opinion on this before because it didn't seem like you agreed and I thought it wasn't going to be merged :) Maybe there should be a dedicated example locust file for load testing without wait. Also the If you don't like the extra import for |
|
Removing the need for an extra import makes it a little bit better, but kind of increases the complexity, and most importantly doesnt help with the "first time user" impact. Just reintroducing it to the quick start example as Jonathan suggested is better I think. Imho, it is still bad from a pedagogy stand point, and I still argue all the things I've said before are true - it is the obvious expected default behaviour, it is this way in almost every other tool (literally, I can't think of any other tool that forces you to specify a wait time), etc - but I can live with having it in one example (even if it is the most important one) We have other default behaviours that are much more arbitrary (ramp up 1 user/s in headless, no connection timeouts, no stop timeout, ignore proxy settings, etc) |
In this first section, there's a large risk that I just repeat stuff I've already said to you in chat @cyberw - sorry if that's the case, but it might as well live on github as well: I have no problem what so ever doing some things different than what other tools do - I'd argue that's what have made Locust successful in the first place. When we created Locust one of the main novel ideas was to generate realistic load by defining the behaviour of typical users. We intentionally did not take a thread or RPS-centric approach, but rather a PSU or user-centric approach. Having zero wait time definitely makes sense for tools that don't have a user centric approach. I think that's maybe the core of our disagreement. Whether the default Locust use-case is to have a wait time > 0.
All of those defaults are things that people would expect to be able to change because they are familiar concepts. The approach that Locust takes to simulate user behaviour is something that most other tools don't, and therefore wait time might not be a familiar concept, which is the exact reason why I think it's important to show it in an introduction to Locust (and why I keep arguing for having it mandatory).
If we were to change it, I think that's a good idea. |
|
If we are to keep it, we could add an info message to the log. Something like this: |
Agreed. And as a precursor to that, whether locust's main strength is being a general purpose, programmable/hackable load testing tool or if its strength is in having a different mental model than other tools. |
Of course I also think it's great that Locust is a general purpose, programmable/hackable load testing tool. Whether Borrowing some of your wording, maybe this could be a boiled down, neutral description of the disagreement: Whether it's a strength or weakness to have a different default mental model than other tools |
Yes. Or at least kind of. Whether it is a strength or weakness to have a different mental model, and that mental model is significantly weakened/obscured by having a default wait time of zero. |
Make wait_time = constant(0) the default instead of forcing our users to specify it on every user class.
This makes a big difference in readability of small User classes and reduces the cognitive overhead for locust beginners. Fixes #1308