Support for custom clients

[heyman]

Support for custom clients

This is a proposal for how we could implement official support for custom request/response based clients.

Reasoning

The reason we would like to do this is both to officially support testing of other request/response based systems than HTTP, but also to provide the ability to swap out the current requests based (http://docs.python-requests.org/) HTTP client, in favor for some other HTTP client. For example, if you're running extremely large load tests where you're doing tenths of thousands of requests per second, the overhead python-requests comes with can actually have a quite large impact.

Proposal

I've given this some thought, but it's most likely not optimal. However it's good to start somewhere, so you can see it as a starting point for a discussion on how to best implement this :).

One would specify the client class on the locust class(es) like this:

class User(Locust):
    client_class = ThriftClient
    ...

We would modify the Locust base class to something like this (which will allow one to either just set client_class on the Locust class, or override the get_client() method):

class Locust(...):
    ...
    def init(self, ...):
        self.client = self.get_client()

    def get_client(self):
        return self.client_class(self)
    ...

The client classes should then expect to get an instance of a Locust subclass to their init method (which can be used to read Locust.host etc.), and the client is also responsible for fire:ing request_success and request_failure events when it does requests (which of course should be clearly documented).

Finally, we should also change the HTTP specific labels that we have in the UI (I think it might only be "Method", which we could rename to "Type").

So, what do you think? ping @cgbystrom @Jahaja

[Jahaja]

Good proposal, I think this is a simple and good way to start.

I got a few comments and edge-cases though:

• How would we display multiple Locusts running at the same time but with different clients?
  What we currently got probably works, but it may need to be improved for clarity.
• Should we have a 1:1 relationship between a Locust class and a client? That is, with the suggested design one would need to split up mixed-client users. e.g:

class HttpLocust(Locust):
    client_class = HttpClient

class ThriftLocust(Locust):
    client_class = ThriftClient

A mixed version would require something like this:

class User(TaskSet):
    @task
    def index(self):
        self.get_client().get("/") # fetches default client (HTTP)
        self.get_client("thrift").someServiceCall()

So the question is rather; should we support mixed-client behaviours? This is obviously not a common use-case, but on the other hand I don't think it would polute things too much either.

• It would be good to be able to handle client-specific "meta-data" in a clean, but yet dynamic, way. Translating HTTP Method to the more generic Type is probably sufficent at the moment, but a more long-term solution for arbitrary differences could perhaps be found - without introducing too much complexity.

[mthurlin]

Do we really need the "client_class" attribute and the get_client() method?

Can't the implementor handle this on his own:

class MyLocust(Locust):
    def init(self, ...):
        self.myClient = MyClient()

(We can provide a default HttpLocust that does this, and a default HttpClient)

[cgbystrom]

Good idea!

+1 for @mthurlin's suggestion. I don't believe Locust should involved in the creation and managing of the client(s) needed to test a system. As long as the clients report back of what they do (num requests, response times, failure rate etc), we should be fine.

Adding a possibility for metadata as @Jahaja is suggestions sounds good. We currently are designing Locust around request/response based scenarios but that may expand so enabling room for metadata seems nice.
I don't that will harm the API design of the current HTTP focus that Locust has.

[heyman]

I think that the absolutely most common use case is still going to be HTTP testing. Therefore, I think it would be bad to make changes that in those cases would add some boilerplate code like:

class MyLocust(Locust):
    def init(self, *args, **kwargs):
        super(MyLocust, self).__init__(*args, **kwargs)
        self.client = HttpClient(self)

One solution to this would be that we would continue to automatically supply each Locust instance with an instance of HttpSession under the client attribute. However, this would make the HTTP client implementation "special", and all other clients "second class citizens", which I don't think would be good either. Therefore I'm still leaning more to the idea where we provide some simple helper method (and attribute that defaults to the HTTP client) that can be easily overridden.

As to the mixed client issue, one could still use the (slightly hackish) method of just instantiating an additional client in the init method for those rare cases.

For the metadata I think that it could be good to start by keeping it simple and just use the Type (change name from method) as the available metadata. Unless we have a good idea on how it should work, and some real use-cases it would solve.

[mthurlin]

Why not have Locust as a clean slate (no client), and then provide an HttpLocust like your example?

class HttpLocust(Locust):
    def init(self, *args, **kwargs):
        super(HttpLocust, self).__init__(*args, **kwargs)
        self.client = HttpClient(self)

No boilerplate necessary for end-users, but HTTP doesn't become "special" (except for the fact that we provide a default implementation).

[heyman]

That's true. 😊 🌴

+1 from me too then :)

[EnTeQuAk]

A big 👍 from me for the

class HttpLocust(Locust):
    def init(self, *args, **kwargs):
        super(HttpLocust, self).__init__(*args, **kwargs)
        self.client = HttpClient(self)

solution and having a Locust alias for backwards compatibility (if we need this).

Making the overwrite of self.client on init is also preferred instead of adding a client_class that can be swapped. The latter one might be easier sometimes but explicitly initiating the client class gives multiple advantages like officially show how to forward custom arguments to the client.

[heyman]

Fixed!

[irshad-qb]

@heyman / @cgbystrom :Could you please give me an example or explanation on how to write locust load test with custom client ( WebSocket Server in my case ). I saw the explanation given in locust documentation but I dint get how exactly the functions__getattr__ and def wrapper(*args, **kwargs): which hooks locust events are getting triggered via locust.