Improve documentation for external event loop

[ezramorris]

Related: #72

I think the documentation could do with some clarification/small additions for external event loop usage.

Whilst incoming MQTT data is quite self-explanatory (set the event loop to watch for read availability on the socket(), and then run loop_read()), the writing of data is less clear:

• Is there any need to watch the socket() for write availability, and call loop_write() as a result? (I'm guessing not, but the docstring for loop_write() seems to suggest otherwise.)
• What's the correct way to wait on new outgoing data? The two examples contributed in the above issue (1 and 2) take different approaches: the first just polls want_write() to determine whether to call loop_write(), and the second watches the internal _sockpairR socket. The second option seems to be more suited to external event loops*.

I would be willing to draft some documentation, with an example using asyncio, if these points could be clarified.

*On a side note, if this is the preferred option for waiting for new outgoing packets, should there be a public interface for this?

[downwith]

There's more than one non-public member in the second example. _state was being used too. I was thinking that using asyncio to manage this and other sockets would be piece of cake as well. Now I'm second guessing myself.

[joernheissler]

In my opinion, the existing documentation on external event loops is sufficient. That is, if you know in general how the select system call work. In a nutshell: You can ask the operating system if there is anything to read from a file descriptor or if there's enough buffer space for writing something to a file descriptor. And select will block until the answer is "yes" to at least one of the questions (or a timeout is hit).

Is there any need to watch the socket() for write availability, and call loop_write() as a result? (I'm guessing not, but the docstring for loop_write() seems to suggest otherwise.)

Yes, there is. There are 4 possible situations here: You want to write something, or you don't. There's space for writing or there isn't.

• There's space and you want to write: You should call loop_write.
• There's space and you don't want to write: loop_write will consume CPU time for nothing. Don't call it. And if there's nothing to do at all, you should be blocking or select until something interesting happens.
• There's no space but you want to write: Don't call loop_write. Wait (i.e. block on select) until there's space or you'll waste CPU time.
• There's no space and you don't want to write: Same.

What's the correct way to wait on new outgoing data?

The correct approach is to first determine if you want to write. If so, ask select if you can write. If so, do write.

The two examples contributed in the above issue (1 and 2) take different approaches: the first just polls want_write() to determine whether to call loop_write(), and the second watches the internal _sockpairR socket. The second option seems to be more suited to external event loops*.

Both examples are bad, see https://dev.eclipse.org/mhonarc/lists/paho-dev/msg03998.html

I would be willing to draft some documentation, with an example using asyncio, if these points could be clarified.

I was thinking that using asyncio to manage this and other sockets would be piece of cake as well. Now I'm second guessing myself.

I believe it's not possible to use asyncio here, without changing paho.mqtt first. See https://dev.eclipse.org/mhonarc/lists/paho-dev/msg03999.html for an idea to fix this. See also https://github.com/mossblaser/aiomqtt "Unfortunately the author was unable to work out how to integrate paho-mqtt's event loop into asyncio". The author is not to blame :-)

I'll try to write a correct example using https://docs.python.org/3/library/select.html#select.select for #72. This should suffice as documentation. And I'll try to implement changes so asyncio and other event loops can actually be used, and provide an example for this too.

[MattBrittan]

Closing this because #235 improved the situation (and this issue is now pretty stale!). If you believe this is still a problem please feel free to open a new issue with updated examples etc.