AttributeError: 'NoneType' object has no attribute 'recv'

[freespy]

ERROR mqtt error 'NoneType' object has no attribute 'recv' 2020-08-06 23:12:47
ERROR Traceback (most recent call last):
  File "/data/dtuapi/dtuapi.py", line 117, in run
    self.client.loop_forever()
  File "/usr/local/lib/python2.7/site-packages/paho/mqtt/client.py", line 1787, in loop_forever
    rc = self.loop(timeout, max_packets)
  File "/usr/local/lib/python2.7/site-packages/paho/mqtt/client.py", line 1182, in loop
    rc = self.loop_read(max_packets)
  File "/usr/local/lib/python2.7/site-packages/paho/mqtt/client.py", line 1573, in loop_read
    rc = self._packet_read()
  File "/usr/local/lib/python2.7/site-packages/paho/mqtt/client.py", line 2276, in _packet_read
    byte = self._sock_recv(1)
  File "/usr/local/lib/python2.7/site-packages/paho/mqtt/client.py", line 660, in _sock_recv
    return self._sock.recv(bufsize)
AttributeError: 'NoneType' object has no attribute 'recv'

[valebes]

I have the same error.

[BadWolf42]

Hello,

I'm facing the same issue and could reproduce quite easily/often.
Enclosed code : paho-mqtt-issue505.py.txt.
Associated log files : 20210225-2310.log, 20210225-2315.log, 20210225-2317.log

Using Python 3.7.3 (default, Jan 22 2021, 20:04:44) and paho-mqtt version 1.5.1 on
Linux mydevhost 4.19.0-16-amd64 #1 SMP Debian 4.19.181-1 (2021-03-19) x86_64 GNU/Linux

I think this issue is coming from the fact that the MainThread destroys the Client object, which calls _reset_sockets(), then _sock_close() and sets self._sock to None...

But at the same time, self._sock is taken for granted bellow and is read by the run_forever() worker Thread:
https://github.com/eclipse/paho.mqtt.python/blob/c339cea2652a957d47de68eafb2a76736c1514e6/src/paho/mqtt/client.py#L662-L665

[BadWolf42]

Hello any update?

[russell-sealand]

I am also having the same issue, it seems related to this case: #345
The cause was that client.loop_forever() was being called in a multithreaded fashion, but apparently it only works in single thread mode.
Given you asked this question months ago, did you manage to find a solution? Thanks.

[russell-sealand]

I fixed the problem by switching to use client.loop_start() instead of client.loop_forever()

[lucasjinreal]

@russell-sealand its work not working, still get this error sometimes.

[russell-sealand]

@lucasjinreal please can you provide more information, for example, under what conditions do you get this error? I am unable to investigate without further information since this now works for me.

[jelbin313]

I was having this issue too. The exception is in loop_read, which checks that self._sock is not None before using it. Therefore, another thread must be setting self._sock to None while loop_read is running. The only function (besides __init__) that sets self._sock to None is sock_close, and this gets called when a disconnect packet is being written in _packet_write. I think the intention is that _packet_write only gets called in the loop_forever thread, because it is only called by loop_write, which should only be called by the loop_forever thread. However, disconnect will actually end up calling loop_write via _packet_queue when we are doing everything in a single thread, which it seems is assumed that we are doing when using loop_foever.
As far as I can tell, loop_forever is not intended to ever have client functions called from different threads, but, in order to disconnect, you have to call disconnect from another thread, since loop_forever is blocking and no other function can be called in the thread that is running loop_forever. (at least from my understanding) In order to disconnect cleanly, it seems like we basically have to use loop_forever in a multithreaded fashion, despite it not being meant for that. This is just my understanding of what is happening though, it may not be correct.

I was able to find a "hack" fix for this by setting the _thread attribute of the client to the current thread before calling loop_forever:

Client._thread = threading.current_thread()
Client.loop_forever(retry_first_connection=True)

This ensures that we do not call loop_write from the thread that is calling disconnect. However, you're definitely not supposed to access the _thread attribute of the client, as it is a protected attribute. This solution may cause unexpected errors as well, although it didn't cause any for me. (yet) If you want to use this solution, use at your own risk and test thoroughly. The real solution is that if you need to disconnect cleanly, you should probably just use loop_start. I'm just putting this here as an option for anyone having this issue who insists on using loop_forever instead of loop_start.

[PierreF]

With the sample code from #505 (comment) I'm able to reproduce the issue.

In that code sample, the issue is in stop() function we call publish() which only submit a packet (it don't yet send it) and we disconnect just after. This means two threads will concurrently run, the one trying to actually send the publish packet and the one doing the disconnection.
I totally agree the library should better handle this concurrency issue, at very least document clearly document what could be called concurrently.

That being said, since disconnect() do an immediate disconnection, it will means your last publish could be lost.

A fix for both the last publish that could be lost and the _sock that could be None could be:

	def stop(self):
		self.mqttclient.publish('mybroker'+str(self.id)+'/status', 'offline', 1, True).wait_for_publish()
		self.mqttclient.disconnect()
		self.mqttthread.join()

As @jelbin313 there is indeed some assumption in the library that (at least some) function are only called from the loop thread (loop_forever, loop_start...). Their seems to have a bit better handling when the loop thread is one from loop_start().

A review of concurrency access / adding some locks should probably be done, but this is a large amount of work.

As side node, disconnect() could be called from the thread doing the loop_forever (depending on application needs), if it's done from a callback like on_message, on_publish... But this really depends on application needs, not all application might be able to disconnect on this condition.