MacOS: scanning in new thread raises BleakError("Bluetooth device is turned off")

[dhalbert]

• bleak version: 0.6.4
• Python version: 3.8.2
• Operating System: MacOS 10.15.4

Trying to do scanning in a new thread raises BleakError("Bluetooth device is turned off"). Run the program below with and without --thread. In the latter case, the error is raised, but the program works fine with everything in the same thread.

Using BleakScanner rather than discover has the same problem.

This program runs fine on Linux with BlueZ, and on Windows 10 with Python 3.7.7 and pythonnet.

This sounds sort of like #93 and #125, but those were BlueZ problems. This stackoverflow Q&A may also be of interest, though I don't know corebluetooth well enough to confirm: https://stackoverflow.com/questions/48958267/how-can-i-use-corebluetooth-for-python-without-giving-up-the-main-thread

The reason I'm using a second thread is that I'm re-implementing an existing non-async BLE API using bleak. I was using a fresh thread for the bleak async side of things, and a janus queue to pass back scanning results.

import asyncio
from bleak import discover
from threading import Thread
import argparse

parser = argparse.ArgumentParser()
parser.add_argument('--thread', action='store_true')
args = parser.parse_args()

async def scan():
    devices = await discover()
    for d in devices:
        print(d)

def run():
    asyncio.run(scan(), debug=True)

if args.thread:
    print("in separate thread")
    t = Thread(target=run)
    t.start()
    t.join()
else:
    print("in main thread")
    loop = asyncio.get_event_loop()
    loop.run_until_complete(scan())

[cspensky]

I have seen this same behavior on my Mac, trying to solve a similar problem.

…

On Mon, May 25, 2020 at 10:28 AM Dan Halbert ***@***.***> wrote: - bleak version: 0.6.4 - Python version: 3.8.2 - Operating System: MacOS 10.15.4 - BlueZ version (bluetoothctl -v) in case of Linux: Trying to do scanning in a new thread raises BleakError("Bluetooth device is turned off"). Run the program below with and without --thread. In the latter case, the error is raised, but the program works fine with everything in the same thread. Using BleakScanner rather than discover has the same problem. This program runs fine on Linux with BlueZ. This sounds sort of like #93 <#93> and #125 <#125>, but those were BlueZ problems. This stackoverflow Q&A may also be of interest, though I don't know corebluetooth well enough to confirm: https://stackoverflow.com/questions/48958267/how-can-i-use-corebluetooth-for-python-without-giving-up-the-main-thread The reason I'm using a second thread is that I'm re-implementing an existing non-async BLE API using bleak. I was using a fresh thread for the bleak async side of things, and a janus queue to pass back scanning results. import asynciofrom bleak import discoverfrom threading import Threadimport argparse parser = argparse.ArgumentParser()parser.add_argument('--thread', action='store_true')args = parser.parse_args() async def scan(): devices = await discover() for d in devices: print(d) def run(): asyncio.run(scan(), debug=True) if args.thread: print("in separate thread") t = Thread(target=run) t.start() t.join()else: print("in main thread") loop = asyncio.get_event_loop() loop.run_until_complete(scan()) — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#206>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAVJBSN3OS2TJEPYOV7Z2M3RTKTFHANCNFSM4NJVTWUQ> .

[hbldh]

Relates to #105. On macOS , one should try to avoid using several threads/event loops in bleak, since the use of a global CentralManagerDelegate will produce problems. Run the bleak programs in different processes if possible and it should work.

This needs to be put into the documentation about the macOS backend.

[LTKI]

I have the same problem. Even when using a different process for bleak, I get the same error. I tried both "spawn" and "fork". Do I understand it correctly that it is therefore not possible to execute bluetooth actions in the background of your program, e.g. waiting for a BLE notify while showing a loading bar to the user/running a gui? Maybe you could point us in the right direction for a fix?

[dhalbert]

More data: even just a new event loop (as you implied/mentioned) causes the problem, without using a new thread. Using asyncio.run(), which creates a new event loop, instead of loop.run_until_complete() on the existing event loop, causes the same problem.

import asyncio
from bleak import discover
from threading import Thread
import argparse

parser = argparse.ArgumentParser()
parser.add_argument('--thread', action='store_true')
parser.add_argument('--run_until_complete', action='store_true')
parser.add_argument('--run', action='store_true')
args = parser.parse_args()

async def scan():
    devices = await discover()
    for d in devices:
        print(d)

def run():
    asyncio.run(scan(), debug=True)

if args.thread:
    print("in separate thread")
    t = Thread(target=run)
    t.start()
    t.join()

if args.run_until_complete:
    print("run_until_complete in main thread")
    loop = asyncio.get_event_loop()
    loop.run_until_complete(scan())

if args.run:
    print("run in main thread")
    asyncio.run(scan())

[dhalbert]

Looking at backends/corebluetooth/__init__.py, I see that the CBAPP singleton of class Application is created on import, and that it grabs the current event loop.

bleak/bleak/backends/corebluetooth/__init__.py

Lines 26 to 27 in 9c87ac0

	def __init__(self):
	self.main_loop = asyncio.get_event_loop()

,

I think a simple fix would be to lazily create the singleton when it's actually referenced for the first time, and have it use either the current event loop at that time, or perhaps optionally the loop passed in to BleakClient, etc. That would accommodate the simple cases in the comments above where a single new event loop is created for all bleak operations.

I'll see if I can work up a PR for this. I see several un-merged corebluetooth PR's, but they don't appear to change how the singleton is created.

[bsiever]

@dhalbert I tried a few variations of the "changing which thread owns the singleton" approach without much success. Unfortunately I didn't have much time and can't be sure of the cause of the failure. Good luck! I hope you're able to find a solution

[superfashi]

I'm encountering the same issue. Is there a reason to use a global CentralManagerDelegate instead of initializing an Application class for each client?

[bsiever]

@superfashi I think a manager per client can probably work ok based on some threads in Apple's forums.

Right now both discovery and clients use the shared Application. There are some minor potential benefits, like allowing peripherals to be cached for faster connecting, although this isn't done in the released version. There would be other ways around this too, like having a client's connect() do a scan and connect as soon as a peripheral is discovered rather than time-limited discovery or pass a peripheral object from the discovery to the client.

[superfashi]

The following code doesn't even seem to work, which I found really confusing:

import asyncio

async def run():
    from bleak import discover
    print(await discover())

loop = asyncio.get_event_loop()
loop.run_until_complete(run())

[hbldh]

This issue will be worked on 2020-06-30, so any additional information that can be added here before that is very welcome.

[dhalbert]

This issue will be worked on 2020-06-30, so any additional information that can be added here before that is very welcome.

I'm not sure how useful this would be, but I'll explain the context for my original report. We are using bleak to (partially) reimplement _bleio , an existing API for CircuitPython. CircuiPython is a friendly fork of MicroPython, a mostly bare-metal implementation of Python for microcontrollers. _bleio was originally designed for the nRF52840 native SoftDevice BLE stack. But we also want to run code written for CircuitPython on CPython on host computers, so we have created various wrapper libraries that go by the name "Blinka", in this case Adafruit_Blinka_bleio. I have Blinka _bleio running on x64 and Raspbian Linux, and Windows 10. Getting it to run on MacOS will complete _bleio support on the three major host platforms, in addition to the original CircuitPython implementation. I am also starting on third implementation of _bleio for direct use of HCI-capable co-processors.

The more identically things work on the these four platforms, the better. We want a single program to be runnable across all four. Obviously some things don't work, such as acting as a peripheral. We also have issues with scanning: the bluez kernel support does de-duplication of advertisements, which is undesirable when advertising is used for changing data transmissions, so we have to use privileged hcitool and hcidump right now on Linux for scanning, and fall back to bleak only if they are not available.

_bleio is not meant for end-user use: instead we have a nicer wrapper library, adafruit_ble. The examples directory there contains simple examples, and then we also have a large number of individual libraries that support various devices.

We are very grateful for bleak, which has allowed us to provide this cross-platform support without having to deal with the native libraries on each platform.

[dlech]

I had the same error as the title of this issue without using threads. The changes I made in pybricks@26ab00f and pybricks@0b37282 fixed the issue for me. But ideally it would be nice to get rid of the global CBAPP object. It causes other os-specific quirks like #111.

(@dhalbert in case you haven't seen, it looks like BlueZ is getting better advertisement support: https://github.com/bluez/bluez/blob/master/doc/advertisement-monitor-api.txt - might be good to make sure it does what is needed while it is still experimental)

[cspensky]

I also ran into this issue and fought with it quite a bit. I was able to get the error to go away after some hackery, but was never actually able to achieve full functionality of bleak in a thread. Happy to help any way I can, this is also a high priority item for me. I was planning to knock it out in July if you don't come up with a fix by then.

…

On Wed, Jun 24, 2020 at 8:24 AM David Lechner ***@***.***> wrote: I had the same error as the title of this issue without using threads. The changes I made in ***@***.*** <pybricks@26ab00f> and ***@***.*** <pybricks@0b37282> fixed the issue for me. But ideally it would be nice to get rid of the global CBAPP object. It causes other os-specific quirks like #111 <#111>. ***@***.*** <https://github.com/dhalbert> in case you haven't seen, it looks like BlueZ is getting better advertisement support: https://github.com/bluez/bluez/blob/master/doc/advertisement-monitor-api.txt - might be good to make sure it does what is needed while it is still experimental) — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#206 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAVJBSMFLIGY5LGJL7BTGTLRYILDHANCNFSM4NJVTWUQ> .

-- *Chad Spensky* *Founder and CEO* *Allthenticate, Inc.*

[dlech]

FYI, I've added pybricks@1b04383 and pybricks@f11a2e1 to our fork which gets the example program in #206 (comment) working in the thread.

Still a bit of work left to do to reconcile it with the other proposes changes to the corebluetooth backend, but it probably wouldn't hurt to try it out if anyone is interested.

[hbldh]

I have tested this issue both with threads and with event loops after merging #227 to develop and it seems to be a solid solution. This will be released later today.

[hbldh]

@dhalbert I have looked at _bleio; very nice! It would be great if bleak could cater nicely to that kind of environment as well! Hitherto, it has been focused on desktop only, with some at least support for Raspberry Pi.

The scanning issue that you are mentioning, could you please open a separate issue for that, because that is something I did not know about. I have been getting PropertiesChanged notifications in from BlueZ during scanning, but that might not yield new data, only updated RSSI values?