BlueZ Bluetooth Stack

[rickjansen-dev]

#64 non-breaking

still todo:

• more testing with more devices and examples
• test with multiple hubs at the same time
• set samples back to winrt bluetooth
• wait for new Tmds.DBus dependency version (new version has some important-ish changes)
• existing devices currently block discovery
• raspberry pi (4) issue: device disconnects on notify enable use bluez-firmware package version 1.2-4+rpt8

[tthiery]

Perfectly fine. No expert in DBus and/or bluez .. so trust it is anyway. code wise, everything fine.

[tthiery]

Awesome work in adding advertisement. We can make that even more prominent in the first section "Features" or however it is called.

Also - with your agreement - I will not be shy promoting your contribution about this.

[rickjansen-dev]

Sure!

[tthiery]

I think once we have this in the master, I will thinking about incorporating the Bluetooth selection into the core .AddPoweredUp() call within a later release. I mean, Windows will be always WinRT and Linux will be bluez realistically. Should be branchable and auto-addable.

[rickjansen-dev]

Excellent idea

[tthiery]

why adding it to the collection before querying the manufacturerdata

[rickjansen-dev]

well, this is related to using certain property changes (RSSI in this case) as the trigger for the discovery handler callback. It could very well be that the manufacturer data comes along with other properties directly during the discovery. If the device is not added to the list before the discovery handler is invoked, the GetDeviceAsync will be called and it would not be able to return the device because it's not in the list of devices.

I know this is not pretty, plus come to think of it, adding it to the list might actually need to be done before the call to Initialize(), i'll have to check that to make sure it always works

[dkurok]

I had the same problem during my use of BlueGiga-adapter: The advertising-packets from the BLE-device(Hubs) are coming in an "unordered" sequence. You've got to fetch them all until you're sure to have alos the manufacturer-data (because this is needed in Bluetooth-kernel of poweredup).

[rickjansen-dev]

I had the same problem during my use of BlueGiga-adapter: The advertising-packets from the BLE-device(Hubs) are coming in an "unordered" sequence. You've got to fetch them all until you're sure to have alos the manufacturer-data (because this is needed in Bluetooth-kernel of poweredup).

I'm not familiar with the bluegiga adapter, does that even use bluez? i'm unsure if unordered sequence of advertisement packets is an actual issue here

[dkurok]

No, the BlueGiga-adapter ("dongle") BLED112 is a serial-device attached to USB. The client (poweredup-program here) talks to it just over a Serial-object. So it doesn't need Bluez or any other BLE-stack in the operating system, because the device itself handles the BLE-communication (timing and alike)

[tthiery]

so you from here you are announcing it to the host (or whoever is using the bluetooth stack). I mean smart when there is no other handle on it.

Tough design otherwise ;)

[rickjansen-dev]

Yeah the difficulty is handling completely new and previously connected devices using bluez/dbus.

BlueZ will notify new devices (and services for that matter) being added, for example during discovery.
But for existing devices, this will never happen (unless you remove the device first).
Then, existing devices come in two flavors: ones that you have connected to since the last bluetooth daemon restart and ones that you connected to before that.

But invoking the discovery handler for existing devices is only supposed to happen when you turn on discovery and you actually turn on the device, not immediately, otherwise you would 'find' any hubs you have ever connected to directly and for example, most of the examples will try to connect to this hub immediately, when it might still be off.

So I figured I could rely on the manufacturer data, since that is sent during discovery, for existing devices. But the property changed event will only fire if it changes, so for existing devices that were connected during the same bluetooth daemon session (say, since last reboot) the manufacturer data will be there already. Therefore no change, not event, no discovery handler.
So the only thing dependable I found was to use the RSSI property. Now this might introduce some delays before connecting in some cases, but it seems to work fine.

So, long story short, yes this is far from pretty, but it's the only way I managed to make it work in all cases. And the root cause is the way bluez announces things using dbus - and that is by design apparently 😐

[tthiery]

Thanks. Mid-term i will migrate this to the main assembly

[tthiery]

That is now on the main repository with the new DeviceInfo abstractions.

[tthiery]

Any suspicion, when is this missing TMDS update incoming?

[rickjansen-dev]

Any suspicion, when is this missing TMDS update incoming?

It was released today :)

---

But during testing I found another issue. I've been developing on my laptop with Linux and everything now works smoothly. Today I also tested the same code on the raspberry pi (4), and the Technic Hub that i'm using for testing disconnects if I enable notifications on the characteristic. It is that specific call to StartNotifyAsync that will make the device disconnect instantly.
Now I found similar-ish reports of this issue in several places, some of them in the python lib issues (virantha/bricknil#21) but that's always kind of difficult to determine if their problems originate from their libs/lang use or if it's actually related
One user on the raspberry pi forums has a remarkably similar experience:
https://www.raspberrypi.org/forums/viewtopic.php?t=287467

I suspect this is an issue in the raspberry pi bluetooth firmware, which would kind of suck.

[rickjansen-dev]

I did some more testing on the raspberry pi, after stumbling across this issue: RPi-Distro/bluez-firmware#6

It's not exactly the same issue I am facing, but downgrading the bluetooth firmware on the pi did work. It seems to be very reliable when connecting to the device (albeit a bit slow). I'll do more testing maybe this weekend or next week.
Hopefully the new firmware for the pi's bluetooth stack will come soon and everything + my code will be stable.

[rickjansen-dev]

Some good news at last, the bluetooth firmware for the raspberry pi was updated twice since my last comment.
I updated the firmware to the latest released version (1.2-4+rpt8) today and everything seems to work, no sudden disconnects. I'm really happy with this. I'll do some more testing and update the PR against the latest master if everything works

[tthiery]

@justxi FYI

[justxi]

@tthiery Thanks.

[romarro]

how can I get this pull request in my fork, so I can play with it. I am really interested in the BlueZ implementation because I use a raspberrypi 3B?

[tthiery]

This is a pull request from an existing fork owned by @vuurbeving into the upstream repository (which this one is). So when you clone/fork this, you will not get it. However, the source of the pull request is here: https://github.com/vuurbeving/powered-up/tree/issue-64-bluez.

The most easiest solution: clone and pull that fork. However, with some git mastery, clone this, add a second git remote and pull the branch into your local clone. All possible ;)

Maybe you can support @vuurbeving a bit by testing it when you get it running.

[romarro]

I will try. Thanks

[tthiery]

Hi @vuurbeving

I downloaded your branch and compiled it on my Raspberry PI 4 after merging it with the latest and greatest. Works for the examples and the CLI (previously reported as failed but debugging server closet to bedroom is not so smart with a wireless device 😀).

Small hints

1. Update your source branch/repo to latest. Accept all incoming changes (the conflict above)
2. Fix Cli/Examples/TestScript projects by adding BlueZ in the csproj AND the serviceCollection Add ops next to BlueGigaBLE !!!
3. Fix the interface implementation (see WinRT variant for very simple shim variant)
   • There is a new interface IPoweredUpBluetoothDeviceInfo to be used instead of PoweredUpBluetoothDeviceInfo with a default implementation PoweredUpBluetoothDeviceInfoWithMacAddress
   • The existing property ulong BluetoothAddress is replaced with MacAddressAsUInt64
   • See the WinRT implementation. It uses the same objects. Fix by replacing it. Beauty later.
4. Compile on Windows regular, on Linux compile using dotnet build -f net5.0 -p:TargetFrameworks=net5.0 (this suppresses errors due to the multi-targeting) (or await Disable Multi-target on Linux #179)
5. Run on Linux using ./bin/Debug/net5.0/SharpBrick.PoweredUp.Examples --EnableTrace true --BluetoothAdapter BlueZ

I could create a PR on your PR but that is a bit crazy ;) ... UPDATE: See next comment

[tthiery]

This is changes as part of step (2)

commit 016dff317a392ae27e6837679d2d40a32dc38567
Merge: 6febc05 6e4e3dd
Author:
Date:   Sat May 8 21:11:01 2021 +0100

    Merge branch 'master' into issue-64-bluez

diff --cc examples/SharpBrick.PoweredUp.Examples/SharpBrick.PoweredUp.Examples.csproj
index fc99d6d,33bb82f..a1626a1
--- a/examples/SharpBrick.PoweredUp.Examples/SharpBrick.PoweredUp.Examples.csproj
+++ b/examples/SharpBrick.PoweredUp.Examples/SharpBrick.PoweredUp.Examples.csproj
@@@ -8,15 -8,23 +8,24 @@@
  
    <ItemGroup>
      <ProjectReference Include="..\..\src\SharpBrick.PoweredUp\SharpBrick.PoweredUp.csproj" />
-     <ProjectReference Include="..\..\src\SharpBrick.PoweredUp.WinRT\SharpBrick.PoweredUp.WinRT.csproj" />
+   </ItemGroup>
+ 
+   <ItemGroup Condition=" '$(TargetFramework)' == 'net5.0' ">
+     <ProjectReference Include="..\..\src\SharpBrick.PoweredUp.BlueGigaBLE\SharpBrick.PoweredUp.BlueGigaBLE.csproj" />
 +    <ProjectReference Include="..\..\src\SharpBrick.PoweredUp.BlueZ\SharpBrick.PoweredUp.BlueZ.csproj" />
    </ItemGroup>
  
+   <ItemGroup Condition=" '$(TargetFramework)' == 'net5.0-windows10.0.19041.0' ">
+     <ProjectReference Include="..\..\src\SharpBrick.PoweredUp.WinRT\SharpBrick.PoweredUp.WinRT.csproj" />
+     <ProjectReference Include="..\..\src\SharpBrick.PoweredUp.BlueGigaBLE\SharpBrick.PoweredUp.BlueGigaBLE.csproj" />
+   </ItemGroup>
+ 
    <ItemGroup>
-     <PackageReference Include="Microsoft.Extensions.DependencyInjection" Version="3.1.4" />
-     <PackageReference Include="Microsoft.Extensions.Logging" Version="3.1.4" />
-     <PackageReference Include="Microsoft.Extensions.Logging.Console" Version="3.1.4" />
-     <PackageReference Include="Microsoft.Windows.SDK.Contracts" Version="10.0.19041.1" />
+     <PackageReference Include="Microsoft.Extensions.Configuration.CommandLine" Version="5.0.0" />
+     <PackageReference Include="Microsoft.Extensions.Configuration.Json" Version="5.0.0" />
+     <PackageReference Include="Microsoft.Extensions.DependencyInjection" Version="5.0.1" />
+     <PackageReference Include="Microsoft.Extensions.Logging" Version="5.0.0" />
+     <PackageReference Include="Microsoft.Extensions.Logging.Console" Version="5.0.0" />
    </ItemGroup>
  
  </Project>

and then (3)

commit 2c11fa2ecec4bc62704216c5c453479803e7c315
Author: 
Date:   Sat May 8 21:33:56 2021 +0100

    Port BlueZ to .NET 5 and multi-targeting

diff --git a/examples/SharpBrick.PoweredUp.Examples/Program.cs b/examples/SharpBrick.PoweredUp.Examples/Program.cs
index 4c68aef..97c8ca9 100644
--- a/examples/SharpBrick.PoweredUp.Examples/Program.cs
+++ b/examples/SharpBrick.PoweredUp.Examples/Program.cs
@@ -81,6 +81,10 @@ namespace SharpBrick.PoweredUp.Examples
 #endif
 
 #if NET5_0_OR_GREATER
+            if (bluetoothAdapter == "BlueZ")
+            {
+                serviceCollection.AddBlueZBluetooth();
+            }
             if (bluetoothAdapter == "BlueGigaBLE")
             {
                 // config for "COMPortName" and "TraceDebug" (either via command line or poweredup.json)
diff --git a/src/SharpBrick.PoweredUp.BlueZ/BlueZPoweredUpBluetoothAdapter.cs b/src/SharpBrick.PoweredUp.BlueZ/BlueZPoweredUpBluetoothAdapter.cs
index 42e9b21..c48ffea 100644
--- a/src/SharpBrick.PoweredUp.BlueZ/BlueZPoweredUpBluetoothAdapter.cs
+++ b/src/SharpBrick.PoweredUp.BlueZ/BlueZPoweredUpBluetoothAdapter.cs
@@ -75,7 +75,7 @@ namespace SharpBrick.PoweredUp.BlueZ
         private async Task<bool> IsLegoWirelessProcotolDevice(IDevice1 device)
             => (await device.GetUUIDsAsync()).NullToEmpty().Any(x => x.ToUpperInvariant() == PoweredUpBluetoothConstants.LegoHubService);
 
-        public async void Discover(Func<PoweredUpBluetoothDeviceInfo, Task> discoveryHandler, CancellationToken cancellationToken = default)
+        public async void Discover(Func<IPoweredUpBluetoothDeviceInfo, Task> discoveryHandler, CancellationToken cancellationToken = default)
         {
             _adapter ??= await GetAdapterAsync();
 
@@ -88,7 +88,7 @@ namespace SharpBrick.PoweredUp.BlueZ
                     var poweredUpDevice = new BlueZPoweredUpBluetoothDevice(device, discoveryHandler);
                     await poweredUpDevice.Initialize();
 
-                    _devices.Add(poweredUpDevice.DeviceInfo.BluetoothAddress, poweredUpDevice);
+                    _devices.Add(poweredUpDevice.DeviceInfo.MacAddressAsUInt64, poweredUpDevice);
 
                     await poweredUpDevice.TryGetManufacturerDataAsync();
                 }
@@ -123,14 +123,16 @@ namespace SharpBrick.PoweredUp.BlueZ
 
                 await poweredUpDevice.Initialize();
                 
-                _devices.Add(poweredUpDevice.DeviceInfo.BluetoothAddress, poweredUpDevice);
+                _devices.Add(poweredUpDevice.DeviceInfo.MacAddressAsUInt64, poweredUpDevice);
 
                 await poweredUpDevice.TryGetManufacturerDataAsync();
             }
         }
 
-        public Task<IPoweredUpBluetoothDevice> GetDeviceAsync(ulong bluetoothAddress)
+        public Task<IPoweredUpBluetoothDevice> GetDeviceAsync(IPoweredUpBluetoothDeviceInfo bluetoothDeviceInfo)
         {
+            var bluetoothAddress = (bluetoothDeviceInfo is PoweredUpBluetoothDeviceInfoWithMacAddress local) ? local.MacAddressAsUInt64 : throw new ArgumentException("DeviceInfo not created by adapter", nameof(bluetoothDeviceInfo));
+
             if (!_devices.ContainsKey(bluetoothAddress))
             {
                 throw new ArgumentOutOfRangeException("Requested bluetooth device is not available from this adapter");
@@ -138,5 +140,12 @@ namespace SharpBrick.PoweredUp.BlueZ
 
             return Task.FromResult<IPoweredUpBluetoothDevice>(_devices[bluetoothAddress]);
         }
+
+        public Task<IPoweredUpBluetoothDeviceInfo> CreateDeviceInfoByKnownStateAsync(object state)
+            => Task.FromResult<IPoweredUpBluetoothDeviceInfo>(state switch
+            {
+                ulong address => new PoweredUpBluetoothDeviceInfoWithMacAddress() { MacAddressAsUInt64 = address },
+                _ => null,
+            });
     }
 }
diff --git a/src/SharpBrick.PoweredUp.BlueZ/BlueZPoweredUpBluetoothDevice.cs b/src/SharpBrick.PoweredUp.BlueZ/BlueZPoweredUpBluetoothDevice.cs
index 59cf0aa..e964fe9 100644
--- a/src/SharpBrick.PoweredUp.BlueZ/BlueZPoweredUpBluetoothDevice.cs
+++ b/src/SharpBrick.PoweredUp.BlueZ/BlueZPoweredUpBluetoothDevice.cs
@@ -10,14 +10,14 @@ namespace SharpBrick.PoweredUp.BlueZ
 {
     internal class BlueZPoweredUpBluetoothDevice : IPoweredUpBluetoothDevice
     {
-        private Func<PoweredUpBluetoothDeviceInfo, Task> _discoveryHandler;
+        private Func<IPoweredUpBluetoothDeviceInfo, Task> _discoveryHandler;
         private IDevice1 _device;
 
-        internal PoweredUpBluetoothDeviceInfo DeviceInfo { get; private set; } = new PoweredUpBluetoothDeviceInfo();
+        internal PoweredUpBluetoothDeviceInfoWithMacAddress DeviceInfo { get; private set; } = new PoweredUpBluetoothDeviceInfoWithMacAddress();
         internal bool Connected { get; private set; } = false;
         internal bool ServicesResolved { get; private set;} = false;
 
-        internal BlueZPoweredUpBluetoothDevice(IDevice1 device, Func<PoweredUpBluetoothDeviceInfo, Task> discoveryHandler = null)
+        internal BlueZPoweredUpBluetoothDevice(IDevice1 device, Func<IPoweredUpBluetoothDeviceInfo, Task> discoveryHandler = null)
         {
             _discoveryHandler = discoveryHandler;
             _device = device;
@@ -82,7 +82,7 @@ namespace SharpBrick.PoweredUp.BlueZ
         internal async Task GetSafeDeviceInfoAsync()
         {
             var btAddress = await _device.GetAddressAsync();
-            DeviceInfo.BluetoothAddress = Utilities.BluetoothAddressFormatter.ConvertToInteger(btAddress);
+            DeviceInfo.MacAddressAsUInt64 = Utilities.BluetoothAddressFormatter.ConvertToInteger(btAddress);
             DeviceInfo.Name = Name = await _device.GetNameAsync();
         }
 
diff --git a/src/SharpBrick.PoweredUp.BlueZ/SharpBrick.PoweredUp.BlueZ.csproj b/src/SharpBrick.PoweredUp.BlueZ/SharpBrick.PoweredUp.BlueZ.csproj
index adaa183..d05845e 100644
--- a/src/SharpBrick.PoweredUp.BlueZ/SharpBrick.PoweredUp.BlueZ.csproj
+++ b/src/SharpBrick.PoweredUp.BlueZ/SharpBrick.PoweredUp.BlueZ.csproj
@@ -1,13 +1,13 @@
 <Project Sdk="Microsoft.NET.Sdk">
 
   <PropertyGroup>
-    <TargetFramework>netstandard2.1</TargetFramework>
+    <TargetFramework>net5.0</TargetFramework>
   </PropertyGroup>
 
   <ItemGroup>
-    <PackageReference Include="Polly" Version="7.2.1" />
+    <PackageReference Include="Polly" Version="7.2.2" />
     <PackageReference Include="Tmds.DBus" Version="0.9.1" />
-    <PackageReference Include="System.Reactive" Version="4.4.1" />
+    <PackageReference Include="System.Reactive" Version="5.0.0" />
   </ItemGroup>
 
   <ItemGroup>
diff --git a/src/SharpBrick.PoweredUp.Cli/Program.cs b/src/SharpBrick.PoweredUp.Cli/Program.cs
index b547ab0..58b9c95 100644
--- a/src/SharpBrick.PoweredUp.Cli/Program.cs
+++ b/src/SharpBrick.PoweredUp.Cli/Program.cs
@@ -260,6 +260,10 @@ namespace SharpBrick.PoweredUp.Cli
 #endif
 
 #if NET5_0_OR_GREATER
+            if (bluetoothAdapter == "BlueZ")
+            {
+                serviceCollection.AddBlueZBluetooth();
+            }
             if (bluetoothAdapter == "BlueGigaBLE")
             {
                 // config for "COMPortName" and "TraceDebug" (either via command line or poweredup.json)
diff --git a/src/SharpBrick.PoweredUp.Cli/SharpBrick.PoweredUp.Cli.csproj b/src/SharpBrick.PoweredUp.Cli/SharpBrick.PoweredUp.Cli.csproj
index 281d422..b6d448f 100644
--- a/src/SharpBrick.PoweredUp.Cli/SharpBrick.PoweredUp.Cli.csproj
+++ b/src/SharpBrick.PoweredUp.Cli/SharpBrick.PoweredUp.Cli.csproj
@@ -14,6 +14,7 @@
 
   <ItemGroup Condition=" '$(TargetFramework)' == 'net5.0' ">
     <ProjectReference Include="..\..\src\SharpBrick.PoweredUp.BlueGigaBLE\SharpBrick.PoweredUp.BlueGigaBLE.csproj" />
+    <ProjectReference Include="..\..\src\SharpBrick.PoweredUp.BlueZ\SharpBrick.PoweredUp.BlueZ.csproj" />
   </ItemGroup>
 
   <ItemGroup Condition=" '$(TargetFramework)' == 'net5.0-windows10.0.19041.0' ">
diff --git a/test/SharpBrick.PoweredUp.TestScript/Program.cs b/test/SharpBrick.PoweredUp.TestScript/Program.cs
index a086382..3bdf376 100644
--- a/test/SharpBrick.PoweredUp.TestScript/Program.cs
+++ b/test/SharpBrick.PoweredUp.TestScript/Program.cs
@@ -36,6 +36,10 @@ namespace SharpBrick.PoweredUp.TestScript
 #endif
 
 #if NET5_0_OR_GREATER
+            if (bluetoothAdapter == "BlueZ")
+            {
+                serviceCollection.AddBlueZBluetooth();
+            }
             if (bluetoothAdapter == "BlueGigaBLE")
             {
                 // config for "COMPortName" and "TraceDebug" (either via command line or poweredup.json)
diff --git a/test/SharpBrick.PoweredUp.TestScript/SharpBrick.PoweredUp.TestScript.csproj b/test/SharpBrick.PoweredUp.TestScript/SharpBrick.PoweredUp.TestScript.csproj
index 56467c2..84eed46 100644
--- a/test/SharpBrick.PoweredUp.TestScript/SharpBrick.PoweredUp.TestScript.csproj
+++ b/test/SharpBrick.PoweredUp.TestScript/SharpBrick.PoweredUp.TestScript.csproj
@@ -13,6 +13,7 @@
 
   <ItemGroup Condition=" '$(TargetFramework)' == 'net5.0' ">
     <ProjectReference Include="..\..\src\SharpBrick.PoweredUp.BlueGigaBLE\SharpBrick.PoweredUp.BlueGigaBLE.csproj" />
+    <ProjectReference Include="..\..\src\SharpBrick.PoweredUp.BlueZ\SharpBrick.PoweredUp.BlueZ.csproj" />
   </ItemGroup>
 
   <ItemGroup Condition=" '$(TargetFramework)' == 'net5.0-windows10.0.19041.0' ">