Unable to Pair Level Bolt

[krbaker]

I'm attempting to pair a level lock (bolt). It gets a bit into pairing (enough to ask me to enter the pairing code) and then it fails to read back this data. Interestingly the device is making the sound like its rebooting which it does not do when I pair it with an iphone.

2020-11-29 21:48:47,985 init.py:0127 DEBUG #3 ios -> accessory: send SRP verify request
Enter device pin (XXX-YY-ZZZ):
2020-11-29 21:49:00,920 init.py:0626 DEBUG entering write function...
2020-11-29 21:49:00,920 init.py:0637 DEBUG sent ...
2020-11-29 21:49:01,922 init.py:0646 DEBUG reading characteristic
2020-11-29 21:49:06,146 init.py:0646 DEBUG reading characteristic
2020-11-29 21:49:06,147 device.py:0109 DEBUG read failed: <homekit.controller.ble_impl.gatt.Characteristic object at 0x7fdd9eb73100> Not connected
object of type 'NoneType' has no len()
2020-11-29 21:49:06,147 pair_ble.py:0075 DEBUG object of type 'NoneType' has no len()
Traceback (most recent call last):
File "/home/syzygy/homekit_python/homekit/pair_ble.py", line 68, in
finish_pairing(pin_function())
File "/home/syzygy/homekit_python/homekit/controller/controller.py", line 505, in finish_pairing
response = write_fun(request, expected)
File "/home/syzygy/homekit_python/homekit/controller/ble_impl/init.py", line 644, in write
while len(data) == 0:
TypeError: object of type 'NoneType' has no len()

Happy to try and get more debugging (I stripped some of the data here obviously)

[jlusiardi]

Hi,
can you give me the name of the lock and a complete trace as gist? I can have a look then ;)

[krbaker]

Hi,
@ mentioned you on the gist of the full logs. You asked for the name of the lock. I'm not entirely sure what you mean. Its a Level Lock 'Bolt' ( https://level.co/products/bolt ). I included the BLE scan in the Gist which does say "Name: A043D7T1" if that is useful? The more I look at this the more I think the device is crashing when it gets the SRP verify request (it chimes like it does when you put the battery in, at first I thought this was a pairing chime). Now I've noticed that the device seems to lose BLE connection and that is why the read doesn't return data.
Thanks

[jlusiardi]

Sorry, I didn't parse level lock (bolt) as a product reference.

There is a slight chance, this might be already fixed in the master (pulled up the tlv8 dependency). Could you try this instead of version 0.18.0 (that I suspect you used)?

[krbaker]

Sent another gist, I think I got all of latest and greatest (using master & fresh vitrualenv to be sure). Tagging you there too

[jlusiardi]

1st Gist: https://gist.github.com/krbaker/f5ba428d3edfa42a58cd700dba8bf751#gistcomment-3556880
2nd Gist: https://gist.github.com/krbaker/bac10a603f44cff6e903bab5c12c3468#gistcomment-3560144

Both looks fine to me until the device drops the connection. As you said, it seems to reboot. So i guess the lock doesn't like how the software talks to it :/ perhaps it has something todo with the Pairing Type Flags mentioned in Revision 2 (page 34).

[los93sol]

I have this issue as well. I also attempted pairing with this lock using homebridge-ranger and get the same results. The lock seems to reboot during pairing.

[jlusiardi]

I seem I can't get hold of one of these in Europe i guess. Also pretty expensive Stuff. But without one to tinker around it seems pretty hard to work on this device.

[los93sol]

I wrote a console app to help debug this. It seems to be related to the size of the payload in M3. Anything over 244 bytes crashes the lock. I also played with breaking the payload down across fragmented PDU's and can prevent the lock from crashing but just get back InvalidMethod responses. Even more interesting, if I shorten the PublicKey value down to 383 bytes instead of 384 and fragment the PDU's, it properly makes it to the lock and I get the expected M4 response with an error that it failed to verify.

I'm at a bit of a loss on this, but I suspect this is more than likely related to ATT_MTU negotiation and likely the root cause of a lot of instability with various devices.

[jlusiardi]

can this be also an issue with the underlying bluetooth subsystem on linux here? this always occurred a little bumpy to me...

[los93sol]

I think so, I am stumped at this point. I wrote my console app in C# and used Windows UWP BLE packages since I’m not a python dev. Long story short, I got it to respond with M4 and a failed auth there so I thought maybe my SRP stuff was wrong.

Then I switched over to this lib and modified the Write method so break the request_tlv into segments and send the first with the normal PDU header and subsequent segments with the continuation PDU header. Firing them off immediately resulted in no crash but an InvalidRequest error from the device. Then I added a 100ms delay between them and now get back M4 with a failed auth error.

Another test I did was implementing the transient and split pairing flags and I see them coming back in M2 so I think it’s working with those as expected.

At this point I’ve effectively tried both Linux and Windows with different BLE stacks and am getting consistent results between both. I’d be curious if anyone has a device that is known to work that will test this code and make sure the device responds as expected. If you’re interested I can throw up a fork of my modifications and a repo with my console app.

[los93sol]

Here's the modification I made in homekit/controller/ble_impl/init.py

With this code I occasionally can get an M4 response out of the device, but 90% of the time it is just kicking back InvalidRequest errors in response to the M3 request. Hoping another set of eyes will see something I'm missing

def create_ble_pair_setup_write(characteristic, characteristic_id):
def write(request, expected):
# TODO document me
body = tlv8.encode(request)
logger.debug('entering write function %s', tlv8.format_string(tlv8.decode(body)))
request_tlv = tlv8.encode([
tlv8.Entry(AdditionalParameterTypes.ParamReturnResponse, bytearray(b'\x01')),
tlv8.Entry(AdditionalParameterTypes.Value, body)
])
transaction_id = random.randrange(0, 255)

    max = int(237)
    i = int(0)

    while len(request_tlv) > 0:
        if i == 0:
            data = bytearray([0x00, HapBleOpCodes.CHAR_WRITE, transaction_id])
            data.extend(characteristic_id.to_bytes(length=2, byteorder='little'))
            data.extend(len(request_tlv).to_bytes(length=2, byteorder='little'))
        else:
            time.sleep(1)
            data = bytearray([0x80, transaction_id])

        if len(request_tlv) < max:
            data.extend(request_tlv)
            request_tlv = []
        else:
            data.extend(request_tlv[:max])
            request_tlv = request_tlv[max:]

        logger.debug('sent %s', bytes(data).hex())
        last = len(request_tlv) == 0
        #logger.debug('last %s', last)

        characteristic.write_value(value=data)
        i += 1

[los93sol]

FYI, I did more testing and found that PairSetup needed to be 0x00, from the responses I observed I believe this means that the device does not have a coprocessor present. Anyways, the first trick was leveraging the continuation and the second trick was setting PairSetup properly. Does anyone know if it's possible to interrogate the device to determine how to pair with it systemically?

[jlusiardi]

Hey @krbaker
thanks a lot for your work. Could you tell me which value exactly you change during pairing?
Also can you use python3 -m homekit.discover_ble --adapter ${ADAPTER} --log DEBUG to find the unpaired bolt?

[los93sol]

@jlusiardi

discover_ble worked as expected and still does

Regarding the pair, I set this flag

homekit_python/homekit/protocol/methods.py

Line 20 in c7defa0

PairSetup = 1

to 0

The other piece of the puzzle was that this particular device negotiates an ATT_MTU of 247, so 3 bytes for the ATT header and 244 remaining for the payload is what was causing the device to reboot. I was able to catch that using btmon to see the negotiation start at 517 and drop to 247. That's where the code above came in, you don't have to implement TLV fragmentation, just PDU fragmentation which as you can see above is pretty trivial, it's really just break the full TLV payload apart, set the PDU header of the first payload as you normally would but make sure the length is the sum of all the individual payloads. Each payload after the first just set the continuation bit on the control byte and the transaction id then appends the payload and send it.

Hope this information proves helpful for further development

[jlusiardi]

I am not sure, but perhaps the discovery leads to some clues on how to figure out which 'mode' to use for pairing?

[Jc2k]

I've long suspected ff in discovery tells us which to use:

• kHAPCharacteristicValue_PairingFeatures_SupportsAppleAuthenticationCoprocessor = 1 << 0
• kHAPCharacteristicValue_PairingFeatures_SupportsSoftwareAuthentication = 1 << 1

See:

• https://github.com/apple/HomeKitADK/blob/4967f698bdcf0af122e13e986a2c9b595a68cdc5/HAP/HAPCharacteristicTypes.h#L865
• Pairing Issues: Eve Energy 1EC0 / LG C9 stuck at #5 ios -> accessory: send SRP exchange request #170

[los93sol]

@Jc2k From what I've seen that is correct

[los93sol]

I was able to confirm that you can read PairingFeatureFlags without being paired to the BLE device and the value does indeed tell you which way to pair to the device.

[jlusiardi]

That's pretty cool! Thanks to all of you.

First question: can this also affect IP based accessories or only those with BLE connection?

I would propose to add a parameter to pair_ble(and also to pair if affected) that specifies how the pairing should be performed. The information for this parameter should obtained by discover_ble(and also discoverif affected).

Second question: is this affecting also the communication after pairing?

[los93sol]

I'm pretty sure you'd only ever see this problem on BLE where the MTU size can be limited. AFAIK IP based MTU values will always be high enough that you'll never see this issue there.

I can't answer whether or not this affects communication after pairing since my project goals aren't actually to use this lib, but instead I'm building a BLE to IP gateway to re-expose these BLE devices as IP so that things like Home Assistant can pick them up and work with them as normal (of course with a little manual translation to create a bonjour TXT record out of the BLE advertisement)

[Jc2k]

The PairSetup issue should apply to both AIUI.

[Jc2k]

@los93sol i am the home assistant homekit _controller maintainer so happy to help you if I can. I do want native support for BLE upstream in ha but haven't had spare time to work on it for a while.

[los93sol]

@Jc2k Yes, PairSetup will and I can confirm that you're stuff is working as expected in HA. I proxy the messages over IP to Home Assistant and PairSetup/PairVerify all go as expected since your code is interpretting it properly. I'm stuck at the moment with getting my accessories API to be routable, for some reason the request from homekit_controller is not landing on my API.

I'd like to do both, native support and as a standalone Pi/Pi Zero 2 image to deal with range issues of BLE. In my case, I'm gonna need at least 2 of these "gateways" deployed

[Jc2k]

Yeah I'm having range issues with an Eve Extend right now, going to have to have multiple of them!

[los93sol]

@jlusiardi Sorry to hijack your project with my stuff!

@Jc2k I went ahead and published what I have so far https://github.com/los93sol/BleToIpGatewayTest
Unfortunately, I'm a C#/Windows developer so my project is .NET 6 targetting Windows specific BLE API's for now. If you're interested and have a windows box I believe it will get as far as homekit_controller trying to call the /accessories API before it vomits with your Eve device. Once this is working I plan on cleaning it all up and abstracting the BLE specific parts away so it's actually x-plat and can be run on a pi as well.

Side note: I currently have my device name hard coded in there so it doesn't blindly expose BLE devices to the network. You'll have to update that for it to even discover your Eve device.

[jlusiardi]

@los93sol you're welcome ;)

[los93sol]

@jlusiardi #199 resolves half of this, but not fully. The other half of this issue was that the M3 payload crashes the device if sent in one shot. It works and pairs of you break it up into multiple with a continuation PDU though