Pairing Issues: Eve Energy 1EC0 / LG C9 stuck at #5 ios -> accessory: send SRP exchange request #170

jlusiardi · 2019-08-30T08:55:07Z

In #125 and #158 there are reports that Eve Energy 1EC0 and LG C9 are both stuck at the same problem. This seems to influence both transports.

Citing @Jc2k:

To me, this looks like a completely normal M1/M2/M3 exchange, then the TV abruptly returns nothing / closes the connection instead of sending us an M4. There are no extra fields or new reserved flags suddenly being used in the M2 to indicate we should behave differently. I wondered if they rolled out a new pairing scheme (different salt? pub key?) There is no new spec, though. That likely means it failed the SRP proof? But that kind of error should (according to the spec) yield an M4 response with a kTLVType_Error.

python3 -m homekit.pair -d XX:XX:XX:XX:XX:XX -a mytv -f pairing.json --log DEBUG
2019-08-27 05:57:08,812 __init__.py:0106 DEBUG #1 ios -> accessory: send SRP start request
2019-08-27 05:57:08,813 tlv.py:0134 DEBUG sending [
  6: (1 bytes/<class 'bytearray'>) 0x01
  0: (1 bytes/<class 'bytearray'>) 0x01
]

2019-08-27 05:57:08,813 tlv.py:0117 DEBUG receiving [
  6: (1 bytes/<class 'bytearray'>) 0x01
  0: (1 bytes/<class 'bytearray'>) 0x01
]

2019-08-27 05:57:08,813 __init__.py:0060 DEBUG write message: [
  6: (1 bytes/<class 'bytearray'>) 0x01
  0: (1 bytes/<class 'bytearray'>) 0x01
]

2019-08-27 05:57:09,023 tlv.py:0117 DEBUG receiving [
  6: (1 bytes/<class 'bytearray'>) 0x02
  3: (384 bytes/<class 'bytearray'>) 0xa4716b8b96d60a92ff6c28c7d00085cd420aaccea34530f300c42f0d3776bc21ee2885e3abcaf2ef4d8466b45801080968384833463ee96ac72edcff2b3b7d506aa16d50524b203874e52f4ca4ff807d24c39c1807ad95f28c12788a930afe8f33aa19fafe42f99f481d9adae48341c75ce4d1e65b2c9f7a34a6e0bdbe0cecfcb017c373557adb7a5d8d470edea65913b877924b92721774b4e325c9c7e3bb988c8cf366308393f4b1efa70a29feff34587079bf103227e8786a9cd0a89ef871ae17ba84d03c9e88d1929ea4df010cd50bbec6a108b582d3c3123b9b81a15aee636fb7f9d1a637c775966d0331f2338eb600f1e22c0f2b46eb27eec9cc80af12d3ad28de55473283a4673ed661f929d7053318720e00ec233725f89b795a3ef393f441eddf1e6998dc31e032e16a53c8052eec5b3699feb3eb3fac89ffdc7d30bc1bd4f47d7779c0ee3736280785053b0a0e32fb272e9c4972e25d6c73349fb43c1eada186ce0c539b8648858b185880fa6195590d9519f70af5eed1832978e3
  2: (16 bytes/<class 'bytearray'>) 0x691039f6b6c8067498ff6166cc336825
]

2019-08-27 05:57:09,024 __init__.py:0067 DEBUG response: [
  6: (1 bytes/<class 'bytearray'>) 0x02
  3: (384 bytes/<class 'bytearray'>) 0xa4716b8b96d60a92ff6c28c7d00085cd420aaccea34530f300c42f0d3776bc21ee2885e3abcaf2ef4d8466b45801080968384833463ee96ac72edcff2b3b7d506aa16d50524b203874e52f4ca4ff807d24c39c1807ad95f28c12788a930afe8f33aa19fafe42f99f481d9adae48341c75ce4d1e65b2c9f7a34a6e0bdbe0cecfcb017c373557adb7a5d8d470edea65913b877924b92721774b4e325c9c7e3bb988c8cf366308393f4b1efa70a29feff34587079bf103227e8786a9cd0a89ef871ae17ba84d03c9e88d1929ea4df010cd50bbec6a108b582d3c3123b9b81a15aee636fb7f9d1a637c775966d0331f2338eb600f1e22c0f2b46eb27eec9cc80af12d3ad28de55473283a4673ed661f929d7053318720e00ec233725f89b795a3ef393f441eddf1e6998dc31e032e16a53c8052eec5b3699feb3eb3fac89ffdc7d30bc1bd4f47d7779c0ee3736280785053b0a0e32fb272e9c4972e25d6c73349fb43c1eada186ce0c539b8648858b185880fa6195590d9519f70af5eed1832978e3
  2: (16 bytes/<class 'bytearray'>) 0x691039f6b6c8067498ff6166cc336825
]

2019-08-27 05:57:09,024 __init__.py:0118 DEBUG #3 ios -> accessory: send SRP verify request
Enter device pin (XXX-YY-ZZZ): 739-34-736
2019-08-27 05:57:33,755 tlv.py:0134 DEBUG sending [
  6: (1 bytes/<class 'bytearray'>) 0x03
  3: (384 bytes/<class 'bytearray'>) 0x73520bb9c3cd416bdf68955569dd1fb506dd851b8ce3ae34a4dfcc2f48edf953bfceccb9ea02fcf10cab8d24cc94b50835b23672debf0a4bf2b35dc030627c8e7060cd1414af3d89e0fc060aa432924a3c18fbf4b77bb703937fb3b55c7c50469b641ee21c532620a20ed7b5ee1f714fe24a33c988ea63f33e3749f0fe23335fe919ba9bec6c2ffbd911f4ee9d970e2ca1225a69350b403168eae727b71af6ad573b9a7f145a441c6da73b1d29c934598d46debd44d6696eb935d98ee0f0db298f051a487ee37334a77538e5668718a9aeb8121603f9dedbddbfa129a7f153fdb4f97602504eb5dee5094b8dc5123bb6354c319f6e1d2bf7429ca95359177dec0b5afbd11155bd07d66678e59f636355a56111c08e306e082bcf0855aeb51cb965d7de65f3618249a3674780c1d690169e71a7e4d627a11ad1a110514fa7368e0289bd8f0fdee7abec1420b15bcf9e46f0a41a547d9873dc3a42accb0cbb613b5ac33fa0e938ca65c42dd18c71e27735a3f2cef1e9d6b085d9eeca84ec790e8d
  4: (64 bytes/<class 'bytearray'>) 0xba082ff088e539e3176a90ce49eb042f5ec1af70796dd72d9f39fe679b84f99848e3f423bdcadf9d7d28d07d509588c791e4f93f55b0cbea0b4957eec741b628
]

2019-08-27 05:57:33,755 tlv.py:0117 DEBUG receiving [
  6: (1 bytes/<class 'bytearray'>) 0x03
  3: (384 bytes/<class 'bytearray'>) 0x73520bb9c3cd416bdf68955569dd1fb506dd851b8ce3ae34a4dfcc2f48edf953bfceccb9ea02fcf10cab8d24cc94b50835b23672debf0a4bf2b35dc030627c8e7060cd1414af3d89e0fc060aa432924a3c18fbf4b77bb703937fb3b55c7c50469b641ee21c532620a20ed7b5ee1f714fe24a33c988ea63f33e3749f0fe23335fe919ba9bec6c2ffbd911f4ee9d970e2ca1225a69350b403168eae727b71af6ad573b9a7f145a441c6da73b1d29c934598d46debd44d6696eb935d98ee0f0db298f051a487ee37334a77538e5668718a9aeb8121603f9dedbddbfa129a7f153fdb4f97602504eb5dee5094b8dc5123bb6354c319f6e1d2bf7429ca95359177dec0b5afbd11155bd07d66678e59f636355a56111c08e306e082bcf0855aeb51cb965d7de65f3618249a3674780c1d690169e71a7e4d627a11ad1a110514fa7368e0289bd8f0fdee7abec1420b15bcf9e46f0a41a547d9873dc3a42accb0cbb613b5ac33fa0e938ca65c42dd18c71e27735a3f2cef1e9d6b085d9eeca84ec790e8d
  4: (64 bytes/<class 'bytearray'>) 0xba082ff088e539e3176a90ce49eb042f5ec1af70796dd72d9f39fe679b84f99848e3f423bdcadf9d7d28d07d509588c791e4f93f55b0cbea0b4957eec741b628
]

2019-08-27 05:57:33,755 __init__.py:0060 DEBUG write message: [
  6: (1 bytes/<class 'bytearray'>) 0x03
  3: (384 bytes/<class 'bytearray'>) 0x73520bb9c3cd416bdf68955569dd1fb506dd851b8ce3ae34a4dfcc2f48edf953bfceccb9ea02fcf10cab8d24cc94b50835b23672debf0a4bf2b35dc030627c8e7060cd1414af3d89e0fc060aa432924a3c18fbf4b77bb703937fb3b55c7c50469b641ee21c532620a20ed7b5ee1f714fe24a33c988ea63f33e3749f0fe23335fe919ba9bec6c2ffbd911f4ee9d970e2ca1225a69350b403168eae727b71af6ad573b9a7f145a441c6da73b1d29c934598d46debd44d6696eb935d98ee0f0db298f051a487ee37334a77538e5668718a9aeb8121603f9dedbddbfa129a7f153fdb4f97602504eb5dee5094b8dc5123bb6354c319f6e1d2bf7429ca95359177dec0b5afbd11155bd07d66678e59f636355a56111c08e306e082bcf0855aeb51cb965d7de65f3618249a3674780c1d690169e71a7e4d627a11ad1a110514fa7368e0289bd8f0fdee7abec1420b15bcf9e46f0a41a547d9873dc3a42accb0cbb613b5ac33fa0e938ca65c42dd18c71e27735a3f2cef1e9d6b085d9eeca84ec790e8d
  4: (64 bytes/<class 'bytearray'>) 0xba082ff088e539e3176a90ce49eb042f5ec1af70796dd72d9f39fe679b84f99848e3f423bdcadf9d7d28d07d509588c791e4f93f55b0cbea0b4957eec741b628
]

2019-08-27 05:57:33,848 tlv.py:0117 DEBUG receiving [
]

2019-08-27 05:57:33,849 __init__.py:0067 DEBUG response: [
]

2019-08-27 05:57:33,849 __init__.py:0169 DEBUG #5 ios -> accessory: send SRP exchange request
list index out of range
2019-08-27 05:57:33,849 pair.py:0085 DEBUG list index out of range
Traceback (most recent call last):
  File "/Users/blake/src/homekit/homekit/pair.py", line 78, in <module>
    finish_pairing(pin_function())
  File "/Users/blake/src/homekit/homekit/controller/controller.py", line 391, in finish_pairing
    pairing = perform_pair_setup_part2(pin, str(uuid.uuid4()), write_fun, salt, pub_key)
  File "/Users/blake/src/homekit/homekit/protocol/__init__.py", line 173, in perform_pair_setup_part2
    assert response_tlv[0][0] == TLV.kTLVType_State and response_tlv[0][1] == TLV.M4, \
IndexError: list index out of range

The text was updated successfully, but these errors were encountered:

niemyjski · 2019-10-03T15:44:38Z

Is there any updates on this? If I can help test anything please let me know.

Jc2k · 2019-10-03T16:20:37Z

There are some other pairing variants in the latest spec, maybe this doesn't support the original pairing variant for some reason.

@niemyjski bit of an ask but are you able to get a pcap dump of traffic flowing between an iOS device and your TV whilst pairing?

niemyjski · 2019-10-04T12:21:30Z

If you can tell me exactly what I'd need to do I'd be more than happy to do it. Thanks -Blake Niemyjski

…

On Thu, Oct 3, 2019 at 11:20 AM Jc2k ***@***.***> wrote: There are some other pairing variants in the latest spec, maybe this doesn't support the original pairing variant for some reason. @niemyjski <https://github.com/niemyjski> bit of an ask but are you able to get a pcap dump of traffic flowing between an iOS device and your TV whilst pairing? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#170?email_source=notifications&email_token=AAHZFI3MPN2RH7IS72CZ4PDQMYLVNA5CNFSM4ISK7IA2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAIX74Y#issuecomment-538017779>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAHZFI4H5IGVMYFHTUSV7KDQMYLVNANCNFSM4ISK7IAQ> .

Jc2k · 2019-10-04T13:05:01Z

It really depends on your network setup, hardware and is.

If you have a desktop or laptop that can run wireshark you might just be able to run your wlan card in monitor mode and tell it to look for traffic between your iOS ip and your tv ip. This can be very OS and hardware dependent and I haven’t done it in a good long time. As your iOS device will be WiFi based this should work even if the tv is Ethernet. I don’t know how well this works on Windows, but I’ve definitely had success with it on Linux. Not sure about Mac - think I’ve seen it work there but could be misremembering.

If your tv is connected by Ethernet and you have a switch which supports mirroring or you have a dumb hub then you can watch traffic flowing to and from the tv by running wire shark on a physical Ethernet port.

You should start by installing wireshark and seeing if you can see your local WiFi traffic.

niemyjski · 2019-10-04T17:34:58Z

I'm running all on ubiquity and the tv and Apple TV are hard wired. I'm on Mac and can run Wireshark on the same network. Last time I tried to sniff HomeKit traffic I didn't have much luck so pointers welcomed. Thanks -Blake Niemyjski

…

On Fri, Oct 4, 2019 at 8:05 AM Jc2k ***@***.***> wrote: It really depends on your network setup, hardware and is. If you have a desktop or laptop that can run wireshark you might just be able to run your wlan card in monitor mode and tell it to look for traffic between your iOS ip and your tv ip. This can be very OS and hardware dependent and I haven’t done it in a good long time. As your iOS device will be WiFi based this should work even if the tv is Ethernet. I don’t know how well this works on Windows, but I’ve definitely had success with it on Linux. Not sure about Mac - think I’ve seen it work there but could be misremembering. If your tv is connected by Ethernet and you have a switch which supports mirroring or you have a dumb hub then you can watch traffic flowing to and from the tv by running wire shark on a physical Ethernet port. You should start by installing wireshark and seeing if you can see your local WiFi traffic. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#170?email_source=notifications&email_token=AAHZFI3E5UEEKXA52AG5U5DQM45P5A5CNFSM4ISK7IA2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEALSNWI#issuecomment-538388185>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAHZFI5GVYLHMXFHTOSYAHDQM45P5ANCNFSM4ISK7IAQ> .

jlusiardi · 2019-10-07T05:33:30Z

Hey,
I guess your mac, the TV and the Apple TV are on the same wired network, but connected via a switch?

A switch is more intelligent than an Ethernet hub, which simply retransmits packets out of every port of the hub except the port on which the packet was received, unable to distinguish different recipients, and achieving an overall lower network efficiency.
(see https://en.wikipedia.org/wiki/Network_switch#Overview)

Perhaps https://wiki.wireshark.org/CaptureSetup/Ethernet#Switched_Ethernet will explain further and can help solve this issue.

-Joachim Lusiardi

Ulrar · 2019-10-07T08:10:16Z

I have HA running on a pi, I'm guessing a tcpdump file from the pi while trying to pair the TV is what you're looking for ? You can open that up in wireshark to look at the packets.
As mentioned in the other issue though I have an LG C9, not an appel TV, but I suppose it's the same problem.

I'm attaching the pcap file, it was running from the moment I clicked "configure" in HA to after the error.
capture.zip

Jc2k · 2019-10-07T08:14:19Z

We could do with a capture between a iOS device and the tv ideally so we can see if it is using one of the new pairing handshakes.

Ulrar · 2019-10-07T08:19:38Z

I don't have any iOS devices so I can't help with that, sorry

…

On Mon 7 Oct 2019, 09:14 Jc2k, ***@***.***> wrote: We could do with a capture between a iOS device and the tv ideally so we can see if it is using one of the new pairing handshakes. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#170?email_source=notifications&email_token=AAGVHT7JSUDKDHIVTBOK3NDQNLVVZA5CNFSM4ISK7IA2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAPMPBI#issuecomment-538888069>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAGVHT3DT6QGU37M3YNSJ7DQNLVVZANCNFSM4ISK7IAQ> .

Ulrar · 2019-10-25T14:22:24Z

Just remembered my girlfriend does have a mac laptop. Googling it looks like that won't do though, is that correct ? No way to use a macos laptop to initiate pairing and capture what you'd need ?

jlusiardi · 2019-10-26T06:27:39Z

Hi,

I found https://appleinsider.com/articles/18/06/05/hands-on-controlling-your-smart-home-with-homekit-on-macos-mojave which states from MacOS Mojave onwards this should be possible. But adding new devices is not possible in Mojave. Perhaps this was improved in MacOS Catalina.

@netmanchris, I saw your cool analysis from https://www.youtube.com/watch?v=y5L21gjEyC4 do you have a Catalina enabled Mac? Does it support pairing?

Regards
Joachim

jzee · 2020-01-05T23:10:31Z

Facing the exact same symptoms and logs on an LG TV and looked for the latest protocol spec.

It appears Apple gives out R2 to non-commercial users. I'm not sure what you @Jc2k have been working with but here's an implementation against R14 uploaded a few days ago. Can you spot anything obvious in the handling of M4? (start at line 445)

Jc2k · 2020-01-06T22:01:17Z

Our pairing code was written against R1, and one thing i've been meaning to dig into was whether the Transient and Split pairing modes that are lightly touched on in R2 of the spec could be at play. In a transient pairing there is no M5 or M6, but that only happens if we set kTLVType_Flags.

In addition, if you trigger split pairing then you can reuse an SRP code. I think you can create a secure transient session and then do additional auth over the transient session to finish the pairing?

Can you paste the discovery output for your device? I'm interested in whether the affected devices have an ff flag set to 2.

https://github.com/apple/HomeKitADK/blob/d1174d26f4986f66402b37b46d7239599e1533b3/HAP/HAPIPServiceDiscovery.c#L86

https://github.com/apple/HomeKitADK/blob/d1174d26f4986f66402b37b46d7239599e1533b3/HAP/HAPAccessoryServer.c#L862

Suggests there are 2 possible flags for ff:

kHAPCharacteristicValue_PairingFeatures_SupportsAppleAuthenticationCoprocessor - 1 << 0
kHAPCharacteristicValue_PairingFeatures_SupportsSoftwareAuthentication - 1 << 1

However i'm sure an earlier bug report was showing an '8' in the ff field which i'm struggling to match to the R14 code.

Some other relevant bits here:

https://github.com/apple/HomeKitADK/blob/d1174d26f4986f66402b37b46d7239599e1533b3/HAP/HAPPairingPairSetup.c#L204

Also we need to figure out kHAPPairingMethod_PairSetupWithAuth vs kHAPPairingMethod_PairSetup.

Jc2k · 2020-01-06T22:20:10Z

https://github.com/apple/HomeKitADK/blob/d1174d26f4986f66402b37b46d7239599e1533b3/HAP/HAPPairingPairSetup.c#L597-L805

I don't think we currently do any of that.

Or this:

https://github.com/apple/HomeKitADK/blob/d1174d26f4986f66402b37b46d7239599e1533b3/HAP/HAPPairingPairSetup.c#L204-L235

jlusiardi · 2020-01-11T13:08:21Z

There seems to be a new way to pair HomeKit accessories present in R2 of
the HAP Specification.

Let's do some comparing of R1 vs. R2.

For easier navigation use the next table:

section	Revision 1	Revision 2
start of pairing	Chapter 4.7 page 39	Chapter 5.6 page 34
definition of methods	Table 4-4 page 60	Table 5-3 page 49
pairing type flags	N/A	Table 5-7 page 51
TLV Values	Table 4-6 page 61	Table 5-6 page 49
BLE Protocol Version Characteristic	02.01.00 (Chapter 6.4.3.1, page 126)	02.02.00 (Chapter 7.4.3.1, page 121)
IP Protocol Version	N/A?	Chapter 6.6.3 page 61 mentions PV from Discovery must be `1.1`.

There seems to be a difference in the values of the methods relevant for
pairing:

Value	Method in R1	Method in R2
0	Reserved	Pair Setup
1	Pair Setup	Pair Setup with Auth

Our code still sends a value of 1 but without given a value for
kTLVType_Flags. This particular key was introduced in R2 with a value
of 0x13.

So I am not sure on how to proceed to be compatible with devices that
work with R1 and Eve Energy 1EC0 / LG C9 from this issue. I will do some
more analysis later.

Is one of the owners (@jzee, @Ulrar, @niemyjski) willing to try to witch
the value of PairSetup to 0:

diff --git a/homekit/protocol/tlv.py b/homekit/protocol/tlv.py
index 153d7b2..b2d3a54 100644
--- a/homekit/protocol/tlv.py
+++ b/homekit/protocol/tlv.py
@@ -32,7 +32,7 @@ class TLV:
     M6 = bytearray(b'\x06')
 
     # Methods (see table 4-4 page 60)
-    PairSetup = bytearray(b'\x01')
+    PairSetup = bytearray(b'\x00')
     PairVerify = bytearray(b'\x02')
     AddPairing = bytearray(b'\x03')
     RemovePairing = bytearray(b'\x04')

jzee · 2020-01-11T15:22:36Z

Did that, modified the constant straight in the installation. Here's the log output. If you DM me the instructions of how to get to a debugging setup so I can run python3 -m homekit.pair -d XX:XX:XX:XX:XX:XX -a mytv -f pairing.json --log DEBUG or the like from the console i can maybe do some more investigations.

Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/components/homekit_controller/config_flow.py", line 247, in async_step_pair await self.hass.async_add_executor_job(self.finish_pairing, code) File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run result = self.fn(*self.args, **self.kwargs) File "/usr/local/lib/python3.7/site-packages/homekit/controller/controller.py", line 391, in finish_pairing pairing = perform_pair_setup_part2(pin, str(uuid.uuid4()), write_fun, salt, pub_key) File "/usr/local/lib/python3.7/site-packages/homekit/protocol/__init__.py", line 173, in perform_pair_setup_part2 assert response_tlv[0][0] == TLV.kTLVType_State and response_tlv[0][1] == TLV.M4, \ IndexError: list index out of range

jzee · 2020-01-11T15:37:20Z

Something just happened, it worked! Without having a lot of information how to debug the install properly, I went into log_support.py and hardcoded the call to getLogger().setLevel(5), then I restarted HA. Note that the "PairSetup" was still set to 0, as @jlusiardi suggested.

The TV shows as paired now and reads out the firmware version properly. I have yet to find out what more I can do with it now but it is definitely paired.

jlusiardi · 2020-01-11T15:46:45Z

ok, great!

can you execute

python -m homekit.get_accessories -a mytv -f pairing.json --log DEBUG and
python -m homekit.discover --log DEBUG
and post the results?

I guess we must find a way to get the version of the protocol before starting a pairing...

jzee · 2020-01-11T16:15:31Z

python -m homekit.get_accessories -a mytv -f pairing.json --log DEBUG

doesn't find pairing.json. I grep'ed through the HA docker instance and the host, there is no such file. Where do I find it?

bash-5.0# python -m homekit.discover --log DEBUG
2020-01-11 17:12:21,959 init.py:0099 DEBUG candidate data {b'md': b'Home Assistant Bridge', b'pv': b'1.0', b'id': b'XX:XX:XX:XX:XX:XX', b'c#': b'2', b's#': b'1', b'ff': b'0', b'ci': b'2', b'sf': b'0', b'sh': b'eJ71Fg=='}
2020-01-11 17:12:21,962 init.py:0107 DEBUG found Homekit IP accessory {'name': 'Home Assistant Bridge._hap._tcp.local.', 'address': '192.168.xx.xx', 'port': 51827, 'c#': '2', 'ff': 0, 'flags': 'No support for HAP Pairing', 'id': 'XX:XX:XX:XX:XX:XX', 'md': 'Home Assistant Bridge', 'pv': '1.0', 's#': '1', 'sf': '0', 'statusflags': 'Accessory has been paired.', 'ci': '2', 'category': 'Bridge'}
2020-01-11 17:12:21,963 init.py:0099 DEBUG candidate data {b'c#': b'1', b'ff': b'8', b'id': b'YY:YY:YY:YY:YY:YY', b'md': b'WEBOS', b'pv': b'1.1', b's#': b'1', b'sf': b'0', b'ci': b'31', b'sh': b'wu5QmQ=='}
Traceback (most recent call last):
File "/usr/local/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/usr/local/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.7/site-packages/homekit/discover.py", line 51, in
results = Controller.discover(args.timeout)
File "/usr/local/lib/python3.7/site-packages/homekit/controller/controller.py", line 90, in discover
return discover_homekit_devices(max_seconds)
File "/usr/local/lib/python3.7/site-packages/homekit/zeroconf_impl/init.py", line 102, in discover_homekit_devices
info.properties
File "/usr/local/lib/python3.7/site-packages/homekit/zeroconf_impl/init.py", line 180, in parse_discovery_properties
d['category'] = Categories[int(category)]
File "/usr/local/lib/python3.7/site-packages/homekit/model/categories.py", line 94, in getitem
raise KeyError('Item {item} not found'.format(item=item))
KeyError: 'Item 31 not found'
bash-5.0#

Jc2k · 2020-01-11T16:32:28Z

I'm guessing you are using Home Assistant's web front end to pair rather than the homekit_python projects CLI? (HA doesn't use a pairing.json and wont understand category 31 yet)

jzee · 2020-01-11T16:34:48Z

that's correct, I'm unaware of how to use the homekit's CLI. I can have a look at a clean install of it, any pointers are appreciated.

Jc2k · 2020-01-11T16:38:41Z

There's a file in configuration/.storage/homekit_controller-entity-map that is similar to (has all the same data as) the output of homekit.get_accessories. Review it before posting - it may have e.g. your TV's serial number.

I can try and tell you how to get the pairing data out of HA when i get back home tonight/tomorrow.

jzee · 2020-01-11T16:47:58Z

that should be it

homekit_controller-entity-map-cleansed.txt

jzee · 2020-01-11T17:01:05Z

I just ran the discover from the CLI client after walking through the installation and it coughs at cat 31 as well, with an identical traceback. I checked out master

This is not backed by any documentation or specification but by #170

jlusiardi · 2020-01-11T17:54:27Z

from homekit_controller-entity-map-cleansed.txt:

{
                                "characteristics": [
                                    {
                                        "ev": false,
                                        "format": "string",
                                        "iid": 18,
                                        "perms": [
                                            "pr"
                                        ],
                                        "type": "00000037-0000-1000-8000-0026BB765291",
                                        "value": "1.1.0"
                                    }
                                ],
                                "hidden": false,
                                "iid": 16,
                                "linked": [],
                                "primary": false,
                                "stype": "service",
                                "type": "000000A2-0000-1000-8000-0026BB765291"
                            },

which states a version of 1.1.0. So far so good. But I think we should have the version information before pairing.

@jzee: I just pushed a new commit with an added category 31 (as TV). Can you pull and try again?

Ulrar · 2020-01-11T18:18:52Z

I just got home and haven't paired my TV at all, happy to help if I can.
Not sure how to pull the latest version of homekit_python into home assistant though, should I just git clone the repo and copy the files or is there an easier way ?

Ulrar · 2020-01-11T19:49:21Z

I can confirm that switching the PairSetup to 0 and then restarting home assistant makes pairing work.
Now unfortunately home assistant doesn't support TVs on this homekit integration, so it doesn't do anything, but at least it got rid of the persistent notification.

Jc2k · 2020-01-11T20:42:59Z

Off topic for here, but @Ulrar can you share your configuration/.storage/homekit_controller-entity-map too? The more TV's I can see the soon/better the HA TV code will arrive/be

Ulrar · 2020-01-11T20:53:53Z

There it is. Not sure what I should or shouldn't edit out so I just removed the mac, serial number and what I assume must be the key.
Let me know if you need anything else, thanks !

https://gist.github.com/Ulrar/30a9fe78e962895cce65413982e2ceaf

jzee · 2020-01-11T21:31:48Z

@jlusiardi this commit fixed it. Here's the discovery output

$ python3 -m homekit.discover --log DEBUG
2020-01-11 22:30:26,390 init.py:0100 DEBUG candidate data {b'md': b'Home Assistant Bridge', b'pv': b'1.0', b'id': b'XX:XX:XX:XX:XX:XX', b'c#': b'2', b's#': b'1', b'ff': b'0', b'ci': b'2', b'sf': b'0', b'sh': b'eJ71Fg=='}
2020-01-11 22:30:26,390 init.py:0108 DEBUG found Homekit IP accessory {'name': 'Home Assistant Bridge._hap._tcp.local.', 'address': '192.168.XX.XX', 'port': 51827, 'c#': '2', 'ff': 0, 'flags': 'No support for HAP Pairing', 'id': 'XX:XX:XX:XX:XX:XX', 'md': 'Home Assistant Bridge', 'pv': '1.0', 's#': '1', 'sf': '0', 'statusflags': 'Accessory has been paired.', 'ci': '2', 'category': 'Bridge'}
2020-01-11 22:30:26,391 init.py:0100 DEBUG candidate data {b'c#': b'1', b'ff': b'8', b'id': b'XX:XX:XX:XX:XX:XX', b'md': b'WEBOS', b'pv': b'1.1', b's#': b'1', b'sf': b'0', b'ci': b'31', b'sh': b'wu5QmQ=='}
2020-01-11 22:30:26,391 init.py:0108 DEBUG found Homekit IP accessory {'name': 'LG webOS TV A716._hap._tcp.local.', 'address': '192.168.XX.XX', 'port': 37517, 'c#': '1', 'ff': 8, 'flags': 'No support for HAP Pairing', 'id': 'XX:XX:XX:XX:XX:XX', 'md': 'WEBOS', 'pv': '1.1', 's#': '1', 'sf': '0', 'statusflags': 'Accessory has been paired.', 'ci': '31', 'category': 'TV'}
Name: Home Assistant Bridge._hap._tcp.local.
Url: http_impl://192.168.XX.XX:51827
Configuration number (c#): 2
Feature Flags (ff): No support for HAP Pairing (Flag: 0)
Device ID (id): XX:XX:XX:XX:XX:XX
Model Name (md): Home Assistant Bridge
Protocol Version (pv): 1.0
State Number (s#): 1
Status Flags (sf): Accessory has been paired. (Flag: 0)
Category Identifier (ci): Bridge (Id: 2)

Name: LG webOS TV A716._hap._tcp.local.
Url: http_impl://192.168.XX.XX:37517
Configuration number (c#): 1
Feature Flags (ff): No support for HAP Pairing (Flag: 8)
Device ID (id): XX:XX:XX:XX:XX:XX
Model Name (md): WEBOS
Protocol Version (pv): 1.1
State Number (s#): 1
Status Flags (sf): Accessory has been paired. (Flag: 0)
Category Identifier (ci): TV (Id: 31)

Jc2k · 2020-01-11T22:17:09Z

So pv is 1.1 for the LG webOS TV, and we have to use PairSetup==0. But looking at my home network both my Hue and Eve Extend are pv 1.1 and work with the existing code. So I don't know if its enough to key off pv.

Both the hue and Eve Extend have ff of 1 and the TV has an ff of 8.

Looking at all the working IP devices in tested_devices, they are a mixture of protocol 1.0 and 1.1 and all have ff 1.

From reading the code @jzee posted, ff 1 seems to mean there is a hardware security chip.

So maybe:

PairSetupWithAuth (1) works if ff is 1
PairSetup (0) works with if ff doesn't have bit 1 set

jlusiardi · 2020-01-12T16:39:23Z

@Jc2k sadly there is not much information about the ff value of the bonjour record in the R2 Spec. It references a table which is pretty useless.

Jc2k · 2020-01-12T17:20:40Z

Yes its weird that there is a table in R2 but its empty.

In the official ADK the bonjour data for ff is set here. It uses the function HAPAccessoryServerGetPairingFeatureFlags to get the value. We can see that the public Apple code confirms the values of kHAPCharacteristicValue_PairingFeatures_SupportsAppleAuthenticationCoprocessor and kHAPCharacteristicValue_PairingFeatures_SupportsSoftwareAuthentication for ff and their values are defined here. So we know what the value means, but I guess we don't know what it implies for the protocol. We are none the wiser about what ff of 8 means though.

I think it makes sense that 1 works for PairSetup only with the hardware encryption chip - they are largely the first wave of devices and all we've seen till recently. I guess the "Auth" is that the device proves it has an Apple crypto chip?

I suppose we could make a mock accessory that set ff to 8 and pv to 1.1 and see if that changes what an official client does for the encryption handshake.

Jc2k · 2020-01-14T22:21:50Z

Another user from HA reported issues pairing. They have a VOCOlinc L3 bulb.

Instead of this exception in their case the pairing process hung after M3 but before the asset / IndexError.

This device has an ff of 2 and pv of 1.1. The same patch as above allowed pairing to work - they were able to pair through the HA front end as well.

This is another case where ff & SupportsAppleAuthenticationCoprocessor was a predictor for whether to use PairSetup or PairSetupWithAuth.

Jc2k · 2020-01-14T22:34:18Z

Trying to find something we can use in lieu of spec...

This is part of the official Apple HAP ADK for PairSetupWithAuth:

https://github.com/apple/HomeKitADK/blob/d1174d26f4986f66402b37b46d7239599e1533b3/HAP/HAPPairingPairSetup.c#L612-L623

My reading of this code is that an accessory made with that spec would return an kHAPError_InvalidState if you tried to do an M4 on a device that didn't have a Mfi Hardware security co-processor. We also know from the other code I linked to that an ff of 1 means that a device does have such a chip, according to the Apple example code at least.

In other words, if we tried to pair with an accessory written with the ADK we would have to have logic that did:

if ff & SupportsAppleAuthenticationCoprocessor:
    PairSetupWithAuth()
else:
    PairSetup()

I.e. we should only attempt PairSetupWithAuth if we detect a security chip, because we know that ADK devices fail otherwise.

jzee · 2020-01-14T23:26:39Z

@Jc2k building a mock accessory is actually what the ADK repo comes with out of the box (try “make Target=Linux Apps” on a Linux VM) - so doing that exercise is fairly easy. I spent the better part of Sunday fiddling with it and getting it on the network showing up as a node on iOS is trivial.

I still failed on getting it to use a setup code so I could get further down the auth process - out of the box the example accessories fail since they aren’t automatically provided with a setup code. If someone can shed some light on how that could work, I can dig down a little further.

jlusiardi · 2020-01-15T07:05:44Z

I think we should collect a table of devices here... Perhaps different (firmware)versions of the same device have different behaviors?

Feel free to add devices:
https://docs.google.com/spreadsheets/d/17KzVh7pLqO6R0eqJUiliJCvOMSgqTjuYDYmBdGhaNvs/edit?usp=sharing

Jc2k · 2020-01-15T08:09:16Z

I used demoaccessory and twiddled the pv and ff bonjour data, and captured the TLV_Method attribute when pairing with an official iOS 13 client.

Results:

pv 1.0
ff 0
method 0

pv 1.0
ff 1
method 1

pv 1.1
ff 1
method 1

pv 1.1
ff 0
method 0

pv 1.0
ff 8
method 1

pv 1.0
ff 2
method 1

An official client will use PairSetup = 0 if ff is 0 (and this is what triggers the uncertified device warning also).

Annoyingly though ff 8 does use the failing version of PairSetup, so i suspect that using PairSetup = 0 is actually working by accident rather than design?

jlusiardi added IP Transport Issues related to IP based Accessory BLE Transport Issues related to BLE based Accessory labels Aug 30, 2019

This was referenced Aug 30, 2019

IP Discovery is not working on MacOs #158

Closed

Eve Energy 1EC0 - (list index out of range) #125

Closed

Jc2k mentioned this issue Oct 3, 2019

homekit_controller: unable to complete pair_setup with LG TV home-assistant/core#25960

Closed

jlusiardi added a commit that referenced this issue Jan 11, 2020

add category for TVs

bbfe9d7

This is not backed by any documentation or specification but by #170

Jc2k mentioned this issue Jan 14, 2020

Homekit device is discovered but can't be configured home-assistant/core#30592

Closed

belkinhappyzj mentioned this issue May 16, 2020

[Pairing Accessory] About kTLVType_Method used in Pair Setup M1 #199

Open

Jc2k mentioned this issue Aug 5, 2020

conflicted pair method types values #204

Closed

Jc2k mentioned this issue Dec 17, 2021

Unable to Pair Level Bolt #211

Closed

Pairing Issues: Eve Energy 1EC0 / LG C9 stuck at #5 ios -> accessory: send SRP exchange request #170

Pairing Issues: Eve Energy 1EC0 / LG C9 stuck at #5 ios -> accessory: send SRP exchange request #170

Comments

jlusiardi commented Aug 30, 2019

niemyjski commented Oct 3, 2019

Jc2k commented Oct 3, 2019

niemyjski commented Oct 4, 2019 via email

Jc2k commented Oct 4, 2019

niemyjski commented Oct 4, 2019 via email

jlusiardi commented Oct 7, 2019

Ulrar commented Oct 7, 2019

Jc2k commented Oct 7, 2019

Ulrar commented Oct 7, 2019 via email

Ulrar commented Oct 25, 2019

jlusiardi commented Oct 26, 2019

jzee commented Jan 5, 2020

Jc2k commented Jan 6, 2020

Jc2k commented Jan 6, 2020

jlusiardi commented Jan 11, 2020 • edited Loading

jzee commented Jan 11, 2020

jzee commented Jan 11, 2020

jlusiardi commented Jan 11, 2020

jzee commented Jan 11, 2020 • edited Loading

Jc2k commented Jan 11, 2020

jzee commented Jan 11, 2020

Jc2k commented Jan 11, 2020

jzee commented Jan 11, 2020

jzee commented Jan 11, 2020

jlusiardi commented Jan 11, 2020

Ulrar commented Jan 11, 2020

Ulrar commented Jan 11, 2020

Jc2k commented Jan 11, 2020

Ulrar commented Jan 11, 2020

jzee commented Jan 11, 2020 • edited Loading

Jc2k commented Jan 11, 2020

jlusiardi commented Jan 12, 2020

Jc2k commented Jan 12, 2020 • edited Loading

Jc2k commented Jan 14, 2020

Jc2k commented Jan 14, 2020

jzee commented Jan 14, 2020

jlusiardi commented Jan 15, 2020

Jc2k commented Jan 15, 2020

jlusiardi commented Jan 11, 2020 •

edited

Loading

jzee commented Jan 11, 2020 •

edited

Loading

jzee commented Jan 11, 2020 •

edited

Loading

Jc2k commented Jan 12, 2020 •

edited

Loading