Extending this to work with active iPhones

[sct1000]

I'm happy to report that with the latest fixes for Series 5, this is doing a great job for detecting presence using our Apple Watches.

I noticed a couple of things. If you have two watches, and one is detected but out-of-range, its RSSI will pollute the reported status.

I was also curious to see if this could be tweaked to detect active/unlocked iPhones in range doing their own Continuity broadcasts, and sure enough, I have this working with an iPhone 11 Pro, an SE 2020, and an XR, if there's a mac OS machine somewhere nearby, presumably sending out its own broadcasts. Maybe we can simulate those?

FWIW, the combined changes from my yaml:

esp32_ble_tracker:
  scan_parameters:
    interval: 1.2s
    window: 500ms
    active: false
  on_ble_advertise:
    - then:
      - lambda: |-
          optional<int16_t> best_rssi = nullopt;
          for (auto data : x.get_manufacturer_datas()) {
            // Guard against non-Apple datagrams, or those that are too small.
            if (data.data.size() < 4 || data.uuid.to_string() != "0x004C" || data.data[0] != 0x10 || data.data[1] < 5) {
              continue;
            }
            const int16_t rssi = x.get_rssi();
            const uint8_t ac = data.data[2];
            const uint8_t sf = data.data[3];
            if (sf == 0x98) {  // Match Apple Watches
              if (ac <= 0x0F) {
                best_rssi = max(rssi, best_rssi.value_or(rssi));
                ESP_LOGD("ble_adv", "Found Apple Watch (mac %s) rssi %i", x.address_str().c_str(), rssi);
              } else {
                ESP_LOGD("ble_adv", "Possible Apple Watch? (mac %s) rssi %i, unrecognised action code %#04x", x.address_str().c_str(), rssi, ac);
              }
            } else if ((ac & 0xf0) == 0x70) { // Match unlocked iPhones?
                best_rssi = max(rssi, best_rssi.value_or(rssi));
                ESP_LOGD("ble_adv", "Found active iPhone (mac %s) rssi %i", x.address_str().c_str(), rssi);
            }
          }
          if (best_rssi) {
            id(apple_watch_rssi).publish_state(*best_rssi);
          }

[dalehumby]

I noticed a couple of things. If you have two watches, and one is detected but out-of-range, its RSSI will pollute the reported status.

Great catch! I was wondering how to solve this issue over the weekend, and you beat me to it. I'll merge your changes in.

Regarding detecting active/unlocked iPhones:

Did you discover (ac & 0xf0) == 0x70 through experimentation? Looking at page 18, it looks like the key flags are "Primary device" and "AirDrop receiving", which makes sense. I guess primary device will always (?) be a phone, and if it's unlocked and a MacOS device is around AirDrop will be ready to receive files.

Was your thinking that if your phone is unlocked and in a room, then that is a very strong signal that you are in the room (using your phone), as opposed to just left your phone on the table?

I have been looking into using Room Assistant's companion app for iPhone detection. How willing are your other household users to adding the companion app to their phone?

The companion app displays a UUID to the user. This can be retrieved via BLE with the following method:

1. Search all BLE broadcasts for a string similar to 4c000100000010040000000000000000000000. (Overflow area) This is a strong hint that this is an iPhone with Companion App installed. (RA source and source)
2. Connect to the device, list all services
3. For service UUID 5403c8a7-5c96-47e9-9ab8-59e373d875a7, get characteristic UUID 21c46f33-e813-4407-8601-2ad281030052 (Companion app source)
4. The value of the characteristic is the UUID displayed on the app screen
5. Cache the mac address until it changes. In the mean time use the mac address to get the rssi

This requires connecting to the device once every ~45 min, and jumping through the hoops of getting service and characteristic IDs. TBH I don't know how to do that in a lambda, but we could probably figure that out as ESPHome BLE sensor.

[sct1000]

(ac & 0xf0) == 0x70 is checking the "action_code" value of 7 according to that slide, assuming that the packets are using network byte ordering. The updates docs here suggest that 3 is Idle, and 7 is active/screen on.

I did a bit of experimenting, and verified values of 3 when the iPhones were off/locked, and 7 when they were active. I didn't see any other values.

The main problems I found were:

1. The iPhones broadcast 7 when a notification pops up on an otherwise locked phone screen (e.g. one that's charging)
2. The iPhones completely stop broadcasting when using a browser or an app with a WKWebView for some reason, unless there's a Mac desktop close by. No idea why.

I haven't don't a huge amount of investigation beyond that. Perhaps there's something in the other flags that could help.
My main use case is to try to use the signal from the unlocked watch/ active phone for presence detection. It's not unusual for one of my kids to leave their phone in an otherwise unoccupied room. The Room Assistant companion app doesn't really help there.

[sct1000]

I just did a little more digging with the status flags and one iPhone (11 Pro, IOS 14).

0x08 is listed as "Unknown" in the docs here, but in my case it seems to correspond to a recent touch interaction. If I touch the screen, it's set in broadcasts for the next five seconds or so.

I'm not seeing any difference in action code or flags in getting a popup notification vs other passive use of the device.

I do see the 0x02 flag (also listed as "Unknown") set occasionally, but I haven't been able to correlate it.

One other thing: I sometimes see an action code of '5' instead of '7' for an active iPhone SE, but unlike what the docs suggest, there's no audio playing.

[dcgrove]

I would be willing to test anything you need. I am not nearly as knowledgeable about this as y'all are, but I can post log miles like a maniac!

[sct1000]

Okay, here's something interesting: When my iPhone stops broadcasting its Nearby Info messages (0x10) because it's in an app with a WKWebView, it instead starts broadcasting handoff messages (0xc) using the same MAC.

As far as I can tell, there's no easy way to passively differentiate between handoff messages -- I also see them broadcast from my macbook, but we could do something stateful in the lambda code to keep track of recently observed phones and interpret a handoff broadcast the same way as the active signal.

[dalehumby]

Thanks for all the info and discovery work. I've still got to get to the bottom of why my iPhone Xs / iOS 14 is not detected, even with my MacBook close by. Hopefully get to that tomorrow.

[sct1000]

Here's the current version I'm running with some success. I've decided that a local MacOS device broadcasting handoff messages is 'active' too and counts as a valid signal for room presence.

sensor:
  - platform: template
    id: idevice_rssi
    name: "$room_name iDevice RSSI"
    device_class: signal_strength
    unit_of_measurement: dBm
    accuracy_decimals: 0
    filters:
      - exponential_moving_average:
          alpha: 0.3
          send_every: 1
    on_value:
      then:
        - lambda: |-
            if (id(idevice_rssi).state > $rssi_present) {
              id(room_presence_debounce).publish_state(1);
            } else if (id(idevice_rssi).state < $rssi_not_present) {
              id(room_presence_debounce).publish_state(0);
            }
        - script.execute: presence_timeout  # Publish 0 if no rssi received
  
  - platform: template
    id: room_presence_debounce
    filters:
      - sliding_window_moving_average:
          window_size: 3
          send_every: 1

binary_sensor:
  - platform: template
    id: room_presence
    name: "$room_name iDevice presence"
    device_class: occupancy
    lambda: |-
      if (id(room_presence_debounce).state > 0.99) {
        return true;
      } else if (id(room_presence_debounce).state < 0.01) {
        return false;
      } else {
        return id(room_presence).state;
      }

script:
  # Publish event every 30 seconds when no rssi received
  id: presence_timeout
  mode: restart
  then:
    - delay: 30s
    - lambda: |-
        id(room_presence_debounce).publish_state(0);
    - script.execute: presence_timeout

# Apple watch/phone tracker https://github.com/dalehumby/ESPHome-Apple-Watch-detection
esp32_ble_tracker:
  scan_parameters:
    interval: 2.0s
    window: 500ms
    active: false
  on_ble_advertise:
    - then:
      # Look for manufacturer data of form:
      # For unlocked Apple watch: 4c 00 10 05 0X 98
      # For active iPhone: 45 00 10 XY (where X is > 3, for any Y)
      # For active iPhone/Mac (handoff): 45 00 0C
      - lambda: |-
          optional<int16_t> best_rssi;
          for (auto data : x.get_manufacturer_datas()) {
            const int16_t rssi = x.get_rssi();
            if (data.data.empty() || data.uuid.to_string() != "0x004C") {
              continue;
            }
            // Handoff is 0x0c (watch/phone/mac)
            if (data.data[0] == 0xc) {
              ESP_LOGD("ble_adv", "Found Apple broadcast for handoff (mac %s) rssi %i", x.address_str().c_str(), rssi);
              best_rssi = max(rssi, best_rssi.value_or(rssi));
              continue;
            }
            // Nearby info is 0x10
            if (data.data[0] != 0x10 || data.data.size() < 4) {
              continue;
            }
            const uint8_t ac = data.data[2];
            const uint8_t sf = data.data[3];
            // Unlocked watch
            if (sf == 0x98 && ac <= 0x0F) {
                ESP_LOGD("ble_adv", "Found Apple Watch (mac %s) rssi %i", x.address_str().c_str(), rssi);
                best_rssi = max(rssi, best_rssi.value_or(rssi));
                continue;
            }
            // Active iPhone
            if (ac >= 0x40) {
                best_rssi = max(rssi, best_rssi.value_or(rssi));
                ESP_LOGD("ble_adv", "Found active iPhone (mac %s; ac=%0x; sf=%0x) rssi %i", x.address_str().c_str(), ac, sf, rssi);
                continue;
            }
          }
          if (best_rssi) {
            id(idevice_rssi).publish_state(*best_rssi);
          }

[sct1000]

Looking at this again, my code for recording the closest RSSI is, of course, total bollocks. :-|

x is invariant in the loop: The automation callback is executed independently for each device. I'll find another solution.

[divemasterjm]

Looking at this again, my code for recording the closest RSSI is, of course, total bollocks. :-|

x is invariant in the loop: The automation callback is executed independently for each device. I'll find another solution.

thanks for your code, when compiling i've got the following error

/config/esphome/bletracker2.yaml: In static member function 'static void std::_Function_handler<void(_ArgTypes ...), _Functor>::_M_invoke(const std::_Any_data&, _ArgTypes&& ...) [with _Functor = setup()::<lambda(const esphome::esp32_ble_tracker::ESPBTDevice&)>; _ArgTypes = {const esphome::esp32_ble_tracker::ESPBTDevice&}]':
/config/esphome/bletracker2.yaml:77:38: warning: '((void)& best_rssi +2)' may be used uninitialized in this function [-Wmaybe-uninitialized]
id(idevice_rssi).publish_state(best_rssi);
^
/config/esphome/bletracker2.yaml:45:25: note: '((void*)& best_rssi +2)' was declared here
optional<int16_t> best_rssi;
^