Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KeyError: "Couldn't find extension field Devialet.CallMeMaybe.dvltServiceOptions" #1

Open
fwestenberg opened this issue Mar 27, 2020 · 16 comments

Comments

@fwestenberg
Copy link

Hello,

First thanks for this impressive piece of work! Personally I'm very interested in creating a Devialet integration for Home Assistant. Especially volume contol is what I'm interested in. Can you please help me getting started with this? First I have the following issue, any idea how to solve this:

C:\Repositories\etincelle>python client_test.py
Traceback (most recent call last):
File "client_test.py", line 8, in
from dvlt_client import WhatsUpClient
File "C:\Repositories\etincelle\dvlt_client.py", line 15, in
dvltServiceOptions = dvlt_pool.FindExtensionByName('Devialet.CallMeMaybe.dvltServiceOptions')
KeyError: "Couldn't find extension field Devialet.CallMeMaybe.dvltServiceOptions"

Also, do you know if it is possible with this repository to fake a speaker? Because my Devialet Phantom's are on DOS2, I am not able to update my Devialet Dialog unless I have a Phantom (or fake one) with DOS1. Thanks in advance!

@unusual-thoughts
Copy link
Owner

unusual-thoughts commented Mar 27, 2020

Hello,

Hi Floris! This project has been sleeping for a while now, glad to see someone else is interested ;)

It should have all the building blocks needed to do what you want, although it's all quite messy and experimental. The most complete example usage is probably in play_mp3.py, which shows the fairly long series of steps to talk to a devialet device, although it didn't quite work up to actually playing the mp3 for some reason that i forget. Also as you'll see there is a lot of debug information as well as support for packet captures to replay traffic.

First thanks for this impressive piece of work! Personally I'm very interested in creating a Devialet integration for Home Assistant. Especially volume contol is what I'm interested in. Can you please help me getting started with this? First I have the following issue, any idea how to solve this:

Volume control should be the one of the easiest use cases, probably using TooManyFlows.SoundControl.

C:\Repositories\etincelle>python client_test.py
Traceback (most recent call last):
File "client_test.py", line 8, in
from dvlt_client import WhatsUpClient
File "C:\Repositories\etincelle\dvlt_client.py", line 15, in
dvltServiceOptions = dvlt_pool.FindExtensionByName('Devialet.CallMeMaybe.dvltServiceOptions')
KeyError: "Couldn't find extension field Devialet.CallMeMaybe.dvltServiceOptions"

Fixed this error, which is due to my pretty intense usage of internal protobuf python apis (because a lot of custom stuff in the devialet protobuf usage) that didn't mesh well with the cpp backend. Fixed with a hack for now.

Also, do you know if it is possible with this repository to fake a speaker? Because my Devialet Phantom's are on DOS2, I am not able to update my Devialet Dialog unless I have a Phantom (or fake one) with DOS1. Thanks in advance!

It could absolutely be used to fake a speaker. Both speakers and apps implement the same kinds of interfaces, specifically they are both WhatsUp servers and clients
You could register the same services as a regular phantom, with dummy hooks for all the interesting methods, and wait for someone to call you during the discovery and update process, then fill in the blanks as needed. it might help to have a capture of what the real phantom says, and the repo contains tools to analyze and extract all the chatter between app and device so that you can replicate the important parts

@fwestenberg
Copy link
Author

Thanks for your quick response! I have been tryin to get a connection, but apparently some things have changed with DOS2. Right now I only use two Phantoms (without dialog). With my Android phone I try to capture the packages from the new Devialet app. But also ports seems to have changed. Both of my Phantoms use different ports for some reason:
Phantom 1: first message on port 37703
Phantom 2: first message on port 35003.

Do you still have your Phantom(s)? What is your configuration?
Thanks again!

@unusual-thoughts
Copy link
Owner

Yeah ports are random and there are multiple ones for all the services that are exposed. Did you have any luck with my pyhon fix?

@fwestenberg
Copy link
Author

I'm trying to get some results now, but nothing satisfying so far. Should I connect to the Phantom directly you think? And what port should I use?

@fwestenberg
Copy link
Author

At least the error is gone now ;) But it's just refusing the connection...

@unusual-thoughts
Copy link
Owner

What is the output ?

@fwestenberg
Copy link
Author

The following test:

hostUid = b'etincelle-' + bytes(uuid.uuid4().hex, 'ascii')

discovered = Queue()
dscvr = DevialetDiscovery(discovered serial=hostUid)
dscvr.start()

serial, dvlt_addr = discovered.get(block=True)
print_info(serial, dvlt_addr)

Results in:

�[34mInfo: Searching for Devialet Device...�[0m

And my most positive try so far (port 37703 is always the first port in my trace file):

def callback_test(arg):
    print_data("Callback with", arg)

# Open a WhatsUp connection
wu_client = WhatsUpClient(name="WhatsUp", addr='192.168.1.211', port=37703, start_time=datetime.now())
wu_client.go()
wu_client.service_discovery()

wu_client.keep_receiving(timeout=2)

Results in:

�[34mInfo: Calling method openConnection from service com.devialet.callmemaybe.connection (0)...�[0m
�[32m... with argument:�[0m
�[37m   version: 1�[0m
�[37m   �[0m
�[32mReply:�[0m
�[37m   None�[0m
�[34mInfo: Opened connection to 192.168.1.211 on port 37703�[0m
�[32mList of services:�[0m
�[37m   �[0m
�[31mError: Service name com.devialet.whatsup.registry not in list []�[0m
�[34mInfo: Calling method listServices from service com.devialet.whatsup.registry (0)...�[0m
�[32m... with argument:�[0m
�[37m   �[0m
�[34mInfo: Getting properties for com.devialet.whatsup.registry�[0m
�[31mError: Service name com.devialet.whatsup.registry not in list []�[0m
�[34mInfo: Calling method propertyGet from service com.devialet.whatsup.registry (0)...�[0m
�[32m... with argument:�[0m
�[37m   �[0m
�[33mWarning: Timed out on port 37703�[0m
�[33mWarning: Closing Connection to 192.168.1.211 on port 37703�[0m
Exception in thread WhatsUp:
Traceback (most recent call last):
  File "C:\Users\Floris\AppData\Local\Programs\Python\Python36-32\lib\threading.py", line 916, in _bootstrap_inner
    self.run()
  File "C:\Users\Floris\Documents\Repositories\etincelle\dvlt_client.py", line 133, in run
    self.keep_receiving()
  File "C:\Users\Floris\Documents\Repositories\etincelle\dvlt_client.py", line 127, in keep_receiving
    while not self.shutdown_signal and self.receive(timeout):
  File "C:\Users\Floris\Documents\Repositories\etincelle\dvlt_client.py", line 107, in receive
    data = self.sock.recv(2048)

@unusual-thoughts
Copy link
Owner

I've had a look at the Devialet app binaries, and it looks like the DOS 2 protocol is quite different, for one they are using protobuf 3 now. This repo works with DOS 1 only for now. I haven't updated my phantom, looks like there are only downsides to it (although the spark app does suck)?

@unusual-thoughts
Copy link
Owner

Can you send me a capture of chatter between your phantom and app ?

@unusual-thoughts
Copy link
Owner

I'm surprised if they also changed the multicast discovery protocol though, I would still expect to see UDP traffic on port 24242

@fwestenberg
Copy link
Author

fwestenberg commented Mar 29, 2020

I sent the capture to your email. When I change to a wifi network without Phantoms, it starts searching with UDP at 224.0.0.251:5353, which seems to be a mDNS discovery(?).

@fwestenberg
Copy link
Author

fwestenberg commented Mar 29, 2020

I haven't updated my phantom, looks like there are only downsides to it (although the spark app does suck)?

I think the DOS2 update is fine, Airplay working and Bluetooth (don't know about Spark?). Also nice you don't need a Dialog any more te create a stereo pair. And I use a Sonos connect to get my Phantoms in one app. However, the Phantoms still have a Master volume in my case, so I just need to get this in my Home Assistant app. :-)

@unusual-thoughts
Copy link
Owner

From your capture, it seems they switched the wire protocol to websockets, and are now using new protobufs for the higher level protocol.
I extracted the new protobufs from the android app, analyzed your capture with https://github.com/mildsunrise/protobuf-inspector, and it looks like the protobufs are Devialet.CallMeMaybe.BaseRequest, with a polymorphic Body

message BaseRequest {
  enum Type {
    InvalidType = 0;
    Request = 1;
    Reply = 2;
    Notification = 3;
  }
  enum SubType {
    InvalidSubType = 0;
    Open = 1;
    Close = 2;
    Call = 3;
    PropertySet = 4;
    PropertyAdd = 5;
    PropertyRemove = 6;
    Exception = 7;
    PropertyReplace = 8;
  }
  .Devialet.CallMeMaybe.BaseRequest.Type type = 1;
  .Devialet.CallMeMaybe.BaseRequest.SubType subType = 2;
  .Devialet.CallMeMaybe.TrackingInfo tracking = 3;
  .Devialet.CallMeMaybe.RequestTarget target = 4;
  .google.protobuf.Any body = 5;
}

which looks a bit like the old

message Request {
  required bytes serverId = 1;
  required fixed32 serviceId = 2;
  required bytes requestId = 3;
  required uint32 type = 4;
  required uint32 subTypeId = 5;
}

message Reply {
  required bytes serverId = 1;
  required fixed32 serviceId = 2;
  required bytes requestId = 3;
  required uint32 type = 4;
  required uint32 subTypeId = 5;
  required sint32 errorCode = 6;
  required bool isMultipart = 7;
}

The chatter looks like:
req

type: Request
subType: Open
tracking {
  id: "\240\205\234\372\337\201O\032\275\202\350b\353\2441\336"
}
target {
  serviceId: 1
}
body: empty

resp

type: Reply
subType: Open
tracking {
  id: "\240\205\234\372\337\201O\032\275\202\350b\353\2441\336"
}
target {
  serviceId: 1
}
body: InterfaceProperties {
    apiVersion: 1
    properties: DaemonRegistryProperties {
        parent: .Devialet.CallMeMaybe.InterfaceProperties empty
        activeInstallation: .Devialet.WhatsUp.InstallationMessage {
            id: bytes (16) "AF A7 EC A5 82 23 52 51 A1 19 52 AC 8A 3E 9F E8"
            name: "Rocket"
        }
        currentAgent: AgentMessage {
            id: AgentIdMessage {
                installationId: bytes (16) (Uuid) "AF A7 EC A5 82 23 52 51 A1 19 52 AC 8A 3E 9F E8"
                hostId: bytes (16) "1E AF 69 D4 FC 85 59 9D 8C 5C FF C1 85 C8 2C 70"
                poolId: bytes (16) "CD 5F 21 AE 8A 06 4D ED 92 0B 69 18 21 88 C3 B5"
            }
            name: "WhatsUp"
            hostName: "SilverPhantom-LXXXXXXXXXXXX"
            serialNumber: "LXXXXXXXXXXXX"
        }
    }
}

@fwestenberg
Copy link
Author

fwestenberg commented Mar 29, 2020

What do you think is the best approach? Perhaps create a new repository with a DOS2 API? (finally it should be in a PyPi library).
For a Home Assistant integration I should create an integration with the next services available:

  • turn_on
  • turn_off
  • toggle
  • volume_up
  • volume_down
  • volume_set
  • volume_mute (volume 0 and restore should work)
  • media_play_pause (toggle function)
  • media_play
  • media_pause
  • media_stop
  • media_next_track
  • media_previous_track
  • clear_playlist
  • shuffle_set
  • play_media
  • select_source
  • select_sound_mode

Bold are most interested to me, I would prefer to start with this. Some functions don't need to be available for Devialet.

@fwestenberg
Copy link
Author

fwestenberg commented Jun 17, 2020 via email

@fwestenberg
Copy link
Author

fwestenberg commented Oct 18, 2020 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants