mobileclient.search() returns an empty 'playlist_hits' list

[juanrubio]

Hi,

Sorry if this has been asked before, but I've noticed that mobileclient.search() now doesn't return any playlist hits. I'm sure this used to return playlist results some time ago. Any idea of what is going on?

I'm using gmusicapi 11.0.3 and a GMusic paid subscription.

...

 u'artist_hits': [{u'artist': {u'artistArtRef': u'http://lh3.googleusercontent.com/UX9wLHPOvVHJHL4HOOetxjvTkZNITP2KkDuqUFohphIdYhEQVP8DUpVV_Glx48FQgOV3Ctz4OXg',
                               u'artistArtRefs': [{u'aspectRatio': u'2',
                                                   u'autogen': False,
                                                   u'kind': u'sj#imageRef',
                                                   u'url': u'http://lh3.googleusercontent.com/UX9wLHPOvVHJHL4HOOetxjvTkZNITP2KkDuqUFohphIdYhEQVP8DUpVV_Glx48FQgOV3Ctz4OXg'}],
                               u'artistId': u'Asofl5rpj6oa6kxnwsb5oqnpasi',
                               u'artist_bio_attribution': {u'kind': u'sj#attribution',
                                                           u'source_title': u'artist representative'},
                               u'kind': u'sj#artist',
                               u'name': u'Metallica'},
                   u'type': u'2'}],
 u'playlist_hits': [],
 u'podcast_hits': [{u'series': {u'art': [{u'aspectRatio': u'1',
                                          u'autogen': False,
                                          u'kind': u'sj#imageRef',
                                          u'url': u'https://lh3.googleusercontent.com/lp61Np9b1U6qmF8eO9RpuNJJ29MSRITMHH7hNEGdhQMh2wvcB9soaAYN6UpJ'}],
                                u'author': u'Fans Not Experts',
                                u'continuationToken': u'',
                                u'description': u'A Metallica Fan Podcast from a lifelong Metallica Fan!',
                                u'explicitType': u'2',
                                u'link': u'http://www.fansnotexperts.com/search/label/Metallicast',
                                u'seriesId': u'Isq43ss3cqzxqzbe63naiwi4lhu',
                                u'title': u'METALLICAST - THE Metallica Podcast',
                                u'totalNumEpisodes': 0},
                    u'type': u'9'},

...

[simon-weber]

I tested this on my accounts and see the same thing. It looks like Google just isn't returning playlist results.

They still show up in the app, so my bet would be they moved that functionality clientside. If that's the case, we should probably just remove the result type from the call.

[thebigmunch]

Playlists are still returned in the call response. My library is showing this behavior as well. Will report back after diagnosing what has changed.

[thebigmunch]

FYI, I broke out my old phone to capture a search call to find it had a different response format. But when I make the same call through my library, it returns the same format as we have always used sans the playlist responses.

So, I'm digging into it.

[thebigmunch]

Alright, there's a new URL param. ic if left off or set to False returns the old response format without playlists, if set to True returns the new response format with everything.

I'll put something together soonish; it's dinner time.

[thebigmunch]

So, the new response format looks like so:

{
    'kind': 'sj#searchresponse',
    'clusterDetail': [
        {
            'cluster': {  # Clusters are always returned whether result type is requested or no results for type.
                'category': '1'  # Have seen 1 and 3 as values.
                'id': 'search_[type]'  # E.g. search_artist, search_genre, search_podcast_series, etc.
                'type': '1'  # Same types as before.
            },
           'displayName': ''  # Not required. Used for stations.
            'resultToken': ''  # These may mean something (EAU, EAI, EAM, etc) but don't seem important.
            'entries': []  # Same format as previous entries list except each entry has a cluster key as above in it on top of having a 'type' key still. That cluster value has, for some reason, a list around the dict. Doesn't exist if not requested or no results.
        }
    ]
}

Genre cluster is returned always but empty.

[thebigmunch]

Just an update. Started this, partially just re-familiarizing myself with the code. Got busy with other things for a few days. Will finish over weekend and submit PR on Monday most likely.

[juanrubio]

I am probably missing something here. I still can't see the playlist results.

This is what I get with 'develop':

9 matches for "_hits" in buffer: gmusicapi-11.0.4rc1.log
      6:{u'album_hits': [{u'album': {u'albumArtRef': u'http://lh3.googleusercontent.com/aAd_flnuXy0e9HvC-e-EscLl-qLVLaJnxjhlOWEW_wQCCAEuE470bgLy3r7vCd35mQSc0f5d8w',
   1597: u'artist_hits': [{u'artist': {u'artistArtRef': u'http://lh3.googleusercontent.com/UX9wLHPOvVHJHL4HOOetxjvTkZNITP2KkDuqUFohphIdYhEQVP8DUpVV_Glx48FQgOV3Ctz4OXg',
   1611: u'genre_hits': [],
   1612: u'playlist_hits': [],
   1613: u'podcast_hits': [],
   1614: u'situation_hits': [],
   1615: u'song_hits': [{u'cluster': [{u'category': u'1',
  30751: u'station_hits': [{u'cluster': [{u'category': u'1',
  31052: u'video_hits': [{u'cluster': [{u'category': u'1',

[thebigmunch]

I am probably missing something here. I still can't see the playlist results.

No, I'm seeing this now, too, in both gmusicapi and my own library. But I definitely got results before the weekend.

Just checking on it quick, I'm getting two separate station clusters. Wondering if one of those is full of playlists. And since I changed things to use the type from the cluster instead of each individual result, maybe that's throwing things off. Will report back.

[thebigmunch]

I was searching an artist, and the two different station clusters is expected as they do separate 'Featuring' result section from other radio station results.

[thebigmunch]

Should also be noted that playlist hits aren't tested in the tests specifically because they have been proven inconsistent in the past. Which is why the tests passed.

[thebigmunch]

Well, this is seemingly a result of Google Music handling the max_results param differently than before. It can no longer be left off, or you get no results for situations nor playlists.

Not sure if we should give this a couple days to see if its temporary or just change the max_results default to match Google Music's. Kinda sucks as leaving it to None does allow more than 100 results from the other hit types.

@juanrubio Just for confirmation. If you set max_results to 100 in your call to the search method, do you get playlist results as well?

[juanrubio]

@thebigmunch Good find!. Yes, with max_results=100 the API returns a non-empty playlist_hits

   1441: u'genre_hits': [],
   1442: u'playlist_hits': [{u'cluster': [{u'category': u'1',
-------
       :                     u'type': u'4'}],
   2382: u'podcast_hits': [],

[thebigmunch]

Alright then, it's up to @simon-weber whether he wants me to submit a PR changing the default right away, documenting the new behavior for those that want to get more than 100 results of certain types and don't care about playlist/situation hits, or wait to see if the behavior reverts.

[simon-weber]

Nice find. I think we should probably just change the default to 100 -- my guess is that the previous behavior was unintended anyway.

[juanrubio]

@simon-weber

Are there plans for a new release?

[simon-weber]

I was hoping to resolve the login issues before releasing, but haven't had a chance to look at it lately. I suppose I should just release anyway.

[juanrubio]

Understood. If there are login issues currently being fixed, I can definitely wait for that. Just for reference, what is the issue number?

[simon-weber]

#620 is the most relevant (it's breaking CI as well).