Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spotify stops polling and then does not restart anymore automatically and without notification. #65

Closed
iluvatyr opened this issue Feb 2, 2023 · 4 comments

Comments

@iluvatyr
Copy link

iluvatyr commented Feb 2, 2023

I was checking my scrobbles and suddenly noticed that many scrobbles of the last days are missing. So I manually checked the logs of the multiscrobber and noticed that it stopped scrobbling tracks from spotify since half a month already. The tracks I played there are therefore lost now.

I think it would be best, if there was a container internal mechanism implemented that tries polling again after a defined time, because there is no way someone wants to constantly check if the scrobbling is still working.

Secondly, it would be great to have an additional email or other notification that can be configured to inform the user that there is an issue regarding something in multi-scrobbler.

Options to inform user about an issue with multi-scrobbler:

  • Health page multi-scrobbler-domain/health that can be monitored using uptime-kuma / uptimerobot
  • Docker internal health check
  • Email / Webhook notification

Logs are here:

2023-01-14T02:21:20+01:00 warn : [Source - Spotify - unnamed] Poll retries (0) equal to max poll retries (0), stopping polling!
2023-01-14T02:21:20+01:00 error : [Source - Spotify - unnamed] Error: getaddrinfo ENOTFOUND api.spotify.com
at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:67:26) ,2023-01-14T02:21:20+01:00 error : [Source - Spotify - unnamed] Error occurred while polling
2023-01-14T02:21:20+01:00 error : [Spotify ] Error: getaddrinfo ENOTFOUND api.spotify.com [{"label":"Spotify"}]
at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:67:26) ,2023-01-14T02:21:20+01:00 error : [Source - Spotify - unnamed] Request failed on retry (1) with no more retries permitted (max 1)
2023-01-14T02:21:18+01:00 warn : [Source - Spotify - unnamed] Request failed but retries (0) less than max (1), retrying request after 2 seconds...```
@FoxxMD
Copy link
Owner

FoxxMD commented Feb 3, 2023

I think it would be best, if there was a container internal mechanism implemented that tries polling again after a defined time

There is! You are looking for maxPollRetries, maxRequestRetries, and retryMultiplier which can be specified for

  • all sources in config.json using the sourceDefaults property or
  • per client in their respective configs (spotify.json) in the data property.

A health check endpoint is a great idea. I don't have time to implement this right now but if you like to give it a go a submit a PR i'd be happy to merge it.

@FoxxMD FoxxMD closed this as completed Feb 3, 2023
@iluvatyr
Copy link
Author

iluvatyr commented Feb 24, 2023

Hello, I see you closed this, but it is not entirely clear to me how to fix this now.
First and foremost, I do not know, which values would be appropriate for the above to not happen anymore.
Secondly, how is setting this possible in an ENV-Configuration, not using config files? is it even possible?

But thanks for letting me know it is possible to configure it via json configuration files!

If you do not have a lot of time for a health check, I think a get request to a custom URL on a poll stopping or error coming up would already be enough for a lot of things

@FoxxMD
Copy link
Owner

FoxxMD commented Feb 24, 2023

which values would be appropriate for the above to not happen anymore.

I've recently rewritten the documentation and it now includes a schema explorer for each config file and the general config.json file. Here is the explorer for the general config (also linked in the docs). The explorer contains descriptions, examples, and a live editor/validator. The sourceDefaults property is the one you are interested in.

To increase the polling attempts you could have a config.json file that contains only this:

{
  "sourceDefaults": {
    "maxRequestRestries": 3,
    "maxPollRetries": 12,
    "retryMultiplier": 10
  }
}

Which will make MS:

  • Attempt any request 3 times before failing
  • Try to restart polling 12 times before giving up
  • Wait # of attempts * 10 seconds seconds between each polling retry attempt

is setting this possible in an ENV-Configuration

No it is not. The ENV based config is for supporting the bare minimum setup for MS.

On the code side I already have to structure all this user data into something equivalent to json (nested objects) so it's much easier to use config files for that input since I can easily validate and parse that complex/nested data. MS already has ~20 ENV variables I have to support alongside the regular config files and I don't plan on adding more if possible.

EDIT: For your specific scenario you must have already setup a config directory since the spotify source has to store its credentials in a file anyway. It should be trivial to create a json file in the same directory.

@iluvatyr
Copy link
Author

Hello FoxxMD,

Thank you! I was just finished with my process of converting all the environment variables to jsons when you posted this. The documentation is showing nicely how to format the json files. It worked on first try.
I just didnt know exactly what values are appropriate since I do not know the polling mechanism and how it should be for not trying too long but still long enough.

Thank you for showing me some values that I will use.

FoxxMD added a commit that referenced this issue Sep 5, 2023
Addresses #5 and #65 -- in default config use-cases where:

 * request/polling retry attempts are low
 * source was initially OK (auth OK, initial communication OK)
 * and there is a non-auth related *upstream* communication issue

 there is no fallback path for restarting source polling after upstream has normalized.

 This feat implements a scheduled heartbeat task that attempts to restart polling every 20 minutes if this scenario is valid so that a good source config will recover without user intervention.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants