Add a backup key in the configuration

[bogdanbujdea]

I set my addon to refresh every 90 seconds, but sometimes this depletes the quota, it depends on how much I drive in that day. Would it be possible to add a "backup key" as well in the configuration for situations like this? I created another app in my Volvo account so this way I could use shorter intervals.
The main reason I don't use the default 300 seconds interval is because I need to detect as soon as possible when I'm starting the car to log the battery/fuel levels when a trip starts.

[Dielee]

Yeah, good idea. I had the same idea a few weeks ago. I will try to implement this feature, but I need your help. If your quota is depleted, which http code will be returned?

EDIT: If you don't know yet, enable debug log and upload your log.

[bogdanbujdea]

I found it in the logs, it's 403 Forbidden.
ERROR: API Call failed. Status Code: 403. Error: { "statusCode": 403, "message": "Out of call volume quota. Quota will be replenished in 06:40:24." }

[Dielee]

Nice, I will take a look.

[Dielee]

Are you able to test a dev release as single docker image, outside HA OS ?

[bogdanbujdea]

I can try. I just need to run the docker image? or what are the steps?

[Dielee]

Yes, I test the changes know and create a dev build later today.
If the test build was successfull, I will send you a few instructions on how to run the dev build.

[Dielee]

Here you go. Start your docker container like this:

docker run -d --pull=always -e CONF_updateInterval=300 -e CONF_babelLocale='de' -e CONF_mqtt='@json {"broker": "", "username": "", "password": "", "port": 1883}' -e CONF_volvoData='@json {"username": "", "password": "", "vin": "", "vccapikey": "", "backupvccapikey": "", "odometerMultiplier": 1, "averageSpeedDivider": 1, "averageFuelConsumptionMultiplier": 1}' -e TZ='Europe/Berlin' --name volvo2mqtt ghcr.io/dielee/volvo2mqtt:backup-vcc-api-key

Please test today and tomorrow. If everything works, I will merge into main.

[bogdanbujdea]

I got this in the logs:

2023-08-07 14:56:19 Traceback (most recent call last):
2023-08-07 14:56:19   File "/usr/local/lib/python3.9/site-packages/dynaconf/utils/parse_conf.py", line 273, in get_converter
2023-08-07 14:56:19     converted_value = converter(value, box_settings=box_settings)
2023-08-07 14:56:19 TypeError: <lambda>() got an unexpected keyword argument 'box_settings'
2023-08-07 14:56:19 
2023-08-07 14:56:19 During handling of the above exception, another exception occurred:
2023-08-07 14:56:19 
2023-08-07 14:56:19 Traceback (most recent call last):
2023-08-07 14:56:19   File "/volvoAAOS2mqtt/./main.py", line 2, in <module>
2023-08-07 14:56:19     from volvo import authorize
2023-08-07 14:56:19   File "/volvoAAOS2mqtt/volvo.py", line 3, in <module>
2023-08-07 14:56:19     import mqtt
2023-08-07 14:56:19   File "/volvoAAOS2mqtt/mqtt.py", line 6, in <module>
2023-08-07 14:56:19     import util
2023-08-07 14:56:19   File "/volvoAAOS2mqtt/util.py", line 9, in <module>
2023-08-07 14:56:19     from const import units
2023-08-07 14:56:19   File "/volvoAAOS2mqtt/const.py", line 82, in <module>
2023-08-07 14:56:19     {"name": "Electric Range", "domain": "sensor", "id": "electric_range", "unit": "km" if not units.get(settings["babelLocale"]) else units[settings["babelLocale"]]["electric_range"]["unit"], "icon": "map-marker-distance", "url": RECHARGE_STATE_URL, "state_class": "measurement"},
2023-08-07 14:56:19   File "/usr/local/lib/python3.9/site-packages/dynaconf/utils/functional.py", line 18, in inner
2023-08-07 14:56:19     self._setup()
2023-08-07 14:56:19   File "/usr/local/lib/python3.9/site-packages/dynaconf/base.py", line 174, in _setup
2023-08-07 14:56:19     self._wrapped = Settings(
2023-08-07 14:56:19   File "/usr/local/lib/python3.9/site-packages/dynaconf/base.py", line 253, in __init__
2023-08-07 14:56:19     self.execute_loaders()
2023-08-07 14:56:19   File "/usr/local/lib/python3.9/site-packages/dynaconf/base.py", line 1035, in execute_loaders
2023-08-07 14:56:19     core_loader.load(self, env, silent=silent, key=key)
2023-08-07 14:56:19   File "/usr/local/lib/python3.9/site-packages/dynaconf/loaders/env_loader.py", line 33, in load
2023-08-07 14:56:19     load_from_env(obj, global_prefix, key, silent, IDENTIFIER + "_global")
2023-08-07 14:56:19   File "/usr/local/lib/python3.9/site-packages/dynaconf/loaders/env_loader.py", line 73, in load_from_env
2023-08-07 14:56:19     data = {
2023-08-07 14:56:19   File "/usr/local/lib/python3.9/site-packages/dynaconf/loaders/env_loader.py", line 74, in <dictcomp>
2023-08-07 14:56:19     key[trim_len:]: parse_conf_data(
2023-08-07 14:56:19   File "/usr/local/lib/python3.9/site-packages/dynaconf/utils/parse_conf.py", line 379, in parse_conf_data
2023-08-07 14:56:19     return _parse_conf_data(data, tomlfy=tomlfy, box_settings=box_settings)
2023-08-07 14:56:19   File "/usr/local/lib/python3.9/site-packages/dynaconf/utils/parse_conf.py", line 343, in _parse_conf_data
2023-08-07 14:56:19     value = get_converter(converter_key, value, box_settings)
2023-08-07 14:56:19   File "/usr/local/lib/python3.9/site-packages/dynaconf/utils/parse_conf.py", line 275, in get_converter
2023-08-07 14:56:19     converted_value = converter(value)
2023-08-07 14:56:19   File "/usr/local/lib/python3.9/site-packages/dynaconf/utils/parse_conf.py", line 250, in <lambda>
2023-08-07 14:56:19     else json.loads(value),
2023-08-07 14:56:19   File "/usr/local/lib/python3.9/json/__init__.py", line 346, in loads
2023-08-07 14:56:19     return _default_decoder.decode(s)
2023-08-07 14:56:19   File "/usr/local/lib/python3.9/json/decoder.py", line 337, in decode
2023-08-07 14:56:19     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
2023-08-07 14:56:19   File "/usr/local/lib/python3.9/json/decoder.py", line 353, in raw_decode
2023-08-07 14:56:19     obj, end = self.scan_once(s, idx)
2023-08-07 14:56:19 json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)

Is it something wrong with the variables I'm using maybe? This is how I run the docker container:

docker run -d --pull=always -e CONF_updateInterval=15 -e CONF_babelLocale='ro_RO' -e CONF_mqtt='@json {"broker": "192.168.x.x", "username": "mymqttusername", "password": "mymqttpass", "port": 1883}' -e CONF_volvoData='@json {"username": "myemail", "password": "myvolvopass", "vin": "mycarvin", "vccapikey": "apikey1", "backupvccapikey": "apikey2", "odometerMultiplier": 1, "averageSpeedDivider": 1, "averageFuelConsumptionMultiplier": 1}' -e TZ='Europe/Bucharest' --name volvo2mqtt ghcr.io/dielee/volvo2mqtt:backup-vcc-api-key

[Dielee]

Your json seems to be invalid. Your pasted command works for me without any errors. Double check the command filled with your real values. Maybe your password contains a " or '?

[bogdanbujdea]

weird, I validated both jsons, and the passwords have just letters and numbers
I tried to run the :latest as well and I have the same error, so it's definitely something from my side
I'll keep trying and let you know when I find it

[bogdanbujdea]

made it work, I have to escape the quotes like this

docker run ... -e CONF_mqtt='@json {"\"broker"\": "\"192.168.x.x"\".....

it might be from my terminal...I don't know for sure
right now it's running fine and I set the interval to 15 seconds, so it should deplete the quota soon

[Dielee]

Thats odd... but okay, it works.
So the addon will use the backup key later this day and will remain using this, until the backup key quota depletes.
This means, the addon should switch to the "default" key again, tomorrow, if everything works.

[bogdanbujdea]

Got it, so if the quota from the default one is depleted, it will switch to the other one immediately, but only until the next day.

[Dielee]

Noep, it will switch if the key is depleted. Then it will try the "other" key. Either the default, or the backup key, depending on what were used last.

[bogdanbujdea]

Done, it works!

2023-08-07 17:31:16 Aug 07 17:31:16 volvo2mqtt [1] - INFO: Mqtt update done. Next run in 15 seconds.
2023-08-07 17:31:31 Aug 07 17:31:31 volvo2mqtt [1] - INFO: Sending mqtt update...
2023-08-07 17:31:31 Aug 07 17:31:31 volvo2mqtt [1] - WARNING: Quota extended. Try to change VCCAPIKEY!
2023-08-07 17:31:31 Aug 07 17:31:31 volvo2mqtt [1] - INFO: Default VCCAPIKEY Quota extended. Start using backup VCCAPIKEY!
2023-08-07 17:31:32 Aug 07 17:31:32 volvo2mqtt [1] - INFO: Mqtt update done. Next run in 15 seconds.
2023-08-07 17:31:47 Aug 07 17:31:47 volvo2mqtt [1] - INFO: Sending mqtt update...
2023-08-07 17:31:49 Aug 07 17:31:49 volvo2mqtt [1] - INFO: Mqtt update done. Next run in 15 seconds.

[Dielee]

Nice, then it should revert to the default key tomorrow

[bogdanbujdea]

I stopped my docker container for a few hours and when I started it again I get this in the logs:

The backup key has only 2k requests, and it's not midnight yet.
Requests today:
2 813 out of 10 000

UPDATE: I think I caught a bug, here's how to reproduce it:

• stop the container
• set a key with depleted quota as default
• set a valid key as backup
• start the container

Expected result would be to gracefully switch from default to backup at startup, but instead it will crash with the errors in the log above.
I tried switching the keys (the default one is valid) and then it works, so just the case above needs to be handled. It's an edge case in my opinion because in order to reproduce it you need to deplete the quota on the default key and restart the addon before the quota gets replenished, but if there's an easy fix then it's better to include it in the next release.

EDIT: File removed. Contains sensitive data.

[Dielee]

Thanks a lot! I will take a look later.

[Dielee]

Should be fixed. Please pull the image again and test some more!

[bogdanbujdea]

It works now, I depleted my default key quota then restarted the addin and it continued with the backup key.

2023-08-08 12:38:27 Aug 08 12:38:27 volvo2mqtt [1] - INFO: Starting volvo2mqtt version v1.7.10
2023-08-08 12:38:27 Aug 08 12:38:27 volvo2mqtt [1] - INFO: Vin: ['ABC'] found!
2023-08-08 12:38:28 Aug 08 12:38:28 volvo2mqtt [1] - WARNING: Quota extended. Try to change VCCAPIKEY!
2023-08-08 12:38:28 Aug 08 12:38:28 volvo2mqtt [1] - INFO: Default VCCAPIKEY Quota extended. Start using backup VCCAPIKEY!
2023-08-08 12:38:28 Aug 08 12:38:28 volvo2mqtt [1] - INFO: Failed, Battery Charge Level is unfortunately not supported by your vehicle.
2023-08-08 12:38:28 Aug 08 12:38:28 volvo2mqtt [1] - INFO: Success! Battery Charge Level is supported by your vehicle.
2023-08-08 12:38:28 Aug 08 12:38:28 volvo2mqtt [1] - INFO: Success! Electric Range is supported by your vehicle.

[Dielee]

We should test a few more times, so take your time. I will merge in the next days, if everything works.

[Dielee]

I found another bug. If both keys are extended, the addon crashes. Are you able to reproduce this behaviour ?

[Dielee]

I reworked the whole vcc api backup key section, as this was a real mess. In the next release, you will be able to specify multiple vcc-api-keys in your config. You can save as much keys as you want, not only one backup key. This seems to be a far better solution for me. Your config will look like this:

username: yourVolvousername@dummy.com
password: yourPassword
vin: ""
vccapikey:
  - vccapikey1
  - vccapikey2
  - vccapikey3
  - etc.
odometerMultiplier: 1
averageSpeedDivider: 1
averageFuelConsumptionMultiplier: 1

[bogdanbujdea]

Indeed, it's a better solution at least from the users's point of view because you're not limited to 2 keys. I didn't had time in the last 2 days but tonight I'll pull the latest image and test again, I'll update you tomorrow on how it goes.

[mkrijtenberg]

+1 for multiple keys!

I'm going to try out it as well!

[mkrijtenberg]

Nice. I've created 4 applications and added all primary api keys (what are the secondary keys for?). My refresh rate is set to 60. I want updates every minute :)

When I start the container, I've used 28 request for the first update. How many requests does it perform every cycle?

[Dielee]

The secondary keys are not used. I don't know what they are for.
The first update uses a few more requests as every enpoint will be checked if supported by your car.
How many requests it performs every cycle depends on your car and supported endpoints.

[Dielee]

This function is available starting release v1.8.0.