Allow multiple vcc api keys (#85)

* Multiple vccapikeys function

* Reduce api calls for key checks

* Change sleeptime

* Optimize logging

* Format log datetime

* Change default conf

* Set device offline, if no working vcc api key can be found

* Fix climate scheduler

* Only redact sensitive data if debug is enabled

README.md

@@ -64,7 +64,7 @@ NOTE: Energy status currently available only for cars in the Europe / Middle Eas
 Just install this addon with the following command.
 Please note to fill in your settings inside the environment variables.
 
-`docker run -d --pull=always -e CONF_updateInterval=300 -e CONF_babelLocale='de' -e CONF_mqtt='@json {"broker": "", "username": "", "password": "", "port": 1883}' -e CONF_volvoData='@json {"username": "", "password": "", "vin": "", "vccapikey": "", "odometerMultiplier": 1, "averageSpeedDivider": 1, "averageFuelConsumptionMultiplier": 1}' -e TZ='Europe/Berlin' --name volvo2mqtt ghcr.io/dielee/volvo2mqtt:latest`
+`docker run -d --pull=always -e CONF_updateInterval=300 -e CONF_babelLocale='de' -e CONF_mqtt='@json {"broker": "", "username": "", "password": "", "port": 1883}' -e CONF_volvoData='@json {"username": "", "password": "", "vin": "", "vccapikey": "", "backupvccapikey": "", "odometerMultiplier": 1, "averageSpeedDivider": 1, "averageFuelConsumptionMultiplier": 1}' -e TZ='Europe/Berlin' --name volvo2mqtt ghcr.io/dielee/volvo2mqtt:latest`
 
 <b>HA Add-On:</b><br>
 
@@ -83,7 +83,7 @@ Here is what every option means:
 | `CONF_volvoData`          | `json`    | `username`                            | **required** | Normally your email address to login into the Volvo App.
 | `CONF_volvoData`          | `json`    | `password`                            | **required** | Your password to login into the Volvo App.
 | `CONF_volvoData`          | `json`    | `vin`                                 | optional     | A single VIN like "VIN1" or a list of VINs like "["VIN1", "VIN2"]". Leave this empty if you don't know your VIN. The addon will use every car that is tied to your account.
-| `CONF_volvoData`          | `json`    | `vccapikey`                           | **required** | API key linked with your volvo developer account. Get your Vccapi key from [here](https://developer.volvocars.com/account/)
+| `CONF_volvoData`          | `json`    | `vccapikey`                           | **required** | VCCAPIKEY linked with your volvo developer account. Get your Vccapi key from [here](https://developer.volvocars.com/account/). <b>Starting version 1.8.0, it is possible to define multiple keys, like this: `["vccapikey1", "vccapikey2", "vccapikey3", "etc..."]`</b>
 | `CONF_volvoData`          | `json`    | `odometerMultiplier`                  | optional     | The multiplier value for the odometer value, as the volvo api delivers inconsistent data. For some cars this setting is 10, for some 1. Try what's right for your car. If you leave it empty, the multiplier will be 1.
 | `CONF_volvoData`          | `json`    | `averageSpeedDivider`                 | optional     | The divider value for the average speed value, as the volvo api delivers inconsistent data. For some cars this setting is 10, for some 1. Try what's right for your car. If you leave it empty, the divider will be 1.
 | `CONF_volvoData`          | `json`    | `averageFuelConsumptionMultiplier`    | optional     | The multiplier value for the average fuel consumption value, as the volvo api delivers inconsistent data. For some cars this setting is 10, for some 1. Try what's right for your car. If you leave it empty, the multiplier will be 1.

src/CHANGELOG.md

@@ -1,3 +1,26 @@
+## v1.8.0
+### 🚀 Features:
+
+- <b>Breaking change: The `vccapikey` setting is now a list. It is possible to
+  add multiple VCCAPIKEYs. This will be helpful if you are extending your 10.000 call limit some times.
+  Change your vccapikey config like this:
+
+  Old:
+  ```
+    "vccapikey": "vccapikey1" 
+  ```
+
+  New:
+  ```
+    vccapikey:
+      - vccapikey1
+      - vccapikey2
+      - vccapikey3
+      - etc.
+  ```
+  More information about this feature [here](https://github.com/Dielee/volvo2mqtt/issues/84).
+    </b>
+
 ## v1.7.10
 ### 🐛 Bug Fixes:
 

src/config.yaml

@@ -1,6 +1,6 @@
 name: "Volvo2Mqtt"
 description: "Volvo AAOS MQTT bridge"
-version: "1.7.10"
+version: "1.8.0"
 slug: "volvo2mqtt"
 init: false
 url: "https://github.com/Dielee/volvo2mqtt"
@@ -42,7 +42,8 @@ schema:
     username: str
     password: str
     vin: str?
-    vccapikey: str
+    vccapikey:
+        - str
     odometerMultiplier: int(1,)
     averageSpeedDivider: int(1,)
     averageFuelConsumptionMultiplier: int(1,)

src/const.py

@@ -1,6 +1,6 @@
 from config import settings
 
-VERSION = "v1.7.10"
+VERSION = "v1.8.0"
 
 OAUTH_URL = "https://volvoid.eu.volvocars.com/as/token.oauth2"
 VEHICLES_URL = "https://api.volvocars.com/connected-vehicle/v1/vehicles"

src/main.py

@@ -10,6 +10,6 @@
     set_mqtt_settings()
     setup_logging()
     logging.info("Starting volvo2mqtt version " + VERSION)
-    authorize()
     connect()
+    authorize()
     update_loop()

src/mqtt.py

@@ -352,3 +352,8 @@ def create_ha_devices():
 
 def send_heartbeat():
     mqtt_client.publish(availability_topic, "online")
+
+
+def send_offline():
+    mqtt_client.publish(availability_topic, "offline")
+

src/settings.json

@@ -12,7 +12,7 @@
     "username": "",
     "password": "",
     "vin": "",
-	"vccapikey": "",
+	"vccapikey": [],
     "odometerMultiplier": "",
     "averageSpeedDivider": "",
     "averageFuelConsumptionMultiplier": ""

src/translations/en.yaml

@@ -16,4 +16,4 @@ configuration:
     description: Leave the settings as they are if you are using the MQTT Mosquitto Addon. If not, take a look at the readme from volvo2mqtt.
   volvoData:
     name: Volvo configuration options
-    description: You have to enter your Volvo app credentials. The username is normally your email, and the password is the password you use for your Volvo app. The vin can stay empty. The Add-On will use any vin inside your Volvo account. VCCAPI key is required and comes from your Volvo developer account. Odometer multiplier is sometimes 10, sometimes 1. The same is applicable for average speed divider and average fuel consumption. Leave it as it is if you don't know what's right. Take a look at the GitHub repo for more information!
+    description: You have to enter your Volvo app credentials. The username is normally your email, and the password is the password you use for your Volvo app. The vin can stay empty. The Add-On will use any vin inside your Volvo account. VCCAPI key is required and comes from your Volvo developer account. This setting works as list, so it is possible to define multiple VCCAPI keys. Odometer multiplier is sometimes 10, sometimes 1. The same is applicable for average speed divider and average fuel consumption. Leave it as it is if you don't know what's right. Take a look at the GitHub repo for more information!

src/util.py

@@ -13,7 +13,8 @@
 TZ = None
 SENSITIVE_PATTERNS = [
     r"[A-Z0-9]{17}",  # VIN
-    r"\d{1,2}\.\d{5,16}"  # Location
+    r"\d{1,2}\.\d{5,16}",  # Location
+    r"[a-z0-9]{32}"  # VCCAPIKEY
 ]
 
 
@@ -48,13 +49,15 @@ def setup_logging():
         '%(asctime)s volvo2mqtt [%(process)d] - %(levelname)s: %(message)s',
         '%b %d %H:%M:%S')
     file_log_handler.setFormatter(formatter)
-    sensitive_data_filter = SensitiveDataFilter(SENSITIVE_PATTERNS)
-    file_log_handler.addFilter(sensitive_data_filter)
-    logger = logging.getLogger()
+
+    if settings["debug"]:
+        sensitive_data_filter = SensitiveDataFilter(SENSITIVE_PATTERNS)
+        file_log_handler.addFilter(sensitive_data_filter)
 
     console_log_handler = logging.StreamHandler(sys.stdout)
     console_log_handler.setFormatter(formatter)
 
+    logger = logging.getLogger()
     logger.addHandler(console_log_handler)
     logger.addHandler(file_log_handler)
     logging.getLogger("urllib3").setLevel(logging.WARNING)

src/volvo.py

@@ -3,6 +3,7 @@
 import mqtt
 import util
 import time
+import re
 from threading import currentThread
 from datetime import datetime, timedelta
 from config import settings
@@ -14,7 +15,7 @@
 
 session = requests.Session()
 session.headers = {
-    "vcc-api-key": settings["volvoData"]["vccapikey"],
+    "vcc-api-key": "",
     "content-type": "application/json",
     "accept": "*/*"
 }
@@ -24,6 +25,7 @@
 vins = []
 supported_endpoints = {}
 cached_requests = {}
+vcc_api_keys = []
 
 
 def authorize():
@@ -42,12 +44,13 @@ def authorize():
     auth = requests.post(OAUTH_URL, data=body, headers=headers)
     if auth.status_code == 200:
         data = auth.json()
-        session.headers.update({'authorization': "Bearer " + data["access_token"]})
+        session.headers.update({"authorization": "Bearer " + data["access_token"]})
 
         global token_expires_at, refresh_token
         token_expires_at = datetime.now(util.TZ) + timedelta(seconds=(data["expires_in"] - 30))
         refresh_token = data["refresh_token"]
 
+        get_vcc_api_keys()
         get_vehicles()
         check_supported_endpoints()
     else:
@@ -77,7 +80,7 @@ def refresh_auth():
 
     if auth.status_code == 200:
         data = auth.json()
-        session.headers.update({'authorization': "Bearer " + data["access_token"]})
+        session.headers.update({"authorization": "Bearer " + data["access_token"]})
 
         global token_expires_at
         token_expires_at = datetime.now(util.TZ) + timedelta(seconds=(data["expires_in"] - 30))
@@ -88,8 +91,8 @@ def get_vehicles():
     global vins
     if not settings.volvoData["vin"]:
         vehicles = session.get(VEHICLES_URL)
+        data = vehicles.json()
         if vehicles.status_code == 200:
-            data = vehicles.json()
             if len(data["data"]) > 0:
                 for vehicle in data["data"]:
                     vins.append(vehicle["vin"])
@@ -115,6 +118,89 @@ def get_vehicles():
         logging.info("Vin: " + str(vins) + " found!")
 
 
+def get_vcc_api_keys(used_key=None):
+    setting_keys = settings.volvoData["vccapikey"]
+    if isinstance(setting_keys, str):
+        set_key_state(setting_keys)
+    elif isinstance(setting_keys, list):
+        for key in setting_keys:
+            set_key_state(key)
+
+    logging.debug(str(vcc_api_keys))
+    working_keys = [key["key"] for key in vcc_api_keys if not key.get("extended") and key.get('key') != used_key]
+    if len(working_keys) < 1:
+        logging.warning("No working VCCAPIKEY found, waiting 10 minutes. Then trying again!")
+        mqtt.send_offline()
+        time.sleep(600)
+        get_vcc_api_keys(used_key=None)
+        return None
+
+    mqtt.send_heartbeat()
+    session.headers.update({"vcc-api-key": working_keys[0]})
+    logging.info("Using VCCAPIKEY: " + working_keys[0])
+    for key_dict in vcc_api_keys:
+        if key_dict["key"] == working_keys[0]:
+            key_dict["in_use"] = True
+
+
+def set_key_state(key):
+    global vcc_api_keys
+    list_index = next((index for (index, d) in enumerate(vcc_api_keys) if d["key"] == key), None)
+
+    if list_index or list_index == 0:
+        extended, extended_until = check_vcc_api_key(key, vcc_api_keys[list_index]["extended_until"])
+        vcc_api_keys[list_index] = ({"key": key, "extended": extended,
+                                     "extended_until": extended_until, "in_use": False})
+    else:
+        extended, extended_until = check_vcc_api_key(key)
+        vcc_api_keys.append({"key": key, "extended": extended,
+                             "extended_until": extended_until, "in_use": False})
+
+
+def check_vcc_api_key(test_key, extended_until=None):
+    if extended_until:
+        if extended_until >= datetime.now():
+            return True, extended_until
+
+    if datetime.now(util.TZ) >= token_expires_at:
+        refresh_auth()
+
+    token = session.headers.get("authorization")
+    headers = {
+        "vcc-api-key": test_key,
+        "content-type": "application/json",
+        "accept": "*/*",
+        "authorization": token
+    }
+
+    response = requests.get(VEHICLES_URL, headers=headers)
+    data = response.json()
+    if response.status_code == 200:
+        logging.debug("VCCAPIKEY " + test_key + " works!")
+        return False, None
+    elif response.status_code == 403 and "message" in data:
+        if "Out of call volume quota" in data["message"]:
+            reuse_search = re.search(r"\d{2}\:\d{2}\:\d{2}", data["message"])
+            if reuse_search:
+                reusable_in = reuse_search.group(0).split(":")
+                now = datetime.now()
+                extended_until = now + timedelta(hours=int(reusable_in[0]),
+                                                 minutes=int(reusable_in[1]),
+                                                 seconds=int(reusable_in[2]) + 10)
+                logging.warning("VCCAPIKEY " + test_key + " is extended and will be reusable at: "
+                                + format_datetime(extended_until, format="medium", locale=settings["babelLocale"]))
+        else:
+            logging.warning("VCCAPIKEY " + test_key + " isn't working! " + data["error"]["message"])
+    else:
+        logging.warning("VCCAPIKEY " + test_key + " isn't working! " + data["error"]["message"])
+    return True, extended_until
+
+
+def change_vcc_api_key():
+    used_vcc_api_key = session.headers.get("vcc-api-key")
+    get_vcc_api_keys(used_vcc_api_key)
+
+
 def get_vehicle_details(vin):
     response = session.get(VEHICLE_DETAILS_URL.format(vin), timeout=15)
     if response.status_code == 200:
@@ -171,8 +257,10 @@ def check_supported_endpoints():
 
 def initialize_scheduler(vins):
     for vin in vins:
+        topic = f"homeassistant/schedule/{vin}/command"
         mqtt.active_schedules[vin] = {"timers": []}
-        mqtt.subscribed_topics = [f"homeassistant/schedule/{vin}/command"]
+        mqtt.subscribed_topics = [topic]
+        mqtt.mqtt_client.subscribe(topic)
 
 
 def initialize_climate(vins):
@@ -223,15 +311,14 @@ def check_engine_status(vin):
         time.sleep(5)
 
 
-def api_call(url, method, vin, sensor_id=None, force_update=False):
-    global token_expires_at
+def api_call(url, method, vin, sensor_id=None, force_update=False, key_change=False):
     if datetime.now(util.TZ) >= token_expires_at:
         refresh_auth()
 
     if url in [RECHARGE_STATE_URL, WINDOWS_STATE_URL, LOCK_STATE_URL, TYRE_STATE_URL,
                STATISTICS_URL, ENGINE_DIAGNOSTICS_URL]:
         # Minimize API calls for endpoints with multiple values
-        response = cached_request(url, method, vin, force_update)
+        response = cached_request(url, method, vin, force_update, key_change)
         if response is None:
             # Exception caught while getting data from volvo api, doing nothing
             return None
@@ -254,9 +341,11 @@ def api_call(url, method, vin, sensor_id=None, force_update=False):
         return None
 
     logging.debug("Response status code: " + str(response.status_code))
+    data = response.json()
+
     if response.status_code == 200:
-        data = response.json()
         logging.debug(response.text)
+        return parse_api_data(data, sensor_id)
     else:
         logging.debug(response.text)
         if url == CLIMATE_START_URL and response.status_code == 503:
@@ -267,13 +356,20 @@ def api_call(url, method, vin, sensor_id=None, force_update=False):
             # Suppress 403 errors for unsupported extended-vehicle api cars
             logging.debug("Suppressed 403 for extended-vehicle API")
             return None
+        elif response.status_code == 403 and "message" in data:
+            if "Out of call volume quota" in data["message"]:
+                logging.warn("Quota extended. Try to change VCCAPIKEY!")
+                change_vcc_api_key()
+                api_call(url, method, vin, sensor_id, force_update, True)
+            else:
+                logging.error(
+                    "API Call failed. Status Code: " + str(response.status_code) + ". Error: " + response.text)
         else:
             logging.error("API Call failed. Status Code: " + str(response.status_code) + ". Error: " + response.text)
         return None
-    return parse_api_data(data, sensor_id)
 
 
-def cached_request(url, method, vin, force_update=False):
+def cached_request(url, method, vin, force_update=False, key_change=False):
     global cached_requests
     if not util.keys_exists(cached_requests, vin + "_" + url):
         # No API Data cached, get fresh data from API
@@ -289,8 +385,9 @@ def cached_request(url, method, vin, force_update=False):
     else:
         if (datetime.now(util.TZ) - cached_requests[vin + "_" + url]["last_update"]).total_seconds() \
                 >= settings["updateInterval"] or (force_update and
-                                                  (datetime.now(util.TZ) - cached_requests[vin + "_" + url]
-                                                  ["last_update"]).total_seconds() >= 2):
+                                                  (datetime.now(util.TZ) - cached_requests[vin + "_" + url][
+                                                      "last_update"]).total_seconds() >= 2) \
+                or key_change:
             # Old Data in Cache, or force mode active, updating
             logging.debug("Starting " + method + " call against " + url)
             try: