Superset has issues using SECRET_KEY to decrypt encrypted data in postgres

[HumairAK]

Upon reboot the init container will crash reporting the following:

  File "/usr/local/lib/python3.7/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 414, in process_result_value
    decrypted_value = self.engine.decrypt(value)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 128, in decrypt
    raise ValueError('Invalid decryption key')
ValueError: Invalid decryption key

But the SECRET_KEY is specified in the superset_config.py and pulled from a secret that never changes, so I'm not sure why this happens. To fix this issue we have to clear out the data that is using this key to encrypt said data. AFAIK this is only in the public.dbs table, the field password and encrypted_extra running the following fixes it:

$ oc rsh ${SUPERSET_DB_POD}
sh-4.2$ psql
postgres=# \connect supersetdb
supersetdb=# UPDATE public.dbs SET password = null;
supersetdb=# UPDATE public.dbs SET encrypted_extra = null;

However doing this requires one to go back and update the database password by re-entering them again, lest they won't work.

Possibly related threads:

• Superset 0.39 => 1.3.0 upgrade fails with ValueError: Invalid decryption key apache/superset#17059
• Change SECRET_KEY and get error ValueError: Invalid decryption key on K8s apache/superset#8538
• [Question] What is SECRET_KEY used in superset for? apache/superset#7190

[HumairAK]

Okay pretty sure the reason why is because we set the SECRET_KEY to an env var that is not available to the init container, thus the init container uses the default value as specified here

Fix should be: operate-first/apps#1227