[Question] WALG_DISABLE_S3_SSE and S3 compliant endpoint

[Lebvanih]

Hello,

Some context for our installation in case this helps:

• S3 used to store the backups
• WAL-G used for backup and restore (we have issue with WAL-E and the AWS_REGION as our S3 "provider" doesn't seems 100% compliant with what is expected..)
• WALG_DISABLE_S3_SSE = False
• Our S3 compliant system consist of MINIO-GW + KES in order to handle SSE at our gateway level, with a Swift S3 backend behind it (that doesn't support SSE). A test using rclone requesting encryption works, and if we inspect the S3 we notice the file are indeed encryption properly by the MINIO-GW.
• All in a K8S environment.

Regarding our issue/question: I am a bit confused regarding the WALG_DISABLE_S3_SSE variable.

It is said in the documentation the following:

WALE_DISABLE_S3_SSE, WALG_DISABLE_S3_SSE: by default wal-e/wal-g are configured to encrypt files uploaded to S3. In order to disable it you can set this environment variable to true.

However we noticed that the files were pushed without any request of encryption. I checked inside the pod in the wal-e envdir, and noticed that the variable WALG_DISABLE_S3_SSE was set to True, while the doc, and our definition said "false".

Looking at the code here https://github.com/zalando/spilo/blob/master/postgres-appliance/scripts/configure_spilo.py#L787 , I see that we are in that case, as we aren't using S3 in AWS (as we have a endpoint defined)
Then a few line below: https://github.com/zalando/spilo/blob/master/postgres-appliance/scripts/configure_spilo.py#L801-802. I see that SSE is disabled without any comment on the reason:

wale.update(WALE_S3_ENDPOINT=wale_endpoint, AWS_ENDPOINT=aws_endpoint,
                        WALG_DISABLE_S3_SSE='true', WALE_DISABLE_S3_SSE='true')

Reading the function seems to not show any other way to enable it.
Could you confirm that my understanding is correct, and that if we use a S3 compliant installation, there is, currently no way, to force WAL-G to request encryption? We did workaround for now using the auto encrypt at our minio gateway to force everything to be encrypted even if not requested, but this sounds like a ugly hack more than anything for us...

Maybe this should be clarified in the documentation as well if it's expected. As the documentation mention that it is supposed to be enabled.

Kr,

[CyberDem0n]

Could you confirm that my understanding is correct, and that if we use a S3 compliant installation, there is, currently no way, to force WAL-G to request encryption?

Originally the WALG_DISABLE_S3_SSE was introduced because it seems that S3 compatible installations didn't support encryption. Later (and I think logically) it was set to true by default.

It shouldn't be hard to fix it and set it to true only if no other value is set:

if not wale.get('WALG_DISABLE_S3_SSE'):
    wale['WALG_DISABLE_S3_SSE'] = 'true'

and the same for WALE_DISABLE_S3_SSE.