✨ Is922/add csv export functionality (⚠️ devops)

[matusdrobuliak66]

What do these changes do?

• ✨ Introducing CSV export functionality using postgres aws-s3 extension (Can be reused for any Postgres to S3 export of data, also tested for huge exports with a couple of GBs)
• Concept:
  • I have added an RDS policy that can export data only! to the simcore-<deployment>-rds-exports bucket.
  • Installed RDS extension
  • You can use the code: SELECT * from aws_s3.query_export_to_s3('{your query}', aws_commons.create_s3_uri('{bucket_name}', '{s3_key}', 'us-east-1'), 'format csv, HEADER true');
  • Using a pre-signed download link we give access to the file for 2 hours.
  • Bucket policy is set that it deletes the data that's there longer than 2 days

First version doesn't expect huge data, therefore discussed with @ignapas and implemented as redirect:

Related issue/s

• relates Add export CSV functionality osparc-issues#1242

How to test

Dev Checklist

• No ENV changes or I properly updated ENV (read the instruction)

DevOps Checklist

• ⚠️ adding all S3_SETTINGS variables
  https://git.speag.com/oSparc/osparc-ops-deployment-configuration/-/merge_requests/317
  • RESOURCE_USAGE_TRACKER_S3 env var filled with the correct value for
    • osparc-staging.io
    • sim4life.io
    • osparc.io

[pcrespov]

Very nice!
Please check the comments and come back to me if you have any questions on my suggetions

[sanderegg]

This is looking very good! I have a few comments/questions and we can gladly go through.

[pcrespov]

🎉

[YuryHrytsuk]

Waiting for the ENV PR 👍

[GitHK]

please consider my suggestion

[sonarqubecloud]

Quality Gate passed

Kudos, no new issues were introduced!

0 New issues
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

[sanderegg]

added some more comments for the next PR coming. thanks a lot.