Refactor Storage Calculator to be more robust

[seanhamlin]

Checklist

• Affected Issues have been mentioned in the Closing issues section
• Documentation has been written/updated
• PR title is ready for changelog and subsystem label(s) applied

Explain the details for making this change. What existing problem does the pull request solve?

It was noticed that storage-calculator was not running for all projects in a large dedicated cluster with it's own Lagoon Core. This PR refactors the storage calculator to:

• Use consistent code style
• Easier to read debug messages
• Move guard statements to the top to avoid unnecessary nesting of code
• Remove exit 1 which potentially halts the storage calculator before it runs over all projects
• Use of --ignore-not-found=true and less piping to /dev/null
• Have a timeout of 30 seconds for the storageCalc pod rollout. To prevent RWO PVs from stalling the script for too long. The current default is 10 minutes.
• Add a new environment variable LAGOON_STORAGE_IGNORE_REGEX to which can be used to selectively skip certain PVCs by name. e.g. it is easy now to skip reading all the solr PVCs now.

I also added README.md file to explain a lot of the things.

Things that would make great follow up PRs:

• Move away from python-yq to Go yq.
• Fixing StorageCalc should not run on RWO PVs that are already mounted #2789

Related issues

• Related to Lagoon API lists bytesUsed when it is actually kiloBytesUsed #2245

[rocketeerbkw]

Adding more details about the issue and reason the changes here will help from internal comms:

• there are a lot of projects (each with 1+ environments)
• projects are enumerated through in id order, meaning new projects are done last
• if there is a RWO PV in use, then the process stalls for 10 minutes to wait for it to fail to spawn
• the cron process physically cannot complete in 24 hours, and thus new projects are just skipped.