How to handle disk space usage of debug files

[emrys90]

Problem Statement

I build for 3 Unity platforms, and the debug files that gets uploaded per platform is over 1gb. So per build that is over 3gb of debug files that gets uploaded.

I typically do beta builds around once a day or so. Which adds up to over 100gb of debug files per month.

What are my options for handling this? I only really care about the most recent beta and production debug files, and not previous builds.

Solution Brainstorm

No response

[hubertdeng123]

You could have a scheduled cron job that cleans up those debug files. If anything is older than a few days old. Something like

self-hosted/docker-compose.yml

Line 481 in da06c0f

command: '"0 0 * * * find /var/lib/sentry-profiles -type f -mtime +$SENTRY_EVENT_RETENTION_DAYS -delete"'

, but cleaning up the sentry-data volume instead, which is what I think houses debug files.

[emrys90]

So there's no built in system for it, for example something like SENTRY_EVENT_RETENTION_DAYS, but that is for debug files instead?

And I don't think deleting anything older than a couple days on its own would work? As according to documentation, debug files are deleted when not used for more than 90 days, rather than being 90 days old. So I would want to delete any files not used for more than 2 weeks or so potentially, which should cover making sure they are always available if needed.

[hubertdeng123]

There is not a built in system for this, no. I have looked into this more, and I believe our sentry cleanup job that runs periodically that should clean up the models corresponding to the files and debug file models that live in postgres. However, I believe in order to actually free up disk space you would need to do a VACUUM FULL or something similar to ensure that the disk space is freed up.

https://www.rdiachenko.com/posts/databases/postgresql/postgresql-does-not-free-up-physical-space-after-delete/#:~:text=Q%3A%20How%20Do%20I%20Reclaim,frees%20space%20for%20future%20use.

[emrys90]

Could I propose this being added as a feature then? Seems like it would be useful for anyone uploading debug files on a regular basis. I have no knowledge of the code base, but I would assume it would just be converting whatever is running the duration for cleaning up debug files to use an environment variable instead so users can configure it.

[hubertdeng123]

Sure, this can be a feature request. Please be aware this is probably not something we can get to in the near term though.

[emrys90]

Great, thank you!