Update objects timestamps on cache hit #542
Conversation
| Object: object, | ||
| } | ||
|
|
||
| _, err := c.mcore.ComposeObject(context.Background(), dst, src) |
There was a problem hiding this comment.
I see that the copying is done server side- nice. Are there any downsides to having this always on?
There was a problem hiding this comment.
Good question - I don't see any downsides in terms of performance or storage usage but I'm running self hosted s3 storage. As I understand AWS s3 pricing documentation there is some cost for coping object - you pay the S3 charges for storage in the selected destination S3 storage classes, for the primary copy, for replication PUT request. So making this turn on by default could potentially generate costs for people using AWS so maybe it's better to leave it as opt-in?
There was a problem hiding this comment.
OK, let's make it opt-in to be on the safe side.
…it is trying to copy an object to itself without changing the object’s metadata` error
|
Thanks for the contribution- merged. |
In this PR I would like to introduce option to leverage S3 storage lifecycle to automatically remove not used objects after specified amount of time.
We can configure S3 bucket to remove objects after certain amount of time from its modification date. To update modification date, on each cache hit we need to copy object in place - that's limitation of s3. We can now pass
s3.update_timestampsoption in config to use it.