Cleanup does not fully delete objects on S3 with versioning enabled

[tnubeo1111]

I encountered an issue with Harbor's cleanup feature when using it with an S3 bucket that has versioning enabled.

After deleting an image from Harbor and running the cleanup feature, I noticed that the associated object on S3 was not fully removed. Specifically:

The original file (e.g., fe10625e-6d4b-d9cf-a158-fa163e3c5253 with a size of 71 B) remains on S3 after the cleanup, without a delete marker.
A new file with name "fe106247-c6d6-f18f-a158-fa163e3c5253" a delete marker (0 B in size) is created at the same time when I run the cleanup on Harbor.
My S3 bucket has versioning enabled, and I see that the cleanup process only creates delete markers for the new versions but does not delete or mark the original version. This leads to older versions of the object remaining on S3.

Questions:

1. Is this the expected behavior of the cleanup feature when working with S3 versioning? If not, is there a way to configure cleanup to fully delete older versions of the objects?
2. How can I ensure that older objects are fully removed from S3 during the cleanup process?
   Thank you!

The S3 I am currently using is Cloudian.

[MinerYang]

Hi @tnubeo1111 ,

For enable S3 storage versioning , any changes to an object will be stored as a new version. It is more like a S3 behavior or limitation, please refer to the upstream distribution issue. Within this contexts, we are not recommend to enable S3 versioning unless you could handle the lifecycle configuration rules for objects yourself properly.

• S3 Storage has issues with S3 versioning & deletion. distribution/distribution#1049