HDDS-11661. ITestS3AContractBulkDelete fails with FSO bucket

[chungen0126]

What changes were proposed in this pull request?

New contract test ITestS3AContractBulkDelete (added by HADOOP-18679 in Hadoop 3.4.1) is failing with FSO bucket.
Bulk delete doesn't delete directories. Therefore, It passed path without trailing slash. In the original implementation, directories would be deleted if paths were passed without trailing slashes.

To fix this, delete the directory only if the trailing slash present in the path on the server side. Also, make sure that directory paths are passed with trailing slashes on the client side.

https://issues.apache.org/jira/browse/HDDS-11661

How was this patch tested?

CI: https://github.com/chungen0126/ozone/actions/runs/12308665244

[adoroszlai]

Thanks @chungen0126 for the patch, and the description of the problem.

To fix this, delete the directory only if the trailing slash present in the path on the server side. Also, make sure that directory paths are passed with trailing slashes on the client side.

I think we should consider client-server compatibility across versions. What if old client sends the request, without /, to new server, which expects /?

Current xcompat tests probably do not cover this operation.

[chungen0126]

@adoroszlai Thanks for your review.

I think we should consider client-server compatibility across versions. What if old client sends the request, without /, to new server, which expects /?

That sounds like a problem. I am adding a new argument for requests from S3 to let the server know where the requests are coming from and only check the requests from S3. This might solve the problem. Do you think we need to add a test case in the xcompat tests for this?

[adoroszlai]

I am adding a new argument for requests from S3 to let the server know where the requests are coming from and only check the requests from S3

Instead of adding the flag, please use ClientVersion, which is already sent in requests. Add a new constant and check for it in OM, like:

ozone/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/bucket/OMBucketCreateRequest.java

Lines 474 to 475 in 008f9a6

	if (ClientVersion.fromProtoValue(req.getVersion())
	.compareTo(ClientVersion.BUCKET_LAYOUT_SUPPORT) < 0) {

Do you think we need to add a test case in the xcompat tests for this?

Yes, please.

[errose28]

I think adding new client versions is still blocked on #6932 or else they will not be correctly validated cc @swamirishi

[adoroszlai]

I think adding new client versions is still blocked on #6932 or else they will not be correctly validated

Nice catch. We need to get that one fixed. This contract test can wait.

[chungen0126]

I think adding new client versions is still blocked on #6932 or else they will not be correctly validated cc @swamirishi

Thanks @errose28 for the mentioning it. I actually encounter that problem.