S3 TransferManager do not handle S3 object version while copying objects

[kamoor]

com.amazonaws.services.s3.transfer.TransferManager ends up with error if you try to copy an object with a version. This is because MetaData object do not consider the version while setting prior copy. See line 1776 in TranferManager.

line 1776
GetObjectMetadataRequest getObjectMetadataRequest = new GetObjectMetadataRequest(
copyObjectRequest.getSourceBucketName(), copyObjectRequest.getSourceKey())
.withSSECustomerKey(copyObjectRequest.getSourceSSECustomerKey());

[varunnvs92]

Which version of the SDK are you using? Can you provide a test case to repo the issue and error log?

[kiiadi]

Are these large objects being copied? The TransferManager automatically uses a multi-part copy if the object is over a certain size (default is 5GB - see here TransferManagerConfiguration.java)

Unfortunately S3 doesn't handle meta-data copy on a multi-part copy - see related issue #367

You can explicitly set the metadata when you initiate the copy request and in this case it will get persisted -

copyObjectRequest.withNewObjectMetadata(metadata);

[kamoor]

@kiiadi I think copy method in TransferManager will always create a new GetObjectMetadataRequest regardless of the fact copyObjectRequest has metadata set. See line 1776 in TransferManager

[kiiadi]

What version of the SDK are you using? The TransferManager creates a CopyCallable object with the details of the CopyObjectRequest - when a multi-part copy is initiated; if newObjectMetadata exists on the CopyObjectRequest it will get used otherwise a new ObjectMetadata object is created.

See https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-s3/src/main/java/com/amazonaws/services/s3/transfer/internal/CopyCallable.java#L239

[kiiadi]

Closing aged issue, please feel free to reopen if this is still a problem.