Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Renaming a file to have the same name causes it to be deleted, error raised #277

Closed
spdjudd opened this issue Oct 14, 2022 · 2 comments
Closed

Comments

@spdjudd
Copy link
Contributor

spdjudd commented Oct 14, 2022

Hi,

Thanks for your work on cloudpathlib, much appreciated!

I came across this issue when running a script to rename a set of files - one of them happened to have the correct name already, but calling CloudPath('s3://bucket/folder/filea.pq').rename(CloudPath('s3://bucket/folder/filea.pq')) causes the file to be lost, and an error raised:

    448 def rename(self, target: "CloudPath") -> "CloudPath":
    449     # for cloud services replace == rename since we don't just rename,
    450     # we actually move files
--> 451     return self.replace(target)

File ~\anaconda3\lib\site-packages\cloudpathlib\cloudpath.py:445, in CloudPath.replace(self, target)
    442 if target.exists():
    443     target.unlink()
--> 445 self.client._move_file(self, target)
    446 return target

File ~\anaconda3\lib\site-packages\cloudpathlib\s3\s3client.py:217, in S3Client._move_file(self, src, dst, remove_src)
    213 if src == dst:
    214     o = self.s3.Object(src.bucket, src.key)
    215     o.copy_from(
    216         CopySource={"Bucket": src.bucket, "Key": src.key},
--> 217         Metadata=self._get_metadata(src).get("extra", {}),
    218         MetadataDirective="REPLACE",
    219     )
    221 else:
    222     target = self.s3.Object(dst.bucket, dst.key)

...
NoSuchKey: An error occurred (NoSuchKey) when calling the GetObject operation: The object 'folder/filea.pq' was not found in the bucket 'bucket'

Seems like checking target to see if it's the same path as self before unlinking could prevent this - happy to create a PR if that sounds ok?

@pjbull
Copy link
Member

pjbull commented Oct 15, 2022

@spdjudd Yep, that seems right, good catch! Would be happy to accept a PR with that fix and a test added. Thanks.

@pjbull
Copy link
Member

pjbull commented Oct 20, 2022

Closed by #279

@pjbull pjbull closed this as completed Oct 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants