Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to preserve POSIX metadata #23

Open
asaldanha-generate opened this issue Aug 30, 2024 · 0 comments
Open

Add option to preserve POSIX metadata #23

asaldanha-generate opened this issue Aug 30, 2024 · 0 comments

Comments

@asaldanha-generate
Copy link
Contributor

When TAR-ing files that are used with FSX or Storage Gateway, or un-TARing files to S3 that may then be used by FSX or Storage Gateway, it would be convenient if POSIX metadata including filetype, permissions, uid and gid could be preserved in the S3 Metadata headers as described in https://docs.aws.amazon.com/fsx/latest/LustreGuide/posix-metadata-support.html. This involves two enhancements

  1. When creating an archive with s3tar, S3 Object Metadata is fetched and used to populate the POSIX file type, permissions, uid and gid.

    • x-amz-meta-file-owner is parsed as decimal to set the UID
    • x-amz-meta-file-group parsed as decimal to set the GID
    • x-amz-meta-file-permissions parsed as octal to set the Mode bits
  2. Conversely, when extracting a file the TAR header is read prior to creating the object in S3, and

    • x-amz-meta-file-owner is set to the decimal representation of the UID
    • x-amz-meta-file-group is set to the decimal representation of the GID
    • x-amz-meta-file-permissions set to the octal representation of the Mode bits
alokito pushed a commit to alokito/amazon-s3-tar-tool that referenced this issue Aug 30, 2024
alokito pushed a commit to alokito/amazon-s3-tar-tool that referenced this issue Aug 30, 2024
alokito pushed a commit to alokito/amazon-s3-tar-tool that referenced this issue Sep 2, 2024
alokito pushed a commit to alokito/amazon-s3-tar-tool that referenced this issue Sep 2, 2024
alokito pushed a commit to alokito/amazon-s3-tar-tool that referenced this issue Sep 3, 2024
alokito pushed a commit to alokito/amazon-s3-tar-tool that referenced this issue Sep 3, 2024
alokito pushed a commit to alokito/amazon-s3-tar-tool that referenced this issue Oct 1, 2024
ctime is not mentioned in AWS docs, but is set by AWS Storage Gateway.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant