Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support UnixFS 1.5 file mode and modification times #34

Closed
wants to merge 8 commits into from

Conversation

kstuart
Copy link

@kstuart kstuart commented Jan 21, 2023

Adds support for storing and retrieving file mode and last modification time.

Support added to:

  • Files
  • LinkFiles
  • Webfiles
  • Directories
  • Tar Archives

When the TAR archive (headers) include a file mode or modification time, the extractor will restore that metadata when supported for the underlying filesystem.

The Golang runtime currently does not support changing the times on a symlink, for Linux and some BSDs a custom solution has been implemented, for Darwin this is not the case so when copying a symlink to the filesystem the last modification time is not updated. Since for concrete files and directories stored modes and modification times are faithfully restored to the filesystem this should not be a breaking issue, a similar solution to that implemented for Linux/BSDs is likely implementable by a developer with access to a Darwin platform.

Replaces PRs:

Relates to ipfs/kubo#6920

@welcome
Copy link

welcome bot commented Jan 21, 2023

Thank you for submitting this PR!
A maintainer will be here shortly to review it.
We are super grateful, but we are also overloaded! Help us by making sure that:

  • The context for this PR is clear, with relevant discussion, decisions
    and stakeholders linked/mentioned.

  • Your contribution itself is clear (code comments, self-review for the
    rest) and in its best form. Follow the code contribution
    guidelines

    if they apply.

Getting other community members to do a review would be great help too on complex PRs (you can ask in the chats/forums). If you are unsure about something, just leave us a comment.
Next steps:

  • A maintainer will triage and assign priority to this PR, commenting on
    any missing things and potentially assigning a reviewer for high
    priority items.

  • The PR gets reviews, discussed and approvals as needed.

  • The PR is merged by maintainers when it has been approved and comments addressed.

We currently aim to provide initial feedback/triaging within two business days. Please keep an eye on any labelling actions, as these will indicate priorities and status of your contribution.
We are very grateful for your contribution!

@codecov
Copy link

codecov bot commented Jan 26, 2023

Codecov Report

Merging #34 (7161a2f) into main (c444cbb) will increase coverage by 2.19%.
The diff coverage is 78.01%.

❗ Current head 7161a2f differs from pull request most recent head 77cdc80. Consider uploading reports for the commit 77cdc80 to get more accurate results

Impacted file tree graph

@@            Coverage Diff             @@
##             main      #34      +/-   ##
==========================================
+ Coverage   63.75%   65.94%   +2.19%     
==========================================
  Files          32       30       -2     
  Lines        1694     1856     +162     
==========================================
+ Hits         1080     1224     +144     
- Misses        477      486       +9     
- Partials      137      146       +9     
Impacted Files Coverage Δ
files/serialfile.go 63.72% <0.00%> (-2.61%) ⬇️
tar/extractor.go 51.04% <51.31%> (-0.36%) ⬇️
tar/util_posix.go 60.00% <60.00%> (ø)
files/multipartfile.go 83.51% <78.84%> (-2.41%) ⬇️
files/util.go 75.86% <89.47%> (+25.86%) ⬆️
files/tarwriter.go 83.52% <93.54%> (+10.91%) ⬆️
files/linkfile.go 64.00% <100.00%> (+24.00%) ⬆️
files/multifilereader.go 85.45% <100.00%> (+3.43%) ⬆️
files/readerfile.go 77.08% <100.00%> (+10.41%) ⬆️
files/slicedirectory.go 80.32% <100.00%> (+6.99%) ⬆️
... and 1 more

... and 3 files with indirect coverage changes

@kstuart kstuart requested a review from a team as a code owner March 12, 2023 21:08
Currently the Golang runtime doesn't support changing the time on a symbolic link.
@kstuart kstuart changed the title support UnixFS 1.5 file mode and modification time feat: support UnixFS 1.5 file mode and modification times Mar 12, 2023
@guseggert guseggert added P1 High: Likely tackled by core team if no one steps up kind/enhancement A net-new feature or improvement to an existing feature need/maintainers-input Needs input from the current maintainer(s) labels Mar 13, 2023
Jorropo pushed a commit to ipfs/go-libipfs-rapide that referenced this pull request Mar 23, 2023
@zacharywhitley
Copy link

Any status on where this is? It seems to be stalled.

@kstuart
Copy link
Author

kstuart commented Oct 30, 2023

@zacharywhitley, unfortunately I don't think there's any real interest in this feature; This has been awaiting review for some time, and the completed ipfs/kubo#6920 PRs prior to the refactoring met the same fate.

Copy link
Member

@lidel lidel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See ipfs/kubo#7754 (review)

Something we should add in this PR is to update boxo/gateway as well, and expose mtime as Last-Modified on HTTP Gateway responses (if present)

t.Fatalf("Expected file modified time to be [%v] but got [%v]", mtime, rf.ModTime())
}
}

func TestMultipartFiles(t *testing.T) {
Copy link
Member

@lidel lidel May 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💭 are these required, and we need to effectively break the wire format? Could we avoid it and be backward compatible?

If so, we want to keep original tests without mode and mtime being sent over the wire.
Perhaps refactor into separate TestMultipartFiles and TestMultipartFilesWithMtime?

@lidel lidel requested a review from hacdias May 17, 2024 15:48
@hacdias hacdias removed their assignment May 27, 2024
@hacdias hacdias removed their request for review May 27, 2024 12:42
gammazero added a commit that referenced this pull request Aug 6, 2024
Replaces PR #34 written by @kstuart

Adds support for storing and retrieving file mode and last modification time.

Support added to:
- [X] Files
- [X] LinkFiles
- [X] Webfiles
- [X] Directories
- [X] Tar Archives

When the TAR archive (headers) include a file mode or modification time, the extractor will restore that metadata when supported for the underlying filesystem.

The Golang runtime currently does not support changing the times on a symlink, for Linux and some BSDs a custom solution has been implemented, for Darwin this is not the case so when copying a symlink to the filesystem the last modification time is not updated. Since for concrete files and directories stored modes and modification times are faithfully restored to the filesystem this should not be a breaking issue, a similar solution to that implemented for Linux/BSDs is likely implementable by a developer with access to a Darwin platform.

Replaces PRs:
- ipfs/go-ipfs-files#31
- ipfs/tar-utils#11
- #34

Relates to ipfs/kubo#6920
@gammazero
Copy link
Contributor

Replaced by PR #653

@gammazero gammazero closed this Aug 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/enhancement A net-new feature or improvement to an existing feature need/maintainers-input Needs input from the current maintainer(s) P1 High: Likely tackled by core team if no one steps up
Projects
No open projects
Status: 🔎 In Review
Development

Successfully merging this pull request may close these issues.

6 participants