gh-107845: Fix symlink handling for tarfile.data_filter #107846

encukou · 2023-08-10T15:39:55Z

Issue: tarfile.data_filter wrongly rejects some tarballs with symlinks #107845

📚 Documentation preview 📚: https://cpython-previews--107846.org.readthedocs.build/

frenzymadness

I've found only one comment which might be improved.

As I'm thinking about more possible tests, I can imagine two more types:

Chains of symlinks/hardlinks, something like symlink → hardlink → file or hardlink → symlink → file etc.
(This can be combined with idea 1) Tests of archives where the symlinks/hardlinks are added before their destinations. So for the first example above, it'd be add symlink pointing to nonexisting hardlink, add hardlink to nonexisting file, and then add the target file.

Lib/test/test_tarfile.py

vstinner

LGTM.

The doc and tarfile file LGTM. I didn't have time to review carefully the tests. Maybe first merge into main, see how it goes on buildbots, and only later backport to all branches. Sometimes, we have bad surprises.

Doc/library/tarfile.rst

Co-authored-by: Victor Stinner <vstinner@python.org> Co-authored-by: Lumír 'Frenzy' Balhar <frenzy.madness@gmail.com>

Lib/test/test_tarfile.py

encukou · 2023-08-17T13:11:24Z

AFAIK you can't extract a hardlink to file that's later in the archive (unless the file already exists on disk). Tarfile only searches entries before the link.
I added a test for link chains.

Co-authored-by: Victor Stinner <vstinner@python.org>

bedevere-bot · 2023-08-17T14:26:44Z

🤖 New build scheduled with the buildbot fleet by @encukou for commit cebb318 🤖

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

bedevere-bot · 2023-08-21T09:13:42Z

🤖 New build scheduled with the buildbot fleet by @encukou for commit cebb318 🤖

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

encukou · 2023-08-21T10:56:30Z

Can't seem to get the buildbots to run, so I'll watch them after merging.

miss-islington · 2023-08-21T11:48:14Z

Thanks @encukou for the PR 🌮🎉.. I'm working now to backport this PR to: 3.9.
🐍🍒⛏🤖

miss-islington · 2023-08-21T11:48:15Z

Thanks @encukou for the PR 🌮🎉.. I'm working now to backport this PR to: 3.10.
🐍🍒⛏🤖

miss-islington · 2023-08-21T11:48:15Z

Thanks @encukou for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

miss-islington · 2023-08-21T11:48:17Z

Sorry, @encukou, I could not cleanly backport this to 3.8 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker acbd3f9c5c5f23e95267714e41236140d84fe962 3.8

miss-islington · 2023-08-21T11:48:24Z

Sorry, @encukou, I could not cleanly backport this to 3.9 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker acbd3f9c5c5f23e95267714e41236140d84fe962 3.9

…GH-107846) (cherry picked from commit acbd3f9) Co-authored-by: Petr Viktorin <encukou@gmail.com> Co-authored-by: Victor Stinner <vstinner@python.org> Co-authored-by: Lumír 'Frenzy' Balhar <frenzy.madness@gmail.com>

bedevere-bot · 2023-08-21T11:48:27Z

GH-108209 is a backport of this pull request to the 3.11 branch.

bedevere-bot · 2023-08-21T11:48:33Z

GH-108210 is a backport of this pull request to the 3.10 branch.

…GH-107846) (cherry picked from commit acbd3f9) Co-authored-by: Petr Viktorin <encukou@gmail.com> Co-authored-by: Victor Stinner <vstinner@python.org> Co-authored-by: Lumír 'Frenzy' Balhar <frenzy.madness@gmail.com>

bedevere-bot · 2023-08-21T11:48:38Z

GH-108211 is a backport of this pull request to the 3.12 branch.

…7846) (#108211) gh-107845: Fix symlink handling for tarfile.data_filter (GH-107846) (cherry picked from commit acbd3f9) Co-authored-by: Petr Viktorin <encukou@gmail.com> Co-authored-by: Victor Stinner <vstinner@python.org> Co-authored-by: Lumír 'Frenzy' Balhar <frenzy.madness@gmail.com>

…7846) (GH-108209) gh-107845: Fix symlink handling for tarfile.data_filter (GH-107846) (cherry picked from commit acbd3f9) Co-authored-by: Petr Viktorin <encukou@gmail.com> Co-authored-by: Victor Stinner <vstinner@python.org> Co-authored-by: Lumír 'Frenzy' Balhar <frenzy.madness@gmail.com>

…ythonGH-107846) (pythonGH-108209) pythongh-107845: Fix symlink handling for tarfile.data_filter (pythonGH-107846) (cherry picked from commit acbd3f9) Co-authored-by: Petr Viktorin <encukou@gmail.com> Co-authored-by: Victor Stinner <vstinner@python.org> Co-authored-by: Lumír 'Frenzy' Balhar <frenzy.madness@gmail.com>

bedevere-bot · 2023-08-22T09:34:11Z

GH-108274 is a backport of this pull request to the 3.9 branch.

…ythonGH-107846) (pythonGH-108209) pythongh-107845: Fix symlink handling for tarfile.data_filter (pythonGH-107846) (cherry picked from commit acbd3f9) Co-authored-by: Petr Viktorin <encukou@gmail.com> Co-authored-by: Victor Stinner <vstinner@python.org> Co-authored-by: Lumír 'Frenzy' Balhar <frenzy.madness@gmail.com>

bedevere-bot · 2023-08-22T11:33:03Z

GH-108279 is a backport of this pull request to the 3.8 branch.

…ythonGH-107846) (pythonGH-108209) pythongh-107845: Fix symlink handling for tarfile.data_filter (pythonGH-107846) (cherry picked from commit acbd3f9) Co-authored-by: Petr Viktorin <encukou@gmail.com> Co-authored-by: Victor Stinner <vstinner@python.org> Co-authored-by: Lumír 'Frenzy' Balhar <frenzy.madness@gmail.com>

…7846) (#108210)

) (#108274) (cherry picked from commit acbd3f9) Co-authored-by: Petr Viktorin <encukou@gmail.com> Co-authored-by: Victor Stinner <vstinner@python.org> Co-authored-by: Lumír 'Frenzy' Balhar <frenzy.madness@gmail.com>

) (#108279) (cherry picked from commit acbd3f9) Co-authored-by: Petr Viktorin <encukou@gmail.com> Co-authored-by: Victor Stinner <vstinner@python.org> Co-authored-by: Lumír 'Frenzy' Balhar <frenzy.madness@gmail.com>

…ythonGH-107846) (python#108274) (cherry picked from commit acbd3f9) Co-authored-by: Petr Viktorin <encukou@gmail.com> Co-authored-by: Victor Stinner <vstinner@python.org> Co-authored-by: Lumír 'Frenzy' Balhar <frenzy.madness@gmail.com>

Fix symlink handling for tarfile.data_filter

f2c619a

encukou requested a review from ethanfurman as a code owner August 10, 2023 15:39

bedevere-bot mentioned this pull request Aug 10, 2023

tarfile.data_filter wrongly rejects some tarballs with symlinks #107845

Closed

bedevere-bot added the awaiting core review label Aug 10, 2023

ethanfurman approved these changes Aug 10, 2023

View reviewed changes

bedevere-bot added awaiting merge and removed awaiting core review labels Aug 10, 2023

encukou added 2 commits August 15, 2023 12:03

Improve tests & an error message

e660734

Account for differences in symlink resolution

aef2658

encukou marked this pull request as draft August 15, 2023 11:59

bedevere-bot removed the awaiting merge label Aug 15, 2023

encukou marked this pull request as ready for review August 16, 2023 09:54

bedevere-bot added the awaiting core review label Aug 16, 2023

frenzymadness reviewed Aug 17, 2023

View reviewed changes

Lib/test/test_tarfile.py Outdated Show resolved Hide resolved

vstinner approved these changes Aug 17, 2023

View reviewed changes

Doc/library/tarfile.rst Outdated Show resolved Hide resolved

Doc/library/tarfile.rst Outdated Show resolved Hide resolved

bedevere-bot added awaiting merge and removed awaiting core review labels Aug 17, 2023

encukou and others added 2 commits August 17, 2023 15:07

Apply suggestions from code review

fa69b69

Co-authored-by: Victor Stinner <vstinner@python.org> Co-authored-by: Lumír 'Frenzy' Balhar <frenzy.madness@gmail.com>

Add a test for (sym)link chains

fbc0642

vstinner reviewed Aug 17, 2023

View reviewed changes

Lib/test/test_tarfile.py Outdated Show resolved Hide resolved

encukou and others added 2 commits August 17, 2023 15:21

Apply suggestions from code review

f85aea9

Co-authored-by: Victor Stinner <vstinner@python.org>

Merge branch 'main' into tarfile-link-fix

cebb318

encukou added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Aug 17, 2023

bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Aug 17, 2023

encukou added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Aug 21, 2023

bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Aug 21, 2023

miss-islington assigned encukou Aug 21, 2023

bedevere-bot removed the needs backport to 3.11 only security fixes label Aug 21, 2023

bedevere-bot removed the needs backport to 3.10 only security fixes label Aug 21, 2023

bedevere-bot removed the needs backport to 3.12 bug and security fixes label Aug 21, 2023

bedevere-bot removed the needs backport to 3.9 only security fixes label Aug 22, 2023

bedevere-bot removed the needs backport to 3.8 label Aug 22, 2023

pablogsal pushed a commit that referenced this pull request Aug 22, 2023

[3.10] gh-107845: Fix symlink handling for tarfile.data_filter (GH-10…

7d44551

…7846) (#108210)

LecrisUT mentioned this pull request Sep 4, 2023

Fix: build wheel workflow spglib/spglib#321

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gh-107845: Fix symlink handling for tarfile.data_filter #107846

gh-107845: Fix symlink handling for tarfile.data_filter #107846

encukou commented Aug 10, 2023 •

edited by github-actions bot

Loading

frenzymadness left a comment

vstinner left a comment

encukou commented Aug 17, 2023

bedevere-bot commented Aug 17, 2023

bedevere-bot commented Aug 21, 2023

encukou commented Aug 21, 2023

miss-islington commented Aug 21, 2023

miss-islington commented Aug 21, 2023

miss-islington commented Aug 21, 2023

miss-islington commented Aug 21, 2023

miss-islington commented Aug 21, 2023

bedevere-bot commented Aug 21, 2023

bedevere-bot commented Aug 21, 2023

bedevere-bot commented Aug 21, 2023

bedevere-bot commented Aug 22, 2023

bedevere-bot commented Aug 22, 2023

gh-107845: Fix symlink handling for tarfile.data_filter #107846

gh-107845: Fix symlink handling for tarfile.data_filter #107846

Conversation

encukou commented Aug 10, 2023 • edited by github-actions bot Loading

frenzymadness left a comment

Choose a reason for hiding this comment

vstinner left a comment

Choose a reason for hiding this comment

encukou commented Aug 17, 2023

bedevere-bot commented Aug 17, 2023

bedevere-bot commented Aug 21, 2023

encukou commented Aug 21, 2023

miss-islington commented Aug 21, 2023

miss-islington commented Aug 21, 2023

miss-islington commented Aug 21, 2023

miss-islington commented Aug 21, 2023

miss-islington commented Aug 21, 2023

bedevere-bot commented Aug 21, 2023

bedevere-bot commented Aug 21, 2023

bedevere-bot commented Aug 21, 2023

bedevere-bot commented Aug 22, 2023

bedevere-bot commented Aug 22, 2023

encukou commented Aug 10, 2023 •

edited by github-actions bot

Loading