Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[3.8] [3.7] test_mkfifo_dir_fd triggers segfaults on macOS CI #102306

Closed
pradyunsg opened this issue Feb 27, 2023 · 5 comments
Closed

[3.8] [3.7] test_mkfifo_dir_fd triggers segfaults on macOS CI #102306

pradyunsg opened this issue Feb 27, 2023 · 5 comments
Assignees
@ambv
Labels
3.8 (EOL) end of life OS-mac type-crash A hard crash of the interpreter, possibly with a core dump

Comments

@pradyunsg
Copy link
Member

pradyunsg commented Feb 27, 2023 •
edited by bedevere-bot
Loading

A PR upgrading pip's wheel on 3.8 (which shouldn't really affect the test_mkfifo_dir_fd) triggered a segfault on the macOS test run.

https://github.com/python/cpython/actions/runs/4269978257/jobs/7433511913#step:6:651

Looking through the issue tracker, I don't see that failure mode reported as is and I'm noting it here mostly since I'm not sure if it's in any way related to this ongoing thing or if it should be a separate issue. It doesn't seem like it is, but I've not followed the failure discussions here closely to know for sure. 😅

Originally posted by @pradyunsg in #101981 (comment)

We've also seen this happen on certain other runs as well since, including a re-run on the aforementioned PR.

Linked PRs
@pradyunsg pradyunsg mentioned this issue Feb 27, 2023
Closed
@ned-deily
Copy link
Member

ned-deily commented Feb 27, 2023 •
edited
Loading

This is the problem identified in #97897. Undoubtedly, the Github image is using macOS 12.6.x but the default SDK in the installed Command Line Tools is linked to a macOS 13 SDK. We did not backport the change to 3.8 because we don't support full macOS "weaklinking" on Python 3.8 but it can be a problem in edge-cases like this (Apple forces a problem by introducing the use of a new release's SDK on the previous system). We should be able to avoid the problem by backporting some code, by explicitly building with a MacOSX12.sdk, or by changing the CI to build with macOS 13 Ventura. (Note this is only a problem when building on macOS 12 with a macOS 13 SDK as explained in the linked issue.) It's @ambv's call.

@bedevere-bot bedevere-bot mentioned this issue Feb 28, 2023
Merged
@ned-deily
Copy link
Member

As a proof-of-concept, #102307 shows that just forcing the SDK avoids the issue.

@ned-deily ned-deily mentioned this issue Feb 28, 2023
Closed
@pradyunsg pradyunsg added OS-mac 3.8 (EOL) end of life type-crash A hard crash of the interpreter, possibly with a core dump labels Feb 28, 2023
ned-deily added a commit that referenced this issue Mar 4, 2023
@ned-deily
[3.8] GH-102306 Avoid GHA CI macOS test_posix failure by using the ap…
4812813 
…propriate macOS SDK (GH-102307)

[3.8] Avoid GHA CI macOS test_posix failure by using the appropriate macOS SDK.
ned-deily added a commit to ned-deily/cpython that referenced this issue Mar 4, 2023
@ned-deily
pythonGH-102306 Avoid GHA CI macOS test_posix failure by using the ap…
887a681 
…propriate macOS SDK.
@bedevere-bot bedevere-bot mentioned this issue Mar 4, 2023
Merged
@ned-deily ned-deily changed the title [3.8] test_mkfifo_dir_fd triggers segfaults on macOS CI [3.8] [3.7] test_mkfifo_dir_fd triggers segfaults on macOS CI Mar 4, 2023
@ned-deily
Copy link
Member

This is also an issue for 3.7.x; #102427 is a backport.

arhadthedev added a commit to ned-deily/cpython that referenced this issue Apr 9, 2023
@arhadthedev
Merge branch '3.7' into pythongh-102306-37
37da82c
ned-deily added a commit that referenced this issue Apr 10, 2023
@ned-deily @arhadthedev
[3.7] GH-102306 Avoid GHA CI macOS test_posix failure by using the ap…
7522ff7 
…propriate macOS SDK. (GH-102427)

Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
@pradyunsg
Copy link
Member Author

With the PRs merged, is there anything more to do here?

@erlend-aasland
Copy link
Contributor

I believe we can close this.

carlosroman added a commit to DataDog/cpython that referenced this issue Jun 22, 2023
@carlosroman @ambv
@benjaminp @ned-deily @miss-islington @thunder-007 @gpshead @ucodery @hugovk @zooba @OTheDev @merwok @arhadthedev @corona10 @Blind4Basics @CAM-Gerlach @pradyunsg @encukou @samcarroll42 @ethanfurman @JelleZijlstra @gaogaotiantian @iritkatriel @stratakis @illia-v
Upgrade to 3.8.17 (#26)
e0a363f 
* Post 3.8.16

* [3.8] Update copyright years to 2023. (pythongh-100852)

* [3.8] Update copyright years to 2023. (pythongh-100848).
(cherry picked from commit 11f9932)

Co-authored-by: Benjamin Peterson <benjamin@python.org>

* Update additional copyright years to 2023.

Co-authored-by: Ned Deily <nad@python.org>

* [3.8] Update copyright year in README (pythonGH-100863) (pythonGH-100867)

(cherry picked from commit 30a6cc4)

Co-authored-by: Ned Deily <nad@python.org>
Co-authored-by: HARSHA VARDHAN <75431678+Thunder-007@users.noreply.github.com>

* [3.8] Correct CVE-2020-10735 documentation (pythonGH-100306) (python#100698)

(cherry picked from commit 1cf3d78)
(cherry picked from commit 88fe8d7)

Co-authored-by: Jeremy Paige <ucodery@gmail.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>

* [3.8] Bump Azure Pipelines to ubuntu-22.04 (pythonGH-101089) (python#101215)

(cherry picked from commit c22a55c)

Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>

* [3.8] pythongh-100180: Update Windows installer to OpenSSL 1.1.1s (pythonGH-100903) (python#101258)

* pythongh-101422: (docs) TarFile default errorlevel argument is 1, not 0 (pythonGH-101424)

(cherry picked from commit ea23271)

Co-authored-by: Owain Davies <116417456+OTheDev@users.noreply.github.com>

* [3.8] pythongh-95778: add doc missing in some places (pythonGH-100627) (python#101630)

(cherry picked from commit 4652182)

* [3.8] pythongh-101283: Improved fallback logic for subprocess with shell=True on Windows (pythonGH-101286) (python#101710)

Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
Co-authored-by: Steve Dower <steve.dower@microsoft.com>

* [3.8] pythongh-101981: Fix Ubuntu SSL tests with OpenSSL (3.1.0-beta1) CI i… (python#102095)

[3.8] pythongh-101981: Fix Ubuntu SSL tests with OpenSSL (3.1.0-beta1) CI issue (pythongh-102079)

* [3.8] pythonGH-102306 Avoid GHA CI macOS test_posix failure by using the appropriate macOS SDK (pythonGH-102307)

[3.8] Avoid GHA CI macOS test_posix failure by using the appropriate macOS SDK.

* [3.8] pythongh-101726: Update the OpenSSL version to 1.1.1t (pythonGH-101727) (pythonGH-101752)

Fixes CVE-2023-0286 (High) and a couple of Medium security issues.
https://www.openssl.org/news/secadv/20230207.txt

Co-authored-by: Gregory P. Smith <greg@krypto.org>
Co-authored-by: Ned Deily <nad@python.org>

* [3.8] pythongh-102627: Replace address pointing toward malicious web page (pythonGH-102630) (pythonGH-102667)

(cherry picked from commit 61479d4)

Co-authored-by: Blind4Basics <32236948+Blind4Basics@users.noreply.github.com>
Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>

* [3.8] pythongh-101997: Update bundled pip version to 23.0.1 (pythonGH-101998). (python#102244)

(cherry picked from commit 89d9ff0)

* [3.8] pythongh-102950: Implement PEP 706 – Filter for tarfile.extractall (pythonGH-102953) (python#104548)

Backport of c8c3956

* [3.8] pythongh-99889: Fix directory traversal security flaw in uu.decode() (pythonGH-104096) (python#104332)

(cherry picked from commit 0aeda29)

Co-authored-by: Sam Carroll <70000253+samcarroll42@users.noreply.github.com>

* [3.8] pythongh-104049: do not expose on-disk location from SimpleHTTPRequestHandler (pythonGH-104067) (python#104121)

Do not expose the local server's on-disk location from `SimpleHTTPRequestHandler` when generating a directory index. (unnecessary information disclosure)

(cherry picked from commit c7c3a60)

Co-authored-by: Ethan Furman <ethan@stoneleaf.us>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>

* [3.8] pythongh-103935: Use `io.open_code()` when executing code in trace and profile modules (pythonGH-103947) (python#103954)

Co-authored-by: Tian Gao <gaogaotiantian@hotmail.com>

* [3.8] pythongh-68966: fix versionchanged in docs (pythonGH-105299)

* [3.8] Update GitHub CI workflow for macOS. (pythonGH-105302)

* [3.8] pythongh-105184: document that marshal functions can fail and need to be checked with PyErr_Occurred (pythonGH-105185) (python#105222)

(cherry picked from commit ee26ca1)

Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>

* [3.8] pythongh-102153: Start stripping C0 control and space chars in `urlsplit` (pythonGH-102508) (pythonGH-104575) (pythonGH-104592) (python#104593) (python#104895)

`urllib.parse.urlsplit` has already been respecting the WHATWG spec a bit pythonGH-25595.

This adds more sanitizing to respect the "Remove any leading C0 control or space from input" [rule](https://url.spec.whatwg.org/GH-url-parsing:~:text=Remove%20any%20leading%20and%20trailing%20C0%20control%20or%20space%20from%20input.) in response to [CVE-2023-24329](https://nvd.nist.gov/vuln/detail/CVE-2023-24329).

I simplified the docs by eliding the state of the world explanatory
paragraph in this security release only backport.  (people will see
that in the mainline /3/ docs)

(cherry picked from commit d7f8a5f)
(cherry picked from commit 2f630e1)
(cherry picked from commit 610cc0a)
(cherry picked from commit f48a96a)

Co-authored-by: Miss Islington (bot) <31488909+miss-islington@users.noreply.github.com>
Co-authored-by: Illia Volochii <illia.volochii@gmail.com>
Co-authored-by: Gregory P. Smith [Google] <greg@krypto.org>

* [3.8] pythongh-103142: Upgrade binary builds and CI to OpenSSL 1.1.1u (pythonGH-105174) (pythonGH-105200) (pythonGH-105205) (python#105370)

Upgrade builds to OpenSSL 1.1.1u.

Also updates _ssl_data_111.h from OpenSSL 1.1.1u, _ssl_data_300.h from 3.0.9.

Manual edits to the _ssl_data_300.h file prevent it from removing any
existing definitions in case those exist in some peoples builds and were
important (avoiding regressions during backporting).

(cherry picked from commit ede89af)
(cherry picked from commit e15de14)

Co-authored-by: Gregory P. Smith <greg@krypto.org>
Co-authored-by: Ned Deily <nad@python.org>

* Python 3.8.17

* Post 3.8.17

* Updated CI to build 3.8.17

---------

Co-authored-by: Łukasz Langa <lukasz@langa.pl>
Co-authored-by: Benjamin Peterson <benjamin@python.org>
Co-authored-by: Ned Deily <nad@python.org>
Co-authored-by: Miss Islington (bot) <31488909+miss-islington@users.noreply.github.com>
Co-authored-by: HARSHA VARDHAN <75431678+Thunder-007@users.noreply.github.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Co-authored-by: Jeremy Paige <ucodery@gmail.com>
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Co-authored-by: Steve Dower <steve.dower@python.org>
Co-authored-by: Owain Davies <116417456+OTheDev@users.noreply.github.com>
Co-authored-by: Éric <earaujo@caravan.coop>
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
Co-authored-by: Steve Dower <steve.dower@microsoft.com>
Co-authored-by: Dong-hee Na <donghee.na@python.org>
Co-authored-by: Blind4Basics <32236948+Blind4Basics@users.noreply.github.com>
Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>
Co-authored-by: Pradyun Gedam <pradyunsg@gmail.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Sam Carroll <70000253+samcarroll42@users.noreply.github.com>
Co-authored-by: Ethan Furman <ethan@stoneleaf.us>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Co-authored-by: Tian Gao <gaogaotiantian@hotmail.com>
Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
Co-authored-by: stratakis <cstratak@redhat.com>
Co-authored-by: Illia Volochii <illia.volochii@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ambv ambv

Labels
3.8 (EOL) end of life OS-mac type-crash A hard crash of the interpreter, possibly with a core dump
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@ambv @pradyunsg @ned-deily @erlend-aasland