Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-41100: Support macOS 11 and Apple Silicon #22855

Merged

Conversation

ronaldoussoren
Copy link
Contributor

@ronaldoussoren ronaldoussoren commented Oct 21, 2020

This merges my other two PRs, and improves on the PR that implements support for older macOS versions.

  • Tests pass on macOS 11 (ARM64)
  • Tests pass on macOS 10.9 using 10.10 SDK
  • Add tests for the time module, similar to the tests added for the posix module
  • Build arm64/x86_64 binary ("universal2") and test on:
    • macOS 11 (arm64)
    • macOS 11 (x86_64)
    • macOS 10.15 (x86_64)
    • macOS 10.14 (x86_64)
    • macOS 10.12 (x86_64)
    • macOS 10.9 (x86_64)
  • _osx_support.py changes to strip "-arch arm64" (and possibly related arguments)
    on systems that don't support his.
  • Make sure CI passes on all platforms
  • Add tests for the changes in _osx_support and distutils
  • Write NEWS blurb

https://bugs.python.org/issue41100

We should consider just dropping support for macOS 10.4 here
to simplify the code.
This is support for ctypes on macOS/arm64 based
on PR 21249 by Lawrence D'Anna (Apple).

Changes:
- changed __builtin_available tests from 11.0 to 10.15
- added test to setup.py for ffi_closure_alloc and use
  that in malloc_closure.c
- Minor change in the code path for ffi_prep_closure_var
  (coding style change)
The preprocessor guard in the old version doesn't work, and
isn't really needed (10.4 only supported 32-bit code where
unsigned long is the same as uint32_t).
This also adds an option to stop building after compiling the
3th-party dependencies, as well as a script for archiving those
dependencies. This makes it easier to work on the build.

There are three changes to build-installer.py related to universal2
support:

1. Add 'universal2' information to build-installer.py;
2. Building OpenSSL for arm64 requires a patch at this time;
3. For some reason I had to patch the Tcl build to avoid a build error.
Needed because my previous workaround doesn't work anymore.

This uses a private API, that should be made public later...
- __builtin_available is not present in the compiler, fall
  back to the ancient way of testing if a symbol is usable.
- Logic error code in pytime.c resulted in a compile error.
…oren/cpython into macos11-deploy-earlier-branch
The tests are not entirely done yet, and something
similar needs to be done for the time module.
…on isn't present.

I don't particularly like the code duplication, but this code
should be pretty stable.
@ronaldoussoren
Copy link
Contributor Author

I've started work on the backport to 3.8. That's more work that the 3.9 backport due to unrelated changes that interfere with merging. I expect to have a PR with a manual complete backport later today, and might create a manual backport of a subset of the functionality as well.

mozillazg pushed a commit to mozillazg/pypy that referenced this pull request Jan 14, 2021
On Big Sur, there is no longer a separate dylib file for each system
library. Instead, they have been combined into a single shared cache
file. As a result, it no longer suffices to check file existence to
determine whether a system library is present. Fix this by calling
_dyld_shared_cache_contains_path to check the dyld shared cache as well.

The changes under lib-python/ are copied from
python/cpython#22855.

The _dyld_shared_cache_contains_path function is exposed to the ctypes
python code using CFFI, similar to what is done in other modules, such
as lib_pypy/_sha3.

Fixes issue #3314.
adorilson pushed a commit to adorilson/cpython that referenced this pull request Mar 13, 2021
Co-authored-by:  Lawrence D’Anna <lawrence_danna@apple.com>

* Add support for macOS 11 and Apple Silicon (aka arm64)
   
  As a side effect of this work use the system copy of libffi on macOS, and remove the vendored copy

* Support building on recent versions of macOS while deploying to older versions

  This allows building installers on macOS 11 while still supporting macOS 10.9.
isuruf added a commit to isuruf/cpython that referenced this pull request May 31, 2021
pythonGH-25806 was a python 3.8 backport of pythonGH-22855 to support macOS 11,
but didn't backport the changes that enabled building python
with a new macOS SDK and target an older macOS version.
This commit adds support for that.
colesbury referenced this pull request in colesbury/nogil Oct 7, 2021
…295)

* [3.9] bpo-41100: Support macOS 11 and Apple Silicon (GH-22855)

Co-authored-by:  Lawrence D’Anna <lawrence_danna@apple.com>

* Add support for macOS 11 and Apple Silicon (aka arm64)

  As a side effect of this work use the system copy of libffi on macOS, and remove the vendored copy

* Support building on recent versions of macOS while deploying to older versions

  This allows building installers on macOS 11 while still supporting macOS 10.9..
(cherry picked from commit 4176193)

Co-authored-by: Ronald Oussoren <ronaldoussoren@mac.com>

* Back port of changes to _decimal to support arm64

* temp_dir is in test.support in 3.9
@@ -331,6 +355,12 @@ def compiler_fixup(compiler_so, cc_args):
except ValueError:
break

elif not _supports_arm64_builds():
# Look for "-arch arm64" and drop that
for idx in range(len(compiler_so)):
Copy link

@ddl-jameslee ddl-jameslee Dec 1, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When idx is len(compiler_so)-1, it will throw an IndexError exception.
So, it should be for idx in range(len(compiler_so)-1):

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dang, looks like you're right

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR was merged over a year ago and has long since been released to the field. Please open a new issue on the bug tracker and, if possible, provide a PR. Otherwise, it will likely be forgotten about it. Thanks!

zware added a commit to zware/cpython that referenced this pull request Dec 26, 2022
It was an ancient, modified copy of libffi that has not been in use
since pythonGH-22855.
zware added a commit to zware/cpython that referenced this pull request Dec 26, 2022
It has had no effect on non-macOS platforms for a long time, and has had
the non-obvious effect of invoking `pkg_config` and not setting
`-DUSING_APPLE_OS_LIBFFI` on macOS since pythonGH-22855.
zware added a commit that referenced this pull request Dec 28, 2022
…00544)

It has had no effect on non-macOS platforms for a long time, and has had
the non-obvious effect of invoking `pkg_config` and not setting
`-DUSING_APPLE_OS_LIBFFI` on macOS since GH-22855.
zware added a commit that referenced this pull request Dec 28, 2022
It was an ancient, modified copy of libffi that has not been in use
since GH-22855.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.