-
Notifications
You must be signed in to change notification settings - Fork 733
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
auto-initialize: new feature to control initializing Python #1347
Conversation
dfa52b2
to
600db5f
Compare
Thanks!
Again, this is a difficult problem, and thank you for tackling it. |
Thanks for the questions. I'm very happy to change the design if we think alternatives are better. Here's my thinking about your proposed alternatives:
|
... I'm thinking that maybe we can use |
e336bbe
to
1543525
Compare
Reworked this PR as discussed but I expect CI to continue to fail until #1350 is merged, because one of the simplifications in that PR fixes how Py_SHARED would be set. |
9d4a410
to
5eb33f7
Compare
Rebased and rebuilt this PR as a single commit; hopefully CI is green this time 🤞 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks!
CI failures are because auto-initialise
is unintentionally invoked. I'm sorry but I have no idea why this happens.
src/gil.rs
Outdated
cfg_if::cfg_if! { | ||
if #[cfg(all(feature = "auto-initialize", Py_SHARED, not(PyPy)))] { | ||
prepare_freethreaded_python(); | ||
} else if #[cfg(all(feature = "auto-initialize", not(feature = "extension-module"), not(Py_SHARED), not(rustdoc)))] { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we can make this simpler by else if all(auto-initialize, PyPy) { ... } else if all(auto_initialize, ...)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
not(extension-module)
is unfortunately necessary to avoid a breaking change - otherwise all extension modules would suddenly start failing to build PyPy
wheels unless users changed their Cargo.toml
to remove default features. We could make it as you suggest in 0.14
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, thanks. That makes sense.
BTW, do you think that we should keep auto-initialize in default-features?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think in the long-run it's better to remove auto-initialize
from default-features, but that's also a breaking change, so let's remove it in 0.14
.
I'll create an issue to track these cleanups and add it to the 0.14
milestone.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍🏼
|
||
- name: Build without default features | ||
- name: Build (no features) | ||
run: cargo build --no-default-features --verbose --target ${{ matrix.platform.rust-target }} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like that auto-initialize
and macros
are enabled in this build, though I don't know why 😕
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's because of the Cargo.toml
changes - see other comment.
|
||
# Run tests (except on PyPy, because no embedding API). | ||
- if: matrix.python-version != 'pypy-3.6' | ||
name: Test | ||
run: cargo test --features "num-bigint num-complex hashbrown" --target ${{ matrix.platform.rust-target }} | ||
run: cargo test --no-default-features --features "macros num-bigint num-complex hashbrown" --target ${{ matrix.platform.rust-target }} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not using auto-initialize
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's automatically included in tests because I put this in [dev-dependencies]
:
# features needed to run the PyO3 test suite
pyo3 = { path = ".", default-features = false, features = ["macros", "auto-initialize"] }
However, that's also the reason why auto-initialize
and macros
are included in the cargo build
command above. Interestingly cargo
is about to stabilize a fix for this as a new resolver = "2"
option in Cargo.toml. rust-lang/cargo#8997 (I tested on nightly and the features are no longer included in cargo build
.)
Because in the future what I've done here will work correctly in the future, I'm going to keep the code mostly as-is for the moment and introduce a hack to avoid the immediate issue with a TODO to tidy it up once we bump MSRV in the future (probably to Rust 1.52).
02e27c1
to
c2cc186
Compare
@kngwyu I pushed a CHANGELOG entry and a new guide page I also added a tiny "hack" with If you are happy with this, I think this is ready to merge! 🚀 |
Is it much simpler to remove the dev-dependency for now? |
If we remove the I'm sure I'll forget to add the feature a lot and it's also annoying to type something so long. I think it's better to leave the very small hack in so that we can make exactly the API we want for users. Especially as we know the fix which removes the need for the hack is going to be stabilised very soon. (Because we only need the hack to fix the pypy CI jobs, we only need to wait until it's in the most recent stable rust version.) |
OK, if you are to release 0.14 before the next stable release, let's go in this way 👍🏼 |
(I force-pushed to fix the commit message. Will merge shortly.) |
This PR add a new feature
auto-initialize
to control whetherPython::with_gil
andPython::acquire_gil
callprepare_freethreaded_python
automatically.I wanted to add this for a couple of reasons:
extension-module
then this functionality is not needed.The PR checks for the static python and PyPy cases and removes the
prepare_freethreaded_python
API, and instead emits acompile_error!
if theauto-initialize
feature is enabled.Closes #742
Closes #762
Closes #1213
TODO:
It would be nice to make this feature opt-in instead of opt-out, but that's a breaking change so we should delay that until 0.14 release.
I'd like to write some more documentation for this in the guide before merging.
Also needs CHANGELOG entry.