Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[C API] Meta issue: add new public functions with doc+tests to replace removed private functions #111481

Closed
vstinner opened this issue Oct 30, 2023 · 15 comments

Comments

@vstinner
Copy link
Member

Many private functions were removed in issue #106320. My plan is to add public functions for removed private functions which are used by 3rd party extensions: see C API: My plan to clarify private vs public functions in Python 3.13.

I don't think that the fact that a private function was used must always justify to directly expose the exact same API to the public C API. Usually, private functions expose implementation details, use weaker or not check such as checking argument types, are not documented, and are not tested.

To make a function public, we should:

I also would like to add new public functions to https://pythoncapi-compat.readthedocs.io/ to provide them to Python 3.12 and older (by calling the old private functions).

This work will take time. Depending on how it goes, we should consider reverting some removals if there is too much pressure, to unblock some critical dependencies.

The target is to make most C extensions compatible with Python 3.13 when Python 3.13 beta1 is released. Obviously, sooner the better.


C extensions which want to access implementation details can use the internal C API. By default, I modified the internal functions to no longer export internal functions, only make them usable by Python internals: they cannot be used by shared libraries such as stdlib C extensions. This part is unclear to me. Should we expose more internal functions. Which ones? How do you decide if a function should be exported or not?

@vstinner
Copy link
Member Author

vstinner commented Oct 30, 2023

Issues asking to add public APIs for removed private APIs.

DONE:

Private API restored:

Rejected:

Note: I update this comment time to time.

@vstinner
Copy link
Member Author

See also discussion: Is Python 3.13 going to be Python 4?.

@encukou
Copy link
Member

encukou commented Nov 2, 2023

Guido said in #106320 (comment):

I strongly prefer revert over fix, for removed APIs that cause problems.

Can we get consensus on that -- that is, revert the removal for these? Or should we get a SC decision?

@vstinner
Copy link
Member Author

vstinner commented Nov 2, 2023

Can we get consensus on that -- that is, revert the removal for these? Or should we get a SC decision?

My plan mentioned at the beginning of this issue is to add public functions asked by users. And if we run out of time before beta1, start to revert and consider adding more public functions later.

Alpha 1 was just released a few days ago. Can we start by adding public functions for now, and only move to the "revert" backup plan later if needed?

Beta 1 is scheduled in May 2024: in 6 months. Usually it takes several months to get more C extensions compatible with the new Python versions. I'm surprised that for Python 3.13, people want suddenly to get "everything" compatible at day 1 of the alpha1 release. Isn't it not a little bit early to expect that?

@gpshead
Copy link
Member

gpshead commented Nov 6, 2023

I'd much prefer 'revert' for any API anyone is found using in 3.13.

We need to treat 3.13 as a more special than usual release and aim to minimize compatibility headaches for existing project code. That way more things that build and run on 3.12 build can run on 3.13 as is or with minimal work.

This will enable ecosystem code owners to focus on the bigger picture task of enabling existing code to be built and tested on an experimental pep703 free-threading build rather than having a pile of unrelated cleanup trivia blocking that.

@encukou
Copy link
Member

encukou commented Nov 6, 2023

I talked to Victor privately, and confirmed that we fundamentally disagree with each other.
@vstinner, correct me if I'm wrong:

Victor believes that the plan (https://discuss.python.org/t/30131) has overwhelming support and no significant opposition, and it doesn't need a PEP.
I believe that is not the case, and it should have been a PEP, to centralize the discussion. (Personally, I'd appreciate a “motivation” section -- a description of the problem being solved -- to be able to suggest alternative approaches. So far I have been unable to find one.)
I have been assured that my opinion was taken into account, but it was decided to go ahead with the plan as it is. By now, I don't see how a personal opinion could change the decision.

@vstinner
Copy link
Member Author

vstinner commented Nov 7, 2023

I'd much prefer 'revert' for any API anyone is found using in 3.13.

This issue is about adding public functions for removed private functions. Would you mind to open a new issue to discuss revert? Otherwise it's unclear what is the scope of this issue. Or are you saying that we must not add public functions, but stick to private functions forever?

@vstinner
Copy link
Member Author

vstinner commented Nov 8, 2023

@vstinner
Copy link
Member Author

See also issue gh-112026: [C API] Revert of private functions removed in Python 3.13 causing most problems.

@vstinner
Copy link
Member Author

Indirectly related: Python 3.13 removed deprecated functions to configure the Python initialization, such as Py_SetPath(). The replacement API PyConfig is excluded from the limited C API. I'm working on a replacement for the stable ABI: Add PyInitConfig C API.

@vstinner
Copy link
Member Author

List of removed C functions reverted in Python 3.13 alpha2: #112026 (comment)

@vstinner vstinner changed the title [C API] Meta issue to replace removed functions with new clean public functions [C API] Meta issue: add new public functions with doc+tests to replace removed private functions Dec 15, 2023
emmatyping added a commit to emmatyping/cpython that referenced this issue May 22, 2024
This is used in pywin32, and the function being moved to internal headers
broke with 3.13. Restore it for now to eventually be considered for being
made a public symbol.

See also pythongh-112026 (previous restoration) and pythongh-111481 (public replacements)
@vstinner
Copy link
Member Author

vstinner commented Jul 18, 2024

mypyc issue: #121489

@vstinner
Copy link
Member Author

vstinner commented Jul 18, 2024

uwsgi issue: #121758

@vstinner
Copy link
Member Author

Pyodide issue: capi-workgroup/problems#79

@vstinner
Copy link
Member Author

All known issues have been fixed, I close this issue. The work can continue in new issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants