-
Notifications
You must be signed in to change notification settings - Fork 30
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Unable to build for Python 3.11 on macOS #62
Comments
Thanks for reporting this! I’m also on Homebrew so you have got my attention 🙂 I’ll have some time in the next few days to try a 3.11 version of the readline module. |
@ludwigschwardt kindly following up - will it be possible to use this in Python 3.11 and later? |
Whoops Milos, I lost track of this - sincere apologies! It's not a Python 3.11 issue per se... I already have 3.11 wheels on PyPI since October 2022 (#59). (Just checking: have you tried I'm currently on Homebrew but only for non-Python libraries. I use I think the main compilation issue is that you don't have any readline headers around anymore. Try adding
to use the local readline headers inside the package. In the meantime I'll fish out my old laptop that still has brew python and see if there is a better solution. |
@ludwigschwardt thanks a lot for responding. My use case is that I just want to have it in the shell so I can use CTRL+R and other readline features on macOS (as would be normal on Linux installs) since brew Python 3.11 removed readline support going forward, meaning Python 3.10 was the last version to be compiled against readline. I tried
I believe that's what I tried to do last time, which failed with the error in my original comment. I just tried running
Appreciate any further help. |
Yes, it plays "nicely" with your Python installation and requires you to call it explicitly (unlike We used to have a direct
to do exactly what you require. This is what I meant by "easy_install" installation. It's different from The problem is that On your build issue: the include directory flag should have been The bigger challenge is to get the gnureadline package to replace the original readline module. Dirty tricks include easy_install, the easy-install.pth file, eggs (as opposed to wheels) and |
For myself, I use |
I have a new scheme to try out... 🙂
import sys
try:
import gnureadline as readline
except ImportError:
import readline
sys.modules['readline'] = readline
del readline
del sys
export PYTHONSTARTUP=~/.pythonrc These commands will run at the start of every interactive Python session, which is when you need tab completion etc. It also works in virtualenvs, if you remember to install Let me know if the resulting keystrokes are what you expect. |
Mmm, I see now that There are two earlier opportunities to tweak things: I had good success with USER_SITE=$(python3 -m site --user-site) # just a temp variable - no need to set in profile
mkdir -p $USER_SITE
cp ~/.pythonrc $USER_SITE/usercustomize.py This also works in virtualenvs 🙂 |
@ludwigschwardt Wow, thank you so much! That works perfectly! Screen.Recording.2023-05-24.at.9.21.19.AM.mov |
Whilst I appreciate you looking into this, I'll stick with the |
For anyone else wanting this solution, here's a single command to copy-paste to set up the gnureadline injection:
After that, just install gnureadline if it isn't already: Thanks again @ludwigschwardt! |
I'm very glad it worked for you, Milos! I know PYTHONSTARTUP but never used The @keeely, glad you could sort out your users too. Your solution makes sense given the constraints. Do you import |
Here is an example of a Python script that updates |
Since Python 3.10 it is possible to build against libedit explicitly. This introduced a WITH_EDITLINE macro in pyconfig.h which causes the readline extension module to include the libedit headers instead of the readline headers. Homebrew Python 3.11 finally took the plunge and enabled this macro, causing builds of the extension code to fail with undeclared identifiers and implicit declarations due to the missing includes. I first tried to add `undef_macros=['WITH_EDITLINE']` to the Extension object in setup.py but that had no effect (probably because pyconfig.h is included later). The next best thing is less elegant but effective: undefine it just before it would be used. This addresses @milosivanovic's build issues in #62.
Since Python 3.10 it is possible to build against libedit explicitly. This introduced a WITH_EDITLINE macro in pyconfig.h which causes the readline extension module to include the libedit headers instead of the readline headers. Homebrew Python 3.11 finally took the plunge and enabled this macro, causing builds of the extension code to fail with undeclared identifiers and implicit declarations due to the missing includes. I first tried to add `undef_macros=['WITH_EDITLINE']` to the Extension object in setup.py but that had no effect (probably because pyconfig.h is included later). The next best thing is less elegant but effective: undefine it just before it would be used. This addresses @milosivanovic's build issues in #62.
For me, if all I want is my python interactive shell to use import atexit
import os
import rlcompleter
try:
# If we have gnureadline use it, and save history to ~/.python_history
import gnureadline as readline
history_path = "~/.python_history"
use_gnureadline = True
except ImportError:
# If we don't have gnureadline, import the regular readline and check if
# it is gnu readline or bsd libedit
import readline
use_gnureadline = False
if readline.__doc__.find("libedit") > 0:
# We're probably on OS X (or maybe another BSD), the history file for
# libedit is different from readline, so save it to another file to
# prevent confusion
history_path = "~/.python_history_libedit"
else:
# We're probably on linux, and readline is gnu readline, use the regular
# history file
history_path = "~/.python_history"
history_path = os.path.expanduser(history_path)
if os.path.exists(history_path):
readline.read_history_file(history_path)
# Set the completer so we get tab completion
readline.set_completer(rlcompleter.Completer().complete)
readline.parse_and_bind('tab: complete')
# The function to run at exit to save the history
def save_history(history_path, use_gnureadline):
if use_gnureadline:
import gnureadline as readline
else:
import readline
readline.write_history_file(history_path)
atexit.register(save_history, history_path, use_gnureadline)
# Remove all the vars from global namespace
del os, atexit, readline, rlcompleter, save_history, history_path, use_gnureadline It's a little more complicated than the bare minimum because:
I don't have to mess with |
Since Python 3.10 it is possible to build against libedit explicitly. This introduced a WITH_EDITLINE macro in pyconfig.h which causes the readline extension module to include the libedit headers instead of the readline headers. Homebrew Python 3.11 finally took the plunge and enabled this macro, causing builds of the extension code to fail with undeclared identifiers and implicit declarations due to the missing includes. I first tried to add `undef_macros=['WITH_EDITLINE']` to the Extension object in setup.py but that had no effect (probably because pyconfig.h is included later). The next best thing is less elegant but effective: undefine it just before it would be used. This addresses @milosivanovic's build issues in #62.
Hi @onlynone, I had a good look at your PYTHONSTARTUP approach again. It makes sense what you are doing. The advantages of PYTHONSTARTUP over user/site customization are:
I've just made a PR (#72) describing my latest approach to the problem. I'm curious what the PYTHONSTARTUP approach does about the import readline # The old libedit readline returns...
readline.parse_and_bind('bind ^I rl_complete') but it doesn't repeat this line: readline.set_completer(rlcompleter.Completer().complete) Is that sufficient to make things work out OK? |
I was probably just copy-pasting stuff I found elsewhere when I came up with that. Looking at the rlcompleter docs now I see:
So maybe making the explicit call to
inside the block that does:
|
Hi Steven, I've now verified for myself that your readline.parse_and_bind('tab: complete') with silly completion code customised for each library: if "libedit" in readline.__doc__:
readline.parse_and_bind("bind e rl_complete")
else:
readline.parse_and_bind("r: complete") So if the I've finally released 8.2.10 to PyPI now - thanks for your patience! |
This should be sorted with the 8.2.10 release. |
Thanks a lot for making this package. Given that Homebrew might be removing GNU readline for good (as discussed here #61), python-gnureadline seemed exactly like what I needed to hopefully get readline back into the brew-provided Python 3.11. I know that the README says it's only been tested up to Python 3.10, but I'm not sure if that's the reason for the compilation failure I'm getting.
From...
...I inferred that I should clone this repo and run
python3.11 setup.py install
since I want readline working in the Python REPL. Did I understand that correctly?After running that command, the build process seemed to progress nicely until it reached this part:
It looks like the first error is that append_history is missing, but I'm not sure where that's supposed to be defined. Could you let me know if this is just a 3.11 incompatibility that has yet to be fixed, or if I'm building it wrong?
The text was updated successfully, but these errors were encountered: