Skip to content

Commit

Permalink
Merge pull request #72 from ludwigschwardt/override-readline
Browse files Browse the repository at this point in the history
Override default readline via `python -m override_readline`
  • Loading branch information
ludwigschwardt authored Jun 10, 2024
2 parents 8474e55 + a9f016d commit 0ae068d
Show file tree
Hide file tree
Showing 9 changed files with 360 additions and 88 deletions.
3 changes: 3 additions & 0 deletions .github/workflows/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@ jobs:
os: [ubuntu-20.04, macos-13, macos-14]
python: ["3.6", "3.7", "3.8", "3.9", "3.10", "3.11", "3.12"]
exclude:
# There is something fishy about this combo (broken standard readline)
- os: macos-13
python: "3.7"
# setup-python only gained arm64 from version 3.10.11 onwards
- os: macos-14
python: "3.6"
Expand Down
94 changes: 66 additions & 28 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@ then you already have GNU Readline and you probably don't need this package

then you've come to the right place.


Still interested?
-----------------

Expand All @@ -35,18 +34,18 @@ macOS via a popular open-source package manager such as Homebrew or MacPorts,
you'll get a readline extension module that calls libedit internally (even
though it's confusingly still called "readline"!).

While a lot of effort has been expended to make GNU Readline and Editline
While a lot of effort has gone into making GNU Readline and Editline
interchangeable within Python, they are not fully equivalent. If you want
proper Readline support, this module provides it by bundling the standard
Python readline module with the GNU Readline source code, which is compiled
and statically linked to it. The end result is a package which is simple to
install and requires no extra shared libraries.
install and only requires the system-dependent ncurses library.

The module is called *gnureadline* so as not to clash with the existing
readline module in the standard library. It supports two general needs:

The module is called *gnureadline* so as not to clash with the readline module
in the standard library. This keeps polite installers such as `pip`_ happy and
is sufficient for shells such as `IPython`_. **Please take note that IPython
does not depend on gnureadline anymore since version 5.0 as it now uses**
`prompt_toolkit`_ **instead.**
Code that explicitly imports readline
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

A typical use case is to override readline in your code like this:

Expand All @@ -57,33 +56,72 @@ A typical use case is to override readline in your code like this:
except ImportError:
import readline
If you want to use this module as a drop-in replacement for readline in the
standard Python shell, it has to be installed with the less polite easy_install
script found in `setuptools`_. **Please take note that easy_install has been
deprecated for a while and is about to be dropped from setuptools. Proceed at
your own risk!**
Tab completion in the standard interactive Python shell
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The above trick does not fix tab completion in the Python shell because by
the time the shell prints its first output to the screen, it's too late...
One solution is to put this workaround in one of the customization modules
imported by the `site`_ module early on during the startup process.

This is conveniently done for you by installing *gnureadline* and running::

<python> -m override_readline

where *<python>* is the specific Python interpreter you want to fix
(for example *python3*). The script first tries to add the workaround to
*usercustomize* and then falls back to *sitecustomize* if the user site is
not enabled (for example in virtualenvs). If you want to go straight to
*sitecustomize*, add the standard *-s* option::

<python> -s -m override_readline

The script explains in detail what it is doing and also refuses to install
the workaround twice. Another benefit of *override_readline* is that the
interactive Python interpreter gains a helpful reminder on startup, like::

Python 3.12.2 (main, Apr 17 2024, 20:25:57) [Clang 15.0.0 (clang-1500.0.40.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Using GNU readline instead of the default readline (see sitecustomize.py)
>>>

You don't have to run the *override_readline* script if *gnureadline* was
installed as a dependency of another package. It's only there to help you fix
tab completion in the standard Python shell.

While *usercustomize* and *sitecustomize* are associated with a specific
Python version, you can also fix tab completion for all Python versions
by adding the workaround to the *PYTHONSTARTUP* file (e.g. *~/.pythonrc*).
This requires some extra setup as seen in this `example pythonrc`_, which also
shows a way to maintain separate history files for libreadline and libedit.
The *PYTHONSTARTUP* file only affects the interactive shell, while
user / site customization affects general scripts using readline as well.
The Python Tutorial has a `section`_ describing these customization options.

**Please take note that** `IPython`_ **does not depend on gnureadline for tab
completion anymore. Since version 5.0 it uses** `prompt_toolkit`_ **instead.**

Versions
--------

The module can be used with both Python 2.x and 3.x, and has been tested with
Python versions 2.6, 2.7, and 3.2 to 3.12. The first three numbers of the module
version reflect the version of the underlying GNU Readline library (major,
minor and patch level), while any additional fourth number distinguishes
different module updates based on the same Readline library.

This module is usually unnecessary on Linux and other Unix systems with default
readline support. An exception is if you have a Python distribution that does
not include GNU Readline due to licensing restrictions (such as ActiveState's
ActivePython in the past). If you are using Windows, which also ships without
GNU Readline, you might want to consider using the `pyreadline`_ module instead,
which is a readline replacement written in pure Python that interacts with the
Windows clipboard.
Python versions 2.6, 2.7, and 3.2 to 3.12. The first three numbers of the
module version reflect the version of the underlying GNU Readline library
(major, minor and patch level), while any additional fourth number
distinguishes different module updates based on the same Readline library.

The latest development version is available from the `GitHub repository`_.

If you are using Windows, which also ships without GNU Readline, you might
want to consider using the `pyreadline3`_ module instead, which is a readline
replacement written in pure Python that interacts with the Windows clipboard.

.. _GNU Readline: http://www.gnu.org/software/readline/
.. _Editline: http://www.thrysoee.dk/editline/
.. _pip: http://www.pip-installer.org/
.. _site: https://docs.python.org/library/site.html
.. _example pythonrc: https://github.com/ludwigschwardt/python-gnureadline/issues/62#issuecomment-1724103579
.. _section: https://python.readthedocs.io/en/latest/tutorial/appendix.html#interactive-mode
.. _IPython: http://ipython.org/
.. _prompt_toolkit: http://python-prompt-toolkit.readthedocs.io/en/stable/
.. _setuptools: https://pypi.python.org/pypi/setuptools
.. _pyreadline: http://pypi.python.org/pypi/pyreadline
.. _GitHub repository: http://github.com/ludwigschwardt/python-gnureadline
.. _pyreadline3: http://pypi.python.org/pypi/pyreadline3
160 changes: 160 additions & 0 deletions override_readline.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
#
# Install readline override code and explain what happened.
#
# Run this as `python -m override_readline`, picking the appropriate Python
# executable. This runs on both Python 2.7 and 3.x, hence the use of `io`,
# `os.path` and explicit Unicode.
#
# First target sitecustomize.py (better for venvs and system-wide overrides)
# and then fall back to usercustomize.py if we don't have permission.
#
# Based on the script in https://stackoverflow.com/a/38606990. Thanks, Alastair!
#

import importlib
import io
import os
import os.path
import site
import sys


_HEADER = """
This script will attempt to install an override in Python's site customization
modules that replaces the default readline module with gnureadline.
First check the existing readline module and its replacement
------------------------------------------------------------
"""

_REPORT = """
The following override was added to {filename}:
(full path: {full_path})
{padded_override}
Feel free to remove this{or_even_file} if not needed anymore
(It is also pretty harmless to leave it in there...)
"""

OVERRIDE = u"""
# Added by override_readline script in gnureadline package
import sys
def add_override_message_to_hook():
try:
old_hook = sys.__interactivehook__
except AttributeError:
return
def hook():
old_hook()
print("Using GNU readline instead of the default readline (see {filename})")
sys.__interactivehook__ = hook
try:
import gnureadline as readline
add_override_message_to_hook()
except ImportError:
import readline
sys.modules["readline"] = readline
# End of override_readline block
"""


def check_module(module_name):
"""Attempt to import `module_name` and report basic features."""
try:
module = importlib.import_module(module_name)
except ImportError:
print("Module {name}: not found".format(name=module_name))
return None
style = "libedit" if "libedit" in module.__doc__ else "GNU readline"
kwargs = dict(name=module_name, style=style, path=module.__file__)
print("Module {name}: based on {style}, {path}".format(**kwargs))
return module


def install_override(customize_path):
"""Add override to specified customization module and report back."""
site_directory, customize_filename = os.path.split(customize_path)
banner = "\nAdd override to {filename}\n--------------------------------\n"
print(banner.format(filename=customize_filename))
if not os.path.exists(site_directory):
os.makedirs(site_directory)
print("Created site directory at {dir}".format(dir=site_directory))
file_mode = "r+t" if os.path.exists(customize_path) else "w+t"
with io.open(customize_path, file_mode) as customize_file:
if file_mode == "w+t":
print("Created customize module at {path}".format(path=customize_path))
existing_text = customize_file.read()
override = OVERRIDE.format(filename=customize_filename)
if override in existing_text:
print("Readline override already enabled, nothing to do")
else:
# File pointer should already be at the end of the file after read()
customize_file.write(override)
kwargs = dict(
filename=customize_filename,
full_path=customize_path,
padded_override="\n ".join(override.split("\n")),
or_even_file=" (or even the entire file)" if file_mode == "w+t" else "",
)
print(_REPORT.format(**kwargs))


def override_usercustomize():
if not site.ENABLE_USER_SITE:
print(
"Could not override usercustomize.py because user site "
"directory is not enabled (maybe you are in a virtualenv?)"
)
return False
try:
import usercustomize
except ImportError:
path = os.path.join(site.getusersitepackages(), "usercustomize.py")
else:
path = usercustomize.__file__
try:
install_override(path)
except OSError:
print("Could not override usercustomize.py because file open/write failed")
return False
else:
return True


def override_sitecustomize():
try:
import sitecustomize
except ImportError:
path = os.path.join(site.getsitepackages()[0], "sitecustomize.py")
else:
path = sitecustomize.__file__
try:
install_override(path)
except OSError:
print("Could not override sitecustomize.py because file open/write failed")
return False
else:
return True


def main():
print(_HEADER)
readline = check_module("readline")
gnureadline = check_module("gnureadline")
if not gnureadline:
raise RuntimeError("Please install gnureadline first")
if readline == gnureadline:
print("It looks like readline is already overridden, but let's make sure")
success = override_usercustomize()
if not success:
success = override_sitecustomize()
return 0 if success else 1


if __name__ == '__main__':
sys.exit(main())
17 changes: 17 additions & 0 deletions readline.py
Original file line number Diff line number Diff line change
@@ -1,2 +1,19 @@
#
# WARNING
# -------
#
# This module is meant for distributions like ActiveState Python that have
# no default readline module at all. Since there is no builtin readline that
# clashes with this, it is straightforward to enable readline support:
# simply install gnureadline and you are done.
#
# If your Python ships with readline, please don't see this module as an
# encouragement to move site-packages higher up in `sys.path` or to perform
# any other PYTHONPATH shenanigans in order to override the system readline.
# That decision will come back to haunt you.
#
# Instead, run the override_readline script that only overrides readline and
# nothing else, using site customization. Run `python -m override_readline`.
#
from gnureadline import *
from gnureadline import __doc__
4 changes: 2 additions & 2 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
from setuptools import setup, Extension

if sys.platform == 'win32':
sys.exit('Error: this module is not meant to work on Windows (try pyreadline instead)')
sys.exit('Error: this module is not meant to work on Windows (try pyreadline3 instead)')
elif sys.platform == 'cygwin':
sys.exit('Error: this module is not needed for Cygwin (and probably does not compile anyway)')

Expand Down Expand Up @@ -124,7 +124,7 @@ def build_extensions(self):
maintainer_email="ludwig.schwardt@gmail.com, srid@srid.ca",
url="http://github.com/ludwigschwardt/python-gnureadline",
include_package_data=True,
py_modules=['readline'],
py_modules=['readline', 'override_readline'],
cmdclass={'build_ext': build_ext_subclass},
ext_modules=[
Extension(name="gnureadline",
Expand Down
57 changes: 0 additions & 57 deletions test.py

This file was deleted.

Loading

0 comments on commit 0ae068d

Please sign in to comment.