Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump minimum Python version to 3.8 #934

Merged
merged 4 commits into from
Mar 23, 2020
Merged

Bump minimum Python version to 3.8 #934

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
a6a56ad
Bump minimum Python version to 3.7
danielmitterdorfer Mar 20, 2020
60193d8
Mention pyenv prereqs
danielmitterdorfer Mar 20, 2020
726f890
Highlight pyenv prereqs
danielmitterdorfer Mar 23, 2020
e49b85a
Require Python 3.8
danielmitterdorfer Mar 23, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .ci/variables.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
{
"python_versions": {
"PY38": "3.8.0",
"PY37": "3.7.5",
"PY36": "3.6.9"
"PY37": "3.7.5"
}
}
9 changes: 2 additions & 7 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ PYENV_REGEX = .pyenv/shims
PY_BIN = python3
# https://github.com/pypa/pip/issues/5599
PIP_WRAPPER = $(PY_BIN) -m pip
PY36 = $(shell jq '.python_versions.PY36' .ci/variables.json)
PY37 = $(shell jq '.python_versions.PY37' .ci/variables.json)
PY38 = $(shell jq '.python_versions.PY38' .ci/variables.json)
VENV_NAME ?= .venv
Expand All @@ -35,10 +34,9 @@ PYENV_PREREQ_HELP = "\033[0;31mIMPORTANT\033[0m: please add \033[0;31meval \"\$$
VE_MISSING_HELP = "\033[0;31mIMPORTANT\033[0m: Couldn't find $(PWD)/$(VENV_NAME); have you executed make venv-create?\033[0m\n"

prereq:
pyenv install --skip-existing $(PY36)
pyenv install --skip-existing $(PY37)
pyenv install --skip-existing $(PY38)
pyenv local $(PY36) $(PY37) $(PY38)
pyenv local $(PY37) $(PY38)
@# Ensure all Python versions are registered for this project
@ jq -r '.python_versions | [.[] | tostring] | join("\n")' .ci/variables.json > .python-version
-@ printf $(PYENV_PREREQ_HELP)
Expand Down Expand Up @@ -108,9 +106,6 @@ precommit: lint
it: check-venv python-caches-clean tox-env-clean
. $(VENV_ACTIVATE_FILE); tox

it36: check-venv python-caches-clean tox-env-clean
. $(VENV_ACTIVATE_FILE); tox -e py36

it37: check-venv python-caches-clean tox-env-clean
. $(VENV_ACTIVATE_FILE); tox -e py37

Expand All @@ -131,4 +126,4 @@ release-checks: check-venv
release: check-venv release-checks clean docs it
. $(VENV_ACTIVATE_FILE); ./release.sh $(release_version) $(next_version)

.PHONY: install clean nondocs-clean docs-clean python-caches-clean tox-env-clean docs serve-docs test it it36 it37 it38 benchmark coverage release release-checks prereq venv-create check-env
.PHONY: install clean nondocs-clean docs-clean python-caches-clean tox-env-clean docs serve-docs test it it37 it38 benchmark coverage release release-checks prereq venv-create check-env
41 changes: 14 additions & 27 deletions docs/install.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,35 +13,29 @@ Prerequisites

Rally does not support Windows and is only actively tested on MacOS and Linux. Install the following packages first.

.. _install_python:

Python
~~~~~~

* Python 3.6 or better available as `python3` on the path. Verify with: ``python3 --version``.
* Python 3.7 or better available as ``python3`` on the path. Verify with: ``python3 --version``.
* Python3 header files (included in the Python3 development package).
* ``pip3`` available on the path. Verify with ``pip3 --version``.

**Debian / Ubuntu**

::

sudo apt-get install gcc python3-pip python3.6-dev


**RHEL / CentOS 6 and 7**

Please refer to the `installation instructions for Python 3.6 in the Red Hat Software Collections <https://www.softwarecollections.org/en/scls/rhscl/rh-python36/>`_.
We recommend to use `pyenv <https://github.com/pyenv/pyenv>`_ to manage installation of Python. For details refer to their `installation instructions <https://github.com/pyenv/pyenv#installation>`_ and `ensure that all of pyenv's prerequisites are installed <https://github.com/pyenv/pyenv/wiki/common-build-problems#prerequisites>`_.

**Amazon Linux**
Once ``pyenv`` is installed, install a compatible Python version::

::

sudo yum install -y gcc python3-pip.noarch python3-devel.x86_64

**MacOS**
# Install Python
pyenv install 3.8.0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we mention specifically the system package prerequisites(for compiling python) as mandatory? These are listed nicely here: https://github.com/pyenv/pyenv/wiki/common-build-problems#prerequisites

Without the packages present, one can hit issues like:

$ pip3 install esrally
Collecting esrally
  Using cached https://files.pythonhosted.org/packages/eb/ca/371dcc49c7abcdb1bc8fec8aa76467658c891d935f0c2967137ff036a324/esrally-1.4.1-py3-none-any.whl
Collecting py-cpuinfo==3.2.0 (from esrally)
  Downloading https://files.pythonhosted.org/packages/48/1d/404dff45f8bdc63063ac0b822b557a5a5df54679de55261021efc882b8dd/py-cpuinfo-3.2.0.tar.gz (76kB)
     |████████████████████████████████| 81kB 4.7MB/s
    ERROR: Command errored out with exit status 1:
     command: /home/vagrant/.pyenv/versions/3.8.0/bin/python3.8 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-h35_rza2/py-cpuinfo/setup.py'"'"'; __file__='"'"'/tmp/pip-install-h35_rza2/py-cpuinfo/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base pip-egg-info
         cwd: /tmp/pip-install-h35_rza2/py-cpuinfo/
    Complete output (11 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/home/vagrant/.pyenv/versions/3.8.0/lib/python3.8/site-packages/setuptools/__init__.py", line 20, in <module>
        from setuptools.dist import Distribution, Feature
      File "/home/vagrant/.pyenv/versions/3.8.0/lib/python3.8/site-packages/setuptools/dist.py", line 35, in <module>
        from setuptools import windows_support
      File "/home/vagrant/.pyenv/versions/3.8.0/lib/python3.8/site-packages/setuptools/windows_support.py", line 2, in <module>
        import ctypes
      File "/home/vagrant/.pyenv/versions/3.8.0/lib/python3.8/ctypes/__init__.py", line 7, in <module>
        from _ctypes import Union, Structure, Array
    ModuleNotFoundError: No module named '_ctypes'
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

which are a bit hard to diagnose.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pondered doing this but this is also linked from Pyenv's installation instructions which is the reason I thought of not replicating it here. But you have a point...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed in 60193d8

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. I'd personally emphasize with bold as below, but it's a personal preference (to ensure maximum visibility for users)

and **ensure that all of** `pyenv's prerequisites <https://github.com/pyenv/pyenv/wiki/common-build-problems#prerequisites>`_ are installed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 726f890.


We recommend that you use `Homebrew <https://brew.sh/>`_::
# select that version for the current user
# see https://github.com/pyenv/pyenv/blob/master/COMMANDS.md#pyenv-global for details
pyenv global 3.8.0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initially I wondered if this is too invasive i.e. will change the default version of Python for the user leading to surprises.

However, as these instructions are targeted also for released versions, I think this is ok. Otherwise we'd suggest something like:

mkdir some_stuff_rally_dir
cd some_stuff_rally_dir
pyenv local 3.8.0
exec $SHELL

# and then install Rally from within this dir
pip3 install esrally


brew install python3
# Install pip3
curl -s https://bootstrap.pypa.io/get-pip.py -o get-pip.py
python3 get-pip.py --user

git
~~~
Expand Down Expand Up @@ -104,7 +98,7 @@ Non-sudo Install
If you don't want to use ``sudo`` when installing Rally, installation is still possible but a little more involved:

1. Specify the ``--user`` option when installing Rally (step 2 above), so the command to be issued is: ``python3 setup.py develop --user``.
2. Check the output of the install script or lookup the `Python documentation on the variable site.USER_BASE <https://docs.python.org/3.5/library/site.html#site.USER_BASE>`_ to find out where the script is located. On Linux, this is typically ``~/.local/bin``.
2. Check the output of the install script or lookup the `Python documentation on the variable site.USER_BASE <https://docs.python.org/3/library/site.html#site.USER_BASE>`_ to find out where the script is located. On Linux, this is typically ``~/.local/bin``.

You can now either add ``~/.local/bin`` to your path or invoke Rally via ``~/.local/bin/esrally`` instead of just ``esrally``.

Expand All @@ -119,13 +113,6 @@ You can also use Virtualenv to install Rally into an isolated Python environment

Whenever you want to use Rally, run the activation script (step 2 above) first. When you are done, simply execute ``deactivate`` in the shell to exit the virtual environment.

PyEnv Install
------------------

Rally can be tested with different Python versions and it uses pyenv to manage them.

Please refer to PyEnv `installation instructions <https://github.com/pyenv/pyenv#installation>`_.

Docker
------

Expand Down
8 changes: 8 additions & 0 deletions docs/migrate.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,14 @@
Migration Guide
===============

Migrating to Rally 1.5.0
------------------------

Minimum Python version is 3.7.0
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Rally 1.5.0 requires Python 3.7.0. Check the :ref:`updated installation instructions <install_python>` for more details.

Migrating to Rally 1.4.1
------------------------

Expand Down
2 changes: 1 addition & 1 deletion docs/quickstart.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Rally is developed for Unix and is actively tested on Linux and MacOS. Rally sup
Install
-------

Install Python 3.6+ including ``pip3``, git 1.9+ and an `appropriate JDK to run Elasticsearch <https://www.elastic.co/support/matrix#matrix_jvm>`_ Be sure that ``JAVA_HOME`` points to that JDK. Then run the following command, optionally prefixed by ``sudo`` if necessary::
Install Python 3.7+ including ``pip3``, git 1.9+ and an `appropriate JDK to run Elasticsearch <https://www.elastic.co/support/matrix#matrix_jvm>`_ Be sure that ``JAVA_HOME`` points to that JDK. Then run the following command, optionally prefixed by ``sudo`` if necessary::

pip3 install esrally

Expand Down
4 changes: 2 additions & 2 deletions esrally/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,8 +72,8 @@


def check_python_version():
if sys.version_info.major != 3 or sys.version_info.minor < 6:
raise RuntimeError("Rally requires at least Python 3.6 but you are using:\n\nPython %s" % str(sys.version))
if sys.version_info.major != 3 or sys.version_info.minor < 7:
raise RuntimeError("Rally requires at least Python 3.7 but you are using:\n\nPython %s" % str(sys.version))


def doc_link(path=None):
Expand Down
2 changes: 1 addition & 1 deletion setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ def str_from_file(name):
long_description = str_from_file("README.rst")

# tuples of (major, minor) of supported Python versions ordered from lowest to highest
supported_python_versions = [(3, 6), (3, 7), (3, 8)]
supported_python_versions = [(3, 7), (3, 8)]

################################################################################################
#
Expand Down
2 changes: 1 addition & 1 deletion tox.ini
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
###############################################################################
[tox]
envlist =
docs, py36, py37, py38
docs, py37, py38
platform =
linux|darwin

Expand Down