Enhance control over python build during pyenv install execution

[PrieJos]

Purpose

Please find in this PR several enhancements made over the current HEAD of your main branch (436b085) which are aimed at improving the control over how pyenv plugin python-build builds the python runtime from source code.

The origin of this PR is from issue #216.

Proposed Enhancements

Enhancement 1: Additional custom .pyenvrc

You can now optionally add a custom .pyenvrc shell script in addition to the one delivered by the role (which created out the Jinja template .pyenvrc.j2. This custom .pyenvrc will be copied as .pyenvrc.custom to the same path as .pyenvrc. It will be sourced then by .pyenvrc delivered by the role if exists.

This will become one way you can customize the python build environment. This way you can for instance set an environment not only as python-build plugin supports according to what its documentation Special Environment Variables describes, but you can also set other environment variables supported by the python source code
compilation itself. This becomes very handy indeed to take full control over the python build customization.

Enhancement 2: Additional variables to customize python build task

It adds the following optional role variables that can be defined by users to alter how python-build plugin compiles python when asked for install:

Role Variable	Environment variable according to python-build
pyenv_tmpdir	TMPDIR
pyenv_python_build_build_path	PYTHON_BUILD_BUILD_PATH
pyenv_python_build_cache_path	PYTHON_BUILD_CACHE_PATH
pyenv_python_build_mirror_url	PYTHON_BUILD_MIRROR_URL
pyenv_python_build_mirror_url_skip_checksum	PYTHON_BUILD_MIRROR_URL_SKIP_CHECKSUM
pyenv_python_build_skip_mirror	PYTHON_BUILD_SKIP_MIRROR
pyenv_python_build_skip_homebrew	PYTHON_BUILD_SKIP_HOMEBREW
pyenv_python_build_root	PYTHON_BUILD_ROOT
pyenv_python_build_definitions	PYTHON_BUILD_DEFINITIONS
pyenv_python_configure_opts	PYTHON_CONFIGURE_OPTS
pyenv_python_cflags	PYTHON_CFLAGS
pyenv_python_make_opts	PYTHON_MAKE_OPTS
pyenv_python_make_install_opts	PYTHON_MAKE_INSTALL_OPTS
pyenv_configure_opts	CONFIGURE_OPTS
pyenv_cc	CC
pyenv_make	MAKE
pyenv_make_opts	MAKE_OPTS
pyenv_make_install_opts	MAKE_INSTALL_OPTS
pyenv_profile_task	PROFILE_TASK

Note: pyenv_profile_task was added to support all options to build python for maximum performance as described in Building for maximum performance.

All those roles variables will be another way to customize the python build environment.

Enhancement 3: Additional options in call pyenv install

The new optional role variable pyenv_install_extra_opts can be defined to pass in extra options to the call to command pyenv install.

[PrieJos]

By the way. I tested the enhancements in a RHEL 7 using a playbook like this:

---
- name: Deploy Python via pyenv
  hosts: "{{ target|default('all') }}"
  gather_facts: true 
  vars:
    pyenv_owner: priejos
    pyenv_owner_group: users
    pyenv_python_versions:
      - 3.11.2
    pyenv_global:
      - automation
    pyenv_virtualenvs:
      - venv_name: automation
        py_version: 3.11.2
    pyenv_path: "{{ ansible_env.HOME }}/.pyenv"
    pyenv_make_opts: "-j4"
    pyenv_python_configure_opts: "--enable-optimizations --with-lto --with-ensurepip=upgrade"
    pyenv_python_cflags: "-march=native -mtune=native"
    pyenv_profile_task: "-m test.regrtest --pgo -j0"
    pyenv_init_options: bash
    pyenv_custom_pyenvrc_file: pyenvrc.custom.rhel7
  tasks:
    - name: Run pyenv role
      ansible.builtin.include_role:
        name: staticdev.pyenv

The custom pyenvrc file pyenvrc.custom.rhel7 contains the next customization of the environment required to compile Python 3.11 on a RHEL 7:

source /opt/rh/devtoolset-11/enable
export CPPFLAGS='-I/usr/include/openssl11'
export LDFLAGS='-L/usr/lib64/openssl11'
export TCLTK_LIBS="-ltk8.5 -ltcl8.5"

[staticdev]

Thanks a lot for your contribution @PrieJos!
Would you kindly also add to the readme information about the variables introduced such as possible values, defaults and what they mean?
Other than that super good!

[PrieJos]

Hi @staticdev

Thanks a lot for your contribution @PrieJos! Would you kindly also add to the readme information about the variables introduced such as possible values, defaults and what they mean? Other than that super good!

You are absolutely welcome. Indeed I should also be thanked to you since I use your role to deploy an Ansible runtime environment. So it is of great help.

Anyway, I added a new commit with the README.md changes as suggested (0f18d5c). So please take your time to test it from your side and let me know if you have any further question. Indeed the sooner you can merge this into a new release upstream, the better for us 😄

Thanks again
Jose M. Prieto

[staticdev]

Thanks again @PrieJos I am thinking of making it a version 3.0.0.

Do you maybe have a suggestion to put groups of variables for the readme? Now we have a huge amount of variables.. I am thinking of a new person that never used the role, if I saw this huge list like in here https://github.com/PrieJos/ansible-role-pyenv/tree/feature-python-build I would think it is too complicated and not a thing I would invest my time ;)

I also think I will add a first simpler example usage (to have 2 instead of 1) just with a minimal number of variables to a person that just a quick installation.

[PrieJos]

Hi @staticdev

Do you maybe have a suggestion to put groups of variables for the readme? Now we have a huge amount of variables.. I am thinking of a new person that never used the role, if I saw this huge list like in here https://github.com/PrieJos/ansible-role-pyenv/tree/feature-python-build I would think it is too complicated and not a thing I would invest my time ;)

So maybe all the new variables, except pyenv_custom_pyenvrc_file and pyenv_install_extra_opts, should be grouped together in a category like Python Compilation, while the rest (yours and the two exceptions before), should be grouped into a category like Pyenv Installation.

I also think I will add a first simpler example usage (to have 2 instead of 1) just with a minimal number of variables to a person that just a quick installation.

So regarding this point, I pasted above a possible 2nd example scenario which was the one I used to test my contribution in a RHEL 7.

Cheers
Jose M. Prieto

[staticdev]

Hi @staticdev

Do you maybe have a suggestion to put groups of variables for the readme? Now we have a huge amount of variables.. I am thinking of a new person that never used the role, if I saw this huge list like in here https://github.com/PrieJos/ansible-role-pyenv/tree/feature-python-build I would think it is too complicated and not a thing I would invest my time ;)

So maybe all the new variables, except pyenv_custom_pyenvrc_file and pyenv_install_extra_opts, should be grouped together in a category like Python Compilation, while the rest (yours and the two exceptions before), should be grouped into a category like Pyenv Installation.

I also think I will add a first simpler example usage (to have 2 instead of 1) just with a minimal number of variables to a person that just a quick installation.

Looks good to me.

So regarding this point, I pasted above a possible 2nd example scenario which was the one I used to test my contribution in a RHEL 7.

Cheers Jose M. Prieto

No problem, and we can even add RHEL to our test images for CI.

[PrieJos]

Hi @staticdev

Any plan by when this PR will be merged into the main branch?

Thanks
Jose M Prieto

[staticdev]

@PrieJos very sorry for the long wait. I am having a lot of trouble with new Ansible Galaxy API / UI. I need to first release a bugfix version with current fixes. Then we can do a new major refactor. Meanwhile, I see there are some conflicts with new ansible syntax on the readme that you could change to new vars syntax. See #223

[staticdev]

Ok, at least now I fixed the release issues. Please just update your PR so we can merge.

[PrieJos]

Hi @staticdev

Thanks for your response.

Ok, at least now I fixed the release issues. Please just update your PR so we can merge.

So I have nothing to update in the MR since the last time I did. Let me know if you need anything else.

Thanks again.

Cheers
Jose M. Prieto

[staticdev]

@PrieJos actually by update i mean rebase to have this conflict on readme solved.

[staticdev]

@PrieJos anyway I fixed it, thanks.