Ansible request for CentOS6 adoptopenjdk_install role

[Willsparker]

Affecting both VPC and the CentOS6 GitHub Check:

• https://ci.adoptopenjdk.net/job/VagrantPlaybookCheck/997/OS=CentOS6,label=vagrant/console

• https://github.com/AdoptOpenJDK/openjdk-infrastructure/pull/1875/checks?check_run_id=1804893220

• Missing install

• Bug in ansible playbook

• Request for new playbook addition

Details:

The issue is with adoptopenjdk_install :

fatal: [localhost]: FAILED! => {"attempts": 3, "changed": false, "msg": "Failed to validate the SSL certificate for github-releases.githubusercontent.com:443. Make sure your managed systems have a valid CA certificate installed. If the website serving the url uses SNI you need python >= 2.7.9 on your managed machine  (the python executable used (/usr/bin/python) is version: 2.6.6 (r266:84292, Jun 20 2019, 14:14:55) [GCC 4.4.7 20120313 (Red Hat 4.4.7-23)]) or you can install the `urllib3`, `pyOpenSSL`, `ndg-httpsclient`, and `pyasn1` python modules to perform SNI verification in python >= 2.6. You can use validate_certs=False if you do not need to confirm the servers identity but this is unsafe and not recommended. Paths checked for this platform: /etc/ssl/certs, /etc/pki/ca-trust/extracted/pem, /etc/pki/tls/certs, /usr/share/ca-certificates/cacert.org, /etc/ansible. The exception msg was: hostname 'github-releases.githubusercontent.com' doesn't match either of '*.githubassets.com', 'githubassets.com'.", "status": -1, "url": "https://api.adoptopenjdk.net/v3/binary/latest/8/ga/linux/x64/jdk/hotspot/normal/adoptopenjdk?project=jdk"}

Apparently it's failing to validate the SSL Certificate for github-releases.githubusercontent.com. If all else fails, we can use validate_certs=False, but this is a last resort.

[Willsparker]

Replicated problem on local CentOS6 VM. Looking at doing as the Issue suggests, and updating Python to 2.7.9, however, we'll still have to ensure the GH check and the VPC machine has Python 2.7.9

[Willsparker]

Hmm, installing 2.7.9 ruined my Ansible install.... Not something I want to do on the VPC machine

[Willsparker]

Oh - my mistake - the error message is talking about the CentOS6 Python install, I think.
Installing it on CentOS6, following this https://www.linuxtweaks.in/how-to-install-python-2-7-in-centosrhel/ (and changing 2.7.8 for 2.7.9). I then symlinked /usr/bin/python to /usr/local/bin/python2.7. This also retains the original python 2.6 at /usr/bin/python2. This makes the role work!
@sxa Because a lot depends on Python, is it okay to have the playbook install 2.7 ? Alternatively, to fix VPC, we can have python2.7 installed in the provisioning script. As for GH check, I can't see where anything might be installed before the playbook is run on the docker image?

[Willsparker]

I've made a Python 2.7.9 install role, which can be seen at https://github.com/Willsparker/openjdk-infrastructure/tree/1877 . Currently testing with VPC : https://ci.adoptopenjdk.net/job/VagrantPlaybookCheck/1002/

[Willsparker]

https://ci.adoptopenjdk.net/job/VagrantPlaybookCheck/1005/
Seems this will have implications later on. I'll try to fix that issue and then if anymore come up , I think I'll abandon the idea of installing Python 2.7.9 and try to find a solution to the original problem.

[Willsparker]

Okay, yep, scrap the Python 2.7.9 idea. I'll put the yaml here, just in case we ever want to come back to it:

---
####################
# Python Bootstrap #
####################

# Check if Python is installed, and it's version. If Python isn't installed, or it's at a version below 2.7.9, install Python 2.7.9.
# Currently only for CentOS6. See: https://github.com/AdoptOpenJDK/openjdk-infrastructure/issues/1877

- name: Check for Python 2's version
  shell: python -V 2>&1 | grep -Po '(?<=Python )(.+)'
  register: python_version
  ignore_errors: yes
  when:
    - (ansible_distribution == "CentOS") and (ansible_distribution_major_version == "6")
  tags: 
    - python_bootstrap

- name: Yum install prerequisites to build Python 2.7.9
  package:
    name: '{{ item }}'
    state: latest
  with_items:
    - zlib-devel
    - bzip2-devel
    - openssl-devel
    - xz-libs
    - wget
  when: 
    - (ansible_distribution == "CentOS") and (ansible_distribution_major_version == "6")
    - (python_version.rc != 0) or (python_version.rc == 0 and python_version.stdout is version_compare('2.7.9', operator='lt') )
  tags:
    - python_bootstrap

- name: Download & extract Python 2.7.9 source
  unarchive: 
    src: http://www.python.org/ftp/python/2.7.9/Python-2.7.9.tar.xz 
    dest: /tmp/
    remote_src: yes
    mode: 0755
  retries: 3 
  delay: 5
  register: python_download
  until: python_download is not failed
  when: 
    - (ansible_distribution == "CentOS") and (ansible_distribution_major_version == "6")
    - (python_version.rc != 0) or (python_version.rc == 0 and python_version.stdout is version_compare('2.7.9', operator='lt') )
  tags:
    - python_bootstrap

- name: Configure, build & make/make altinstall
  shell: |
    cd /tmp/Python-2.7.9 && ./configure --prefix=/usr/local && make && make altinstall
  when:
    - (ansible_distribution == "CentOS") and (ansible_distribution_major_version == "6")
    - (python_version.rc != 0) or (python_version.rc == 0 and python_version.stdout is version_compare('2.7.9', operator='lt') )
  tags:
    - python_bootstrap

- name: Symlink Python 2.7.9 to /usr/bin/python
  file:
    src: /usr/local/bin/python2.7
    dest: /usr/bin/python
    state: link
    force: yes
  when:
    - (ansible_distribution == "CentOS") and (ansible_distribution_major_version == "6")
    - (python_version.rc != 0) or (python_version.rc == 0 and python_version.stdout is version_compare('2.7.9', operator='lt') )
  tags:
    - python_bootstrap

Onwards

[Willsparker]

Okay, I tried installing the 4 pip modules that the error message suggests, and while installing pyOpenSSL, the following error occurs:

[vagrant@adoptopenjdkC6 ~]$ sudo pip install pyOpenSSL
You are using pip version 7.1.0, however version 21.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting pyOpenSSL
  Downloading https://files.pythonhosted.org/packages/b2/5e/06351ede29fd4899782ad335c2e02f1f862a887c20a3541f17c3fa1a3525/pyOpenSSL-20.0.1-py2.py3-none-any.whl (54kB)
    100% |████████████████████████████████| 57kB 4.2MB/s 
Collecting cryptography>=3.2 (from pyOpenSSL)
  Downloading https://files.pythonhosted.org/packages/b7/82/f7a4ddc1af185936c1e4fa000942ffa8fb2d98cff26b75afa7b3c63391c4/cryptography-3.3.1.tar.gz (539kB)
    100% |████████████████████████████████| 540kB 203kB/s 
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 20, in <module>
      File "/tmp/pip-build-hDZHat/cryptography/setup.py", line 25, in <module>
        exec (f.read(), about)
      File "<string>", line 31, in <module>
    ValueError: zero length field name in format
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-hDZHat/cryptography

And much the same error message installing ndg-httpsclient :

[vagrant@adoptopenjdkC6 ~]$ sudo pip install ndg-httpsclient
You are using pip version 7.1.0, however version 21.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting ndg-httpsclient
  Downloading https://files.pythonhosted.org/packages/bf/b2/26470fde7ff55169df8e071fb42cb1f83e22bd952520ab2b5c5a5edc2acd/ndg_httpsclient-0.5.1-py2-none-any.whl
Collecting PyOpenSSL (from ndg-httpsclient)
  Using cached https://files.pythonhosted.org/packages/b2/5e/06351ede29fd4899782ad335c2e02f1f862a887c20a3541f17c3fa1a3525/pyOpenSSL-20.0.1-py2.py3-none-any.whl
Collecting pyasn1>=0.1.1 (from ndg-httpsclient)
  Downloading https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl (77kB)
    100% |████████████████████████████████| 77kB 4.2MB/s 
Collecting cryptography>=3.2 (from PyOpenSSL->ndg-httpsclient)
  Using cached https://files.pythonhosted.org/packages/b7/82/f7a4ddc1af185936c1e4fa000942ffa8fb2d98cff26b75afa7b3c63391c4/cryptography-3.3.1.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 20, in <module>
      File "/tmp/pip-build-vqod4k/cryptography/setup.py", line 25, in <module>
        exec (f.read(), about)
      File "<string>", line 31, in <module>
    ValueError: zero length field name in format
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-vqod4k/cryptography

[aahlenst]

You're trying things with ancient versions of pip. python -m pip install --upgrade pip should help. While you are at it, use the latest version of Python 2.7 (2.7.18). If I remember correctly, Ansible favours Python 3 nowadays. Maybe it compiles on CentOS 6? Would give us patches.

[Willsparker]

Yeah ... it may be worth bringing up the conversation of running our playbooks on Python 3.
I'll trying upgrading pip, and seeing if that changes anything, thanks :-)

[Willsparker]

On a fresh Vagrant VM:

[vagrant@adoptopenjdkC6 ~]$ pip install --upgrade pip
You are using pip version 7.1.0, however version 21.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting pip
  Downloading https://files.pythonhosted.org/packages/b7/2d/ad02de84a4c9fd3b1958dc9fb72764de1aa2605a9d7e943837be6ad82337/pip-21.0.1.tar.gz (1.5MB)
    100% |████████████████████████████████| 1.6MB 151kB/s 
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 20, in <module>
      File "/tmp/pip-build-0PsDnh/pip/setup.py", line 77, in <module>
        "pip{}=pip._internal.cli.main:main".format(sys.version_info[0]),
    ValueError: zero length field name in format
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-0PsDnh/pip

🤦

[Willsparker]

I'm just testing to see if validating_certs: no will fix the issue - as a temporary fix, it may be considered passable considering it's coming from our api server. But it shouldn't be permanent.
https://ci.adoptopenjdk.net/job/VagrantPlaybookCheck/1012/

[sxa]

I'm just testing to see if validating_certs: no will fix the issue - as a temporary fix, it may be considered passable considering it's coming from our api server.

It isn't - the API server redirects to github. I would possibly be ok with ignoring the cert if either we verified a checksum on the download, although that would need to be updated on each release and the role is currently generic across all versions and platforms so that's non-trivial

[Willsparker]

Yeah, as you say, it's non trivial - I'd prefer to work on finding the fix over finding the optimal workaround :-)

I may go back to building Python 2.7.9 as part of the playbook and dealing with the problems from that.

[Willsparker]

Okay, so the issue I'm having when using Python 3.6.10 on a CentOS6 VM is as follows:

< TASK [Common : Enable EPEL release (not CentOS8)] >
fatal: [127.0.0.1]: FAILED! => {"changed": false, "msg": "The Python 2 bindings for rpm are needed for this module. If you require Python 3 support use the `dnf` Ansible module instead.. The Python 2 yum module is needed for this module. If you require Python 3 support use the `dnf` Ansible module instead."}

Note for future me: You can force the ansible_python_interpreter by using -e 'ansible_python_interpreter=/usr/local/bin/python3' in the ansible-playbook command.

Apparently this is a known issue of the dnf module (not specifically the yum module) (ansible/ansible#67083 (comment)). I assume this affects yum too.

[Willsparker]

From what I can see, CentOS6 's Yum is dependent on Python2.6, and so we can't just globally set the ansible_python_interpreter to the newer version of Python. And as far as I can see, there's no way of setting the interpreter to a specific python for a single task.

[aahlenst]

If the problem is only the missing Let's Encrypt root certificate, you should be able to put those somewhere in /etc, invoke some tool, and CentOS should re-generate the trust stores for all tools (see https://manuals.gfi.com/en/kerio/connect/content/server-configuration/ssl-certificates/adding-trusted-root-certificates-to-the-server-1605.html). That's the short-term solution. The long-term solution is to do what Oracle does: Install a CentOS 6 sysroot onto a CentOS 7 system.

[Willsparker]

Ah okay... I googled a bit, and found that the SSL Cert can be found with openssl s_client -showcerts -servername api.adoptopenjdk.net -connect api.adoptopenjdk.net:443 < /dev/null - however, that's outputting 2 different certificates.

[aahlenst]

You want ISRG Root X1, and ISRG Root X2 from https://letsencrypt.org/certificates/.

[Willsparker]

Do you know how I can retrieve them from the command line? The command I put above doesn't output anything about ISRG, and I can't find anything online about how to retrieve it

[aahlenst]

First, check with curl -vI https://valid-isrgrootx1.letsencrypt.org/ whether it fails.

To get the certificates:

curl -O https://letsencrypt.org/certs/isrgrootx1.pem
curl -O https://letsencrypt.org/certs/isrg-root-x1-cross-signed.pem
curl -O https://letsencrypt.org/certs/isrg-root-x2.pem
curl -O https://letsencrypt.org/certs/isrg-root-x2-cross-signed.pem

After importing the certificates into the OS, check again with curl -vI https://valid-isrgrootx1.letsencrypt.org/ and it should work now.

[A client program has to bring the root certificates itself, otherwise the whole chain of trust wouldn't work. The client program trusts those certificates and by trusting those, it trusts every certificate signed with one of those root certificates. If the root certificates were supplied by the server or if you had none, It would be like connecting to a server with a self-signed certificate because there's no pre-established trust. That's why openssl s_client is useless for this task.]

[Willsparker]

Ah okay. I retrieved them and added them to the system as per https://manuals.gfi.com/en/kerio/connect/content/server-configuration/ssl-certificates/adding-trusted-root-certificates-to-the-server-1605.html , however the curl -vI https://valid-isrgrootx1.letsencrypt.org/ command seemed to output the same before and after adding those certs. I am able to wget an archive, but I get the error when using the unarchive module in Ansible

[aahlenst]

Can you paste the curl output here and run curl -vI api.adoptopenjdk.net, too? What does python -c "import ssl; print(ssl.get_default_verify_paths()) print?

[Willsparker]

[vagrant@adoptopenjdkC6 ~]$ curl -vI api.adoptopenjdk.net
* Rebuilt URL to: api.adoptopenjdk.net/
*   Trying 104.17.159.60...
* TCP_NODELAY set
* Connected to api.adoptopenjdk.net (104.17.159.60) port 80 (#0)
> HEAD / HTTP/1.1
> Host: api.adoptopenjdk.net
> User-Agent: curl/7.61.1
> Accept: */*
> 
< HTTP/1.1 301 Moved Permanently
HTTP/1.1 301 Moved Permanently
< Date: Tue, 16 Feb 2021 09:18:01 GMT
Date: Tue, 16 Feb 2021 09:18:01 GMT
< Connection: keep-alive
Connection: keep-alive
< Cache-Control: max-age=3600
Cache-Control: max-age=3600
< Expires: Tue, 16 Feb 2021 10:18:01 GMT
Expires: Tue, 16 Feb 2021 10:18:01 GMT
< Location: https://api.adoptopenjdk.net/
Location: https://api.adoptopenjdk.net/
< cf-request-id: 084bbaf1470000e658c838a000000001
cf-request-id: 084bbaf1470000e658c838a000000001
< Server: cloudflare
Server: cloudflare
< CF-RAY: 62262dc87ae9e658-LHR
CF-RAY: 62262dc87ae9e658-LHR
< alt-svc: h3-27=":443"; ma=86400, h3-28=":443"; ma=86400, h3-29=":443"; ma=86400
alt-svc: h3-27=":443"; ma=86400, h3-28=":443"; ma=86400, h3-29=":443"; ma=86400

< 
* Connection #0 to host api.adoptopenjdk.net left intact

and

[vagrant@adoptopenjdkC6 ~]$ python -c "import ssl; print(ssl.get_default_verify_paths())"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
AttributeError: 'module' object has no attribute 'get_default_verify_paths'

Doesn't look like get_default_verify_paths exists. I can't even check the ssl.OPENSSL_VERSION supposedly because Python2.6 doesn't support that.

[aahlenst]

curl -vI api.adoptopenjdk.net needs a https. You're testing port 80 at the moment.

[Willsparker]

Redone:

[vagrant@adoptopenjdkC6 ~]$ curl -vI https://api.adoptopenjdk.net
* Rebuilt URL to: https://api.adoptopenjdk.net/
*   Trying 104.17.158.60...
* TCP_NODELAY set
* Connected to api.adoptopenjdk.net (104.17.158.60) port 443 (#0)
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* NPN, negotiated HTTP1.1
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Next protocol (67):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-ECDSA-AES128-GCM-SHA256
* Server certificate:
*  subject: C=US; ST=CA; L=San Francisco; O=Cloudflare, Inc.; CN=adoptopenjdk.net
*  start date: Jul  2 00:00:00 2020 GMT
*  expire date: Jul  2 12:00:00 2021 GMT
*  subjectAltName: host "api.adoptopenjdk.net" matched cert's "*.adoptopenjdk.net"
*  issuer: C=US; O=Cloudflare, Inc.; CN=Cloudflare Inc ECC CA-3
*  SSL certificate verify ok.
> HEAD / HTTP/1.1
> Host: api.adoptopenjdk.net
> User-Agent: curl/7.61.1
> Accept: */*
> 
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< Date: Tue, 16 Feb 2021 11:38:16 GMT
Date: Tue, 16 Feb 2021 11:38:16 GMT
< Content-Length: 1846
Content-Length: 1846
< Connection: keep-alive
Connection: keep-alive
< Set-Cookie: __cfduid=d07de1c281a7523a818f9401eb9100b3f1613475496; expires=Thu, 18-Mar-21 11:38:16 GMT; path=/; domain=.adoptopenjdk.net; HttpOnly; SameSite=Lax
Set-Cookie: __cfduid=d07de1c281a7523a818f9401eb9100b3f1613475496; expires=Thu, 18-Mar-21 11:38:16 GMT; path=/; domain=.adoptopenjdk.net; HttpOnly; SameSite=Lax
< Accept-Ranges: bytes
Accept-Ranges: bytes
< cache-control: public, max-age=86400
cache-control: public, max-age=86400
< last-modified: Tue, 12 Jan 2021 11:54:22 GMT
last-modified: Tue, 12 Jan 2021 11:54:22 GMT
< vary: accept-encoding
vary: accept-encoding
< Set-Cookie: b7b892882bae631693e1ea44963ef628=fde9d519ef5131c720794443840a88a0; path=/; HttpOnly; Secure
Set-Cookie: b7b892882bae631693e1ea44963ef628=fde9d519ef5131c720794443840a88a0; path=/; HttpOnly; Secure
< Cache-control: private
Cache-control: private
< CF-Cache-Status: DYNAMIC
CF-Cache-Status: DYNAMIC
< cf-request-id: 084c3b5a240000423374814000000001
cf-request-id: 084c3b5a240000423374814000000001
< Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
< Server: cloudflare
Server: cloudflare
< CF-RAY: 6226fb3d0fa54233-LHR
CF-RAY: 6226fb3d0fa54233-LHR
< alt-svc: h3-27=":443"; ma=86400, h3-28=":443"; ma=86400, h3-29=":443"; ma=86400
alt-svc: h3-27=":443"; ma=86400, h3-28=":443"; ma=86400, h3-29=":443"; ma=86400

< 
* Connection #0 to host api.adoptopenjdk.net left intact

[Willsparker]

Upon further investigation with @aahlenst . we've ascertained that this probably can't be done properly. Python 2.6 doesn't support SNI, which is the issue when connecting with http://github-releases.githubusercontent.com/ . We can't install a different version of Python, otherwise yum won't work.

So, we have a few options, none of them great:

1. Merge UnixPB: Temporarily fix CentOS6 Adoptopenjdk_install role #1913 - which will stop validating the certs.
2. use the get_url module to retrieve the archive and decompress it in a separate task (however, due to adoptopenjdk_install being a generalised role, the checksum can't be put in, so I don't think it's safer than validate_certs: no ).
3. Try to do what Oracle does with CentOS6, which is install a CentOS6 sysroot on a CentOS7 system - however; I don't know how easy that'd be to implement into VPC / the GitHub Check.

regarding option 2: Can the API provide a checksum of the archive that's being requested? If so, it could be assigned to a variable easily enough

[sxa]

We can't install a different version of Python, otherwise yum won't work.

Is it not possible to put Python2.7 in a different location on the machine and only use that with ansible? i.e. don't make it the default on the machine, therefore yum should be unaffected?

use the get_url module to retrieve the archive and decompress it in a separate task

Does that work and validate the certificate (Seems unlikely if unarchive doesn't)? If so I don't see why that's a problem ...

[sxa]

I've just built my own Python27 and pointed ansible at it after setting ansible_python_interpreter=/usr/local/python2/bin/python as a variable to ansible and it was able to complete the adoptopenjdk_install role. Note that if you set that value it also uses it for operations on the machine that you're running ansible-playbook on so it's not ideal unless you're running ansible-playbook on the target machine unless you symlink /usr/local/python2/bin/python to the default python on your local system. (There must be another way of doing that :-) ) - Although is we only need this in docker containers, then we're probably running it on the machine anyway, and the updated python can be installed via the Dockerfile, although we'll need some magic for VPC ... magic may be the aforementioned symlink, or just extracting the python27 onto the vagrant machine)

tarball is at https://ci.adoptopenjdk.net/userContent/usrlocalPython27.tar.xz if we can make use of it (Nothing special done to make it, just created from this source, then ./configure --prefix=/usr/local/python2; make -j10; make install)

[Willsparker]

With that, did you run the whole playbook or just the adoptopenjdk_install role? If it's just the adoptopenjdk-install role, try running the Common role (or anything that uses Ansible's yum module).

From my brief googling, wget does validate the SSL cert, I think? I suppose the difference is that the unarchive module uses Python, and wget on the machine doesn't. however, I haven't tested the get_url module in Ansible, so that may have the same issue.

[sxa]

The Common role can't easily be tested because it hits a Error: Cannot find a valid baseurl for repo: base for me. I don't see why get_url would behave differently from unarchive in which case we likely have two options if the Common role won't work (Although why ansible's yum operation needs to use the same python interpreter as it does is ... grrr)

• Use wget or another utility to download before the unarchive
• Use a separate unarchive step for RHEL/Cent6 with e.g.:

  vars:
    ansible_python_interpreter: /usr/local/python2/bin/python

[Willsparker]

I think the latter, personally - seems cleaner and it's more obvious that it's done that way because CentOS6 is the outlier

[Willsparker]

I have a branch doing that at https://github.com/Willsparker/openjdk-infrastructure/tree/centos6_Python and testing on VPC at https://ci.adoptopenjdk.net/job/VagrantPlaybookCheck/1062/ :-)