[18.0.0-proposed][zuul]Use crc 2.39 that has ocp 4.16

[gibizer]

(cherry picked from commit dd61c93)

[SeanMooney]

yep we want to test on 4.16 for release

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: gibizer, SeanMooney

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [SeanMooney,gibizer]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[softwarefactory-project-zuul]

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/f000b5e8911c4a2bb4493d88107d683f

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 50m 59s
✔️ nova-operator-kuttl SUCCESS in 40m 38s
✔️ nova-operator-tempest-multinode SUCCESS in 2h 03m 04s
❌ nova-operator-tempest-multinode-ceph FAILURE in 1h 17m 27s

[mrkisaolamb]

recheck

[gibizer]

the ceph job failed with setting up some resources (probably for tempest). This is new.

2024-07-22 16:34:38.990468 | controller | TASK [os_net_setup : Create subnets _raw_params={{ lookup('ansible.builtin.template', _template_file) }}
2024-07-22 16:34:38.990482 | controller | ] ***
2024-07-22 16:34:38.990496 | controller | Monday 22 July 2024  16:34:30 -0400 (0:00:00.140)       0:49:24.786 ***********
2024-07-22 16:34:38.990519 | controller | fatal: [localhost]: FAILED! => changed=true
2024-07-22 16:34:39.107832 | controller |   cmd: |-
2024-07-22 16:34:39.107950 | controller |     set -euo pipefail
2024-07-22 16:34:39.107968 | controller |     oc exec -n openstack openstackclient -- openstack subnet create \
2024-07-22 16:34:39.107983 | controller |         --allocation-pool start=192.168.122.171,end=192.168.122.250 \
2024-07-22 16:34:39.107998 | controller |         --subnet-range 192.168.122.0/24 \
2024-07-22 16:34:39.108012 | controller |         --gateway 192.168.122.1 \
2024-07-22 16:34:39.108036 | controller |         --network public \
2024-07-22 16:34:39.108051 | controller |         public_subnet
2024-07-22 16:34:39.108065 | controller |   delta: '0:00:08.682889'
2024-07-22 16:34:39.108079 | controller |   end: '2024-07-22 16:34:38.944875'
2024-07-22 16:34:39.108094 | controller |   msg: non-zero return code
2024-07-22 16:34:39.108108 | controller |   rc: 1
2024-07-22 16:34:39.108122 | controller |   start: '2024-07-22 16:34:30.261986'
2024-07-22 16:34:39.108136 | controller |   stderr: |-
2024-07-22 16:34:39.108151 | controller |     BadRequestException: 400: Client Error for url: https://neutron-public-openstack.apps-crc.testing/v2.0/subnets, Invalid input for operation: Requested subnet with cidr: 192.168.122.0/24 for network: 9e744fc8-4d95-4dfd-a63b-a8bc90793a1a overlaps with another subnet.
2024-07-22 16:34:39.108165 | controller |     command terminated with exit code 1
2024-07-22 16:34:39.108215 | controller |   stderr_lines: <omitted>
2024-07-22 16:34:39.108229 | controller |   stdout: ''
2024-07-22 16:34:39.108243 | controller |   stdout_lines: <omitted>

[softwarefactory-project-zuul]

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/e9dd541203424dbbb88d43db1b1746af

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 41m 39s
✔️ nova-operator-kuttl SUCCESS in 37m 55s
❌ nova-operator-tempest-multinode FAILURE in 1h 11m 06s
✔️ nova-operator-tempest-multinode-ceph SUCCESS in 2h 23m 57s

[gibizer]

recheck now the local storage job failed at net setup

2024-07-23 04:30:01.001442 | controller | TASK [os_net_setup : Create networks _raw_params={{ lookup('ansible.builtin.template', _template_file) }}
2024-07-23 04:30:01.001453 | controller | ] ***
2024-07-23 04:30:01.001482 | controller | Tuesday 23 July 2024  04:29:57 -0400 (0:00:00.095)       0:41:28.182 **********
2024-07-23 04:30:01.001498 | controller | fatal: [localhost]: FAILED! => changed=true
2024-07-23 04:30:01.115717 | controller |   cmd: |-
2024-07-23 04:30:01.115797 | controller |     set -euo pipefail
2024-07-23 04:30:01.115806 | controller |     oc exec -n openstack openstackclient -- openstack network create \
2024-07-23 04:30:01.115813 | controller |         --external \
2024-07-23 04:30:01.115819 | controller |         --default \
2024-07-23 04:30:01.115826 | controller |         --provider-network-type flat \
2024-07-23 04:30:01.115832 | controller |         --provider-physical-network datacentre \
2024-07-23 04:30:01.115839 | controller |         --no-share \
2024-07-23 04:30:01.115845 | controller |         public
2024-07-23 04:30:01.115851 | controller |   delta: '0:00:03.299831'
2024-07-23 04:30:01.115858 | controller |   end: '2024-07-23 04:30:00.981075'
2024-07-23 04:30:01.115864 | controller |   msg: non-zero return code
2024-07-23 04:30:01.115870 | controller |   rc: 1
2024-07-23 04:30:01.115877 | controller |   start: '2024-07-23 04:29:57.681244'
2024-07-23 04:30:01.115883 | controller |   stderr: |-
2024-07-23 04:30:01.115889 | controller |     Error while executing command: HttpException: 502, The server returned an invalid or incomplete response.: 502 Bad Gateway
2024-07-23 04:30:01.115896 | controller |     command terminated with exit code 1
2024-07-23 04:30:01.115902 | controller |   stderr_lines: <omitted>
2024-07-23 04:30:01.115908 | controller |   stdout: ''
2024-07-23 04:30:01.115914 | controller |   stdout_lines: <omitted>
2024-07-23 04:30:01.115921 | controller |
2024-07-23 04:30:01.115927 | controller | NO MORE HOSTS LEFT ******************