Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug 1862114: bump RHCOS images for CVE-2020-10713 #3987

Merged
merged 1 commit into from
Aug 11, 2020

Conversation

miabbott
Copy link
Member

The mitigation path for OCP customers is to reprovision nodes, so we
should bump the boot images on the installer as well.

@openshift-ci-robot openshift-ci-robot added bugzilla/severity-medium Referenced Bugzilla bug's severity is medium for the branch this PR is targeting. bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. labels Jul 30, 2020
@openshift-ci-robot
Copy link
Contributor

@miabbott: This pull request references Bugzilla bug 1862114, which is invalid:

  • expected dependent Bugzilla bug 1862111 to be in one of the following states: VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), but it is POST instead
  • expected dependent Bugzilla bug 1862112 to be in one of the following states: VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), but it is NEW instead
  • expected dependent Bugzilla bug 1862113 to be in one of the following states: VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), but it is NEW instead
  • expected dependent Bugzilla bug 1862111 to target a release in 4.4.0, 4.4.z, but it targets "4.6.0" instead
  • expected dependent Bugzilla bug 1862112 to target a release in 4.4.0, 4.4.z, but it targets "4.5.z" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

Bug 1862114: bump RHCOS images for CVE-2020-10713

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@miabbott
Copy link
Member Author

$ ./differ.py -fe api.ci -fr rhcos-4.3/43.81.202003111353.0 -se api.ci -sr rhcos-4.3/43.82.202007291953.0
{
    "sources": {
        "rhcos-4.3/43.81.202003111353.0": "https://releases-art-rhcos.svc.ci.openshift.org/art/storage/releases/rhcos-4.3/43.81.202003111353.0/x86_64/commitmeta.json",
        "rhcos-4.3/43.82.202007291953.0": "https://releases-art-rhcos.svc.ci.openshift.org/art/storage/releases/rhcos-4.3/43.82.202007291953.0/x86_64/commitmeta.json"
    },
    "diff": {
        "NetworkManager": {
            "rhcos-4.3/43.81.202003111353.0": "NetworkManager-1.20.0-5.el8_1.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "NetworkManager-1.22.8-5.el8_2.x86_64"
        },
        "NetworkManager-libnm": {
            "rhcos-4.3/43.81.202003111353.0": "NetworkManager-libnm-1.20.0-5.el8_1.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "NetworkManager-libnm-1.22.8-5.el8_2.x86_64"
        },
        "NetworkManager-ovs": {
            "rhcos-4.3/43.81.202003111353.0": "NetworkManager-ovs-1.20.0-5.el8_1.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "NetworkManager-ovs-1.22.8-5.el8_2.x86_64"
        },
        "adcli": {
            "rhcos-4.3/43.81.202003111353.0": "adcli-0.8.2-3.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "adcli-0.8.2-5.el8.x86_64"
        },
        "afterburn": {
            "rhcos-4.3/43.81.202003111353.0": "afterburn-4.1.1-3.rhaos4.2.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "afterburn-4.3.3-1.rhaos4.3.el8.x86_64"
        },
        "audit": {
            "rhcos-4.3/43.81.202003111353.0": "audit-3.0-0.13.20190507gitf58ec40.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "audit-3.0-0.17.20191104git1c2f876.el8.x86_64"
        },
        "audit-libs": {
            "rhcos-4.3/43.81.202003111353.0": "audit-libs-3.0-0.13.20190507gitf58ec40.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "audit-libs-3.0-0.17.20191104git1c2f876.el8.x86_64"
        },
        "bind-export-libs": {
            "rhcos-4.3/43.81.202003111353.0": "bind-export-libs-9.11.4-26.P2.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "bind-export-libs-9.11.13-5.el8_2.x86_64"
        },
        "bind-libs": {
            "rhcos-4.3/43.81.202003111353.0": "bind-libs-9.11.4-26.P2.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "bind-libs-9.11.13-5.el8_2.x86_64"
        },
        "bind-libs-lite": {
            "rhcos-4.3/43.81.202003111353.0": "bind-libs-lite-9.11.4-26.P2.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "bind-libs-lite-9.11.13-5.el8_2.x86_64"
        },
        "bind-license": {
            "rhcos-4.3/43.81.202003111353.0": "bind-license-9.11.4-26.P2.el8.noarch",
            "rhcos-4.3/43.82.202007291953.0": "bind-license-9.11.13-5.el8_2.noarch"
        },
        "bind-utils": {
            "rhcos-4.3/43.81.202003111353.0": "bind-utils-9.11.4-26.P2.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "bind-utils-9.11.13-5.el8_2.x86_64"
        },
        "bubblewrap": {
            "rhcos-4.3/43.81.202003111353.0": "bubblewrap-0.3.0-1.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "bubblewrap-0.4.0-1.el8.x86_64"
        },
        "cifs-utils": {
            "rhcos-4.3/43.81.202003111353.0": "cifs-utils-6.8-2.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "cifs-utils-6.8-3.el8.x86_64"
        },
        "clevis": {
            "rhcos-4.3/43.81.202003111353.0": "clevis-11-2.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "clevis-11-9.el8.x86_64"
        },
        "clevis-dracut": {
            "rhcos-4.3/43.81.202003111353.0": "clevis-dracut-11-2.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "clevis-dracut-11-9.el8.x86_64"
        },
        "clevis-luks": {
            "rhcos-4.3/43.81.202003111353.0": "clevis-luks-11-2.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "clevis-luks-11-9.el8.x86_64"
        },
        "clevis-systemd": {
            "rhcos-4.3/43.81.202003111353.0": "clevis-systemd-11-2.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "clevis-systemd-11-9.el8.x86_64"
        },
        "conmon": {
            "rhcos-4.3/43.81.202003111353.0": "conmon-2.0.9-1.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "conmon-2.0.17-1.rhaos4.3.el8.x86_64"
        },
        "console-login-helper-messages": {
            "rhcos-4.3/43.81.202003111353.0": "console-login-helper-messages-0.16-1.rhaos4.2.el8.noarch",
            "rhcos-4.3/43.82.202007291953.0": "console-login-helper-messages-0.18-1.rhaos4.3.el8.noarch"
        },
        "console-login-helper-messages-issuegen": {
            "rhcos-4.3/43.81.202003111353.0": "console-login-helper-messages-issuegen-0.16-1.rhaos4.2.el8.noarch",
            "rhcos-4.3/43.82.202007291953.0": "console-login-helper-messages-issuegen-0.18-1.rhaos4.3.el8.noarch"
        },
        "console-login-helper-messages-profile": {
            "rhcos-4.3/43.81.202003111353.0": "console-login-helper-messages-profile-0.16-1.rhaos4.2.el8.noarch",
            "rhcos-4.3/43.82.202007291953.0": "console-login-helper-messages-profile-0.18-1.rhaos4.3.el8.noarch"
        },
        "container-selinux": {
            "rhcos-4.3/43.81.202003111353.0": "container-selinux-2.124.0-1.module+el8.1.1+5259+bcdd613a.noarch",
            "rhcos-4.3/43.82.202007291953.0": "container-selinux-2.135.0-1.module+el8.2.1+6849+893e4f4a.noarch"
        },
        "containernetworking-plugins": {
            "rhcos-4.3/43.81.202003111353.0": "containernetworking-plugins-0.8.3-4.module+el8.1.1+5259+bcdd613a.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "containernetworking-plugins-0.8.6-1.rhaos4.3.el8.x86_64"
        },
        "containers-common": {
            "rhcos-4.3/43.81.202003111353.0": "containers-common-0.1.40-8.module+el8.1.1+5351+506397b0.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "containers-common-1.0.0-1.module+el8.2.1+6676+604e1b26.x86_64"
        },
        "coreutils": {
            "rhcos-4.3/43.81.202003111353.0": "coreutils-8.30-6.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "coreutils-8.30-7.el8_2.1.x86_64"
        },
        "coreutils-common": {
            "rhcos-4.3/43.81.202003111353.0": "coreutils-common-8.30-6.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "coreutils-common-8.30-7.el8_2.1.x86_64"
        },
        "cri-o": {
            "rhcos-4.3/43.81.202003111353.0": "cri-o-1.16.3-28.dev.rhaos4.3.git9aad8e4.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "cri-o-1.16.6-18.rhaos4.3.git538d861.el8.x86_64"
        },
        "criu": {
            "rhcos-4.3/43.81.202003111353.0": "criu-3.12-9.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "criu-3.14-2.module+el8.2.1+6750+e53a300c.x86_64"
        },
        "crypto-policies": {
            "rhcos-4.3/43.81.202003111353.0": "crypto-policies-20190807-1.git9b1477b.el8.noarch",
            "rhcos-4.3/43.82.202007291953.0": "crypto-policies-20191128-2.git23e1bf1.el8.noarch"
        },
        "cryptsetup": {
            "rhcos-4.3/43.81.202003111353.0": "cryptsetup-2.2.0-2.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "cryptsetup-2.2.2-1.el8.x86_64"
        },
        "cryptsetup-libs": {
            "rhcos-4.3/43.81.202003111353.0": "cryptsetup-libs-2.2.0-2.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "cryptsetup-libs-2.2.2-1.el8.x86_64"
        },
        "cryptsetup-reencrypt": {
            "rhcos-4.3/43.81.202003111353.0": "cryptsetup-reencrypt-2.2.0-2.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "cryptsetup-reencrypt-2.2.2-1.el8.x86_64"
        },
        "cups-libs": {
            "rhcos-4.3/43.81.202003111353.0": "cups-libs-2.2.6-28.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "cups-libs-2.2.6-33.el8.x86_64"
        },
        "curl": {
            "rhcos-4.3/43.81.202003111353.0": "curl-7.61.1-11.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "curl-7.61.1-12.el8.x86_64"
        },
        "dbus": {
            "rhcos-4.3/43.81.202003111353.0": "dbus-1.12.8-9.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "dbus-1.12.8-10.el8_2.x86_64"
        },
        "dbus-common": {
            "rhcos-4.3/43.81.202003111353.0": "dbus-common-1.12.8-9.el8.noarch",
            "rhcos-4.3/43.82.202007291953.0": "dbus-common-1.12.8-10.el8_2.noarch"
        },
        "dbus-daemon": {
            "rhcos-4.3/43.81.202003111353.0": "dbus-daemon-1.12.8-9.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "dbus-daemon-1.12.8-10.el8_2.x86_64"
        },
        "dbus-libs": {
            "rhcos-4.3/43.81.202003111353.0": "dbus-libs-1.12.8-9.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "dbus-libs-1.12.8-10.el8_2.x86_64"
        },
        "dbus-tools": {
            "rhcos-4.3/43.81.202003111353.0": "dbus-tools-1.12.8-9.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "dbus-tools-1.12.8-10.el8_2.x86_64"
        },
        "device-mapper": {
            "rhcos-4.3/43.81.202003111353.0": "device-mapper-1.02.163-5.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "device-mapper-1.02.169-3.el8.x86_64"
        },
        "device-mapper-event": {
            "rhcos-4.3/43.81.202003111353.0": "device-mapper-event-1.02.163-5.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "device-mapper-event-1.02.169-3.el8.x86_64"
        },
        "device-mapper-event-libs": {
            "rhcos-4.3/43.81.202003111353.0": "device-mapper-event-libs-1.02.163-5.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "device-mapper-event-libs-1.02.169-3.el8.x86_64"
        },
        "device-mapper-libs": {
            "rhcos-4.3/43.81.202003111353.0": "device-mapper-libs-1.02.163-5.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "device-mapper-libs-1.02.169-3.el8.x86_64"
        },
        "device-mapper-multipath": {
            "rhcos-4.3/43.81.202003111353.0": "device-mapper-multipath-0.8.0-5.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "device-mapper-multipath-0.8.3-3.el8.x86_64"
        },
        "device-mapper-multipath-libs": {
            "rhcos-4.3/43.81.202003111353.0": "device-mapper-multipath-libs-0.8.0-5.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "device-mapper-multipath-libs-0.8.3-3.el8.x86_64"
        },
        "device-mapper-persistent-data": {
            "rhcos-4.3/43.81.202003111353.0": "device-mapper-persistent-data-0.8.5-2.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "device-mapper-persistent-data-0.8.5-3.el8.x86_64"
        },
        "dhcp-client": {
            "rhcos-4.3/43.81.202003111353.0": "dhcp-client-4.3.6-34.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "dhcp-client-4.3.6-40.el8.x86_64"
        },
        "dhcp-common": {
            "rhcos-4.3/43.81.202003111353.0": "dhcp-common-4.3.6-34.el8.noarch",
            "rhcos-4.3/43.82.202007291953.0": "dhcp-common-4.3.6-40.el8.noarch"
        },
        "dhcp-libs": {
            "rhcos-4.3/43.81.202003111353.0": "dhcp-libs-4.3.6-34.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "dhcp-libs-4.3.6-40.el8.x86_64"
        },
        "diffutils": {
            "rhcos-4.3/43.81.202003111353.0": "diffutils-3.6-5.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "diffutils-3.6-6.el8.x86_64"
        },
        "dnsmasq": {
            "rhcos-4.3/43.81.202003111353.0": "dnsmasq-2.79-6.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "dnsmasq-2.79-11.el8_2.1.x86_64"
        },
        "e2fsprogs": {
            "rhcos-4.3/43.81.202003111353.0": "e2fsprogs-1.44.6-3.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "e2fsprogs-1.45.4-3.el8.x86_64"
        },
        "e2fsprogs-libs": {
            "rhcos-4.3/43.81.202003111353.0": "e2fsprogs-libs-1.44.6-3.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "e2fsprogs-libs-1.45.4-3.el8.x86_64"
        },
        "elfutils-default-yama-scope": {
            "rhcos-4.3/43.81.202003111353.0": "elfutils-default-yama-scope-0.176-5.el8.noarch",
            "rhcos-4.3/43.82.202007291953.0": "elfutils-default-yama-scope-0.178-7.el8.noarch"
        },
        "elfutils-libelf": {
            "rhcos-4.3/43.81.202003111353.0": "elfutils-libelf-0.176-5.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "elfutils-libelf-0.178-7.el8.x86_64"
        },
        "elfutils-libs": {
            "rhcos-4.3/43.81.202003111353.0": "elfutils-libs-0.176-5.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "elfutils-libs-0.178-7.el8.x86_64"
        },
        "file": {
            "rhcos-4.3/43.81.202003111353.0": "file-5.33-8.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "file-5.33-13.el8.x86_64"
        },
        "file-libs": {
            "rhcos-4.3/43.81.202003111353.0": "file-libs-5.33-8.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "file-libs-5.33-13.el8.x86_64"
        },
        "firewalld-filesystem": {
            "rhcos-4.3/43.81.202003111353.0": "firewalld-filesystem-0.7.0-5.el8.noarch",
            "rhcos-4.3/43.82.202007291953.0": "firewalld-filesystem-0.8.0-4.el8.noarch"
        },
        "fuse-overlayfs": {
            "rhcos-4.3/43.81.202003111353.0": "fuse-overlayfs-0.7.2-1.module+el8.1.1+5259+bcdd613a.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "fuse-overlayfs-1.0.0-2.module+el8.2.1+6465+1a51e8b6.x86_64"
        },
        "git-core": {
            "rhcos-4.3/43.81.202003111353.0": "git-core-2.18.2-1.el8_1.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "git-core-2.18.4-2.el8_2.x86_64"
        },
        "glib2": {
            "rhcos-4.3/43.81.202003111353.0": "glib2-2.56.4-7.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "glib2-2.56.4-8.el8.x86_64"
        },
        "glibc": {
            "rhcos-4.3/43.81.202003111353.0": "glibc-2.28-72.el8_1.1.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "glibc-2.28-101.el8.x86_64"
        },
        "glibc-all-langpacks": {
            "rhcos-4.3/43.81.202003111353.0": "glibc-all-langpacks-2.28-72.el8_1.1.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "glibc-all-langpacks-2.28-101.el8.x86_64"
        },
        "glibc-common": {
            "rhcos-4.3/43.81.202003111353.0": "glibc-common-2.28-72.el8_1.1.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "glibc-common-2.28-101.el8.x86_64"
        },
        "glusterfs": {
            "rhcos-4.3/43.81.202003111353.0": "glusterfs-6.0-20.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "glusterfs-6.0-37.el8.x86_64"
        },
        "glusterfs-client-xlators": {
            "rhcos-4.3/43.81.202003111353.0": "glusterfs-client-xlators-6.0-20.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "glusterfs-client-xlators-6.0-37.el8.x86_64"
        },
        "glusterfs-fuse": {
            "rhcos-4.3/43.81.202003111353.0": "glusterfs-fuse-6.0-20.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "glusterfs-fuse-6.0-37.el8.x86_64"
        },
        "glusterfs-libs": {
            "rhcos-4.3/43.81.202003111353.0": "glusterfs-libs-6.0-20.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "glusterfs-libs-6.0-37.el8.x86_64"
        },
        "gnutls": {
            "rhcos-4.3/43.81.202003111353.0": "gnutls-3.6.8-8.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "gnutls-3.6.8-11.el8_2.x86_64"
        },
        "grub2-common": {
            "rhcos-4.3/43.81.202003111353.0": "grub2-common-2.02-78.el8_1.1.noarch",
            "rhcos-4.3/43.82.202007291953.0": "grub2-common-2.02-87.el8_2.noarch"
        },
        "grub2-efi-x64": {
            "rhcos-4.3/43.81.202003111353.0": "grub2-efi-x64-2.02-78.el8_1.1.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "grub2-efi-x64-2.02-87.el8_2.x86_64"
        },
        "grub2-pc": {
            "rhcos-4.3/43.81.202003111353.0": "grub2-pc-2.02-78.el8_1.1.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "grub2-pc-2.02-87.el8_2.x86_64"
        },
        "grub2-pc-modules": {
            "rhcos-4.3/43.81.202003111353.0": "grub2-pc-modules-2.02-78.el8_1.1.noarch",
            "rhcos-4.3/43.82.202007291953.0": "grub2-pc-modules-2.02-87.el8_2.noarch"
        },
        "grub2-tools": {
            "rhcos-4.3/43.81.202003111353.0": "grub2-tools-2.02-78.el8_1.1.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "grub2-tools-2.02-87.el8_2.x86_64"
        },
        "grub2-tools-extra": {
            "rhcos-4.3/43.81.202003111353.0": "grub2-tools-extra-2.02-78.el8_1.1.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "grub2-tools-extra-2.02-87.el8_2.x86_64"
        },
        "grub2-tools-minimal": {
            "rhcos-4.3/43.81.202003111353.0": "grub2-tools-minimal-2.02-78.el8_1.1.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "grub2-tools-minimal-2.02-87.el8_2.x86_64"
        },
        "grubby": {
            "rhcos-4.3/43.81.202003111353.0": "grubby-8.40-37.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "grubby-8.40-38.el8.x86_64"
        },
        "gssproxy": {
            "rhcos-4.3/43.81.202003111353.0": "gssproxy-0.8.0-14.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "gssproxy-0.8.0-15.el8.x86_64"
        },
        "hwdata": {
            "rhcos-4.3/43.81.202003111353.0": "hwdata-0.314-8.2.el8_1.noarch",
            "rhcos-4.3/43.82.202007291953.0": "hwdata-0.314-8.4.el8.noarch"
        },
        "ignition": {
            "rhcos-4.3/43.81.202003111353.0": "ignition-0.34.0-2.rhaos4.3.git92f874c.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "ignition-0.34.0-4.rhaos4.3.git92f874c.el8.x86_64"
        },
        "info": {
            "rhcos-4.3/43.81.202003111353.0": "info-6.5-4.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "info-6.5-6.el8.x86_64"
        },
        "initscripts": {
            "rhcos-4.3/43.81.202003111353.0": "initscripts-10.00.4-1.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "initscripts-10.00.6-1.el8_2.1.x86_64"
        },
        "ipcalc": {
            "rhcos-4.3/43.81.202003111353.0": "ipcalc-0.2.4-3.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "ipcalc-0.2.4-4.el8.x86_64"
        },
        "iproute": {
            "rhcos-4.3/43.81.202003111353.0": "iproute-4.18.0-15.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "iproute-5.3.0-1.el8.x86_64"
        },
        "iptables": {
            "rhcos-4.3/43.81.202003111353.0": "iptables-1.8.4-9.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "iptables-1.8.4-10.el8_2.1.x86_64"
        },
        "iptables-libs": {
            "rhcos-4.3/43.81.202003111353.0": "iptables-libs-1.8.4-9.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "iptables-libs-1.8.4-10.el8_2.1.x86_64"
        },
        "iscsi-initiator-utils": {
            "rhcos-4.3/43.81.202003111353.0": "iscsi-initiator-utils-6.2.0.877-1.gitf71581b.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "iscsi-initiator-utils-6.2.0.878-4.gitd791ce0.el8.x86_64"
        },
        "iscsi-initiator-utils-iscsiuio": {
            "rhcos-4.3/43.81.202003111353.0": "iscsi-initiator-utils-iscsiuio-6.2.0.877-1.gitf71581b.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "iscsi-initiator-utils-iscsiuio-6.2.0.878-4.gitd791ce0.el8.x86_64"
        },
        "kernel": {
            "rhcos-4.3/43.81.202003111353.0": "kernel-4.18.0-147.5.1.el8_1.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "kernel-4.18.0-193.14.3.el8_2.x86_64"
        },
        "kernel-core": {
            "rhcos-4.3/43.81.202003111353.0": "kernel-core-4.18.0-147.5.1.el8_1.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "kernel-core-4.18.0-193.14.3.el8_2.x86_64"
        },
        "kernel-modules": {
            "rhcos-4.3/43.81.202003111353.0": "kernel-modules-4.18.0-147.5.1.el8_1.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "kernel-modules-4.18.0-193.14.3.el8_2.x86_64"
        },
        "kernel-modules-extra": {
            "rhcos-4.3/43.81.202003111353.0": "kernel-modules-extra-4.18.0-147.5.1.el8_1.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "kernel-modules-extra-4.18.0-193.14.3.el8_2.x86_64"
        },
        "kmod": {
            "rhcos-4.3/43.81.202003111353.0": "kmod-25-13.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "kmod-25-16.el8.x86_64"
        },
        "kmod-libs": {
            "rhcos-4.3/43.81.202003111353.0": "kmod-libs-25-13.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "kmod-libs-25-16.el8.x86_64"
        },
        "kpartx": {
            "rhcos-4.3/43.81.202003111353.0": "kpartx-0.8.0-5.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "kpartx-0.8.3-3.el8.x86_64"
        },
        "krb5-libs": {
            "rhcos-4.3/43.81.202003111353.0": "krb5-libs-1.17-9.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "krb5-libs-1.17-18.el8.x86_64"
        },
        "libblkid": {
            "rhcos-4.3/43.81.202003111353.0": "libblkid-2.32.1-17.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libblkid-2.32.1-22.el8.x86_64"
        },
        "libcap": {
            "rhcos-4.3/43.81.202003111353.0": "libcap-2.26-1.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libcap-2.26-3.el8.x86_64"
        },
        "libcap-ng": {
            "rhcos-4.3/43.81.202003111353.0": "libcap-ng-0.7.9-4.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libcap-ng-0.7.9-5.el8.x86_64"
        },
        "libcom_err": {
            "rhcos-4.3/43.81.202003111353.0": "libcom_err-1.44.6-3.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libcom_err-1.45.4-3.el8.x86_64"
        },
        "libcurl": {
            "rhcos-4.3/43.81.202003111353.0": "libcurl-7.61.1-11.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libcurl-7.61.1-12.el8.x86_64"
        },
        "libdnet": {
            "rhcos-4.3/43.81.202003111353.0": "libdnet-1.12-26.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "Not present"
        },
        "libdrm": {
            "rhcos-4.3/43.81.202003111353.0": "libdrm-2.4.98-2.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libdrm-2.4.100-1.el8.x86_64"
        },
        "libfdisk": {
            "rhcos-4.3/43.81.202003111353.0": "libfdisk-2.32.1-17.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libfdisk-2.32.1-22.el8.x86_64"
        },
        "libgcc": {
            "rhcos-4.3/43.81.202003111353.0": "libgcc-8.3.1-4.5.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libgcc-8.3.1-5.el8.x86_64"
        },
        "libgomp": {
            "rhcos-4.3/43.81.202003111353.0": "libgomp-8.3.1-4.5.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libgomp-8.3.1-5.el8.x86_64"
        },
        "libicu": {
            "rhcos-4.3/43.81.202003111353.0": "libicu-60.3-1.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "Not present"
        },
        "libipa_hbac": {
            "rhcos-4.3/43.81.202003111353.0": "libipa_hbac-2.2.0-19.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libipa_hbac-2.2.3-20.el8.x86_64"
        },
        "libldb": {
            "rhcos-4.3/43.81.202003111353.0": "libldb-1.5.4-2.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libldb-2.0.7-3.el8.x86_64"
        },
        "libluksmeta": {
            "rhcos-4.3/43.81.202003111353.0": "libluksmeta-9-2.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libluksmeta-9-4.el8.x86_64"
        },
        "libmaxminddb": {
            "rhcos-4.3/43.81.202003111353.0": "libmaxminddb-1.2.0-6.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libmaxminddb-1.2.0-7.el8.x86_64"
        },
        "libmodulemd1": {
            "rhcos-4.3/43.81.202003111353.0": "libmodulemd1-1.8.11-4.el8_1.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libmodulemd1-1.8.16-0.2.8.2.1.x86_64"
        },
        "libmount": {
            "rhcos-4.3/43.81.202003111353.0": "libmount-2.32.1-17.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libmount-2.32.1-22.el8.x86_64"
        },
        "libmspack": {
            "rhcos-4.3/43.81.202003111353.0": "libmspack-0.7-0.1.alpha.el8.3.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libmspack-0.7-0.3.alpha.el8.4.x86_64"
        },
        "libndp": {
            "rhcos-4.3/43.81.202003111353.0": "libndp-1.7-1.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libndp-1.7-3.el8.x86_64"
        },
        "libnfsidmap": {
            "rhcos-4.3/43.81.202003111353.0": "libnfsidmap-2.3.3-26.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libnfsidmap-2.3.3-31.el8.x86_64"
        },
        "libnghttp2": {
            "rhcos-4.3/43.81.202003111353.0": "libnghttp2-1.33.0-1.el8_0.1.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libnghttp2-1.33.0-3.el8_2.1.x86_64"
        },
        "libnl3": {
            "rhcos-4.3/43.81.202003111353.0": "libnl3-3.4.0-5.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libnl3-3.5.0-1.el8.x86_64"
        },
        "librepo": {
            "rhcos-4.3/43.81.202003111353.0": "librepo-1.10.3-3.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "librepo-1.11.0-2.el8.x86_64"
        },
        "libreport-filesystem": {
            "rhcos-4.3/43.81.202003111353.0": "libreport-filesystem-2.9.5-9.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libreport-filesystem-2.9.5-10.el8.x86_64"
        },
        "libselinux": {
            "rhcos-4.3/43.81.202003111353.0": "libselinux-2.9-2.1.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libselinux-2.9-3.el8.x86_64"
        },
        "libselinux-utils": {
            "rhcos-4.3/43.81.202003111353.0": "libselinux-utils-2.9-2.1.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libselinux-utils-2.9-3.el8.x86_64"
        },
        "libsemanage": {
            "rhcos-4.3/43.81.202003111353.0": "libsemanage-2.9-1.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libsemanage-2.9-2.el8.x86_64"
        },
        "libsmartcols": {
            "rhcos-4.3/43.81.202003111353.0": "libsmartcols-2.32.1-17.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libsmartcols-2.32.1-22.el8.x86_64"
        },
        "libsmbclient": {
            "rhcos-4.3/43.81.202003111353.0": "libsmbclient-4.10.4-101.el8_1.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libsmbclient-4.11.2-13.el8.x86_64"
        },
        "libsolv": {
            "rhcos-4.3/43.81.202003111353.0": "libsolv-0.7.4-3.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libsolv-0.7.7-1.el8.x86_64"
        },
        "libss": {
            "rhcos-4.3/43.81.202003111353.0": "libss-1.44.6-3.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libss-1.45.4-3.el8.x86_64"
        },
        "libsss_autofs": {
            "rhcos-4.3/43.81.202003111353.0": "libsss_autofs-2.2.0-19.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libsss_autofs-2.2.3-20.el8.x86_64"
        },
        "libsss_certmap": {
            "rhcos-4.3/43.81.202003111353.0": "libsss_certmap-2.2.0-19.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libsss_certmap-2.2.3-20.el8.x86_64"
        },
        "libsss_idmap": {
            "rhcos-4.3/43.81.202003111353.0": "libsss_idmap-2.2.0-19.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libsss_idmap-2.2.3-20.el8.x86_64"
        },
        "libsss_nss_idmap": {
            "rhcos-4.3/43.81.202003111353.0": "libsss_nss_idmap-2.2.0-19.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libsss_nss_idmap-2.2.3-20.el8.x86_64"
        },
        "libsss_sudo": {
            "rhcos-4.3/43.81.202003111353.0": "libsss_sudo-2.2.0-19.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libsss_sudo-2.2.3-20.el8.x86_64"
        },
        "libstdc++": {
            "rhcos-4.3/43.81.202003111353.0": "libstdc++-8.3.1-4.5.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libstdc++-8.3.1-5.el8.x86_64"
        },
        "libtalloc": {
            "rhcos-4.3/43.81.202003111353.0": "libtalloc-2.1.16-3.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libtalloc-2.2.0-7.el8.x86_64"
        },
        "libtdb": {
            "rhcos-4.3/43.81.202003111353.0": "libtdb-1.3.18-2.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libtdb-1.4.2-2.el8.x86_64"
        },
        "libtevent": {
            "rhcos-4.3/43.81.202003111353.0": "libtevent-0.9.39-2.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libtevent-0.10.0-2.el8.x86_64"
        },
        "libuuid": {
            "rhcos-4.3/43.81.202003111353.0": "libuuid-2.32.1-17.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libuuid-2.32.1-22.el8.x86_64"
        },
        "libwbclient": {
            "rhcos-4.3/43.81.202003111353.0": "libwbclient-4.10.4-101.el8_1.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libwbclient-4.11.2-13.el8.x86_64"
        },
        "libxml2": {
            "rhcos-4.3/43.81.202003111353.0": "libxml2-2.9.7-5.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libxml2-2.9.7-7.el8.x86_64"
        },
        "libxslt": {
            "rhcos-4.3/43.81.202003111353.0": "libxslt-1.1.32-3.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "libxslt-1.1.32-4.el8.x86_64"
        },
        "linux-firmware": {
            "rhcos-4.3/43.81.202003111353.0": "linux-firmware-20190516-94.git711d3297.el8.noarch",
            "rhcos-4.3/43.82.202007291953.0": "linux-firmware-20191202-97.gite8a0f4c9.el8.noarch"
        },
        "luksmeta": {
            "rhcos-4.3/43.81.202003111353.0": "luksmeta-9-2.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "luksmeta-9-4.el8.x86_64"
        },
        "lvm2": {
            "rhcos-4.3/43.81.202003111353.0": "lvm2-2.03.05-5.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "lvm2-2.03.08-3.el8.x86_64"
        },
        "lvm2-libs": {
            "rhcos-4.3/43.81.202003111353.0": "lvm2-libs-2.03.05-5.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "lvm2-libs-2.03.08-3.el8.x86_64"
        },
        "machine-config-daemon": {
            "rhcos-4.3/43.81.202003111353.0": "machine-config-daemon-4.3.6-202003110901.git.0.b4999a5.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "machine-config-daemon-4.3.30-202007230708.p0.git.2188.9f34d7f.el8.x86_64"
        },
        "make": {
            "rhcos-4.3/43.81.202003111353.0": "make-4.2.1-9.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "make-4.2.1-10.el8.x86_64"
        },
        "mdadm": {
            "rhcos-4.3/43.81.202003111353.0": "mdadm-4.1-9.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "mdadm-4.1-13.el8.x86_64"
        },
        "microcode_ctl": {
            "rhcos-4.3/43.81.202003111353.0": "microcode_ctl-20190618-1.20191115.3.el8_1.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "microcode_ctl-20191115-4.20200609.1.el8_2.x86_64"
        },
        "mozjs60": {
            "rhcos-4.3/43.81.202003111353.0": "mozjs60-60.9.0-3.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "mozjs60-60.9.0-4.el8.x86_64"
        },
        "nfs-utils": {
            "rhcos-4.3/43.81.202003111353.0": "nfs-utils-2.3.3-26.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "nfs-utils-2.3.3-31.el8.x86_64"
        },
        "nftables": {
            "rhcos-4.3/43.81.202003111353.0": "nftables-0.9.0-14.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "nftables-0.9.3-12.el8.x86_64"
        },
        "numactl-libs": {
            "rhcos-4.3/43.81.202003111353.0": "numactl-libs-2.0.12-7.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "numactl-libs-2.0.12-9.el8.x86_64"
        },
        "open-vm-tools": {
            "rhcos-4.3/43.81.202003111353.0": "open-vm-tools-10.3.10-3.el8_1.1.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "open-vm-tools-11.0.5-3.el8.x86_64"
        },
        "openshift-clients": {
            "rhcos-4.3/43.81.202003111353.0": "openshift-clients-4.3.6-202003110601.git.0.4b8c32c.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "openshift-clients-4.3.31-202007250052.p0.git.3329.59998b9.el8.x86_64"
        },
        "openshift-hyperkube": {
            "rhcos-4.3/43.81.202003111353.0": "openshift-hyperkube-4.3.6-202003090404.git.0.737076f.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "openshift-hyperkube-4.3.31-202007280738.p0.git.0.9884401.el8.x86_64"
        },
        "openssl": {
            "rhcos-4.3/43.81.202003111353.0": "openssl-1.1.1c-2.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "openssl-1.1.1c-15.el8.x86_64"
        },
        "openssl-libs": {
            "rhcos-4.3/43.81.202003111353.0": "openssl-libs-1.1.1c-2.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "openssl-libs-1.1.1c-15.el8.x86_64"
        },
        "openssl-pkcs11": {
            "rhcos-4.3/43.81.202003111353.0": "openssl-pkcs11-0.4.8-2.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "openssl-pkcs11-0.4.10-2.el8.x86_64"
        },
        "openvswitch-selinux-extra-policy": {
            "rhcos-4.3/43.81.202003111353.0": "openvswitch-selinux-extra-policy-1.0-22.el8fdp.noarch",
            "rhcos-4.3/43.82.202007291953.0": "openvswitch-selinux-extra-policy-1.0-23.el8fdp.noarch"
        },
        "openvswitch2.11": {
            "rhcos-4.3/43.81.202003111353.0": "openvswitch2.11-2.11.0-48.el8fdp.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "openvswitch2.11-2.11.0-56.20200327gita4efc59.el8fdp.x86_64"
        },
        "ostree": {
            "rhcos-4.3/43.81.202003111353.0": "ostree-2019.5-1.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "ostree-2019.6-2.el8.x86_64"
        },
        "ostree-grub2": {
            "rhcos-4.3/43.81.202003111353.0": "ostree-grub2-2019.5-1.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "ostree-grub2-2019.6-2.el8.x86_64"
        },
        "ostree-libs": {
            "rhcos-4.3/43.81.202003111353.0": "ostree-libs-2019.5-1.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "ostree-libs-2019.6-2.el8.x86_64"
        },
        "pam": {
            "rhcos-4.3/43.81.202003111353.0": "pam-1.3.1-4.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "pam-1.3.1-8.el8.x86_64"
        },
        "passwd": {
            "rhcos-4.3/43.81.202003111353.0": "passwd-0.80-2.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "passwd-0.80-3.el8.x86_64"
        },
        "platform-python": {
            "rhcos-4.3/43.81.202003111353.0": "platform-python-3.6.8-15.1.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "platform-python-3.6.8-23.el8.x86_64"
        },
        "platform-python-pip": {
            "rhcos-4.3/43.81.202003111353.0": "platform-python-pip-9.0.3-15.el8.noarch",
            "rhcos-4.3/43.82.202007291953.0": "platform-python-pip-9.0.3-16.el8.noarch"
        },
        "podman": {
            "rhcos-4.3/43.81.202003111353.0": "podman-1.6.4-7.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "podman-1.9.3-2.module+el8.2.1+6867+366c07d6.x86_64"
        },
        "podman-manpages": {
            "rhcos-4.3/43.81.202003111353.0": "podman-manpages-1.6.4-7.el8.noarch",
            "rhcos-4.3/43.82.202007291953.0": "Not present"
        },
        "policycoreutils": {
            "rhcos-4.3/43.81.202003111353.0": "policycoreutils-2.9-3.el8_1.1.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "policycoreutils-2.9-9.el8.x86_64"
        },
        "policycoreutils-python-utils": {
            "rhcos-4.3/43.81.202003111353.0": "policycoreutils-python-utils-2.9-3.el8_1.1.noarch",
            "rhcos-4.3/43.82.202007291953.0": "policycoreutils-python-utils-2.9-9.el8.noarch"
        },
        "polkit": {
            "rhcos-4.3/43.81.202003111353.0": "polkit-0.115-9.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "polkit-0.115-11.el8.x86_64"
        },
        "polkit-libs": {
            "rhcos-4.3/43.81.202003111353.0": "polkit-libs-0.115-9.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "polkit-libs-0.115-11.el8.x86_64"
        },
        "psmisc": {
            "rhcos-4.3/43.81.202003111353.0": "psmisc-23.1-3.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "psmisc-23.1-4.el8.x86_64"
        },
        "python3-audit": {
            "rhcos-4.3/43.81.202003111353.0": "python3-audit-3.0-0.13.20190507gitf58ec40.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "python3-audit-3.0-0.17.20191104git1c2f876.el8.x86_64"
        },
        "python3-bind": {
            "rhcos-4.3/43.81.202003111353.0": "python3-bind-9.11.4-26.P2.el8.noarch",
            "rhcos-4.3/43.82.202007291953.0": "python3-bind-9.11.13-5.el8_2.noarch"
        },
        "python3-libs": {
            "rhcos-4.3/43.81.202003111353.0": "python3-libs-3.6.8-15.1.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "python3-libs-3.6.8-23.el8.x86_64"
        },
        "python3-libselinux": {
            "rhcos-4.3/43.81.202003111353.0": "python3-libselinux-2.9-2.1.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "python3-libselinux-2.9-3.el8.x86_64"
        },
        "python3-libsemanage": {
            "rhcos-4.3/43.81.202003111353.0": "python3-libsemanage-2.9-1.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "python3-libsemanage-2.9-2.el8.x86_64"
        },
        "python3-pip-wheel": {
            "rhcos-4.3/43.81.202003111353.0": "python3-pip-wheel-9.0.3-15.el8.noarch",
            "rhcos-4.3/43.82.202007291953.0": "python3-pip-wheel-9.0.3-16.el8.noarch"
        },
        "python3-ply": {
            "rhcos-4.3/43.81.202003111353.0": "python3-ply-3.9-7.el8.noarch",
            "rhcos-4.3/43.82.202007291953.0": "python3-ply-3.9-8.el8.noarch"
        },
        "python3-policycoreutils": {
            "rhcos-4.3/43.81.202003111353.0": "python3-policycoreutils-2.9-3.el8_1.1.noarch",
            "rhcos-4.3/43.82.202007291953.0": "python3-policycoreutils-2.9-9.el8.noarch"
        },
        "python3-setools": {
            "rhcos-4.3/43.81.202003111353.0": "python3-setools-4.2.2-1.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "python3-setools-4.2.2-2.el8.x86_64"
        },
        "python3-sssdconfig": {
            "rhcos-4.3/43.81.202003111353.0": "python3-sssdconfig-2.2.0-19.el8.noarch",
            "rhcos-4.3/43.82.202007291953.0": "python3-sssdconfig-2.2.3-20.el8.noarch"
        },
        "rdma-core": {
            "rhcos-4.3/43.81.202003111353.0": "rdma-core-22.3-1.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "rdma-core-26.0-8.el8.x86_64"
        },
        "redhat-release-coreos": {
            "rhcos-4.3/43.81.202003111353.0": "redhat-release-coreos-43.80-4.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "redhat-release-coreos-43.82-6.el8.x86_64"
        },
        "rpcbind": {
            "rhcos-4.3/43.81.202003111353.0": "rpcbind-1.2.5-4.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "rpcbind-1.2.5-7.el8.x86_64"
        },
        "rpm": {
            "rhcos-4.3/43.81.202003111353.0": "rpm-4.14.2-25.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "rpm-4.14.2-37.el8.x86_64"
        },
        "rpm-libs": {
            "rhcos-4.3/43.81.202003111353.0": "rpm-libs-4.14.2-25.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "rpm-libs-4.14.2-37.el8.x86_64"
        },
        "rpm-plugin-selinux": {
            "rhcos-4.3/43.81.202003111353.0": "rpm-plugin-selinux-4.14.2-25.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "rpm-plugin-selinux-4.14.2-37.el8.x86_64"
        },
        "rsync": {
            "rhcos-4.3/43.81.202003111353.0": "rsync-3.1.3-6.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "rsync-3.1.3-7.el8.x86_64"
        },
        "runc": {
            "rhcos-4.3/43.81.202003111353.0": "runc-1.0.0-65.rc10.rhaos4.3.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "runc-1.0.0-67.rc10.rhaos4.3.el8.x86_64"
        },
        "samba-client-libs": {
            "rhcos-4.3/43.81.202003111353.0": "samba-client-libs-4.10.4-101.el8_1.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "samba-client-libs-4.11.2-13.el8.x86_64"
        },
        "samba-common": {
            "rhcos-4.3/43.81.202003111353.0": "samba-common-4.10.4-101.el8_1.noarch",
            "rhcos-4.3/43.82.202007291953.0": "samba-common-4.11.2-13.el8.noarch"
        },
        "samba-common-libs": {
            "rhcos-4.3/43.81.202003111353.0": "samba-common-libs-4.10.4-101.el8_1.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "samba-common-libs-4.11.2-13.el8.x86_64"
        },
        "selinux-policy": {
            "rhcos-4.3/43.81.202003111353.0": "selinux-policy-3.14.3-20.el8.noarch",
            "rhcos-4.3/43.82.202007291953.0": "selinux-policy-3.14.3-41.el8_2.5.noarch"
        },
        "selinux-policy-targeted": {
            "rhcos-4.3/43.81.202003111353.0": "selinux-policy-targeted-3.14.3-20.el8.noarch",
            "rhcos-4.3/43.82.202007291953.0": "selinux-policy-targeted-3.14.3-41.el8_2.5.noarch"
        },
        "setup": {
            "rhcos-4.3/43.81.202003111353.0": "setup-2.12.2-2.el8_1.1.noarch",
            "rhcos-4.3/43.82.202007291953.0": "setup-2.12.2-5.el8.noarch"
        },
        "sg3_utils": {
            "rhcos-4.3/43.81.202003111353.0": "sg3_utils-1.44-3.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "sg3_utils-1.44-5.el8.x86_64"
        },
        "sg3_utils-libs": {
            "rhcos-4.3/43.81.202003111353.0": "sg3_utils-libs-1.44-3.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "sg3_utils-libs-1.44-5.el8.x86_64"
        },
        "shim-x64": {
            "rhcos-4.3/43.81.202003111353.0": "shim-x64-15-11.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "shim-x64-15-14.el8_2.x86_64"
        },
        "skopeo": {
            "rhcos-4.3/43.81.202003111353.0": "skopeo-0.1.40-8.module+el8.1.1+5351+506397b0.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "skopeo-1.0.0-1.module+el8.2.1+6676+604e1b26.x86_64"
        },
        "slirp4netns": {
            "rhcos-4.3/43.81.202003111353.0": "slirp4netns-0.4.2-4.git21fdece.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "slirp4netns-1.0.1-1.module+el8.2.1+6595+03641d72.x86_64"
        },
        "socat": {
            "rhcos-4.3/43.81.202003111353.0": "socat-1.7.3.2-6.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "socat-1.7.3.3-2.el8.x86_64"
        },
        "sqlite-libs": {
            "rhcos-4.3/43.81.202003111353.0": "sqlite-libs-3.26.0-4.el8_1.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "sqlite-libs-3.26.0-6.el8.x86_64"
        },
        "sssd": {
            "rhcos-4.3/43.81.202003111353.0": "sssd-2.2.0-19.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "sssd-2.2.3-20.el8.x86_64"
        },
        "sssd-ad": {
            "rhcos-4.3/43.81.202003111353.0": "sssd-ad-2.2.0-19.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "sssd-ad-2.2.3-20.el8.x86_64"
        },
        "sssd-client": {
            "rhcos-4.3/43.81.202003111353.0": "sssd-client-2.2.0-19.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "sssd-client-2.2.3-20.el8.x86_64"
        },
        "sssd-common": {
            "rhcos-4.3/43.81.202003111353.0": "sssd-common-2.2.0-19.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "sssd-common-2.2.3-20.el8.x86_64"
        },
        "sssd-common-pac": {
            "rhcos-4.3/43.81.202003111353.0": "sssd-common-pac-2.2.0-19.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "sssd-common-pac-2.2.3-20.el8.x86_64"
        },
        "sssd-ipa": {
            "rhcos-4.3/43.81.202003111353.0": "sssd-ipa-2.2.0-19.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "sssd-ipa-2.2.3-20.el8.x86_64"
        },
        "sssd-krb5": {
            "rhcos-4.3/43.81.202003111353.0": "sssd-krb5-2.2.0-19.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "sssd-krb5-2.2.3-20.el8.x86_64"
        },
        "sssd-krb5-common": {
            "rhcos-4.3/43.81.202003111353.0": "sssd-krb5-common-2.2.0-19.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "sssd-krb5-common-2.2.3-20.el8.x86_64"
        },
        "sssd-ldap": {
            "rhcos-4.3/43.81.202003111353.0": "sssd-ldap-2.2.0-19.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "sssd-ldap-2.2.3-20.el8.x86_64"
        },
        "sssd-nfs-idmap": {
            "rhcos-4.3/43.81.202003111353.0": "sssd-nfs-idmap-2.2.0-19.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "sssd-nfs-idmap-2.2.3-20.el8.x86_64"
        },
        "sssd-proxy": {
            "rhcos-4.3/43.81.202003111353.0": "sssd-proxy-2.2.0-19.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "sssd-proxy-2.2.3-20.el8.x86_64"
        },
        "strace": {
            "rhcos-4.3/43.81.202003111353.0": "strace-4.24-5.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "strace-4.24-9.el8.x86_64"
        },
        "subscription-manager-rhsm-certificates": {
            "rhcos-4.3/43.81.202003111353.0": "subscription-manager-rhsm-certificates-1.25.17-1.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "subscription-manager-rhsm-certificates-1.26.17-1.el8_2.x86_64"
        },
        "sudo": {
            "rhcos-4.3/43.81.202003111353.0": "sudo-1.8.25p1-8.el8_1.1.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "sudo-1.8.29-5.el8.x86_64"
        },
        "systemd": {
            "rhcos-4.3/43.81.202003111353.0": "systemd-239-27.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "systemd-239-31.el8_2.2.x86_64"
        },
        "systemd-journal-remote": {
            "rhcos-4.3/43.81.202003111353.0": "systemd-journal-remote-239-27.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "systemd-journal-remote-239-31.el8_2.2.x86_64"
        },
        "systemd-libs": {
            "rhcos-4.3/43.81.202003111353.0": "systemd-libs-239-27.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "systemd-libs-239-31.el8_2.2.x86_64"
        },
        "systemd-pam": {
            "rhcos-4.3/43.81.202003111353.0": "systemd-pam-239-27.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "systemd-pam-239-31.el8_2.2.x86_64"
        },
        "systemd-udev": {
            "rhcos-4.3/43.81.202003111353.0": "systemd-udev-239-27.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "systemd-udev-239-31.el8_2.2.x86_64"
        },
        "tpm2-tools": {
            "rhcos-4.3/43.81.202003111353.0": "tpm2-tools-3.1.4-5.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "tpm2-tools-3.2.1-1.el8.x86_64"
        },
        "tzdata": {
            "rhcos-4.3/43.81.202003111353.0": "tzdata-2019c-1.el8.noarch",
            "rhcos-4.3/43.82.202007291953.0": "tzdata-2020a-1.el8.noarch"
        },
        "unbound-libs": {
            "rhcos-4.3/43.81.202003111353.0": "unbound-libs-1.7.3-8.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "unbound-libs-1.7.3-11.el8_2.x86_64"
        },
        "util-linux": {
            "rhcos-4.3/43.81.202003111353.0": "util-linux-2.32.1-17.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "util-linux-2.32.1-22.el8.x86_64"
        },
        "which": {
            "rhcos-4.3/43.81.202003111353.0": "which-2.21-10.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "which-2.21-12.el8.x86_64"
        },
        "xfsprogs": {
            "rhcos-4.3/43.81.202003111353.0": "xfsprogs-5.0.0-1.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "xfsprogs-5.0.0-2.el8.x86_64"
        },
        "zlib": {
            "rhcos-4.3/43.81.202003111353.0": "zlib-1.2.11-10.el8.x86_64",
            "rhcos-4.3/43.82.202007291953.0": "zlib-1.2.11-13.el8.x86_64"
        },
        "elfutils-debuginfod-client": {
            "rhcos-4.3/43.81.202003111353.0": "Not present",
            "rhcos-4.3/43.82.202007291953.0": "elfutils-debuginfod-client-0.178-7.el8.x86_64"
        },
        "kbd": {
            "rhcos-4.3/43.81.202003111353.0": "Not present",
            "rhcos-4.3/43.82.202007291953.0": "kbd-2.0.4-8.el8.x86_64"
        },
        "kbd-legacy": {
            "rhcos-4.3/43.81.202003111353.0": "Not present",
            "rhcos-4.3/43.82.202007291953.0": "kbd-legacy-2.0.4-8.el8.noarch"
        },
        "kbd-misc": {
            "rhcos-4.3/43.81.202003111353.0": "Not present",
            "rhcos-4.3/43.82.202007291953.0": "kbd-misc-2.0.4-8.el8.noarch"
        },
        "libslirp": {
            "rhcos-4.3/43.81.202003111353.0": "Not present",
            "rhcos-4.3/43.82.202007291953.0": "libslirp-4.3.0-3.module+el8.2.1+6816+bedf4f91.x86_64"
        },
        "libxkbcommon": {
            "rhcos-4.3/43.81.202003111353.0": "Not present",
            "rhcos-4.3/43.82.202007291953.0": "libxkbcommon-0.9.1-1.el8.x86_64"
        },
        "libzstd": {
            "rhcos-4.3/43.81.202003111353.0": "Not present",
            "rhcos-4.3/43.82.202007291953.0": "libzstd-1.4.2-2.el8.x86_64"
        },
        "xkeyboard-config": {
            "rhcos-4.3/43.81.202003111353.0": "Not present",
            "rhcos-4.3/43.82.202007291953.0": "xkeyboard-config-2.28-1.el8.noarch"
        }
    }
}

@miabbott
Copy link
Member Author

This update also includes RHEL 8.2

@miabbott
Copy link
Member Author

/hold

While we've tested that booting this RHCOS version via qemu using Secure Boot is successful, there are reports of problems booting with the grub2 mitigation in place - https://bugzilla.redhat.com/show_bug.cgi?id=1861977

I'm going to tentatively hold this until we have more confidence

@openshift-ci-robot openshift-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 30, 2020
@cgwalters
Copy link
Member

The mitigation path for OCP customers is to reprovision nodes, so we
should bump the boot images on the installer as well.
@miabbott
Copy link
Member Author

miabbott commented Aug 3, 2020

/unhold

The latest RHCOS builds include the fixed shim package to address BZ#1861977

Notably we also get a new toolbox package.

$ ./differ.py -fe api.ci -fr rhcos-4.3/43.82.202007291953.0 -se api.ci -sr rhcos-4.3/43.82.2020080
10953.0
{
    "sources": {
        "rhcos-4.3/43.82.202007291953.0": "https://releases-art-rhcos.svc.ci.openshift.org/art/storage/releases/rhcos-4.3/43.82.202007291953.0/x86_64/commitmeta.json",
        "rhcos-4.3/43.82.202008010953.0": "https://releases-art-rhcos.svc.ci.openshift.org/art/storage/releases/rhcos-4.3/43.82.202008010953.0/x86_64/commitmeta.json"
    },
    "diff": {
        "conmon": {
            "rhcos-4.3/43.82.202007291953.0": "conmon-2.0.17-1.rhaos4.3.el8.x86_64",
            "rhcos-4.3/43.82.202008010953.0": "conmon-2.0.20-1.rhaos4.3.el8.x86_64"
        },
        "containers-common": {
            "rhcos-4.3/43.82.202007291953.0": "containers-common-1.0.0-1.module+el8.2.1+6676+604e1b26.x86_64",
            "rhcos-4.3/43.82.202008010953.0": "containers-common-1.1.1-1.rhaos4.3.el8.x86_64"
        },
        "openshift-clients": {
            "rhcos-4.3/43.82.202007291953.0": "openshift-clients-4.3.31-202007250052.p0.git.3329.59998b9.el8.x86_64",
            "rhcos-4.3/43.82.202008010953.0": "openshift-clients-4.3.32-202008010046.p0.git.3329.9a170c9.el8.x86_64"
        },
        "openshift-hyperkube": {
            "rhcos-4.3/43.82.202007291953.0": "openshift-hyperkube-4.3.31-202007280738.p0.git.0.9884401.el8.x86_64",
            "rhcos-4.3/43.82.202008010953.0": "openshift-hyperkube-4.3.32-202008010046.p0.git.0.68a46ff.el8.x86_64"
        },
        "shim-x64": {
            "rhcos-4.3/43.82.202007291953.0": "shim-x64-15-14.el8_2.x86_64",
            "rhcos-4.3/43.82.202008010953.0": "shim-x64-15-15.el8_2.x86_64"
        },
        "skopeo": {
            "rhcos-4.3/43.82.202007291953.0": "skopeo-1.0.0-1.module+el8.2.1+6676+604e1b26.x86_64",
            "rhcos-4.3/43.82.202008010953.0": "skopeo-1.1.1-1.rhaos4.3.el8.x86_64"
        },
        "toolbox": {
            "rhcos-4.3/43.82.202007291953.0": "toolbox-0.0.7-1.rhaos4.3.el8.noarch",
            "rhcos-4.3/43.82.202008010953.0": "toolbox-0.0.8-1.rhaos4.3.el8.noarch"
        }
    }
}

@openshift-ci-robot openshift-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 3, 2020
@darkmuggle
Copy link
Contributor

/retest

@miabbott
Copy link
Member Author

miabbott commented Aug 4, 2020

/retest

e2e-gcp-upgrade doesn't look great; has failed multiple times while the cluster is being disrupted due to the upgrade. Haven't been able to sort out the reason.

e2e-openstack keeps failing with Some cluster operators are still updating: authentication, console, image-registry, ingress, monitoring" before it can reach 100% rollout of the cluster.

@miabbott
Copy link
Member Author

miabbott commented Aug 4, 2020

Interestingly, I was able to get cluster-bot to do test upgrade 4.3 openshift/installer#3987 gcp successfully - https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/release-openshift-origin-installer-launch-gcp/1290731756546166784

Though that looks like it didn't actually test the upgrade path.... 🤔

@miabbott
Copy link
Member Author

miabbott commented Aug 6, 2020

/test e2e-azure
/test e2e-metal
/test e2e-vsphere
/test e2e-ovirt
/test e2e-gcp

@miabbott
Copy link
Member Author

miabbott commented Aug 6, 2020

@abhinavdahiya @sdodson The final boot image bump in the series for Boot Hole.

@sdodson sdodson added bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. and removed bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. labels Aug 6, 2020
@miabbott
Copy link
Member Author

miabbott commented Aug 7, 2020

/retest

openstack, gcp, metal, azure, vsphere failed during build of the installer with:

http://mirror.centos.org/centos/7/cloud/x86_64/openstack-stein/Packages/p/python2-jinja2-2.10.1-1.el7.noarch.rpm: [Errno 14] HTTP Error 404 - Not Found

@sdodson
Copy link
Member

sdodson commented Aug 7, 2020

/test e2e-azure
e2e-gcp and e2e-metal install completed but many many e2e test failures, most on sig-storage tests
@openshift/sig-storage PTAL at these tests

@sdodson
Copy link
Member

sdodson commented Aug 7, 2020

/test e2e-gcp
/test e2e-metal
/test e2e-azure

@sdodson
Copy link
Member

sdodson commented Aug 7, 2020

CI registry flakes everywhere
/test e2e-gcp
/test e2e-azure
/test e2e-metal

@sdodson
Copy link
Member

sdodson commented Aug 8, 2020

/test e2e-gcp
/test e2e-azure
/test e2e-metal

@miabbott
Copy link
Member Author

miabbott commented Aug 8, 2020

/bugzilla refresh

@openshift-ci-robot openshift-ci-robot removed the bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. label Aug 8, 2020
@openshift-ci-robot
Copy link
Contributor

@miabbott: This pull request references Bugzilla bug 1862114, which is invalid:

  • expected dependent Bugzilla bug 1862111 to target a release in 4.4.0, 4.4.z, but it targets "4.6.0" instead
  • expected dependent Bugzilla bug 1862112 to target a release in 4.4.0, 4.4.z, but it targets "4.5.z" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci-robot
Copy link
Contributor

@sdodson: This pull request references Bugzilla bug 1862114, which is valid. The bug has been moved to the POST state. The bug has been updated to refer to the pull request using the external bug tracker.

6 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.3.z) matches configured target release for branch (4.3.z)
  • bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)
  • dependent bug Bugzilla bug 1862113 is in the state VERIFIED, which is one of the valid states (VERIFIED, RELEASE_PENDING, CLOSED (ERRATA))
  • dependent Bugzilla bug 1862113 targets the "4.4.z" release, which is one of the valid target releases: 4.4.0, 4.4.z
  • bug has dependents

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci-robot openshift-ci-robot added bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. and removed bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. labels Aug 8, 2020
@miabbott
Copy link
Member Author

miabbott commented Aug 9, 2020

@sdodson Understood. This bump brings in RHEL 8.2 as well as the mitigations for Boot Hole CVE. We are using the same RHEL 8.2 content in 4.5 + 4.4, so it is puzzling why the tests are failing in this branch.

@miabbott
Copy link
Member Author

miabbott commented Aug 9, 2020

In the e2e-azuretest, I see some abort/stack traces which look like they are coming from the oc command; not sure it is related to the all the failures:

    error running &{/usr/bin/kubectl [kubectl --server=https://api.ci-op-prc3dnm9-2373a.ci.azure.devcluster.openshift.com:6443 --kubeconfig=/var/run/secrets/ci.openshift.io/multi-stage/kubeconfig get pods -o template --template={{range.items}}{{.metadata.name}} {{end}} -l name=update-demo --namespace=e2e-kubectl-4288] []  <nil>  runtime/cgo: pthread_create failed: Resource temporarily unavailable
    SIGABRT: abort
    PC=0x7f837ea6c387 m=5 sigcode=18446744073709551610
    
    goroutine 0 [idle]:
    runtime: unknown pc 0x7f837ea6c387
    stack: frame={sp:0x7f837b020978, fp:0x0} stack=[0x7f837a8212a8,0x7f837b020ea8)
    00007f837b020878:  0000000000000000  0000000000000000 
    00007f837b020888:  0000000000000000  0000000000000000 
    00007f837b020898:  0000000000000002  000000c000097fd0 
    00007f837b0208a8:  0000000000000000  0000000000000000 
    00007f837b0208b8:  0000000000000000  0000000000000000 
    00007f837b0208c8:  0000000000000000  0000000000000000 
    00007f837b0208d8:  0000000000000000  0000000000000000 
    00007f837b0208e8:  0000000000000000  0000000000000000 
    00007f837b0208f8:  0000000000000000  000000000046c931 <runtime.goexit+1> 
    00007f837b020908:  000000000019e320  0000000000000000 
    00007f837b020918:  0000000000000000  0000000000000000 
    00007f837b020928:  0000000000000000  0000000000428594 <runtime.(*fixalloc).alloc+116> 
    00007f837b020938:  00007f837f409008  0000000000000030 
    00007f837b020948:  00007f837edfe868  00000000034902bb 
    00007f837b020958:  00007f836c0008c0  0000000000000011 
    00007f837b020968:  00000000032ec854  0000000000000000 
    00007f837b020978: <00007f837ea6da78  0000000000000020 
    00007f837b020988:  0000000000000000  0000000000000000 
    00007f837b020998:  0000000000000000  0000000000000000 
    00007f837b0209a8:  0000000000000000  0000000000000000 
    00007f837b0209b8:  0000000000000000  0000000000000000 
    00007f837b0209c8:  0000000000000000  0000000000000000 
    00007f837b0209d8:  0000000000000000  0000000000000000 
    00007f837b0209e8:  0000000000000000  0000000000000000 
    00007f837b0209f8:  0000000000000000  000000000000000d 
    00007f837b020a08:  00007f837eb25a29  0000000000000000 
    00007f837b020a18:  00007f837eab0263  000000000000000d 
    00007f837b020a28:  00007f837edfe1c0  0000000000000001 
    00007f837b020a38:  00007f837edfe243  00007f837edfa380 
    00007f837b020a48:  00007f837eab1a7e  00007f837edfe1c0 
    00007f837b020a58:  000000000000000a  00007f836c0008c0 
    00007f837b020a68:  0000000000000011  00000000032ec854 
    runtime: unknown pc 0x7f837ea6c387
    stack: frame={sp:0x7f837b020978, fp:0x0} stack=[0x7f837a8212a8,0x7f837b020ea8)
    00007f837b020878:  0000000000000000  0000000000000000 
    00007f837b020888:  0000000000000000  0000000000000000 
    00007f837b020898:  0000000000000002  000000c000097fd0 
    00007f837b0208a8:  0000000000000000  0000000000000000 
    00007f837b0208b8:  0000000000000000  0000000000000000 
    00007f837b0208c8:  0000000000000000  0000000000000000 
    00007f837b0208d8:  0000000000000000  0000000000000000 
    00007f837b0208e8:  0000000000000000  0000000000000000 
    00007f837b0208f8:  0000000000000000  000000000046c931 <runtime.goexit+1> 
    00007f837b020908:  000000000019e320  0000000000000000 
    00007f837b020918:  0000000000000000  0000000000000000 
    00007f837b020928:  0000000000000000  0000000000428594 <runtime.(*fixalloc).alloc+116> 
    00007f837b020938:  00007f837f409008  0000000000000030 
    00007f837b020948:  00007f837edfe868  00000000034902bb 
    00007f837b020958:  00007f836c0008c0  0000000000000011 
    00007f837b020968:  00000000032ec854  0000000000000000 
    00007f837b020978: <00007f837ea6da78  0000000000000020 
    00007f837b020988:  0000000000000000  0000000000000000 
    00007f837b020998:  0000000000000000  0000000000000000 
    00007f837b0209a8:  0000000000000000  0000000000000000 
    00007f837b0209b8:  0000000000000000  0000000000000000 
    00007f837b0209c8:  0000000000000000  0000000000000000 
    00007f837b0209d8:  0000000000000000  0000000000000000 
    00007f837b0209e8:  0000000000000000  0000000000000000 
    00007f837b0209f8:  0000000000000000  000000000000000d 
    00007f837b020a08:  00007f837eb25a29  0000000000000000 
    00007f837b020a18:  00007f837eab0263  000000000000000d 
    00007f837b020a28:  00007f837edfe1c0  0000000000000001 
    00007f837b020a38:  00007f837edfe243  00007f837edfa380 
    00007f837b020a48:  00007f837eab1a7e  00007f837edfe1c0 
    00007f837b020a58:  000000000000000a  00007f836c0008c0 
    00007f837b020a68:  0000000000000011  00000000032ec854 
    
    goroutine 1 [semacquire]:
    sync.runtime_Semacquire(0xc0020d0b38)
    	/usr/local/go/src/runtime/sema.go:56 +0x39
    sync.(*WaitGroup).Wait(0xc0020d0b30)
    	/usr/local/go/src/sync/waitgroup.go:130 +0x65
    github.com/openshift/oc/vendor/k8s.io/client-go/discovery.fetchGroupVersionResources(0x33e9f60, 0xc002047d80, 0xc002047dc0, 0x20, 0x21)
    	/go/src/github.com/openshift/oc/vendor/k8s.io/client-go/discovery/discovery_client.go:377 +0x125
    github.com/openshift/oc/vendor/k8s.io/client-go/discovery.ServerGroupsAndResources(0x33e9f60, 0xc002047d80, 0xc000d5b520, 0x13e43b1, 0x2a59e40, 0xc0020d26c0, 0x0, 0x0, 0x0, 0x0)
    	/go/src/github.com/openshift/oc/vendor/k8s.io/client-go/discovery/discovery_client.go:264 +0x18e
    github.com/openshift/oc/vendor/k8s.io/client-go/discovery/cached/disk.(*CachedDiscoveryClient).ServerResources(0xc002047d80, 0x0, 0x0, 0x0, 0x0, 0x0)
    	/go/src/github.com/openshift/oc/vendor/k8s.io/client-go/discovery/cached/disk/cached_discovery.go:96 +0x37
    github.com/openshift/oc/vendor/k8s.io/client-go/restmapper.discoveryCategoryExpander.Expand(0x33e9f60, 0xc002047d80, 0x7fff695c8048, 0x4, 0x3345300, 0xc0020ad710, 0x0, 0x0)
    	/go/src/github.com/openshift/oc/vendor/k8s.io/client-go/restmapper/category_expansion.go:61 +0x4c
    github.com/openshift/oc/vendor/k8s.io/cli-runtime/pkg/resource.(*Builder).ReplaceAliases(0xc001de9400, 0x7fff695c8048, 0x4, 0xc00203a500, 0x0)
    	/go/src/github.com/openshift/oc/vendor/k8s.io/cli-runtime/pkg/resource/builder.go:589 +0x281
    github.com/openshift/oc/vendor/k8s.io/cli-runtime/pkg/resource.(*Builder).ResourceTypeOrNameArgs(0xc001de9400, 0x1, 0xc00203a510, 0x1, 0x9, 0xda9f85a8)
    	/go/src/github.com/openshift/oc/vendor/k8s.io/cli-runtime/pkg/resource/builder.go:553 +0xce5
    github.com/openshift/oc/vendor/k8s.io/kubernetes/pkg/kubectl/cmd/get.(*GetOptions).Run(0xc00063f1d0, 0x3400460, 0xc001dae540, 0xc001fd3680, 0xc00203a510, 0x1, 0x9, 0x0, 0x0)
    	/go/src/github.com/openshift/oc/vendor/k8s.io/kubernetes/pkg/kubectl/cmd/get/get.go:479 +0x214
    github.com/openshift/oc/vendor/k8s.io/kubernetes/pkg/kubectl/cmd/get.NewCmdGet.func1(0xc001fd3680, 0xc00203a510, 0x1, 0x9)
    	/go/src/github.com/openshift/oc/vendor/k8s.io/kubernetes/pkg/kubectl/cmd/get/get.go:168 +0x12f
    github.com/openshift/oc/vendor/github.com/spf13/cobra.(*Command).execute(0xc001fd3680, 0xc00203a480, 0x9, 0x9, 0xc001fd3680, 0xc00203a480)
    	/go/src/github.com/openshift/oc/vendor/github.com/spf13/cobra/command.go:830 +0x2ae
    github.com/openshift/oc/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc001dadb80, 0x7, 0xc001dadb80, 0x7)
    	/go/src/github.com/openshift/oc/vendor/github.com/spf13/cobra/command.go:914 +0x2fc
    github.com/openshift/oc/vendor/github.com/spf13/cobra.(*Command).Execute(...)
    	/go/src/github.com/openshift/oc/vendor/github.com/spf13/cobra/command.go:864
    main.main()
    	/go/src/github.com/openshift/oc/cmd/oc/oc.go:110 +0x835```

@miabbott
Copy link
Member Author

miabbott commented Aug 9, 2020

e2e-metal showing oc blowing up too:

Aug  8 17:48:10.398: INFO: Running 'oc --namespace=e2e-test-build-valuefrom-2ds44 --config=/tmp/configfile484652386 create -f /tmp/fixture-testdata-dir825035343/test/extended/testdata/builds/valuefrom/test-configmap.yaml'
goroutine 1 [running]:
runtime/debug.Stack(0x7f2403cf26d0, 0x0, 0x4dc054)
	/usr/local/go/src/runtime/debug/stack.go:24 +0x9d
github.com/openshift/origin/test/extended/util.FatalErr(0x4bdec20, 0xc0029c2070)
	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/test/extended/util/client.go:680 +0x26
github.com/openshift/origin/test/extended/util.(*CLI).Output(0xc001889b00, 0x2, 0x2, 0xc002d08c30, 0x447642)
	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/test/extended/util/client.go:576 +0x5e9
github.com/openshift/origin/test/extended/util.(*CLI).Execute(0xc001889b00, 0xc0027f6bd0, 0x2)
	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/test/extended/util/client.go:669 +0x40
github.com/openshift/origin/test/extended/builds.glob..func39.1.3()
	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/test/extended/builds/valuefrom.go:53 +0x944
github.com/openshift/origin/vendor/github.com/onsi/ginkgo/internal/leafnodes.(*runner).runSync(0xc0015d0840, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/onsi/ginkgo/internal/leafnodes/runner.go:113 +0x9c
github.com/openshift/origin/vendor/github.com/onsi/ginkgo/internal/leafnodes.(*runner).run(0xc0015d0840, 0x8e, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/onsi/ginkgo/internal/leafnodes/runner.go:64 +0xcf
github.com/openshift/origin/vendor/github.com/onsi/ginkgo/internal/leafnodes.(*SetupNode).Run(0xc0016588a0, 0x61a8e80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/onsi/ginkgo/internal/leafnodes/setup_nodes.go:15 +0x64
github.com/openshift/origin/vendor/github.com/onsi/ginkgo/internal/spec.(*Spec).runSample(0xc002958f00, 0x0, 0x61a8e80, 0xc000271480)
	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/onsi/ginkgo/internal/spec/spec.go:205 +0x44f
github.com/openshift/origin/vendor/github.com/onsi/ginkgo/internal/spec.(*Spec).Run(0xc002958f00, 0x61a8e80, 0xc000271480)
	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/onsi/ginkgo/internal/spec/spec.go:138 +0xf4
github.com/openshift/origin/vendor/github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).runSpec(0xc001a68f00, 0xc002958f00, 0x0)
	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/onsi/ginkgo/internal/specrunner/spec_runner.go:200 +0x10f
github.com/openshift/origin/vendor/github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).runSpecs(0xc001a68f00, 0x1)
	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/onsi/ginkgo/internal/specrunner/spec_runner.go:170 +0x124
github.com/openshift/origin/vendor/github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).Run(0xc001a68f00, 0xc0013d6a88)
	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/onsi/ginkgo/internal/specrunner/spec_runner.go:66 +0x117
github.com/openshift/origin/vendor/github.com/onsi/ginkgo/internal/suite.(*Suite).Run(0xc000279900, 0x61a7740, 0xc001cade50, 0x0, 0x0, 0xc000e72020, 0x1, 0x1, 0x6298860, 0xc000271480, ...)
	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/onsi/ginkgo/internal/suite/suite.go:62 +0x42e
github.com/openshift/origin/pkg/test/ginkgo.(*TestOptions).Run(0xc001b83740, 0xc001b871b0, 0x1, 0x1, 0x0, 0x0)
	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/pkg/test/ginkgo/cmd_runtest.go:59 +0x4d2
main.newRunTestCommand.func1(0xc000c84000, 0xc001b871b0, 0x1, 0x1, 0x0, 0x0)
	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/cmd/openshift-tests/openshift-tests.go:233 +0x11f
github.com/openshift/origin/vendor/github.com/spf13/cobra.(*Command).execute(0xc000c84000, 0xc001b87170, 0x1, 0x1, 0xc000c84000, 0xc001b87170)
	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/spf13/cobra/command.go:826 +0x465
github.com/openshift/origin/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc000c21180, 0x61af478, 0xa571d18, 0xa571d18)
	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/spf13/cobra/command.go:914 +0x2fc
github.com/openshift/origin/vendor/github.com/spf13/cobra.(*Command).Execute(...)
	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/spf13/cobra/command.go:864
main.main.func1(0xc000c21180, 0x0, 0x0)
	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/cmd/openshift-tests/openshift-tests.go:68 +0x93
main.main()
	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/cmd/openshift-tests/openshift-tests.go:69 +0x327

Aug  8 17:48:10.399: INFO: unable to execute "oc": fork/exec /usr/bin/oc: resource temporarily unavailable

@miabbott
Copy link
Member Author

miabbott commented Aug 9, 2020

e2e-aws appears to fail because the F30 image isn't getting delivered properly:

Aug  8 17:52:04.538: INFO: At 2020-08-08 17:49:07 +0000 UTC - event for get-bootstrap-creds: {kubelet ip-10-0-136-96.ec2.internal} Pulling: Pulling image "registry.fedoraproject.org/fedora:30"
Aug  8 17:52:04.538: INFO: At 2020-08-08 17:49:14 +0000 UTC - event for get-bootstrap-creds: {kubelet ip-10-0-136-96.ec2.internal} Failed: Failed to pull image "registry.fedoraproject.org/fedora:30": rpc error: code = Unknown desc = Error reading blob sha256:ec1dd3aa5ab3ec9f5a9528154e3290ac9b6f81354b13869bd15f1f98abd531f8: Digest did not match, expected sha256:ec1dd3aa5ab3ec9f5a9528154e3290ac9b6f81354b13869bd15f1f98abd531f8, got sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Aug  8 17:52:04.538: INFO: At 2020-08-08 17:49:14 +0000 UTC - event for get-bootstrap-creds: {kubelet ip-10-0-136-96.ec2.internal} Failed: Error: ErrImagePull
Aug  8 17:52:04.538: INFO: At 2020-08-08 17:49:14 +0000 UTC - event for get-bootstrap-creds: {kubelet ip-10-0-136-96.ec2.internal} BackOff: Back-off pulling image "registry.fedoraproject.org/fedora:30"
Aug  8 17:52:04.538: INFO: At 2020-08-08 17:49:14 +0000 UTC - event for get-bootstrap-creds: {kubelet ip-10-0-136-96.ec2.internal} Failed: Error: ImagePullBackOff

@miabbott
Copy link
Member Author

miabbott commented Aug 9, 2020

For the e2e-aws failure, it looks like we need openshift/origin@0dc769a back-ported to 4.4 + 4.3

openshift/origin#25393 openshift/origin#25348
openshift/origin#25394

@sdodson
Copy link
Member

sdodson commented Aug 10, 2020

/test e2e-aws
/test e2e-azure
/test e2e-gcp
/test e2e-metal
/test e2e-vsphere

@miabbott
Copy link
Member Author

Opened a dummy PR to see if similar failures are observed - #4035

@miabbott
Copy link
Member Author

Well, the dummy PR (#4035) is failing some of the same tests as this PR, but in different ways.

We continue to see golang panics/stacks in e2e-metal, e2e-aws, e2e-gcp which are concerning and I'm not sure how to debug them.

@sdodson
Copy link
Member

sdodson commented Aug 10, 2020

/lgtm
/approve
We're moving forward with this, all jobs completed installation however pre-submit e2e tests are in very poor state.

@openshift-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sdodson

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot openshift-ci-robot added lgtm Indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Aug 10, 2020
@sdodson sdodson added the cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. label Aug 10, 2020
@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

7 similar comments
@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-ci-robot
Copy link
Contributor

openshift-ci-robot commented Aug 11, 2020

@miabbott: The following test failed, say /retest to rerun all failed tests:

Test name Commit Details Rerun command
ci/prow/e2e-vsphere cb6d677 link /test e2e-vsphere

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-merge-robot openshift-merge-robot merged commit 43f11cf into openshift:release-4.3 Aug 11, 2020
@openshift-ci-robot
Copy link
Contributor

@miabbott: All pull requests linked via external trackers have merged: openshift/installer#3987. Bugzilla bug 1862114 has been moved to the MODIFIED state.

In response to this:

Bug 1862114: bump RHCOS images for CVE-2020-10713

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. bugzilla/severity-medium Referenced Bugzilla bug's severity is medium for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. lgtm Indicates that a PR is ready to be merged.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants