Improve certificate renewal #324
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mz-pdm
requested review from
sgratch,
ljelinkova,
mwperina,
emesika,
ahadas,
bennyz,
michalskrivanek,
oliel and
didib
as code owners
...sible-runner-service-project/project/roles/ovirt-host-deploy-ovn-certificates/tasks/main.yml
Outdated
Show resolved
Hide resolved
ahadas
reviewed
May 3, 2022
...end/manager/modules/bll/src/main/java/org/ovirt/engine/core/bll/validator/HostValidator.java
Show resolved
Hide resolved
Non-web certificates are not subjects to the lifetime limitations required by web browsers. VDS certificates must be renewed when the host is in maintenance, which makes frequent upgrades difficult. Let’s extend the lifetime of non-web certificates to 5 years (a value selected by Michal) to make the certificate updates much less frequent. Bug-Url: https://bugzilla.redhat.com/2079835
…host-upgrade Otherwise the whole playbook will stop immediately and host upgrade fails. We want to proceed instead and update the certificates. Bug-Url: https://bugzilla.redhat.com/2079890
Currently, we warn about host certificate expiration 120 days in advance and alert and perform renewal on host upgrades 30 days in advance. These proved to be too late, many users don’t pay appropriate attention to warnings and alerts and when there is no host upgrade during the relatively short period, the users may end up with expired certificates and non-responsive hosts. Now, when we have extended the lifetime of host certificates to 3650 days, we can warn about host certificate expiration earlier and force renewal on host upgrades already during the warning period. This increases the chances there will be a host upgrade (with certificate renewal) during the period. Value of CertExpirationWarnPeriodInDays is increased to 365. This change also extends the warning period for all other certificates except for web certificates. Bug-Url: https://bugzilla.redhat.com/2079890
When a host certificate expires the host becomes non-responsive. There is no longer a way to connect to the host via the Vdsm API. But it would be still possible to renew the certificate if the operation wasn’t blocked in the web UI. Let’s permit enrolling certificates from the menu also for hosts in non-responsive state, to make life easier for users who ignore warning about upcoming certificate expiration until it actually happens. Of course, a host can become non-responsive also for other reasons than its certificate expiration. If the users decide to renew certificates in such states, it’s their own decision and risk. A host in such a state is not in a good shape anyway and restarting the services running on it, as needed by the certificate renewal, is not that much an issue. Bug-Url: https://bugzilla.redhat.com/2079901
OVN services are not restarted after OVN certificate update. Unlike other services, which are restarted after enrolling certificates, these ones are not restarted together with libvirtd restart and need their own restart action. Bug-Url: https://bugzilla.redhat.com/2079901
After updating Vdsm certificates, the corresponding services are not restarted. Or, actually, they are, but only as a part of ovirt-host-deploy-vnc-certificates role if those certificates happened to be updated as well (they usually do), which is not entirely sufficient. And ovirt-imageio is not restarted at all. Let’s make sure the services are always restarted after Vdsm certificates update. It’s sufficient to restart libvirtd (it restarts vdsmd etc. too) and ovirt-imageio (vdsmd wants it started but doesn’t ensure its restart). We check for both ‘enabled’ status and ‘running’ states. Some services may be stopped before updates, so checking for ‘running’ wouldn’t be sufficient. And ovirt-imageio may be running but not enabled, which is probably a bug but it doesn’t harm to check for both. This change also means that some services may be restarted more than once during a certificate update. Fixing this would need introducing inter-role service handling, which would be too complicated at the moment. Multiple restarts should be usually harmless because this is normally done in the maintenance mode. Bug-Url: https://bugzilla.redhat.com/2079901
|
Besides the things discussed above, I added another commit ensuring that services get restarted after Vdsm certificates update, including ovirt-imageio, which wasn't restarted in such a case at all, even not by side-effect (like the other services). |
michalskrivanek
approved these changes
May 4, 2022
mwperina
approved these changes
May 4, 2022
Expose VdsCertificateValidityInDays in engine-config to allow to set custom validity period of hypervisor certificates. Bug-Url: https://bugzilla.redhat.com/2079890
michalskrivanek
approved these changes
May 4, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This should fix some of the problems that users not paying proper attention to certificate expiration warnings and alerts may experience:
Read the corresponding commit messages for more details.
Not renewing some of the vmconsole certificates will be handled in a separate PR later.
Related bugs: