efibootmgr exit code 5 during leapp efi_finalization_fix

[mliguori-crestron]

Hi,

I'm not sure what is happening on this particular machine, that is failing on efibootmgr with an exit code 5, and saying "Could not prepare Boot variable: No such file or directory". My team members and I have run the AlmaOS migration from CentOS 7.9 to Alma 8.8 on many different machines (same models) and are not sure why this is happening here. This is the last step to trigger AlmaOS to auto-reboot, but stops at the last step. You can see a gap in time in the timestamp below, this is when we are looking at the box and eventually a user, manually triggers a power cycle and the box comes up into AlmaOS and operates properly. The goal is to never require a user to manually powercycle and it just seems so odd to see it fail at this part.

I post this because maybe something can be done in the future to help with this for others even if not for my timeframe.

Can you help me with this? I'm not sure if you are the right team to talk too about exit code 5 and better explain to me what has happened here.

Error in log:
2023-08-14 16:20:20.0 INFO PID: 3 leapp.workflow.Finalization: Executing actor efi_finalization_fix
2023-08-14 16:20:20.68 DEBUG PID: 2102 leapp.workflow.Finalization.efi_finalization_fix: External command has started: ['/sbin/efibootmgr', '-c', '-d', '/dev/nvme0n1p', '-p', '1', '-l', '\\EFI\\almalinux\\shimx64.efi', '-L', 'AlmaLinux']
2023-08-14 16:20:20.114 DEBUG PID: 2102 leapp.workflow.Finalization.efi_finalization_fix: Could not prepare Boot variable: No such file or directory
2023-08-14 16:20:20.134 DEBUG PID: 2102 leapp.workflow.Finalization.efi_finalization_fix: Command ['/sbin/efibootmgr', '-c', '-d', '/dev/nvme0n1p', '-p', '1', '-l', '\\EFI\\almalinux\\shimx64.efi', '-L', 'AlmaLinux'] failed with exit code 5.

LOG:
2023-08-14 16:20:19.260 DEBUG PID: 1772 leapp.workflow.Finalization.force_default_boot_to_target_kernel_version: kernelopts=root=/dev/mapper/centos-root ro crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb quiet
2023-08-14 16:20:19.297 DEBUG PID: 1772 leapp.workflow.Finalization.force_default_boot_to_target_kernel_version: External command has finished: ['grub2-editenv', 'list']
2023-08-14 16:20:19.316 DEBUG PID: 1772 leapp.workflow.Finalization.force_default_boot_to_target_kernel_version: External command has started: ['grubby', '--default-kernel']
2023-08-14 16:20:19.436 DEBUG PID: 1772 leapp.workflow.Finalization.force_default_boot_to_target_kernel_version: /boot/vmlinuz-4.18.0-477.13.1.el8_8.x86_64
2023-08-14 16:20:19.476 DEBUG PID: 1772 leapp.workflow.Finalization.force_default_boot_to_target_kernel_version: External command has finished: ['grubby', '--default-kernel']
2023-08-14 16:20:19.495 DEBUG PID: 1772 leapp.workflow.Finalization.force_default_boot_to_target_kernel_version: External command has started: ['grubby', '--default-index']
2023-08-14 16:20:19.617 DEBUG PID: 1772 leapp.workflow.Finalization.force_default_boot_to_target_kernel_version: 0
2023-08-14 16:20:19.655 DEBUG PID: 1772 leapp.workflow.Finalization.force_default_boot_to_target_kernel_version: External command has finished: ['grubby', '--default-index']
2023-08-14 16:20:19.675 DEBUG PID: 1772 leapp.workflow.Finalization.force_default_boot_to_target_kernel_version: External command has started: ['grubby', '--default-title']
2023-08-14 16:20:19.797 DEBUG PID: 1772 leapp.workflow.Finalization.force_default_boot_to_target_kernel_version: AlmaLinux (4.18.0-477.13.1.el8_8.x86_64) 8.8 (Sapphire Caracal)
2023-08-14 16:20:19.835 DEBUG PID: 1772 leapp.workflow.Finalization.force_default_boot_to_target_kernel_version: External command has finished: ['grubby', '--default-title']
2023-08-14 16:20:19.853 DEBUG PID: 1772 leapp.workflow.Finalization.force_default_boot_to_target_kernel_version: External command has started: ['grub2-editenv', 'list']
2023-08-14 16:20:19.899 DEBUG PID: 1772 leapp.workflow.Finalization.force_default_boot_to_target_kernel_version: saved_entry=f20816fa9c244729937fb2cd34bb89ee-4.18.0-477.13.1.el8_8.x86_64
2023-08-14 16:20:19.918 DEBUG PID: 1772 leapp.workflow.Finalization.force_default_boot_to_target_kernel_version: kernelopts=root=/dev/mapper/centos-root ro crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb quiet
2023-08-14 16:20:19.955 DEBUG PID: 1772 leapp.workflow.Finalization.force_default_boot_to_target_kernel_version: External command has finished: ['grub2-editenv', 'list']
2023-08-14 16:20:20.0 INFO PID: 3 leapp.workflow.Finalization: Executing actor efi_finalization_fix
2023-08-14 16:20:20.68 DEBUG PID: 2102 leapp.workflow.Finalization.efi_finalization_fix: External command has started: ['/sbin/efibootmgr', '-c', '-d', '/dev/nvme0n1p', '-p', '1', '-l', '\EFI\almalinux\shimx64.efi', '-L', 'AlmaLinux']
2023-08-14 16:20:20.114 DEBUG PID: 2102 leapp.workflow.Finalization.efi_finalization_fix: Could not prepare Boot variable: No such file or directory
2023-08-14 16:20:20.134 DEBUG PID: 2102 leapp.workflow.Finalization.efi_finalization_fix: Command ['/sbin/efibootmgr', '-c', '-d', '/dev/nvme0n1p', '-p', '1', '-l', '\EFI\almalinux\shimx64.efi', '-L', 'AlmaLinux'] failed with exit code 5.
2023-08-14 16:20:20.171 DEBUG PID: 2102 leapp.workflow.Finalization.efi_finalization_fix: External command has finished: ['/sbin/efibootmgr', '-c', '-d', '/dev/nvme0n1p', '-p', '1', '-l', '\EFI\almalinux\shimx64.efi', '-L', 'AlmaLinux']
2023-08-14 16:20:20.197 ERROR PID: 3 leapp: Actor efi_finalization_fix unexpectedly terminated with exit code: 1
2023-08-14 16:20:20.249 INFO PID: 3 leapp: Answerfile will be created at /var/log/leapp/answerfile
2023-08-14 16:20:20.381 ERROR PID: 3 leapp: Upgrade workflow failed, check log for details
### LEAPP reboot ###
-- Logs begin at Mon 2023-08-14 20:12:44 UTC, end at Mon 2023-08-14 20:20:20 UTC. --
Aug 14 20:12:44 localhost kernel: Linux version 4.18.0-477.13.1.el8_8.x86_64 (mockbuild@x64-builder02.almalinux.org) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-18) (GCC)) #1 SMP Tue May 30 14:53:41 EDT 2023
Aug 14 20:12:44 localhost kernel: Command line: BOOT_IMAGE=/vmlinuz-upgrade.x86_64 root=/dev/mapper/centos-root ro crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb quiet LANG=en_US.UTF-8 enforcing=0 rd.plymouth=0 plymouth.enable=0
Aug 14 20:12:44 localhost kernel: x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
Aug 14 20:12:44 localhost kernel: x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'