CentOS not booting after 'yum update' (using zfs-kmod) [WORKAROUND]

I am using ZFS as a root filesystem (zfs-kmod) installed like described in this guide.

After I updated the system with 'yum update' and rebooted, the system does not boot.

Update: I was wrong - did not changed the version of the repo from 7.3 to 7.4. When updating your CentOS system with kmod-zfs, please update also your zfs.repo.

The relevant error message was:

Failed to start Import ZFS pools by cache file.
[FAILED] See 'systemctl status zfs-import-cache.service' for details.

Also there was many lines "dracut ...", but these lines was not helpful for debugging.

dracut-initqueue[249]: Warning: dracut-initqueue timout - starting timeout scripts
dracut-initqueue[249]: Warning: dracut-initqueue timout - starting timeout scripts
dracut-initqueue[249]: Warning: dracut-initqueue timout - starting timeout scripts
dracut-initqueue[249]: Warning: dracut-initqueue timout - starting timeout scripts
...

After entering the emergency mode I confirmed that the ZFS kernel module is missing ('modprobe zfs' gives error message).

When rebooting I see this menu:

It does not contain the new kernels.

When I tried to run the system by selecting the second line in the menu, I got this:

I rebooted the system again and went into command prompt mode by pressing the 'c'.

To see the list of available kernels, I wrote this command:

grub>  ls (hd0,msdos1)/ROOT/@/boot/ -la

After exiting the command prompt mode I selected the first item in the menu and pressed the key 'e' (to enter the edit mode):

I tried different kernels, but it worked only after I changed the version from "3.10.0-514.16.1" to "3.10.0-514.26.2":

After the successful boot I added all of the kernels to the menu:

# grub2-mkconfig -o /boot/grub2/grub.cfg

After reboot I need to select the correct kernel from the menu. It does not boot when I leave the default.

Now, it's much more easy to run the system - I need just to select the correct kernel from the Grub menu (instead of manually editing the configuration by pressing 'e' at the correct moment).

The moral of the story:

• Using ZFS with CentOS is risky, it might not boot after update.
• Do not delete the old kernels!
• Before to reboot make sure all of the kernels are added to the Grub menu (because it's annoying to edit the configuration manually).
• Do not reboot if it's not necessary.

Also, before to run 'yum update', make a copy of your kernels:

[root@localhost boot]# cp vmlinuz-3.10.0-514.10.2.el7.x86_64 vmlinuz-3.10.0-514.10.2.el7.x86_64.bak1
[root@localhost boot]# cp initramfs-3.10.0-514.10.2.el7.x86_64.img initramfs-3.10.0-514.10.2.el7.x86_64.img.bak1

Or better:

[root@localhost boot]# cp vmlinuz-3.10.0-514.10.2.el7.x86_64 vmlinuz-3.10.0-514.10.2.bak1.el7.x86_64
[root@localhost boot]# cp initramfs-3.10.0-514.10.2.el7.x86_64.img initramfs-3.10.0-514.10.2.bak1.el7.x86_64.img

This way the boot loader grub is adding the 'bak1' kernels properly (tested it with another install).

If the kernel 3.10.0-514.26.2.el7.x86_64 is missing in your install read this: Bug in ZFS 0.7.3-1 on CentOS - the zfs module is missing and the system is not booting (kmod-zfs-0.7.3-1.el7_3.x86_64) [WORKAROUND, NOT FIXED!].

A simple workaround is to migrate from kmod-zfs to zfs-dkms like described here.

Update: I was wrong

When updating your CentOS system with kmod-zfs, please update also your zfs.repo. I did not changed the version of the repo from 7.3 to 7.4.