Verify that the kernel module zfs.ko is included in the initramfs image with this bash script [CentOS 7, ZFS]

If you are using kmod-zfs you should expect that some of the kernels will not boot (because the zfs.ko module is not included in the corresponding initramfs images):

# check-initramfs-zfs.ko.centos.bash
/boot/initramfs-0-rescue-7a7f370813524dd4b0a8bf9710545d76.img: zfs.ko MISSING
/boot/initramfs-3.10.0-514.16.1.el7.x86_64.img: zfs.ko MISSING
/boot/initramfs-3.10.0-514.21.1.el7.x86_64.img: zfs.ko MISSING
/boot/initramfs-3.10.0-514.26.1.el7.x86_64.img: zfs.ko MISSING
/boot/initramfs-3.10.0-514.26.2.bak1.el7.x86_64.img: OK
/boot/initramfs-3.10.0-514.26.2.el7.x86_64.img: zfs.ko MISSING
/boot/initramfs-3.10.0-693.5.2.el7.x86_64.img: OK

Make backup of the working kernel:

# cd /boot/
# cp initramfs-3.10.0-693.5.2.el7.x86_64.img initramfs-3.10.0-693.5.2.bak1.el7.x86_64.img
# cp vmlinuz-3.10.0-693.5.2.el7.x86_64 vmlinuz-3.10.0-693.5.2.bak1.el7.x86_64

The script check-initramfs-zfs.ko.centos.bash:

#!/bin/bash

ls /boot/initramfs*.img | while read initramfsimage; do

lines=$(lsinitrd $initramfsimage | grep zfs.ko | wc -l)

echo -ne "$initramfsimage:";

if [ "$lines" -eq "0" ]; then
 echo " zfs.ko MISSING"
else 
 echo " OK"
fi

done

Don't forget to change the version of the ZFS repo when updating your system. If you forget you will not get the zfs.ko for the latest kernel.

Setting the default kernel to Grub

Update the list of kernels in the Grub configuration:

# export ZPOOL_VDEV_NAME_PATH=YES
# grub2-mkconfig -o /boot/grub2/grub.cfg
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-3.10.0-693.5.2.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-693.5.2.el7.x86_64.img
Found linux image: /boot/vmlinuz-3.10.0-693.5.2.bak1.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-693.5.2.bak1.el7.x86_64.img
Found linux image: /boot/vmlinuz-3.10.0-514.26.2.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-514.26.2.el7.x86_64.img
Found linux image: /boot/vmlinuz-3.10.0-514.26.2.bak1.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-514.26.2.bak1.el7.x86_64.img
Found linux image: /boot/vmlinuz-3.10.0-514.26.1.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-514.26.1.el7.x86_64.img
Found linux image: /boot/vmlinuz-3.10.0-514.21.1.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-514.21.1.el7.x86_64.img
Found linux image: /boot/vmlinuz-3.10.0-514.16.1.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-514.16.1.el7.x86_64.img
Found linux image: /boot/vmlinuz-0-rescue-7a7f370813524dd4b0a8bf9710545d76
Found initrd image: /boot/initramfs-0-rescue-7a7f370813524dd4b0a8bf9710545d76.img
done

Verify that the GRUB_DEFAULT variable is set to 'saved':

# grep DEFAULT  /etc/default/grub
GRUB_DEFAULT=saved

The current default kernel is:

# grub2-editenv list
saved_entry=CentOS Linux (3.10.0-514.16.1.el7.x86_64) 7 (Core)

Get the list of the kernels (and their number):

# awk -F\' '$1=="menuentry " {print i++ " : " $2}' /etc/grub2.cfg
0 : CentOS Linux (3.10.0-693.5.2.el7.x86_64) 7 (Core)
1 : CentOS Linux (3.10.0-693.5.2.bak1.el7.x86_64) 7 (Core)
2 : CentOS Linux (3.10.0-514.26.2.el7.x86_64) 7 (Core)
3 : CentOS Linux (3.10.0-514.26.2.bak1.el7.x86_64) 7 (Core)
4 : CentOS Linux (3.10.0-514.26.1.el7.x86_64) 7 (Core)
5 : CentOS Linux (3.10.0-514.21.1.el7.x86_64) 7 (Core)
6 : CentOS Linux (3.10.0-514.16.1.el7.x86_64) 7 (Core)
7 : CentOS Linux (0-rescue-7a7f370813524dd4b0a8bf9710545d76) 7 (Core)

Before to run the above command you must update the grub configuration (grub2-mkconfig) because it may give you incorrect numbers.

I set the number 1, because I want the second kernel (with 'bak1' in the filename). This way I am sure that the initramfs image will not be rebuilt (zfs.ko is present in the 'bak1' version of the initramfs image):

[root@server4 ~]# grub2-set-default 1

Verify the default kernel:

# grub2-editenv list
saved_entry=1

Alternatively, we can use the name of the kernel (instead of the number):

# grub2-set-default "CentOS Linux (3.10.0-693.5.2.bak1.el7.x86_64) 7 (Core)"

Verify the default kernel:

# grub2-editenv list
saved_entry=CentOS Linux (3.10.0-693.5.2.bak1.el7.x86_64) 7 (Core)

Now we can safely reboot the system. It will boot with the kernel 3.10.0-693.5.2.bak1.el7.x86_64 (image file /boot/initramfs-3.10.0-693.5.2.bak1.el7.x86_64.img). Just in case I make another test:

# lsinitrd /boot/initramfs-3.10.0-693.5.2.bak1.el7.x86_64.img | grep zfs.ko
-rw-r--r--   1 root     root      3191696 Nov 14 21:12 usr/lib/modules/3.10.0-693.2.2.el7.x86_64/extra/zfs/zfs/zfs.ko
lrwxrwxrwx   1 root     root           58 Nov 14 21:11 usr/lib/modules/3.10.0-693.5.2.el7.x86_64/weak-updates/zfs/zfs/zfs.ko -> ../../../../3.10.0-693.2.2.el7.x86_64/extra/zfs/zfs/zfs.ko

And reboot:

# reboot

The reboot was successful. We verify that the default kernel was the one we selected:

# dmesg  | grep BOOT_IMAGE
[    0.000000] Command line: BOOT_IMAGE=/ROOT@/boot/vmlinuz-3.10.0-693.5.2.bak1.el7.x86_64 root=ZFS=rpool/ROOT ro rhgb quiet
[    0.000000] Kernel command line: BOOT_IMAGE=/ROOT@/boot/vmlinuz-3.10.0-693.5.2.bak1.el7.x86_64 root=ZFS=rpool/ROOT ro rhgb quiet