How to: Change to a Different Firmware and Kernel Combo

[Lolonois]

Synopsis: Sometimes it is needed to change the Raspbian kernel and
firmware because some releases do create a regression (for example in video
playback, audio drop outs, ..., see issue tracker at [0]). This post will
show you how to do change your kernel/firmware version without reflashing a
pristine RetroPie image.

Outline (aka Travel Guide):

1. Where am I?: What is your current firmware and kernel?
2. Finding the route where to go to: Selecting and applying a firmware and
   kernel revision
3. I like this place of my journey, I want to stay: Make a firmware kernel
   combo stick
4. Traveling with pets? No problem: Linux headers for your beloved kernel
   modules
5. Enjoy your journey: Firmware/kernel versions confirmed to be robust

NB: All commands to be executed on the command line (ssh to your
raspberry host).

1. Current Firmware and Kernel

• Command to display the kernel version: uname -r

• GPU Firmware date and internal revision: vcgencmd version. See [1]
  for discussion on versioning. TL;DR: You can only use the date (+/- 1
  day) to map to the commit dates in this repo [2] so it is a loose fit /
  educated guess this way around.

• In the next step you will see how to get from a kernel version to a
  firmware release matching that kernel: This is the other way around.

• Additional commands (handy if you seeking help here, not needed for this
  how to):

  • Echo the /boot/config.txt settings: vcgencmd get_config int
  • Bootprocess and status of firmware: sudo vcdbg log msg

2. Selecting and Applying a Firmware and Kernel Revision

To identify which firmware is matching your kernel (and is most likely
installed):

1. Remember your kernel version (see above)
2. Then head to this repo [3] and search for Bump to <your kernel version>
3. It should give you exactly one match in the commits
4. Note the commit value

Lookup Example:

• Kernel is (output from uname -r): 4.19.97-v7l+
• Then search for Bump to 4.19.97 (omit the suffix)
• Commit value is: 993f47507f287f5da56495f718c2d0cd05ccbc19

Now you know how to search for a different kernel and matching firmware.

Apply Example:

1. First and foremost: Have/make a full backup of the current system
2. Lets assume you want kernel 4.19.118 installed
3. Search for Bump to 4.19.118 in [3]
4. The commit string is: e1050e94821a70b2e4c72b318d6c6c968552e9a2
5. Then issue: sudo rpi-update e1050e94821a70b2e4c72b318d6c6c968552e9a2
6. Follow the on screen instructions
7. Reboot
8. After reboot issue uname -r: Should display kernel 4.9.118.
9. Smile

Extra:

If you want to dissect if an issue is caused by a specific firmware you can
keep your current kernel and only adjust the firmware. Obviously you should
not pick a firmware relating to a smaller kernel version than your current
kernel version. To keep your current kernel add SKIP_KERNEL=1, thus: sudo SKIP_KERNEL=1 rpi-update ...

3. Make a Firmware/Kernel Combo Stick

If you update the raspberry OS (apt full-update stance) chances are that
your working kernel/firmware combo gets also upgraded.

To avoid this to happen you can mark these packages to be held in the
current version. Do issue:

sudo apt-mark hold libraspberrypi-bin
sudo apt-mark hold raspberrypi-kernel
sudo apt-mark hold raspberrypi-bootloader
sudo apt-mark hold raspberrypi-kernel-headers
sudo apt-mark hold raspi-config
sudo apt-mark hold rpi-eeprom

Unmarking/unholding, if needed, can be done with sudo apt-mark unhold $(apt-mark showhold).

Personally, I refrain from using full-upgrade because it may update too
many packages and potentially break (but not brick) RetroPie. You may
consider aptitude safe-upgrade which does not do a big-bang upgrade. See
man page of aptitude after installing with sudo apt install aptitude or
look it up with your search engine of least distrust.

4. Linux Headers for Your Beloved Kernel Modules

Depending on what modules you have installed you may note that your kernel
modules (for example: mkarcadejoystick, or one of
those) do not get loaded as
before your kernel/firmware changes. You can verify if they are loaded with
lsmod | grep <part_of_modulename>, if you don't get a match and your input
device (joystick/gamepad) is not functioning, continue reading.

Again some commit IDs have to be mapped, but this time it can be done
automagically.

1. Navigate to https://github.com/RPi-Distro/rpi-source
2. Follow the instruction on how to install
3. Make sure you have booted into the desired kernel (uname -r is your
   friend)
4. Run sudo rpi-source -d /usr/src, this will automagically download and
   pre-configure the kernel headers for kernel of uname -r. These must
   match the current kernel version and are needed to compile the before
   mentioned kernel modules.
5. Let the process continue, you will get output on what is happening
6. Either it will stop because the package ncurses-devel is not
   installed or else exit (Ctrl-C) the process once the compile process
   has left the scripts/ folder
7. Now you should be able to compile and install your kernel module as
   usual
8. Recheck afterwards with lsmod | grep <part_of_modulename>

Epilogue: You will find a folder below /usr/src/ named
linux-<commitstring>. If you plan to download additional kernel header
files, take notes on the commitstring and to which kernel version it
relates. If you want to rerun rpi-source for a different kernel version
remove the symbolic link /usr/src/linux before doing so (sudo rm /usr/src/linux). Do so also when you want reuse different kernel headers,
in this case also review your notes and do run sudo ln -s /usr/src/linux-<commitstring> /usr/src/linux. Then compile your kernel
module(s).

5. Firmware/Kernel Versions Confirmed to be Robust

From my experience I recommend to try this order and kernel/firmware combos
if you have issues:

• 4.19.118 (stable)
• 5.4.83 (next best to stable). Note: You have to search for Bump to
  5.4.81, you will get two commit messages as match. The real kernel 5.4.83
  commit is 453e49bdd87325369b462b40e809d5f3187df21d, as can be verified
  in this file [4], dated Dec 14, 2020.
• 5.10.x (hit and miss)
• Update 2022-03: 5.10.x may have matured in the meantime. Did not try any kernel beyond 5.10 (yet).

That's it, have a lot of fun with your arcade setup. (:

References

[0] https://github.com/Hexxeh/rpi-firmware/issues?q=is%3Aissue+is%3Aclosed

[1] https://raspberrypi.stackexchange.com/questions/322/how-can-i-determine-the-current-firmware-version-number/323#323

[2] https://github.com/raspberrypi/firmware

[3] ~https://github.com/Hexxeh/rpi-firmware~ superseded by https://github.com/raspberrypi/rpi-firmware

[4] https://github.com/Hexxeh/rpi-firmware/blob/453e49bdd87325369b462b40e809d5f3187df21d/uname_string7