usb boot from 5tb hdd

Lolonois

@korn16ftl3 Looks correct, except you are missing one hex-number in the product ID (should be :25ee and not :5ee).

The quiet parameter is double but it does not have an impact: The kernel parameters are parsed left to right and the last one wins, if there are two identical parameters.

My new question is since we are playing with the cmdline.txt, what are the other parameters that are there?

The kernel parameters are documented in the kernel documentation, in the raspberry pi documentation, and the kernel parameter of plymouth and info about plymouth itself are documented for example in the Arch wiki,

korn16ftl3

@Lolonois said in usb boot from 5tb hdd:

@korn16ftl3 Looks correct, except you are missing one hex-number in the product ID (should be :25ee and not :5ee).

The quiet parameter is double but it does not have an impact: The kernel parameters are parsed left to right and the last one wins, if there are two identical parameters.

My new question is since we are playing with the cmdline.txt, what are the other parameters that are there?

The kernel parameters are documented in the kernel documentation, in the raspberry pi documentation, and the kernel parameter of plymouth and info about plymouth itself are documented for example in the Arch wiki,

i made the previously mentioned changes to the cmdline.txt and am getting the same error readouts.

im starting to think/wonder if the GPT portion of thi is what is causing some of my problems as retropie its self if i understand correctly was written for MBR and everything boots just fine in MBR and GPT when everything is <2tb the second i convert the GPT partition manually (before initial retropie boot and partition resize) everything ceases toi load/boot.

im going to remove the

quiet init=/usr/lib/raspi-config/init_resize.sh

from the cmdline.txt since perhaps it is hanging on attempt to resize a partition that has already been resized for one and for two is GPT and not sure how to handle it.

lets see what that does i suppose.

also debating on wiping everything abd starting over using gparted and specifically raspberry pi os to try to set tis up as well maybe that ii yield different results.

will report back after a little bit of experimenting and a bit more research.

alternatively i have seen some things online that enable UEFI booting ( [https://www.linux.it/~ema/posts/secure-boot-rpi/](link url) , [https://github.com/pftf/RPi4](link url) ) would this be any help with trying to boot from >2TB partitions as UEFI booting supports it? would there be any drawbacks or risks to experimenting with these options?

korn16ftl3

So I went back to tinkering on things today and rather than trying to get retropie it’s self to load from the usb hdd I tried to get raspberry pi OS X64 lite to run since this is the base of retropie and I would then have the option to crate my own retropie from it if I were able to get it working.

I deleted all partitions on the 12tb hdd and used the raspberry pi image program to write the image.

I changed the drive from MBR to GPT on a windows computer then loaded raspberry pi OS X64 full from the SD card and used it to grow the root partition passed the original 2tb and attempted to boot the system, as per usual this failed and gave me the following error:

[     0.953845] mmc1: controller never released inhibit bit(s) 

Give up waiting for root file system device.  Common problems: 
- Check rootdelay= (did the system wait long enough?)
- Missing modules (cat /proc/modules: last /dev)

ALERT!  PARTUUID=d9c86127-02 does not exist. Dropping to a shell!

Busybox v1.35.0 (Debian 1:1.35.0-4+b3) built-in shell (ash)

Enter ‘help’ for a list of built in commands.

(Initramfs) _

What I’m failing to understand is isn’t mmc1 the SD card? I have flashed the usb booting information to the pi so why is it trying to boot from the sd card?

That first line is also familiar from the attempted retropie installs.

I’m also reading an article over at the raspberry pi forums about booting from usb as well looking for other ideas and things to try: https://forums.raspberrypi.com/viewtopic.php?t=196778

I’m not sure how I would run those programs from within retropie itself. I’m thinking download the zip in that thread, unpack on a windows system in the root of the usb stick, and browsing to the usb drive from within retropie itself and trying things that way.

Just wanted to let everyone know where I was at with this. It seems things start to go defunct every time I go past the 2tb mark with the root partitions as retropie, and other OS’s load just fine when the drive is in either MBR or GPT but the second I grow the root passed that 2tb line it looses functionality for whatever reason.

Another viable idea would be to install retropie on the USB hdd in MBR, grow retropie’s root partition to lets say 1tb for system files, game media and so on, convert to GPT, partition the remaining unallocated space for rom storage. Would this have any negative effect on performance however?

mitu

@korn16ftl3 said in usb boot from 5tb hdd:

I changed the drive from MBR to GPT on a windows computer then loaded raspberry pi OS X64 full from the SD card and used it to grow the root partition passed the original 2tb and attempted to boot the system, as per usual this failed and gave me the following error:

The error seem to indicate the a partiion ID has changed and the kernel cannot find the partition (seems it's the first partition). You may want to check /etc/fstab and compare the partition IDs from it with the ones present on the disk (i.e. by running sudo lsblk which would show the partitions present on all disks and their IDs).

What I’m failing to understand is isn’t mmc1 the as card? I have flashed the usb booting information to the pi so why is it trying to boot from the sd card?

Those are the controller driver messages, the fact that you got so far with the boot means that the kernel and initramfs have been loaded from the disk, so the boot has been succesful.

... Would this have any negative effect on performance however?

No.

korn16ftl3

@mitu said in usb boot from 5tb hdd:

The error seem to indicate the a partiion ID has changed and the kernel cannot find the partition (seems it's the first partition). You may want to check /etc/fstab and compare the partition IDs from it with the ones present on the disk (i.e. by running sudo lsblk which would show the partitions present on all disks and their IDs).

the fstab file reads the following:

proc            /proc           proc    defaults          0       0
PARTUUID=d9c86127-01  /boot/firmware  vfat    defaults          0       2
PARTUUID=d9c86127-02  /               ext4    defaults,noatime  0       1

the output of sudo lsblk :

NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
sda           8:0    0  10.9T  0 disk
├─sda1        8:1    0   512M  0 part /media/pi/bootfs
└─sda2        8:2    0  10.9T  0 part /media/pi/rootfs
mmcblk0     179:0    0 476.7G  0 disk
├─mmcblk0p1 179:1    0   512M  0 part /boot/firmware
└─mmcblk0p2 179:2    0 476.2G  0 part /

upon tinkering with gdisk (no write actions taken i quit after getting the output) i simply type

sudo gdisk /dev/sda

and get the following information as well in regards to the partitions:

GPT fdisk (gdisk) version 1.0.9

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.

im not sure what MBR protective is but im wondering if thats the tiny greyed out partition at the beginning of the hdd i cant delete or change when looking at the hdd in gparted

@mitu said in usb boot from 5tb hdd:

Those are the controller driver messages, the fact that you got so far with the boot means that the kernel and initramfs have been loaded from the disk, so the boot has been succesful.

im wondering if whatever the issue is with my raspberry pi OS X64 lite install is if that is the same issue with my retropie install as well after growing the root partition.

mitu

@korn16ftl3 said in usb boot from 5tb hdd:

the fstab file reads the following:
...
the output of sudo lsblk :

Sorry, the command to show the partition ID's is sudo blkid. IMHO, you should use gparted for converting the partition table to GPT instead of using Windows to do it. If putting the right partition ID in /etc/fstab doesn't fix your issue, that would be the next thing to try.

korn16ftl3

@mitu said in usb boot from 5tb hdd:

the command to show the partition ID's is sudo blkid. IMHO

the read out for sudo blkid:

/dev/mmcblk0p1: LABEL_FATBOOT="bootfs" LABEL="bootfs" UUID="F737-8E10" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="90ace319-01"
/dev/mmcblk0p2: LABEL="rootfs" UUID="d6ecfcd5-2703-41bf-9301-10c403b6fb0c" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="90ace319-02"
/dev/sda2: LABEL="rootfs" UUID="d4cc7d63-da78-48ad-9bdd-64ffbba449a8" BLOCK_SIZE="4096" TYPE="ext4" PARTLABEL="Basic data partition" PARTUUID="0000046b-eba0-706e-cf40-dc01d6080000"
/dev/sda1: LABEL_FATBOOT="bootfs" LABEL="bootfs" UUID="EC36-4DE1" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="Basic data partition" PARTUUID="701069d0-00cf-01dc-e814-08b86700ee00"

@mitu said in usb boot from 5tb hdd:

IMHO, you should use gparted for converting the partition table to GPT instead of using Windows to do it.

I've tried this but for some reason (even when in MBR) when i try to make edits to the partition tables it doesn't want to let me. i launch gparted from the console with sudo gparted however when i right click the partitions to edit or delete them it wont let me click any options or use any of the buttons on the tops of the GUI.

im willing to let someone remote desktop access my PI OS x64 and have a look around for themselves as its no security risk I'm really only using it for a bootable os to have a look at things and try to sort this mess out.

mitu

@korn16ftl3 said in usb boot from 5tb hdd:

/dev/sda2: LABEL="rootfs" UUID="d4cc7d63-da78-48ad-9bdd-64ffbba449a8" BLOCK_SIZE="4096" TYPE="ext4" PARTLABEL="Basic data partition" PARTUUID="0000046b-eba0-706e-cf40-dc01d6080000"

The PARTUUID of the disk is not the one you have in /etc/fstab - the one in /etc/fstab actually refers to the partition from the uSD card.

Btw, did you post the output of /etc/fstab from the uSD card or the one from the disk ? Check that the /etc/fstab on the disk contains the correct Partition UUIDs - the ones from /dev/sda1 and /dev/sda as shown by the blkid command and ensure that the cmdline.txt from the disk has the right partition ID (of /dev/sda2) also.

korn16ftl3

@mitu said in usb boot from 5tb hdd:

@korn16ftl3 said in usb boot from 5tb hdd:

/dev/sda2: LABEL="rootfs" UUID="d4cc7d63-da78-48ad-9bdd-64ffbba449a8" BLOCK_SIZE="4096" TYPE="ext4" PARTLABEL="Basic data partition" PARTUUID="0000046b-eba0-706e-cf40-dc01d6080000"

The PARTUUID of the disk is not the one you have in /etc/fstab - the one in /etc/fstab actually refers to the partition from the uSD card.

Btw, did you post the output of /etc/fstab from the uSD card or the one from the disk ? Check that the /etc/fstab on the disk contains the correct Partition UUIDs - the ones from /dev/sda1 and /dev/sda as shown by the blkid command and ensure that the cmdline.txt from the disk has the right partition ID (of /dev/sda2) also.

I opened the fstab in a txt editor from the hard drive it's self from the /etc/fstab rather than running any commands in the shell to receive an output so the fstab should be for the hdd

I will do a cli readout when I return home a little later.

It's looking like I need to fix the cmdline.txt and the fstab on the hdd like u are suggesting. I will post pre edit and post edits of everything I change to double check the correctness of it.

If I'm understanding correctly the cmdline.txt (located in the boot partition) the fstab (located in /etc) must all have matching UUID'S for the corresponding partitions.

korn16ftl3

so having a look here to see what matches and what to replace in the fstab and what I'm seeing is the following:

/dev/mmcblk0p1: LABEL_FATBOOT="bootfs" LABEL="bootfs" UUID="F737-8E10" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="90ace319-01"
/dev/mmcblk0p2: LABEL="rootfs" UUID="d6ecfcd5-2703-41bf-9301-10c403b6fb0c" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="90ace319-02"

those would be the SD card labels if im not mistaken and you had said:
@mitu said in usb boot from 5tb hdd:

The PARTUUID of the disk is not the one you have in /etc/fstab - the one in /etc/fstab actually refers to the partition from the uSD card.

however i am not seeing the PARTUUID d9c86127 anywhere in the mmcblk0 read outs. I'm curious how you came to that conclusion.

do i change the entire PARTUUID in the fstab to match the ones in the blkid readout:
i.e.
from this in fstab:

PARTUUID=d9c86127-01  /boot/firmware  vfat    defaults          0       2

to this:

PARTUUID=701069d0-00cf-01dc-e814-08b86700ee00-01  /boot/firmware  vfat    defaults          0       2

I'm assuming that the -01 at the very end directs it to partition 1 on the hdd as it is labeled /boot/firmware under the proc column.

lastly i should change the cmdline.txt from this:

code_textconsole=serial0,115200 console=tty1 root=PARTUUID=d9c86127-02 rootfstype=ext4 fsck.repair=yes rootwait quiet init=/usr/lib/raspberrypi-sys-mods/firstboot cfg80211.ieee80211_regdom=NA systemd.run=/boot/firstrun.sh systemd.run_success_action=reboot systemd.unit=kernel-command-line.target

to this:

console=serial0,115200 console=tty1 root=PARTUUID=701069d0-00cf-01dc-e814-08b86700ee00-02 rootfstype=ext4 fsck.repair=yes rootwait quiet init=/usr/lib/raspberrypi-sys-mods/firstboot cfg80211.ieee80211_regdom=NA systemd.run=/boot/firstrun.sh systemd.run_success_action=reboot systemd.unit=kernel-command-line.target

what kind of throws me for a loop is the fact that the HDD PARTUUID has so many more characters than the original PARTUUID. Is there any particular reason for this?

UPDATE:
tried the above changes an got a message from the system stating something about running blkid or something to that effect. the message kept appearing.

im going to switch back to the 5tb drve and start fresh, so i can pick up work on the 12tb drive later sine we have established the problem is not the drive(s) just the way the system is handling them.

this time i suppose its time to leave my comfort zone a little more and try doing everything within linux and on the pi itself.

im going to start by tinkering around with th tools talked about here:
https://forums.raspberrypi.com/viewtopic.php?t=196778

i will keep this thread up to date with things i have tried and if i succeed or not because i personally hate when people dont follow up with their threads for other readers that may be having the same issues or attempting the same things and aren't sure what to do or where to go.

mitu

@korn16ftl3 said in usb boot from 5tb hdd:

however i am not seeing the PARTUUID d9c86127 anywhere in the mmcblk0 read outs. I'm curious how you came to that conclusion.

There's no mmcblk0 partition and I didn't mention any d9c86127 in my reply, I just mentioned that the Paritition UUID for the uSD card partitions (that you posted):

/dev/mmcblk0p1: LABEL_FATBOOT="bootfs" LABEL="bootfs" UUID="F737-8E10" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="90ace319-01"
/dev/mmcblk0p2: LABEL="rootfs" UUID="d6ecfcd5-2703-41bf-9301-10c403b6fb0c" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="90ace319-02"

are the Partition UUIDs that also appear in /etc/fstab.

In the /etc/fstab file from the hard disk (not uSD), it's the Partition UUIDs from /dev/sda[1|2] that should be used, not the ones from the uSD - the error you got with the kernel not finding the Partition UUID is caused by a mismatch in the UUID(s).

korn16ftl3

so in some more googling i found this:
https://raspberrypi.stackexchange.com/questions/120525/is-it-possible-to-boot-a-raspberry-pi-4-from-a-gpt-usb-connected-disk

in one of the reply's it says:
"By the way, you will have to update your /boot/cmdline.txt and /etc/fstab files, as your partitions will have different PARTUUID values. Also you have to change the boot order with raspi-config.

I recommend you set up /dev/sdX values in those files and then boot the raspberry and update the values with the right PARTUUID values. I tried to use the PARTUUID values I got when the disk was attached to my laptop, but they are different, for some reason."

what does he mean by reccomending that you set up /dev/sdX in those files? i know X is a variable (sda, sdb, whatever the external storage device is assigned) but is he suggesting i add those in place of the PARTUUID value in case of a bad readout, as he reports having an issue with?

example:

code_textconsole=serial0,115200 console=tty1 root=PARTUUID=/dev/sdX rootfstype=ext4 fsck.repair=yes rootwait quiet init=/usr/lib/raspberrypi-sys-mods/firstboot cfg80211.ieee80211_regdom=NA systemd.run=/boot/firstrun.sh systemd.run_success_action=reboot systemd.unit=kernel-command-line.target

Lolonois

@korn16ftl3 the legacy format is root=/dev/sda1 (or whatever drive and partition holds / (the root filesystem). Same format a la /dev/sda1 goes for /etc/fstab.

FWIW, there is a good reason why the legacy format is discouraged: https://wiki.archlinux.org/title/Persistent_block_device_naming

korn16ftl3

@Lolonois said in usb boot from 5tb hdd:

@korn16ftl3 the legacy format is root=/dev/sda1 (or whatever drive and partition holds / (the root filesystem). Same format a la /dev/sda1 goes for /etc/fstab.

FWIW, there is a good reason why the legacy format is discouraged: https://wiki.archlinux.org/title/Persistent_block_device_naming

I haven’t read that link yet but the sda and sdb are all determined by the order of of detection at boot up or order of being plugged in I’m not mistaken so i can understand how this is discouraged and no longer the way to do things.

On a positive note I did get this all working with BOTH the original 5tb and the later 12tb hard drives. The solution…..so it all in Linux. I’ll write a guide here in this thread to help others and maybe a separate one individually so it can be stickied and not follow this trail of confusing stuff?

Long story short I/we were on the right patch to begin with…just don’t use windows to change to gpt for some reason that screw’s it all up.

Simple rundown is as follows:
Write SD boot firmware to SD card with pi imager> boot SD boot firmware sd card in pi >Write retropie image to 2+tb device > write raspberry pi OS to micro SD card > boot raspberry pi OS from SD > ensure all updates and such (use the config editor with the raspberry pi imager to speed up a few things like ssh and WiFi setup) > ensure GParted and gdisk are installed > use gdisk to change hard retropie drive to GPT > use GParted to expand the root file system > check the retropie partitions partuuid in the cmd prompt > edit the cmdline.txt partuuid to match the ones you just looked up > mount the root drive for the retropie storage device with cmd prompts > load the FStab with nano > edit both partitions partuuid to the corresponding ones you just found > unmount > attempt boot

It does give some errors upon the first boot after this process and scans the hard drive and says it’s in emergency mode because it can’t boot the kernel. Just let the hard drive finish its scan and then when you can’t do anything more (it won’t even do anything if you hit enter to continue) just unplug the pi and plug it back in everything should boot accordingly and be fine after that.

Lolonois

@korn16ftl3 Thanks for the heads up and hints. Happy retro-gaming.

korn16ftl3

@Lolonois said in usb boot from 5tb hdd:

@korn16ftl3 Thanks for the heads up and hints. Happy retro-gaming.

It should also be noted to check and see if you have an energy saver in the firmware for your HDD. Mine did and it interfered with it somehow and gave me problems after a shit down and did some weird stuff…it eventually straightened its self out HOWEVER if you have the energy saver “feature” on your particular hard drive it should be disabled if you plan to leave the system on for long periods of time. I was able to do this with a windows pc using the WD utilities software. Others may have to seek other solutions to keep the drive from entering sleep mode.

I must say that even tho this whole thing was frustrating it was fun at the same time and felt accomplishing once I actually got the results I was after. Oh and I learned a lot about Linux along the way besides the little I already knew.

korn16ftl3

Quick question I'm hoping someone has an answer to.

After getting this far I'm so close to having this perfect I can taste it.

Below is a photo of my start up that I get, it seems almost random.

One of the guides I read about moving to a drive (the guide was for an SSD whit I am not using) said to add I think it was a wait=10 I'm the cmdline.txt (I'd have to look at my cmdline.txt to tell u exactly what it was).

Is that what is causing this odd boot up I keep getting?
If not what is or how can I fix it?

https://ibb.co/xtN52zF0