usb boot from 5tb hdd

[Lolonois]

@korn16ftl3 good you dug up some options.

Just to cross other issues out: The first Pi4 revision has an issue with some quality USB chargers. Check the output of cat /sys/firmware/devicetree/base/model, if it shows board revision 1.1 you may need an official Pi4 USB-C power supply. Otherwise you are on the safe side.

The mmc1 controller never released inhibit bits is most likely not related with the USB drive issue. Most likely you can mute it by inserting a blank sd card (no partition on it). [1]

The USB WD Drive has a custom chipset (see discussion in that thread [2]), which may cause the issue in context with the Raspberry Pi. You can try some USB quirks to mitigate the issue.
May be you can use t as flag (disables some UAS commands which may help already), but at least u should remediate the situation.
(add either to cmdline.txt) (see usb-storage.quirks in [3])

Last but not least you can detect which mode (Driver=uas or Driver=usb-storage) the drive uses with lsusb -t (look for the USB drive name in the output).

[1] https://forums.raspberrypi.com/viewtopic.php?t=351746#p2123679
[2] https://community.wd.com/t/5-tb-mypassport-drive-only/242638/2
[3] https://www.kernel.org/doc/Documentation/admin-guide/kernel-parameters.txt

[korn16ftl3]

@Lolonois
Thanks for some more directions to go, as yours seem WD targeted i will begin with them in my troubleshooting process.

I was unaware that there was a particular issue with some pi4 models that forced me to use an official pi4 power supply. Perhaps I should attempt this first to resolve my issues or start here to eliminate the possibility.

I have just received my external usb enclosure and have shucked the drive. To my disappointment the drive has no data interface and in fact has an integrated USB interface.

I suppose to proceed with even considering putting this in an external sata enclosure I would have to match this hdd model # with another that has the actual sata interface pcb to use as a donor for this drive. I’m not even sure how well this would work but in theory it should. Does anyone have any other possible work arounds for this issue? I don’t even know when they started doing this, it used to be buy an external shuck it and put it in whatever device you chose.

Does anyone else have any input thoughts or ideas with this?

This has quickly gotten way over my head….maybe I will just buy a different hdd and move on I don’t know yet.

[korn16ftl3]

ok as an update:

since i started tinkering with this and realized perhaps this is the controller interface on the USB HDD and that the usb portion was integrated to the HDD its self i decided to get a 3.5 12TB WD mybook external HDD. this assures there is appropriate power as well as that i am able to actually shuck the drive from the enclosure if i so choose.

Upon trying a different drive i have attempted some of the steps above that i have already attempted with the other drive with the same results. so far this leads me to believe this has to do with booting from larger that 2TB drives (GPT vs MBR) as was assumed from the beginning of this thread.

once I started getting similar results from the previous drive i started dabbeling with the options that @Lolonois presented me with.

im not exactly certain what to make of the readout of the lsusb -t but i am going to post a picture of it and perhaps someone can help me.

Im going to set up SSH and boot raspberry pi OS with only my HDD plugged in to the usb ports so at least I know the readouts are only for the USB HDD and nothing else. This will perhaps eliminate some of the confusion with the readout (at lest for me)

I do know (and its also noted in the photo of my read out) that my rpi 4 is a board v1.4 so I should be safe from the previously mentions flaw in board rev. 1.1.

i started reading threw some of the usb-storage.quirks and am making a bit of sense out of it. I'm not exactly certain what to do with the information.

the section of quirks i am reading is:

usb-storage.quirks=
			[UMS] A list of quirks entries to supplement or
			override the built-in unusual_devs list.  List
			entries are separated by commas.  Each entry has
			the form VID:PID:Flags where VID and PID are Vendor
			and Product ID values (4-digit hex numbers) and
			Flags is a set of characters, each corresponding
			to a common usb-storage quirk flag as follows:
				a = SANE_SENSE (collect more than 18 bytes
					of sense data, not on uas);
				b = BAD_SENSE (don't collect more than 18
					bytes of sense data, not on uas);
				c = FIX_CAPACITY (decrease the reported
					device capacity by one sector);
				d = NO_READ_DISC_INFO (don't use
					READ_DISC_INFO command, not on uas);
				e = NO_READ_CAPACITY_16 (don't use
					READ_CAPACITY_16 command);
				f = NO_REPORT_OPCODES (don't use report opcodes
					command, uas only);
				g = MAX_SECTORS_240 (don't transfer more than
					240 sectors at a time, uas only);
				h = CAPACITY_HEURISTICS (decrease the
					reported device capacity by one
					sector if the number is odd);
				i = IGNORE_DEVICE (don't bind to this
					device);
				j = NO_REPORT_LUNS (don't use report luns
					command, uas only);
				k = NO_SAME (do not use WRITE_SAME, uas only)
				l = NOT_LOCKABLE (don't try to lock and
					unlock ejectable media, not on uas);
				m = MAX_SECTORS_64 (don't transfer more
					than 64 sectors = 32 KB at a time,
					not on uas);
				n = INITIAL_READ10 (force a retry of the
					initial READ(10) command, not on uas);
				o = CAPACITY_OK (accept the capacity
					reported by the device, not on uas);
				p = WRITE_CACHE (the device cache is ON
					by default, not on uas);
				r = IGNORE_RESIDUE (the device reports
					bogus residue values, not on uas);
				s = SINGLE_LUN (the device has only one
					Logical Unit);
				t = NO_ATA_1X (don't allow ATA(12) and ATA(16)
					commands, uas only);
				u = IGNORE_UAS (don't bind to the uas driver);
				w = NO_WP_DETECT (don't test whether the
					medium is write-protected).
				y = ALWAYS_SYNC (issue a SYNCHRONIZE_CACHE
					even if the device claims no cache,

the example given is by the link:
quirks=0419:aaf5:rl,0421:0433:rc

now if I understand correctly the rl after the 0419:aaf5 for example are the quirks activate but what/were does the 0419:aaf5 come from? I'm guessing it is a device ID of some sort? how do i get this device ID as I did not see anything like it from the lsusb -t command (again see the photo)

I do know where to put these lines as I'm assuming there is a cmdline.txt in the fat/boot portion of the HDD/SD card where I add them and again guessing I add them at the very end of the txt file.

Anyway I'm back at this again and have not given up......just decided to go a different direction to try to mitigate some potential issues and give more more options than the propitiatory USB interface on the previous USB HDD offered.

where to go from here.

i know booting from a large hdd is possible I'm just not certain how to achieve it just yet.

here is a link to the photo:
https://ibb.co/DP1sjXvb

[Lolonois]

@korn16ftl3 ah my bad, I guess I should have added -v to lsusb. You may run again and keep only Mass Storage devices, e.g. lsusb -vt | grep -A1 "Mass Storage"

Which will output something like:

        |__ Port 3: Dev 11, If 0, Class=Mass Storage, Driver=uas, 480M
            ID 152d:0583 JMicron Technology Corp. / JMicron USA Technology Corp. JMS583Gen 2 to PCIe Gen3x2 Bridge

The hexadecimal numbers right after ID are the USB Vendor ID and Product ID. Use these to add the usbquirks in the cmdline.txt in the /boot (or /boot/firmware folder). Make sure to add it at the same line (no newline) and only use spaces before the whole kernel parameter and value, e.g.:

<existing cmdline.txt line> usb-storage.quirks=<vendorId>:<productId>:t 

<existing cmdline.txt line> usb-storage.quirks=<vendorId>:<productId>:u

You may try first with the flag/quirk :t and check if it remediates the situation or jump right to :u, which will most likely -judging from the distance- remediate the situation.

[korn16ftl3]

@Lolonois said in usb boot from 5tb hdd:

@korn16ftl3 ah my bad, I guess I should have added -v to lsusb. You may run again and keep only Mass Storage devices, e.g. lsusb -vt | grep -A1 "Mass Storage"

Which will output something like:

        |__ Port 3: Dev 11, If 0, Class=Mass Storage, Driver=uas, 480M
            ID 152d:0583 JMicron Technology Corp. / JMicron USA Technology Corp. JMS583Gen 2 to PCIe Gen3x2 Bridge

The hexadecimal numbers right after ID are the USB Vendor ID and Product ID. Use these to add the usbquirks in the cmdline.txt in the /boot (or /boot/firmware folder). Make sure to add it at the same line (no newline) and only use spaces before the whole kernel parameter and value, e.g.:

<existing cmdline.txt line> usb-storage.quirks=<vendorId>:<productId>:t 

<existing cmdline.txt line> usb-storage.quirks=<vendorId>:<productId>:u

You may try first with the flag/quirk :t and check if it remediates the situation or jump right to :u, which will most likely -judging from the distance- remediate the situation.

ok so now i have something that makes sense :-)

this is now my readout for the drive i am trying to boot retropie from:
https://ibb.co/0jXRnDNR

now to follow the rest of your instructions:

my original cmdline.txt is as follows:

console=serial0,115200 console=tty1 root=PARTUUID=3de64a5b-02 rootfstype=ext4 fsck.repair=yes rootwait quiet loglevel=3 consoleblank=0 plymouth.enable=0 quiet init=/usr/lib/raspi-config/init_resize.sh

if I'm understanding what the instructions are it should be this for the quirks:

console=serial0,115200 console=tty1 root=PARTUUID=3de64a5b-02 rootfstype=ext4 fsck.repair=yes rootwait quiet loglevel=3 consoleblank=0 plymouth.enable=0 quiet init=/usr/lib/raspi-config/init_resize.sh usb-storage.quirks=1058:5ee:u

My new question is since we are playing with the cmdline.txt, what are the other parameters that are there?

[Lolonois]

@korn16ftl3 Looks correct, except you are missing one hex-number in the product ID (should be :25ee and not :5ee).

The quiet parameter is double but it does not have an impact: The kernel parameters are parsed left to right and the last one wins, if there are two identical parameters.

My new question is since we are playing with the cmdline.txt, what are the other parameters that are there?

The kernel parameters are documented in the kernel documentation, in the raspberry pi documentation, and the kernel parameter of plymouth and info about plymouth itself are documented for example in the Arch wiki,

[korn16ftl3]

@Lolonois said in usb boot from 5tb hdd:

@korn16ftl3 Looks correct, except you are missing one hex-number in the product ID (should be :25ee and not :5ee).

The quiet parameter is double but it does not have an impact: The kernel parameters are parsed left to right and the last one wins, if there are two identical parameters.

My new question is since we are playing with the cmdline.txt, what are the other parameters that are there?

The kernel parameters are documented in the kernel documentation, in the raspberry pi documentation, and the kernel parameter of plymouth and info about plymouth itself are documented for example in the Arch wiki,

i made the previously mentioned changes to the cmdline.txt and am getting the same error readouts.

im starting to think/wonder if the GPT portion of thi is what is causing some of my problems as retropie its self if i understand correctly was written for MBR and everything boots just fine in MBR and GPT when everything is <2tb the second i convert the GPT partition manually (before initial retropie boot and partition resize) everything ceases toi load/boot.

im going to remove the

quiet init=/usr/lib/raspi-config/init_resize.sh

from the cmdline.txt since perhaps it is hanging on attempt to resize a partition that has already been resized for one and for two is GPT and not sure how to handle it.

lets see what that does i suppose.

also debating on wiping everything abd starting over using gparted and specifically raspberry pi os to try to set tis up as well maybe that ii yield different results.

will report back after a little bit of experimenting and a bit more research.

alternatively i have seen some things online that enable UEFI booting ( [https://www.linux.it/~ema/posts/secure-boot-rpi/](link url) , [https://github.com/pftf/RPi4](link url) ) would this be any help with trying to boot from >2TB partitions as UEFI booting supports it? would there be any drawbacks or risks to experimenting with these options?

[korn16ftl3]

So I went back to tinkering on things today and rather than trying to get retropie it’s self to load from the usb hdd I tried to get raspberry pi OS X64 lite to run since this is the base of retropie and I would then have the option to crate my own retropie from it if I were able to get it working.

I deleted all partitions on the 12tb hdd and used the raspberry pi image program to write the image.

I changed the drive from MBR to GPT on a windows computer then loaded raspberry pi OS X64 full from the SD card and used it to grow the root partition passed the original 2tb and attempted to boot the system, as per usual this failed and gave me the following error:

[     0.953845] mmc1: controller never released inhibit bit(s) 

Give up waiting for root file system device.  Common problems: 
- Check rootdelay= (did the system wait long enough?)
- Missing modules (cat /proc/modules: last /dev)

ALERT!  PARTUUID=d9c86127-02 does not exist. Dropping to a shell!

Busybox v1.35.0 (Debian 1:1.35.0-4+b3) built-in shell (ash)

Enter ‘help’ for a list of built in commands.

(Initramfs) _

What I’m failing to understand is isn’t mmc1 the SD card? I have flashed the usb booting information to the pi so why is it trying to boot from the sd card?

That first line is also familiar from the attempted retropie installs.

I’m also reading an article over at the raspberry pi forums about booting from usb as well looking for other ideas and things to try: https://forums.raspberrypi.com/viewtopic.php?t=196778

I’m not sure how I would run those programs from within retropie itself. I’m thinking download the zip in that thread, unpack on a windows system in the root of the usb stick, and browsing to the usb drive from within retropie itself and trying things that way.

Just wanted to let everyone know where I was at with this. It seems things start to go defunct every time I go past the 2tb mark with the root partitions as retropie, and other OS’s load just fine when the drive is in either MBR or GPT but the second I grow the root passed that 2tb line it looses functionality for whatever reason.

Another viable idea would be to install retropie on the USB hdd in MBR, grow retropie’s root partition to lets say 1tb for system files, game media and so on, convert to GPT, partition the remaining unallocated space for rom storage. Would this have any negative effect on performance however?

[mitu]

@korn16ftl3 said in usb boot from 5tb hdd:

I changed the drive from MBR to GPT on a windows computer then loaded raspberry pi OS X64 full from the SD card and used it to grow the root partition passed the original 2tb and attempted to boot the system, as per usual this failed and gave me the following error:

The error seem to indicate the a partiion ID has changed and the kernel cannot find the partition (seems it's the first partition). You may want to check /etc/fstab and compare the partition IDs from it with the ones present on the disk (i.e. by running sudo lsblk which would show the partitions present on all disks and their IDs).

What I’m failing to understand is isn’t mmc1 the as card? I have flashed the usb booting information to the pi so why is it trying to boot from the sd card?

Those are the controller driver messages, the fact that you got so far with the boot means that the kernel and initramfs have been loaded from the disk, so the boot has been succesful.

... Would this have any negative effect on performance however?

No.

[korn16ftl3]

@mitu said in usb boot from 5tb hdd:

The error seem to indicate the a partiion ID has changed and the kernel cannot find the partition (seems it's the first partition). You may want to check /etc/fstab and compare the partition IDs from it with the ones present on the disk (i.e. by running sudo lsblk which would show the partitions present on all disks and their IDs).

the fstab file reads the following:

proc            /proc           proc    defaults          0       0
PARTUUID=d9c86127-01  /boot/firmware  vfat    defaults          0       2
PARTUUID=d9c86127-02  /               ext4    defaults,noatime  0       1

the output of sudo lsblk :

NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
sda           8:0    0  10.9T  0 disk
├─sda1        8:1    0   512M  0 part /media/pi/bootfs
└─sda2        8:2    0  10.9T  0 part /media/pi/rootfs
mmcblk0     179:0    0 476.7G  0 disk
├─mmcblk0p1 179:1    0   512M  0 part /boot/firmware
└─mmcblk0p2 179:2    0 476.2G  0 part /

upon tinkering with gdisk (no write actions taken i quit after getting the output) i simply type

sudo gdisk /dev/sda

and get the following information as well in regards to the partitions:

GPT fdisk (gdisk) version 1.0.9

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.

im not sure what MBR protective is but im wondering if thats the tiny greyed out partition at the beginning of the hdd i cant delete or change when looking at the hdd in gparted

@mitu said in usb boot from 5tb hdd:

Those are the controller driver messages, the fact that you got so far with the boot means that the kernel and initramfs have been loaded from the disk, so the boot has been succesful.

im wondering if whatever the issue is with my raspberry pi OS X64 lite install is if that is the same issue with my retropie install as well after growing the root partition.

[mitu]

@korn16ftl3 said in usb boot from 5tb hdd:

the fstab file reads the following:
...
the output of sudo lsblk :

Sorry, the command to show the partition ID's is sudo blkid. IMHO, you should use gparted for converting the partition table to GPT instead of using Windows to do it. If putting the right partition ID in /etc/fstab doesn't fix your issue, that would be the next thing to try.

[korn16ftl3]

@mitu said in usb boot from 5tb hdd:

the command to show the partition ID's is sudo blkid. IMHO

the read out for sudo blkid:

/dev/mmcblk0p1: LABEL_FATBOOT="bootfs" LABEL="bootfs" UUID="F737-8E10" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="90ace319-01"
/dev/mmcblk0p2: LABEL="rootfs" UUID="d6ecfcd5-2703-41bf-9301-10c403b6fb0c" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="90ace319-02"
/dev/sda2: LABEL="rootfs" UUID="d4cc7d63-da78-48ad-9bdd-64ffbba449a8" BLOCK_SIZE="4096" TYPE="ext4" PARTLABEL="Basic data partition" PARTUUID="0000046b-eba0-706e-cf40-dc01d6080000"
/dev/sda1: LABEL_FATBOOT="bootfs" LABEL="bootfs" UUID="EC36-4DE1" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="Basic data partition" PARTUUID="701069d0-00cf-01dc-e814-08b86700ee00"

@mitu said in usb boot from 5tb hdd:

IMHO, you should use gparted for converting the partition table to GPT instead of using Windows to do it.

I've tried this but for some reason (even when in MBR) when i try to make edits to the partition tables it doesn't want to let me. i launch gparted from the console with sudo gparted however when i right click the partitions to edit or delete them it wont let me click any options or use any of the buttons on the tops of the GUI.

im willing to let someone remote desktop access my PI OS x64 and have a look around for themselves as its no security risk I'm really only using it for a bootable os to have a look at things and try to sort this mess out.

[mitu]

@korn16ftl3 said in usb boot from 5tb hdd:

/dev/sda2: LABEL="rootfs" UUID="d4cc7d63-da78-48ad-9bdd-64ffbba449a8" BLOCK_SIZE="4096" TYPE="ext4" PARTLABEL="Basic data partition" PARTUUID="0000046b-eba0-706e-cf40-dc01d6080000"

The PARTUUID of the disk is not the one you have in /etc/fstab - the one in /etc/fstab actually refers to the partition from the uSD card.

Btw, did you post the output of /etc/fstab from the uSD card or the one from the disk ? Check that the /etc/fstab on the disk contains the correct Partition UUIDs - the ones from /dev/sda1 and /dev/sda as shown by the blkid command and ensure that the cmdline.txt from the disk has the right partition ID (of /dev/sda2) also.

[korn16ftl3]

@mitu said in usb boot from 5tb hdd:

@korn16ftl3 said in usb boot from 5tb hdd:

/dev/sda2: LABEL="rootfs" UUID="d4cc7d63-da78-48ad-9bdd-64ffbba449a8" BLOCK_SIZE="4096" TYPE="ext4" PARTLABEL="Basic data partition" PARTUUID="0000046b-eba0-706e-cf40-dc01d6080000"

The PARTUUID of the disk is not the one you have in /etc/fstab - the one in /etc/fstab actually refers to the partition from the uSD card.

Btw, did you post the output of /etc/fstab from the uSD card or the one from the disk ? Check that the /etc/fstab on the disk contains the correct Partition UUIDs - the ones from /dev/sda1 and /dev/sda as shown by the blkid command and ensure that the cmdline.txt from the disk has the right partition ID (of /dev/sda2) also.

I opened the fstab in a txt editor from the hard drive it's self from the /etc/fstab rather than running any commands in the shell to receive an output so the fstab should be for the hdd

I will do a cli readout when I return home a little later.

It's looking like I need to fix the cmdline.txt and the fstab on the hdd like u are suggesting. I will post pre edit and post edits of everything I change to double check the correctness of it.

If I'm understanding correctly the cmdline.txt (located in the boot partition) the fstab (located in /etc) must all have matching UUID'S for the corresponding partitions.

[korn16ftl3]

so having a look here to see what matches and what to replace in the fstab and what I'm seeing is the following:

/dev/mmcblk0p1: LABEL_FATBOOT="bootfs" LABEL="bootfs" UUID="F737-8E10" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="90ace319-01"
/dev/mmcblk0p2: LABEL="rootfs" UUID="d6ecfcd5-2703-41bf-9301-10c403b6fb0c" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="90ace319-02"

those would be the SD card labels if im not mistaken and you had said:
@mitu said in usb boot from 5tb hdd:

The PARTUUID of the disk is not the one you have in /etc/fstab - the one in /etc/fstab actually refers to the partition from the uSD card.

however i am not seeing the PARTUUID d9c86127 anywhere in the mmcblk0 read outs. I'm curious how you came to that conclusion.

do i change the entire PARTUUID in the fstab to match the ones in the blkid readout:
i.e.
from this in fstab:

PARTUUID=d9c86127-01  /boot/firmware  vfat    defaults          0       2

to this:

PARTUUID=701069d0-00cf-01dc-e814-08b86700ee00-01  /boot/firmware  vfat    defaults          0       2

I'm assuming that the -01 at the very end directs it to partition 1 on the hdd as it is labeled /boot/firmware under the proc column.

lastly i should change the cmdline.txt from this:

code_textconsole=serial0,115200 console=tty1 root=PARTUUID=d9c86127-02 rootfstype=ext4 fsck.repair=yes rootwait quiet init=/usr/lib/raspberrypi-sys-mods/firstboot cfg80211.ieee80211_regdom=NA systemd.run=/boot/firstrun.sh systemd.run_success_action=reboot systemd.unit=kernel-command-line.target

to this:

console=serial0,115200 console=tty1 root=PARTUUID=701069d0-00cf-01dc-e814-08b86700ee00-02 rootfstype=ext4 fsck.repair=yes rootwait quiet init=/usr/lib/raspberrypi-sys-mods/firstboot cfg80211.ieee80211_regdom=NA systemd.run=/boot/firstrun.sh systemd.run_success_action=reboot systemd.unit=kernel-command-line.target

what kind of throws me for a loop is the fact that the HDD PARTUUID has so many more characters than the original PARTUUID. Is there any particular reason for this?

UPDATE:
tried the above changes an got a message from the system stating something about running blkid or something to that effect. the message kept appearing.

im going to switch back to the 5tb drve and start fresh, so i can pick up work on the 12tb drive later sine we have established the problem is not the drive(s) just the way the system is handling them.

this time i suppose its time to leave my comfort zone a little more and try doing everything within linux and on the pi itself.

im going to start by tinkering around with th tools talked about here:
https://forums.raspberrypi.com/viewtopic.php?t=196778

i will keep this thread up to date with things i have tried and if i succeed or not because i personally hate when people dont follow up with their threads for other readers that may be having the same issues or attempting the same things and aren't sure what to do or where to go.

[mitu]

@korn16ftl3 said in usb boot from 5tb hdd:

however i am not seeing the PARTUUID d9c86127 anywhere in the mmcblk0 read outs. I'm curious how you came to that conclusion.

There's no mmcblk0 partition and I didn't mention any d9c86127 in my reply, I just mentioned that the Paritition UUID for the uSD card partitions (that you posted):

/dev/mmcblk0p1: LABEL_FATBOOT="bootfs" LABEL="bootfs" UUID="F737-8E10" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="90ace319-01"
/dev/mmcblk0p2: LABEL="rootfs" UUID="d6ecfcd5-2703-41bf-9301-10c403b6fb0c" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="90ace319-02"

are the Partition UUIDs that also appear in /etc/fstab.

In the /etc/fstab file from the hard disk (not uSD), it's the Partition UUIDs from /dev/sda[1|2] that should be used, not the ones from the uSD - the error you got with the kernel not finding the Partition UUID is caused by a mismatch in the UUID(s).