usb boot from 5tb hdd

korn16ftl3

@korn16ftl3 said in usb boot from 5tb hdd:

After that I use a windows machine and the pi image writer to write retries (and I’ve even tried raspi os) to the usb hdd. The problem im finding is that it just turns the hdd back to MBR.

You need to do the conversion after you write the image, otherwise writing the image again will just re-initialize the disk with a MBR boot signature.

So I've tried a couple things:

wrote retrope image to HDD then used MiniTool Partition Wizard (windows) to change the MBR to GPT try the first boot of retropie (where it resizes) and it stalls after post with the following 3 lines on screen

[ 1.494426] mmc1: controller never released inhabit bit(s)
[ 3.839951] sd 0:0:0:0: [sda] no caching mode page found
[ 3.840041] sd 0:0:0:0: [sda] assuming drive cache: write through

a. I've taken the previous results (just for fun) with a fresh image write of retropie on the hdd, wrote a pi OS 64 bit image to an sd card, installed gparted, to pi os changed the MBR to GPT with pi OS and got the same results from above.

b. I used gparted to play with the boot partition flags.
The boot partition flag WAS originally set to msftres so i obviously tried setting the boot flag first and still got the same 3 lines above. Ive also tried setting the boot partition flag to lba (just because) again same results as well with that.

wrote a fresh retropie image to the hdd booted it up to receive the unable to resize partition error loaded to controller settings hi F4 then used sudo shutdown now to shut the pi down. I then plugged the pi into a windows pc and again used MiniTool Partition Wizard to convert the hdd to GPT and tried to boot up the retropie hdd again but this time it entered some kind of boot loop at post (the pink screen) complaining about the hdd though it scrolled so quickly I didn't catch it all. Also note i have only attempted this particular method once so perhaps user error?

Anyhow I'm curious what those 3 lines mean? i know 3 lines show up on a successful boot at that portion as well then continues to boot but once again they go by so quickly I'm not sure what they say....like at all.

it seems I'm getting part of this correct somehow as its doing something it does on a normal boot but what am i missing here?

mitu

@korn16ftl3 said in usb boot from 5tb hdd:

Anyhow I'm curious what those 3 lines mean? i know 3 lines show up on a successful boot at that portion as well then continues to boot but once again they go by so quickly I'm not sure what they say....like at all.

The sda related messages means the disc has been read by the system and I would expect to see a list of partitions found on the disc in the following messages ( referring to sda1, sda2 since there are 2 partition).
Maybe there's not enough power supplied by the 'charger' you're using and the disc cannot be initialized ? Can you try with a RPI certified power source ?

korn16ftl3

@mitu said in usb boot from 5tb hdd:

@korn16ftl3 said in usb boot from 5tb hdd:

Anyhow I'm curious what those 3 lines mean? i know 3 lines show up on a successful boot at that portion as well then continues to boot but once again they go by so quickly I'm not sure what they say....like at all.

The sda related messages means the disc has been read by the system and I would expect to see a list of partitions found on the disc in the following messages ( referring to sda1, sda2 since there are 2 partition).
Maybe there's not enough power supplied by the 'charger' you're using and the disc cannot be initialized ? Can you try with a RPI certified power source ?

I do not have a certified rpi power source available to me however I find it hard to believe that there is insufficient power supplied as the ROG ally oem charger should be sufficient at the following specifications:

The ROG Ally OEM charger is a 65W USB Type C power adapter.
Here's a breakdown of its specifications:
Model: ADP-65JWY2A or ADP-65SDD2A.
Power: 65W.
Voltage: 20V.
Amperage: 3.25A.
Connector: USB-C.
Plug Type: US (American).

I did find a different link when googling some of those errors and found that it’s possible it is the data to usb chipset on the external hdd (source: https://forums.raspberrypi.com/viewtopic.php?t=330384 )

The above site does list a few things to try and to try to eliminate the still potential power problem an maybe the data to usb controller problem I ordered a different external enclosure that is independently powered as well ( this enclosure: https://a.co/d/fWlN5rW ). I already have a different sabernet data to usb adapter that is NOT independently powered that works ok with my (off brand and for some reason slow) 2.5 ssd hdd so hopefully the powered one solves my issues.

If I still have problems I will try some of the other remedies mentioned there as well as the way back machine link that is in that thread as well.

If you have any other feedback or ideas please feel free to post them and feel free to have a read on that thread as well.

Lolonois

@korn16ftl3 good you dug up some options.

Just to cross other issues out: The first Pi4 revision has an issue with some quality USB chargers. Check the output of cat /sys/firmware/devicetree/base/model, if it shows board revision 1.1 you may need an official Pi4 USB-C power supply. Otherwise you are on the safe side.

The mmc1 controller never released inhibit bits is most likely not related with the USB drive issue. Most likely you can mute it by inserting a blank sd card (no partition on it). [1]

The USB WD Drive has a custom chipset (see discussion in that thread [2]), which may cause the issue in context with the Raspberry Pi. You can try some USB quirks to mitigate the issue.
May be you can use t as flag (disables some UAS commands which may help already), but at least u should remediate the situation.
(add either to cmdline.txt) (see usb-storage.quirks in [3])

Last but not least you can detect which mode (Driver=uas or Driver=usb-storage) the drive uses with lsusb -t (look for the USB drive name in the output).

[1] https://forums.raspberrypi.com/viewtopic.php?t=351746#p2123679
[2] https://community.wd.com/t/5-tb-mypassport-drive-only/242638/2
[3] https://www.kernel.org/doc/Documentation/admin-guide/kernel-parameters.txt

korn16ftl3

@Lolonois
Thanks for some more directions to go, as yours seem WD targeted i will begin with them in my troubleshooting process.

I was unaware that there was a particular issue with some pi4 models that forced me to use an official pi4 power supply. Perhaps I should attempt this first to resolve my issues or start here to eliminate the possibility.

I have just received my external usb enclosure and have shucked the drive. To my disappointment the drive has no data interface and in fact has an integrated USB interface.

I suppose to proceed with even considering putting this in an external sata enclosure I would have to match this hdd model # with another that has the actual sata interface pcb to use as a donor for this drive. I’m not even sure how well this would work but in theory it should. Does anyone have any other possible work arounds for this issue? I don’t even know when they started doing this, it used to be buy an external shuck it and put it in whatever device you chose.

Does anyone else have any input thoughts or ideas with this?

This has quickly gotten way over my head….maybe I will just buy a different hdd and move on I don’t know yet.

korn16ftl3

ok as an update:

since i started tinkering with this and realized perhaps this is the controller interface on the USB HDD and that the usb portion was integrated to the HDD its self i decided to get a 3.5 12TB WD mybook external HDD. this assures there is appropriate power as well as that i am able to actually shuck the drive from the enclosure if i so choose.

Upon trying a different drive i have attempted some of the steps above that i have already attempted with the other drive with the same results. so far this leads me to believe this has to do with booting from larger that 2TB drives (GPT vs MBR) as was assumed from the beginning of this thread.

once I started getting similar results from the previous drive i started dabbeling with the options that @Lolonois presented me with.

im not exactly certain what to make of the readout of the lsusb -t but i am going to post a picture of it and perhaps someone can help me.

Im going to set up SSH and boot raspberry pi OS with only my HDD plugged in to the usb ports so at least I know the readouts are only for the USB HDD and nothing else. This will perhaps eliminate some of the confusion with the readout (at lest for me)

I do know (and its also noted in the photo of my read out) that my rpi 4 is a board v1.4 so I should be safe from the previously mentions flaw in board rev. 1.1.

i started reading threw some of the usb-storage.quirks and am making a bit of sense out of it. I'm not exactly certain what to do with the information.

the section of quirks i am reading is:

usb-storage.quirks=
			[UMS] A list of quirks entries to supplement or
			override the built-in unusual_devs list.  List
			entries are separated by commas.  Each entry has
			the form VID:PID:Flags where VID and PID are Vendor
			and Product ID values (4-digit hex numbers) and
			Flags is a set of characters, each corresponding
			to a common usb-storage quirk flag as follows:
				a = SANE_SENSE (collect more than 18 bytes
					of sense data, not on uas);
				b = BAD_SENSE (don't collect more than 18
					bytes of sense data, not on uas);
				c = FIX_CAPACITY (decrease the reported
					device capacity by one sector);
				d = NO_READ_DISC_INFO (don't use
					READ_DISC_INFO command, not on uas);
				e = NO_READ_CAPACITY_16 (don't use
					READ_CAPACITY_16 command);
				f = NO_REPORT_OPCODES (don't use report opcodes
					command, uas only);
				g = MAX_SECTORS_240 (don't transfer more than
					240 sectors at a time, uas only);
				h = CAPACITY_HEURISTICS (decrease the
					reported device capacity by one
					sector if the number is odd);
				i = IGNORE_DEVICE (don't bind to this
					device);
				j = NO_REPORT_LUNS (don't use report luns
					command, uas only);
				k = NO_SAME (do not use WRITE_SAME, uas only)
				l = NOT_LOCKABLE (don't try to lock and
					unlock ejectable media, not on uas);
				m = MAX_SECTORS_64 (don't transfer more
					than 64 sectors = 32 KB at a time,
					not on uas);
				n = INITIAL_READ10 (force a retry of the
					initial READ(10) command, not on uas);
				o = CAPACITY_OK (accept the capacity
					reported by the device, not on uas);
				p = WRITE_CACHE (the device cache is ON
					by default, not on uas);
				r = IGNORE_RESIDUE (the device reports
					bogus residue values, not on uas);
				s = SINGLE_LUN (the device has only one
					Logical Unit);
				t = NO_ATA_1X (don't allow ATA(12) and ATA(16)
					commands, uas only);
				u = IGNORE_UAS (don't bind to the uas driver);
				w = NO_WP_DETECT (don't test whether the
					medium is write-protected).
				y = ALWAYS_SYNC (issue a SYNCHRONIZE_CACHE
					even if the device claims no cache,

the example given is by the link:
quirks=0419:aaf5:rl,0421:0433:rc

now if I understand correctly the rl after the 0419:aaf5 for example are the quirks activate but what/were does the 0419:aaf5 come from? I'm guessing it is a device ID of some sort? how do i get this device ID as I did not see anything like it from the lsusb -t command (again see the photo)

I do know where to put these lines as I'm assuming there is a cmdline.txt in the fat/boot portion of the HDD/SD card where I add them and again guessing I add them at the very end of the txt file.

Anyway I'm back at this again and have not given up......just decided to go a different direction to try to mitigate some potential issues and give more more options than the propitiatory USB interface on the previous USB HDD offered.

where to go from here.

i know booting from a large hdd is possible I'm just not certain how to achieve it just yet.

here is a link to the photo:
https://ibb.co/DP1sjXvb

Lolonois

@korn16ftl3 ah my bad, I guess I should have added -v to lsusb. You may run again and keep only Mass Storage devices, e.g. lsusb -vt | grep -A1 "Mass Storage"

Which will output something like:

        |__ Port 3: Dev 11, If 0, Class=Mass Storage, Driver=uas, 480M
            ID 152d:0583 JMicron Technology Corp. / JMicron USA Technology Corp. JMS583Gen 2 to PCIe Gen3x2 Bridge

The hexadecimal numbers right after ID are the USB Vendor ID and Product ID. Use these to add the usbquirks in the cmdline.txt in the /boot (or /boot/firmware folder). Make sure to add it at the same line (no newline) and only use spaces before the whole kernel parameter and value, e.g.:

<existing cmdline.txt line> usb-storage.quirks=<vendorId>:<productId>:t

<existing cmdline.txt line> usb-storage.quirks=<vendorId>:<productId>:u

You may try first with the flag/quirk :t and check if it remediates the situation or jump right to :u, which will most likely -judging from the distance- remediate the situation.

korn16ftl3

@Lolonois said in usb boot from 5tb hdd:

@korn16ftl3 ah my bad, I guess I should have added -v to lsusb. You may run again and keep only Mass Storage devices, e.g. lsusb -vt | grep -A1 "Mass Storage"

Which will output something like:
        |__ Port 3: Dev 11, If 0, Class=Mass Storage, Driver=uas, 480M
            ID 152d:0583 JMicron Technology Corp. / JMicron USA Technology Corp. JMS583Gen 2 to PCIe Gen3x2 Bridge
The hexadecimal numbers right after ID are the USB Vendor ID and Product ID. Use these to add the usbquirks in the cmdline.txt in the /boot (or /boot/firmware folder). Make sure to add it at the same line (no newline) and only use spaces before the whole kernel parameter and value, e.g.:
<existing cmdline.txt line> usb-storage.quirks=<vendorId>:<productId>:t 
<existing cmdline.txt line> usb-storage.quirks=<vendorId>:<productId>:u
You may try first with the flag/quirk :t and check if it remediates the situation or jump right to :u, which will most likely -judging from the distance- remediate the situation.

ok so now i have something that makes sense :-)

this is now my readout for the drive i am trying to boot retropie from:
https://ibb.co/0jXRnDNR

now to follow the rest of your instructions:

my original cmdline.txt is as follows:

console=serial0,115200 console=tty1 root=PARTUUID=3de64a5b-02 rootfstype=ext4 fsck.repair=yes rootwait quiet loglevel=3 consoleblank=0 plymouth.enable=0 quiet init=/usr/lib/raspi-config/init_resize.sh

if I'm understanding what the instructions are it should be this for the quirks:

console=serial0,115200 console=tty1 root=PARTUUID=3de64a5b-02 rootfstype=ext4 fsck.repair=yes rootwait quiet loglevel=3 consoleblank=0 plymouth.enable=0 quiet init=/usr/lib/raspi-config/init_resize.sh usb-storage.quirks=1058:5ee:u

My new question is since we are playing with the cmdline.txt, what are the other parameters that are there?

Lolonois

@korn16ftl3 Looks correct, except you are missing one hex-number in the product ID (should be :25ee and not :5ee).

The quiet parameter is double but it does not have an impact: The kernel parameters are parsed left to right and the last one wins, if there are two identical parameters.

My new question is since we are playing with the cmdline.txt, what are the other parameters that are there?

The kernel parameters are documented in the kernel documentation, in the raspberry pi documentation, and the kernel parameter of plymouth and info about plymouth itself are documented for example in the Arch wiki,

korn16ftl3

@Lolonois said in usb boot from 5tb hdd:

@korn16ftl3 Looks correct, except you are missing one hex-number in the product ID (should be :25ee and not :5ee).

The quiet parameter is double but it does not have an impact: The kernel parameters are parsed left to right and the last one wins, if there are two identical parameters.

My new question is since we are playing with the cmdline.txt, what are the other parameters that are there?

The kernel parameters are documented in the kernel documentation, in the raspberry pi documentation, and the kernel parameter of plymouth and info about plymouth itself are documented for example in the Arch wiki,

i made the previously mentioned changes to the cmdline.txt and am getting the same error readouts.

im starting to think/wonder if the GPT portion of thi is what is causing some of my problems as retropie its self if i understand correctly was written for MBR and everything boots just fine in MBR and GPT when everything is <2tb the second i convert the GPT partition manually (before initial retropie boot and partition resize) everything ceases toi load/boot.

im going to remove the

quiet init=/usr/lib/raspi-config/init_resize.sh

from the cmdline.txt since perhaps it is hanging on attempt to resize a partition that has already been resized for one and for two is GPT and not sure how to handle it.

lets see what that does i suppose.

also debating on wiping everything abd starting over using gparted and specifically raspberry pi os to try to set tis up as well maybe that ii yield different results.

will report back after a little bit of experimenting and a bit more research.

alternatively i have seen some things online that enable UEFI booting ( [https://www.linux.it/~ema/posts/secure-boot-rpi/](link url) , [https://github.com/pftf/RPi4](link url) ) would this be any help with trying to boot from >2TB partitions as UEFI booting supports it? would there be any drawbacks or risks to experimenting with these options?