usb boot from 5tb hdd

korn16ftl3

Pi Model or other hardware: raspberry pi 4 8GB

Power Supply used: OEM ROG ally USB C charger

RetroPie Version Used : 4.8

Built From: pre-build from site filename: retropie-buster-4.8-rpi4_400.img.gz

USB Devices connected: 5TB WD passport ultra (boot device), Dell USB keyboard, onn brand USB mouse

Controller used: INNEXT clone snes controller

Error messages received: could not expand filesystem, please try raspi-config or rc_GUI. Root partition resize failed.

Guide used: did the normal install, updated bootloader via sd flash for USB boot, wrote retropie image to the USB HDD with raspberry pi imager

How to replicate the problem: simply write the retropie image to my hdd and try to load the OS, does not repeat on USB stick however so it leads me to believe i have some kind of problem with the hdd.

I'm trying to get retropie to boot from a 5TB usb HDD and i can write the image and it loads but the problem seems to be that when it first boots up its supposed to resize the partitoon to the full size of the drive, however it seems that for some reason this fails on the hdd.

I get a big grey and blue screen that comes up telling me it cant resize the partition, and asks me to try with raspi-config, however when I get into raspi-config (either threw cli or the retropie GUI) option 7 (where the resize option is supposed to be) is missing and unable to be selected. I have no idea what rc_GUI is and have not attempted this as of yet.

Is it even possible to boot from a 5TB hdd or am i missing, doing something wrong here?

mitu

I think for >4Tb drivers you need a GPT partition table, whereas the one provided with the RetroPie image is just MBR.

Is the current RaspiOS Lite version able to resize the partition ? You may want to install the RaspiOS first then do a manual install if it works.

korn16ftl3

@mitu said in usb boot from 5tb hdd:

I think for >4Tb drivers you need a GPT partition table, whereas the one provided with the RetroPie image is just MBR.

Is the current RaspiOS Lite version able to resize the partition ? You may want to install the RaspiOS first then do a manual install if it works.

i tried using gparted and manually resizing it anf that also failed.

i also installed raspian os on the hdd and it did everything in MBR, even split the drive into 2 partitions one was unallocated.

its not looking like this is a possibility

mitu

@korn16ftl3 said in usb boot from 5tb hdd:

its not looking like this is a possibility

I know users in the RPi forums have made this work, but you have to use gparted to convert the partition table to GPT (from MBR), from a separate/different system - a Linux system which can run gparted on the (unmounted) SSD disk. If you have a PC system you can use gparted from there (with a Live USB Linux - https://gparted.org/liveusb.php).

korn16ftl3

@mitu said in usb boot from 5tb hdd:

@korn16ftl3 said in usb boot from 5tb hdd:

its not looking like this is a possibility

I know users in the RPi forums have made this work, but you have to use gparted to convert the partition table to GPT (from MBR), from a separate/different system - a Linux system which can run gparted on the (unmounted) SSD disk. If you have a PC system you can use gparted from there (with a Live USB Linux - https://gparted.org/liveusb.php).

I have made a similar attempt like what you are mentioning (at least I think) by loading into raspberry pi os from as card installing gparted then plugging in the 5tb usb had (non-ssd) and using gparted to convert from MBR to GPT. After that I use a windows machine and the pi image writer to write retries (and I’ve even tried raspi os) to the usb hdd. The problem im finding is that it just turns the hdd back to MBR.

After some thought I do believe there is a way (with some of my windows hdd tools) to convert an MBR hdd to GPT with the OS installed and not destroying the data. I will have a look later on today. Not sure how this will fair however as after the image writer the hdd will be in Linux format (ext4 I think it is?) so maybe I’ll have a look with both windows tools an gparted after the image write and see what options I have.

mitu

@korn16ftl3 said in usb boot from 5tb hdd:

After that I use a windows machine and the pi image writer to write retries (and I’ve even tried raspi os) to the usb hdd. The problem im finding is that it just turns the hdd back to MBR.

You need to do the conversion after you write the image, otherwise writing the image again will just re-initialize the disk with a MBR boot signature.

korn16ftl3

@mitu said in usb boot from 5tb hdd:

@korn16ftl3 said in usb boot from 5tb hdd:

After that I use a windows machine and the pi image writer to write retries (and I’ve even tried raspi os) to the usb hdd. The problem im finding is that it just turns the hdd back to MBR.

You need to do the conversion after you write the image, otherwise writing the image again will just re-initialize the disk with a MBR boot signature.

So I've tried a couple things:

wrote retrope image to HDD then used MiniTool Partition Wizard (windows) to change the MBR to GPT try the first boot of retropie (where it resizes) and it stalls after post with the following 3 lines on screen

[ 1.494426] mmc1: controller never released inhabit bit(s)
[ 3.839951] sd 0:0:0:0: [sda] no caching mode page found
[ 3.840041] sd 0:0:0:0: [sda] assuming drive cache: write through

a. I've taken the previous results (just for fun) with a fresh image write of retropie on the hdd, wrote a pi OS 64 bit image to an sd card, installed gparted, to pi os changed the MBR to GPT with pi OS and got the same results from above.

b. I used gparted to play with the boot partition flags.
The boot partition flag WAS originally set to msftres so i obviously tried setting the boot flag first and still got the same 3 lines above. Ive also tried setting the boot partition flag to lba (just because) again same results as well with that.

wrote a fresh retropie image to the hdd booted it up to receive the unable to resize partition error loaded to controller settings hi F4 then used sudo shutdown now to shut the pi down. I then plugged the pi into a windows pc and again used MiniTool Partition Wizard to convert the hdd to GPT and tried to boot up the retropie hdd again but this time it entered some kind of boot loop at post (the pink screen) complaining about the hdd though it scrolled so quickly I didn't catch it all. Also note i have only attempted this particular method once so perhaps user error?

Anyhow I'm curious what those 3 lines mean? i know 3 lines show up on a successful boot at that portion as well then continues to boot but once again they go by so quickly I'm not sure what they say....like at all.

it seems I'm getting part of this correct somehow as its doing something it does on a normal boot but what am i missing here?

mitu

@korn16ftl3 said in usb boot from 5tb hdd:

Anyhow I'm curious what those 3 lines mean? i know 3 lines show up on a successful boot at that portion as well then continues to boot but once again they go by so quickly I'm not sure what they say....like at all.

The sda related messages means the disc has been read by the system and I would expect to see a list of partitions found on the disc in the following messages ( referring to sda1, sda2 since there are 2 partition).
Maybe there's not enough power supplied by the 'charger' you're using and the disc cannot be initialized ? Can you try with a RPI certified power source ?

korn16ftl3

@mitu said in usb boot from 5tb hdd:

@korn16ftl3 said in usb boot from 5tb hdd:

Anyhow I'm curious what those 3 lines mean? i know 3 lines show up on a successful boot at that portion as well then continues to boot but once again they go by so quickly I'm not sure what they say....like at all.

The sda related messages means the disc has been read by the system and I would expect to see a list of partitions found on the disc in the following messages ( referring to sda1, sda2 since there are 2 partition).
Maybe there's not enough power supplied by the 'charger' you're using and the disc cannot be initialized ? Can you try with a RPI certified power source ?

I do not have a certified rpi power source available to me however I find it hard to believe that there is insufficient power supplied as the ROG ally oem charger should be sufficient at the following specifications:

The ROG Ally OEM charger is a 65W USB Type C power adapter.
Here's a breakdown of its specifications:
Model: ADP-65JWY2A or ADP-65SDD2A.
Power: 65W.
Voltage: 20V.
Amperage: 3.25A.
Connector: USB-C.
Plug Type: US (American).

I did find a different link when googling some of those errors and found that it’s possible it is the data to usb chipset on the external hdd (source: https://forums.raspberrypi.com/viewtopic.php?t=330384 )

The above site does list a few things to try and to try to eliminate the still potential power problem an maybe the data to usb controller problem I ordered a different external enclosure that is independently powered as well ( this enclosure: https://a.co/d/fWlN5rW ). I already have a different sabernet data to usb adapter that is NOT independently powered that works ok with my (off brand and for some reason slow) 2.5 ssd hdd so hopefully the powered one solves my issues.

If I still have problems I will try some of the other remedies mentioned there as well as the way back machine link that is in that thread as well.

If you have any other feedback or ideas please feel free to post them and feel free to have a read on that thread as well.

Lolonois

@korn16ftl3 good you dug up some options.

Just to cross other issues out: The first Pi4 revision has an issue with some quality USB chargers. Check the output of cat /sys/firmware/devicetree/base/model, if it shows board revision 1.1 you may need an official Pi4 USB-C power supply. Otherwise you are on the safe side.

The mmc1 controller never released inhibit bits is most likely not related with the USB drive issue. Most likely you can mute it by inserting a blank sd card (no partition on it). [1]

The USB WD Drive has a custom chipset (see discussion in that thread [2]), which may cause the issue in context with the Raspberry Pi. You can try some USB quirks to mitigate the issue.
May be you can use t as flag (disables some UAS commands which may help already), but at least u should remediate the situation.
(add either to cmdline.txt) (see usb-storage.quirks in [3])

Last but not least you can detect which mode (Driver=uas or Driver=usb-storage) the drive uses with lsusb -t (look for the USB drive name in the output).

[1] https://forums.raspberrypi.com/viewtopic.php?t=351746#p2123679
[2] https://community.wd.com/t/5-tb-mypassport-drive-only/242638/2
[3] https://www.kernel.org/doc/Documentation/admin-guide/kernel-parameters.txt

korn16ftl3

@Lolonois
Thanks for some more directions to go, as yours seem WD targeted i will begin with them in my troubleshooting process.

I was unaware that there was a particular issue with some pi4 models that forced me to use an official pi4 power supply. Perhaps I should attempt this first to resolve my issues or start here to eliminate the possibility.

I have just received my external usb enclosure and have shucked the drive. To my disappointment the drive has no data interface and in fact has an integrated USB interface.

I suppose to proceed with even considering putting this in an external sata enclosure I would have to match this hdd model # with another that has the actual sata interface pcb to use as a donor for this drive. I’m not even sure how well this would work but in theory it should. Does anyone have any other possible work arounds for this issue? I don’t even know when they started doing this, it used to be buy an external shuck it and put it in whatever device you chose.

Does anyone else have any input thoughts or ideas with this?

This has quickly gotten way over my head….maybe I will just buy a different hdd and move on I don’t know yet.

korn16ftl3

ok as an update:

since i started tinkering with this and realized perhaps this is the controller interface on the USB HDD and that the usb portion was integrated to the HDD its self i decided to get a 3.5 12TB WD mybook external HDD. this assures there is appropriate power as well as that i am able to actually shuck the drive from the enclosure if i so choose.

Upon trying a different drive i have attempted some of the steps above that i have already attempted with the other drive with the same results. so far this leads me to believe this has to do with booting from larger that 2TB drives (GPT vs MBR) as was assumed from the beginning of this thread.

once I started getting similar results from the previous drive i started dabbeling with the options that @Lolonois presented me with.

im not exactly certain what to make of the readout of the lsusb -t but i am going to post a picture of it and perhaps someone can help me.

Im going to set up SSH and boot raspberry pi OS with only my HDD plugged in to the usb ports so at least I know the readouts are only for the USB HDD and nothing else. This will perhaps eliminate some of the confusion with the readout (at lest for me)

I do know (and its also noted in the photo of my read out) that my rpi 4 is a board v1.4 so I should be safe from the previously mentions flaw in board rev. 1.1.

i started reading threw some of the usb-storage.quirks and am making a bit of sense out of it. I'm not exactly certain what to do with the information.

the section of quirks i am reading is:

usb-storage.quirks=
			[UMS] A list of quirks entries to supplement or
			override the built-in unusual_devs list.  List
			entries are separated by commas.  Each entry has
			the form VID:PID:Flags where VID and PID are Vendor
			and Product ID values (4-digit hex numbers) and
			Flags is a set of characters, each corresponding
			to a common usb-storage quirk flag as follows:
				a = SANE_SENSE (collect more than 18 bytes
					of sense data, not on uas);
				b = BAD_SENSE (don't collect more than 18
					bytes of sense data, not on uas);
				c = FIX_CAPACITY (decrease the reported
					device capacity by one sector);
				d = NO_READ_DISC_INFO (don't use
					READ_DISC_INFO command, not on uas);
				e = NO_READ_CAPACITY_16 (don't use
					READ_CAPACITY_16 command);
				f = NO_REPORT_OPCODES (don't use report opcodes
					command, uas only);
				g = MAX_SECTORS_240 (don't transfer more than
					240 sectors at a time, uas only);
				h = CAPACITY_HEURISTICS (decrease the
					reported device capacity by one
					sector if the number is odd);
				i = IGNORE_DEVICE (don't bind to this
					device);
				j = NO_REPORT_LUNS (don't use report luns
					command, uas only);
				k = NO_SAME (do not use WRITE_SAME, uas only)
				l = NOT_LOCKABLE (don't try to lock and
					unlock ejectable media, not on uas);
				m = MAX_SECTORS_64 (don't transfer more
					than 64 sectors = 32 KB at a time,
					not on uas);
				n = INITIAL_READ10 (force a retry of the
					initial READ(10) command, not on uas);
				o = CAPACITY_OK (accept the capacity
					reported by the device, not on uas);
				p = WRITE_CACHE (the device cache is ON
					by default, not on uas);
				r = IGNORE_RESIDUE (the device reports
					bogus residue values, not on uas);
				s = SINGLE_LUN (the device has only one
					Logical Unit);
				t = NO_ATA_1X (don't allow ATA(12) and ATA(16)
					commands, uas only);
				u = IGNORE_UAS (don't bind to the uas driver);
				w = NO_WP_DETECT (don't test whether the
					medium is write-protected).
				y = ALWAYS_SYNC (issue a SYNCHRONIZE_CACHE
					even if the device claims no cache,

the example given is by the link:
quirks=0419:aaf5:rl,0421:0433:rc

now if I understand correctly the rl after the 0419:aaf5 for example are the quirks activate but what/were does the 0419:aaf5 come from? I'm guessing it is a device ID of some sort? how do i get this device ID as I did not see anything like it from the lsusb -t command (again see the photo)

I do know where to put these lines as I'm assuming there is a cmdline.txt in the fat/boot portion of the HDD/SD card where I add them and again guessing I add them at the very end of the txt file.

Anyway I'm back at this again and have not given up......just decided to go a different direction to try to mitigate some potential issues and give more more options that the propitiatory USB interface on the previous USB HDD offered.

where to go from here.

i know booting from a large hdd is possible I'm just not certain how to achieve it just yet.

here is a link to the photo:
https://ibb.co/DP1sjXvb

Lolonois

@korn16ftl3 ah my bad, I guess I should have added -v to lsusb. You may run again and keep only Mass Storage devices, e.g. lsusb -vt | grep -A1 "Mass Storage"

Which will output something like:

        |__ Port 3: Dev 11, If 0, Class=Mass Storage, Driver=uas, 480M
            ID 152d:0583 JMicron Technology Corp. / JMicron USA Technology Corp. JMS583Gen 2 to PCIe Gen3x2 Bridge

The hexadecimal numbers right after ID are the USB Vendor ID and Product ID. Use these to add the usbquirks in the cmdline.txt in the /boot (or /boot/firmware folder). Make sure to add it at the same line (no newline) and only use spaces before the whole kernel parameter and value, e.g.:

<existing cmdline.txt line> usb-storage.quirks=<vendorId>:<productId>:t

<existing cmdline.txt line> usb-storage.quirks=<vendorId>:<productId>:u

You may try first with the flag/quirk :t and check if it remediates the situation or jump right to :u, which will most likely -judging from the distance- remediate the situation.