[PI3B+ & Retropie 4.4.4] Limit on Roms stored?

magicfly · 6 Jan 2019, 13:00

Hi,
first of all thanks to everybody involved in Retropie project, i've been enjoying a lot of hours with your project.
I'm a newbie and so i've read almost all the guides all over the internet,, maybe not all but a lot :) on how properly use an external usb hdd without buying expensive sd cards.
First, my specs are:

Pi Model or other hardware: 3B+
Power Supply used: yes, the default which come with Raspberry PI3B+
RetroPie Version Used (eg 3.6, 3.8.1, 4.1 - do not write latest): 4.4.4
Built From: Pre made SD Image on RetroPie website
USB Devices connected: 3 (usb keyboard logtech k400 - usb joypad gamesir GS4 - usb hdd western digital my passport 1TB ntfs)
Controller used: usb joypad gamesir GS4
Error messages received:
Log found in /dev/shm/runcommand.log (if relevant):
Guide used: (Mention if you followed a guide)
File: (File with issue - with FULL path)
Emulator: (Name of emulator - if applicable) all
Attachment of config files: (PLEASE USE PASTEBIN.COM FOR LARGE LOGS)
How to replicate the problem:

Second, this is what i've done:
Installed retropie on a SD 128 GB card from a Pre made SD Image on RetroPie website and then set all configuration about country, wifi ecc. everything ok.
Used Eazy-Hax-RetroPie-Toolkit to use both sd card and my USB hdd ntfs formatted, it creates a folder called addonusb on home/pi/ where i put the rom folder for every system i inted to use, linked to my hard disk.
Everything works fine, es_system.cfg is well configured and roms start regularly.
BUT, in the meantime i've addedother systems and added many roms to existing folder, and at the moment i've ~500000 roms.
Yes, a lot are dupes, but now my PI3B+ won't load any roms, i see the system scraped ok but everytime i click on a random game of a random system it goes to a black screen and after 4, 5 seconds goes back to the menu, without the message "launching ecc...".

The question is: is there a limit of roms i can store on my hdd (not sd) ? for example at 70000 roms every game works perfectly...

Thanks a lot!!

mitu · 6 Jan 2019, 13:03

It probably is a limit - you can max out the memory of Emulationstation with that many ROMs. What's the output of /dev/shm/runcommand.log, after you try and launch a game ?

magicfly · 6 Jan 2019, 13:10

@mitu said in [PI3B+ & Retropie 4.4.4] Limit on Roms stored?:

u can m

it's strange, i don't have that log; just rpmanager.log from the retropie manager.
can i locate it elsewhere?
Thanks

mitu · 6 Jan 2019, 13:16

@magicfly The log only appears after running a ROM and it's not persisten, if you rebooted the system it will be gone.

magicfly · 6 Jan 2019, 13:21

@mitu said in [PI3B+ & Retropie 4.4.4] Limit on Roms stored?:

@magicfly The log only appears after running a ROM and it's not persisten, if you rebooted the system it will be gone.

just tried with ten different roms from many systems but runcommand.log hasn't been written, without rebooting.
Thanks

cyperghost · 6 Jan 2019, 13:28

@mitu it is a premade image. Eazy Hax 4 da 1337z

mitu · 6 Jan 2019, 13:31

@cyperghost He said he used the image from the RetroPie site. The Eazy-Hax-RetroPie-Toolkit is just an addon people can install over RetroPie (adding hacks is so cool!) from https://github.com/Shakz76/Eazy-Hax-RetroPie-Toolkit. It's usually present in most of the 3rd party ROM filles/badly configured images.

mitu · 6 Jan 2019, 13:31

@magicfly How much free memory do you have on the system, with Emulationstaton started ?

cyperghost · 6 Jan 2019, 16:03

@mitu said in [PI3B+ & Retropie 4.4.4] Limit on Roms stored?:

It's usually present in most of the 3rd party ROM filles/badly configured images

Yes I know :) And there are good reasons that it is not installable by RetroPie itself.

I've downloaded some from these images to see how they are configured.
Most of them are just crap. Using old RP versions, Broken Wifi, Inconsistent file layout, modified RetroPie-Setups.....

But with tonnage of ROMs with different file headers, beta versions....

All in all I agree with you ;)

magicfly · 6 Jan 2019, 16:43

@mitu said in [PI3B+ & Retropie 4.4.4] Limit on Roms stored?:

@magicfly How much free memory do you have on the system, with Emulationstaton started ?

Hi, thi is the situation with ES running.
I'm not a Linux expert so I typed free -m to see the free resources, let me know if i'm wrong.

@cyperghost i've found that toolkit because its ease to configure an hard drive with roms, but i'm ready to uninstall it and i'm open to any solution, but i need help to make it happens because i' not so linux addicted.
Thanks in advance to everybody!

cyperghost · 1 Jun 2019, 16:55

@magicfly said in [PI3B+ & Retropie 4.4.4] Limit on Roms stored?:

but i'm ready to uninstall it and i'm open to any solution,

No it's okay as long as you took the original image from here.
But as @mitu already said these scripts are spreaded widly on ready setted images and then the chances (and willingness) to help users is near to zero. Because we do really know nothing about these internal hacks,improvements and file versions.

About the Eazy Hax-Toolkit. Well it tells you to format your devices to NTFS and my experience was that NTFS is incredible slow in loading. So I use ext3 (best choice) and for the sakeness of portability FAT32. The only advantage for NTFS over FAT32 is the ability for symlinks....

Read more about here: Running ROMs from a USB drive and @mitu is one of our experts here - so his advices are helpfull ;)

magicfly · 6 Jan 2019, 16:57

yeah i tried with symlinks, the /home/pi/retropie/roms was correctly linked to the one on my drive but it's the same problem, no roms are loaded, black screen and goto back to the list :(

mitu · 6 Jan 2019, 17:04

@magicfly Follow the docs - https://retropie.org.uk/docs/Running-ROMs-from-a-USB-drive/ - to run your ROMs from an USB drive and don't rely on the 'Easy...'. How large is your external disk ?

magicfly · 6 Jan 2019, 17:23

@mitu said in [PI3B+ & Retropie 4.4.4] Limit on Roms stored?:

@magicfly Follow the docs - https://retropie.org.uk/docs/Running-ROMs-from-a-USB-drive/ - to run your ROMs from an USB drive and don't rely on the 'Easy...'. How large is your external disk ?

should i follow the manual method?
My hd is 1TB, and it's a WD elements portable see here

BuZz · 1 Jun 2019, 18:10

Looking at the code briefly for the toolkit I would definitely not recommend running anything from it. A right mess.

I don't think we should offer support for anyone who has run this.

magicfly · 1 Jun 2019, 18:17

oh well, you convinced me to uninstall it; this is just why i didn't know it was so dangerous.. :)
btw thank you, i'll try the manual mode. just a question, do you suggest to use ntfs or fat32 (i'm on windows...) Thanks.

omen_peter · 6 Jan 2019, 18:22

@magicfly said in [PI3B+ & Retropie 4.4.4] Limit on Roms stored?:

@mitu said in [PI3B+ & Retropie 4.4.4] Limit on Roms stored?:

@magicfly Follow the docs - https://retropie.org.uk/docs/Running-ROMs-from-a-USB-drive/ - to run your ROMs from an USB drive and don't rely on the 'Easy...'. How large is your external disk ?

should i follow the manual method?
My hd is 1TB, and it's a WD elements portable see here

I use a WD Elements external HDD mounted using the automatic method. It's NTFS formatted and works fine for me. Worked straight out of the box (no additional formatting required).

mitu · 1 Jun 2019, 19:44

@magicfly Use Fat32 if you can. The only limit is the 4Gb per-file limit, but since you're running on a PI, I assume you don't want to run Wii/GC games.

magicfly · 6 Jan 2019, 18:23

@omen_peter how many roms do you have?

cyperghost · 6 Jan 2019, 19:38

@BuZz said in [PI3B+ & Retropie 4.4.4] Limit on Roms stored?:

Looking at the code briefly for the toolkit I would definitely not recommend running anything from it. A right mess.

I don't think we should offer support for anyone who has run this.

As I said: Eazy Hax 4 da 1337z
I did not take a deep look, but it uses SSH remote scripts. So I stand to my sentence

And there are good reasons that it is not installable by RetroPie itself.

I think it would also to be a good idea to explain in the FAQ why you should use suggestions made. So the advantage for FAT32 is the 100% compatibilty for a wide spread of Operating Systems (Unix, Mac and Windows)
NTFS is a proprietary file system ......
My personal experience with Raspbian it is awfully slow for file operations and only usefull if you need symlinks (this is not possible with FAT)