Are excessively large collections a problem for start up?

Sisyphus

I'm in the mind of discovery. I don't wish to curate, parse or scrape (is that lazy of me?) but basically drop in all TOSEC, maybe World of Spectrum and all No Intro sets for many, many systems and go where the mood takes me. The problem is these sets combined have really massive numbers of roms and Emulation Station does and will fall over with such massive numbers.

What is the maximum number of roms combined across all systems it can handle before start up and/or crash is a problem?

Is it really necessary to go through checks? Eg. if I symlink within a system to a specific folder on a large HDD, could it build a list only when I go in to that particular folder?

I suppose if I built a massive gameslist.xml and not parse, would this help or would that be just the same as that's the way ES works? Too big/too many entries and it'll fail?

windg

I have my roms on a usb stick 128 gb with only 4 gb free. It takes under 10 seconds for the sytem to read it during the start up. I don't think you will have a problem.

mitu

@windg And how many games ? Over 80k games and you may not
have enough memory on a Pi3 to load them all.

windg

@mitu said in Are excessively large collections a problem for start up?:

@windg And how many games ? Over 80k games and you may not
have enough memory on a Pi3 to load them all.

~~I don't know how many there are, I haven't counted them so far. Most of the space is taken up by cd games.~~
My rom folder have 20.512 files.

Sisyphus

Lady Eklipse's total zx spectrum collection alone is over 87,000 unique items I think 3.25gb

mitu

What is the maximum number of roms combined across all systems it can handle before start up and/or crash is a problem?

Haven't done any extensive testing, but I recall a while ago a forum member tried to load more than 100k games and ES crashed due to insufficient memory. That was on a RPI 3, which had something less than 800Mb RAM available (the rest being taken by the GPU memory).

On a RPI4, with more RAM available, things might be better. Of course, if you're not using a SBC, RAM may not be an issue.

Is it really necessary to go through checks? Eg. if I symlink within a system to a specific folder on a large HDD, could it build a list only when I go in to that particular folder?

Symlink-ing won't help much, you can already mount your ROM folders from an USB drive and this would speed things up for the initial scan.

I suppose if I built a massive gameslist.xml and not parse, would this help or would that be just the same as that's the way ES works?

It would help with the initial loading, otherwise you'll need to wait for ES to re-scan the folders on start and depending on how many games you have, that can take a while (not only an a Pi).