How is memory utilized on EmulationStation startup?

Used2BeRX

I've run into a brick wall with the Pi Zero. I'm hoping that a Pi 3 and beyond won't have these problems with the additional CPU and ram. Having too many games in the gamelist.XML files as well as artwork seems to have "broke" the Pi Zero, and there really weren't all that many.... just the Atari 2600, 5200, 7800 and LYNX (with a lot of prototypes, hacks and other roms beyond the standard Official games).

Here's my questions.

Does EmulationStation load every piece of artwork/videos for every system that has a gamelist.xml into memory when it loads?
Does artwork that is not pointed to in the gamelist.xml file get loaded into memory or is it ignored?
Do the number of roms on your system effect performance, even if you don't have a gamelist.xml file for that system?
Does having pointers in your gamelist.xml file that currently point to nothing effect performance in EmulationStation?

I can't think of any more questions at the moment, but if anybody has any other useful information on memory for me other than those 4 questions I'd appreciate it.

Rascas

@Used2BeRX
1 and 2: All images/artwork in ES are loaded into GPU dedicated memory. Since some time now, not sure exactly at which point, something around Retropie 4, you have an option, somewhere in the start menu, to define a limit of GPU memory that will be used for that, the default is 100 MB. If, depending how many systems, roms, etc, (also some themes use more memory than others), excedds that, a cache system will be used which is slower. So play with that setting to find what is best for you.

3: It affects only ES startup time (to load everything into memory), and eventually interface navigation, if it starts using system cache.

4: Not sure what you meant by that but most probably not.

pjft

@Used2BeRX hi. I'm on my phone, so will be short.

Thanks for this report. I've seen quite a few by now, so I'm curious. My answers should be the following, from some experience with the code. They are not canonical, though, but they're my understanding of it.

No. Shouldn't happen. It will load the textures for the current and neighboring systems, for cache purposes, but not the games (worst case, only the selected game per neighboring system). But I doubt that. You can see how video memory is being used by running

sudo vcdbg reloc
Ignored.
Well, yes, as they are loaded into memory. That being said, not as much as the ones with artwork and such. These are just data structures and such. The number of systems has a bigger impact, as a new system means a background texture, logos and such. Box art shouldn't take a lot of memory. Do you have videos?
Shouldn't be the case.

Now, describe "breaking". What do you experience?

I recommend using Carbon and reducing the VRAM value to 50 or so, and still then adjust as needed, going as low as you need to to get it to work while now getting to a state where it fails to load the theme altogether.

Best.

Used2BeRX

@Rascas Thanks for the reply. I'm going to try that out later and see if tweaking the GPU memory. What's happening is after displaying around 1,200 games worth of images and synopsis metadata, emulators actually refuse to work and just kick you back out to the romlist. I was still able to load NES, but SNES and Atari 5200 were just going to a black screen and kicking me back out.

Here's an example of what I meant by number 4:

<game>
		<path>./(1) Atari 5200/(1) Licensed/Astro Chase.bin</path>
		<image>/home/pi/RetroPie/Media/ATARI800/Artwork/Box Front/Astro Chase.jpg</image>
		<cart>/home/pi/RetroPie/Media/ATARI800/Artwork/Cart/Astro Chase.jpg</cart>
                <title>/home/pi/RetroPie/Media/ATARI800/Artwork/Titles/Astro Chase.jpg</title>
                <action>/home/pi/RetroPie/Media/ATARI800/Artwork/Action/Astro Chase.jpg</action>
                <threedbox>/home/pi/RetroPie/Media/ATARI800/Artwork/3D Boxart/Astro Chase.png</threedbox>
                <video>/home/pi/RetroPie/Media/ATARI800/Movies/Astro Chase.wmv</video>
                <gamefaq>/home/pi/RetroPie/Media/ATARI800/GameFAQs/Astro Chase.zip</gamefaq>
                <manual>/home/pi/RetroPie/Media/ATARI800/Manuals/Astro Chase.zip</manual>
                <vgmap>/home/pi/RetroPie/Media/ATARI800/VGMaps/Astro Chase.zip</vgmap>
		<name>Astro Chase</name>
		<platform>Atari 5200</platform>
		<region>USA</region>
		<media>Cartridge</media>
		<controller>5200 Joystick</controller>
		<genre>Shoot 'em Up - Multi-Directional</genre>
		<gametype>Licensed</gametype>
		<releasedate>1983</releasedate>	
		<developer>First Star Software</developer>
		<publisher>Parker Brothers</publisher>
		<players>1</players>
		<programmer>Fernando Herrera</programmer>
		<desc>As pilot of a spaceship, your job is to save the planet Earth! Eight mega mines are slowly homing in, and you need to destroy them all before they reach Earth. Trying to stop you are a variety of flying saucers which will constantly be chasing you around the planet.

Your ship is equipped with lasers which can be used to destroy enemy saucers and the mega mines. Located throughout the space around Earth are eight shield depots; if you fly through one of these, you will temporarily become invincible. Your spaceship also has a limited energy supply; if you get too low on energy, it can be resupplied at one of the generators located in the four corners of the play area. When all of the mega mines have been destroyed, you will move on to the next, faster chase!

http://www.mobygames.com/game/atari-5200/astro-chase
		</desc>		
	</game>

The first 10 lines after the rom image are really just placeholders for the future. Some of these I have media for every one of them for the game, but a lot are only some of them. I'm working with somebody on making a program to create the gamelist.xml files for all of the games with the collections a group I'm with already have for another platform and knowing how memory is used is going to be important when making the program.

Used2BeRX

@pjft No problem. Glad to help in any way I can. I will play with that sudo vcdbg reloc command. Thanks for that.

I do have videos, but they do not work with the Pi Zero. They will start and play for a second or two, but then they freeze up. The audio keeps playing but the video stops.

By "breaking", I mean that the emus won't work at all. I explain it more in detail in my prevoius reply to Rascas.

I'm using the Carbon skin. I will try to play with VRAM and see if that helps.

I'm in the middle of re-installing the newest Genesis emulators after following the instructions to change the "Megadrive" icon to "Genesis" in retropie, so it will be a little while till I can get to it. :)

pjft

@Used2BeRX if it's the emulators that are failing, then it's not as much about the GPU memory as it is the CPU RAM. And yes, in that respect I can see larger connections having an impact, but would be good to have you run "free" from the command line and see how much memory is being taken by EmulationStation.

The textures are unloaded from memory at game launch.

Used2BeRX

@pjft said in How is memory utilized on EmulationStation startup?:

@Used2BeRX if it's the emulators that are failing, then it's not as much about the GPU memory as it is the CPU RAM. And yes, in that respect I can see larger connections having an impact, but would be good to have you run "free" from the command line and see how much memory is being taken by EmulationStation.
The textures are unloaded from memory at game launch.

Okay. I will check when I'm able to and I'll let you know what I see.

Also....
I asked about the effect of just having roms as well, because even after renaming the gamelist.xml files to gamelistBAK.xml so they don't load when EmulationStation starts, I have noticed that the system icons are just "white" and fade to the proper emu when scrolling through systems. I had also added about twice as many roms, so I thought this might have something to do with that.

pjft

@Used2BeRX system icons being white stems from running out of GPU memory. Reducing VRAM will hopefully help.

Rascas

@Used2BeRX I maybe misunderstood how this vram option work, go for pjft recommendations since I think it was him who done that option. Either way failling to launch one rom is strange. Do you have something running on background that might consume some resources? Follow pjft advices, like via ssh run free and sudo vcdbg reloc to check if it is not a memory issue.

Used2BeRX

@Rascas I will try pjft's suggestions and report back when I have the opportunity.

It wasn't failing only to launch one rom. Any Atari 5200 or SNES roms that I was trying to load were failing to load. Once I renamed all of may gamelist.xml files to gamelistBAK.xml and restarted EmulationStation the problem went away and all the games would load again.

As far as I know I have nothing running in the background other than these XML files being loaded. What other things could possibly be consuming system resources?

Used2BeRX

@pjft So I tried the sudo vcdbg reloc. Unfortunately, I'm doing this on an old 720 resolution projection screen TV at the moment, so everything in the console is pretty unreadable. Is there anyway that I could direct that output to a file so I can post the results here? I don't know if I would understand the output on my own even if I could read it.

What I have noticed is that EmulationStation seems to load quicker, now that I've changed the VRAM limit to 50MB. Also, I now have the ability to load games in Atari5200 and SNES again even with the four Atari Gamelist.xml files loaded up.

Unfortunately, after loading a game and coming back out to the EmulationStation main menu, I'm still having that issue with the icons flashing white again, and it seems to get worse after trying out a few emulators.

I'll let you know how long it takes for me to "break" it again once I start adding more XML files and artwork. I should be starting making some new ones some time this weekend if the program beta is created on schedule.

Here's the thread about that, if you haven't seen it already: https://retropie.org.uk/forum/topic/10406/making-premade-gamelist-xml-with-xtra-media-collections

EDIT:

Here's a list of the systems and how many roms I have for them. Only the Atari systems have a gamelist.xml and artwork at this point:

Atari 2600: 723
Atari 5200: 94
Atari 7800: 85
Atari LYNX: 85
Nintendo; 1721
Super Nintendo: 1141
Gameboy: 668
Gameboy Color: 595
Gameboy Advance: 1117
Sega Genesis: 951
Sega Master System: 320
Sega Game Gear: 308
Sega SG-1000: 66
Turbo Grafx 16: 220
Colecovision: 189

Total Systems: 15
Total Games: 8,283