Collections Causing Lockups/White Backgrounds

hansolo77

UPDATE Problem solved. It's not a problem with the addition, it's a problem with memory. The solution was to decrease the VRAM to 80. VRAM adjustment is the limitation you're imposing on ES before it flushes it's memory. Too high, and it tries to fill it up beyond usefulness. So, the lower the better.

This is in response to @pjft requesting a new thread be set up to help troubleshoot some problems I'm having.

I have a stable RetroPie system right now. 30 total systems, 28 if you take out the symlinked Genesis and RetroPie menus. I'm using @TMNTturtlguy 's fabulous ComicBook theme. This is on a Raspberry Pi 3, with an 8gb SD card and a 2tb 2.5 inch USB drive holding the roms and video screenshots.

Earlier today I updated my ES from the RetroPie settings menu via source. I've been experimenting with the new Collections feature.

Initially, I added just 1 collection,Last Played. This list was buggy for me, but I don't plan on using it so I didn't report it. The list was inaccurate, displaying games I hadn't played, and I couldn't get it to sort by the date, just by game name.

While keeping Last Played active, I went through my Genesis system to create some games for the Favorites collection. I did that by identifying a game I wanted to add, then edited the metadata via ES and turned the option ON for "Favorite". I did that for a couple games, then enabled the Favorites collection. This caused an immediate crash. I had to restart the Pi. Once rebooted, the Favorites menu item appeared next to the Last Played one. When I entered it, it was empty. So the changes I had made didn't save. Either the method of editing the metadata was wrong, or the crash happened before the settings could be saved. In either case, it didn't crash when I opened the empty menu. I thought maybe having 2 collections open is what caused it to crash, so I went back in and DISABLED Last Played I went back to re-add the games after that. It was then that I noticed the Help System bar saying to press Y to add a game to favorites. Much easier. ^_^ I added a few games, then backed out to see if they got added to the Favorites menu. Scrolling through the top menu to get to Favorites, I discovered MAME and PC-Engine systems had lost their backgrounds. It was a white screen (with the ComicBook theme's black frame outlines). This had me concerned, so I exited and had the system Restart ES. Once back up, all the menu backgrounds were present, and if I entered the Favorites menu, my games were there. So that's good. I backed out of Favorites to go and find more games to add. It was then that I noticed OTHER systems were now missing their backgrounds, Nintendo and MegaDrive specifically this 2nd time. So I once again Restarted ES, and verified the backgrounds still existed. In both times during the restart, I noticed briefly that the terminal screen was showing an error about something like gl#### error locating. I might have that wrong, I can't reproduce it at the moment. In either case, I didn't notice the white backgrounds until AFTER I entered the Favorites Collection menu.

At this point, I reported it in the "announcement" thread as a potential memory leak. This lead to the discussion about VRAM. During my investigations, I discovered that I had indeed NOT changed my Memory Split. Interestingly too, I saw the in the DOCS that there is a possible way to add an additional 16mb by setting total_mem=1024 in the config.txt. I thought this might help improve at least a little, so I was trying to get that to work. As of right now, I still can't seem to access that 16mb. My tests show I'm still just using 256mb for GPU and 752mb for ARM. I then saw a response in the "announcement" thread to try increasing VRAM to remedy the white screens. I first did 110, then 120, then 130, rebooting after each attempt and verifying the setting was saved. The only impact this did was cause MORE backgrounds to appear white (on the top menu) and the videos screens to no longer play when browsing games. I've since DECREASED the VRAM to 90, and now 80. So far, the white backgrounds and video screens appear to be fixed, but I've still ran into an occasional completely system lock up.

The lockup/freezes have been appearing randomly. At first I thought it was the batteries in my Xbox360 controller I'm using dying. But typically they don't just DIE without first causing the LEDs to blink. The system is completely unresponsive. I have a wireless keyboard hooked up too, and can't get it to move inside ES or even close it. The only solution is a complete power off and reboot. I have a Mausberry Circuit, so a lot of the time a simple power button push will reboot it. But other times, not even that works (the power button/Mausberry runs a script to power off the system). So when it doesn't work, that's where the whole system is completely dead. Another symptom I've seen with the lockup is where I'm browsing the game lists, a video is playing but I can't move. The video just continues to repeat itself over and over until I power cycle. This has led me to think that maybe the the lockup is caused by excessive fast scrolling, or something with the gamelists, but it still appears to happen totally randomly.

When trying to report problems, I understand the value of being as descriptive and detailed as possible to help narrow down potential causes. I've been a PAID beta tester in the past, so I know what kind of information you're looking for. :) I hope this has been helpful.

I will monitor TOP (didn't even know of that command) via an SSH terminal and report my findings the next time I have a crash/lockup/white screen. I can't troubleshoot any more tonight because I'm on my way to bed (work sucks). But I will definitely keep you guys updated.

hansolo77

Oops! Forgot these bits:

Favorites: 27
Atari2600: 473
Atari5200: 105
Atari7800: 66
Lynx: 84
FDS: 270
Game Gear: 377
GameBoy: 1229
GameBoy Advance: 1840
GameBoy Color: 1028
Genesis: 1216
MAME: 4160
Master System: 390
Mega Drive: 1216
Nintendo64: 424
Nintendo: 2001
NeoGeo Pocket: 10
NeoGeo Pocket Color 76
PC Engine: 317
Ports: 10
PlayStation: 159
Retropie: null
Sega 32X: 36
SegaCD: 14
SG-1000: 110
Super NES: 2197
Vectrex: 44
Virtual Boy: 25
Wonderswan: 110
Wonderswan Color 91

TMNTturtlguy

@hansolo77 great report! Just for clarification, can you confirm you were using the ComicBook theme prior to updating ES to the last played/favorites build. Can you also confirm that you did not experience these issues prior to updating ES? Lastly, can you confirm you updated the ComicBook themes so that the Last Played and Favorites logos and themes are present. Thanks again for the great report. I hope that this can be resolved.

pjft

Thanks for the exhaustive reply - this is good to know!

Let me try to address a few of the items here, to the best of my ability. Answers inline:

Initially, I added just 1 collection,Last Played. This list was buggy for me, but I don't plan on using it so I didn't report it. The list was inaccurate, displaying games I hadn't played, and I couldn't get it to sort by the date, just by game name.

So, to start: this collection is only as good as your metadata upkeep. I can assure you that it should only display games that you have played at least once - meaning, the "Times Played" metadata entry is higher than zero. If you do see a game there that has "Times Played: 0" then that is a bug, but otherwise whether you've played it or not, that's the data ES has access to and relies on. :)

As for the sorting, it should be default sorted by last played date - once again, it will always depend on how accurate the "Last Played" date is in the metadata.

The only exception is that if you open the "select" menu, the sorting entry there will list "Name, Ascending", so on exiting that menu it will sort it alphabetically - unless you explicitly re-select "Last Played, Descending" (or any other sorting order). That's a separate issue with ES not storing the current sort value, but it's on my list of future things to tackle.

It could be that you have entries in your gamelists with "Times Played" higher than 0, and no date for last played? I don't see why that would be the case, but that could be an option. If you feel there's any particular entry that shouldn't be there - or that has "Times Played" set to "0", then do send over the gamelist, as I'm keen on trying to understand why that'd be the case.

While keeping Last Played active, I went through my Genesis system to create some games for the Favorites collection. I did that by identifying a game I wanted to add, then edited the metadata via ES and turned the option ON for "Favorite". I did that for a couple games, then enabled the Favorites collection. This caused an immediate crash. I had to restart the Pi. Once rebooted, the Favorites menu item appeared next to the Last Played one. When I entered it, it was empty. So the changes I had made didn't save. Either the method of editing the metadata was wrong, or the crash happened before the settings could be saved. In either case, it didn't crash when I opened the empty menu.

Ok, this is a great use case! I'll need to test it on my own, but I certainly didn't test adding favorites without having the system enabled, and then enabling it. Thank you!

As for it not having saved, that is correct: metadata is only saved on ES exit, so if it crashed, it did not save it. But the method of editing is correct anyway - either Metadata editor or the button would work.

I thought maybe having 2 collections open is what caused it to crash, so I went back in and DISABLED Last Played I went back to re-add the games after that. It was then that I noticed the Help System bar saying to press Y to add a game to favorites. Much easier. ^_^ I added a few games, then backed out to see if they got added to the Favorites menu. Scrolling through the top menu to get to Favorites, I discovered MAME and PC-Engine systems had lost their backgrounds. It was a white screen (with the ComicBook theme's black frame outlines). This had me concerned, so I exited and had the system Restart ES. Once back up, all the menu backgrounds were present, and if I entered the Favorites menu, my games were there. So that's good. I backed out of Favorites to go and find more games to add. It was then that I noticed OTHER systems were now missing their backgrounds, Nintendo and MegaDrive specifically this 2nd time. So I once again Restarted ES, and verified the backgrounds still existed. In both times during the restart, I noticed briefly that the terminal screen was showing an error about something like gl#### error locating. I might have that wrong, I can't reproduce it at the moment. In either case, I didn't notice the white backgrounds until AFTER I entered the Favorites Collection menu.

Correct: white backgrounds and glError 0x0505 go hand in hand. That's the code for "GL_OUT_OF_MEMORY". Lowering VRAM will sort those out. I've probably written way too many posts about why that is happening, but I suppose I should attempt to fix that one day, as it keeps happening to people.

My personal setting is 80, which seems to work well, but try it out on your own and see what works. Summarizing: that's the threshold at which ES will force unload textures from memory to load new ones, in order to prevent running out of video memory. However, if the threshold is too high, it will try to load textures until it runs out of memory, but ES doesn't know how to handle that, and so the white backgrounds show up (textures that are not fully loaded).

I wouldn't necessarily attribute it to the collections themselves, other than for the fact that they are, effectively, more systems being loaded at the time, which take up more video memory. But my assumption is that if you'd add 3 more systems you'd end up running into similar problems anyway.

The lockup/freezes have been appearing randomly. At first I thought it was the batteries in my Xbox360 controller I'm using dying. But typically they don't just DIE without first causing the LEDs to blink. The system is completely unresponsive. I have a wireless keyboard hooked up too, and can't get it to move inside ES or even close it. The only solution is a complete power off and reboot. I have a Mausberry Circuit, so a lot of the time a simple power button push will reboot it. But other times, not even that works (the power button/Mausberry runs a script to power off the system). So when it doesn't work, that's where the whole system is completely dead. Another symptom I've seen with the lockup is where I'm browsing the game lists, a video is playing but I can't move. The video just continues to repeat itself over and over until I power cycle. This has led me to think that maybe the the lockup is caused by excessive fast scrolling, or something with the gamelists, but it still appears to happen totally randomly.
When trying to report problems, I understand the value of being as descriptive and detailed as possible to help narrow down potential causes. I've been a PAID beta tester in the past, so I know what kind of information you're looking for. :) I hope this has been helpful.

This was extremely helpful. In fact, it may be that ES is not actually locking up, but crashing. Are you using OMX Player? That would explain why you keep seeing the video looping, but can't interact with ES anymore - it's that ES has crashed in the background, but since OMX Player is a separate process, it keeps running.

When you run "top" you may find that the ES process is no longer running when these "full" lockups happen.

I'd need a bit more detail or info around those, or maybe a video of what you're doing and then the crash, system, whatever. It might be a crash with parsing a specific metadata entry, but right now I'm grasping at straws without further data.

I will monitor TOP (didn't even know of that command) via an SSH terminal and report my findings the next time I have a crash/lockup/white screen. I can't troubleshoot any more tonight because I'm on my way to bed (work sucks). But I will definitely keep you guys updated.

Thanks. I don't expect you to get any more white screens, anyway, if VRAM is under control.

In fact, a nice command for you to use if/when you get a white screen (or just for fun) is

sudo vcdbg reloc

which will show the video memory in use. The first few lines will have all the relevant information for you:

total space allocated is 236M, with 234M relocatable, 2.3M legacy and 0 offline

(...)

200M free memory in 30 free block(s)
largest free block is 114M bytes

Total video memory
Total free memory in (XXX) blocks
Largest free block (largest allocatable block)

These are the key entries. What may be helpful as well is the content of ~/.emulationstation/es_log.txtthough there might be a lot of non-useful entries there which look scary at the moment. :) Still, worth a shot getting that.

I may share with you a test binary at a later stage if you'd be available to test it, in case we can narrow down some of those crashes.

Let me know if that would work for you.

Thanks!

hansolo77

Good stuff! Glad to help.

@tmntturtlguy said in Collections Causing Lockups/White Backgrounds:

@hansolo77 great report! Just for clarification, can you confirm you were using the ComicBook theme prior to updating ES to the last played/favorites build. Can you also confirm that you did not experience these issues prior to updating ES? Lastly, can you confirm you updated the ComicBook themes so that the Last Played and Favorites logos and themes are present. Thanks again for the great report. I hope that this can be resolved.

Yes I have been using the ComicBook theme prior to updating ES. I didn't have any of these white background issues before updating either, but I have experienced the occasional lockup. Lastly, I didn't update the theme prior to updating ES, but already had the collections logos in place, so I must have already updated it on a prior date.

@pjft said in Collections Causing Lockups/White Backgrounds:

So, to start: this collection is only as good as your metadata upkeep. I can assure you that it should only display games that you have played at least once - meaning, the "Times Played" metadata entry is higher than zero. If you do see a game there that has "Times Played: 0" then that is a bug, but otherwise whether you've played it or not, that's the data ES has access to and relies on. :)

I will doublecheck this when I get home, to see if the games presented truly have a "Times Played" entry. Since I'm using the Video Screensaver now, I think I might have started a game or 2 from within the screensaver, without actually playing it. Does the metadata for "Times Played", etc get updated as soon as the the game launches (when the Run Command is still present) or when the game returns to ES?

As for the sorting, it should be default sorted by last played date - once again, it will always depend on how accurate the "Last Played" date is in the metadata.

The only exception is that if you open the "select" menu, the sorting entry there will list "Name, Ascending", so on exiting that menu it will sort it alphabetically - unless you explicitly re-select "Last Played, Descending" (or any other sorting order). That's a separate issue with ES not storing the current sort value, but it's on my list of future things to tackle.

As I recall, the games were listed initially in aphabetical order. I've played through the entire game of Doom (port) and have started playing Doom 2 (port). When I opened "Last Played", it showed Doom as the first entry with Doom 2 under it. The most recent game I had played was Sonic 1 (Genesis). This collection only showed I had played like 15 games total, which isn't accurate either. But when I used the "select" menu to change the sort, it didn't save it when I exited. I'll have to test a bit more, as it might have been because I exited with the B button rather than going to the "back" menu button.

It could be that you have entries in your gamelists with "Times Played" higher than 0, and no date for last played? I don't see why that would be the case, but that could be an option. If you feel there's any particular entry that shouldn't be there - or that has "Times Played" set to "0", then do send over the gamelist, as I'm keen on trying to understand why that'd be the case.

I will check this when I get home, and send gamelists if you think it'll help.

Ok, this is a great use case! I'll need to test it on my own, but I certainly didn't test adding favorites without having the system enabled, and then enabling it. Thank you!

As for it not having saved, that is correct: metadata is only saved on ES exit, so if it crashed, it did not save it. But the method of editing is correct anyway - either Metadata editor or the button would work.

It may have been something else that caused the crash. The crash was caused ES to close to terminal, not "lockup". But it makes a lot of sense, that if you don't have the Favorites collection enabled prior to adding favorites it would crash.

Correct: white backgrounds and glError 0x0505 go hand in hand. That's the code for "GL_OUT_OF_MEMORY". Lowering VRAM will sort those out. I've probably written way too many posts about why that is happening, but I suppose I should attempt to fix that one day, as it keeps happening to people.

My personal setting is 80, which seems to work well, but try it out on your own and see what works. Summarizing: that's the threshold at which ES will force unload textures from memory to load new ones, in order to prevent running out of video memory. However, if the threshold is too high, it will try to load textures until it runs out of memory, but ES doesn't know how to handle that, and so the white backgrounds show up (textures that are not fully loaded).

I wouldn't necessarily attribute it to the collections themselves, other than for the fact that they are, effectively, more systems being loaded at the time, which take up more video memory. But my assumption is that if you'd add 3 more systems you'd end up running into similar problems anyway.

Good stuff. I assumed that the MORE is usally better. So by increasing VRAM to the max would just make the system that much better. But if the way it works is by telling ES how much memory you want it to use so it can clear out the memory before going over the limit, then REDUCING makes sense. I think I better way to handle this would be to change the menu wording a bit. Maybe something like Max VRAM available to ES (less is better!)

This was extremely helpful. In fact, it may be that ES is not actually locking up, but crashing. Are you using OMX Player? That would explain why you keep seeing the video looping, but can't interact with ES anymore - it's that ES has crashed in the background, but since OMX Player is a separate process, it keeps running.

When you run "top" you may find that the ES process is no longer running when these "full" lockups happen.

I'd need a bit more detail or info around those, or maybe a video of what you're doing and then the crash, system, whatever. It might be a crash with parsing a specific metadata entry, but right now I'm grasping at straws without further data.

Yes I am using the OMX Player. It was the latest "fix" I needed to get the video screenshots to work right. Without it, I was getting a lot of green screens, and massive pixelation. Using the OMX Player solved all of that. I will monitor TOP and and see if ES is no longer running the next time that happens.

hansolo77

Heh even better actually:
VRAM clearing threshold (less is better)

pjft

Wonderful!

I will doublecheck this when I get home, to see if the games presented truly have a "Times Played" entry. Since I'm using the Video Screensaver now, I think I might have started a game or 2 from within the screensaver, without actually playing it. Does the metadata for "Times Played", etc get updated as soon as the the game launches (when the Run Command is still present) or when the game returns to ES?

It will get updated on return to ES, but regardless of whether the emulator is actually launched. So, only if you had somehow rebooted the Pi without returning to ES, the moment you select "launch" and it exists ES, on return it will update it.

As I recall, the games were listed initially in aphabetical order. I've played through the entire game of Doom (port) and have started playing Doom 2 (port). When I opened "Last Played", it showed Doom as the first entry with Doom 2 under it. The most recent game I had played was Sonic 1 (Genesis). This collection only showed I had played like 15 games total, which isn't accurate either. But when I used the "select" menu to change the sort, it didn't save it when I exited. I'll have to test a bit more, as it might have been because I exited with the B button rather than going to the "back" menu button.

Hm. Unfortunately, whether you click "back" or "ok" on that menu, the sorting should still apply (it is quite dumb that there's no "Cancel" option, I know...).

As for it only showing 15 games in total, that would certainly be weird, unless you don't have metadata being saved?

There's a reasonably easy test you can perform: go to the "All games" system and sort by "Times Played, Descending". It should show all the games sorted by the number of times you have played them, as present in the metadata.

If there's a discrepancy in the games listed with more than "zero" times, and the games in the "Last Played" list (they won't necessarily be sorted the same way, mind you, as Last Played sorts by date (according to the metadata, once again), whereas here you're sorting by the actual number), I'd want to look into that.

I will check this when I get home, and send gamelists if you think it'll help.

We'll see if we need to. I know it's a pain to get the gamelists one by one, so I'd want to avoid that if we can :)

It may have been something else that caused the crash. The crash was caused ES to close to terminal, not "lockup". But it makes a lot of sense, that if you don't have the Favorites collection enabled prior to adding favorites it would crash.

Yeah, crashes manifest themselves in different ways, depending on the type. It can crash to the terminal, it can stay with the video (as OMX Player is a separate process)... In principle, it shouldn't have crashed in this scenario, as the collection does exist, it's just not displayed. That being said, I'd not be shocked that there's something I've missed there, so I will look into that. This is an excellent test scenario I had not gone through.

Good stuff. I assumed that the MORE is usally better. So by increasing VRAM to the max would just make the system that much better. But if the way it works is by telling ES how much memory you want it to use so it can clear out the memory before going over the limit, then REDUCING makes sense. I think I better way to handle this would be to change the menu wording a bit. Maybe something like Max VRAM available to ES (less is better!)

Yeah. I'll figure something out. Eventually. I am not that familiar with that code, but we should be able to prevent users from shooting themselves in the foot. For sure.

Yes I am using the OMX Player. It was the latest "fix" I needed to get the video screenshots to work right. Without it, I was getting a lot of green screens, and massive pixelation. Using the OMX Player solved all of that. I will monitor TOP and and see if ES is no longer running the next time that happens.

Agreed, OMX Player is a great help for videos on the Pi! I bet that that's what happening then, which unfortunately doesn't help us much as it doesn't bring us any closer to solving this. :) That being said, you mentioned that you had had some lockups prior to this update? Were they similar - meaning, video stays playing, nothing is responsive?

Thanks!

hansolo77

@pjft said in Collections Causing Lockups/White Backgrounds:

It will get updated on return to ES, but regardless of whether the emulator is actually launched. So, only if you had somehow rebooted the Pi without returning to ES, the moment you select "launch" and it exists ES, on return it will update it.

This sounds good to me. The random game in "Last Played" collection I don't remember playing must have just been one of those times I simply started it from the video screensaver as a test.

Hm. Unfortunately, whether you click "back" or "ok" on that menu, the sorting should still apply (it is quite dumb that there's no "Cancel" option, I know...).

As for it only showing 15 games in total, that would certainly be weird, unless you don't have metadata being saved?

I have it set to Parse gamelists only to "ON" so it doesn't generate new gamelists on bootup. Then I have it set to Save Metadata on Exit also set to "ON". This actually just got me thinking.. If it doesn't save metadata until exit (from ES right?), maybe that's why the "Last Played" collection isn't getting updated? I've been playing around with keeping the screensaver running all day, so nothing gets saved until it crashes. But the "Last Played" or "Times Played" flags in the metadata should be getting added immediately right?

There's a reasonably easy test you can perform: go to the "All games" system and sort by "Times Played, Descending". It should show all the games sorted by the number of times you have played them, as present in the metadata.

If there's a discrepancy in the games listed with more than "zero" times, and the games in the "Last Played" list (they won't necessarily be sorted the same way, mind you, as Last Played sorts by date (according to the metadata, once again), whereas here you're sorting by the actual number), I'd want to look into that.

I actually just did a little experiment when I got home. I re-enabled the Last Played collection and looked. It's now showing 24 games. I know for a fact I've played a LOT more than that. But still, going forward. I noticed it appeared to be sorted by system. That might have just been coincidence. But I remember playing Doom 2 last night because the RetroAchievements site was having issues, so I just played that. It doesn't show up on my list till WAY down at the bottom:
alt text
That "The Battle of Olympus" I don't remember ever playing... It looks like it might be sorted by system, but it's not alphabetical as I originally though. "NES" comes after "GENESIS", and "PINBALL" should come after "METROID", so I guess that's right. To further the experiment, after taking that snapshot, I loaded up Doom 2, then backed out, and re-opened that collection. Doom 2 now shows up at the top:

alt text
So apparently it's working right?

Yeah. I'll figure something out. Eventually. I am not that familiar with that code, but we should be able to prevent users from shooting themselves in the foot. For sure.

I will say, that in the short time I've been playing around with it now with the VRAM set to 80, it definitely "appears" to be more stable. It hasn't had a hard lockup, crash back to terminal, or have white background screens.

Agreed, OMX Player is a great help for videos on the Pi! I bet that that's what happening then, which unfortunately doesn't help us much as it doesn't bring us any closer to solving this. :) That being said, you mentioned that you had had some lockups prior to this update? Were they similar - meaning, video stays playing, nothing is responsive?

Yeah I will try to keep a close eye on this and see if I can get it to trigger. The lockups I was having (video continuing to play with no input response) seem to be cleared out for now. I almost want to try to put my VRAM back up to 100 to see if I can trigger it again, just to see if ES is still running. At this point, without it crashing anymore, I don't really have anything more to try. I've purposely tried to spam the lists, scrolling really fast, etc because it really felt like it was doing it if I didn't give the video time to load up. Nothing is happening now. When I was having the issues before updating, my VRAM was set at the default 100, so maybe lowering does the trick.

Thanks!

Glad to help! I can't code, even though I'd love to learn. So just by providing a report of issues is about all I can do to feel helpful. :)

hansolo77

One other quick observation...
If you have the Video Screensaver enabled, but are in the "Start" or "Select" menus, the screensaver doesn't work. It's just blanking the screen to black. Not related, just thought I'd mention it. I noticed it when I was typing up my reply about having "Save Metadata on Exit" enabled.

pjft

I have it set to Parse gamelists only to "ON" so it doesn't generate new gamelists on bootup. Then I have it set to Save Metadata on Exit also set to "ON". This actually just got me thinking.. If it doesn't save metadata until exit (from ES right?), maybe that's why the "Last Played" collection isn't getting updated? I've been playing around with keeping the screensaver running all day, so nothing gets saved until it crashes. But the "Last Played" or "Times Played" flags in the metadata should be getting added immediately right?

No, only on ES exit are the gamelists saved to the disk.

What do you mean "I've been playing around with keeping the screensaver running all day, so nothing gets saved until it crashes."? Does it crash when you leave the screensaver running all day? That should hopefully not be the case, it's not the best user experience :P

I actually just did a little experiment when I got home. I re-enabled the Last Played collection and looked. It's now showing 24 games. I know for a fact I've played a LOT more than that. But still, going forward. I noticed it appeared to be sorted by system. That might have just been coincidence. But I remember playing Doom 2 last night because the RetroAchievements site was having issues, so I just played that. It doesn't show up on my list till WAY down at the bottom:
That "The Battle of Olympus" I don't remember ever playing... It looks like it might be sorted by system, but it's not alphabetical as I originally though. "NES" comes after "GENESIS", and "PINBALL" should come after "METROID", so I guess that's right. To further the experiment, after taking that snapshot, I loaded up Doom 2, then backed out, and re-opened that collection. Doom 2 now shows up at the top:
So apparently it's working right?

Hm. I'm not sure. We do have a new sorting option for "sorting per console", etc. It would be a big coincidence that the games you had played would have been sorted by system, and by name, descending... it can be, but I'm not ready to believe it yet. I'm assuming you did not enter the Select menu, nor did you try to sort it.

Would you want to share your Genesis, NES and Ports gamelists via PasteBin or something? Also, I'm assuming the time/date on your Pi is correct (i.e. is it connected to the internet?).

Yeah I will try to keep a close eye on this and see if I can get it to trigger. The lockups I was having (video continuing to play with no input response) seem to be cleared out for now. I almost want to try to put my VRAM back up to 100 to see if I can trigger it again, just to see if ES is still running. At this point, without it crashing anymore, I don't really have anything more to try. I've purposely tried to spam the lists, scrolling really fast, etc because it really felt like it was doing it if I didn't give the video time to load up. Nothing is happening now. When I was having the issues before updating, my VRAM was set at the default 100, so maybe lowering does the trick.

No need to increase VRAM and try to force crash it - I mean, if it crashes because it runs out of memory, there's not much we can do other than try to get it to handle things gracefully. :) But, once again, it's early days, so keep using it with 80 and we'll see how it goes.

Glad to help! I can't code, even though I'd love to learn. So just by providing a report of issues is about all I can do to feel helpful. :)

By all means, thank you for the thorough explanation and investigation here.

As for the note on video screensaver not triggering when you're on a menu, I had run into that as well. I suppose I would need to look into that one day, I'll add it to the list of things to check out. :)

Thanks!

hansolo77

@pjft said in Collections Causing Lockups/White Backgrounds:

No, only on ES exit are the gamelists saved to the disk.

What do you mean "I've been playing around with keeping the screensaver running all day, so nothing gets saved until it crashes."? Does it crash when you leave the screensaver running all day? That should hopefully not be the case, it's not the best user experience :P

No it doesn't crash when I exit the screensaver. I was merely saying that the screensaver is running fine, then I try to play some games and experience the crash. So after a full day of the screensaver working fine, the system crashes once I start to navigate. I can have it working fine, play a few games or whatever, screensaver loads, come back home to test, and it crashes. So it never saves the xml's. It's not related to the screensaver, just ignore that bit. :)

Hm. I'm not sure. We do have a new sorting option for "sorting per console", etc. It would be a big coincidence that the games you had played would have been sorted by system, and by name, descending... it can be, but I'm not ready to believe it yet. I'm assuming you did not enter the Select menu, nor did you try to sort it.

Would you want to share your Genesis, NES and Ports gamelists via PasteBin or something? Also, I'm assuming the time/date on your Pi is correct (i.e. is it connected to the internet?).

Because my gamelists are so large (complete sets) I had to upload them to my Dropbox.
https://www.dropbox.com/sh/gnjsjj0ge5tx6kz/AAC9JdOqquIxWUUKUOtK0iu8a?dl=0

No need to increase VRAM and try to force crash it - I mean, if it crashes because it runs out of memory, there's not much we can do other than try to get it to handle things gracefully. :) But, once again, it's early days, so keep using it with 80 and we'll see how it goes.

I only said "almost". ^_^ I don't want to purposely cause it to lock up. With all the hard crashes and lockups, I'm probably going to corrupt my SD card. So I'd like to minimize it where I can.

By all means, thank you for the thorough explanation and investigation here.

As for the note on video screensaver not triggering when you're on a menu, I had run into that as well. I suppose I would need to look into that one day, I'll add it to the list of things to check out. :)

Yeah I thought it was something I did. But I tested the theory and was able to duplicate it perfectly by just keeping those menus up. Not a biggie, I"m sure it's a rare issue.

pjft

@hansolo77 said in Collections Causing Lockups/White Backgrounds:

So after a full day of the screensaver working fine, the system crashes once I start to navigate. I can have it working fine, play a few games or whatever, screensaver loads, come back home to test, and it crashes. So it never saves the xml's. It's not related to the screensaver, just ignore that bit. :)

Oh my. Does this happen all the time?

I'd be curious to learn whether this also happens if you're not using the video screensaver.

I have left it on overnight a few times and didn't experience such crashes, but that being said, I haven't done so that many times.

As for the sorting, this actually clarifies that it's working as intended:

https://docs.google.com/spreadsheets/d/129M3inADmelPb4jR3dL31B8FsT1yYAj1J9g7Fg58tOk/edit#gid=0

Those games seem to have been opened in sequence, probably as part of you setting up an Achievements subfolder of sorts? Unsure about that, but at least the data on your gamelists matches the exact sorting order shown in your screens.

One less mystery to solve :)

Thanks!

hansolo77

No no no!! It has nothing to do with the screensaver! Before I adjusted the VRAM to 80, I was having things lockup and crash all the time. Randomly. All I was saying was that I had the system working, played a game, left for work, came back with the screensaver running, browsed around to find another game and then it would (or would not) crash. Had I not had the screensaver running, or the DIM screensaver, it would have crashed anyway. The only reason I mentioned it was to reiterate that the system was stable enough to run for HOURS until I started moving around in the menus. Now that's I've dropped back the VRAM, I've not seen it happen. The issue is that the gamelist xml's aren't saving when the system crashes. I've played MANY games and saw that the "Last Played" list had not been properly updated. THAT was the issue I was reporting.

Now that I think about it, it might have something to do with the Mausberry Power Circuit I'm using. As I recall, pushing the power button causes it send a command to the Pi to shutdown. BUT, I don't think it's sending a signal to ES first to close it down. So the Pi shuts off gracefully, but not ES. So the lists aren't saving. They only save if I tell it to shut down ES (or restart it, etc..). Not knowing anything about how all that works, I wouldn't know where to being. But I'm a smart fellow, I could probably figure it out. Just need to add a line to the shutdown script to first exit ES. Is there a terminal command I should be looking for to try? I was thinking sudo killall emulationstation, but will that initiate the "save on exit"?

And yep.. I created subfolders in each of my rom folders for games that have Achievements. I then copied the roms that have them into it. When I did the scrape, it added that folder path to the gamelist.xml. I didn't know if it would work, but it does! I added a "_" to the folder name, hoping to have it show show up at the TOP of the gamelist, but at the bottom works just as well.

pjft

@hansolo77 Hm.

It might certainly be related, though I'm not sure. I mean, you clearly have some metadata being saved at times.

I'd probably test playing a game, or adding a favorite, then shutting it down with the Mausberry, and then seeing if it sticks.

If not, you can then test sending out that signal to ES and seeing if it sticks.

Trial and error, I suppose.

hansolo77

I did actually test it. I think that's the problem. Before shutting down via the Mausberry, I checked the "Last Played" collection and it had 2 or 3 more games on the list after my screenshot. The I shut down. Waited, powered on, and checked. The added games weren't there. So I went and played a game or 2, then checked again. The games were on the list. Did a quick Mausberry shutdown again and once again the games are gone.

I tried adding the sudo killall emulationstation to the script on the line before sudo poweroff but it didn't make any difference in the following test. I wonder if that's not the best command to use? Either that, or should I add some kind of delay to have the script pause until it confirms the task has finished before initiating the poweroff? I'm not sure how to do that.

I can confirm however that the sudo killall emulationstation command DOES work via SSH. I started a game, exited, verified it was in "Last Played", then SSH'd and sent the command. It existed to terminal. I then ran emulationstation to get it back up, and the game was there in "Last Played" like it should.

So I just need to figure out how to make shut down ES properlly before sending out the poweroff command.

pjft

@hansolo77 maybe a post in the power block section may help, I'm sure others would have had the same problem in the past. I'm curious about it though :)

hansolo77

Updated the OP to indicate this problem is solved, and the solution.