White flicker on moving through UI after update to 4.2.3

TMNTturtlguy

@pjft do you suspect that is because the .svg has the same name? If it used two different names it would not be the case at it would "remember" them each independently?

Does this happen only within the same system you entered a rom in? Or all systems after entering a rom, exiting, and then switching systems?

pjft

@josete2k Correct. It should low slightly slower first time around (instead of flickering, it ensures it's loaded).

@TMNTturtlguy If the theme used two different names, that would likely work, but that would kind of subvert the purpose of having a SVG in the first place, which is to be able to resize it at will without quality loss - and if you can re-use it in the same theme, that should still be supported. Not to mention that it would take up double the memory, which is what the "re-use" option attempts to fix :)

I'll see what I can do about this.

I like to know that ON/ON and OFF/OFF are suitable solutions for different kinds of problems people have with themes, but I'd like to avoid OFF/OFF, precisely because of the performance hit.

Thanks a lot everyone! I'm calling it a night.

TMNTturtlguy

@pjft thanks for your hard work! Get some rest and have a nice night!

RedBatman

I'm just glad people here are so helpful

DarkWolf

@RedBatman
This community "offers" some very kind people! I love it

pjft

@josete2k I should have fixed it now. I'm trying to force the SVG to be kept in memory to always be the highest resolution used in the theme.

Hopefully this fixes things. Changes have been made both to this build as well as to the OMX branch.

Download and test using the exact same instructions as before - it's the same link. You'll know you're in the right one as it will no longer have the Multithreaded and Re-Use SVGs options.

To ensure you're using it right, though, you may want to remove the two new option values from your ES-Settings (multithreaded images, and re-use SVGs), or set them to "true, true" in your current build before reinstalling as they are now gone from the menu, and defaulted to "true, true".

All of you testing, let me know how it goes. And thanks a lot for catching that one yesterday, @josete2k !

Edit: I've been testing a bit more, and while it seems to be fairly improved, it's still not eradicated. I'll keep digging into this.

josete2k

@pjft yes flickering is still there.

TMNTturtlguy

@pjft I will be on late tonight, probably while you are getting some rest. I plan to test on my comic book theme and carbon theme. I will test with my 12 systems and then load up 20 or so and see what happens.

If there are any specific tests or things you want time to look into let me know here or on the PR comments. I will check both and report back. Thanks!

pjft

@josete2k hopefully the issue with the system logos has disappeared, though? And the flickering should hopefully not be as bad as it was before?

@TMNTturtlguy thanks. I'm trying one final thing - hoping it solves it, but compilation takes quite a bit, so still waiting.

josete2k

Another point...

With the "old" emulationstation, if I set ON Re-Use theme images and Multithreaded in OFF, the white stripe (where the system.svg is loaded) is thinner than normal.

josete2k

@pjft yes, the flickering is... a little "shorter" now.

It happens with the logo in the right side on the screen.

Before it was until it arrived to the center.

Controllers are shown OK.

pjft

@josete2k said in White flicker on moving through UI after update to 4.2.3:

Another point...

With the "old" emulationstation, if I set ON Re-Use theme images and Multithreaded in OFF, the white stripe (where the system.svg is loaded) is thinner than normal.

Can you test with the new one only and check whether it is normal? That combination should not be useful anymore.

I'm still looking into this - thanks for the details!

Update:

Yeah, this is still somewhat borked. I'm calling it a night, tomorrow is a work day.

I'll keep looking into this in the coming days and see if there's anything that I can think of further.

TMNTturtlguy

@pjft I have done extensive testing of your update, and first off, let me just say, you are a miracle worker!

Now to some quick notes - I will also leave feedback on the github PR.

Test 1: Comic Book Theme (heavy theme, 17 systems loaded, uses .svg files) VRAM at 100

After boot up it looks amazing, i can slide with little to no stutter at all, the slide is very smooth, logos and backgrounds load very fast and clean. Every now and then there is a mili-second of a black flash if you want to call it that, but compared to the very large white flashing, this looks so smooth and clean.
After entering a game and exiting: .png background reload time can vary, sometimes fast, sometimes a slight stutter. Exiting back to system select screen. The first time scrolling to the right, there is a large black flash, and the .png system logos now have a white box flash for a half second before the .png logo loads. After one full cycle, everything goes back to completely smooth. I also noted this does not happen after every time i exit a game. Sometimes, it works smoothly.

Video of this test:

Test 2: Carbon Theme 17 systems loaded VRAM 100

This reacts the exact same as the comic book theme did. I notice the same stutter after exiting roms and the same white box on the logos when sliding right. These logos are .svg in carbon. From my testing, it seems like your current solution as fixed the use of .svg and .png, not it is just figuring out how to get the white boxes to disappear after exiting a game.

Test 3: Comic Book Theme (heavy theme, 17 systems loaded, uses .svg files) VRAM at 120

This is absolutely amazing! I can scroll through as fast as I can hit my arrow buttons and never flash or loose the background! After existing a game, the first time though there is a black flash and the logos are still white, after the first time through, it is back to lightning fast speeds.

Video of this test:

Overall - amazing work! I have not seen any negative affects, only major positives. I would suggest this is an upgrade to the current White Screen of Death fix. Even if not perfect yet. I plan to keep this running as my main ES and will report anything else i see back.

EDIT: After more testing with the comic book theme I can create glget errors. More information on this at the Github PR.

pjft

@TMNTturtlguy Thanks for the testing.

As usual, I'd be hoping that my changes would not have caused any particular increase of glGet errors, at least nothing that wouldn't have happened before, but that's yet another thing to chase.

Yes, thanks to @josete2k 's feedback I have indeed narrowed it down to mostly how to fix things after launching a game and coming back.

ES is unloading all textures when it launches a game, so that we have enough memory available to play the games. However, when we get back I am so far struggling to make it work or understanding why that is happening. I have a few theories I'll be chasing down, but it's a puzzle for me. Nothing short of killing ES and actually restarting it seems to fix it for good. Even fully reloading a theme doesn't seem to make a difference.

So yeah, that's where we're at.

I'll also be testing it with a very old ES version, to understand whether this is a regression or whether this used to happen in the past, as that might yield some clues.

We'll see.

Thanks all!

TMNTturtlguy

@pjft I am not sure that your new changes have caused a new way to get glget errors. Take a look at my more detailed testing and report on the PR comments. I have additional notes, testing and another video there. I did or want to duplicate my posts on both sites, so I kept the forum post to a synopsis of my larger post.

As you and I both found, we had been able to reproduce glget errors with several systems running on the comic book theme prior to your modification. I had found that it ran fine at VRAM 100 but in order to operate as smooth as possible and avoid Glget errors it was beneficial to run at a different VRAM, slightly lower or higher than 100. I was also never able to scroll at extremely high speeds as I can now. Lastly, as noted on GitHub I cannot create glget errors on other themes such as carbon or futura.

pjft

@TMNTturtlguy Thanks for the reply. I'll make sure to dutifully read both posts, but thanks for the extensive posting. Hopefully you had a great family weekend! :)

paffley

Hi,

Don't know if this helps but I managed to get rid of the White Flash altogether on my created theme.

Rpi3
Latest RetopiePie build
Video & Marquee support enabled
18 Systems and adding
Scraped boxart within retropie
Game shot images dl'd from web
different backgrounds for system view and gamelist view (PNG 30kb, 6kb)
10 extra image elements added to both system view and gamelist view* (PNG 85kb, 4kb, 2kb, 8kb, 6kb, 10kb, 4kb, 4kb, 31kb, 8kb)

What I did...
I changed my 'background png' from 1920x1080 to 1280x720 and batch shrank my scraped and dl'd images (boxart/game shot) down from approx. 300kb to 50kb - This seemed to stop the white flash completely on my theme and runs smooth on all parts.

My thoughts
I believe it was the weight of my theme (pretty light) and the weight of the scraped boxart/game shot images together causing the White Flash in my opinion, only happened since RP4.2.

Hope this helps...

Cheers,

pjft

Thanks @paffley for sharing. Indeed, there's plenty of things that can - and should! - be done at a theme level, no doubt about that, as well as for the artwork people use in their own libraries.

The main thing that has come up recently, as well, is that since a few ES builds ago there have been an increase of odd white flashes, not fully related to the themes themselves - as even Carbon was now suffering from them - and that's kind of the issue we had been trying to tackle here.

@josete2k - I put together a new build. Same place, same instructions. It should have reverted the changes that caused the white flashes to start showing up. Please try it. It is not the latest - the final one will have the option to Re-Use SVGs as it helps saving memory in larger themes. That is not the case in Carbon, and it should work well.
@TMNTturtlguy - You can also find a new build in my repository if it helps. It will have the option to Re-Use SVGs. Don't use the build I'm sharing with @josete2k as it will fail miserably on your theme. Heavy themes still have problems with the limited memory on the Pi, even after reverting the changes that caused the white screens to become more prevalent. The Re-Use SVG option is the one you want to use for your theme.

In general, and after testing and profiling your theme fairly extensively recently, a few recommendations:

Definitely get all systems to point to the same SVG border file.
Select the "Re-Use SVG" option to reduce memory consumption.
Use "Fade" transition rather than "Slide", especially on the Pi. Slide works best when it can load every single system in memory, so that it doesn't have to load things on demand. It's not that it won't work, but if you hold your finger on the right or left button to scroll a lot of systems, you'll see a lot of black screens as it frantically tries to load the textures from memory. This also causes white flashes on the logos after returning from a game, as it tries to load all the textures at once again. This also happened in the old build :(
Don't set your VRAM to more than 100, unless you have increased your memory split further. Indeed 120MB makes it run as smooth as butter, but that's because it thinks it can load everything at once into memory :) You'll run out of memory after a few systems (10-15, I think) and then you get glGet errors :) I still haven't fixed ES to recover gracefully from that, so it just results in blank screens. I actually run your theme with 70MB of VRAM, I find it a good compromise, as well as using fade.

You may test the last known good build here. You will need to remove the background color from the theme's systemInfo text, as it wasn't supported at the time.

I'll still tinker with this in the coming days, as I'd rather not have the re-use SVG option as an option, but rather as a default. The problem is that it causes the flashing in Carbon, as its logos are SVGs. I have some thoughts on how to adjust that, but will see.

I'm close to calling it a day for these developments. We'll see.

Sorry the outcome isn't extremely better, but at least it should not be worse than where we were a few months ago - hopefully slightly better even.

Thanks.

EDIT: @josete2k - I have just updated the build, it should now have the option as well. You'll want to set it to false (so, do NOT re-use SVGs).

TMNTturtlguy

@pjft Thanks! You are working so hard on this, we all owe you big time.

For your information regarding my Comic Book Theme, Part of the optimization i have been working on for both the 4:3 and 16:9 versions has been reducing files sizes and deleting non used information and pointing all artwork and sound clips that are shared to one location in the art folder, so yes, all systems point to the border .svg in one location now. The testing i rand yesterday and today all incorporated those changes already.

I will run some extensive testing tonight to see what i can come up with and re-create. I think you have done an amazing job of improving to this point, I don't think any of us are expecting you to be able to make the demanding themes work perfectly, there are limits to what we can do! Get some rest!

Oh. by the way, do you want me to post my testing results to the PR comments? I can do a short summary here and a more detailed report on github? Or i can just comment here if you prefer. Thanks

pjft

@TMNTturtlguy Either way works - I suppose here is probably good!

Thanks for the encouraging words - I'm actually thankful you insisted on this being a regression, so much that I started noticing it more and it annoyed me enough to get me to spend time on this.

Unfortunately, though, it seems I won't take it a lot further, but at least it's back to where it was - with warts and all :)

No need for extensive testing, really - you're investing a lot of time in this, which we all certainly appreciate, but at this stage the changes I'm proposing are so few now that, unless you notice something really breaking, it shouldn't be a cause for significant concern.

You may test if things work acceptably for you without enabling the re-use SVG option, though, as I'd love to be able to remove it. But it will be a memory hog without it, so it's kind of a mixed feeling for me. I'f love to at least sort that one out.

But no, just enjoy your RetroPie setup and have fun - otherwise you remind me of me enjoying spending so much time reading about old games that I never find the time to play them :)